linux-kernel - Re: [PATCH v2] fix get_scan_count for working well with small targets

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <20110427143121.e2a7e158.kamezawa.hiroyu@jp.fujitsu.com>
Date:	Wed, 27 Apr 2011 14:31:21 +0900
From:	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
To:	Minchan Kim <minchan.kim@...il.com>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"nishimura@....nes.nec.co.jp" <nishimura@....nes.nec.co.jp>,
	"kosaki.motohiro@...fujitsu.com" <kosaki.motohiro@...fujitsu.com>,
	"mgorman@...e.de" <mgorman@...e.de>, Ying Han <yinghan@...gle.com>
Subject: Re: [PATCH v2] fix get_scan_count for working well with small
 targets

On Wed, 27 Apr 2011 14:08:18 +0900
Minchan Kim <minchan.kim@...il.com> wrote:

> Hi Kame,
> 
> On Wed, Apr 27, 2011 at 10:50 AM, KAMEZAWA Hiroyuki
> <kamezawa.hiroyu@...fujitsu.com> wrote:
> > On Tue, 26 Apr 2011 13:59:34 -0700
> > Andrew Morton <akpm@...ux-foundation.org> wrote:
> >
> >> What about simply removing the nr_saved_scan logic and permitting small
> >> scans?  That simplifies the code and I bet it makes no measurable
> >> performance difference.
> >>
> >
> > ok, v2 here. How this looks ?
> > For memcg, I think I should add select_victim_node() for direct reclaim,
> > then, we'll be tune big memcg using small memory on a zone case.
> >
> > ==
> > At memory reclaim, we determine the number of pages to be scanned
> > per zone as
> >        (anon + file) >> priority.
> > Assume
> >        scan = (anon + file) >> priority.
> >
> > If scan < SWAP_CLUSTER_MAX, the scan will be skipped for this time
> > and priority gets higher. This has some problems.
> >
> >  1. This increases priority as 1 without any scan.
> >     To do scan in this priority, amount of pages should be larger than 512M.
> >     If pages>>priority < SWAP_CLUSTER_MAX, it's recorded and scan will be
> >     batched, later. (But we lose 1 priority.)
> 
> Nice catch!  It looks to be much enhance.
> 
> >     But if the amount of pages is smaller than 16M, no scan at priority==0
> >     forever.
> 


> Before reviewing the code, I have a question about this.
> Now, in case of (priority = 0), we don't do shift operation with priority.>
 So nr_saved_scan would be the number of lru list pages. ie, 16M.
> Why no-scan happens in case of (priority == 0 and 16M lru pages)?
> What am I missing now?
> 
An, sorry. My comment is wrong. no scan at priority == DEF_PRIORITY.
I'll fix description.

But....
Now, in direct reclaim path
==
static void shrink_zones(int priority, struct zonelist *zonelist,
                                        struct scan_control *sc)
{
....
                if (scanning_global_lru(sc)) {
                        if (!cpuset_zone_allowed_hardwall(zone, GFP_KERNEL))
                                continue;
                        if (zone->all_unreclaimable && priority != DEF_PRIORITY)
                                continue;       /* Let kswapd poll it */
                }
==

And in kswapd path
==
                /*
                 * Scan in the highmem->dma direction for the highest
                 * zone which needs scanning
                 */
                for (i = pgdat->nr_zones - 1; i >= 0; i--) {
                        struct zone *zone = pgdat->node_zones + i;

                        if (!populated_zone(zone))
                                continue;

                        if (zone->all_unreclaimable && priority != DEF_PRIORITY)
                                continue;
....
               for (i = 0; i <= end_zone; i++) {
                        if (zone->all_unreclaimable && priority != DEF_PRIORITY)
                                continue;

==

So, all_unreclaimable zones are only scanned when priority==DEF_PRIORITY.
But, in DEF_PRIORITY, scan count is always zero because of priority shift.
So, yes, no scan even if priority==0 even after setting all_unreclaimable == true.

Thanks,
-Kame

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/