linux-kernel - Re: [PATCH] mm: vmscan: Obey indeed proportional scanning for kswapd and memcg

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20151130090444.GA2520@yaowei-K42JY>
Date:	Mon, 30 Nov 2015 17:04:44 +0800
From:	Yaowei Bai <baiyaowei@...s.chinamobile.com>
To:	Mel Gorman <mgorman@...e.de>
Cc:	akpm@...ux-foundation.org, riel@...hat.com, mhocko@...e.cz,
	hannes@...xchg.org, kamezawa.hiroyu@...fujitsu.com, jslaby@...e.cz,
	Valdis.Kletnieks@...edu, zcalusic@...sync.net, vbabka@...e.cz,
	vdavydov@...allels.com, linux-mm@...ck.org,
	linux-kernel@...r.kernel.org
Subject: Re: [PATCH] mm: vmscan: Obey indeed proportional scanning for kswapd
 and memcg

On Wed, Nov 25, 2015 at 11:28:51AM +0000, Mel Gorman wrote:
> On Wed, Nov 25, 2015 at 12:48:20PM +0800, Yaowei Bai wrote:
> > Commit e82e0561dae9f3ae5 ("mm: vmscan: obey proportional scanning
> > requirements for kswapd") intended to preserve the proportional scanning
> > and reclaim what was requested by get_scan_count() for kswapd and memcg
> > by stopping reclaiming one type(anon or file) LRU and reducing the other's
> > amount of scanning proportional to the original scan target.
> > 
> > So the way to determine which LRU should be stopped reclaiming should be
> > comparing scanned/unscanned percentages to the original scan target of two
> > lru types instead of absolute values what implemented currently, because
> > larger absolute value doesn't mean larger percentage, there shall be
> > chance that larger absolute value with smaller percentage, for instance:
> > 
> > 	target_file = 1000
> > 	target_anon = 500
> > 	nr_file = 500
> > 	nr_anon = 400
> > 
> > in this case, because nr_file > nr_anon, according to current implement,
> > we will stop scanning anon lru and shrink file lru. This breaks
> > proportional scanning intent and makes more unproportional.
> > 
> > This patch changes to compare percentage to the original scan target to
> > determine which lru should be shrunk.
> > 
> > Signed-off-by: Yaowei Bai <baiyaowei@...s.chinamobile.com>
> 
> This one has gone back and forth a few times in the past. It really was

Sorry for reply late. Yes, I noticed Johannes Weiner has recommended this in
the discussion thread about commit e82e0561dae9f3ae5 ("mm: vmscan: obey
proportional scanning requirements for kswapd"):

 http://marc.info/?l=linux-kernel&m=136397130117394&w=2

and you thought it was out of scope of that series at that moment.
But i didn't see this in the upstream git history.

> deliberate that the scanning was proportional to the scan target. While

Yes, i see the evolvement of the source code and do believe that the scanning
was proportional to the scan target is the right direction and we're already in
that direction with current implementation. At the very beginning, you wanted
to subtract min from all of LRUs to perform proportional scan and i think that
is a very good start and simple and useful enough approxiamtion. And then Johannes
Weiner suggersted that swappiness is about page types and comparing the sum of
file pages with the sum of anon pages and then knocking out the smaller pair would
be better.You agreed and implemented it with applying scanned percentage of the
smaller pair to the remaining LRUs. But considering the example case mentioned
above we will scan even more unproportionally as we cann't guarantee scanning
all LRUs 100% evenly.

> I see what your concern is, it's unclear what the actual impact is. Have
> you done any testing to check if the proposed new behaviour is actually
> better?

I didn't test this patch. Maybe it's difficult to catch this situation of
the example case because mostly we scan LRUs evenly. but i think it's advantage
is also obvious because it covers the case mentioned above to achieve indeed
proportional without introducing extra overhead and makes the code match with
the comments and more understandable to reduce people's confusion.

Did i miss something?

> 
> -- 
> Mel Gorman
> SUSE Labs

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/