linux-kernel - Re: [patch] mm, compaction: abort free scanner if split fails

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.DEB.2.10.1606221502140.146497@chino.kir.corp.google.com>
Date:	Wed, 22 Jun 2016 15:06:29 -0700 (PDT)
From:	David Rientjes <rientjes@...gle.com>
To:	Andrew Morton <akpm@...ux-foundation.org>
cc:	Vlastimil Babka <vbabka@...e.cz>, Minchan Kim <minchan@...nel.org>,
	Joonsoo Kim <iamjoonsoo.kim@....com>,
	Mel Gorman <mgorman@...hsingularity.net>,
	Hugh Dickins <hughd@...gle.com>, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org, stable@...r.kernel.org
Subject: Re: [patch] mm, compaction: abort free scanner if split fails

On Wed, 22 Jun 2016, Andrew Morton wrote:

> On Tue, 21 Jun 2016 18:22:49 -0700 (PDT) David Rientjes <rientjes@...gle.com> wrote:
> 
> > If the memory compaction free scanner cannot successfully split a free
> > page (only possible due to per-zone low watermark), terminate the free 
> > scanner rather than continuing to scan memory needlessly.  If the 
> > watermark is insufficient for a free page of order <= cc->order, then 
> > terminate the scanner since all future splits will also likely fail.
> > 
> > This prevents the compaction freeing scanner from scanning all memory on 
> > very large zones (very noticeable for zones > 128GB, for instance) when 
> > all splits will likely fail while holding zone->lock.
> > 
> 
> This collides pretty heavily with Joonsoo's "mm/compaction: split
> freepages without holding the zone lock".
> 

Sorry if it wasn't clear, but I was proposing this patch for 4.7 
inclusion and Vlastimil agreed we should ask for that.  Joonsoo said he 
was prepared to rebase on top of that.  Is 
mm-compaction-split-freepages-without-holding-the-zone-lock.patch and 
friends going into 4.7 or are we deferring this fix until 4.8?

compaction_alloc() iterating a 128GB zone has been benchmarked to take 
over 400ms on some systems whereas any free page isolated and ready to be 
split ends up failing in split_free_page() because of the low watermark 
check and thus the iteration continues.

The next time compaction occurs, the freeing scanner will likely start at 
the end of the zone again since no success was made previously and we get 
the same lengthy iteration until the zone is brought above the low 
watermark.  All thp page faults can take >400ms in such a state without 
this fix.