linux-kernel - Re: [PATCH 06/11] mm, compaction: distinguish between full and partial COMPACT

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160411124653.GG23157@dhcp22.suse.cz>
Date:	Mon, 11 Apr 2016 14:46:53 +0200
From:	Michal Hocko <mhocko@...nel.org>
To:	Vlastimil Babka <vbabka@...e.cz>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Johannes Weiner <hannes@...xchg.org>,
	Mel Gorman <mgorman@...e.de>,
	David Rientjes <rientjes@...gle.com>,
	Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>,
	Joonsoo Kim <js1304@...il.com>,
	Hillf Danton <hillf.zj@...baba-inc.com>, linux-mm@...ck.org,
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 06/11] mm, compaction: distinguish between full and
 partial COMPACT_COMPLETE

On Mon 11-04-16 14:10:26, Vlastimil Babka wrote:
> On 04/05/2016 01:25 PM, Michal Hocko wrote:
> >From: Michal Hocko <mhocko@...e.com>
> >
> >COMPACT_COMPLETE now means that compaction and free scanner met. This is
> >not very useful information if somebody just wants to use this feedback
> >and make any decisions based on that. The current caller might be a poor
> >guy who just happened to scan tiny portion of the zone and that could be
> >the reason no suitable pages were compacted. Make sure we distinguish
> >the full and partial zone walks.
> >
> >Consumers should treat COMPACT_PARTIAL_SKIPPED as a potential success
> >and be optimistic in retrying.
> >
> >The existing users of COMPACT_COMPLETE are conservatively changed to
> >use COMPACT_PARTIAL_SKIPPED as well but some of them should be probably
> >reconsidered and only defer the compaction only for COMPACT_COMPLETE
> >with the new semantic.
> >
> >This patch shouldn't introduce any functional changes.
> >
> >Signed-off-by: Michal Hocko <mhocko@...e.com>
> 
> Acked-by: Vlastimil Babka <vbabka@...e.cz>

Thanks!

> With some notes:
> 
> >@@ -1463,6 +1466,10 @@ static enum compact_result compact_zone(struct zone *zone, struct compact_contro
> >  		zone->compact_cached_migrate_pfn[0] = cc->migrate_pfn;
> >  		zone->compact_cached_migrate_pfn[1] = cc->migrate_pfn;
> >  	}
> >+
> >+	if (cc->migrate_pfn == start_pfn)
> >+		cc->whole_zone = true;
> >+
> 
> This assumes that migrate scanner at initial position implies also free
> scanner at the initial position. That should be true, because migration
> scanner is the first to run. But getting the zone->compact_cached_*_pfn is
> racy. Worse, zone->compact_cached_migrate_pfn is array distinguishing sync
> and async compaction, so it's possible that async compaction has advanced
> both its own migrate scanner cached position, and the shared free scanner
> cached position, and then sync compaction starts migrate scanner at
> start_pfn, but free scanner has already advanced.

OK, I see. The whole thing smelled racy but I thought it wouldn't be
such a big deal. Even if we raced then only a marginal part of the zone
wouldn't be scanned, right? Or is it possible that free_pfn would appear
in the middle of the zone because of the race?

> So you might still see a false positive COMPACT_COMPLETE, just less
> frequently and probably with much lower impact.
> But if you need to be truly reliable, check also that cc->free_pfn ==
> round_down(end_pfn - 1, pageblock_nr_pages)

I do not think we need the precise check if the race window (in the
skipped zone range) is always small.

Thanks!
-- 
Michal Hocko
SUSE Labs