Message-ID: <54D22F1D.6060906@suse.cz>
Date:	Wed, 04 Feb 2015 15:39:25 +0100
From:	Vlastimil Babka <vbabka@...e.cz>
To:	Joonsoo Kim <js1304@...il.com>
CC:	Joonsoo Kim <iamjoonsoo.kim@....com>,
	Linux Memory Management List <linux-mm@...ck.org>,
	LKML <linux-kernel@...r.kernel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	David Rientjes <rientjes@...gle.com>,
	Christoph Lameter <cl@...ux.com>, Mel Gorman <mgorman@...e.de>,
	Michal Nazarewicz <mina86@...a86.com>,
	Minchan Kim <minchan@...nel.org>,
	Naoya Horiguchi <n-horiguchi@...jp.nec.com>,
	Rik van Riel <riel@...hat.com>
Subject: Re: [RFC PATCH 0/5] compaction: changing initial position of scanners

On 02/03/2015 06:07 PM, Joonsoo Kim wrote:
> 2015-02-04 0:51 GMT+09:00 Vlastimil Babka <vbabka@...e.cz>:
>> Ah, I think I see where the misunderstanding comes from now. So to clarify,
>> let's consider
>>
>> 1. single compaction run - single invocation of compact_zone(). It can start
>> from cached pfn's from previous run, or zone boundaries (or pivot, after this
>> series), and terminate with scanners meeting or not meeting.
>>
>> 2. full zone compaction - consists one or more compaction runs, where the first
>> run starts at boundaries (pivot). It ends when scanners meet -
>> compact_finished() returns COMPACT_COMPLETE
>>
>> 3. compaction after full defer cycle - this is full zone compaction, where
>> compaction_restarting() returns true in its first run
>>
>> My understanding is that you think pivot changing occurs after each full zone
>> compaction (definition 2), but in fact it occurs only each defer cycle
>> (definition 3). See patch 5 for detailed reasoning. I don't think it's short
>> term. It means full zone compactions (def 2) already failed many times and then
>> was deferred for further time, using the same unchanged pivot.
>
> Ah... thanks for clarifying. I actually think pivot changing occurs at
> definition 2
> as you guess. :)

Great it's clarified :)

>> I think any of the alternatives you suggested below where migrate scanner
>> processes whole zone during full zone compaction (2), would necessarily result
>> in shorter-term back and forth migration than this scheme. On the other hand,
>> the pivot changing proposed here might be too long-term. But it's a first
>> attempt, and the frequency can be further tuned.
>
> Yes, your proposal would be less problematic on back and forth problem than
> my suggestion.
>
> Hmm...nevertheless, I can't completely agree with pivot approach.
>
> I'd like to remove dependency of migrate scanner and free scanner such as
> termination criteria at this chance. Meeting position of both scanner is roughly

Well at some point compaction should terminate if it scanned the whole 
zone, and failed. How else to check that than using the scanner positions?

> determined by on amount of free memory in the zone. If 200 MB is free in
> the zone, migrate scanner can scan at maximum 200 MB from the start pfn
> of the pivot. Without changing pivot quickly, we can scan only
> this region regardless zone size so it cause bad effect to high order
> allocation for a long time.
>
> In stress-highalloc test, it doesn't matter since we try to attempt a lot of
> allocations. This bad effect would not appear easily. Although middle of
> allocation attempts are failed, latter attempts would succeed
> since pivot would be changed in the middle of attempts.

OK, that might be true. It's not a perfect benchmark.

> But, in real world scenario, all allocation attempts are precise and
> it'd be better
> first come high order allocation request to succeed and this is another problem
> than allocation success rate in stress-highalloc test. To accomplish it, we
> need to change pivot as soon as possible. Without it, we could miss some
> precise allocation attempt until pivot is changed. For this purpose, we should
> remove defer logic or change it more loosely and then, resetting pivot would
> occur soon so we could encounter back and forth problem frequently.

It seems to me that you can't have both the "migration scanner should 
try scanning whole zone during single compaction (or during relatively 
few attempts)" and "we shouldn't migrate pages that we have just 
(relatively recently) migrated", in any scheme including the two you 
proposed in previous mail. These features just go against each other.

In any scheme you should divide the zone between part that's scanned for 
pages to migrate from, and part that scanned for free pages as migration 
targets. If you don't divide, then you end up migrating back and forth 
instantly, which would be bad.

So what happens after you don't have any free pages in the part that was 
for the free scanner (this is what happen in current scheme when 
scanners meet). If you wanted to continue with the migration scanner, 
the only free pages are in the part which the migration scanner just 
processed. And funnily enough, the pivot changing scheme will put the 
free scanner just in the position to scan this part. But doing that 
immediately could mean excessive migration.

Your alternative scheme where free scanner follows the migration scanner 
at some distance is not very different in this sense. If you manage to 
tune the distance properly, you will also scan for free pages the part 
that was just processed by the migration scanner. It might be more 
efficient in that you don't rescan the part that the migration scanner 
didn't reach both before and after pivot change. But fundamentally, it 
means migrating pages that were recently migrated.

> Therefore, it's better to change compaction logic more fundamentally.

Maybe it's indeed better to excessively migrate than keep rescanning the 
same pageblocks and then defer compaction. But we shouldn't forget that 
immediate success rate is not the only criteria. We should also keep the 
overhead sane. That's why there's pageblock suitability bitfield, 
deferred compaction etc, which I'm not sure how those would fit into the 
"continuously progressing migration scanner" scheme.

So what I think should precede such increase in compact aggressivity:
- on direct compaction, only try migrate when successfully isolated all 
pages needed for merging the desired order page. I've had such patch 
already in one series last year, but it affected the anti-fragmentation 
effects of compaction.
- no more THP page faults (also for other good reasons), leave 
collapsing to khugepaged, or rather task_work, leaving only the 
expensive sync compaction to dedicated per-node daemons. These should 
hopefully solve the anti-fragmentation issue as well.

> Thanks.
>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists