linux-kernel - Re: [PATCH 01 of 66] disable lumpy when compaction is enabled

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 10 Nov 2010 17:03:52 +0100
From:	Andrea Arcangeli <aarcange@...hat.com>
To:	Mel Gorman <mel@....ul.ie>
Cc:	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	linux-mm@...ck.org, Linus Torvalds <torvalds@...ux-foundation.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	linux-kernel@...r.kernel.org,
	Marcelo Tosatti <mtosatti@...hat.com>,
	Adam Litke <agl@...ibm.com>, Avi Kivity <avi@...hat.com>,
	Hugh Dickins <hugh.dickins@...cali.co.uk>,
	Rik van Riel <riel@...hat.com>,
	Dave Hansen <dave@...ux.vnet.ibm.com>,
	Benjamin Herrenschmidt <benh@...nel.crashing.org>,
	Ingo Molnar <mingo@...e.hu>, Mike Travis <travis@....com>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	Christoph Lameter <cl@...ux-foundation.org>,
	Chris Wright <chrisw@...s-sol.org>, bpicco@...hat.com,
	Balbir Singh <balbir@...ux.vnet.ibm.com>,
	"Michael S. Tsirkin" <mst@...hat.com>,
	Peter Zijlstra <peterz@...radead.org>,
	Johannes Weiner <hannes@...xchg.org>,
	Daisuke Nishimura <nishimura@....nes.nec.co.jp>,
	Chris Mason <chris.mason@...cle.com>,
	Borislav Petkov <bp@...en8.de>
Subject: Re: [PATCH 01 of 66] disable lumpy when compaction is enabled

On Wed, Nov 10, 2010 at 02:27:04PM +0000, Mel Gorman wrote:
> Agreed. Any performance increase from THP is not likely to offset the
> cost of lumpy reclaim.

Exactly. Furthermore the improvement will still happen later by
polling compaction once every 10 sec with khugepaged (this is also
required in case some other guest or application quit releasing tons
of ram maybe natively order 9 in the buddy without requiring any
further compaction invocation).

What the default should be I don't know, but I like a default that
fails without causing swap storms. If you want the swap storms and to
drop all ptes regardless of their young bits, you should ask
explicitly for it I think. Anybody asking for high order allocation
and pretending to succeed despite the anti-frag and movable pageblocks
migrated with compaction aren't enough to succeed should be able to
handle a full graceful failure like THP does by design (or worst case
to return error to userland). As far as I can tell tg3 atomic order 2
allocation also provides for a graceful fallback for the same reason
(however in new mainline it floods the dmesg with tons of printk,
which it didn't used to with older kernels but it's not an actual
regression).

> Again agreed, I have no problem with lumpy reclaim being pushed aside.
> I'm just less keen on it being disabled altogether. I have high hopes
> for the series I'm working on that it can be extended slightly to suit
> the needs of THP.

Great. Well this is also why I disabled it with the smallest possible
modification, to avoid stepping on your toes.

> Nah, the first thing I did was eliminate being "my fault" :). It would
> have surprised me because the patches in isolation worked fine. It
> thought the inode changes might have had something to do with it so I
> was chasing blind alleys for a while. Hopefully d065bd81 will prove to
> be the real problem.

Well I wasn't sure if you tested it already on that very workload, the
patches weren't from you (even if you were in the signoffs). I
mentioned it just in case, glad it's not related :).
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/