linux-kernel - Re: [PATCH] Reduce vm_stat cacheline contention in __vm_enough

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20111019145458.GA9266@sgi.com>
Date:	Wed, 19 Oct 2011 09:54:58 -0500
From:	Dimitri Sivanich <sivanich@....com>
To:	David Rientjes <rientjes@...gle.com>
Cc:	Andi Kleen <andi@...stfloor.org>, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org, Christoph Lameter <cl@...two.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Mel Gorman <mel@....ul.ie>
Subject: Re: [PATCH] Reduce vm_stat cacheline contention in
 __vm_enough_memory

On Tue, Oct 18, 2011 at 06:16:21PM -0700, David Rientjes wrote:
> On Tue, 18 Oct 2011, Andi Kleen wrote:
> 
> > > Would it make sense to have the ZVC delta be tuneable (via /proc/sys/vm?), keeping the
> > > same default behavior as what we currently have?
> > 
> > Tunable is bad. We don't really want a "hundreds of lines magic shell script to
> > make large systems perform". Please find a way to auto tune.
> > 
> 
> Agreed, and I think even if we had a tunable that it would result in 
> potentially erradic VM performance because some areas depend on "fairly 
> accurate" ZVCs and it wouldn't be clear that you're trading other unknown 
> VM issues that will affect your workload because you've increased the 
> deltas.  Let's try to avoid having to ask "what is your ZVC delta tunable 
> set at?" when someone reports a bug about reclaim stopping preemptively.

Yes, I'm inclined to agree.

> 
> That said, perhaps we need higher deltas by default and then hints in key 
> areas in the form of sync_stats_if_delta_above(x) calls that would do 
> zone_page_state_add() only when that kind of precision is actually needed.  
> For public interfaces, that would be very easy to audit to see what the 
> level of precision is when parsing the data.

I did some manual tuning to see what deltas would be needed to achieve the
greatest tmpfs writeback performance on a system with 640 cpus and 64 nodes:

For 120 threads writing in parallel (each to it's own mountpoint), the
threshold needs to be on the order of 1000.  At a threshold of 750, I
start to see a slowdown of 50-60 MB/sec.

For 400 threads writing in parallel, the threshold needs to be on the order
of 2000 (although we're off by about 40 MB/sec at that point).

The necessary deltas in these cases are quite a bit higher than the current
125 maximum (see calculate*threshold in mm/vmstat.c).

I like the idea of having certain areas triggering vm_stat sync, as long
as we know what those key areas are and how often they might be called.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/