linux-kernel - Re: [PATCH 5/5] slub: Only IPI CPUs that have per cpu obj to flush

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1317030352.9084.76.camel@twins>
Date:	Mon, 26 Sep 2011 11:45:52 +0200
From:	Peter Zijlstra <a.p.zijlstra@...llo.nl>
To:	Gilad Ben-Yossef <gilad@...yossef.com>
Cc:	linux-kernel@...r.kernel.org,
	Frederic Weisbecker <fweisbec@...il.com>,
	Russell King <linux@....linux.org.uk>,
	Chris Metcalf <cmetcalf@...era.com>, linux-mm@...ck.org,
	Christoph Lameter <cl@...ux-foundation.org>,
	Pekka Enberg <penberg@...nel.org>,
	Matt Mackall <mpm@...enic.com>
Subject: Re: [PATCH 5/5] slub: Only IPI CPUs that have per cpu obj to flush

On Mon, 2011-09-26 at 11:35 +0300, Gilad Ben-Yossef wrote:
> Yes, the alloc in the flush_all path definitively needs to go. I
> wonder if just to resolve that allocating the mask per cpu and not in
> kmem_cache itself is not better - after all, all we need is a single
> mask per cpu when we wish to do a flush_all and no per cache. The
> memory overhead of that is slightly better. This doesn't cover the
> cahce bounce issue.
> 
> My thoughts regarding that were that since the flush_all() was a
> rather rare operation it is preferable to do some more
> work/interference here, if it allows us to avoid having to do more
> work in the hotter alloc/dealloc paths, especially since it allows us
> to have less IPIs that I figured are more intrusive then cacheline
> steals (are they?)
> 
> After all, for each CPU that actually needs to do a flush, we are
> making the flush a bit more expensive because of the cache bounce just
> before we send the IPI, but that IPI and further operations are an
> expensive operations anyway. For CPUs that don't need to do a flush, I
> replaced an IPI for a cacheline(s) steal. I figured it was still a
> good bargain

Hard to tell really, I've never really worked with these massive
machines, biggest I've got is 2 nodes and for that I think your
for_each_online_cpu() loop might indeed still be a win when compared to
extra accounting on the alloc/free paths.

The problem with a per-cpu cpumask is that you need to disable
preemption over the whole for_each_online_cpu() scan and that's not
really sane on very large machines as that can easily take a very long
time indeed.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/