linux-kernel - Re: [patch 02/12] Immediate Values

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20090928104617.9c4b868a.akpm@linux-foundation.org>
Date:	Mon, 28 Sep 2009 10:46:17 -0700
From:	Andrew Morton <akpm@...ux-foundation.org>
To:	Andi Kleen <andi@...stfloor.org>
Cc:	Mathieu Desnoyers <mathieu.desnoyers@...ymtl.ca>,
	Ingo Molnar <mingo@...e.hu>, linux-kernel@...r.kernel.org,
	Jason Baron <jbaron@...hat.com>,
	Rusty Russell <rusty@...tcorp.com.au>,
	Adrian Bunk <bunk@...sta.de>,
	Christoph Hellwig <hch@...radead.org>
Subject: Re: [patch 02/12] Immediate Values - Architecture Independent Code

On Mon, 28 Sep 2009 03:23:37 +0200 Andi Kleen <andi@...stfloor.org> wrote:

> On Thu, Sep 24, 2009 at 09:20:13PM -0700, Andrew Morton wrote:
> > On Thu, 24 Sep 2009 09:26:28 -0400 Mathieu Desnoyers <mathieu.desnoyers@...ymtl.ca> wrote:
> > 
> > > Immediate values are used as read mostly variables that are rarely updated. They
> > > use code patching to modify the values inscribed in the instruction stream. It
> > > provides a way to save precious cache lines that would otherwise have to be used
> > > by these variables.
> > 
> > What a hare-brained concept.
> 
> The concept makes a lot of sense.

But does it?

> Cache misses are extremly costly
> on modern CPUs and when the workload has blown the caches away in user space
> it can literally be hundreds or even thousands of cycles to fetch
> a data cache line.

Well yes.  But for a kernel dcache entry to have been replaced by a
userspace one, userspace will, on average, have itself incurred a *lot*
of dcache misses.  So we just spent a lot of CPU cycles in userspace,
so the cost of the in-kernel dcache miss is relatively small.

That's how caches work!  If a kernel variable is read frequently, it's
still in dcache.  If it's read infrequently, it falls out of dcache but
that doesn't matter much because it's read infrequently!

And lo, it appears that we're unable to observe any measurable benefit
from the changes, so we're cooking up weird fake testcases to be able to
drag this thing out of the noise floor.

Obviously the change will have _some_ performance benefit.  But is it
enough to justify the addition of yet more tricksy code to maintain? 
That's a very different question.  

> There's a lot of data around that the kernel has very little IPC
> due to a lot of cache misses in some workloads.

Kernel gets a lot of cache misses, but that's usually against
userspace, pagecache, net headers/data, etc.  I doubt if it gets many
misses against a small number of small, read-mostly data items which is
what this patch addresses.

And it is a *small* number of things to which this change is
applicable.  This is because the write operation for these read-mostly
variables becomes very expensive indeed.  This means that we cannot use
"immediate values" for any variable which can conceivable be modified
at high frequency by any workload.

For example, how do we know it's safe to use immediate-values for
anything which can be modified from userspace, such as a sysfs-accessed
tunable?  How do we know this won't take someone's odd-but-legitimate
workload and shoot it in the head?

Summary:

- at this stage no real-world beenefit has been demonstrated afaict

- the feature is narrowly applicable anyway

- it addes complexity and maintenance cost

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/