Message-ID: <e0e7e253412240b3b427624a984642e6@AcuMS.aculab.com>
Date: Tue, 12 Mar 2024 22:18:39 +0000
From: David Laight <David.Laight@...LAB.COM>
To: 'Pasha Tatashin' <pasha.tatashin@...een.com>, "H. Peter Anvin"
	<hpa@...or.com>
CC: "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>, "akpm@...ux-foundation.org"
	<akpm@...ux-foundation.org>, "x86@...nel.org" <x86@...nel.org>,
	"bp@...en8.de" <bp@...en8.de>, "brauner@...nel.org" <brauner@...nel.org>,
	"bristot@...hat.com" <bristot@...hat.com>, "bsegall@...gle.com"
	<bsegall@...gle.com>, "dave.hansen@...ux.intel.com"
	<dave.hansen@...ux.intel.com>, "dianders@...omium.org"
	<dianders@...omium.org>, "dietmar.eggemann@....com"
	<dietmar.eggemann@....com>, "eric.devolder@...cle.com"
	<eric.devolder@...cle.com>, "hca@...ux.ibm.com" <hca@...ux.ibm.com>,
	"hch@...radead.org" <hch@...radead.org>, "jacob.jun.pan@...ux.intel.com"
	<jacob.jun.pan@...ux.intel.com>, "jgg@...pe.ca" <jgg@...pe.ca>,
	"jpoimboe@...nel.org" <jpoimboe@...nel.org>, "jroedel@...e.de"
	<jroedel@...e.de>, "juri.lelli@...hat.com" <juri.lelli@...hat.com>,
	"kent.overstreet@...ux.dev" <kent.overstreet@...ux.dev>,
	"kinseyho@...gle.com" <kinseyho@...gle.com>,
	"kirill.shutemov@...ux.intel.com" <kirill.shutemov@...ux.intel.com>,
	"lstoakes@...il.com" <lstoakes@...il.com>, "luto@...nel.org"
	<luto@...nel.org>, "mgorman@...e.de" <mgorman@...e.de>, "mic@...ikod.net"
	<mic@...ikod.net>, "michael.christie@...cle.com"
	<michael.christie@...cle.com>, "mingo@...hat.com" <mingo@...hat.com>,
	"mjguzik@...il.com" <mjguzik@...il.com>, "mst@...hat.com" <mst@...hat.com>,
	"npiggin@...il.com" <npiggin@...il.com>, "peterz@...radead.org"
	<peterz@...radead.org>, "pmladek@...e.com" <pmladek@...e.com>,
	"rick.p.edgecombe@...el.com" <rick.p.edgecombe@...el.com>,
	"rostedt@...dmis.org" <rostedt@...dmis.org>, "surenb@...gle.com"
	<surenb@...gle.com>, "tglx@...utronix.de" <tglx@...utronix.de>,
	"urezki@...il.com" <urezki@...il.com>, "vincent.guittot@...aro.org"
	<vincent.guittot@...aro.org>, "vschneid@...hat.com" <vschneid@...hat.com>
Subject: RE: [RFC 00/14] Dynamic Kernel Stacks

...
> I re-read my cover letter, and I do not see where "kernel memory" is
> mentioned. We are talking about kernel stacks overhead that is
> proportional to the user workload, as every active thread has an
> associated kernel stack. The idea is to save memory by not
> pre-allocating all pages of kernel-stacks, but instead use it as a
> safeguard when a stack actually becomes deep. Come-up with a solution
> that can handle rare deeper stacks only when needed. This could be
> done through faulting on the supported hardware (as proposed in this
> series), or via pre-map on every schedule event, and checking the
> access when thread goes off cpu (as proposed by Andy Lutomirski to
> avoid double faults on x86) .
> 
> In other words, this feature is only about one very specific type of
> kernel memory that is not even directly mapped (the feature required
> vmapped stacks).

Just for interest how big does the register save area get?
In the 'good old days' it could be allocated from the low end of the
stack memory. But AVX512 starts making it large - never mind some
other things that (IIRC) might get to 8k.
Even the task area is probably non-trivial since far fewer things
can be shared than one might hope.

I'm sure I remember someone contemplating not allocating stacks to
each thread. I think that requires waking up with a system call
restart for some system calls - plausibly possible for futex() and poll().

Another option is to do a proper static analysis of stack usage
and fix the paths that have deep stacks and remove all recursion.
I'm pretty sure objtool knows the stack offsets of every call instruction.
The indirect call hashes (fine IBT?) should allow indirect calls
be handled as well as direct calls.
Processing the 'A calls B at offset n' to generate a max depth
is just a SMOP.

At the moment I think all 'void (*)(void *)' function have the same hash?
So the compiler would need a function attribute to seed the hash.

With that you might be able to remove all the code paths that actually
use a lot of stack - instead of just guessing and limiting individual
stack frames.

My 'gut feel' from calculating the stack use that way for an embedded
system back in the early 1980s is that the max use will be inside
printk() inside an obscure error path and if you actually hit it
things will explode.
(We didn't have enough memory to allocate big enough stacks!)

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

Powered by blists - more mailing lists