linux-kernel - Re: tipc_init(), WARNING: at arch/x86/mm/highmem_32.c:52, [2.6.24-rc4-git5: Reported regressions from 2.6.23]

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sun, 9 Dec 2007 12:51:39 +0100
From:	Ingo Molnar <mingo@...e.hu>
To:	Pekka Enberg <penberg@...helsinki.fi>
Cc:	Linus Torvalds <torvalds@...ux-foundation.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Matt Mackall <mpm@...enic.com>,
	"Rafael J. Wysocki" <rjw@...k.pl>,
	LKML <linux-kernel@...r.kernel.org>,
	Christoph Lameter <clameter@....com>
Subject: Re: tipc_init(), WARNING: at arch/x86/mm/highmem_32.c:52,
	[2.6.24-rc4-git5: Reported regressions from 2.6.23]

* Pekka Enberg <penberg@...helsinki.fi> wrote:

> I mostly live in the legacy 32-bit UMA/UP land still so I cannot 
> verify this myself but the kind folks at SGI claim the following 
> (again from the announcement):
> 
> "On our systems with 1k nodes / processors we have several gigabytes
>  just tied up for storing references to objects for those queues This 
>  does not include the objects that could be on those queues. One fears 
>  that the whole memory of the machine could one day be consumed by 
>  those queues."

Yes, you can find gigs tied up on systems that have 100 GB of RAM, or 
you can have gigs tied up if you over-size your caches. I'd like to see 
an accurate calculation done on this.

> The problem is that for each cache, you have an "per-node alien 
> queues" for each node (see struct kmem_cache nodelists -> struct 
> kmem_list3 alien). Moving slab metadata to struct page solves this but 
> now you can only have one "queue" thats part of the same struct.

yes, it's what i referred to as "distributed, per node cache". It has no 
"quadratic overhead". It has SLAB memory spread out amongst nodes. I.e. 
1 million pages are distributed amongst 1k nodes with 1000 pages per 
node with each node having 1 page.

But that memory is not lost and it's disingenous to call it 'overhead' 
and it very much comes handy for performance _IF_ there's global 
workload that involves cross-node allocations. It's simply a cache that 
is mis-sized and mis-constructed on large node count systems but i bet 
it makes quite a performance difference on low-node-count systems.

On high node-count systems it might make sense to reduce the amount of 
cross-node caching and to _structure_ the distributed NUMA SLAB cache in 
an intelligent way (perhaps along cpuset boundaries) - but a total, 
design level _elimination_ of this caching concept, using very 
misleading arguments, just looks stupid to me ...

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/