linux-kernel - Re: [PATCH v2 2/2] SLUB: Mark merged slab caches in /proc/slabinfo

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.DEB.2.00.1009141320090.7123@chino.kir.corp.google.com>
Date:	Tue, 14 Sep 2010 13:56:18 -0700 (PDT)
From:	David Rientjes <rientjes@...gle.com>
To:	Linus Torvalds <torvalds@...ux-foundation.org>
cc:	Pekka Enberg <penberg@...nel.org>, linux-kernel@...r.kernel.org,
	Christoph Lameter <cl@...ux.com>
Subject: Re: [PATCH v2 2/2] SLUB: Mark merged slab caches in /proc/slabinfo

On Tue, 14 Sep 2010, Linus Torvalds wrote:

> > I can understand how it's confusing that only the first slab cache name is
> > being emitted, and I think that can be changed, but this shows way too
> > much information that is already available when CONFIG_SLUB_DEBUG is used
> > via the sysfs interface.
> 
> Umm. Not in any readable form, it isn't.
> 

The debug information is available via /proc/slabinfo, the relationships 
amongst merged caches is available via /sys/kernel/slab.  The former 
depends on the latter via Kconfig, so slub requires using both interfaces 
because of its implementation of merged caches to understand each cache's 
slab usage.

I agree the only way to show this information only via /proc/slabinfo so 
that you can ignore the sysfs interface is by doing exactly what is done 
here.  The problem is that it's an incomplete solution since it's 
truncated at 63 chars, it delimits cache names by '+' which is allowed in 
names already, and is obfuscated because it doesn't imply caches are 
merged (in fact, it implies additional rather than subsumption.

> The cause for this is that I made a bug-report about the wrong slab
> info, with me claiming 400+ thousand entries (taking up 10M of memory)
> for a slub cache that turned out to be entirely innocent.
> 
> That's what /proc/slabinfo said, and quite frankly, /proc/slabinfo was
> simply _lying_. It gave very misleading output.
> 

I understand, and that's addressed with my suggestion of using the unique 
cache identifier for each line in /proc/slabinfo which would map 1:1 to 
the sysfs dentries for those merged caches.  /proc/slabinfo would then 
represent the true slab state by listing actual caches and their usage.

Aggregating 63 chars of cache names on each line of /proc/slabinfo 
wouldn't have helped you when it reports 400+ thousand entries because you 
still can't identify the guilty cache.  For that issue, this would simply 
give you a list of suspects, and sysfs is a better inteface for that since 
it's complete and not truncated.

> In my not-so-humble opinion, either the merging needs to go away
> entirely, or the misleading output needs to be fixed.

Cache merging may have been advertised as a bigger performance improvement 
than it actually is, and I don't do it in my own slab allocator for other 
reasons, but it does lead to more effective memory use by reducing slab 
fragmentation.  On one of my benchmarking servers, over 60% of caches are 
merged and /sys/kernel/slab/.../partial reports roughly the same percent 
of fewer total partial slabs over the system in comparison to 
slub_nomerge.

Indeed, to debug egregious slab usage on your system you would opt-out of 
the more efficient memory use by using slub_nomerge and then checking 
/proc/slabinfo anyway.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/