linux-kernel - Re: Idea for reducing sysfs memory usage

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20160217023025.GB26016@kroah.com>
Date:	Tue, 16 Feb 2016 18:30:25 -0800
From:	Greg Kroah-Hartman <gregkh@...uxfoundation.org>
To:	David Ahern <dsa@...ulusnetworks.com>
Cc:	Edward Cree <ec429@...tab.net>, linux-kernel@...r.kernel.org
Subject: Re: Idea for reducing sysfs memory usage

On Tue, Feb 16, 2016 at 05:44:54PM -0700, David Ahern wrote:
> On 2/16/16 5:37 PM, Edward Cree wrote:
> >On 16/02/16 23:55, Greg Kroah-Hartman wrote:
> >>On Tue, Feb 16, 2016 at 11:46:49PM +0000, Edward Cree wrote:
> >>>Sorry if this has been suggested before, but if so I couldn't find it.
> >>>Short version: could a sysfs dir reference a list of default attributes
> >>>rather than having to instantiate them all?
> >>Shorter version, why do you think it is?  :)
> >>
> >>Have you done some testing of the amount of memory that sysfs entries
> >>consume and found any problems with it?
> >Two reasons:
> >a) in his netdev1.1 talk "Scaling the Number of Network Interfaces on
> >Linux",
> >    David Ahern claimed a memory overhead of (iirc) about 45kB per
> >netdevice,
> >    of which he attributed (again, iirc) about 20kB to sysfs entries. He
> >also
> >    indicated that this was a problem for his use case.  (My apologies to
> >    David if I've misrepresented him.  CCed him so he can correct me.)
> 
> Close enough. :-)
> 
> I analyzed memory allocations for the creation of a dummy netdevice. All
> total 36,450 bytes were requested resulting in 43,944 bytes allocated. Of
> that kobject and sysfs is 14,568. So kobject and sysfs is roughly 1/3 the
> memory overhead of a netdevice and is also a significant overhead in the
> time to create the netdevice.

Time?  Hm, I thought we reduced the time down by a lot a few years ago,
any pointers to where we are spending too much time?  The PPC people did
some work on our list traversal issues for thousands of devices, perhaps
that doesn't scale to 10s of thousands?

As for 1/3 the memory needed, is that acceptable given all that the
interface and functionality that the code gives you?  14k does seem
pretty large to me, how many attributes/files are assigned to each
device?

thanks,

greg k-h