linux-kernel - Re: fs/stat: Reduce memory requirements for stat

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <5399B301.9030108@canonical.com>
Date:	Thu, 12 Jun 2014 16:02:41 +0200
From:	Stefan Bader <stefan.bader@...onical.com>
To:	Heiko Carstens <heiko.carstens@...ibm.com>
CC:	linux-kernel@...r.kernel.org,
	Eric Dumazet <eric.dumazet@...il.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>
Subject: Re: fs/stat: Reduce memory requirements for stat_open

On 12.06.2014 15:41, Heiko Carstens wrote:
> On Thu, Jun 12, 2014 at 03:00:17PM +0200, Stefan Bader wrote:
>> When reading from /proc/stat we allocate a large buffer to maximise
>> the chances of the results being from a single run and thus internally
>> consistent.  This currently is sized at 128 * num_possible_cpus() which,
>> in the face of kernels sized to handle large configurations (256 cpus
>> plus), results in the buffer being an order-4 allocation or more.
>> When system memory becomes fragmented these cannot be guarenteed, leading
>> to read failures due to allocation failures.
>>
>> There seem to be two issues in play here.  Firstly the allocation is
>> going to be vastly over sized in the common case, as we only consume the
>> buffer based on the num_online_cpus().  Secondly, regardless of size we
>> should not be requiring allocations greater than PAGE_ALLOC_COSTLY_ORDER
>> as allocations above this order are significantly more likely to fail.
>>
>> The following patch addesses both of these issues.  Does that make sense
>> generally? It seemed to stop top complaining wildly for the reporter
>> at least.
> 
> Hi Stefan,
> 
> see also https://lkml.org/lkml/2014/5/21/341

Hi Heiko,

doh, so you guys have been hit by that before. And I have missed the fact that
single_open is special. Which makes the change for the upper limit do the wrong
thing. While long-term it sounds like changing it to vmalloc or iterative reads
sounds better, maybe the change from possible to online cpus might be something
that is better acceptable as a quick fix... Not sure. At least this giving back
a bit of attention to the matter and it is not only affecting zSeries. x86
starts to see hw that requires a similar high possible cpus... :)

-Stefan

> 
> and one possible solution:
> 
> https://lkml.org/lkml/2014/5/30/191
> 
> and the other one:
> 
> https://lkml.org/lkml/2014/6/12/92
> https://lkml.org/lkml/2014/6/12/107
> 
> Thanks,
> Heiko
> 



Download attachment "signature.asc" of type "application/pgp-signature" (902 bytes)