linux-kernel - Re: [RFC PATCH 2/2] fs/dcache: Add negative-dentry-ratio config

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <a0a311d6-bf23-4b62-e2ed-874c116bda6b@oracle.com>
Date:   Thu, 31 Mar 2022 13:37:01 -0700
From:   Stephen Brennan <stephen.s.brennan@...cle.com>
To:     Al Viro <viro@...iv.linux.org.uk>
Cc:     Andrew Morton <akpm@...ux-foundation.org>,
        Luis Chamberlain <mcgrof@...nel.org>,
        Arnd Bergmann <arnd@...db.de>,
        Matthew Wilcox <willy@...radead.org>,
        James Bottomley <James.Bottomley@...senpartnership.com>,
        Gao Xiang <hsiangkao@...ux.alibaba.com>,
        Dave Chinner <david@...morbit.com>,
        Roman Gushchin <roman.gushchin@...ux.dev>,
        Colin Walters <walters@...bum.org>,
        linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [RFC PATCH 2/2] fs/dcache: Add negative-dentry-ratio config

On 3/31/22 12:45, Al Viro wrote:
> On Thu, Mar 31, 2022 at 12:08:27PM -0700, Stephen Brennan wrote:
>> Negative dentry bloat is a well-known problem. For systems without
>> memory pressure, some workloads (like repeated stat calls) can create an
>> unbounded amount of negative dentries quite quickly. In the best case,
>> these dentries could speed up a subsequent name lookup, but in the worst
>> case, they are never used and their memory never freed.
>>
>> While systems without memory pressure may not need that memory for other
>> purposes, negative dentry bloat can have other side-effects, such as
>> soft lockups when traversing the d_subdirs list or general slowness with
>> managing them. It is a good idea to have some sort of mechanism for
>> controlling negative dentries, even outside memory pressure.
>>
>> This patch attempts to do so in a fair way. Workloads which create many
>> negative dentries must create many dentries, or convert dentries from
>> positive to negative. Thus, negative dentry management is best done
>> during these same operations, as it will amortize its cost, and
>> distribute the cost to the perpetrators of the dentry bloat. We
>> introduce a sysctl "negative-dentry-ratio" which sets a maximum number
>> of negative dentries per positive dentry, N:1. When a dentry is created
>> or unlinked, the next N+1 dentries of the parent are scanned. If no
>> positive dentries are found, then a candidate negative dentry is killed.
> 
> Er...  So what's to stop d_move() from leaving you with your cursor
> pointer poiting into the list of children of another parent?
> 
> What's more, your dentry_unlist() logics will be defeated by that -
> if victim used to have a different parent, got moved, then evicted,
> it looks like you could end up with old parent cursor pointing
> to the victim and left unmodified by dentry_unlist() (since it looks
> only at the current parent's cursor).  Wait for it to be freed and
> voila - access to old parent's cursor will do unpleasant things.
> 
> What am I missing here?

Thanks for this catch. Since d_move holds the parent's lock, it should
be possible to include the same condition as dentry_unlist() to ensure
the cursor gets advanced if necessary. I could make it a small inline
helper to make things easier to read. I will go ahead and fix that.

Thanks,
Stephen