linux-kernel - RE: [PATCH v3 0/5] fs/dcache: Limit # of negative dentries

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87B246BB5ED53A4C98E4F9A35839EDE18539A2F4@nkgeml514-mbx.china.huawei.com>
Date:   Tue, 22 Aug 2017 02:59:57 +0000
From:   "Wangkai (Kevin,C)" <wangkai86@...wei.com>
To:     Waiman Long <longman@...hat.com>,
        Alexander Viro <viro@...iv.linux.org.uk>,
        Jonathan Corbet <corbet@....net>
CC:     "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-doc@...r.kernel.org" <linux-doc@...r.kernel.org>,
        "linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
        "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Ingo Molnar <mingo@...nel.org>,
        Miklos Szeredi <mszeredi@...hat.com>,
        Matthew Wilcox <willy@...radead.org>,
        Larry Woodman <lwoodman@...hat.com>,
        James Bottomley <James.Bottomley@...senPartnership.com>
Subject: RE: [PATCH v3 0/5] fs/dcache: Limit # of negative dentries



> -----Original Message-----
> From: Waiman Long [mailto:longman@...hat.com]
> Sent: Monday, August 21, 2017 9:35 PM
> To: Wangkai (Kevin,C); Alexander Viro; Jonathan Corbet
> Cc: linux-kernel@...r.kernel.org; linux-doc@...r.kernel.org;
> linux-fsdevel@...r.kernel.org; Paul E. McKenney; Andrew Morton; Ingo Molnar;
> Miklos Szeredi; Matthew Wilcox; Larry Woodman; James Bottomley
> Subject: Re: [PATCH v3 0/5] fs/dcache: Limit # of negative dentries
> 
> On 08/20/2017 11:23 PM, Wangkai (Kevin,C) wrote:
> >
> > Yes, I have add some trace info for the dentry state changed, with dentry flag
> and reference count:
> >
> > File create:
> > [   42.636675] dentry [xxxx_1234] 0xffff880230be8180 flag 0x0 ref 1 ev
> dentry alloc
> > File close:
> > [   42.637421] dentry [xxxx_1234] 0xffff880230be8180 flag 0x4800c0 ref 0
> ev dput called
> >
> > Unlink lookup:
> > [  244.658086] dentry [xxxx_1234] 0xffff880230be8180 flag 0x4800c0 ref
> > 1 ev d_lookup Unlink d_delete:
> > [  244.658254] dentry [xxxx_1234] 0xffff880230be8180 flag 0x800c0 ref
> > 1 ev d_lockref ref 1 Unlink dput:
> > [  244.658438] dentry [xxxx_1234] 0xffff880230be8180 flag 0x800c0 ref
> > 0 ev dput called
> >
> > The end, dentry's flag stay at 0x800c0, but this dentry was not freed,
> > keeped by the dcache as unused, After tens of thousands of the
> > dentries slow down the dentry lookup performance, kernel memory usage
> Keep high.
> >
> > Regards,
> > Kevin
> 
> That is expected. The kernel does not get rid of negative dentries until the
> shrinker is called because of memory pressure. Negative dentries do help to
> improve file lookup performance. However, too much of negative dentries
> suppress the amount of free memory available for other use.
> That is why I send out my patch to limit the number of negative dentries
> outstanding.
> 
I think there are two issue:
1. when a file was removed, the dentry should be deleted, this is as a bug, if the
Dentry memory cannot be reclaimed, there is a memory leak.

2. limit the dentries number can improve when there were lots of files operations,
And all the files were valid.

Regards,
Kevin