linux-kernel - Re: [sshfs] inode problem when using git on a sshfs filesystem

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <E1PqA19-00080P-Dy@pomaz-ex.szeredi.hu>
Date:	Thu, 17 Feb 2011 20:59:59 +0100
From:	Miklos Szeredi <miklos@...redi.hu>
To:	"Yann Droneaud" <yann@...neaud.fr>
CC:	miklos@...redi.hu, miklos@...redi.hu,
	fuse-devel@...ts.sourceforge.net, linux-kernel@...r.kernel.org,
	fuse-sshfs@...ts.sourceforge.net, git@...r.kernel.org
Subject: Re: [sshfs] inode problem when using git on a sshfs filesystem

On Thu, 17 Feb 2011, Yann Droneaud wrote:
> > The VFS (part of the linux kernel that implements the generic
> > filesystem logic) clears the directory entry from the cache prior to
> > actually trying to remove the directory.  This has the effect that any
> > children of the directory are also cleared from the cache, hence the
> > behavior you see in rmdir.
> >
> 
> I tried to check that behavor: if the VFS is dropping dentry before doing
> a rmdir(), them lookup files under this directory should be slower than
> before rmdir().
> 
> On a ext4 filesystem, directory with 338 sub directories and 1992 files, 
> i've tried the following commands:
> 
> /* drop all */
> # echo 3 > /proc/sys/vm/drop_caches
> # time ls -alR directory > /dev/null
> real	0m0.140s
> user	0m0.000s
> sys	0m0.080s
> 
> /* drop cache */
> # echo 1 > /proc/sys/vm/drop_caches
> # time ls -alR directory > /dev/null
> real	0m0.119s
> user	0m0.000s
> sys	0m0.072s
> 
> /* drop dentry and inode */
> # echo 2 > /proc/sys/vm/drop_caches
> # time ls -alR directory > /dev/null
> real	0m0.089s
> user	0m0.016s
> sys	0m0.040s
> 
> /* read from cache */
> # time ls -alR directory > /dev/null
> real	0m0.063s
> user	0m0.004s
> sys	0m0.036s
> 
> $ rmdir directory
> rmdir: failed to remove `directory': Directory not empty
> 
> /* re read from cache */
> $ time ls -alR directory > /dev/null
> real	0m0.065s
> user	0m0.012s
> sys	0m0.036s
> 
> As you can see, after doing rmdir(), the time taken to walk trough the
> directories is the same than before calling it, so the dentry seems not
> flushed out of the cache after the rmdir().

tucsk:~ # time find /usr -ls > /dev/null

real    0m1.922s
user    0m1.104s
sys     0m0.800s
tucsk:~ # rmdir /usr
rmdir: failed to remove `/usr': Directory not empty
tucsk:~ # time find /usr -ls > /dev/null

real    0m2.637s
user    0m1.172s
sys     0m1.448s
tucsk:~ # time find /usr -ls > /dev/null

real    0m1.943s
user    0m1.068s
sys     0m0.848s
tucsk:~ # 

As you can see it does get significantly slower after the rmdir.

> I really prefer this behavior. The one you explained would be a real pain:
> if one user call rmdir on / inside a loop, the full dentry cache will be
> dropped each time: this would affect performance for the whole system.

Yes.  But in practice this doesn't really matter, removing non-empty
directories happens rarely.

Thanks,
Miklos
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/