linux-kernel - Re: [sshfs] inode problem when using git on a sshfs filesystem

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <ac00e83c1b57e2eeda1c6631ec611e75.squirrel@webmail.ocsa-data.net>
Date:	Thu, 17 Feb 2011 19:11:14 +0100
From:	"Yann Droneaud" <yann@...neaud.fr>
To:	"Miklos Szeredi" <miklos@...redi.hu>
Cc:	"Yann Droneaud" <yann@...neaud.fr>, miklos@...redi.hu,
	fuse-sshfs@...ts.sourceforge.net, fuse-devel@...ts.sourceforge.net,
	git@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [sshfs] inode problem when using git on a sshfs filesystem

> On Thu, 17 Feb 2011, Yann Droneaud wrote:
>> > On Wed, 16 Feb 2011, Yann Droneaud wrote:
>> >> Hi,
>> >>
>> >> For some days, my usage of git is not as seamless as before.
>> >>
>> >> I'm using git along sshfs/fuse (don't blame me for that), and
>> >> each time I try to rebase one of my branch, I have a conflict when
>> >> applying
>> >> the third commit. Doing the same operation on a local filesystem
>> works
>> >> without any problem.
>> >
>> > Yann, thanks for looking into this.
>> >
>> > Your findings are not surprising: unlike NFS, sshfs doesn't provide
>> > inode numbers and the fuse library also doesn't guarantee stable inode
>> > numbers by default.
>> >
>>
>> But why does it have such behavior when trying to rmdir() a non empty
>> directory ?
>
> The VFS (part of the linux kernel that implements the generic
> filesystem logic) clears the directory entry from the cache prior to
> actually trying to remove the directory.  This has the effect that any
> children of the directory are also cleared from the cache, hence the
> behavior you see in rmdir.
>

I tried to check that behavor: if the VFS is dropping dentry before doing
a rmdir(), them lookup files under this directory should be slower than
before rmdir().

On a ext4 filesystem, directory with 338 sub directories and 1992 files, 
i've tried the following commands:

/* drop all */
# echo 3 > /proc/sys/vm/drop_caches
# time ls -alR directory > /dev/null
real	0m0.140s
user	0m0.000s
sys	0m0.080s

/* drop cache */
# echo 1 > /proc/sys/vm/drop_caches
# time ls -alR directory > /dev/null
real	0m0.119s
user	0m0.000s
sys	0m0.072s

/* drop dentry and inode */
# echo 2 > /proc/sys/vm/drop_caches
# time ls -alR directory > /dev/null
real	0m0.089s
user	0m0.016s
sys	0m0.040s

/* read from cache */
# time ls -alR directory > /dev/null
real	0m0.063s
user	0m0.004s
sys	0m0.036s

$ rmdir directory
rmdir: failed to remove `directory': Directory not empty

/* re read from cache */
$ time ls -alR directory > /dev/null
real	0m0.065s
user	0m0.012s
sys	0m0.036s

As you can see, after doing rmdir(), the time taken to walk trough the
directories is the same than before calling it, so the dentry seems not
flushed out of the cache after the rmdir().

I really prefer this behavior. The one you explained would be a real pain:
if one user call rmdir on / inside a loop, the full dentry cache will be
dropped each time: this would affect performance for the whole system.

But perhaps I doesn't understand how the VFS is working, which is the most
probable solution. Please correct me.

Regards.

-- 
Yann Droneaud


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/