linux-kernel - Re: Beyond inotify recursive watches

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CALkWK0=kMGcPMfQWWt97Ae8JWvXDT+CGfAC-JWObF00KbYiZ_Q@mail.gmail.com>
Date:	Thu, 11 Apr 2013 17:29:39 +0530
From:	Ramkumar Ramachandra <artagnon@...il.com>
To:	Jan Kara <jack@...e.cz>
Cc:	Al Viro <viro@...iv.linux.org.uk>, linux-kernel@...r.kernel.org
Subject: Re: Beyond inotify recursive watches

Jan Kara wrote:
>   Initially, you will have to flip the flag on every directory in the
> subtree. But the flag is persistently stored on disk so you have to do it
> once when the directory is created and then each time you notice the
> directory has changed and the flag has been cleared.

How is this any better than setting up inotify recursively on the
directory tree?  I'll have to readdir() each directory in the tree,
looking for more directories to set the flag on.

>   I think Al is asking how do we lock kernel dentry cache so that we can
> safely walk up the tree and update time stamps in presence of other
> modifications happening to the directory tree in parallel.

>   It's not as much timestamp updates themselves racing against each other
> but rather things like moving directories in the directory tree racing with
> us walking up the tree and updating time stamps - in Linux, directory
> locking happens in top-bottom manner (like when you do lookup of a path) so
> when you are climbing up, one has to be careful not to introduce races.

Oh.  So we need to carefully code very fine-grained locking (so that
the entire fs isn't unusable when this recursive update is happening).

>   One reason why we need things to be fs-internal is that we want to store
> everything permanently on disk so that e.g. if there's reboot between
> modification of a git tree and 'git add -u', you will still find what has
> changed since last time you've checked (without walking the whole tree).

Makes sense.  However, doesn't this mean that we have to patch every
filesystem separately for this feature, as opposed to just patching
VFS?

>   My concern is that if you have a directory tree where there are lots of
> entries in each directory, then you still have to check a lot of entries
> before you find what has changed because you have to scan all entries in
> each directory on the modified path. If there was a way to iterate only
> through entries in a directory which had the flag cleared, things could be
> considerably faster.

What are your thoughts on introducing a version of readdir() that only
lists dentries with this flag?

Can you get deeper into the implementation, and point me to the parts
of the code to look at?  Do you have any WIP patches that I can look
at?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/