linux-kernel - Re: Issues with using fanotify for a filesystem indexer

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 02 Apr 2009 18:29:04 +0200
From:	Alexander Larsson <alexl@...hat.com>
To:	Jan Kara <jack@...e.cz>
Cc:	eparis@...hat.com, linux-kernel@...r.kernel.org
Subject: Re: Issues with using fanotify for a filesystem indexer

On Thu, 2009-04-02 at 16:54 +0200, Jan Kara wrote:

>   Some time ago I was trying to solve a similar problem and I've come up
> with a solution which I've called recursive mtime. The general idea is
> that with each directory, kernel additionally keeps a flag and a
> timestamp. When a directory is modified, we do:
>   dir = changed dir;
>   while dir has flag set do
>     update timestamp to current time
>     clear flag
>     dir = parent dir
> 
>   When a file is modified, you just start with a parent directory of
> that file. With this scheme, you are able to find reasonably quickly
> (without looking at unchanged directories) what has changed since
> you've looked last time (you look for timestamps newer than the time
> when you started last scan and you set flags as you go). Also the scheme
> is quite cheap to maintain and has no problems with overflowing event
> queues etc. (observe that the scheme works perfectly fine for several
> independent scanners in parallel). As a bonus, if you store the flag +
> timestamp persistently on disk, you can use this scheme to speedup things
> like rsync.
>   What gets nasty (but solvable) are hardlinks and bind mounts. I was
> writing a library to handle these last summer but then had to work on
> something else and didn't get back to it yet.

Another potential issue with this is that every change bubbles up to the
top, modifying the recursive mtime of that. This will become very
contented, and may imply a partial serialization of fs activity, which
is kinda costly.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/