linux-kernel - Re: Better support for (desktop) file search / indexing applications

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <201211101753.46211.Martin@lichtvoll.de>
Date:	Sat, 10 Nov 2012 17:53:45 +0100
From:	Martin Steigerwald <Martin@...htvoll.de>
To:	linux-kernel@...r.kernel.org
Cc:	Tvrtko Ursulin <tvrtko.ursulin@...lan.co.uk>,
	Jan Kara <jack@...e.cz>,
	John McCutchan <john@...nmccutchan.com>,
	Robert Love <rlove@...ve.org>,
	Eric Paris <eparis@...isplace.org>,
	Eric Paris <eparis@...hat.com>,
	Nepomuk Mailing List <nepomuk@....org>,
	Linux Filesystem Development Mailinglist 
	<linux-fsdevel@...r.kernel.org>
Subject: Re: Better support for (desktop) file search / indexing applications

Am Donnerstag, 1. November 2012 schrieb Tvrtko Ursulin:
> On Thursday 01 November 2012 13:52:42 Martin Steigerwald wrote:
> ...
> 
> > The following two main issues led to the discussion about adding
> > notification about user inotify watch limit or even having it raised
> > automatically via some policy kit mechanism:
> > 
> > 1) Watches are not working recursively. Thus one has to add a watch
> > to each sub directory.
> > 
> > 2) There are inotify file move events. But one has to watch source
> > and destination directory to get notified of a file move between
> > these. Thus one has to watch each directory again. File moves
> > outside the watched home directory will go unnotified unless every
> > other accessible directory is watched as well.
> > 
> > 
> > What would be nice to have for file indexers would be:
> > 
> > 1) Recursive notifications. I.e. one watch for /home/martin can
> > notify about everything what happens in sub directories of that
> > directory.
> > 
> > 2) File move events that work from the source directory. I.e. if
> > watching a directory like /home/martin recursively it would be nice
> > to be notified about:
> > 
> > a) A file is moved from one sub directory inside /home/martin to
> > another one inside it.
> > 
> > b) A file is moved outside /home/martin
> > 
> > While these enhancement would likely fix the issues desktop file
> > search applications have with the kernel notification APIs, there
> > might be other approaches I did not yet thought off... so feel free
> > to comment with your thoughts on it.
> 
> I will only comment on the real time indexing part since I had some
> part in the inception of fanotify and still remember a thing or two
> about it.
> 
> Perhaps you should look into how hard would it be to add directory or
> rename, and unlink events to fanotify. It may not be too hard.
> 
> In that case, even though it does not support recursive directory
> watches (I tried to implement this some time around 2009. but found it
> impossible to wedge into the fanotify locking model), it does support
> mount point watches. Which for the desktop use might be sufficient,
> assuming /home is typically a separate filesystem.
> 
> Downside with  this approach is that you have to filter out the events
> you do not care about like /home/some-other-user, or even more if
> /home is not a separate filesystem. Which with the current fanotify
> state can be done using paths, but that includes resolving a link in
> procfs which may be a too expensive thing to do.
> 
> Or perhaps it is acceptable, if you for example only cared about
> CLOSE_WRITE events (closure of file which were open for writing).
> 
> So I think for this part you have two options, have a go of extending
> directory watches to be recursive, or live with the mount watches
> giving you too much traffic.

Thanks for your suggestions.

Still fanotify needs root access and thus this would need a daemon running 
as root and some policy kit stuff to access it and in case of mount point 
watches robust and secure code so that each user may only see his/her own 
results.

Any other ideas from anyone?

Thanks.
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/