linux-kernel - Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20070726122330.GA21750@one.firstfloor.org>
Date:	Thu, 26 Jul 2007 14:23:30 +0200
From:	Andi Kleen <andi@...stfloor.org>
To:	Al Viro <viro@....linux.org.uk>
Cc:	Ingo Molnar <mingo@...e.hu>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Frank Kingswood <frank@...gswood-consulting.co.uk>,
	Andi Kleen <andi@...stfloor.org>,
	Nick Piggin <nickpiggin@...oo.com.au>,
	Ray Lee <ray-lk@...rabbit.org>,
	Jesper Juhl <jesper.juhl@...il.com>,
	ck list <ck@....kolivas.org>, Paul Jackson <pj@....com>,
	linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

> BTW, I really wonder how much pain could be avoided if updatedb recorded
> mtime of directories and checked it.  I.e. instead of just doing blind
> find(1), walk the stored directory tree comparing timestamps with those
> in filesystem.  If directory mtime has not changed, don't bother rereading
> it and just go for (stored) subdirectories.  If it has changed - reread the
> sucker.  If we have a match for stored subdirectory of changed directory,
> check inumber; if it doesn't match, consider the entire subtree as new
> one.  AFAICS, that could eliminate quite a bit of IO...

That would just save reading the directories. Not sure
it helps that much. Much better would be actually if it didn't stat the 
individual files (and force their dentries/inodes in). I bet it does that to 
find out if they are directories or not. But in a modern system it could just 
check the type in the dirent on file systems that support 
that and not do a stat. Then you would get much less dentries/inodes.

Also I expect in general the new slub dcache freeing that is pending
will improve things a lot.

But even if updatedb was fixed to be more efficient we probably
still need a general solution for other tree walking programs
that cannot be optimized this way.

-Andi

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/