linux-kernel - Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 27 Jul 2007 09:23:41 +0200
From:	Mike Galbraith <efault@....de>
To:	Andrew Morton <akpm@...ux-foundation.org>
Cc:	Ingo Molnar <mingo@...e.hu>,
	Frank Kingswood <frank@...gswood-consulting.co.uk>,
	Andi Kleen <andi@...stfloor.org>,
	Nick Piggin <nickpiggin@...oo.com.au>,
	Ray Lee <ray-lk@...rabbit.org>,
	Jesper Juhl <jesper.juhl@...il.com>,
	ck list <ck@....kolivas.org>, Paul Jackson <pj@....com>,
	linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: RFT: updatedb "morning after" problem [was: Re: -mm merge
	plans for 2.6.23]

On Fri, 2007-07-27 at 07:13 +0200, Mike Galbraith wrote:
> On Thu, 2007-07-26 at 11:05 -0700, Andrew Morton wrote:
> > > drops caches prior to both updatedb runs.
> > 
> > I think that was the wrong thing to do.  That will leave gobs of free
> > memory for updatedb to populate with dentries and inodes.
> > 
> > Instead, fill all of memory up with pagecache, then do the updatedb.  See
> > how much pagecache is left behind and see how large the vfs caches end up.

I didn't _fill_ memory, but loaded it up a bit with some real workload
data...

I tried time sh -c 'git diff v2.6.11 HEAD > /dev/null' to populate the
cache, and tried different values for vfs_cache_pressure.  Nothing
prevented git's data from being trashed by updatedb.  Turning the knob
downward rapidly became very unpleasant due to swap, (with 0 not
surprisingly being a true horror) but turning it up didn't help git one
bit.  The amount of data that had to be re-read with stock 100 or 10000
was the same, or at least so close that you couldn't see a difference in
vmstat and wall-clock.  Cache sizes varied, but the bottom line didn't.
(wasn't surprised, seems quite reasonable that git's data looks old and
useless to the reclaim logic when updatedb runs in between git runs)

	-Mike

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/