linux-kernel - Re: updatedb

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1185528368.7851.44.camel@Homer.simpson.net>
Date:	Fri, 27 Jul 2007 11:26:08 +0200
From:	Mike Galbraith <efault@....de>
To:	Rene Herman <rene.herman@...il.com>
Cc:	Jesper Juhl <jesper.juhl@...il.com>,
	Andika Triwidada <andika@...il.com>,
	Robert Deaton <false.hopes@...il.com>,
	linux-kernel@...r.kernel.org, ck list <ck@....kolivas.org>,
	linux-mm@...ck.org
Subject: Re: updatedb

On Fri, 2007-07-27 at 10:28 +0200, Rene Herman wrote:
> On 07/27/2007 09:54 AM, Mike Galbraith wrote:
> 
> > On Fri, 2007-07-27 at 08:00 +0200, Rene Herman wrote:
> > 
> >> The remaining issue of updatedb unnecessarily blowing away VFS caches is 
> >> being discussed (*) in a few thread-branches still running.
> > 
> > If you solve that, the swap thing dies too, they're one and the same
> > problem.
> 
> I still wonder what the "the swap thing" is though. People just kept saying 
> that swap-prefetch helped which would seem to indicate their problem didnt 
> have anything to do with updatedb.

I haven't rummaged around in the VM in quite a long while, so don't know
exactly where the balance lies any more, and have never looked at
swap-prefetch, but the mechanism of how swap-prefetch can help the
"morning after syndrome" seems simple enough:

Reclaim (swapout) a slew of application pages because there are
truckloads of utterly bored pages laying about when updatedb comes along
and introduces memory pressure in the middle of the night.  Updatedb
finishes, freeing some ram (doesn't matter how much) swap-prefetch
detects idle CPU, and begins faulting swapped out pages back in.  In the
process of doing so, memory pressure is generated, and now these freshly
accessed pages are a less lovely target than the now aging VFS caches
that updatedb bloated up, so they shrink back down enough that the
balance you had before updatedb ran is restored... with the notable
exception that cached data is now toast, so what you gained by faulting
god knows how frequently used pages back in isn't _necessarily_ going to
help you.  Heck, it could even step on what was left of your cached
working set after updatedb finished.

> Also, I know shit about the VFS so this may well be not very educated but to 
> me something like FADV_NOREUSE on a dirfd sounds like a much more promising 
> approach than the convoluted userspace schemes being discussed, if only 
> because it'll actually be implemented/used.

I like Andrew's mention of a future option... put that sucker and
everybody who looks like him in a resource limited container.

	-Mike

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/