linux-kernel - Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20070726033809.c1691bb0.akpm@linux-foundation.org>
Date:	Thu, 26 Jul 2007 03:38:09 -0700
From:	Andrew Morton <akpm@...ux-foundation.org>
To:	Ingo Molnar <mingo@...e.hu>
Cc:	Frank Kingswood <frank@...gswood-consulting.co.uk>,
	Andi Kleen <andi@...stfloor.org>,
	Nick Piggin <nickpiggin@...oo.com.au>,
	Ray Lee <ray-lk@...rabbit.org>,
	Jesper Juhl <jesper.juhl@...il.com>,
	ck list <ck@....kolivas.org>, Paul Jackson <pj@....com>,
	linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans
 for 2.6.23]

On Thu, 26 Jul 2007 12:27:30 +0200 Ingo Molnar <mingo@...e.hu> wrote:

> 
> * Andrew Morton <akpm@...ux-foundation.org> wrote:
> 
> > > > > ( we _do_ want to baloon the dentry cache otherwise - for things like 
> > > > >   "find" - having a fast VFS is important. But known-use-once things 
> > > > >   like the daily updatedb job can clearly be annotated properly. )
> > > > 
> > > > Mutter.  /proc/sys/vm/vfs_cache_pressure has been there for what, 
> > > > three years?  Are any distros raising it during the updatedb run yet?
> > > 
> > > but ... that's system-wide, and the 'dont baloon the dcache' is only a 
> > > property of updatedb.
> > 
> > Sure, but it's practical, isn't it?  Who runs (and cares about) 
> > vfs-intensive workloads during their wee-small-hours updatedb run?
> 
> there's another side-effect: it likely results in the zapping of 
> thousands of dentries that were cached nicely before. So we might 
> exchange 'all my apps are swapped out' experience with 'all file access 
> is slow'. The latter is _probably_ still an improvement over the 
> balooning, but i'm not sure.

Yup.  Nobody has begun to think about preserving dcache/icache across load
shifts yet, afaik.  Hard.

> What we _really_ want is an updatedb that 
> does not disturb the dcache.

Well.  Hopefully this time next year you can prep a 16MB container and toss
your updatedb inside that.  Maybe set its peak disk bandwidth utilisation
too.  However that won't work ;) because I don't think anyone is looking
at containerisation of vfs cache memory yet.  Perhaps full-on openvz has it,
dunno.

But updatedb is a special case, because it is so vfs-intensive.  For lots
of other workloads (those which use heaps of pagecache), resource
management via containerisation will work nicely.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/