linux-kernel - Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <46AAF1CF.6060908@gmail.com>
Date:	Sat, 28 Jul 2007 09:35:43 +0200
From:	Rene Herman <rene.herman@...il.com>
To:	Björn Steinbrink <B.Steinbrink@....de>,
	Rene Herman <rene.herman@...il.com>,
	Daniel Hazelton <dhazelton@...er.net>,
	Mike Galbraith <efault@....de>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Ingo Molnar <mingo@...e.hu>,
	Frank Kingswood <frank@...gswood-consulting.co.uk>,
	Andi Kleen <andi@...stfloor.org>,
	Nick Piggin <nickpiggin@...oo.com.au>,
	Ray Lee <ray-lk@...rabbit.org>,
	Jesper Juhl <jesper.juhl@...il.com>,
	ck list <ck@....kolivas.org>, Paul Jackson <pj@....com>,
	linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: RFT: updatedb "morning after" problem [was: Re: -mm merge	plans
 for 2.6.23]

On 07/28/2007 01:15 AM, Björn Steinbrink wrote:

> On 2007.07.27 20:16:32 +0200, Rene Herman wrote:

>> Here's swap-prefetch's author saying the same:
>>
>> http://lkml.org/lkml/2007/2/9/112
>>
>> | It can't help the updatedb scenario. Updatedb leaves the ram full and
>> | swap prefetch wants to cost as little as possible so it will never
>> | move anything out of ram in preference for the pages it wants to swap
>> | back in.
>>
>> Now please finally either understand this, or tell us how we're wrong.
> 
> Con might have been wrong there for boxes with really little memory.

Note -- with "the updatedb scenario" both he in the above and I are talking 
about the "VFS caches filling memory cause the problem" not updatedb in 
particular.

> My desktop box has not even 300k inodes in use (IIRC someone posted a df 
> -i output showing 1 million inodes in use). Still, the memory footprint 
> of the "sort" process grows up to about 50MB. Assuming that the average 
> filename length stays, that would mean 150MB for the 1 million inode 
> case, just for the "sort" process.

Even if it's not 150MB, 50MB is already a lot on a 128 or even a 256MB box. 
So, yes, we're now at the expected scenario of some hog pushing out things 
and freeing it upon exit again and it's something swap-prefetch definitely 
has potential to help with.

Said early in the thread it's hard to imagine how it would not help in any 
such situation so that the discussion may as far as I'm concerned at that 
point concentrate on whether swap-prefetch hurts anything in others.

Some people I believe are not convinced it helps very significantly due to 
at that point _everything_ having been thrown out but a copy of openoffice 
with a large spreadsheet open should come back to life much quicker it would 
seem.

> Any faults in that reasoning?

No. If the machine goes idle after some memory hog _itself_ pushes things 
out and then exits, swap-prefetch helps, at the veryvery least potentially.

By the way -- I'm unable to make my slocate grow substantial here but I'll 
try what GNU locate does. If it's really as bad as I hear then regardless of 
anything else it should really be either fixed or dumped...

Rene.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/