linux-kernel - Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20070726110549.da3a7a0d.akpm@linux-foundation.org>
Date:	Thu, 26 Jul 2007 11:05:49 -0700
From:	Andrew Morton <akpm@...ux-foundation.org>
To:	Mike Galbraith <efault@....de>
Cc:	Ingo Molnar <mingo@...e.hu>,
	Frank Kingswood <frank@...gswood-consulting.co.uk>,
	Andi Kleen <andi@...stfloor.org>,
	Nick Piggin <nickpiggin@...oo.com.au>,
	Ray Lee <ray-lk@...rabbit.org>,
	Jesper Juhl <jesper.juhl@...il.com>,
	ck list <ck@....kolivas.org>, Paul Jackson <pj@....com>,
	linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans
 for 2.6.23]

On Thu, 26 Jul 2007 14:46:58 +0200 Mike Galbraith <efault@....de> wrote:

> On Thu, 2007-07-26 at 03:09 -0700, Andrew Morton wrote:
> 
> > Setting it to zero will maximise the preservation of the vfs caches.  You
> > wanted 10000 there.
> > 
> > <bets that nobody will test this>
> 
> drops caches prior to both updatedb runs.

I think that was the wrong thing to do.  That will leave gobs of free
memory for updatedb to populate with dentries and inodes.

Instead, fill all of memory up with pagecache, then do the updatedb.  See
how much pagecache is left behind and see how large the vfs caches end up.

> root@...er: df -i
> Filesystem            Inodes   IUsed   IFree IUse% Mounted on
> /dev/hdc3            12500992 1043544 11457448    9% /
> udev                  129162    1567  127595    2% /dev
> /dev/hdc1              26104      87   26017    1% /boot
> /dev/hda1             108144   90676   17468   84% /windows/C
> /dev/hda5              11136    3389    7747   31% /windows/D
> /dev/hda6                  0       0       0    -  /windows/E
> 
> vfs_cache_pressure=10000, updatedb freshly completed:
> procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
>  1  0     48  76348 420356 104748    0    0     0     0 1137  912  3  1 97  0
> 
> ext3_inode_cache  315153 316274    524    7    1 : tunables   54   27    8 : slabdata  45182  45182      0
> dentry_cache      224829 281358    136   29    1 : tunables  120   60    8 : slabdata   9702   9702      0
> buffer_head       156624 159728     56   67    1 : tunables  120   60    8 : slabdata   2384   2384      0
> 
> vfs_cache_pressure=100 (stock), updatedb freshly completed:
> 
> procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
> 1  0    148  83824 270088 116340    0    0     0     0 1095  330  2  1 97  0
>  
> ext3_inode_cache  467257 502495    524    7    1 : tunables   54   27    8 : slabdata  71785  71785      0
> dentry_cache      292695 408958    136   29    1 : tunables  120   60    8 : slabdata  14102  14102      0
> buffer_head       118329 184384     56   67    1 : tunables  120   60    8 : slabdata   2752   2752      1
> 
> Note:  updatedb doesn't bother my box, not running enough leaky apps I
> guess.
> 

So you ended up with a couple hundred MB of pagecache preserved.

Capturing before-and-after /proc/meminfo would be nice - it's a useful
summary.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/