linux-kernel - Disk Cache, Was: O

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1168637967.183616.9.camel@localhost>
Date:	Fri, 12 Jan 2007 14:39:27 -0700
From:	Zan Lynx <zlynx@....org>
To:	Michael Tokarev <mjt@....msk.ru>
Cc:	Chris Mason <chris.mason@...cle.com>,
	Linus Torvalds <torvalds@...l.org>,
	dean gaudet <dean@...tic.org>, Viktor <vvp01@...ox.ru>,
	Aubrey <aubreylee@...il.com>, Hua Zhong <hzhong@...il.com>,
	Hugh Dickins <hugh@...itas.com>, linux-kernel@...r.kernel.org,
	hch@...radead.org, kenneth.w.chen@...el.com, akpm@...l.org
Subject: Disk Cache, Was: O_DIRECT question

On Sat, 2007-01-13 at 00:03 +0300, Michael Tokarev wrote:
[snip]
> And sure thing, withOUT O_DIRECT, the whole system is almost dead under this
> load - because everything is thrown away from the cache, even caches of /bin
> /usr/bin etc... ;)  (For that, fadvise() seems to help a bit, but not alot).

One thing that I've been using, and seems to work well, is a customized
version of the readahead program several distros use during boot up.

Mine starts off doing:
mlockall(MCL_CURRENT|MCL_FUTURE);
...yadda, yadda...

and for each file listed:
...open, stat stuff...
       if( NULL == mmap(
                NULL, stat_buf.st_size, 
                PROT_READ, MAP_SHARED|MAP_LOCKED|MAP_POPULATE,
                fd, 0)
        ) {
                fprintf(stderr, "'%s' ", file);
                perror("mmap");
        }
...more stuff...
and then ends with:
pause();
and it sits there forever.

As far as I can tell, this makes the program and library code stay in
RAM.  At least, after a drop_caches nautilus doesn't load 12 MB off
disk, it just starts.  It has to be reloaded after software updates and
after prelinking.  I find the 250 MB used to be worthwhile, even if its
kinda Windowsey.

Something like that could keep your system responsive no matter what the
disk cache is doing otherwise.
-- 
Zan Lynx <zlynx@....org>

Download attachment "signature.asc" of type "application/pgp-signature" (190 bytes)