linux-kernel - Re: [PATCH 0/3] readahead drop behind and size adjustment

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1185344325.7105.91.camel@perkele>
Date:	Wed, 25 Jul 2007 02:18:45 -0400
From:	Eric St-Laurent <ericstl34@...patico.ca>
To:	Nick Piggin <nickpiggin@...oo.com.au>
Cc:	Rusty Russell <rusty@...tcorp.com.au>,
	Fengguang Wu <fengguang.wu@...il.com>,
	Dave Jones <davej@...hat.com>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	linux-kernel <linux-kernel@...r.kernel.org>,
	riel <riel@...hat.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Tim Pepper <lnxninja@...ibm.com>,
	Chris Snook <csnook@...hat.com>
Subject: Re: [PATCH 0/3] readahead drop behind and size adjustment

On Wed, 2007-25-07 at 15:19 +1000, Nick Piggin wrote:

> What *I* think is supposed to happen is that newly read in pages get
> put on the inactive list, and unless they get accessed againbefore
> being reclaimed, they are allowed to fall off the end of the list
> without disturbing active data too much.
> 
> I think there is a missing piece here, that we used to ease the reclaim
> pressure off the active list when the inactive list grows relatively
> much larger than it (which could indicate a lot of use-once pages in
> the system).

Maybe a new list should be added to put newly read pages in it. If they
are not used or used once after a certain period, they can be moved to
the inactive list (or whatever).

Newly read pages...

- ... not used after this period are excessive readahead, we discard
immediately.
- ... used only once after this period, we discard soon.
- ... used many/frequently are moved to active list.

Surely the scan rate (do I make sense?) should be different for this
newly-read list and the inactive list. 

I also remember your split mapped/unmapped active list patches from a
while ago.

Can someone point me to a up-to-date documentation about the Linux VM?
The books and documents I've seen are outdated.

> I think I've been banned from touching vmscan.c, but if you're keen to
> try a patch, I might be convinced to come out of retirement :)

I'm more than willing!  Now that CFS is merged, redirect your energies
from nicksched to nick-vm ;)

Patches against any tree (stable, linus, mm, rt) are good. But I prefer
the last stable release because it narrows down the possible problems
that a moving target like the development tree may have.

I test this on my main system, so patches with basic testing and
reasonable stability are preferred. I just want to avoid data corruption
bugs. FYI, I used to run the -rt tree most of the time.

> One man's trash is another's treasure: some people will want the
> files to remain in cache because they'll use them again (copy it
> somewhere else, or start editing it after being copied or whatever).
> 
> But yeah, we can probably do better at the sequential read/write
> case.

Sure, but there are many hints to detect this: *large* (> most of the
RAM), *streaming*, *used once*

But if a program mmap() a 3/4 of the RAM area and "play" in it, it's a
good sign that the streaming code shouldn't be active.

- Eric

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/