linux-kernel - Re: How can we make page replacement smarter

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <46ABF184.40803@redhat.com>
Date:	Sat, 28 Jul 2007 21:46:44 -0400
From:	Rik van Riel <riel@...hat.com>
To:	Al Boldi <a1426z@...ab.com>
CC:	Chris Snook <csnook@...hat.com>, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org
Subject: Re: How can we make page replacement smarter

Al Boldi wrote:

> Good idea, but unless we understand the problems involved, we are bound to 
> repeat it.  So my first question would be:  Why is swap-in so slow?
> 
> As I have posted in other threads, swap-in of consecutive pages suffers a 2x 
> slowdown wrt swap-out, whereas swap-in of random pages suffers over 6x 
> slowdown.
> 
> Because it is hard to quantify the expected swap-in speed for random pages, 
> let's first tackle the swap-in of consecutive pages, which should be at 
> least as fast as swap-out.  So again, why is swap-in so slow?

I suspect that this is a locality of reference issue.

Anonymous memory can get jumbled up by repeated free and
malloc cycles of many smaller objects.  The amount of
anonymous memory is often smaller than or roughly the same
size as system memory.

Locality of refenence to anonymous memory tends to be
temporal in nature, with the same sets of pages being
accessed over and over again.

Files are different.  File content tends to be grouped
in large related chunks, both logically in the file and
on disk.  Generally there is a lot more file data on a
system than what fits in memory.

Locality of reference to file data tends to be spatial
in nature, with one file access leading up to the system
accessing "nearby" data.  The data is not necessarily
touched again any time soon.

> Once we understand this problem, we may be able to suggest a smart 
> improvement.

Like the one on http://linux-mm.org/PageoutFailureModes ?

I have the LRU lists split and am working on getting SEQ
replacement implemented for the anonymous pages.

The most recent (untested) patches are attached.

-- 
Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is.  Each group
calls the other unpatriotic.

View attachment "linux-2.6.22-vmsplit.patch" of type "text/x-patch" (50691 bytes)

View attachment "linux-2.6.22-vmsplit-swapfree.patch" of type "text/x-patch" (588 bytes)