linux-kernel - [RFC][PATCH] split file and anonymous page queues #3

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <46005B4A.6050307@redhat.com>
Date:	Tue, 20 Mar 2007 18:08:10 -0400
From:	Rik van Riel <riel@...hat.com>
To:	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
CC:	linux-mm <linux-mm@...ck.org>
Subject: [RFC][PATCH] split file and anonymous page queues #3

[ OK, I suck.  I edited yesterday's email with the new info, but forgot
   to change the attachment to today's patch.  Here is today's patch. ]

Split the anonymous and file backed pages out onto their own pageout
queues.  This we do not unnecessarily churn through lots of anonymous
pages when we do not want to swap them out anyway.

This should (with additional tuning) be a great step forward in
scalability, allowing Linux to run well on very large systems where
scanning through the anonymous memory (on our way to the page cache
memory we do want to evict) is slowing systems down significantly.

The file backed queues and anon/swap queues receive different
pageout pressure.  Basically the larger the fraction of pages
on each queue that were recently referenced, the more we let
off the pressure, since that queue contains useful data.

For example, if 75% of the pages scanned on the anon/swap queues
were referenced, but only 25% of the pages scanned on the file
queues were referenced (ignoring used-once pages), we will put
3 times as much pressure on each file page as on each anon page.
This is further modified by the /proc/sys/vm/swappiness parameter.

---
This patch has been stress tested and seems to work, but has not
been fine tuned or benchmarked yet.  For now the swappiness parameter
can be used to tweak swap aggressiveness up and down as desired, but
in the long run we may want to simply measure IO cost of page cache
and anonymous memory and auto-adjust.

Please take this patch for a spin and let me know what goes well
and what goes wrong.

More info on the patch can be found on:

http://linux-mm.org/PageReplacementDesign

Signed-off-by: Rik van Riel <riel@...hat.com>

Changelog #3:
- Change some whitespace on Andrew's request.
- Use unsigned long, not ULL since the calculations in
    get_scan_ratio() no longer need numbers that big.
Changelog #2:
- Fix page_anon() to put all the file pages really on the
     file list.
- Fix get_scan_ratio() to return more stable numbers, by
     properly keeping track of the scanned anon and file pages.

-- 
All Rights Reversed

View attachment "linux-2.6-vm-split.patch" of type "text/x-patch" (51325 bytes)