Message-Id: <20080611184214.605110868@redhat.com>
Date:	Wed, 11 Jun 2008 14:42:14 -0400
From:	Rik van Riel <riel@...hat.com>
To:	linux-kernel@...r.kernel.org
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Lee Schermerhorn <lee.schermerhorn@...com>,
	Kosaki Motohiro <kosaki.motohiro@...fujitsu.com>
Subject: [PATCH -mm 00/24] VM pageout scalability improvements (V12)

Andrew, this is the promised drop-in replacement for the patch series,
with all the cleanups you requested since friday as well as the bugfixes
I came up with this morning.

You can drop vmscan-add-some-sanity-checks-to-get_scan_ratio.patch
and the incremental changes - those are all folded in to these patches.


On large memory systems, the VM can spend way too much time scanning
through pages that it cannot (or should not) evict from memory. Not
only does it use up CPU time, but it also provokes lock contention
and can leave large systems under memory presure in a catatonic state.

This patch series improves VM scalability by:

1) putting filesystem backed, swap backed and unevictable pages
   onto their own LRUs, so the system only scans the pages that it
   can/should evict from memory

2) switching to two handed clock replacement for the anonymous LRUs,
   so the number of pages that need to be scanned when the system
   starts swapping is bound to a reasonable number

3) keeping unevictable pages off the LRU completely, so the
   VM does not waste CPU time scanning them. ramfs, ramdisk,
   SHM_LOCKED shared memory segments and mlock()ed VMA pages
   are keept on the unevictable list.

More info on the overall design can be found at:

	http://linux-mm.org/PageReplacementDesign

An all-in-one patch can be found at:

	http://people.redhat.com/riel/splitvm/

Changelog:
- fix the merge bugs
- leave swappiness at 60, if only to demonstrate why that value is
  wrong with the new code (hi Andrew)
- update Documentation/vm/unevictable-lru.txt until my hands hurt
  from typing
- rename try_to_unlock to try_to_munlock
- remove CONFIG_NORECLAIM_MLOCK, only use CONFIG_UNEVICTABLE_LRU
- Aunt Tillified the CONFIG_UNEVICTABLE_LRU description
- make CONFIG_NORECLAIM_LRU no longer depend on 64BIT and default y
- rename NORECLAIM to UNEVICTABLE as suggested by Andrew Morton
- fix vmscan-fix-pagecache-reclaim-referenced-bit-check.patch so
  the referenced bit set test is the same as the shrink_page_list test
- add comments everywhere
- rename inactive_anon_low to inactive_anon_is_low
- use an array for lru stats in meminfo_read_proc
- pull patch 08/25 into 06 and 07 to ease reviewing
  Andrew: please remove vmscan-add-some-sanity-checks-to-get_scan_ratio.patch
- rename page_file_cache to page_is_file_cache
- pull page_lru() into patch 02/25 (Rik van Riel)
- use NR_LRU_BASE and LRU_BASE as base for calculations
- make zone->lru stuff cache friendly as suggested by Andrew Morton

- merge in all of Lee's mlock changes

- fix shrink_list memcgroup balancing (KOSAKI Motohiro)
- fix balancing stats in shrink_active_list (Daisuke Nishimura)

- make sure previously active pagecache pages get reactivated
  on the first access (Rik van Riel)
- compile fix when !CONFIG_SWAP (MinChan Kim)
- clean up page-flags.h defines when !CONFIG_NORECLAIM_LRU
  (Lee Schermerhorn)
- fix some race conditions around moving pages to and from
  the noreclaim list (Lee Schermerhorn, KOSAKI Motohiro)
- use putback_lru_page() for page migration (Lee Schermerhorn)
- fix potential SHM_UNLOCK race in scan_mapping_noreclaim_pages()
  (Lee Schermerhorn, KOSAKI Motohiro)

- improve swap space freeing to deal with COW shared space
  (Lee Schermerhorn, Daisuke Nishimura & Minchan Kim)
- clean up PG_swapbacked setting in swapin path (Minchan Kim)
- properly invoke shrink_active_list for background aging (Minchan Kim)
- add authorship info to all patches (Rik van Riel)
- clean up (or move below ---) the comments for the commit logs (Rik van Riel)
- after some tests, reduce default swappiness to 20 for now (Rik van Riel)

- several code cleanups (minchan Kim)
- noreclaim patch refactoring and improvements (Lee Schermerhorn)
- several PROT_NONE and vma merging fixes (KOSAKI Motohiro)
- SMP bugfixes and efficiency improvements (Rik van Riel, Lee Schermerhorn)
- fix NUMA node stats printing (Lee Schermerhorn)
- remove the mlocked-VMA-noreclaim code for now, it still has
  bugs on IA64 and is holding up the merge (Rik van Riel)

- make page_alloc.c compile without CONFIG_NORECLAIM_MLOCK (minchan Kim)
- BUG() does not take an argument (minchan Kim) 
- clean up is_active_lru and is_file_lru (Andy Whitcroft)
- clean up shrink_active_list temp list names (KOSAKI Motohiro)
- add total active & inactive memory totals for vmstat -a (KOSAKI Motohiro)
- only try global anon page aging on global lru scans (KOSAKI Motohiro)
- make function descriptions follow the kernel-doc format (Rik van Riel)
- simplify mlock_vma_pages_range and munlock_vma_pages_range (Lee Schermerhorn)
- remove some more arguments, rename to mlock_vma_pages_all (Lee Schermerhorn)
- many code cleanups (Lee Schermerhorn)
- pass correct vma arg to mlock_vma_pages_range from do_brk (Rik van Riel)
- port to 2.6.25-rc3-mm1

- pull the memcontrol lru arrayification earlier into the patch series
- use a pagevec array similar to the lru array
- clean up the code in various places
- improved pageout balancing and reduced pageout cpu use

- fix compilation on PPC and without memcontrol
- make page_is_pagecache more readable
- replace get_scan_ratio with correct version

- merge memcontroller split LRU code into the main split LRU patch,
  since it is not functionally different (it was split up only to help
  people who had seen the last version of the patch series review it)
- drop the page_file_cache debugging patch, since it never triggered
- reintroduce code to not scan anon list if swap is full
- add code to scan anon list if page cache is very small already
- use lumpy reclaim more aggressively for smaller order > 1 allocations

-- 
All Rights Reversed

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists