linux-kernel - RFC: Easy-Reclaimable LRU list

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <4FE012CD.6010605@kernel.org>
Date:	Tue, 19 Jun 2012 14:49:01 +0900
From:	Minchan Kim <minchan@...nel.org>
To:	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	LKML <linux-kernel@...r.kernel.org>
CC:	Rik van Riel <riel@...hat.com>, Mel Gorman <mgorman@...e.de>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	KOSAKI Motohiro <kosaki.motohiro@...il.com>,
	Johannes Weiner <hannes@...xchg.org>,
	Andrea Arcangeli <aarcange@...hat.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Anton Vorontsov <anton.vorontsov@...aro.org>,
	John Stultz <john.stultz@...aro.org>,
	Pekka Enberg <penberg@...nel.org>,
	Wu Fengguang <fengguang.wu@...el.com>,
	Hugh Dickins <hughd@...gle.com>
Subject: RFC:  Easy-Reclaimable LRU list

Hi everybody!

Recently, there are some efforts to handle system memory pressure.

1) low memory notification - [1]
2) fallocate(VOLATILE) - [2]
3) fadvise(NOREUSE) - [3]

For them, I would like to add new LRU list, aka "Ereclaimable" which is opposite of "unevictable".
Reclaimable LRU list includes _easy_ reclaimable pages.
For example, easy reclaimable pages are following as. 

1. invalidated but remained LRU list.
2. pageout pages for reclaim(PG_reclaim pages)
3. fadvise(NOREUSE)
4. fallocate(VOLATILE)

Their pages shouldn't stir normal LRU list and compaction might not migrate them, even.
Reclaimer can reclaim Ereclaimable pages before normal lru list and will avoid unnecessary
swapout in anon pages in easy-reclaimable LRU list.
It also can make admin measure how many we have available pages at the moment without latency.
It's very important in recent mobile systems because page reclaim/writeback is very critical
of application latency. Of course, it could affect normal desktop, too.
With it, we can calculate fast-available pages more exactly with NR_FREE_PAGES + NR_ERECLAIMABLE_PAGES,
for example. If it's below threshold we defined, we could trigger 1st level notification
if we really need prototying low memory notification.

We may change madvise(DONTNEED) implementation instead of zapping page immediately.
If memory pressure doesn't happen, pages are in memory so we can avoid so many minor fault.
Of course, we can discard instead of swap out if system memory pressure happens.
We might implement it madvise(VOLATILE) instead of DONTNEED, but anyway it's off-topic in this thread.

As a another example, we can implement CFLRU(Clean-First LRU) which reclaims unmapped-clean cache page firstly.
The rationale is that in non-rotation device, read/write cost is much asynchronous.
Read is very fast while write is very slow so it would be a gain while we can avoid writeback of dirty pages
if possible although we need several reads. It can be implemented easily with Ereclaimable pages, too.

Anyway, it's just a brain-storming phase and never implemented yet but decide posting before it's too late.
I hope listen others opinion before get into the code.

Any comment are welcome.
Thanks.

[1] http://lkml.org/lkml/2012/5/1/97
[2] https://lkml.org/lkml/2012/6/1/322
[3] https://lkml.org/lkml/2011/6/24/136

-- 
Kind regards,
Minchan Kim
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/