linux-kernel - [PATCH 0/5][RFC] Fallocate Volatile Ranges v6

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-Id: <1343447832-7182-1-git-send-email-john.stultz@linaro.org>
Date:	Fri, 27 Jul 2012 23:57:07 -0400
From:	John Stultz <john.stultz@...aro.org>
To:	LKML <linux-kernel@...r.kernel.org>
Cc:	John Stultz <john.stultz@...aro.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Android Kernel Team <kernel-team@...roid.com>,
	Robert Love <rlove@...gle.com>, Mel Gorman <mel@....ul.ie>,
	Hugh Dickins <hughd@...gle.com>,
	Dave Hansen <dave@...ux.vnet.ibm.com>,
	Rik van Riel <riel@...hat.com>,
	Dmitry Adamushko <dmitry.adamushko@...il.com>,
	Dave Chinner <david@...morbit.com>, Neil Brown <neilb@...e.de>,
	Andrea Righi <andrea@...terlinux.com>,
	"Aneesh Kumar K.V" <aneesh.kumar@...ux.vnet.ibm.com>,
	Mike Hommey <mh@...ndium.org>, Jan Kara <jack@...e.cz>,
	KOSAKI Motohiro <kosaki.motohiro@...il.com>,
	Michel Lespinasse <walken@...gle.com>,
	Minchan Kim <minchan@...nel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>
Subject: [PATCH 0/5][RFC] Fallocate Volatile Ranges v6

So after not getting too much positive feedback on my last
attempt at trying to use a non-shrinker method for managing
& purging volatile ranges, I decided I'd go ahead and try
to implement something along Minchan's ERECLAIM LRU list
idea.

Again this patchset has two parts:

The first 3 patches add generic volatile range management code, as 
well as tmpfs support for FALLOC_FL_MARK_VOLATILE, which uses a
shrinker to purge ranges, and converts ashmem to use
FALLOC_FL_MARK_VOLATILE, almost reducing the driver in half.


Since Kosaki-san objected to using the shrinker, as its not numa
aware, and is only called after we shrink normal lru lists. The 
second half of this patch set provides a different method that is
not shrinker based.

The last two patches introduce a new lru list, LRU_VOLATILE.
When pages are marked volatile, they are moved to this lru list,
which we will shrink first when trying to free up memory.

The reason why I'm keeping this in two parts is that I want
to be able to easily do performance comparisons between
the more lightweight, but numa-unaware, shrinker method vs the
more correct, but slower (due to the page-by-page management)
method of handling it deeper in the VM.

I know the way this is currently implemented is really bad for
performance, since we add/remove pages to the LRU_VOLATILE list
page by page instead of batching. So I know this needs more work,
but I wanted to get just some initial reactions to this approach
versus the earlier ones.

Also, I know this isn't exactly the same as the ERECLAIM lru list
that Minchan suggested, since that might also contain inactive
clean file pages as well, but I wanted to stick to just volatile
pages for now as I learn more about how the core vm works. I'll
be fine to rename LRU_VOLATILE later if its appropriate.

What's new in this iteration:
* Dropped the writepage style purging instead for the LRU_VOLATILE

CC: Andrew Morton <akpm@...ux-foundation.org>
CC: Android Kernel Team <kernel-team@...roid.com>
CC: Robert Love <rlove@...gle.com>
CC: Mel Gorman <mel@....ul.ie>
CC: Hugh Dickins <hughd@...gle.com>
CC: Dave Hansen <dave@...ux.vnet.ibm.com>
CC: Rik van Riel <riel@...hat.com>
CC: Dmitry Adamushko <dmitry.adamushko@...il.com>
CC: Dave Chinner <david@...morbit.com>
CC: Neil Brown <neilb@...e.de>
CC: Andrea Righi <andrea@...terlinux.com>
CC: Aneesh Kumar K.V <aneesh.kumar@...ux.vnet.ibm.com>
CC: Mike Hommey <mh@...ndium.org>
CC: Jan Kara <jack@...e.cz>
CC: KOSAKI Motohiro <kosaki.motohiro@...il.com>
CC: Michel Lespinasse <walken@...gle.com>
CC: Minchan Kim <minchan@...nel.org>
CC: linux-mm@...ck.org <linux-mm@...ck.org>


John Stultz (5):
  [RFC] Add volatile range management code
  [RFC] tmpfs: Add FALLOC_FL_MARK_VOLATILE/UNMARK_VOLATILE handlers
  [RFC] ashmem: Convert ashmem to use volatile ranges
  [RFC][HACK] Add LRU_VOLATILE support to the VM
  [RFC][HACK] Switch volatile/shmem over to LRU_VOLATILE

 drivers/staging/android/ashmem.c |  331 +--------------------------
 fs/open.c                        |    3 +-
 include/linux/falloc.h           |    7 +-
 include/linux/fs.h               |    1 +
 include/linux/mm_inline.h        |    2 +
 include/linux/mmzone.h           |    1 +
 include/linux/page-flags.h       |    3 +
 include/linux/swap.h             |    3 +
 include/linux/volatile.h         |   39 ++++
 mm/Makefile                      |    2 +-
 mm/memcontrol.c                  |    1 +
 mm/page_alloc.c                  |    1 +
 mm/shmem.c                       |  118 ++++++++++
 mm/swap.c                        |   71 ++++++
 mm/vmscan.c                      |   76 ++++++-
 mm/volatile.c                    |  459 ++++++++++++++++++++++++++++++++++++++
 16 files changed, 788 insertions(+), 330 deletions(-)
 create mode 100644 include/linux/volatile.h
 create mode 100644 mm/volatile.c

-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/