linux-kernel - [RFC PATCH] frontswap (v15). +--------+ |zsmalloc| +--------+ +---------+ +------------+ | | swap +--->| frontswap + v +---------+ +------------| +--------+ +----->| zcache

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <1334958255-6612-1-git-send-email-konrad.wilk@oracle.com>
Date:	Fri, 20 Apr 2012 17:44:09 -0400
From:	Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>
To:	linux-mm@...ck.org, akpm@...ux-foundation.org,
	linux-kernel@...r.kernel.org, ngupta@...are.org,
	sjenning@...ux.vnet.ibm.com, rcj@...ux.vnet.ibm.com,
	aarcange@...hat.com, dhowells@...hat.com, riel@...hat.com,
	JBeulich@...ell.com
Subject: [RFC PATCH] frontswap (v15). +--------+ |zsmalloc| +--------+ +---------+    +------------+          | | swap    +--->| frontswap  +          v +---------+    +------------|      +--------+  +----->| zcache | +--------+

[Example usage, others are tmem, ramster, and RFC KVM]

Frontswap provides a "transcendent memory" interface for swap pages.
In some environments, dramatic performance savings may be obtained because
swapped pages are saved in RAM (or a RAM-like device) instead of a swap disk.
A nice overview of it is visible at: http://lwn.net/Articles/454795/

The backends, such as zcache provides compression of pages resulting
in speed up [https://lkml.org/lkml/2012/3/22/383]. If the backends
are not enabled and CONFIG_FRONTSWAP is compiled there is no adverse
performance hit (details further down).

Patches are also at:
git://git.kernel.org/pub/scm/linux/kernel/git/konrad/mm.git  stable/frontswap.v15.squashed

The last time these patches were posted [https://lkml.org/lkml/2011/10/27/206]
the discussion got very technical - and the feeling I got was that:
 - The API is too simple. The hard decisions (what to do when memory
   is low and the pages are mlocked, disk is faster than the CPU compression,
   need some way to shed pages when OOM conditions are close by, which pages
   to compress) are left to the backends. Adding VM pressure hooks could solve 
   some (if not all) of these issues? Dan is working on figuring this out for
   zcache.
 - the backends - like zcache - are tied in how this API is used. This means
   that to get zcache out of staging need to think of the frontswap and
   zcache (and also the other backends).

So to rehash, I think the core issues were with the backends [note, I 
ommitted some here - that is not b/c I choose to ignore them - it just that 
there were some many and I believe many of them got resolved in the discussion?]
 - IRQ latency of zcache. Code needs to be rewritten in zcache a bit.
 - VM pressure - need to be able to influence the backends (or the
   frontswap API as a whole)
 - backends need to handle batched requests.
 - backend needs to fix glaring atrocities (casting of 'struct page *' to
   'char *' and then back).

frontswap API:
 - perhaps make it a device driver and stack it (similar to loopback)? So
   frontswap_ioctl /dev/sda3 /dev/frontswap1 [bind it]
   swapon /dev/frontswap1

   The work required here means more invasive patches in the swap code and
   block code to deal with a dynamic sized disk.
 - only do it on recently activate pages [https://lkml.org/lkml/2012/1/26/520]


Compared to the last posting [https://lkml.org/lkml/2011/10/27/206]
 - a writethrough option. Meaning hat every swap page gets written
   to both frontswap AND to the swap disk. The backend can choose to tell frontswap
   to stop sending pages to it. This would allow the backend to react to memory pressure
   or to shut down if it is taking too much time compressing and pages end up in
   proper swap.
 - change the put/get to a different name: store/load
 - make the return values be bool instead of the int - it was getting confusing
 - seperate the CONFIG_DEBUGFS options out

Documentation/vm/frontswap.txt        |  278 +++++++++++++++++++++++++++++
 MAINTAINERS                           |    7 +
 drivers/staging/ramster/zcache-main.c |    8 +-
 drivers/staging/zcache/zcache-main.c  |   10 +-
 drivers/xen/tmem.c                    |    8 +-
 include/linux/frontswap.h             |  127 +++++++++++++
 include/linux/swap.h                  |    4 +
 include/linux/swapfile.h              |   13 ++
 mm/Kconfig                            |   17 ++
 mm/Makefile                           |    1 +
 mm/frontswap.c                        |  314 +++++++++++++++++++++++++++++++++
 mm/page_io.c                          |   12 ++
 mm/swapfile.c                         |   54 +++++--
 13 files changed, 827 insertions(+), 26 deletions(-)
Dan Magenheimer (4):
      mm: frontswap: add frontswap header file
      mm: frontswap: core swap subsystem hooks and headers
      mm: frontswap: core frontswap functionality
      mm: frontswap: config and doc files

Konrad Rzeszutek Wilk (2):
      MAINTAINER: Add myself for the frontswap API
      frontswap: s/put_page/store/g s/get_page/load

Benchmarks:

James asked about if compiled in but not used (so FRONTSWAP=y but no 'zcache' on
the Linux comnand line argument). Dan did some work in this and found that the
that CONFIG_FRONTSWAP=y is faster... or actually realistically just well within any
measureable noise. 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/