Date:	Mon, 16 Apr 2012 12:22:10 -0600
From:	Stephan Uphoff <ups@...gle.com>
To:	Arnd Bergmann <arnd@...db.de>
Cc:	Minchan Kim <minchan@...nel.org>, linaro-kernel@...ts.linaro.org,
	android-kernel@...glegroups.com, linux-mm@...ck.org,
	"Luca Porzio (lporzio)" <lporzio@...ron.com>,
	Alex Lemberg <alex.lemberg@...disk.com>,
	linux-kernel@...r.kernel.org, Saugata Das <saugata.das@...aro.org>,
	Venkatraman S <venkat@...aro.org>,
	Yejin Moon <yejin.moon@...sung.com>,
	Hyojin Jeong <syr.jeong@...sung.com>,
	"linux-mmc@...r.kernel.org" <linux-mmc@...r.kernel.org>
Subject: Re: swap on eMMC and other flash

I really like where this is going and would like to use the
opportunity to plant a few ideas.

In contrast to rotational disks read/write operation overhead and
costs are not symmetric.
While random reads are much faster on flash - the number of write
operations is limited by wearout and garbage collection overhead.
To further improve swapping on eMMC or similar flash media I believe
that the following issues need to be addressed:

1) Limit average write bandwidth to eMMC to a configurable level to
guarantee a minimum device lifetime
2) Aim for a low write amplification factor to maximize useable write bandwidth
3) Strongly favor read over write operations

Lowering write amplification (2) has been discussed in this email
thread - and the only observation I would like to add is that
over-provisioning the internal swap space compared to the exported
swap space significantly can guarantee a lower write amplification
factor with the indirection and GC techniques discussed.

I believe the swap functionality is currently optimized for storage
media where read and write costs are nearly identical.
As this is not the case on flash I propose splitting the anonymous
inactive queue (at least conceptually) - keeping clean anonymous pages
with swap slots on a separate queue as the cost of swapping them
out/in is only an inexpensive read operation. A variable similar to
swapiness (or a more dynamic algorithmn) could determine the
preference for swapping out clean pages or dirty pages. ( A similar
argument could be made for splitting up the file inactive queue )

The problem of limiting the average write bandwidth reminds me of
enforcing cpu utilization limits on interactive workloads.
Just as with cpu workloads - using the resources to the limit produces
poor interactivity.
When interactivity suffers too much I believe the only sane response
for an interactive device is to limit usage of the swap device and
transition into a low memory situation - and if needed - either
allowing userspace to reduce memory usage or invoking the OOM killer.
As a result low memory situations could not only be encountered on new
memory allocations but also on workload changes that increase the
number of dirty pages.

A wild idea to avoid some writes altogether is to see if
de-duplication techniques can be used to (partially?) match pages
previously written so swap.
In case of unencrypted swap  (or encrypted swap with a static key)
swap pages on eMMC could even be re-used across multiple reboots.
A simple version would just compare dirty pages with data in their
swap slots as I suspect (but really don't know) that some user space
algorithms (garbage collection?) dirty a page just temporarily -
eventually reverting it to the previous content.

Stephan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists