linux-kernel - Re: [PATCH v2] z3fold: the 3-fold allocator for compressed pages

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 25 Apr 2016 09:54:51 -0500
From:	Seth Jennings <sjenning@...hat.com>
To:	Vlastimil Babka <vbabka@...e.cz>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Vitaly Wool <vitalywool@...il.com>,
	Linux-MM <linux-mm@...ck.org>,
	LKML <linux-kernel@...r.kernel.org>,
	Dan Streetman <ddstreet@...e.org>
Subject: Re: [PATCH v2] z3fold: the 3-fold allocator for compressed pages

On Mon, Apr 25, 2016 at 2:28 AM, Vlastimil Babka <vbabka@...e.cz> wrote:
> On 04/22/2016 01:22 AM, Andrew Morton wrote:
>>
>> On Tue, 19 Apr 2016 11:48:45 +0200 Vitaly Wool <vitalywool@...il.com>
>> wrote:
>>
>>> This patch introduces z3fold, a special purpose allocator for storing
>>> compressed pages. It is designed to store up to three compressed pages
>>> per
>>> physical page. It is a ZBUD derivative which allows for higher
>>> compression
>>> ratio keeping the simplicity and determinism of its predecessor.
>>>
>>> The main differences between z3fold and zbud are:
>>> * unlike zbud, z3fold allows for up to PAGE_SIZE allocations
>>> * z3fold can hold up to 3 compressed pages in its page
>>>
>>> This patch comes as a follow-up to the discussions at the Embedded Linux
>>> Conference in San-Diego related to the talk [1]. The outcome of these
>>> discussions was that it would be good to have a compressed page allocator
>>> as stable and deterministic as zbud with with higher compression ratio.
>>>
>>> To keep the determinism and simplicity, z3fold, just like zbud, always
>>> stores an integral number of compressed pages per page, but it can store
>>> up to 3 pages unlike zbud which can store at most 2. Therefore the
>>> compression ratio goes to around 2.5x while zbud's one is around 1.7x.
>>>
>>> The patch is based on the latest linux.git tree.
>>>
>>> This version of the patch has updates related to various concurrency
>>> fixes
>>> made after intensive testing on SMP/HMP platforms.
>>>
>>>
>>> [1]https://openiotelc2016.sched.org/event/6DAC/swapping-and-embedded-compression-relieves-the-pressure-vitaly-wool-softprise-consulting-ou
>>>
>>
>> So...  why don't we just replace zbud with z3fold?  (Update the changelog
>> to answer this rather obvious question, please!)
>
>
> There was discussion between Seth and Vitaly on v1. Without me knowing the
> details myself, it looked like Seth's objections were addressed, but then
> the thread died. I think there should first be a more clear answer from Seth
> whether z3fold really looks like a clear win (i.e. not workload-dependent)
> over zbud, in which case zbud could be extended?

(sorry for the dup Vlastimil, didn't reply-to-all)

It seems like it could be in the case that most of the pages in your
system compress to 1/3 their original size (on average).  In my
original research, I found that, using lzo, 1/2 a page was more
typical.  However, if you used deflate, you might be able to push the
average down.

IMO I do think we should try to merge zbud and z3fold with zbud being
the default mode (2 object per page) and have an option to enable the
3 objects per page logic.  IIRC that 3rd object logic seemed to be
fairly contained.  Having the separate would duplicate a lot of very
similar code.

However, if Andrew is ok with yet another z- allocator, it can just be
another zpool backend.  I'm fine either way.  Just my two cents.

Seth

>