linux-kernel - Re: zsmalloc zbud hybrid design discussion?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130411193534.GB28296@cerebellum>
Date:	Thu, 11 Apr 2013 14:35:34 -0500
From:	Seth Jennings <sjenning@...ux.vnet.ibm.com>
To:	Dan Magenheimer <dan.magenheimer@...cle.com>
Cc:	Konrad Wilk <konrad.wilk@...cle.com>,
	Minchan Kim <minchan@...nel.org>, Bob Liu <bob.liu@...cle.com>,
	Robert Jennings <rcj@...ux.vnet.ibm.com>,
	Nitin Gupta <ngupta@...are.org>,
	Wanpeng Li <liwanp@...ux.vnet.ibm.com>, linux-mm@...ck.org,
	linux-kernel@...r.kernel.org
Subject: Re: zsmalloc zbud hybrid design discussion?

On Wed, Mar 27, 2013 at 01:04:25PM -0700, Dan Magenheimer wrote:
> Seth and all zproject folks --
> 
> I've been giving some deep thought as to how a zpage
> allocator might be designed that would incorporate the
> best of both zsmalloc and zbud.
> 
> Rather than dive into coding, it occurs to me that the
> best chance of success would be if all interested parties
> could first discuss (on-list) and converge on a design
> that we can all agree on.  If we achieve that, I don't
> care who writes the code and/or gets the credit or
> chooses the name.  If we can't achieve consensus, at
> least it will be much clearer where our differences lie.
> 
> Any thoughts?

I'll put some thoughts, keeping in mind that I'm not throwing zsmalloc under
the bus here.  Just what I would do starting from scratch given all that has
happened.

Simplicity - the simpler the better

High density - LZO best case is ~40 bytes. That's around 1/100th of a page.
I'd say it should support up to at least 64 object per page in the best case.
(see Reclaim effectiveness before responding here)

No slab - the slab approach limits LRU and swap slot locality within the pool
pages.  Also swap slots have a tendency to be freed in clusters.  If we improve
locality within each pool page, it is more likely that page will be freed
sooner as the zpages it contains will likely be invalidated all together.
Also, take a note out of the zbud playbook at track LRU based on pool pages,
not zpages.  One would fill allocation requests from the most recently used
pool page.

Reclaim effectiveness - conflicts with density. As the number of zpages per
page increases, the odds decrease that all of those objects will be
invalidated, which is necessary to free up the underlying page, since moving
objects out of sparely used pages would involve compaction (see next).  One
solution is to lower the density, but I think that is self-defeating as we lose
much the compression benefit though fragmentation. I think the better solution
is to improve the likelihood that the zpages in the page are likely to be freed
together through increased locality.

Not a requirement:

Compaction - compaction would basically involve creating a virtual address
space of sorts, which zsmalloc is capable of through its API with handles,
not pointer.  However, as Dan points out this requires a structure the maintain
the mappings and adds to complexity.  Additionally, the need for compaction
diminishes as the allocations are short-lived with frontswap backends doing
writeback and cleancache backends shrinking.

So just some thoughts to start some specific discussion.  Any thoughts?

Thanks,
Seth

> 
> Thanks,
> Dan
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/