linux-kernel - Re: [RFC][PATCH v2 2/3] zram: use zs_get_huge_class_size

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160222035448.GB11961@swordfish>
Date:	Mon, 22 Feb 2016 12:54:48 +0900
From:	Sergey Senozhatsky <sergey.senozhatsky.work@...il.com>
To:	Minchan Kim <minchan@...nel.org>
Cc:	Sergey Senozhatsky <sergey.senozhatsky.work@...il.com>,
	Sergey Senozhatsky <sergey.senozhatsky@...il.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Joonsoo Kim <js1304@...il.com>, linux-mm@...ck.org,
	linux-kernel@...r.kernel.org
Subject: Re: [RFC][PATCH v2 2/3] zram: use zs_get_huge_class_size_watermark()

On (02/22/16 11:57), Minchan Kim wrote:
[..]
> > > Yes, I mean if we have backing storage, we could mitigate the problem
> > > like the mentioned approach. Otherwise, we should solve it in allocator
> > > itself and you suggested the idea and I commented first step.
> > > What's the problem, now?
> > 
> > well, I didn't say I have problems.
> > so you want a backing device that will keep only 'bad compression'
> > objects and use zsmalloc to keep there only 'good compression' objects?
> > IOW, no huge classes in zsmalloc at all? well, that can work out. it's
> > a bit strange though that to solve zram-zsmalloc issues we would ask
> > someone to create a additional device. it looks (at least for now) that
> > we can address those issues in zram-zsmalloc entirely; w/o user
> > intervention or a 3rd party device.
> 
> Agree. That's what I want. zram shouldn't be aware of allocator's
> internal implementation. IOW, zsmalloc should handle it without
> exposing any internal limitation.

well, at the same time zram must not dictate what to do. zram simply spoils
zsmalloc; it does not offer guaranteed good compression, and it does not let
zsmalloc to do it's job. zram has only excuses to be the way it is.
the existing zram->zsmalloc dependency looks worse than zsmalloc->zram to me.

> Backing device issue is orthogonal but what I said about thing
> was it could solve the issue too without exposing zsmalloc's
> limitation to the zram.

well, backing device would not reduce the amount of pages we request.
and that's the priority issue, especially if we are talking about
embedded system with a low free pages capability. we would just move huge
objects from zsmalloc to backing device. other than that we would still
request 1000 (for example) pages to store 1000 objects. it's zsmalloc's
"page sharing" that permits us to request less than 1000 pages to store
1000 objects.

so yes, I agree, increasing ZS_MAX_ZSPAGE_ORDER and do more tests is
the step #1 to take.

> Let's summary my points in here.
> 
> Let's make zsmalloc smarter to reduce wasted space. One of option is
> dynamic page creation which I agreed.
>
> Before the feature, we should test how memory footprint is bigger
> without the feature if we increase ZS_MAX_ZSPAGE_ORDER.
> If it's not big, we could go with your patch easily without adding
> more complex stuff(i.e, dynamic page creation).

yes, agree. alloc_zspage()/init_zspage() and friends must be the last
thing to touch. only if increased ZS_MAX_ZSPAGE_ORDER will turn out not
to be good enough.

> Please, check max_used_pages rather than mem_used_total for seeing
> memory footprint at the some moment and test very fragmented scenario
> (creating files and free part of files) rather than just full coping.

sure, more tests will follow.

	-ss