linux-kernel - Re: [RFC 0/4] ZRAM: make it just store the high compression rate page

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160905021852.GB22701@bbox>
Date:   Mon, 5 Sep 2016 11:18:52 +0900
From:   Minchan Kim <minchan@...nel.org>
To:     Hui Zhu <teawater@...il.com>
Cc:     Sergey Senozhatsky <sergey.senozhatsky.work@...il.com>,
        Hui Zhu <zhuhui@...omi.com>, ngupta@...are.org,
        Hugh Dickins <hughd@...gle.com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Ingo Molnar <mingo@...hat.com>,
        Peter Zijlstra <peterz@...radead.org>, acme@...nel.org,
        alexander.shishkin@...ux.intel.com,
        Andrew Morton <akpm@...ux-foundation.org>, mhocko@...e.com,
        hannes@...xchg.org, mgorman@...hsingularity.net, vbabka@...e.cz,
        redkoi@...tuozzo.com, luto@...nel.org,
        kirill.shutemov@...ux.intel.com, geliangtang@....com,
        baiyaowei@...s.chinamobile.com, dan.j.williams@...el.com,
        vdavydov@...tuozzo.com, aarcange@...hat.com, dvlasenk@...hat.com,
        jmarchan@...hat.com, koct9i@...il.com, yang.shi@...aro.org,
        dave.hansen@...ux.intel.com, vkuznets@...hat.com,
        vitalywool@...il.com, ross.zwisler@...ux.intel.com,
        Thomas Gleixner <tglx@...utronix.de>,
        kwapulinski.piotr@...il.com, axboe@...com, mchristi@...hat.com,
        Joe Perches <joe@...ches.com>, namit@...are.com,
        Rik van Riel <riel@...hat.com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        Linux Memory Management List <linux-mm@...ck.org>
Subject: Re: [RFC 0/4] ZRAM: make it just store the high compression rate page

On Thu, Aug 25, 2016 at 04:25:30PM +0800, Hui Zhu wrote:
> On Thu, Aug 25, 2016 at 2:09 PM, Sergey Senozhatsky
> <sergey.senozhatsky.work@...il.com> wrote:
> > Hello,
> >
> > On (08/22/16 16:25), Hui Zhu wrote:
> >>
> >> Current ZRAM just can store all pages even if the compression rate
> >> of a page is really low.  So the compression rate of ZRAM is out of
> >> control when it is running.
> >> In my part, I did some test and record with ZRAM.  The compression rate
> >> is about 40%.
> >>
> >> This series of patches make ZRAM can just store the page that the
> >> compressed size is smaller than a value.
> >> With these patches, I set the value to 2048 and did the same test with
> >> before.  The compression rate is about 20%.  The times of lowmemorykiller
> >> also decreased.
> >
> > I haven't looked at the patches in details yet. can you educate me a bit?
> > is your test stable? why the number of lowmemorykill-s has decreased?
> > ... or am reading "The times of lowmemorykiller also decreased" wrong?
> >
> > suppose you have X pages that result in bad compression size (from zram
> > point of view). zram stores such pages uncompressed, IOW we have no memory
> > savings - swapped out page lands in zsmalloc PAGE_SIZE class. now you
> > don't try to store those pages in zsmalloc, but keep them as unevictable.
> > so the page still occupies PAGE_SIZE; no memory saving again. why did it
> > improve LMK?
> 
> No, zram will not save this page uncompressed with these patches.  It
> will set it as non-swap and kick back to shrink_page_list.
> Shrink_page_list will remove this page from swapcache and kick it to
> unevictable list.
> Then this page will not be swaped before it get write.
> That is why most of code are around vmscan.c.

If I understand Sergey's point right, he means there is no gain
to save memory between before and after.

With your approach, you can prevent unnecessary pageout(i.e.,
uncompressible page swap out) but it doesn't mean you save the
memory compared to old so why does your patch decrease the number of
lowmemory killing?

A thing I can imagine is without this feature, zram could be full of
uncompressible pages so good-compressible page cannot be swapped out.
Hui, is this scenario right for your case?

Thanks.