linux-kernel - Re: [PATCH] erofs: get rid of ->lru usage

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <YXX2h26Fm/hztdaZ@B-P7TQMD6M-0146.local>
Date:   Mon, 25 Oct 2021 08:12:55 +0800
From:   Gao Xiang <hsiangkao@...ux.alibaba.com>
To:     Kent Overstreet <kent.overstreet@...il.com>
Cc:     linux-erofs@...ts.ozlabs.org, LKML <linux-kernel@...r.kernel.org>,
        Matthew Wilcox <willy@...radead.org>
Subject: Re: [PATCH] erofs: get rid of ->lru usage

On Sun, Oct 24, 2021 at 12:59:22PM -0400, Kent Overstreet wrote:
> On Fri, Oct 22, 2021 at 05:01:20PM +0800, Gao Xiang wrote:
> > Currently, ->lru is a way to arrange non-LRU pages and has some
> > in-kernel users. In order to minimize noticable issues of page
> > reclaim and cache thrashing under high memory presure, limited
> > temporary pages were all chained with ->lru and can be reused
> > during the request. However, it seems that ->lru could be removed
> > when folio is landing.
> > 
> > Let's use page->private to chain temporary pages for now instead
> > and transform EROFS formally after the topic of the folio / file
> > page design is finalized.
> > 
> > Cc: Matthew Wilcox <willy@...radead.org>
> > Cc: Kent Overstreet <kent.overstreet@...il.com>
> > Cc: Chao Yu <chao@...nel.org>
> > Signed-off-by: Gao Xiang <hsiangkao@...ux.alibaba.com>
> 
> Would it not be an option to use an array of pointers to pages, instead of a
> linked list? Arrays are faster than lists, and page->private is another thing we
> prefer not to use if we don't have to.

Our requirement is to maintain variable-sized temporary pages, that
may be short or maybe long during decompression process (according to
different algorithms and whether it's inplace I/O or not) rather than
before I/O submission:

 For LZ4, that maybe 16 pages for lz4 sliding window and other
temporary pages;
 For LZMA, since it has a internal dictionary (sliding window) so we
don't have to allocate any temporary pages for cached decompression
(outplace I/O), but we still need allocating temporary pages to
handle overlapped decompression due to inplace I/O (think about it,
if each compressed pcluster is 4pages -- 16KiB, we have to allocate
for example another 128KiB compressed pages due to read request for
outplace I/O, but instead 0 pages for inplace I/O during I/O process,
and only 4 pages for decompression process. It saves much memory in
the low memory scenarios by using inplace I/O and important to the
performance).

Currently we only add some page to the head or pick some page from
head, so a singly-linked list is also fine to us for now.

> 
> That said - this is definitely preferable to using page->lru - thank you.
> 
> Reviewed-by: Kent Overstreet <kent.overstreet@...il.com>

Thank you!

Thanks,
Gao Xiang