linux-kernel - Re: [PATCH 2/4] erofs: support adjust lz4 history window size

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20210327101548.GB2995728@xiangao.remote.csb>
Date:   Sat, 27 Mar 2021 18:15:48 +0800
From:   Gao Xiang <hsiangkao@...hat.com>
To:     Chao Yu <yuchao0@...wei.com>
Cc:     Gao Xiang <hsiangkao@....com>, linux-erofs@...ts.ozlabs.org,
        Chao Yu <chao@...nel.org>, LKML <linux-kernel@...r.kernel.org>,
        Huang Jianan <huangjianan@...o.com>,
        Guo Weichao <guoweichao@...o.com>
Subject: Re: [PATCH 2/4] erofs: support adjust lz4 history window size

Hi Chao,

On Sat, Mar 27, 2021 at 05:34:33PM +0800, Chao Yu wrote:
> On 2021/3/27 11:49, Gao Xiang wrote:
> > From: Huang Jianan <huangjianan@...o.com>
> > 
> > lz4 uses LZ4_DISTANCE_MAX to record history preservation. When
> > using rolling decompression, a block with a higher compression
> > ratio will cause a larger memory allocation (up to 64k). It may
> > cause a large resource burden in extreme cases on devices with
> > small memory and a large number of concurrent IOs. So appropriately
> > reducing this value can improve performance.
> > 
> > Decreasing this value will reduce the compression ratio (except
> > when input_size <LZ4_DISTANCE_MAX). But considering that erofs
> > currently only supports 4k output, reducing this value will not
> > significantly reduce the compression benefits.
> > 
> > The maximum value of LZ4_DISTANCE_MAX defined by lz4 is 64k, and
> > we can only reduce this value. For the old kernel, it just can't
> > reduce the memory allocation during rolling decompression without
> > affecting the decompression result.
> > 
> > Signed-off-by: Huang Jianan <huangjianan@...o.com>
> > Signed-off-by: Guo Weichao <guoweichao@...o.com>
> > [ Gao Xiang: introduce struct erofs_sb_lz4_info for configurations. ]
> > Signed-off-by: Gao Xiang <hsiangkao@...hat.com>
> > ---
> >   fs/erofs/decompressor.c | 21 +++++++++++++++++----
> >   fs/erofs/erofs_fs.h     |  3 ++-
> >   fs/erofs/internal.h     | 19 +++++++++++++++++++
> >   fs/erofs/super.c        |  4 +++-
> >   4 files changed, 41 insertions(+), 6 deletions(-)
> > 
> > diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c
> > index 80e8871aef71..93411e9df9b6 100644
> > --- a/fs/erofs/decompressor.c
> > +++ b/fs/erofs/decompressor.c
> > @@ -28,6 +28,17 @@ struct z_erofs_decompressor {
> >   	char *name;
> >   };
> > +int z_erofs_load_lz4_config(struct super_block *sb,
> > +			    struct erofs_super_block *dsb)
> > +{
> > +	u16 distance = le16_to_cpu(dsb->lz4_max_distance);
> > +
> > +	EROFS_SB(sb)->lz4.max_distance_pages = distance ?
> > +					DIV_ROUND_UP(distance, PAGE_SIZE) + 1 :
> > +					LZ4_MAX_DISTANCE_PAGES;
> > +	return 0;
> > +}
> > +
> >   static int z_erofs_lz4_prepare_destpages(struct z_erofs_decompress_req *rq,
> >   					 struct list_head *pagepool)
> >   {
> > @@ -36,6 +47,8 @@ static int z_erofs_lz4_prepare_destpages(struct z_erofs_decompress_req *rq,
> >   	struct page *availables[LZ4_MAX_DISTANCE_PAGES] = { NULL };
> >   	unsigned long bounced[DIV_ROUND_UP(LZ4_MAX_DISTANCE_PAGES,
> >   					   BITS_PER_LONG)] = { 0 };
> > +	unsigned int lz4_max_distance_pages =
> > +				EROFS_SB(rq->sb)->lz4.max_distance_pages;
> >   	void *kaddr = NULL;
> >   	unsigned int i, j, top;
> > @@ -44,14 +57,14 @@ static int z_erofs_lz4_prepare_destpages(struct z_erofs_decompress_req *rq,
> >   		struct page *const page = rq->out[i];
> >   		struct page *victim;
> > -		if (j >= LZ4_MAX_DISTANCE_PAGES)
> > +		if (j >= lz4_max_distance_pages)
> >   			j = 0;
> >   		/* 'valid' bounced can only be tested after a complete round */
> >   		if (test_bit(j, bounced)) {
> > -			DBG_BUGON(i < LZ4_MAX_DISTANCE_PAGES);
> > -			DBG_BUGON(top >= LZ4_MAX_DISTANCE_PAGES);
> > -			availables[top++] = rq->out[i - LZ4_MAX_DISTANCE_PAGES];
> > +			DBG_BUGON(i < lz4_max_distance_pages);
> > +			DBG_BUGON(top >= lz4_max_distance_pages);
> > +			availables[top++] = rq->out[i - lz4_max_distance_pages];
> >   		}
> >   		if (page) {
> > diff --git a/fs/erofs/erofs_fs.h b/fs/erofs/erofs_fs.h
> > index 9ad1615f4474..b27d0e4e4ab5 100644
> > --- a/fs/erofs/erofs_fs.h
> > +++ b/fs/erofs/erofs_fs.h
> > @@ -39,7 +39,8 @@ struct erofs_super_block {
> >   	__u8 uuid[16];          /* 128-bit uuid for volume */
> >   	__u8 volume_name[16];   /* volume name */
> >   	__le32 feature_incompat;
> > -	__u8 reserved2[44];
> > +	__le16 lz4_max_distance;
> 
> It missed to add comments, otherwise it looks good to me.
>

Yes, it needs some inline comment description, will update in
the next version.

> Reviewed-by: Chao Yu <yuchao0@...wei.com>
> 

Thanks for the review!

Thanks,
Gao Xiang

> Thanks,
>