linux-kernel - Re: lz4hc compression in UBIFS?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 24 Oct 2013 19:15:23 +0400
From:	Konstantin Tokarev <annulen@...dex.ru>
To:	Yann Collet <yann.collet.73@...il.com>,
	"linux-mtd@...ts.infradead.org" <linux-mtd@...ts.infradead.org>,
	Brent Taylor <motobud@...il.com>,
	Artem Bityutskiy <dedekind1@...il.com>,
	linux-kernel@...r.kernel.org, akpm@...ux-foundation.org
Subject: Re: lz4hc compression in UBIFS?



24.10.2013, 18:20, "Konstantin Tokarev" <annulen@...dex.ru>:
> 23.10.2013, 22:26, "Yann Collet" <yann.collet.73@...il.com>:
>
>>  UBIFS error (pid 4288): ubifs_decompress: cannot decompress 12 bytes,
>>  (...)
>>          data size      12
>>          data:
>>          00000000: 1f 00 01 00 ff e8 50 00 00 00 00 00
>>
>>  The compressed format is correct.
>>  It describes a flow of 00, of size ~500.
>>
>>  So the problem seems more likely on the decompression side.
>>
>>  Are you sure you are providing "12" as the "input size" parameter ? and that
>>  the "maximum output size" parameter is correctly provided ? (i.e. >= to
>>  original data size)
>
> Decompression code in kernel[1] is heavily modified. In particular, lz4_uncompress
> function (used in this case) does not have input size parameter at all,
> while it's present in lz4_uncompress_unknownoutputsize.
>
> [1] lib/lz4/lz4_decompress.c

Attached patch to crypto API wrapper for lz4hc seems to fix the issue. I can save
and read files on LZ4HC-compressed volume, and I'm running on LZ4HC-compressed rootfs,
flashed from mkfs.ubifs generated image (patch by Elie De Brauwer). My software
works correctly.

Unfortunately, on my board LZ4HC-compressed UBIFS volume performs noticeable worse
than LZO, while still faster than zlib. I guess the reason is that CPU is no longer a
bottleneck for my system, but NAND speed.

Thank you all for your help!

-- 
Regards,
Konstantin
View attachment "crypto_lz4.patch" of type "text/x-diff" (852 bytes)