linux-kernel - [PATCH 0/3] Update LZ4 compressor module

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 20 Dec 2016 19:53:09 +0100
From:   Sven Schmidt <4sschmid@...ormatik.uni-hamburg.de>
To:     akpm@...ux-foundation.org
Cc:     bongkyu.kim@....com, sergey.senozhatsky@...il.com,
        gregkh@...uxfoundation.org, linux-kernel@...r.kernel.org
Subject: [PATCH 0/3] Update LZ4 compressor module

This patchset is for updating the LZ4 compression module to a version based
on LZ4 v1.7.2 allowing to use the fast compression algorithm aka LZ4 fast
which provides an "acceleration" parameter as a tradeoff between
compression ratio and compression speed.

We will use LZ4 fast in order to support compression in lustre. LZ4 fast empowers
us to do client-side as well as server-side compression/decompression while
being able to provide appropriate parameters to enable users to tune lustre's
behaviour to obtain the best performance/compression/etc. on their behalf
(adapative compression).

Also, it will be useful for other users of LZ4 compression,
as with LZ4 fast it is possible to enable applications to use fast and/or high
compression depending of the usecase. E.g. a developer can use very
high compression (low acceleration) for sending data over a network with
limited rate of transmission or he trades the compression ratio for higher
compression speed.

LZ4 homepage: http://www.lz4.org/
LZ4 source repository: https://github.com/lz4/lz4
Source version: 1.7.2
Compression in lustre: http://wiki.lustre.org/Enhanced_Adaptive_Compression_in_Lustre

Benchmark (taken from [1], Core i5-4300U @1.9GHz):
----------------|--------------|----------------|----------
Compressor      | Compression  | Decompression  | Ratio
----------------|--------------|----------------|----------
memcpy          |  4200 MB/s   |  4200 MB/s     | 1.000
LZ4 fast 50     |  1080 MB/s   |  2650 MB/s     | 1.375
LZ4 fast 17     |   680 MB/s   |  2220 MB/s     | 1.607
LZ4 fast 5      |   475 MB/s   |  1920 MB/s     | 1.886
LZ4 default     |   385 MB/s   |  1850 MB/s     | 2.101

[1] http://fastcompression.blogspot.de/2015/04/sampling-or-faster-lz4.html

[PATCH 1/3] crypto: Change lz4 modules to work with new lz4
[PATCH 2/3] fs/pstore: fs/squashfs: Change lz4 compressor functions
[PATCH 3/3] lib: Update LZ4 compressor module based on LZ4 v1.7.2.