linux-kernel - Re: [PATCH 0/3] Update LZ4 compressor module

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1482431714-1536-1-git-send-email-4sschmid@informatik.uni-hamburg.de>
Date:   Thu, 22 Dec 2016 19:35:14 +0100
From:   Sven Schmidt <4sschmid@...ormatik.uni-hamburg.de>
To:     gregkh@...uxfoundation.org
Cc:     akpm@...ux-foundation.org, bongkyu.kim@....com,
        sergey.senozhatsky@...il.com, linux-kernel@...r.kernel.org,
        Sven Schmidt <4sschmid@...ormatik.uni-hamburg.de>
Subject: Re: [PATCH 0/3] Update LZ4 compressor module

On 12/22/2016 06:29 PM, Greg KH wrote:
> On Tue, Dec 20, 2016 at 07:53:09PM +0100, Sven Schmidt wrote:
>>
>> This patchset is for updating the LZ4 compression module to a version based
>> on LZ4 v1.7.2 allowing to use the fast compression algorithm aka LZ4 fast
>> which provides an "acceleration" parameter as a tradeoff between
>> compression ratio and compression speed.
>
> But why do this?
>
>> We will use LZ4 fast in order to support compression in lustre. LZ4 fast empowers
>> us to do client-side as well as server-side compression/decompression while
>> being able to provide appropriate parameters to enable users to tune lustre's
>> behaviour to obtain the best performance/compression/etc. on their behalf
>> (adapative compression).
>
> We don't care about lustre, especially as it is not merged into the main
> portion of the kernel tree :)
>
> Seriously, work on fixing up the known issues in lustre before adding
> additional features, I've only been saying this for a few _years_ now...
>

Hey Greg and thanks for your time. Actually, we're not the guys behind lustre,
hence I'd leave fixing the bugs in lustre to them ;)

I'm working with the research group for scientific computing on the University
of Hamburg on the German Climate Computing Centre. The research group is working
with high performance storage systems etc. In this case, we investigate
data reduction techniques in behalf of the increasing gap between computational speed,
network speed and storage capacity. That's why we're ultimately aiming
for compression support in lustre.

Initial studies have shown that an adequate compression ratio for scientific data
can be archieved using LZ4. Since the currently available version of LZ4
in the kernel is about three years old, we would love if you accept 
our work on getting a more current version into the kernel.

>> Also, it will be useful for other users of LZ4 compression,
>> as with LZ4 fast it is possible to enable applications to use fast and/or high
>> compression depending of the usecase. E.g. a developer can use very
>> high compression (low acceleration) for sending data over a network with
>> limited rate of transmission or he trades the compression ratio for higher
>> compression speed.
>
> This whole patch series is broken, always test-build your code, there's
> nothing we could do with these patches even if we wanted to :(
>
> greg k-h
>

I'm sorry, this is my first patchset, so please be kind :(
Already got rid of the issues on my local machine and reviewed the output from
the buildbots. Will send an updated patchset later!

Thanks
Sven