linux-kernel - Re: [PATCH v3 4/4] squashfs: Add zstd support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 31 Jul 2017 21:30:08 +0000
From:   Nick Terrell <terrelln@...com>
To:     Phillip Lougher <phillip.lougher@...il.com>
CC:     "Austin S . Hemmelgarn" <ahferroin7@...il.com>,
        Herbert Xu <herbert@...dor.apana.org.au>,
        Kernel Team <Kernel-team@...com>, Chris Mason <clm@...com>,
        Yann Collet <cyan@...com>, Adam Borowski <kilobyte@...band.pl>,
        David Sterba <dsterba@...e.cz>,
        SquashFS developers <squashfs-devel@...ts.sourceforge.net>,
        "linux-btrfs@...r.kernel.org" <linux-btrfs@...r.kernel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        "Sean Purcell" <me@...np.xyz>
Subject: Re: [PATCH v3 4/4] squashfs: Add zstd support

On 7/30/17, 6:50 PM, "Phillip Lougher" <phillip.lougher@...il.com> wrote:
> On Thu, Jul 20, 2017 at 10:27 PM, Nick Terrell <terrelln@...com> wrote:
>> Add zstd compression and decompression support to SquashFS. zstd is a
>> great fit for SquashFS because it can compress at ratios approaching xz,
>> while decompressing twice as fast as zlib. For SquashFS in particular,
>> it can decompress as fast as lzo and lz4. It also has the flexibility
>> to turn down the compression ratio for faster compression times.
>
> Hi Nick,
>
> This patch (and none of the previous versions) is showing up on
> squashfs-devel@...ts.sourceforge.net.  I also think you should have
> emailed me directly as a courtesy, as I'm the Squashfs author and
> maintainer.

Sorry about that Phillip, it was an oversight on my part. I've added you to
the CC list going forward, and have subscribed to the mailing list.

>> | Method>> | Ratio | Compression MB/s | Decompression MB/s |
>> |----------------|-------|------------------|--------------------|
>> | gzip>>   |  2.92 |>>>   15 |>>>>128 |
>> | lzo>>>|  2.64 |>>>  9.5 |>>>>217 |
>> | lz4>>>|  2.12 |>>>   94 |>>>>218 |
>> | xz>>> |  3.43 |>>>  5.5 |>>>> 35 |
>> | xz 256 KB>  |  3.53 |>>>  5.4 |>>>> 40 |
>> | zstd 1>> |  2.71 |>>>   96 |>>>>210 |
>> | zstd 5>> |  2.93 |>>>   69 |>>>>198 |
>> | zstd 10>>|  3.01 |>>>   41 |>>>>225 |
>> | zstd 15>>|  3.13 |>>> 11.4 |>>>>224 |
>> | zstd 16 256 KB |  3.24 |>>>  8.1 |>>>>210 |
>
>
> Those numbers look good to me.
>
>>
>> This patch was written by Sean Purcell <me@...np.xyz>, but I will be
>> taking over the submission process.
>>
>> [1] http://releases.ubuntu.com/16.10/
>> [2] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/squashfs-benchmark.sh
>
> I can't find your patch that adds zstd to the user-land
> Squashfs-tools.  That would be handy to do any testing :-)
>
> Phillip

I'll include the squashfs-tools patch that Sean just posted in the next
version.