linux-kernel - Re: [PATCH v3 0/4] Add xxhash and zstd modules

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <dd429ccc-cafd-3042-a35e-4cc6c4784f6a@gmail.com>
Date:   Fri, 21 Jul 2017 07:18:30 -0400
From:   "Austin S. Hemmelgarn" <ahferroin7@...il.com>
To:     Nick Terrell <terrelln@...com>
Cc:     Herbert Xu <herbert@...dor.apana.org.au>, kernel-team@...com,
        Chris Mason <clm@...com>, Yann Collet <cyan@...com>,
        Adam Borowski <kilobyte@...band.pl>,
        David Sterba <dsterba@...e.cz>,
        squashfs-devel@...ts.sourceforge.net, linux-btrfs@...r.kernel.org,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH v3 0/4] Add xxhash and zstd modules

On 2017-07-21 07:16, Austin S. Hemmelgarn wrote:
> On 2017-07-20 17:27, Nick Terrell wrote:
Well this is embarrassing, forgot to type anything before hitting send...
>> Hi all,
>>
>> This patch set adds xxhash, zstd compression, and zstd decompression
>> modules. It also adds zstd support to BtrFS and SquashFS.
>>
>> Each patch has relevant summaries, benchmarks, and tests.
>>
>> Best,
>> Nick Terrell
I've compile tested all of this on my end now, and have gotten some 
preliminary testing of patches 2-4, and everything looks good so far 
here.  I'll reply to this if I end up finding issues, but it doesn't 
look like I will right now considering the testing has run over night 
without issue.
>>
>> Changelog:
>>
>> v1 -> v2:
>> - Make pointer in lib/xxhash.c:394 non-const (1/4)
>> - Use div_u64() for division of u64s (2/4)
>> - Reduce stack usage of ZSTD_compressSequences(), ZSTD_buildSeqTable(),
>>    ZSTD_decompressSequencesLong(), FSE_buildDTable(), 
>> FSE_decompress_wksp(),
>>    HUF_writeCTable(), HUF_readStats(), HUF_readCTable(),
>>    HUF_compressWeights(), HUF_readDTableX2(), and HUF_readDTableX4() 
>> (2/4)
>> - No zstd function uses more than 400 B of stack space (2/4)
>>
>> v2 -> v3:
>> - Work around gcc-7 bug 
>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81388
>>    (2/4)
>> - Fix bug in dictionary compression from upstream commit cc1522351f (2/4)
>> - Port upstream BtrFS commits e1ddce71d6, 389a6cfc2a, and 6acafd1eff 
>> (3/4)
>> - Change default compression level for BtrFS to 3 (3/4)
>>
>> Nick Terrell (4):
>>    lib: Add xxhash module
>>    lib: Add zstd modules
>>    btrfs: Add zstd support
>>    squashfs: Add zstd support
>>
>>   fs/btrfs/Kconfig           |    2 +
>>   fs/btrfs/Makefile          |    2 +-
>>   fs/btrfs/compression.c     |    1 +
>>   fs/btrfs/compression.h     |    6 +-
>>   fs/btrfs/ctree.h           |    1 +
>>   fs/btrfs/disk-io.c         |    2 +
>>   fs/btrfs/ioctl.c           |    6 +-
>>   fs/btrfs/props.c           |    6 +
>>   fs/btrfs/super.c           |   12 +-
>>   fs/btrfs/sysfs.c           |    2 +
>>   fs/btrfs/zstd.c            |  435 ++++++
>>   fs/squashfs/Kconfig        |   14 +
>>   fs/squashfs/Makefile       |    1 +
>>   fs/squashfs/decompressor.c |    7 +
>>   fs/squashfs/decompressor.h |    4 +
>>   fs/squashfs/squashfs_fs.h  |    1 +
>>   fs/squashfs/zstd_wrapper.c |  150 ++
>>   include/linux/xxhash.h     |  236 +++
>>   include/linux/zstd.h       | 1157 +++++++++++++++
>>   include/uapi/linux/btrfs.h |    8 +-
>>   lib/Kconfig                |   11 +
>>   lib/Makefile               |    3 +
>>   lib/xxhash.c               |  500 +++++++
>>   lib/zstd/Makefile          |   18 +
>>   lib/zstd/bitstream.h       |  374 +++++
>>   lib/zstd/compress.c        | 3479 
>> ++++++++++++++++++++++++++++++++++++++++++++
>>   lib/zstd/decompress.c      | 2526 ++++++++++++++++++++++++++++++++
>>   lib/zstd/entropy_common.c  |  243 ++++
>>   lib/zstd/error_private.h   |   53 +
>>   lib/zstd/fse.h             |  575 ++++++++
>>   lib/zstd/fse_compress.c    |  795 ++++++++++
>>   lib/zstd/fse_decompress.c  |  332 +++++
>>   lib/zstd/huf.h             |  212 +++
>>   lib/zstd/huf_compress.c    |  770 ++++++++++
>>   lib/zstd/huf_decompress.c  |  960 ++++++++++++
>>   lib/zstd/mem.h             |  151 ++
>>   lib/zstd/zstd_common.c     |   75 +
>>   lib/zstd/zstd_internal.h   |  250 ++++
>>   lib/zstd/zstd_opt.h        | 1014 +++++++++++++
>>   39 files changed, 14382 insertions(+), 12 deletions(-)
>>   create mode 100644 fs/btrfs/zstd.c
>>   create mode 100644 fs/squashfs/zstd_wrapper.c
>>   create mode 100644 include/linux/xxhash.h
>>   create mode 100644 include/linux/zstd.h
>>   create mode 100644 lib/xxhash.c
>>   create mode 100644 lib/zstd/Makefile
>>   create mode 100644 lib/zstd/bitstream.h
>>   create mode 100644 lib/zstd/compress.c
>>   create mode 100644 lib/zstd/decompress.c
>>   create mode 100644 lib/zstd/entropy_common.c
>>   create mode 100644 lib/zstd/error_private.h
>>   create mode 100644 lib/zstd/fse.h
>>   create mode 100644 lib/zstd/fse_compress.c
>>   create mode 100644 lib/zstd/fse_decompress.c
>>   create mode 100644 lib/zstd/huf.h
>>   create mode 100644 lib/zstd/huf_compress.c
>>   create mode 100644 lib/zstd/huf_decompress.c
>>   create mode 100644 lib/zstd/mem.h
>>   create mode 100644 lib/zstd/zstd_common.c
>>   create mode 100644 lib/zstd/zstd_internal.h
>>   create mode 100644 lib/zstd/zstd_opt.h
>>
>> -- 
>> 2.9.3
>>
>