linux-kernel - Re: [PATCH v3 0/4] Add xxhash and zstd modules

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <cb9deeb3-dc04-45a4-9583-1da9e8a3f779@gmail.com>
Date:   Fri, 21 Jul 2017 11:56:21 -0400
From:   "Austin S. Hemmelgarn" <ahferroin7@...il.com>
To:     Nick Terrell <terrelln@...com>
Cc:     Herbert Xu <herbert@...dor.apana.org.au>, kernel-team@...com,
        Chris Mason <clm@...com>, Yann Collet <cyan@...com>,
        Adam Borowski <kilobyte@...band.pl>,
        David Sterba <dsterba@...e.cz>,
        squashfs-devel@...ts.sourceforge.net, linux-btrfs@...r.kernel.org,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH v3 0/4] Add xxhash and zstd modules

On 2017-07-20 17:27, Nick Terrell wrote:
> Hi all,
> 
> This patch set adds xxhash, zstd compression, and zstd decompression
> modules. It also adds zstd support to BtrFS and SquashFS.
> 
> Each patch has relevant summaries, benchmarks, and tests.
> 
> Best,
> Nick Terrell

For patches 2-3, I've compile tested and had runtime testing running for 
about 18 hours now with no issues, so you can add:

Tested-by: Austin S. Hemmelgarn <ahferroin7@...il.com>

For patch 1, I've only compile tested it, but had no issues and got no 
warnings about it when booting to test 2-4.

For patch 4, I've compile tested it and done some really basic runtime 
testing using a couple of hand-crafted SquashFS images.  It appears to 
run fine, but I've not done enough that I feel it warrants a Tested-by 
tag from me.
> 
> Changelog:
> 
> v1 -> v2:
> - Make pointer in lib/xxhash.c:394 non-const (1/4)
> - Use div_u64() for division of u64s (2/4)
> - Reduce stack usage of ZSTD_compressSequences(), ZSTD_buildSeqTable(),
>    ZSTD_decompressSequencesLong(), FSE_buildDTable(), FSE_decompress_wksp(),
>    HUF_writeCTable(), HUF_readStats(), HUF_readCTable(),
>    HUF_compressWeights(), HUF_readDTableX2(), and HUF_readDTableX4() (2/4)
> - No zstd function uses more than 400 B of stack space (2/4)
> 
> v2 -> v3:
> - Work around gcc-7 bug https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81388
>    (2/4)
> - Fix bug in dictionary compression from upstream commit cc1522351f (2/4)
> - Port upstream BtrFS commits e1ddce71d6, 389a6cfc2a, and 6acafd1eff (3/4)
> - Change default compression level for BtrFS to 3 (3/4)
> 
> Nick Terrell (4):
>    lib: Add xxhash module
>    lib: Add zstd modules
>    btrfs: Add zstd support
>    squashfs: Add zstd support
> 
>   fs/btrfs/Kconfig           |    2 +
>   fs/btrfs/Makefile          |    2 +-
>   fs/btrfs/compression.c     |    1 +
>   fs/btrfs/compression.h     |    6 +-
>   fs/btrfs/ctree.h           |    1 +
>   fs/btrfs/disk-io.c         |    2 +
>   fs/btrfs/ioctl.c           |    6 +-
>   fs/btrfs/props.c           |    6 +
>   fs/btrfs/super.c           |   12 +-
>   fs/btrfs/sysfs.c           |    2 +
>   fs/btrfs/zstd.c            |  435 ++++++
>   fs/squashfs/Kconfig        |   14 +
>   fs/squashfs/Makefile       |    1 +
>   fs/squashfs/decompressor.c |    7 +
>   fs/squashfs/decompressor.h |    4 +
>   fs/squashfs/squashfs_fs.h  |    1 +
>   fs/squashfs/zstd_wrapper.c |  150 ++
>   include/linux/xxhash.h     |  236 +++
>   include/linux/zstd.h       | 1157 +++++++++++++++
>   include/uapi/linux/btrfs.h |    8 +-
>   lib/Kconfig                |   11 +
>   lib/Makefile               |    3 +
>   lib/xxhash.c               |  500 +++++++
>   lib/zstd/Makefile          |   18 +
>   lib/zstd/bitstream.h       |  374 +++++
>   lib/zstd/compress.c        | 3479 ++++++++++++++++++++++++++++++++++++++++++++
>   lib/zstd/decompress.c      | 2526 ++++++++++++++++++++++++++++++++
>   lib/zstd/entropy_common.c  |  243 ++++
>   lib/zstd/error_private.h   |   53 +
>   lib/zstd/fse.h             |  575 ++++++++
>   lib/zstd/fse_compress.c    |  795 ++++++++++
>   lib/zstd/fse_decompress.c  |  332 +++++
>   lib/zstd/huf.h             |  212 +++
>   lib/zstd/huf_compress.c    |  770 ++++++++++
>   lib/zstd/huf_decompress.c  |  960 ++++++++++++
>   lib/zstd/mem.h             |  151 ++
>   lib/zstd/zstd_common.c     |   75 +
>   lib/zstd/zstd_internal.h   |  250 ++++
>   lib/zstd/zstd_opt.h        | 1014 +++++++++++++
>   39 files changed, 14382 insertions(+), 12 deletions(-)
>   create mode 100644 fs/btrfs/zstd.c
>   create mode 100644 fs/squashfs/zstd_wrapper.c
>   create mode 100644 include/linux/xxhash.h
>   create mode 100644 include/linux/zstd.h
>   create mode 100644 lib/xxhash.c
>   create mode 100644 lib/zstd/Makefile
>   create mode 100644 lib/zstd/bitstream.h
>   create mode 100644 lib/zstd/compress.c
>   create mode 100644 lib/zstd/decompress.c
>   create mode 100644 lib/zstd/entropy_common.c
>   create mode 100644 lib/zstd/error_private.h
>   create mode 100644 lib/zstd/fse.h
>   create mode 100644 lib/zstd/fse_compress.c
>   create mode 100644 lib/zstd/fse_decompress.c
>   create mode 100644 lib/zstd/huf.h
>   create mode 100644 lib/zstd/huf_compress.c
>   create mode 100644 lib/zstd/huf_decompress.c
>   create mode 100644 lib/zstd/mem.h
>   create mode 100644 lib/zstd/zstd_common.c
>   create mode 100644 lib/zstd/zstd_internal.h
>   create mode 100644 lib/zstd/zstd_opt.h
> 
> --
> 2.9.3
>