linux-kernel - Re: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20210512194935.GY7604@twin.jikos.cz>
Date:   Wed, 12 May 2021 21:49:35 +0200
From:   David Sterba <dsterba@...e.cz>
To:     Nick Terrell <terrelln@...com>
Cc:     Nick Terrell <nickrterrell@...il.com>,
        Herbert Xu <herbert@...dor.apana.org.au>,
        "linux-crypto@...r.kernel.org" <linux-crypto@...r.kernel.org>,
        Btrfs BTRFS <linux-btrfs@...r.kernel.org>,
        "squashfs-devel@...ts.sourceforge.net" 
        <squashfs-devel@...ts.sourceforge.net>,
        "linux-f2fs-devel@...ts.sourceforge.net" 
        <linux-f2fs-devel@...ts.sourceforge.net>,
        LKML <linux-kernel@...r.kernel.org>,
        Kernel Team <Kernel-team@...com>, Chris Mason <clm@...com>,
        Petr Malat <oss@...at.biz>, Johannes Weiner <jweiner@...com>,
        Niket Agarwal <niketa@...com>, Yann Collet <cyan@...com>,
        Christoph Hellwig <hch@...radead.org>,
        Michał Mirosław <mirq-linux@...e.qmqm.pl>,
        David Sterba <dsterba@...e.cz>,
        Oleksandr Natalenko <oleksandr@...alenko.name>,
        Felix Handte <felixh@...com>,
        Eric Biggers <ebiggers@...nel.org>,
        Randy Dunlap <rdunlap@...radead.org>,
        torvalds@...ux-foundation.org
Subject: Re: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

On Tue, May 11, 2021 at 08:53:41PM +0000, Nick Terrell wrote:
> Pinging this series. Is there anything I should do to help get this
> merged?
> 
> The use of zstd in the kernel is continuously increasing over time,
> both in terms of number of use cases, and number of users that
> actually enable zstd compression in production. E.g. Fedora is
> making btrfs with zstd compression enabled the default.
> 
> I would love to see the zstd code updated to the latest upstream
> and be kept up to date. The latest upstream brings bug fixes, and
> significant performance improvements. Additionally, the latest
> upstream code is continuously fuzzed.

The btrfs community and I in particular have interest to get zstd
updated but also there's the patch 3 that goes against what kernel
requires regarding patch size and logical split of changes.

That the update is so large shouldn't have happened, it covers 3 years
of development, the syncs should have happened more often, but here we
are.

Other points that have been raised in the past:

* new wrappers - there are new wrappers changing users of the API, the
  new names are more conforming, eg ZSTD_decompressDCtx -> zstd_decompress_dctx,
  sounds like an improvement to me

* high stack usage - mentioned in patch 3, slight increase but bounded
  and upstream now monitors that so it does not increase

Other points that are worth mentioning:

* bisectability - the version switch happens in one patch, so the
  effects before/after the patch are only runtime as there's no change
  in format etc, so ok

* will be maintained - no such huge update should happen again

So I suggest to merge in current form. I'm not sure what was the
original plan if it was supposed to go via Herbert's crypto tree, but
that was before Nick added himself as maintainer.

I think that Nick can send the pull request to Linus, perhaps with acks
to all changes that are in the non-zstd code (patch 1).

Cover letter v11: https://lore.kernel.org/linux-btrfs/20210430013157.747152-1-nickrterrell@gmail.com/