linux-kernel - Re: [PATCH 5/9] btrfs: zstd: Switch to the zstd-1.4.6 API

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200916182742.GB4373@sol.localdomain>
Date:   Wed, 16 Sep 2020 11:27:42 -0700
From:   Eric Biggers <ebiggers@...nel.org>
To:     Christoph Hellwig <hch@...radead.org>
Cc:     Chris Mason <clm@...com>, squashfs-devel@...ts.sourceforge.net,
        Herbert Xu <herbert@...dor.apana.org.au>,
        Nick Terrell <nickrterrell@...il.com>,
        Yann Collet <cyan@...com>, Petr Malat <oss@...at.biz>,
        linux-kernel@...r.kernel.org,
        linux-f2fs-devel@...ts.sourceforge.net,
        Nick Terrell <terrelln@...com>, linux-crypto@...r.kernel.org,
        Kernel Team <Kernel-team@...com>,
        Niket Agarwal <niketa@...com>, linux-btrfs@...r.kernel.org,
        Johannes Weiner <jweiner@...com>
Subject: Re: [PATCH 5/9] btrfs: zstd: Switch to the zstd-1.4.6 API

On Wed, Sep 16, 2020 at 03:46:18PM +0100, Christoph Hellwig wrote:
> On Wed, Sep 16, 2020 at 10:43:04AM -0400, Chris Mason wrote:
> > Otherwise we just end up with drift and kernel-specific bugs that are harder
> > to debug.  To the extent those APIs make us contort the kernel code, I???m
> > sure Nick is interested in improving things in both places.
> 
> Seriously, we do not care elsewhere.  Why would zlib be any different?
> 
> > There are probably 1000 constructive ways to have that conversation.  Please
> > choose one of those instead of being an asshole.
> 
> I think you are the asshole here by ignoring the practices we are using
> elsewhere and think your employers pet project is somehow special.  It
> is not, and claiming so is everything but constructive.
> 

The userspace Zstandard library is widely used and has been heavily reviewed,
tested, and fuzzed.

The options are either (a) write and maintain a separate kernel implementation
of Zstandard, or (b) periodically sync from upstream and make minimal, easily
reviewable changes to integrate with the kernel.

I don't see option (a) working for Zstandard.  For short and simple code, it's
the usual Linux kernel practice and it works fine.  But it's far too hard to
write and maintain a good implementation of Zstandard -- meaning correct, fast,
fully fuzzed, and supporting all needed compression levels.  Optimizing
compressors and decompressors is really hard.  A "naive" implementation wouldn't
be too hard, but it would be slow and wouldn't support high compression ratios.

Similarly, some of the crypto assembly code in the kernel is taken from the
OpenSSL project, since the kernel community doesn't have the capacity to
properly optimize algorithms like Poly1305 for x86, arm, arm64, mips, ...

If your main concern is about the camel case function naming, that doesn't seem
very important, relatively speaking.

- Eric