| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <20230718125847.3869700-1-ardb@kernel.org>
Date: Tue, 18 Jul 2023 14:58:26 +0200
From: Ard Biesheuvel <ardb@...nel.org>
To: linux-crypto@...r.kernel.org
Cc: Ard Biesheuvel <ardb@...nel.org>,
Herbert Xu <herbert@...dor.apana.org.au>,
Eric Biggers <ebiggers@...nel.org>,
Kees Cook <keescook@...omium.org>,
Haren Myneni <haren@...ibm.com>,
Nick Terrell <terrelln@...com>,
Minchan Kim <minchan@...nel.org>,
Sergey Senozhatsky <senozhatsky@...omium.org>,
Jens Axboe <axboe@...nel.dk>,
Giovanni Cabiddu <giovanni.cabiddu@...el.com>,
Richard Weinberger <richard@....at>,
David Ahern <dsahern@...nel.org>,
Eric Dumazet <edumazet@...gle.com>,
Jakub Kicinski <kuba@...nel.org>,
Paolo Abeni <pabeni@...hat.com>,
Steffen Klassert <steffen.klassert@...unet.com>,
linux-kernel@...r.kernel.org, linux-block@...r.kernel.org,
qat-linux@...el.com, linuxppc-dev@...ts.ozlabs.org,
linux-mtd@...ts.infradead.org, netdev@...r.kernel.org
Subject: [RFC PATCH 00/21] crypto: consolidate and clean up compression APIs
This series is presented as an RFC, because I haven't quite convinced
myself that the acomp API really needs both scatterlists and request
objects to encapsulate the in- and output buffers, and perhaps there are
more drastic simplifications that we might consider.
However, the current situation with comp, scomp and acomp APIs is
definitely something that needs cleaning up, and so I implemented this
series under the working assumption that we will keep the current acomp
semantics wrt scatterlists and request objects.
Patch #1 drops zlib-deflate support in software, along with the test
cases we have for it. This has no users and should have never been
added.
Patch #2 removes the support for on-the-fly allocation of destination
buffers and scatterlists from the Intel QAT driver. This is never used,
and not even implemented by all drivers (the HiSilicon ZIP driver does
not support it). The diffstat of this patch makes a good case why the
caller should be in charge of allocating the memory, not the driver.
Patch #3 removes this on-the-fly allocation from the core acomp API.
Patch #4 does a minimal conversion of IPcomp to the acomp API.
Patch #5 and #6 are independent UBIFS fixes for things I ran into while
working on patch #7.
Patch #7 converts UBIFS to the acomp API.
Patch #8 converts the zram block driver to the acomp API.
Patches #9 to #19 remove the existing 'comp' API implementations as well
as the core plumbing, now that all clients of the API have been
converted. (Note that pstore stopped using the 'comp' API as well, but
these changes are already queued elsewhere)
Patch #20 converts the generic deflate compression driver to the acomp
API, so that it can natively operate on discontiguous buffers, rather
than requiring scratch buffers. This is the only IPcomp compression
algorithm we actually implement in software in the kernel, and this
conversion could help IPcomp if we decide to convert it further, and
remove the code that 'linearizes' SKBs in order to present them to the
compression API as a contiguous range.
Patch #21 converts the acomp-to-scomp adaptation layer so it no longer
requires per-CPU scratch buffers. This takes advantage of the fact that
all existing users of the acomp API pass contiguous memory regions, and
so scratch buffers are only needed in exceptional cases, and can be
allocated and deallocated on the fly. This removes the need for
preallocated per-CPU scratch buffers that can easily add up to tens of
megabytes on modern systems with high core counts and SMT.
These changes have been build tested and only lightly runtime tested. In
particular, I haven't performed any thorough testing on the acomp
conversions of IPcomp, UBIFS and ZRAM. Any hints on which respective
methods and test cases to use here are highly appreciated.
Cc: Herbert Xu <herbert@...dor.apana.org.au>
Cc: Eric Biggers <ebiggers@...nel.org>
Cc: Kees Cook <keescook@...omium.org>
Cc: Haren Myneni <haren@...ibm.com>
Cc: Nick Terrell <terrelln@...com>
Cc: Minchan Kim <minchan@...nel.org>
Cc: Sergey Senozhatsky <senozhatsky@...omium.org>
Cc: Jens Axboe <axboe@...nel.dk>
Cc: Giovanni Cabiddu <giovanni.cabiddu@...el.com>
Cc: Richard Weinberger <richard@....at>
Cc: David Ahern <dsahern@...nel.org>
Cc: Eric Dumazet <edumazet@...gle.com>
Cc: Jakub Kicinski <kuba@...nel.org>
Cc: Paolo Abeni <pabeni@...hat.com>
Cc: Steffen Klassert <steffen.klassert@...unet.com>
Cc: linux-crypto@...r.kernel.org
Cc: linux-kernel@...r.kernel.org
Cc: linux-block@...r.kernel.org
Cc: qat-linux@...el.com
Cc: linuxppc-dev@...ts.ozlabs.org
Cc: linux-mtd@...ts.infradead.org
Cc: netdev@...r.kernel.org
Ard Biesheuvel (21):
crypto: scomp - Revert "add support for deflate rfc1950 (zlib)"
crypto: qat - Drop support for allocating destination buffers
crypto: acompress - Drop destination scatterlist allocation feature
net: ipcomp: Migrate to acomp API from deprecated comp API
ubifs: Pass worst-case buffer size to compression routines
ubifs: Avoid allocating buffer space unnecessarily
ubifs: Migrate to acomp compression API
zram: Migrate to acomp compression API
crypto: nx - Migrate to scomp API
crypto: 842 - drop obsolete 'comp' implementation
crypto: deflate - drop obsolete 'comp' implementation
crypto: lz4 - drop obsolete 'comp' implementation
crypto: lz4hc - drop obsolete 'comp' implementation
crypto: lzo-rle - drop obsolete 'comp' implementation
crypto: lzo - drop obsolete 'comp' implementation
crypto: zstd - drop obsolete 'comp' implementation
crypto: cavium/zip - drop obsolete 'comp' implementation
crypto: compress_null - drop obsolete 'comp' implementation
crypto: remove obsolete 'comp' compression API
crypto: deflate - implement acomp API directly
crypto: scompress - Drop the use of per-cpu scratch buffers
Documentation/crypto/architecture.rst | 2 -
crypto/842.c | 63 +---
crypto/Makefile | 2 +-
crypto/acompress.c | 6 -
crypto/api.c | 4 -
crypto/compress.c | 32 --
crypto/crypto_null.c | 31 +-
crypto/crypto_user_base.c | 16 -
crypto/crypto_user_stat.c | 4 -
crypto/deflate.c | 386 ++++++--------------
crypto/lz4.c | 61 +---
crypto/lz4hc.c | 63 +---
crypto/lzo-rle.c | 60 +--
crypto/lzo.c | 60 +--
crypto/proc.c | 3 -
crypto/scompress.c | 169 ++++-----
crypto/testmgr.c | 184 +---------
crypto/testmgr.h | 75 ----
crypto/zstd.c | 56 +--
drivers/block/zram/zcomp.c | 67 +++-
drivers/block/zram/zcomp.h | 7 +-
drivers/block/zram/zram_drv.c | 12 +-
drivers/crypto/cavium/zip/zip_crypto.c | 40 --
drivers/crypto/cavium/zip/zip_crypto.h | 10 -
drivers/crypto/cavium/zip/zip_main.c | 50 +--
drivers/crypto/intel/qat/qat_common/qat_bl.c | 159 --------
drivers/crypto/intel/qat/qat_common/qat_bl.h | 6 -
drivers/crypto/intel/qat/qat_common/qat_comp_algs.c | 86 +----
drivers/crypto/intel/qat/qat_common/qat_comp_req.h | 10 -
drivers/crypto/nx/nx-842.c | 34 +-
drivers/crypto/nx/nx-842.h | 14 +-
drivers/crypto/nx/nx-common-powernv.c | 30 +-
drivers/crypto/nx/nx-common-pseries.c | 32 +-
fs/ubifs/compress.c | 61 +++-
fs/ubifs/file.c | 46 +--
fs/ubifs/journal.c | 33 +-
fs/ubifs/ubifs.h | 15 +-
include/crypto/acompress.h | 21 +-
include/crypto/internal/scompress.h | 2 -
include/crypto/scatterwalk.h | 2 +-
include/linux/crypto.h | 49 +--
include/net/ipcomp.h | 4 +-
net/xfrm/xfrm_algo.c | 7 +-
net/xfrm/xfrm_ipcomp.c | 107 ++++--
44 files changed, 502 insertions(+), 1679 deletions(-)
delete mode 100644 crypto/compress.c
--
2.39.2
Powered by blists - more mailing lists