| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <20230718125847.3869700-1-ardb@kernel.org> Date: Tue, 18 Jul 2023 14:58:26 +0200 From: Ard Biesheuvel <ardb@...nel.org> To: linux-crypto@...r.kernel.org Cc: Ard Biesheuvel <ardb@...nel.org>, Herbert Xu <herbert@...dor.apana.org.au>, Eric Biggers <ebiggers@...nel.org>, Kees Cook <keescook@...omium.org>, Haren Myneni <haren@...ibm.com>, Nick Terrell <terrelln@...com>, Minchan Kim <minchan@...nel.org>, Sergey Senozhatsky <senozhatsky@...omium.org>, Jens Axboe <axboe@...nel.dk>, Giovanni Cabiddu <giovanni.cabiddu@...el.com>, Richard Weinberger <richard@....at>, David Ahern <dsahern@...nel.org>, Eric Dumazet <edumazet@...gle.com>, Jakub Kicinski <kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>, Steffen Klassert <steffen.klassert@...unet.com>, linux-kernel@...r.kernel.org, linux-block@...r.kernel.org, qat-linux@...el.com, linuxppc-dev@...ts.ozlabs.org, linux-mtd@...ts.infradead.org, netdev@...r.kernel.org Subject: [RFC PATCH 00/21] crypto: consolidate and clean up compression APIs This series is presented as an RFC, because I haven't quite convinced myself that the acomp API really needs both scatterlists and request objects to encapsulate the in- and output buffers, and perhaps there are more drastic simplifications that we might consider. However, the current situation with comp, scomp and acomp APIs is definitely something that needs cleaning up, and so I implemented this series under the working assumption that we will keep the current acomp semantics wrt scatterlists and request objects. Patch #1 drops zlib-deflate support in software, along with the test cases we have for it. This has no users and should have never been added. Patch #2 removes the support for on-the-fly allocation of destination buffers and scatterlists from the Intel QAT driver. This is never used, and not even implemented by all drivers (the HiSilicon ZIP driver does not support it). The diffstat of this patch makes a good case why the caller should be in charge of allocating the memory, not the driver. Patch #3 removes this on-the-fly allocation from the core acomp API. Patch #4 does a minimal conversion of IPcomp to the acomp API. Patch #5 and #6 are independent UBIFS fixes for things I ran into while working on patch #7. Patch #7 converts UBIFS to the acomp API. Patch #8 converts the zram block driver to the acomp API. Patches #9 to #19 remove the existing 'comp' API implementations as well as the core plumbing, now that all clients of the API have been converted. (Note that pstore stopped using the 'comp' API as well, but these changes are already queued elsewhere) Patch #20 converts the generic deflate compression driver to the acomp API, so that it can natively operate on discontiguous buffers, rather than requiring scratch buffers. This is the only IPcomp compression algorithm we actually implement in software in the kernel, and this conversion could help IPcomp if we decide to convert it further, and remove the code that 'linearizes' SKBs in order to present them to the compression API as a contiguous range. Patch #21 converts the acomp-to-scomp adaptation layer so it no longer requires per-CPU scratch buffers. This takes advantage of the fact that all existing users of the acomp API pass contiguous memory regions, and so scratch buffers are only needed in exceptional cases, and can be allocated and deallocated on the fly. This removes the need for preallocated per-CPU scratch buffers that can easily add up to tens of megabytes on modern systems with high core counts and SMT. These changes have been build tested and only lightly runtime tested. In particular, I haven't performed any thorough testing on the acomp conversions of IPcomp, UBIFS and ZRAM. Any hints on which respective methods and test cases to use here are highly appreciated. Cc: Herbert Xu <herbert@...dor.apana.org.au> Cc: Eric Biggers <ebiggers@...nel.org> Cc: Kees Cook <keescook@...omium.org> Cc: Haren Myneni <haren@...ibm.com> Cc: Nick Terrell <terrelln@...com> Cc: Minchan Kim <minchan@...nel.org> Cc: Sergey Senozhatsky <senozhatsky@...omium.org> Cc: Jens Axboe <axboe@...nel.dk> Cc: Giovanni Cabiddu <giovanni.cabiddu@...el.com> Cc: Richard Weinberger <richard@....at> Cc: David Ahern <dsahern@...nel.org> Cc: Eric Dumazet <edumazet@...gle.com> Cc: Jakub Kicinski <kuba@...nel.org> Cc: Paolo Abeni <pabeni@...hat.com> Cc: Steffen Klassert <steffen.klassert@...unet.com> Cc: linux-crypto@...r.kernel.org Cc: linux-kernel@...r.kernel.org Cc: linux-block@...r.kernel.org Cc: qat-linux@...el.com Cc: linuxppc-dev@...ts.ozlabs.org Cc: linux-mtd@...ts.infradead.org Cc: netdev@...r.kernel.org Ard Biesheuvel (21): crypto: scomp - Revert "add support for deflate rfc1950 (zlib)" crypto: qat - Drop support for allocating destination buffers crypto: acompress - Drop destination scatterlist allocation feature net: ipcomp: Migrate to acomp API from deprecated comp API ubifs: Pass worst-case buffer size to compression routines ubifs: Avoid allocating buffer space unnecessarily ubifs: Migrate to acomp compression API zram: Migrate to acomp compression API crypto: nx - Migrate to scomp API crypto: 842 - drop obsolete 'comp' implementation crypto: deflate - drop obsolete 'comp' implementation crypto: lz4 - drop obsolete 'comp' implementation crypto: lz4hc - drop obsolete 'comp' implementation crypto: lzo-rle - drop obsolete 'comp' implementation crypto: lzo - drop obsolete 'comp' implementation crypto: zstd - drop obsolete 'comp' implementation crypto: cavium/zip - drop obsolete 'comp' implementation crypto: compress_null - drop obsolete 'comp' implementation crypto: remove obsolete 'comp' compression API crypto: deflate - implement acomp API directly crypto: scompress - Drop the use of per-cpu scratch buffers Documentation/crypto/architecture.rst | 2 - crypto/842.c | 63 +--- crypto/Makefile | 2 +- crypto/acompress.c | 6 - crypto/api.c | 4 - crypto/compress.c | 32 -- crypto/crypto_null.c | 31 +- crypto/crypto_user_base.c | 16 - crypto/crypto_user_stat.c | 4 - crypto/deflate.c | 386 ++++++-------------- crypto/lz4.c | 61 +--- crypto/lz4hc.c | 63 +--- crypto/lzo-rle.c | 60 +-- crypto/lzo.c | 60 +-- crypto/proc.c | 3 - crypto/scompress.c | 169 ++++----- crypto/testmgr.c | 184 +--------- crypto/testmgr.h | 75 ---- crypto/zstd.c | 56 +-- drivers/block/zram/zcomp.c | 67 +++- drivers/block/zram/zcomp.h | 7 +- drivers/block/zram/zram_drv.c | 12 +- drivers/crypto/cavium/zip/zip_crypto.c | 40 -- drivers/crypto/cavium/zip/zip_crypto.h | 10 - drivers/crypto/cavium/zip/zip_main.c | 50 +-- drivers/crypto/intel/qat/qat_common/qat_bl.c | 159 -------- drivers/crypto/intel/qat/qat_common/qat_bl.h | 6 - drivers/crypto/intel/qat/qat_common/qat_comp_algs.c | 86 +---- drivers/crypto/intel/qat/qat_common/qat_comp_req.h | 10 - drivers/crypto/nx/nx-842.c | 34 +- drivers/crypto/nx/nx-842.h | 14 +- drivers/crypto/nx/nx-common-powernv.c | 30 +- drivers/crypto/nx/nx-common-pseries.c | 32 +- fs/ubifs/compress.c | 61 +++- fs/ubifs/file.c | 46 +-- fs/ubifs/journal.c | 33 +- fs/ubifs/ubifs.h | 15 +- include/crypto/acompress.h | 21 +- include/crypto/internal/scompress.h | 2 - include/crypto/scatterwalk.h | 2 +- include/linux/crypto.h | 49 +-- include/net/ipcomp.h | 4 +- net/xfrm/xfrm_algo.c | 7 +- net/xfrm/xfrm_ipcomp.c | 107 ++++-- 44 files changed, 502 insertions(+), 1679 deletions(-) delete mode 100644 crypto/compress.c -- 2.39.2
Powered by blists - more mailing lists