| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20240902105656.1383858-1-senozhatsky@chromium.org> Date: Mon, 2 Sep 2024 19:55:48 +0900 From: Sergey Senozhatsky <senozhatsky@...omium.org> To: Andrew Morton <akpm@...ux-foundation.org> Cc: Minchan Kim <minchan@...nel.org>, linux-kernel@...r.kernel.org, Sergey Senozhatsky <senozhatsky@...omium.org> Subject: [PATCHv7 00/24] zram: introduce custom comp backends API This series introduces support for run-time compression algorithms tuning, so users, for instance, can adjust compression/acceleration levels and provide pre-trained compression/decompression dictionaries which certain algorithms support. At this point we stop supporting (old/deprecated) comp API. We may add new acomp API support in the future, but before that zram needs to undergo some major rework (we are not ready for async compression). Some benchmarks for reference (look at column #2) *** init zstd /sys/block/zram0/mm_stat 1750659072 504622188 514355200 0 514355200 1 0 34204 34204 *** init zstd dict=/home/ss/zstd-dict-amd64 /sys/block/zram0/mm_stat 1750650880 465908890 475398144 0 475398144 1 0 34185 34185 *** init zstd level=8 dict=/home/ss/zstd-dict-amd64 /sys/block/zram0/mm_stat 1750654976 430803319 439873536 0 439873536 1 0 34185 34185 *** init lz4 /sys/block/zram0/mm_stat 1750646784 664266564 677060608 0 677060608 1 0 34288 34288 *** init lz4 dict=/home/ss/lz4-dict-amd64 /sys/block/zram0/mm_stat 1750650880 619990300 632102912 0 632102912 1 0 34278 34278 *** init lz4hc /sys/block/zram0/mm_stat 1750630400 609023822 621232128 0 621232128 1 0 34288 34288 *** init lz4hc dict=/home/ss/lz4-dict-amd64 /sys/block/zram0/mm_stat 1750659072 505133172 515231744 0 515231744 1 0 34278 34278 Recompress init zram zstd (prio=0), zstd level=5 (prio 1), zstd with dict (prio 2) *** zstd /sys/block/zram0/mm_stat 1750982656 504630584 514269184 0 514269184 1 0 34204 34204 *** idle recompress priority=1 (zstd level=5) /sys/block/zram0/mm_stat 1750982656 488645601 525438976 0 514269184 1 0 34204 34204 *** idle recompress priority=2 (zstd dict) /sys/block/zram0/mm_stat 1750982656 460869640 517914624 0 514269184 1 0 34185 34204 v6..v7 -- new device attribute algorithm_params to tune and tweak algorithms (instead of extending comp_algorithm/recomp_algorithm attributes) -- added missing sysfs file documentation -- reshufleed docuemntation a tad -- recompress device attr now accepts priority argument Sergey Senozhatsky (24): lib: zstd: export API needed for dictionary support lib: lz4hc: export LZ4_resetStreamHC symbol lib: zstd: fix null-deref in ZSTD_createCDict_advanced2() zram: introduce custom comp backends API zram: add lzo and lzorle compression backends support zram: add lz4 compression backend support zram: add lz4hc compression backend support zram: add zstd compression backend support zram: pass estimated src size hint to zstd zram: add zlib compression backend support zram: add 842 compression backend support zram: check that backends array has at least one backend zram: introduce zcomp_params structure zram: recalculate zstd compression params once zram: introduce algorithm_params device attribute zram: add support for dict comp config zram: introduce zcomp_req structure zram: introduce zcomp_ctx structure zram: move immutable comp params away from per-CPU context zram: add dictionary support to lz4 zram: add dictionary support to lz4hc zram: add dictionary support to zstd backend Documentation/zram: add documentation for algorithm parameters zram: support priority parameter in recompression Documentation/ABI/testing/sysfs-block-zram | 7 + Documentation/admin-guide/blockdev/zram.rst | 64 ++++-- drivers/block/zram/Kconfig | 76 +++++-- drivers/block/zram/Makefile | 8 + drivers/block/zram/backend_842.c | 61 ++++++ drivers/block/zram/backend_842.h | 10 + drivers/block/zram/backend_deflate.c | 146 +++++++++++++ drivers/block/zram/backend_deflate.h | 10 + drivers/block/zram/backend_lz4.c | 127 +++++++++++ drivers/block/zram/backend_lz4.h | 10 + drivers/block/zram/backend_lz4hc.c | 128 +++++++++++ drivers/block/zram/backend_lz4hc.h | 10 + drivers/block/zram/backend_lzo.c | 59 +++++ drivers/block/zram/backend_lzo.h | 10 + drivers/block/zram/backend_lzorle.c | 59 +++++ drivers/block/zram/backend_lzorle.h | 10 + drivers/block/zram/backend_zstd.c | 226 ++++++++++++++++++++ drivers/block/zram/backend_zstd.h | 10 + drivers/block/zram/zcomp.c | 194 +++++++++-------- drivers/block/zram/zcomp.h | 71 ++++-- drivers/block/zram/zram_drv.c | 136 +++++++++++- drivers/block/zram/zram_drv.h | 1 + include/linux/zstd.h | 167 +++++++++++++++ lib/lz4/lz4hc_compress.c | 1 + lib/zstd/compress/zstd_compress.c | 2 + lib/zstd/zstd_compress_module.c | 49 +++++ lib/zstd/zstd_decompress_module.c | 36 ++++ 27 files changed, 1544 insertions(+), 144 deletions(-) create mode 100644 drivers/block/zram/backend_842.c create mode 100644 drivers/block/zram/backend_842.h create mode 100644 drivers/block/zram/backend_deflate.c create mode 100644 drivers/block/zram/backend_deflate.h create mode 100644 drivers/block/zram/backend_lz4.c create mode 100644 drivers/block/zram/backend_lz4.h create mode 100644 drivers/block/zram/backend_lz4hc.c create mode 100644 drivers/block/zram/backend_lz4hc.h create mode 100644 drivers/block/zram/backend_lzo.c create mode 100644 drivers/block/zram/backend_lzo.h create mode 100644 drivers/block/zram/backend_lzorle.c create mode 100644 drivers/block/zram/backend_lzorle.h create mode 100644 drivers/block/zram/backend_zstd.c create mode 100644 drivers/block/zram/backend_zstd.h -- 2.46.0.469.g59c65b2a67-goog
Powered by blists - more mailing lists