linux-kernel - [PATCH v2] kbuild: Add CONFIG_LZ4_LEVEL for choosing compression level

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <20250122-kernel-compress-fast-v2-1-bde9d71b6b88@google.com>
Date: Wed, 22 Jan 2025 16:08:22 +0100
From: Jann Horn <jannh@...gle.com>
To: Masahiro Yamada <masahiroy@...nel.org>, 
 Nathan Chancellor <nathan@...nel.org>, Nicolas Schier <nicolas@...sle.eu>
Cc: Andrew Morton <akpm@...ux-foundation.org>, linux-kbuild@...r.kernel.org, 
 linux-kernel@...r.kernel.org, Tor Vic <torvic9@...lbox.org>, 
 Ard Biesheuvel <ardb@...nel.org>, Jann Horn <jannh@...gle.com>
Subject: [PATCH v2] kbuild: Add CONFIG_LZ4_LEVEL for choosing compression
 level

Allow choosing the LZ4 compression level using the new config flag
CONFIG_LZ4_LEVEL.

This will allow developers who want to quickly iterate with one-off kernel
builds to speed up kernel compression at the cost of creating a larger
kernel image.

LZ4 at level 9 (which the kernel uses by default) takes ~2.8
seconds to compress a 110M large vmlinux.bin on my machine; LZ4 at level 1
just needs ~220ms wall clock time.

Signed-off-by: Jann Horn <jannh@...gle.com>
---
My patch provides kconfig for choosing compression level of LZ4; Tor Vic
mentioned using kconfig for choosing the compression level of ZSTD.

Does my current approach of having a kconfig option specifically for
LZ4 compression level look good (and we can add the same for ZSTD later
if necessary); or should there be a single kconfig option that sets a
compression level that is used across all compression algorithms?
I guess doing it separately for different algorithms makes more sense
because different compression levels might mean different things for
different algorithms?

Do I need to somehow make it explicit that cmd_lz4 and cmd_lz4_with_size
rely on CONFIG_LZ4_LEVEL, or does the build system figure that out on
its own somehow? Experimentally it seems like the build system triggers
a rebuild when the config option changes, but I'm not sure how that
actually works.
---
Changes in v2:
- removed x86/EFI subsystems from recipients since I'm no longer
  touching those
- rewrote patch to make LZ4 level configurable instead of offering
  uncompressed builds on x86 (Ard)
- Link to v1: https://lore.kernel.org/r/20250121-kernel-compress-fast-v1-1-fa693b6167d4@google.com
---
 init/Kconfig         | 16 ++++++++++++++++
 scripts/Makefile.lib |  4 ++--
 2 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/init/Kconfig b/init/Kconfig
index a20e6efd3f0fbdd7f0df2448854cc30734a0ee4f..3fd93bbe98390997e7e2d41426241b90d3b33053 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -358,6 +358,9 @@ config KERNEL_LZ4
 	  is about 8% bigger than LZO. But the decompression speed is
 	  faster than LZO.
 
+	  If you are working on the kernel and recompiling frequently, you can
+	  adjust the compression level with CONFIG_LZ4_LEVEL.
+
 config KERNEL_ZSTD
 	bool "ZSTD"
 	depends on HAVE_KERNEL_ZSTD
@@ -380,6 +383,19 @@ config KERNEL_UNCOMPRESSED
 
 endchoice
 
+config LZ4_LEVEL
+	int "LZ4 compression level"
+	range 1 12
+	depends on KERNEL_LZ4
+	default 9
+	help
+	  This option controls the compression level used for LZ4 invocations,
+	  in particular for compressing the kernel image.
+
+	  If you are working on the kernel and recompiling frequently, you can
+	  set this to a low value like 1 to speed up kernel builds at the cost
+	  of using more disk space.
+
 config DEFAULT_INIT
 	string "Default init path"
 	default ""
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 7395200538da89a2f6e6d21f8959f3f60d291d79..c3eedbc4acf133d5f9e7f13d637af09ad1968930 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -425,10 +425,10 @@ quiet_cmd_lzo_with_size = LZO     $@
       cmd_lzo_with_size = { cat $(real-prereqs) | $(KLZOP) -9; $(size_append); } > $@
 
 quiet_cmd_lz4 = LZ4     $@
-      cmd_lz4 = cat $(real-prereqs) | $(LZ4) -l -9 - - > $@
+      cmd_lz4 = cat $(real-prereqs) | $(LZ4) -l -$(CONFIG_LZ4_LEVEL) - - > $@
 
 quiet_cmd_lz4_with_size = LZ4     $@
-      cmd_lz4_with_size = { cat $(real-prereqs) | $(LZ4) -l -9 - -; \
+      cmd_lz4_with_size = { cat $(real-prereqs) | $(LZ4) -l -$(CONFIG_LZ4_LEVEL) - -; \
                   $(size_append); } > $@
 
 # U-Boot mkimage

---
base-commit: 95ec54a420b8f445e04a7ca0ea8deb72c51fe1d3
change-id: 20250121-kernel-compress-fast-350ce5801c28

-- 
Jann Horn <jannh@...gle.com>