linux-kernel - Re: [PATCH RFC] x86: Add CONFIG_KERNEL

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAG48ez2hVHk-C4XAGW2GieHZ9JAF0RrFfpZF7XhYc80pznMwbA@mail.gmail.com>
Date: Wed, 22 Jan 2025 14:54:01 +0100
From: Jann Horn <jannh@...gle.com>
To: Ard Biesheuvel <ardb@...nel.org>
Cc: Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>, 
	Dave Hansen <dave.hansen@...ux.intel.com>, x86@...nel.org, 
	"H. Peter Anvin" <hpa@...or.com>, Masahiro Yamada <masahiroy@...nel.org>, 
	Nathan Chancellor <nathan@...nel.org>, Nicolas Schier <nicolas@...sle.eu>, linux-efi@...r.kernel.org, 
	Andrew Morton <akpm@...ux-foundation.org>, linux-kbuild@...r.kernel.org, 
	linux-kernel@...r.kernel.org
Subject: Re: [PATCH RFC] x86: Add CONFIG_KERNEL_UNCOMPRESSED support

On Wed, Jan 22, 2025 at 2:31 PM Ard Biesheuvel <ardb@...nel.org> wrote:
> Hi Jann,
>
> On Tue, 21 Jan 2025 at 23:16, Jann Horn <jannh@...gle.com> wrote:
> >
> > Support storing the kernel uncompressed for developers who want to quickly
> > iterate with one-off kernel builds.
> > Store it in the usual format with a 4-byte length suffix and keep this new
> > codepath as close as possible to the normal path where decompression
> > happens.
> >
> > The other compression methods offered by the kernel take some time;
> > even LZ4 (which the kernel uses at compression level 9) takes ~2.8
> > seconds to compress a 110M large vmlinux.bin on my machine.
> >
> > An alternate approach to this would be to offer customization of the LZ4
> > compression level through a kconfig variable; and yet another approach
> > would be to abuse the existing gzip decompression logic by storing the
> > kernel as "non-compressed" DEFLATE blocks, so that the decompression code
> > will essentially end up just doing a bunch of memcpy() calls.
> >
>
> This all seems pretty complicated, and adding yet another
> (pseudo-)compression method is not great in terms of maintenance
> burden, especially because there are other consumers of the compressed
> images (both for bzImage and EFI zboot)
>
> Did you try running gzip with -1 instead of -9? On my build machine,
> this reduces the compression time of a defconfig bzImage build from
> 4.3 seconds to 0.9 seconds.

I tried lz4 with -1; that is very fast (240ms wall clock time on my
machine, and just 120ms user time):

$ ls -lh arch/x86/boot/compressed/vmlinux.bin
-rwxr-x--- 1 [...] 110M Jan 22 00:01 arch/x86/boot/compressed/vmlinux.bin
$ cat arch/x86/boot/compressed/vmlinux.bin | time lz4 -l -9 - - | wc -c
2.86user 0.04system 0:02.96elapsed 97%CPU (0avgtext+0avgdata 15756maxresident)k
0inputs+0outputs (0major+220minor)pagefaults 0swaps
46309676
$ cat arch/x86/boot/compressed/vmlinux.bin | time lz4 -l -1 - - | wc -c
0.12user 0.06system 0:00.24elapsed 75%CPU (0avgtext+0avgdata 15524maxresident)k
0inputs+0outputs (0major+94minor)pagefaults 0swaps
56029608

But I wasn't sure how to wire that up in a nice way. I guess the
nicest option would be to create a separate kconfig variable for the
compression level to use for any cmd_lz4/cmd_lz4_with_size invocations
in the build process; and then maybe only make this option visible if
LZ4 is selected as kernel compression method?

Another option would be to create a new option in the "Kernel
compression mode" choice menu with a name like "LZ4 (fast)", turn
CONFIG_KERNEL_LZ4 into an internal flag that is selected by both LZ4
variants shown in the choice menu, and duplicate some of the make
rules, but that seems overly complicated.