linux-kernel - Re: [PATCH RFC] x86: Add CONFIG_KERNEL

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAMj1kXFT6wkP=eRemR1Y=C-fk2VxNurLHMy74VRFLNmx6NkOAA@mail.gmail.com>
Date: Wed, 22 Jan 2025 15:21:33 +0100
From: Ard Biesheuvel <ardb@...nel.org>
To: Jann Horn <jannh@...gle.com>
Cc: Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>, 
	Dave Hansen <dave.hansen@...ux.intel.com>, x86@...nel.org, 
	"H. Peter Anvin" <hpa@...or.com>, Masahiro Yamada <masahiroy@...nel.org>, 
	Nathan Chancellor <nathan@...nel.org>, Nicolas Schier <nicolas@...sle.eu>, linux-efi@...r.kernel.org, 
	Andrew Morton <akpm@...ux-foundation.org>, linux-kbuild@...r.kernel.org, 
	linux-kernel@...r.kernel.org
Subject: Re: [PATCH RFC] x86: Add CONFIG_KERNEL_UNCOMPRESSED support

On Wed, 22 Jan 2025 at 14:54, Jann Horn <jannh@...gle.com> wrote:
>
> On Wed, Jan 22, 2025 at 2:31 PM Ard Biesheuvel <ardb@...nel.org> wrote:
> > Hi Jann,
> >
> > On Tue, 21 Jan 2025 at 23:16, Jann Horn <jannh@...gle.com> wrote:
> > >
> > > Support storing the kernel uncompressed for developers who want to quickly
> > > iterate with one-off kernel builds.
> > > Store it in the usual format with a 4-byte length suffix and keep this new
> > > codepath as close as possible to the normal path where decompression
> > > happens.
> > >
> > > The other compression methods offered by the kernel take some time;
> > > even LZ4 (which the kernel uses at compression level 9) takes ~2.8
> > > seconds to compress a 110M large vmlinux.bin on my machine.
> > >
> > > An alternate approach to this would be to offer customization of the LZ4
> > > compression level through a kconfig variable; and yet another approach
> > > would be to abuse the existing gzip decompression logic by storing the
> > > kernel as "non-compressed" DEFLATE blocks, so that the decompression code
> > > will essentially end up just doing a bunch of memcpy() calls.
> > >
> >
> > This all seems pretty complicated, and adding yet another
> > (pseudo-)compression method is not great in terms of maintenance
> > burden, especially because there are other consumers of the compressed
> > images (both for bzImage and EFI zboot)
> >
> > Did you try running gzip with -1 instead of -9? On my build machine,
> > this reduces the compression time of a defconfig bzImage build from
> > 4.3 seconds to 0.9 seconds.
>
> I tried lz4 with -1; that is very fast (240ms wall clock time on my
> machine, and just 120ms user time):
>
> $ ls -lh arch/x86/boot/compressed/vmlinux.bin
> -rwxr-x--- 1 [...] 110M Jan 22 00:01 arch/x86/boot/compressed/vmlinux.bin
> $ cat arch/x86/boot/compressed/vmlinux.bin | time lz4 -l -9 - - | wc -c
> 2.86user 0.04system 0:02.96elapsed 97%CPU (0avgtext+0avgdata 15756maxresident)k
> 0inputs+0outputs (0major+220minor)pagefaults 0swaps
> 46309676
> $ cat arch/x86/boot/compressed/vmlinux.bin | time lz4 -l -1 - - | wc -c
> 0.12user 0.06system 0:00.24elapsed 75%CPU (0avgtext+0avgdata 15524maxresident)k
> 0inputs+0outputs (0major+94minor)pagefaults 0swaps
> 56029608
>

Excellent.

> But I wasn't sure how to wire that up in a nice way. I guess the
> nicest option would be to create a separate kconfig variable for the
> compression level to use for any cmd_lz4/cmd_lz4_with_size invocations
> in the build process; and then maybe only make this option visible if
> LZ4 is selected as kernel compression method?
>
> Another option would be to create a new option in the "Kernel
> compression mode" choice menu with a name like "LZ4 (fast)", turn
> CONFIG_KERNEL_LZ4 into an internal flag that is selected by both LZ4
> variants shown in the choice menu, and duplicate some of the make
> rules, but that seems overly complicated.
>

I didn't realise that KERNEL_UNCOMPRESSED already exists and you are
just wiring it up for x86. But I still think that we should avoid
that, not only because it is yet another bzImage format but also
because I still see a 3x size reduction even with the fastest setting.

I think adding one Kconfig symbol that depends on KERNEL_LZ4 and
switches from -9 to -1 for LZ4 only is reasonable.