linux-kernel - Re: [PATCH RFC v2] selftests/nolibc: don't embed initramfs into kernel image

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <ZQmKYz6WRkznqdWV@fjasle.eu>
Date:   Tue, 19 Sep 2023 13:47:47 +0200
From:   Nicolas Schier <nicolas@...sle.eu>
To:     Thomas Weißschuh <linux@...ssschuh.net>
Cc:     Masahiro Yamada <masahiroy@...nel.org>,
        Nathan Chancellor <nathan@...nel.org>,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        Willy Tarreau <w@....eu>, Shuah Khan <shuah@...nel.org>,
        linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
        linux-kselftest@...r.kernel.org
Subject: Re: [PATCH RFC v2] selftests/nolibc: don't embed initramfs into
 kernel image

On Sun, Sep 17, 2023 at 05:21:38PM +0200 Thomas Weißschuh wrote:
> When the initramfs is embedded into the kernel each rebuild of it will
> trigger a full kernel relink and all the expensive postprocessing steps.
> 
> Currently nolibc-test and therefore the initramfs are always rebuild,
> even without source changes, leading to lots of slow kernel relinks.
> 
> Instead of linking the initramfs into the kernel assemble it manually
> and pass it explicitly to qemu.
> This avoids all of the kernel relinks.
> 
> Signed-off-by: Thomas Weißschuh <linux@...ssschuh.net>
> ---
> Currently the nolibc testsuite embeds the test executable into a kernel
> image with CONFIG_INITRAMFS_SOURCE.
> This forces a full kernel relink everytime the test executable is
> updated.
> 
> This relinking step dominates the test cycle.
> It is slower than building and running the test in qemu together.
> 
> With a bit of Makefile-shuffling the relinking can be avoided.
> ---
> Changes in v2:
> - avoid need to modify top-level Makefile
> - drop patch removing "rerun" target
> - add kernel-standalone target
> - Link to v1: https://lore.kernel.org/r/20230916-nolibc-initramfs-v1-0-4416ecedca6d@weissschuh.net
> ---

Thanks, seems to work as described (and I am surprised how fast the 'run'
target is) and patch looks good to me.

Reviewed-by: Nicolas Schier <nicolas@...sle.eu>


>  tools/testing/selftests/nolibc/Makefile | 42 ++++++++++++++++++++-------------
>  1 file changed, 26 insertions(+), 16 deletions(-)
> 
> diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
> index 689658f81a19..ee6a9ad28cfd 100644
> --- a/tools/testing/selftests/nolibc/Makefile
> +++ b/tools/testing/selftests/nolibc/Makefile
> @@ -131,18 +131,20 @@ REPORT  ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{if (!f) printf("\n"); f++
>  
>  help:
>  	@echo "Supported targets under selftests/nolibc:"
> -	@echo "  all          call the \"run\" target below"
> -	@echo "  help         this help"
> -	@echo "  sysroot      create the nolibc sysroot here (uses \$$ARCH)"
> -	@echo "  nolibc-test  build the executable (uses \$$CC and \$$CROSS_COMPILE)"
> -	@echo "  libc-test    build an executable using the compiler's default libc instead"
> -	@echo "  run-user     runs the executable under QEMU (uses \$$XARCH, \$$TEST)"
> -	@echo "  initramfs    prepare the initramfs with nolibc-test"
> -	@echo "  defconfig    create a fresh new default config (uses \$$XARCH)"
> -	@echo "  kernel       (re)build the kernel with the initramfs (uses \$$XARCH)"
> -	@echo "  run          runs the kernel in QEMU after building it (uses \$$XARCH, \$$TEST)"
> -	@echo "  rerun        runs a previously prebuilt kernel in QEMU (uses \$$XARCH, \$$TEST)"
> -	@echo "  clean        clean the sysroot, initramfs, build and output files"
> +	@echo "  all               call the \"run\" target below"
> +	@echo "  help              this help"
> +	@echo "  sysroot           create the nolibc sysroot here (uses \$$ARCH)"
> +	@echo "  nolibc-test       build the executable (uses \$$CC and \$$CROSS_COMPILE)"
> +	@echo "  libc-test         build an executable using the compiler's default libc instead"
> +	@echo "  run-user          runs the executable under QEMU (uses \$$XARCH, \$$TEST)"
> +	@echo "  initramfs.cpio    prepare the initramfs archive with nolibc-test"
> +	@echo "  initramfs         prepare the initramfs tree with nolibc-test"
> +	@echo "  defconfig         create a fresh new default config (uses \$$XARCH)"
> +	@echo "  kernel            (re)build the kernel (uses \$$XARCH)"
> +	@echo "  kernel-standalone (re)build the kernel with the initramfs (uses \$$XARCH)"
> +	@echo "  run               runs the kernel in QEMU after building it (uses \$$XARCH, \$$TEST)"
> +	@echo "  rerun             runs a previously prebuilt kernel in QEMU (uses \$$XARCH, \$$TEST)"
> +	@echo "  clean             clean the sysroot, initramfs, build and output files"
>  	@echo ""
>  	@echo "The output file is \"run.out\". Test ranges may be passed using \$$TEST."
>  	@echo ""
> @@ -195,6 +197,9 @@ run-user: nolibc-test
>  	$(Q)qemu-$(QEMU_ARCH) ./nolibc-test > "$(CURDIR)/run.out" || :
>  	$(Q)$(REPORT) $(CURDIR)/run.out
>  
> +initramfs.cpio: kernel nolibc-test
> +	$(QUIET_GEN)echo 'file /init nolibc-test 755 0 0' | $(srctree)/usr/gen_init_cpio - > initramfs.cpio
> +
>  initramfs: nolibc-test
>  	$(QUIET_MKDIR)mkdir -p initramfs
>  	$(call QUIET_INSTALL, initramfs/init)
> @@ -203,17 +208,20 @@ initramfs: nolibc-test
>  defconfig:
>  	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare
>  
> -kernel: initramfs
> +kernel:
> +	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME)
> +
> +kernel-standalone: initramfs
>  	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME) CONFIG_INITRAMFS_SOURCE=$(CURDIR)/initramfs
>  
>  # run the tests after building the kernel
> -run: kernel
> -	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
> +run: kernel initramfs.cpio
> +	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -initrd initramfs.cpio -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
>  	$(Q)$(REPORT) $(CURDIR)/run.out
>  
>  # re-run the tests from an existing kernel
>  rerun:
> -	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
> +	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -initrd initramfs.cpio -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
>  	$(Q)$(REPORT) $(CURDIR)/run.out
>  
>  # report with existing test log
> @@ -227,6 +235,8 @@ clean:
>  	$(Q)rm -f nolibc-test
>  	$(call QUIET_CLEAN, libc-test)
>  	$(Q)rm -f libc-test
> +	$(call QUIET_CLEAN, initramfs.cpio)
> +	$(Q)rm -rf initramfs.cpio
>  	$(call QUIET_CLEAN, initramfs)
>  	$(Q)rm -rf initramfs
>  	$(call QUIET_CLEAN, run.out)
> 
> ---
> base-commit: 3f79a57865b33f49fdae6655510bd27c8e6610e0
> change-id: 20230916-nolibc-initramfs-4fd00eac3256
> 
> Best regards,
> -- 
> Thomas Weißschuh <linux@...ssschuh.net>

-- 
epost|xmpp: nicolas@...sle.eu          irc://oftc.net/nsc
↳ gpg: 18ed 52db e34f 860e e9fb  c82b 7d97 0932 55a0 ce7f
     -- frykten for herren er opphav til kunnskap --