linux-kernel - Re: [PATCH] kbuild: modinst: remove the multithread option from zstd compression

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CA+icZUWpeZF-PGQJLR1tt0u7sFVZ+MANX4hE-DUCEt=PhXGs3w@mail.gmail.com>
Date: Sun, 4 Aug 2024 08:35:47 +0200
From: Sedat Dilek <sedat.dilek@...il.com>
To: Masahiro Yamada <masahiroy@...nel.org>
Cc: linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org, 
	Nathan Chancellor <nathan@...nel.org>, Nick Terrell <terrelln@...com>, Nicolas Schier <nicolas@...sle.eu>
Subject: Re: [PATCH] kbuild: modinst: remove the multithread option from zstd compression

On Sun, Aug 4, 2024 at 5:33 AM Masahiro Yamada <masahiroy@...nel.org> wrote:
>
> Parallel execution is supported by GNU Make:
>
>   $ make -j<N> modules_install
>
> It is questionable to enable multithreading within each zstd process
> by default.
>
> If you still want to do it, you can use the environment variable:
>
>   $ ZSTD_NBTHREADS=<N> make modules_install
>

Hi Masahiro,

I have some understanding problems.

[ start-build.txt ]
dileks     24225   24217  0 17:55 tty2     00:00:00 /usr/bin/perf stat
make V=1 -k -j4 ARCH=x86_64 LLVM=1 LLVM=/opt/llvm/bin/
PAHOLE=/opt/pahole/bin/pahole KBUILD_BUILD_HOST=iniza
KBUILD_BUILD_USER=sedat.dilek@...il.com
KBUILD_BUILD_TIMESTAMP=2024-08-03
KDEB_PKGVERSION=6.10.3-1~trixie+dileks1
LOCALVERSION=-1-amd64-clang18-kcfi olddefconfig bindeb-pkg

^^ How shall someone pass so that ... ZSTD_NBTHREADS=<N> make
modules_install ... is used?

As far as I understood, each kernel-module file is taken - in the
Debian build-process - sequentially file for file - ZSTD compressed
and afterwards deleted.
Is there a benefit when 'make -j<N>' is used?

[ EXAMPLE - with my patch from [1] ]
zstd -T0 -19 --rm -f -q
debian/linux-image-6.10.3-1-amd64-clang18-kcfi/lib/modules/6.10.3-1-amd64-clang18-kcfi/kernel/drivers/gpu/drm/amd/amdgpu/amdgpu.ko

^^ How should ZSTD_NBTHREADS=<N> replace 'zstd -T0'?

When I pass '-T0 -L19' to ZSTD two cores are used on my CPU and a
better compress-level produces smaller kernel-module files.
I would like to benefit from both.

Thanks.

Best regards,
-Sedat-

Link: https://lore.kernel.org/all/CA+icZUUQadYjAXiCNx7PmKDV20WctvnzkXC3R2F7FM4Gzcm39Q@mail.gmail.com/
[1]

> Signed-off-by: Masahiro Yamada <masahiroy@...nel.org>
> ---
>
>  scripts/Makefile.modinst | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
> index 0afd75472679..04f5229efa6b 100644
> --- a/scripts/Makefile.modinst
> +++ b/scripts/Makefile.modinst
> @@ -146,7 +146,7 @@ quiet_cmd_gzip = GZIP    $@
>  quiet_cmd_xz = XZ      $@
>        cmd_xz = $(XZ) --check=crc32 --lzma2=dict=1MiB -f $<
>  quiet_cmd_zstd = ZSTD    $@
> -      cmd_zstd = $(ZSTD) -T0 --rm -f -q $<
> +      cmd_zstd = $(ZSTD) --rm -f -q $<
>
>  $(dst)/%.ko.gz: $(dst)/%.ko FORCE
>         $(call cmd,gzip)
> --
> 2.43.0
>
>