linux-kernel - Re: [PATCH] kbuild: modinst: Enable multithread xz compression

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 23 Feb 2023 13:16:04 -0700
From:   Nathan Chancellor <nathan@...nel.org>
To:     André Almeida <andrealmeid@...lia.com>
Cc:     linux-kernel@...r.kernel.org, linux-kbuild@...r.kernel.org,
        Masahiro Yamada <masahiroy@...nel.org>, kernel-dev@...lia.com,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        Nicolas Schier <nicolas@...sle.eu>
Subject: Re: [PATCH] kbuild: modinst: Enable multithread xz compression

On Wed, Feb 22, 2023 at 09:16:07PM -0300, André Almeida wrote:
> As it's done for zstd compression, enable multithread compression for
> xz to speed up module installation.
> 
> Signed-off-by: André Almeida <andrealmeid@...lia.com>

This seems reasonable to me.

Reviewed-by: Nathan Chancellor <nathan@...nel.org>

If for some reason Masahiro does not want to take this, you could set
XZ_OPT=-T0 in your build environment, which should accomplish the same
thing.

> ---
> 
> On my setup xz is a bottleneck during module installation. Here are the
> numbers to install it in a local directory, before and after this patch:
> 
> $ time make INSTALL_MOD_PATH=/home/tonyk/codes/.kernel_deploy/ modules_install -j16
> Executed in  100.08 secs
> 
> $ time make INSTALL_MOD_PATH=/home/tonyk/codes/.kernel_deploy/ modules_install -j16
> Executed in   28.60 secs
> ---
>  scripts/Makefile.modinst | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
> index 4815a8e32227..28dcc523d2ee 100644
> --- a/scripts/Makefile.modinst
> +++ b/scripts/Makefile.modinst
> @@ -99,7 +99,7 @@ endif
>  quiet_cmd_gzip = GZIP    $@
>        cmd_gzip = $(KGZIP) -n -f $<
>  quiet_cmd_xz = XZ      $@
> -      cmd_xz = $(XZ) --lzma2=dict=2MiB -f $<
> +      cmd_xz = $(XZ) --lzma2=dict=2MiB -f -T0 $<
>  quiet_cmd_zstd = ZSTD    $@
>        cmd_zstd = $(ZSTD) -T0 --rm -f -q $<
>  
> -- 
> 2.39.2
>