linux-kernel - Re: [PATCH 2/3] kbuild: do not create intermediate *.tar for source tarballs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20230407181105.GC1018455@dev-arch.thelio-3990X>
Date:   Fri, 7 Apr 2023 11:11:05 -0700
From:   Nathan Chancellor <nathan@...nel.org>
To:     Masahiro Yamada <masahiroy@...nel.org>
Cc:     linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        Nicolas Schier <nicolas@...sle.eu>
Subject: Re: [PATCH 2/3] kbuild: do not create intermediate *.tar for source
 tarballs

On Fri, Apr 07, 2023 at 07:16:28PM +0900, Masahiro Yamada wrote:
> Since commit 05e96e96a315 ("kbuild: use git-archive for source package
> creation"), source tarballs are created in two steps; create *.tar file
> then compress it. I split the compression as a separate rule because I
> just thought 'git archive' supported only gzip for compression. I admit
> the unneeded *.tar file is annoying.
> 
> For other compression algorithms, I could pipe the two commands:
> 
>   $ git archive HEAD | xz > linux.tar.xz
> 
> I read git-archive(1) carefully, and I realized GIT had provided a
> more elegant way:

Hooray for documentation :)

>   $ git -c tar.tar.xz.command=xz archive -o linux.tar.xz HEAD
> 
> This commit uses 'tar.tar.*.command' configuration to specify the
> compression backend so we can create a compressed tarball directly.
> 
> GIT commit 767cf4579f0e ("archive: implement configurable tar filters")
> is more than a decade old, so it should be available on almost all build
> environments.

git 1.7.7 it seems, certainly ancientware in my opinion. If people have
issues with this, they can just upgrade git; even RHEL7 has git 1.8.x.

> Signed-off-by: Masahiro Yamada <masahiroy@...nel.org>

Reviewed-by: Nathan Chancellor <nathan@...nel.org>

> ---
> 
>  scripts/Makefile.package | 24 +++++++++++++++++-------
>  1 file changed, 17 insertions(+), 7 deletions(-)
> 
> diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> index a205617730c6..7707975f729b 100644
> --- a/scripts/Makefile.package
> +++ b/scripts/Makefile.package
> @@ -57,16 +57,23 @@ check-git:
>  		false; \
>  	fi
>  
> +archive-config-tar.gz  = -c tar.tar.gz.command="$(KGZIP)"
> +archive-config-tar.bz2 = -c tar.tar.bz2.command="$(KBZIP2)"
> +archive-config-tar.xz  = -c tar.tar.xz.command="$(XZ)"
> +archive-config-tar.zst = -c tar.tar.zst.command="$(ZSTD)"
> +
>  quiet_cmd_archive = ARCHIVE $@
> -      cmd_archive = git -C $(srctree) archive \
> +      cmd_archive = git -C $(srctree) $(archive-config-tar$(suffix $@)) archive \
>                      --output=$$(realpath $@) --prefix=$(basename $@)/ $(archive-args)
>  
>  # Linux source tarball
>  # ---------------------------------------------------------------------------
>  
> -targets += linux.tar
> -linux.tar: archive-args = $$(cat $<)
> -linux.tar: .tmp_HEAD FORCE
> +linux-tarballs := $(addprefix linux, .tar.gz)

Is there any reason not to allow other compression formats for linux
like you do for perf?

> +
> +targets += $(linux-tarballs)
> +$(linux-tarballs): archive-args = $$(cat $<)
> +$(linux-tarballs): .tmp_HEAD FORCE
>  	$(call if_changed,archive)
>  
>  # rpm-pkg
> @@ -185,9 +192,12 @@ perf-archive-args = --add-file=$$(realpath $(word 2, $^)) \
>  	--add-file=$$(realpath $(word 3, $^)) \
>  	$$(cat $(word 2, $^))^{tree} $$(cat $<)
>  
> -targets += perf-$(KERNELVERSION).tar
> -perf-$(KERNELVERSION).tar: archive-args = $(perf-archive-args)
> -perf-$(KERNELVERSION).tar: tools/perf/MANIFEST .tmp_perf/HEAD .tmp_perf/PERF-VERSION-FILE FORCE
> +
> +perf-tarballs := $(addprefix perf-$(KERNELVERSION), .tar .tar.gz .tar.bz2 .tar.xz .tar.zst)
> +
> +targets += $(perf-tarballs)
> +$(perf-tarballs): archive-args = $(perf-archive-args)
> +$(perf-tarballs): tools/perf/MANIFEST .tmp_perf/HEAD .tmp_perf/PERF-VERSION-FILE FORCE
>  	$(call if_changed,archive)
>  
>  PHONY += perf-tar-src-pkg
> -- 
> 2.37.2
>