linux-kernel - Re: [PATCH 2/4] kbuild: deb-pkg: create source package without cleaning

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CA+icZUWTdAihUVSjEgV2BVOVCktUYccd--9y3bv18PrJDX5nhw@mail.gmail.com>
Date:   Sun, 29 Jan 2023 00:30:40 +0100
From:   Sedat Dilek <sedat.dilek@...il.com>
To:     Masahiro Yamada <masahiroy@...nel.org>
Cc:     linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
        Nathan Chancellor <nathan@...nel.org>,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        Nicolas Schier <nicolas@...sle.eu>, Tom Rix <trix@...hat.com>,
        llvm@...ts.linux.dev
Subject: Re: [PATCH 2/4] kbuild: deb-pkg: create source package without cleaning

On Sat, Jan 28, 2023 at 6:40 PM Masahiro Yamada <masahiroy@...nel.org> wrote:
>
> If you run 'make deb-pkg', all objects are lost due to 'make clean',
> which makes the incremental builds impossible.
>
> Instead of cleaning, pass the exclude list to tar's --exclude-from
> option.
>
> Previously, *.diff.gz contained some check-in files such as
> .clang-format, .cocciconfig.
>
> With this commit, *.diff.gz will only contain the .config and debian/.
> The other source files will go into the tarball.
>

Thanks for the patch.

While at this...

...why not switch over to Debian's packaging default XZ compressor:
*.orig.xz and *.diff.xz (or *.debian.tar.xz)?

EXAMPLE binutils:

DSC: http://deb.debian.org/debian/pool/main/b/binutils/binutils_2.40-2.dsc
TAR: http://deb.debian.org/debian/pool/main/b/binutils/binutils_2.40.orig.tar.xz
DIFF: http://deb.debian.org/debian/pool/main/b/binutils/binutils_2.40-2.debian.tar.xz

-Sedat-

[1] https://packages.debian.org/sid/binutils

> Signed-off-by: Masahiro Yamada <masahiroy@...nel.org>
> ---
>
>  scripts/Makefile.package | 27 ++++++++++++++++++++++-----
>  scripts/package/mkdebian | 27 +++++++++++++++++++++++++++
>  2 files changed, 49 insertions(+), 5 deletions(-)
>
> diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> index dfbf40454a99..cb135c99a273 100644
> --- a/scripts/Makefile.package
> +++ b/scripts/Makefile.package
> @@ -50,6 +50,21 @@ fi ; \
>  tar -I $(KGZIP) -c $(RCS_TAR_IGNORE) -f $(2).tar.gz \
>         --transform 's:^:$(2)/:S' $(TAR_CONTENT) $(3)
>
> +# Source Tarball
> +# ---------------------------------------------------------------------------
> +
> +quiet_cmd_exclude_list = GEN     $@
> +      cmd_exclude_list = $(srctree)/scripts/gen-exclude.py --prefix=./ --rootdir=$(srctree) > $@; echo "./$@" >> $@
> +
> +.exclude-list: FORCE
> +       $(call cmd,exclude_list)
> +
> +quiet_cmd_tar = TAR     $@
> +      cmd_tar = tar -I $(KGZIP) -c -f $@ -C $(srctree) --exclude-from=$< --exclude=./$@ --transform 's:^\.:linux:S' .
> +
> +%.tar.gz: .exclude-list
> +       $(call cmd,tar)
> +
>  # rpm-pkg
>  # ---------------------------------------------------------------------------
>  PHONY += rpm-pkg
> @@ -81,12 +96,11 @@ binrpm-pkg:
>
>  PHONY += deb-pkg
>  deb-pkg:
> -       $(MAKE) clean
>         $(CONFIG_SHELL) $(srctree)/scripts/package/mkdebian
> -       $(call cmd,src_tar,$(KDEB_SOURCENAME))
> -       origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
> -               mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
> -       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) --source-option=-sP -i.git -us -uc
> +       $(Q)origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
> +               $(MAKE) -f $(srctree)/scripts/Makefile.package ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
> +       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) \
> +               --build=source,binary --source-option=-sP -nc -us -uc
>
>  PHONY += bindeb-pkg
>  bindeb-pkg:
> @@ -174,4 +188,7 @@ help:
>         @echo '  perf-tarxz-src-pkg  - Build $(perf-tar).tar.xz source tarball'
>         @echo '  perf-tarzst-src-pkg - Build $(perf-tar).tar.zst source tarball'
>
> +PHONY += FORCE
> +FORCE:
> +
>  .PHONY: $(PHONY)
> diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
> index c3bbef7a6754..12c057ffbe6e 100755
> --- a/scripts/package/mkdebian
> +++ b/scripts/package/mkdebian
> @@ -84,6 +84,8 @@ set_debarch() {
>         fi
>  }
>
> +rm -rf debian
> +
>  # Some variables and settings used throughout the script
>  version=$KERNELRELEASE
>  if [ -n "$KDEB_PKGVERSION" ]; then
> @@ -135,6 +137,31 @@ fi
>  mkdir -p debian/source/
>  echo "1.0" > debian/source/format
>
> +cat<<'EOF' > debian/source/local-options
> +#
> +# Ugly: ignore anything except .config or debian/
> +# (is there a cleaner way to do this?)
> +#
> +diff-ignore
> +
> +extend-diff-ignore = ^[^.d]
> +
> +extend-diff-ignore = ^\.[^c]
> +extend-diff-ignore = ^\.c($|[^o])
> +extend-diff-ignore = ^\.co($|[^n])
> +extend-diff-ignore = ^\.con($|[^f])
> +extend-diff-ignore = ^\.conf($|[^i])
> +extend-diff-ignore = ^\.confi($|[^g])
> +extend-diff-ignore = ^\.config.
> +
> +extend-diff-ignore = ^d($|[^e])
> +extend-diff-ignore = ^de($|[^b])
> +extend-diff-ignore = ^deb($|[^i])
> +extend-diff-ignore = ^debi($|[^a])
> +extend-diff-ignore = ^debia($|[^n])
> +extend-diff-ignore = ^debian[^/]
> +EOF
> +
>  echo $debarch > debian/arch
>  extra_build_depends=", $(if_enabled_echo CONFIG_UNWINDER_ORC libelf-dev:native)"
>  extra_build_depends="$extra_build_depends, $(if_enabled_echo CONFIG_SYSTEM_TRUSTED_KEYRING libssl-dev:native)"
> --
> 2.34.1
>