linux-kernel - Re: [PATCH 2/4] kbuild: deb-pkg: create source package without cleaning

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <CA+icZUUT2TpgCSrGZA_tLLq3LmYppOmyxh2o8V6w9B+t-J_CLQ@mail.gmail.com>
Date:   Sun, 29 Jan 2023 09:21:45 +0100
From:   Sedat Dilek <sedat.dilek@...il.com>
To:     Masahiro Yamada <masahiroy@...nel.org>
Cc:     linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
        Nathan Chancellor <nathan@...nel.org>,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        Nicolas Schier <nicolas@...sle.eu>, Tom Rix <trix@...hat.com>,
        llvm@...ts.linux.dev
Subject: Re: [PATCH 2/4] kbuild: deb-pkg: create source package without cleaning

On Sun, Jan 29, 2023 at 9:08 AM Masahiro Yamada <masahiroy@...nel.org> wrote:
>
> On Sun, Jan 29, 2023 at 8:31 AM Sedat Dilek <sedat.dilek@...il.com> wrote:
> >
> > On Sat, Jan 28, 2023 at 6:40 PM Masahiro Yamada <masahiroy@...nel.org> wrote:
> > >
> > > If you run 'make deb-pkg', all objects are lost due to 'make clean',
> > > which makes the incremental builds impossible.
> > >
> > > Instead of cleaning, pass the exclude list to tar's --exclude-from
> > > option.
> > >
> > > Previously, *.diff.gz contained some check-in files such as
> > > .clang-format, .cocciconfig.
> > >
> > > With this commit, *.diff.gz will only contain the .config and debian/.
> > > The other source files will go into the tarball.
> > >
> >
> > Thanks for the patch.
> >
> > While at this...
> >
> > ...why not switch over to Debian's packaging default XZ compressor:
> > *.orig.xz and *.diff.xz (or *.debian.tar.xz)?
>
>
> Does debian support *.diff.xz?
> I do not think so.
>
> *.debian.tar.xz requires "Format: 3.0 (quilt)" migration.
>
>
> See scripts/package/mkdebian.
> We use "Format: 1.0", which only supports gzip.
>

Ah OK.

>
> It is true the Debian kernel uses "Format: 3.0 (quilt)",
> but I do not think it will fit to the upstream kernel.
>
> Rather, I want to use the Native format since I do not
> see much sense in the *.orig.tar.gz / *.diff.gz split
> for the upstream project.
>
>
> My plan is to stop cleaning first,
> then change the source format if it is desirable.
>

Yes, makes sense.

-Sedat-

>
>
>
>
> >
> > EXAMPLE binutils:
> >
> > DSC: http://deb.debian.org/debian/pool/main/b/binutils/binutils_2.40-2.dsc
>
> This is "Format: 3.0 (quilt)".
>
>
> > TAR: http://deb.debian.org/debian/pool/main/b/binutils/binutils_2.40.orig.tar.xz
> > DIFF: http://deb.debian.org/debian/pool/main/b/binutils/binutils_2.40-2.debian.tar.xz
>
>
> This is not a diff.  It is a tarball of the debian/ directory.
> Real diffs are stored in debian/patches/.
>
>
>
>
>
> >
> > -Sedat-
> >
> > [1] https://packages.debian.org/sid/binutils
> >
> > > Signed-off-by: Masahiro Yamada <masahiroy@...nel.org>
> > > ---
> > >
> > >  scripts/Makefile.package | 27 ++++++++++++++++++++++-----
> > >  scripts/package/mkdebian | 27 +++++++++++++++++++++++++++
> > >  2 files changed, 49 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> > > index dfbf40454a99..cb135c99a273 100644
> > > --- a/scripts/Makefile.package
> > > +++ b/scripts/Makefile.package
> > > @@ -50,6 +50,21 @@ fi ; \
> > >  tar -I $(KGZIP) -c $(RCS_TAR_IGNORE) -f $(2).tar.gz \
> > >         --transform 's:^:$(2)/:S' $(TAR_CONTENT) $(3)
> > >
> > > +# Source Tarball
> > > +# ---------------------------------------------------------------------------
> > > +
> > > +quiet_cmd_exclude_list = GEN     $@
> > > +      cmd_exclude_list = $(srctree)/scripts/gen-exclude.py --prefix=./ --rootdir=$(srctree) > $@; echo "./$@" >> $@
> > > +
> > > +.exclude-list: FORCE
> > > +       $(call cmd,exclude_list)
> > > +
> > > +quiet_cmd_tar = TAR     $@
> > > +      cmd_tar = tar -I $(KGZIP) -c -f $@ -C $(srctree) --exclude-from=$< --exclude=./$@ --transform 's:^\.:linux:S' .
> > > +
> > > +%.tar.gz: .exclude-list
> > > +       $(call cmd,tar)
> > > +
> > >  # rpm-pkg
> > >  # ---------------------------------------------------------------------------
> > >  PHONY += rpm-pkg
> > > @@ -81,12 +96,11 @@ binrpm-pkg:
> > >
> > >  PHONY += deb-pkg
> > >  deb-pkg:
> > > -       $(MAKE) clean
> > >         $(CONFIG_SHELL) $(srctree)/scripts/package/mkdebian
> > > -       $(call cmd,src_tar,$(KDEB_SOURCENAME))
> > > -       origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
> > > -               mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
> > > -       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) --source-option=-sP -i.git -us -uc
> > > +       $(Q)origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
> > > +               $(MAKE) -f $(srctree)/scripts/Makefile.package ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
> > > +       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) \
> > > +               --build=source,binary --source-option=-sP -nc -us -uc
> > >
> > >  PHONY += bindeb-pkg
> > >  bindeb-pkg:
> > > @@ -174,4 +188,7 @@ help:
> > >         @echo '  perf-tarxz-src-pkg  - Build $(perf-tar).tar.xz source tarball'
> > >         @echo '  perf-tarzst-src-pkg - Build $(perf-tar).tar.zst source tarball'
> > >
> > > +PHONY += FORCE
> > > +FORCE:
> > > +
> > >  .PHONY: $(PHONY)
> > > diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
> > > index c3bbef7a6754..12c057ffbe6e 100755
> > > --- a/scripts/package/mkdebian
> > > +++ b/scripts/package/mkdebian
> > > @@ -84,6 +84,8 @@ set_debarch() {
> > >         fi
> > >  }
> > >
> > > +rm -rf debian
> > > +
> > >  # Some variables and settings used throughout the script
> > >  version=$KERNELRELEASE
> > >  if [ -n "$KDEB_PKGVERSION" ]; then
> > > @@ -135,6 +137,31 @@ fi
> > >  mkdir -p debian/source/
> > >  echo "1.0" > debian/source/format
> > >
> > > +cat<<'EOF' > debian/source/local-options
> > > +#
> > > +# Ugly: ignore anything except .config or debian/
> > > +# (is there a cleaner way to do this?)
> > > +#
> > > +diff-ignore
> > > +
> > > +extend-diff-ignore = ^[^.d]
> > > +
> > > +extend-diff-ignore = ^\.[^c]
> > > +extend-diff-ignore = ^\.c($|[^o])
> > > +extend-diff-ignore = ^\.co($|[^n])
> > > +extend-diff-ignore = ^\.con($|[^f])
> > > +extend-diff-ignore = ^\.conf($|[^i])
> > > +extend-diff-ignore = ^\.confi($|[^g])
> > > +extend-diff-ignore = ^\.config.
> > > +
> > > +extend-diff-ignore = ^d($|[^e])
> > > +extend-diff-ignore = ^de($|[^b])
> > > +extend-diff-ignore = ^deb($|[^i])
> > > +extend-diff-ignore = ^debi($|[^a])
> > > +extend-diff-ignore = ^debia($|[^n])
> > > +extend-diff-ignore = ^debian[^/]
> > > +EOF
> > > +
> > >  echo $debarch > debian/arch
> > >  extra_build_depends=", $(if_enabled_echo CONFIG_UNWINDER_ORC libelf-dev:native)"
> > >  extra_build_depends="$extra_build_depends, $(if_enabled_echo CONFIG_SYSTEM_TRUSTED_KEYRING libssl-dev:native)"
> > > --
> > > 2.34.1
> > >
>
>
>
> --
> Best Regards
> Masahiro Yamada