Message-ID: <CAK7LNARYNqjcMkdnaY2oAkxttFTtTEgJ9VuOZOn0i4AuXp-How@mail.gmail.com>
Date: Wed, 17 Jul 2024 17:34:09 +0900
From: Masahiro Yamada <masahiroy@...nel.org>
To: Nathan Chancellor <nathan@...nel.org>
Cc: Thomas Weißschuh <linux@...ssschuh.net>, 
	Nicolas Schier <nicolas@...sle.eu>, 
	"Jan Alexander Steffens (heftig)" <heftig@...hlinux.org>, linux-kernel@...r.kernel.org, 
	linux-kbuild@...r.kernel.org
Subject: Re: [PATCH v6] kbuild: add script and target to generate pacman package

On Wed, Jul 17, 2024 at 10:15 AM Nathan Chancellor <nathan@...nel.org> wrote:
>
> Hi Thomas,
>
> On Tue, Jul 16, 2024 at 07:52:14PM +0200, Thomas Weißschuh wrote:
> > pacman is the package manager used by Arch Linux and its derivates.
> > Creating native packages from the kernel tree has multiple advantages:
> >
> > * The package triggers the correct hooks for initramfs generation and
> >   bootloader configuration
> > * Uninstallation is complete and also invokes the relevant hooks
> > * New UAPI headers can be installed without any manual bookkeeping
> >
> > The PKGBUILD file is a simplified version of the one used for the
> > downstream Arch Linux "linux" package.
> > Extra steps that should not be necessary for a development kernel have
> > been removed and an UAPI header package has been added.
> >
> > Signed-off-by: Thomas Weißschuh <linux@...ssschuh.net>
> > Reviewed-by: Nathan Chancellor <nathan@...nel.org>
> > Tested-by: Nathan Chancellor <nathan@...nel.org>
> > Reviewed-by: Nicolas Schier <nicolas@...sle.eu>
> > ---
> > Changes in v6:
> > - Drop reference to srctree/Makefile
> > - Drop $(realpath $(srctree))
> > - Make use of the fact that $(objtree) is always "."
> > - Align coding style to kernel and drop vim config line
> > - Drop indirection through `$MAKE run-command`
> > - Unify shell variable syntax to "${var}"
> > - Add explanations to custom variables
> > - Add makedepends
> > - Link to v5: https://lore.kernel.org/r/20240714-kbuild-pacman-pkg-v5-1-0598460bc918@weissschuh.net
> >
> > Changes in v5:
> > - Rebase onto kbuild/for-next
> > - Use new path to build-version script (from kbuild/for-next)
> > - Ensure submake jobserver delegation works
> > - Simplify $modulesdir/pkgbase file creation
> > - Add Reviewed-by from Nicolas
> > - Link to v4: https://lore.kernel.org/r/20240710-kbuild-pacman-pkg-v4-1-507bb5b79b2a@weissschuh.net
> >
> > Changes in v4:
> > - Update MRPROPER_FILES
> > - Unify shell variable syntax
> > - Link to v3: https://lore.kernel.org/r/20240708-kbuild-pacman-pkg-v3-1-885df3cbc740@weissschuh.net
> >
> > Changes in v3:
> > - Enforce matching architectures for installation
> > - Add Reviewed-by and Tested-by from Nathan
> > - Link to v2: https://lore.kernel.org/r/20240706-kbuild-pacman-pkg-v2-1-613422a03a7a@weissschuh.net
> >
> > Changes in v2:
> > - Replace ${MAKE} with $MAKE for consistency with other variables
> > - Use $MAKE for "-s image_name"
> > - Avoid permission warnings from build directory
> > - Clarify reason for /build symlink removal
> > - Install System.map and config
> > - Install dtbs where available
> > - Allow cross-build through arch=any
> > - Sort Contributor/Maintainer chronologically
> > - Disable some unneeded makepkg options
> > - Use DEPMOD=true for consistency with rpm-package
> > - Link to v1: https://lore.kernel.org/r/20240704-kbuild-pacman-pkg-v1-1-ac2f63f5fa7b@weissschuh.net
> > ---
> >  .gitignore               |  6 +++
> >  Makefile                 |  2 +-
> >  scripts/Makefile.package | 14 +++++++
> >  scripts/package/PKGBUILD | 99 ++++++++++++++++++++++++++++++++++++++++++++++++
> >  4 files changed, 120 insertions(+), 1 deletion(-)
> >
> > diff --git a/.gitignore b/.gitignore
> > index c59dc60ba62e..7902adf4f7f1 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -92,6 +92,12 @@ modules.order
> >  #
> >  /tar-install/
> >
> > +#
> > +# pacman files (make pacman-pkg)
> > +#
> > +/PKGBUILD
> > +/pacman/
> > +
> >  #
> >  # We don't want to ignore the following even if they are dot-files
> >  #
> > diff --git a/Makefile b/Makefile
> > index 7372ea45ed3f..768d3dc107f8 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -1481,7 +1481,7 @@ CLEAN_FILES += vmlinux.symvers modules-only.symvers \
> >  # Directories & files removed with 'make mrproper'
> >  MRPROPER_FILES += include/config include/generated          \
> >                 arch/$(SRCARCH)/include/generated .objdiff \
> > -               debian snap tar-install \
> > +               debian snap tar-install PKGBUILD pacman \
> >                 .config .config.old .version \
> >                 Module.symvers \
> >                 certs/signing_key.pem \
> > diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> > index bf016af8bf8a..0aaa0832279c 100644
> > --- a/scripts/Makefile.package
> > +++ b/scripts/Makefile.package
> > @@ -141,6 +141,19 @@ snap-pkg:
> >       cd $(objtree)/snap && \
> >       snapcraft --target-arch=$(UTS_MACHINE)
> >
> > +# pacman-pkg
> > +# ---------------------------------------------------------------------------
> > +
> > +PHONY += pacman-pkg
> > +pacman-pkg:
> > +     @ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
> > +     +objtree="$(realpath $(objtree))" \
> > +             BUILDDIR=pacman \
> > +             CARCH="$(UTS_MACHINE)" \
> > +             KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
> > +             KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
> > +             makepkg
> > +
> >  # dir-pkg tar*-pkg - tarball targets
> >  # ---------------------------------------------------------------------------
> >
> > @@ -221,6 +234,7 @@ help:
> >       @echo '  bindeb-pkg          - Build only the binary kernel deb package'
> >       @echo '  snap-pkg            - Build only the binary kernel snap package'
> >       @echo '                        (will connect to external hosts)'
> > +     @echo '  pacman-pkg          - Build only the binary kernel pacman package'
> >       @echo '  dir-pkg             - Build the kernel as a plain directory structure'
> >       @echo '  tar-pkg             - Build the kernel as an uncompressed tarball'
> >       @echo '  targz-pkg           - Build the kernel as a gzip compressed tarball'
> > diff --git a/scripts/package/PKGBUILD b/scripts/package/PKGBUILD
> > new file mode 100644
> > index 000000000000..eb3957fad915
> > --- /dev/null
> > +++ b/scripts/package/PKGBUILD
> > @@ -0,0 +1,99 @@
> > +# SPDX-License-Identifier: GPL-2.0-only
> > +# Maintainer: Thomas Weißschuh <linux@...ssschuh.net>
> > +# Contributor: Jan Alexander Steffens (heftig) <heftig@...hlinux.org>
> > +
> > +pkgbase=linux-upstream
> > +pkgname=("${pkgbase}" "${pkgbase}-headers" "${pkgbase}-api-headers")
> > +pkgver="${KERNELRELEASE//-/_}"
> > +# The PKGBUILD is evaluated multiple times.
> > +# Running scripts/build-version from here would introduce inconsistencies.
> > +pkgrel="${KBUILD_REVISION}"
> > +pkgdesc='Linux'
> > +url='https://www.kernel.org/'
> > +# Enable flexible cross-compilation
> > +arch=(${CARCH})
> > +license=(GPL-2.0-only)
> > +makedepends=(
> > +     base-devel
> > +     bc
> > +     cpio
> > +     gettext
> > +     libelf
> > +     openssl
> > +     pahole
> > +     perl
> > +     python
> > +     rsync
> > +     tar
> > +)
> > +options=(!debug !strip !buildflags !makeflags)
> > +
> > +build() {
> > +     # MAKEFLAGS from makepkg.conf override the ones inherited from kbuild.
> > +     # Bypass this override with a custom variable.
> > +     export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> > +     cd "${objtree}"
> > +
> > +     # makepkg does a "chmod a-srw", triggering warnings during kbuild
> > +     chmod 0755 "${pkgdirbase}" || true
> > +
> > +     ${MAKE}
> > +}
> > +
> > +package_linux-upstream() {
> > +     pkgdesc="The ${pkgdesc} kernel and modules"
> > +
> > +     export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> > +     cd "${objtree}"
> > +     local modulesdir="${pkgdir}/usr/${MODLIB}"
> > +
> > +     echo "Installing boot image..."
> > +     # systemd expects to find the kernel here to allow hibernation
> > +     # https://github.com/systemd/systemd/commit/edda44605f06a41fb86b7ab8128dcf99161d2344
> > +     install -Dm644 "$(${MAKE} -s image_name)" "${modulesdir}/vmlinuz"
> > +
> > +     # Used by mkinitcpio to name the kernel
> > +     echo "${pkgbase}" > "${modulesdir}/pkgbase"
> > +
> > +     echo "Installing modules..."
> > +     ${MAKE} INSTALL_MOD_PATH="${pkgdir}/usr" INSTALL_MOD_STRIP=1 \
> > +             DEPMOD=true modules_install
> > +
> > +     if [ -d "${srctree}/arch/${SRCARCH}/boot/dts" ]; then
>
> Does this reference to srctree...


'srctree' is exported by the top Makefile.


This is based on the assumption that we always run
'make pacman-pkg', and we never do 'makepkg' directly.


Do you mean PKGBUILD should be self-contained
so that 'makepkg' works from the command line?



Debian and RPM work that way because we can
build a source package, then we can run
dpkg-buildpackage / rpmbuild later.



>
> > +             echo "Installing dtbs..."
> > +             ${MAKE} INSTALL_DTBS_PATH="${modulesdir}/dtb" dtbs_install
> > +     fi
> > +
> > +     # remove build link, will be part of -headers package
> > +     rm -f "${modulesdir}/build"
> > +}
> > +
> > +package_linux-upstream-headers() {
> > +     pkgdesc="Headers and scripts for building modules for the ${pkgdesc} kernel"
> > +
> > +     export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> > +     cd "${objtree}"
> > +     local builddir="${pkgdir}/usr/${MODLIB}/build"
> > +
> > +     echo "Installing build files..."
> > +     "${srctree}/scripts/package/install-extmod-build" "${builddir}"
>
> and this one still work with srctree no longer being passed to makepkg?
>
> I tried myself but it looks like this version of the patch might be
> broken?
>
> $ make -skj"$(nproc)" ARCH=x86_64 CROSS_COMPILE=x86_64-linux- O=/mnt/nvme/tmp/build/linux mrproper defconfig pacman-pkg
> find: ‘./pacman/linux-upstream/src/pacman/linux-upstream/pkg’: Permission denied
> ==> Making package: linux-upstream 6.10.0_rc7_00051_g818e9c998b04_dirty-1 (Tue 16 Jul 2024 06:08:29 PM MST)
> ==> Checking runtime dependencies...
> ==> Checking buildtime dependencies...
> ==> Retrieving sources...
> ==> Extracting sources...
> ==> Starting build()...
> chmod: cannot access 'pacman/linux-upstream/pkg': No such file or directory
> find: ‘./pacman/linux-upstream/src/pacman/linux-upstream/pkg’: Permission denied
> ==> Entering fakeroot environment...
> chmod: cannot access '/mnt/nvme/tmp/build/linux/pacman/linux-upstream/pkg': No such file or directory
> /usr/bin/fakeroot: line 178: 2633185 User defined signal 1   FAKEROOTKEY=$FAKEROOTKEY LD_LIBRARY_PATH="$PATHS" LD_PRELOAD="$FAKEROOT_LIB" "$@"
> make[4]: *** [scripts/Makefile.package:150: pacman-pkg] Error 138


Sorry, this is due to my suggestion, BUILDDIR=pacman.


Unfortunately, makepkg is tripped up by a relative path.

The 'pkg' directory is nested under the 'src' directory.

  pacman/linux-upstream/src/pacman/linux-upstream/pkg













> Cheers,
> Nathan
>
> > +     echo "Installing System.map and config..."
> > +     cp System.map "${builddir}/System.map"
> > +     cp .config "${builddir}/.config"
> > +
> > +     echo "Adding symlink..."
> > +     mkdir -p "${pkgdir}/usr/src"
> > +     ln -sr "${builddir}" "${pkgdir}/usr/src/${pkgbase}"
> > +}
> > +
> > +package_linux-upstream-api-headers() {
> > +     pkgdesc="Kernel headers sanitized for use in userspace"
> > +     provides=(linux-api-headers)
> > +     conflicts=(linux-api-headers)
> > +
> > +     export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> > +     cd "${objtree}"
> > +
> > +     ${MAKE} headers_install INSTALL_HDR_PATH="${pkgdir}/usr"
> > +}
> >
> > ---
> > base-commit: 818e9c998b04d6c69a510d5255a93d0e3b8d4993
> > change-id: 20240625-kbuild-pacman-pkg-b4f87e19d036
> >
> > Best regards,
> > --
> > Thomas Weißschuh <linux@...ssschuh.net>
> >
>


-- 
Best Regards
Masahiro Yamada

Powered by blists - more mailing lists