Message-ID: <CAK7LNAT5u6trSDUCC4Cvg1CsHZhwnDtTFcRaZZA=Qvt8yNO2iA@mail.gmail.com>
Date: Sat, 16 Nov 2024 17:03:25 +0900
From: Masahiro Yamada <masahiroy@...nel.org>
To: Charlie Jenkins <charlie@...osinc.com>
Cc: Ron Economos <re@...z.net>, Nicolas Schier <nicolas@...sle.eu>, linux-kbuild@...r.kernel.org, 
	linux-kernel@...r.kernel.org, Kees Cook <kees@...nel.org>, 
	Nathan Chancellor <nathan@...nel.org>, Ben Hutchings <ben@...adent.org.uk>
Subject: Re: [PATCH 4/4] kbuild: cross-compile linux-headers package when possible

On Fri, Nov 15, 2024 at 5:57 AM Charlie Jenkins <charlie@...osinc.com> wrote:
>
> On Thu, Oct 17, 2024 at 12:34:49PM -0700, Ron Economos wrote:
> > On 10/17/24 12:24 PM, Nicolas Schier wrote:
> > > On Thu, Oct 17, 2024 at 07:45:57AM -0700 Ron Economos wrote:
> > > > On 7/27/24 12:42 AM, Masahiro Yamada wrote:
> > > > > A long standing issue in the upstream kernel packaging is that the
> > > > > linux-headers package is not cross-compiled.
> > > > >
> > > > > For example, you can cross-build Debian packages for arm64 by running
> > > > > the following command:
> > > > >
> > > > >     $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg
> > > > >
> > > > > However, the generated linux-headers-*_arm64.deb is useless because the
> > > > > host programs in it were built for your build machine architecture
> > > > > (likely x86), not arm64.
> > > > >
> > > > > The Debian kernel maintains its own Makefiles to cross-compile host
> > > > > tools without relying on Kbuild. [1]
> > > > >
> > > > > Instead of adding such full custom Makefiles, this commit adds a small
> > > > > piece of code to cross-compile host programs located under the scripts/
> > > > > directory.
> > > > >
> > > > > A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it
> > > > > would also cross-compile scripts/basic/fixdep, which needs to be native
> > > > > to process the if_changed_dep macro. (This approach may work under some
> > > > > circumstances; you can execute foreign architecture programs with the
> > > > > help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC,
> > > > > but it would require installing QEMU and libc for that architecture.)
> > > > >
> > > > > A trick is to use the external module build (KBUILD_EXTMOD=), which
> > > > > does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link
> > > > > userspace programs (CONFIG_CC_CAN_LINK=y).
> > > > >
> > > > > There are known limitations:
> > > > >
> > > > >    - GCC plugins
> > > > >
> > > > >      It would possible to rebuild GCC plugins for the target architecture
> > > > >      by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages
> > > > >      installed, but gcc on the installed system emits
> > > > >      "cc1: error: incompatible gcc/plugin versions". I did not find a
> > > > >      solution for this because 'gcc' on a foreign architecture is a
> > > > >      different compiler after all.
> > > > >
> > > > >    - objtool and resolve_btfids
> > > > >
> > > > >      These are built by the tools build system. They are not covered by
> > > > >      the current solution.
> > > > >
> > > > > I only tested this with Debian, but it should work for other package
> > > > > systems as well.
> > > > >
> > > > > [1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586
> > > > >
> > > > > Signed-off-by: Masahiro Yamada <masahiroy@...nel.org>
> > > > > ---
> > > > >
> > > > >    scripts/package/install-extmod-build | 34 ++++++++++++++++++++++++++++
> > > > >    1 file changed, 34 insertions(+)
> > > > >
> > > > > diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build
> > > > > index cc335945dfbc..0b56d3d7b48f 100755
> > > > > --- a/scripts/package/install-extmod-build
> > > > > +++ b/scripts/package/install-extmod-build
> > > > > @@ -43,4 +43,38 @@ mkdir -p "${destdir}"
> > > > >         fi
> > > > >    } | tar -c -f - -T - | tar -xf - -C "${destdir}"
> > > > > +# When ${CC} and ${HOSTCC} differ, we are likely cross-compiling. Rebuild host
> > > > > +# programs using ${CC}. This assumes CC=${CROSS_COMPILE}gcc, which is usually
> > > > > +# the case for package building. It does not cross-compile when CC=clang.
> > > > > +#
> > > > > +# This caters to host programs that participate in Kbuild. objtool and
> > > > > +# resolve_btfids are out of scope.
> > > > > +if [ "${CC}" != "${HOSTCC}" ] && is_enabled CONFIG_CC_CAN_LINK; then
> > > > > +       echo "Rebuilding host programs with ${CC}..."
> > > > > +
> > > > > +       cat <<-'EOF' >  "${destdir}/Kbuild"
> > > > > +       subdir-y := scripts
> > > > > +       EOF
> > > > > +
> > > > > +       # HOSTCXX is not overridden. The C++ compiler is used to build:
> > > > > +       # - scripts/kconfig/qconf, which is unneeded for external module builds
> > > > > +       # - GCC plugins, which will not work on the installed system even with
> > > > > +       #   being rebuilt.
> > > > > +       #
> > > > > +       # Use the single-target build to avoid the modpost invocation, which
> > > > > +       # would overwrite Module.symvers.
> > > > > +       "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/
> > > > > +
> > > > > +       cat <<-'EOF' >  "${destdir}/scripts/Kbuild"
> > > > > +       subdir-y := basic
> > > > > +       hostprogs-always-y := mod/modpost
> > > > > +       mod/modpost-objs := $(addprefix mod/, modpost.o file2alias.o sumversion.o symsearch.o)
> > > > > +       EOF
> > > > > +
> > > > > +       # Run once again to rebuild scripts/basic/ and scripts/mod/modpost.
> > > > > +       "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/
> > > > > +
> > > > > +       rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild"
> > > > > +fi
> > > > > +
> > > > >    find "${destdir}" \( -name '.*.cmd' -o -name '*.o' \) -delete
> > > > This patch causes a build error when cross-compiling for RISC-V. I'm using
> > > > the cross-compiler from https://github.com/riscv-collab/riscv-gnu-toolchain.
> > > > When trying to build .debs with:
> > > >
> > > > make CROSS_COMPILE=riscv64-unknown-linux-gnu- ARCH=riscv INSTALL_MOD_STRIP=1
> > > > "KCFLAGS=-mtune=sifive-7-series" LOCALVERSION= bindeb-pkg
> > > >
> > > > I get the following error:
> > > >
> > > > Rebuilding host programs with riscv64-unknown-linux-gnu-gcc...
> > > >    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/genksyms.o
> > > >    YACC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/parse.tab.[ch]
> > > >    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/parse.tab.o
> > > >    LEX debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/lex.lex.c
> > > >    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/lex.lex.o
> > > >    HOSTLD debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/genksyms
> > > >    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/selinux/genheaders/genheaders
> > > >    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/selinux/mdp/mdp
> > > >    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/kallsyms
> > > >    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sorttable
> > > >    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/asn1_compiler
> > > >    HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file
> > > >
> > > > debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file.c:25:10:
> > > > fatal error: openssl/opensslv.h: No such file or directory
> > > >     25 | #include <openssl/opensslv.h>
> > > >        |          ^~~~~~~~~~~~~~~~~~~~
> > > > compilation terminated.
> > > I guess you have openssl/opensslv.h available on your system, do you?  (In
> > > Debian/Ubuntu package libssl-dev or similar)
> > >
> > > Can you natively build a kernel with a similar kernel config?
> > >
> > > Kind regards,
> > > Nicolas
> >
> > Yes, I have /usr/include/openssl/opensslv.h on my system. But that's the x86
> > version. The cross compiler can't use that.
>
> You'll need to add the package for cross-compilation. If you are using
> ubuntu and the ubuntu riscv64 toolchain, you can add the riscv64
> architecture `dpkg --add-architecture riscv64`, swap out your
> sources.list file to specify the architecture `sed -i 's/^deb/deb
> [arch=amd64]/' /etc/apt/sources.list`, add the riscv64 debs to your
> sources.list:
>
> deb [arch=riscv64] http://ports.ubuntu.com/ubuntu-ports jammy main restricted multiverse universe
> deb [arch=riscv64] http://ports.ubuntu.com/ubuntu-ports jammy-updates main
> deb [arch=riscv64] http://ports.ubuntu.com/ubuntu-ports jammy-security main
>
> Then `apt update` and `apt install libssl-dev:riscv64`. I imagine there
> is a similar procedure for other distros. If using a custom compiler,
> you'll need to copy over the installed headers to whatever location your
> compiler is looking for them.


Right, this build dependency is enforced since commit
e2c318225ac13083cdcb4780cdf5b90edaa8644d






>
> - Charlie
>
> >
> > A native build works fine.
> >
> > Ron
> >
> >



-- 
Best Regards
Masahiro Yamada

Powered by blists - more mailing lists