linux-kernel - Re: clang-nightly: vdso/compat_gettimeofday.h:152:15: error: instruction variant requires ARMv6 or later

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20231204223317.GA2053629@dev-arch.thelio-3990X>
Date:   Mon, 4 Dec 2023 15:33:17 -0700
From:   Nathan Chancellor <nathan@...nel.org>
To:     Naresh Kamboju <naresh.kamboju@...aro.org>
Cc:     clang-built-linux <llvm@...ts.linux.dev>,
        Linux ARM <linux-arm-kernel@...ts.infradead.org>,
        open list <linux-kernel@...r.kernel.org>,
        Linux Regressions <regressions@...ts.linux.dev>,
        lkft-triage@...ts.linaro.org,
        Russell King - ARM Linux <linux@...linux.org.uk>,
        Arnd Bergmann <arnd@...db.de>,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        Sylvestre Ledru <sylvestre@...ian.org>
Subject: Re: clang-nightly: vdso/compat_gettimeofday.h:152:15: error:
 instruction variant requires ARMv6 or later

On Mon, Dec 04, 2023 at 11:13:04AM -0700, Nathan Chancellor wrote:
> Hi Naresh,
> 
> On Mon, Dec 04, 2023 at 05:33:26PM +0530, Naresh Kamboju wrote:
> > Following build errors noticed on Linux next-20231204 tag with clang-nightly
> > for arm and arm64.
> > 
> > ## Test Regressions (compared to next-20231201)
> > * arm64, build
> >   - clang-nightly-defconfig
> >   - clang-nightly-defconfig-40bc7ee5
> >   - clang-nightly-lkftconfig
> >   - clang-nightly-lkftconfig-kselftest
> > 
> > * arm, build
> >   - clang-nightly-allnoconfig
> >   - clang-nightly-axm55xx_defconfig
> >   - clang-nightly-bcm2835_defconfig
> >   - clang-nightly-clps711x_defconfig
> >   - clang-nightly-defconfig
> >   - clang-nightly-exynos_defconfig
> >   - clang-nightly-imx_v6_v7_defconfig
> >   - clang-nightly-keystone_defconfig
> >   - clang-nightly-lkftconfig
> >   - clang-nightly-lkftconfig-kselftest
> >   - clang-nightly-omap2plus_defconfig
> >   - clang-nightly-pxa910_defconfig
> >   - clang-nightly-s3c6400_defconfig
> >   - clang-nightly-s5pv210_defconfig
> >   - clang-nightly-sama5_defconfig
> >   - clang-nightly-shmobile_defconfig
> >   - clang-nightly-tinyconfig
> >   - clang-nightly-u8500_defconfig
> >   - clang-nightly-vexpress_defconfig
> > 
> > 
> > Reported-by: Linux Kernel Functional Testing <lkft@...aro.org>
> > 
> > 
> > Build log on arm64:
> > ---------
> > In file included from lib/vdso/gettimeofday.c:5:
> > In file included from include/vdso/datapage.h:135:
> > arch/arm64/include/asm/vdso/compat_gettimeofday.h:152:15: error:
> > instruction variant requires ARMv6 or later
> >   152 |         asm volatile("mov %0, %1" : "=r"(ret) : "r"(_vdso_data));
> >       |                      ^
> > <inline asm>:1:2: note: instantiated into assembly here
> >     1 |         mov r4, r1
> >       |         ^
> > In file included from <built-in>:3:
> > lib/vdso/gettimeofday.c:139:3: error: invalid instruction
> >   139 |                 smp_rmb();
> >       |                 ^
> > 
> > Build log on arm:
> > ---------
> > In file included from arch/arm/vfp/vfpmodule.c:23:
> > arch/arm/include/asm/cp15.h:101:2: error: instruction requires: data-barriers
> >   101 |         isb();
> >       |         ^
> 
> This is caused by a change to Debian's LLVM that changes the internal
> defaults of the arm-linux-gnueabi and arm-linux-gnueabihf tuples:
> 
> https://salsa.debian.org/pkg-llvm-team/llvm-toolchain/-/commit/907baf024b9a5a1626893d9e731b6c79ccf45c87
> 
> We use arm-linux-gnueabi for the kernel (see scripts/Makefile.clang) so
> now we have a hardcoded armv5te CPU, even if we are building for armv7
> or such.
> 
> I am still investigating into what (if anything) can be done to resolve
> this on the kernel side. We could potentially revert commit
> ddc72c9659b5 ("kbuild: clang: do not use CROSS_COMPILE for target
> triple") but I am not sure that will save us from that change, as
> tuxmake's CROSS_COMPILE=arm-linux-gnueabihf will cause us to have an
> armv7 CPU even though we may not be building for armv7.

Okay, this is a pretty awful situation the more I look into it :(

The arm64 compat vDSO build is easy enough to fix because we require use
of the integrated assembler, which means we can add '-mcpu=generic' (the
default in LLVM for those files based on my debugging) to those files
and be done with it:

diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
index 1f911a76c5af..5f5cb722cfc2 100644
--- a/arch/arm64/kernel/vdso32/Makefile
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -9,6 +9,10 @@ include $(srctree)/lib/vdso/Makefile
 ifeq ($(CONFIG_CC_IS_CLANG), y)
 CC_COMPAT ?= $(CC)
 CC_COMPAT += --target=arm-linux-gnueabi
+# Some distributions (such as Debian) change the default CPU for the
+# arm-linux-gnueabi target triple, which can break the build. Explicitly set
+# the CPU to generic, which is the default for Linux in LLVM.
+CC_COMPAT += -mcpu=generic
 else
 CC_COMPAT ?= $(CROSS_COMPILE_COMPAT)gcc
 endif

The failures for all the ARCH=arm configurations appear to be much more
difficult to fix because the default CPU value changes based on the
'-march' value, which basically means that we would have to hardcode
LLVM's default CPU logic into the kernel's Makefile, which is just not
maintainable in my opinion. Just doing a multi_v7_defconfig build of
arch/arm/ shows the value returned from ARM::getARMCPUForArch() in
llvm/lib/TargetParser/ARMTargetParser.cpp can vary between "arm7tdmi" or
"generic". Supplying '-mcpu=generic' explicitly won't work with
LLVM_IAS=0 because GNU as does not support it and clang just happily
passes it along, even though it does not do that in the implicit default
case.

Sylvestre, I strongly believe you should consider reverting that change
or give us some compiler flag that allows us to fallback to upstream
LLVM's default CPU selection logic. I think that hardcoding Debian's
architecture defintions based on the target triple into the compiler
could cause issues for other projects as well. For example,
'--target=arm-linux-gnueabi -march=armv7-a' won't actually target ARMv7:

  $ echo 'int main(void) { asm("dsb"); return 0; }' | \
        clang --target=arm-linux-gnueabi -march=armv7-a \
        -x c -c -o /dev/null -v -
  ...
   "/usr/bin/clang-17" -cc1 -triple armv7-unknown-linux-gnueabi ...
  ...

vs.

  $ echo 'int main(void) { asm("dsb"); return 0; }' | \
        clang --target=arm-linux-gnueabi -march=armv7-a \
        -x c -c -o /dev/null -v -
  ...
  "<prefix>/bin/clang-18" -cc1 -triple armv5e-unknown-linux-gnueabi ...
  ...

Cheers,
Nathan