Date:   Sun, 31 Oct 2021 02:18:01 +0100
From:   Sedat Dilek <sedat.dilek@...il.com>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     Nick Desaulniers <ndesaulniers@...gle.com>,
        Nathan Chancellor <nathan@...nel.org>,
        Masahiro Yamada <masahiroy@...nel.org>,
        Michal Marek <michal.lkml@...kovi.net>,
        X86 ML <x86@...nel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>,
        llvm@...ts.linux.dev
Subject: Re: [PATCH] kbuild: Support clang-$ver builds

On Sat, Oct 30, 2021 at 4:34 PM Peter Zijlstra <peterz@...radead.org> wrote:
>
>
> Latestest greatness... :-)
>
> ---
> Subject: kbuild: Fix clang/llvm build infra
>
> Debian (and derived) distros ship their compilers as -$ver suffixed
> binaries. For gcc it is sufficent to use:
>
>  $ make CC=gcc-12
>
> However, clang builds (esp. clang-lto) need a whole array of tools to be
> exactly right, leading to unweildy stuff like:
>
>  $ make CC=clang-13 LD=ld.lld=14 AR=llvm-ar-13 NM=llvm-nm-13 OBJCOPY=llvm-objcopy-13 OBJDUMP=llvm-objdump-13 READELF=llvm-readelf-13 STRIP=llvm-strip-13 LLVM=1
>
> which is, quite franktly, totally insane and unusable. Instead make
> the CC variable DTRT, enabling one such as myself to use:
>
>  $ make CC=clang-13
>
> This also lets one quickly test different clang versions.
> Additionally, also support path based LLVM suites like:
>
>  $ make CC=/opt/llvm/bin/clang
>

Hi Peter,

Thanks for bringing this up again.

Your issue is not new for Debianist and Linux-LLVM/Clang folks.

But let me comment.

I agree, it is preferable or should be treated the same way in using
clang-$ver like gcc-$ver.

Background:
Debian uses as meta-package llvm-toolchain-$ver and splits stuff into
different Debian packages like the Clang compiler. LLD linker and LLVM
(bin)utils.
These packages place their files in separate directories to
distinguish llvm-toolchain-$ver.
Binaries like clang-$ver, ld.lld-$ver, llvm-strip-$ver, etc. you find
in the /usr/bin/ directory which are symlinks.
Clang compiler is shipped with clang-$ver Debian package.
LLD linker is shipped as a separate Debian package called lld-$ver.
LLVM (bin)utils like llvm-strip-$ver are shipped with the llvm-$ver
Debian package.

Link: https://www.kernel.org/doc/html/latest/kbuild/llvm.html#llvm-utilities

EXAMPLES (here: llvm-toolchain-13):

$ dpkg -S $(which clang-13)
clang-13: /usr/bin/clang-13

$ dpkg -S /usr/bin/ld.lld-13
lld-13: /usr/bin/ld.lld-13

$ dpkg -S /usr/bin/llvm-strip-13
llvm-13: /usr/bin/llvm-strip-13

$ LC_ALL=C ll /usr/bin/clang-13
lrwxrwxrwx 1 root root 24 Oct 23 10:33 /usr/bin/clang-13 ->
../lib/llvm-13/bin/clang

$ LC_ALL=C ll /usr/bin/ld.lld-13
lrwxrwxrwx 1 root root 25 Oct 23 10:33 /usr/bin/ld.lld-13 ->
../lib/llvm-13/bin/ld.lld

$ LC_ALL=C ll /usr/bin/llvm-strip-13
lrwxrwxrwx 1 root root 29 Oct 23 10:33 /usr/bin/llvm-strip-13 ->
../lib/llvm-13/bin/llvm-strip

As a compromise I use in my build_linux-llvm-toolchain.sh script
$LLVM_TOOLCHAIN_PATH:
...
### GCC and LLVM version settings
##GCC_MAJOR_VER="10"
LLVM_MAJOR_VER="13"

### LLVM toolchain path options
# Explicitly set PATH to simplify `make LLVM=1` handling (see "LLVM
tools options")
# Option-1: Use system's LLVM toolchain
# Option-2: Use selfmade LLVM toolchain (here: ThinLTO+PGO optimized
via tc-build)
# Option-3: Set custom PATH in ~/.bashrc
##LLVM_TOOLCHAIN_PATH="/usr/lib/llvm-${LLVM_MAJOR_VER}/bin"
LLVM_TOOLCHAIN_PATH="/opt/llvm-toolchain/bin"
if [ -d ${LLVM_TOOLCHAIN_PATH} ]; then
   export PATH="${LLVM_TOOLCHAIN_PATH}:${PATH}"
fi
...

But...

...for several new features like Clang-LTO we need the LLVM full "eco system".
LLVM eco system means compiler, linker and binutils.
In several talks at LPC 2021 talkers used the term "LLVM/binutils"
like for GNU/binutils.
The LLVM/Clang kernel-docs talk about "LLVM util(itie)s" which is set
by LLVM=1 or not (LLVM=0).

Link: https://www.kernel.org/doc/html/latest/kbuild/llvm.html#llvm-utilities

AFAICS your patch has a primary focus to fix the LLVM/Clang
infrastructure when CONFIG_LTO_CLANG=y, right?
Can you massage the commit message to reflect this?

IMHO your patch can be improved to check for CONFIG_LTO_CLANG=y where
LLVM=1 is mandatory and IIRC LLVM_IAS=1, too.
( The default to use LLVM/Clang-IAS (Integrated ASsembler) was changed
recently for some combinations arches + Clang-LTO? )

Before I forgot:
Can you add a comment for the PPC and S390 situation in the commit message.

Please remember people also want to use combinations like gcc-$ver and
LLVM/binutils like ld.lld-$ver.
GCC-10 + LLD-$VER was the fastest combination to build a Linux Debian
package here on my Intel SandyBridge CPU system.
( There exists overrides like LD=... STRIP=... and I use
XXX_FOR_BUILD=... (see my attached build-script). )

Thanks.

Regards,
- Sedat -

> Signed-off-by: Peter Zijlstra (Intel) <peterz@...radead.org>
> ---
>  Makefile                       |   45 +++++++++++++++++++++++++++---------
>  tools/scripts/Makefile.include |   50 ++++++++++++++++++++++++++++-------------
>  2 files changed, 68 insertions(+), 27 deletions(-)
>
> --- a/Makefile
> +++ b/Makefile
> @@ -423,9 +423,29 @@ HOST_LFS_CFLAGS := $(shell getconf LFS_C
>  HOST_LFS_LDFLAGS := $(shell getconf LFS_LDFLAGS 2>/dev/null)
>  HOST_LFS_LIBS := $(shell getconf LFS_LIBS 2>/dev/null)
>
> -ifneq ($(LLVM),)
> -HOSTCC = clang
> -HOSTCXX        = clang++
> +# powerpc and s390 don't yet work with LLVM as a whole
> +ifeq ($(ARCH),powerpc)
> +LLVM = 0
> +endif
> +ifeq ($(ARCH),s390)
> +LLVM = 0
> +endif
> +
> +# otherwise, if CC=clang, default to using LLVM to enable LTO
> +CC_BASE := $(shell echo $(CC) | sed 's/.*\///')
> +CC_NAME := $(shell echo $(CC_BASE) | cut -b "1-5")
> +ifeq ($(shell test "$(CC_NAME)" = "clang"; echo $$?),0)
> +LLVM ?= 1
> +LLVM_PFX := $(shell echo $(CC) | sed 's/\(.*\/\)\?.*/\1/')
> +LLVM_SFX := $(shell echo $(CC_BASE) | cut -b "6-")
> +endif
> +
> +# if not set by now, do not use LLVM
> +LLVM ?= 0
> +
> +ifneq ($(LLVM),0)
> +HOSTCC = $(LLVM_PFX)clang$(LLVM_SFX)
> +HOSTCXX        = $(LLVM_PFX)clang++$(LLVM_SFX)
>  else
>  HOSTCC = gcc
>  HOSTCXX        = g++
> @@ -442,15 +462,15 @@ KBUILD_HOSTLDLIBS   := $(HOST_LFS_LIBS)
>
>  # Make variables (CC, etc...)
>  CPP            = $(CC) -E
> -ifneq ($(LLVM),)
> -CC             = clang
> -LD             = ld.lld
> -AR             = llvm-ar
> -NM             = llvm-nm
> -OBJCOPY                = llvm-objcopy
> -OBJDUMP                = llvm-objdump
> -READELF                = llvm-readelf
> -STRIP          = llvm-strip
> +ifneq ($(LLVM),0)
> +CC             = $(LLVM_PFX)clang$(LLVM_SFX)
> +LD             = $(LLVM_PFX)ld.lld$(LLVM_SFX)
> +AR             = $(LLVM_PFX)llvm-ar$(LLVM_SFX)
> +NM             = $(LLVM_PFX)llvm-nm$(LLVM_SFX)
> +OBJCOPY                = $(LLVM_PFX)llvm-objcopy$(LLVM_SFX)
> +OBJDUMP                = $(LLVM_PFX)llvm-objdump$(LLVM_SFX)
> +READELF                = $(LLVM_PFX)llvm-readelf$(LLVM_SFX)
> +STRIP          = $(LLVM_PFX)llvm-strip$(LLVM_SFX)
>  else
>  CC             = $(CROSS_COMPILE)gcc
>  LD             = $(CROSS_COMPILE)ld
> @@ -461,6 +481,7 @@ OBJDUMP             = $(CROSS_COMPILE)objdump
>  READELF                = $(CROSS_COMPILE)readelf
>  STRIP          = $(CROSS_COMPILE)strip
>  endif
> +
>  PAHOLE         = pahole
>  RESOLVE_BTFIDS = $(objtree)/tools/bpf/resolve_btfids/resolve_btfids
>  LEX            = flex
> --- a/tools/scripts/Makefile.include
> +++ b/tools/scripts/Makefile.include
> @@ -51,12 +51,32 @@ define allow-override
>      $(eval $(1) = $(2)))
>  endef
>
> -ifneq ($(LLVM),)
> -$(call allow-override,CC,clang)
> -$(call allow-override,AR,llvm-ar)
> -$(call allow-override,LD,ld.lld)
> -$(call allow-override,CXX,clang++)
> -$(call allow-override,STRIP,llvm-strip)
> +# powerpc and s390 don't yet work with LLVM as a whole
> +ifeq ($(ARCH),powerpc)
> +LLVM = 0
> +endif
> +ifeq ($(ARCH),s390)
> +LLVM = 0
> +endif
> +
> +# otherwise, if CC=clang, default to using LLVM to enable LTO
> +CC_BASE := $(shell echo $(CC) | sed 's/.*\///')
> +CC_NAME := $(shell echo $(CC_BASE) | cut -b "1-5")
> +ifeq ($(shell test "$(CC_NAME)" = "clang"; echo $$?),0)
> +LLVM ?= 1
> +LLVM_PFX := $(shell echo $(CC) | sed 's/\(.*\/\)\?.*/\1/')
> +LLVM_SFX := $(shell echo $(CC_BASE) | cut -b "6-")
> +endif
> +
> +# if not set by now, do not use LLVM
> +LLVM ?= 0
> +
> +ifneq ($(LLVM),0)
> +$(call allow-override,CC,$(LLVM_PFX)clang$(LLVM_SFX))
> +$(call allow-override,AR,$(LLVM_PFX)llvm-ar$(LLVM_SFX))
> +$(call allow-override,LD,$(LLVM_PFX)ld.lld$(LLVM_SFX))
> +$(call allow-override,CXX,$(LLVM_PFX)clang++$(LLVM_SFX))
> +$(call allow-override,STRIP,$(LLVM_PFX)llvm-strip$(LLVM_SFX))
>  else
>  # Allow setting various cross-compile vars or setting CROSS_COMPILE as a prefix.
>  $(call allow-override,CC,$(CROSS_COMPILE)gcc)
> @@ -68,10 +88,10 @@ endif
>
>  CC_NO_CLANG := $(shell $(CC) -dM -E -x c /dev/null | grep -Fq "__clang__"; echo $$?)
>
> -ifneq ($(LLVM),)
> -HOSTAR  ?= llvm-ar
> -HOSTCC  ?= clang
> -HOSTLD  ?= ld.lld
> +ifneq ($(LLVM),0)
> +HOSTAR  ?= $(LLVM_PFX)llvm-ar$(LLVM_SFX)
> +HOSTCC  ?= $(LLVM_PFX)clang$(LLVM_SFX)
> +HOSTLD  ?= $(LLVM_PFX)ld.lld$(LLVM_SFX)
>  else
>  HOSTAR  ?= ar
>  HOSTCC  ?= gcc
> @@ -79,11 +99,11 @@ HOSTLD  ?= ld
>  endif
>
>  # Some tools require Clang, LLC and/or LLVM utils
> -CLANG          ?= clang
> -LLC            ?= llc
> -LLVM_CONFIG    ?= llvm-config
> -LLVM_OBJCOPY   ?= llvm-objcopy
> -LLVM_STRIP     ?= llvm-strip
> +CLANG          ?= $(LLVM_PFX)clang$(LLVM_SFX)
> +LLC            ?= $(LLVM_PFX)llc$(LLVM_SFX)
> +LLVM_CONFIG    ?= $(LLVM_PFX)llvm-config$(LLVM_SFX)
> +LLVM_OBJCOPY   ?= $(LLVM_PFX)llvm-objcopy$(LLVM_SFX)
> +LLVM_STRIP     ?= $(LLVM_PFX)llvm-strip$(LLVM_SFX)
>
>  ifeq ($(CC_NO_CLANG), 1)
>  EXTRA_WARNINGS += -Wstrict-aliasing=3

Download attachment "build_linux-llvm-toolchain.sh" of type "application/x-shellscript" (14509 bytes)

Powered by blists - more mailing lists