linux-kernel - Re: [PATCH] kbuild: add script check for cross compilation utilities

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAK7LNAQA7YdtnuTKzO2UZdUqXvn+zxcU-GUD5RBQkMHYs7ZY3Q@mail.gmail.com>
Date:   Sun, 12 May 2019 11:21:28 +0900
From:   Masahiro Yamada <yamada.masahiro@...ionext.com>
To:     Nick Desaulniers <ndesaulniers@...gle.com>
Cc:     clang-built-linux <clang-built-linux@...glegroups.com>,
        Michal Marek <michal.lkml@...kovi.net>,
        Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] kbuild: add script check for cross compilation utilities

Hi.

On Fri, May 10, 2019 at 5:19 AM Nick Desaulniers
<ndesaulniers@...gle.com> wrote:
>
> When cross compiling via setting CROSS_COMPILE, if the prefixed tools
> are not found, then the host utilities are often instead invoked, and
> produce often difficult to understand errors.  This is most commonly the
> case for developers new to cross compiling the kernel that have yet to
> install the proper cross compilation toolchain. Rather than charge
> headlong into a build that will fail obscurely, check that the tools
> exist before starting to compile, and fail with a friendly error
> message.

I see one drawback in adding the check script
so early in the top Makefile.

Make targets that do not require toolchain at all
("make clean", "make headers_install", etc.)
would fail too.


>
> Before:
> $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang
> ...
> /usr/bin/as: unrecognized option '-EL'
> clang: error: assembler command failed with exit code 1 (use -v to see
> invocation)
> make[2]: *** [../scripts/Makefile.build:279: scripts/mod/empty.o] Error 1
> make[2]: *** Waiting for unfinished jobs....
> make[1]: *** [/linux/Makefile:1118:
> prepare0] Error 2
> make: *** [Makefile:179: sub-make] Error 2


What a coincidence, I had sent this patch before:
https://patchwork.kernel.org/patch/10936811/


With my patch applied, the command above would be failed like this:


$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf-  CC=clang defconfig all
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  LEX     scripts/kconfig/lexer.lex.c
  YACC    scripts/kconfig/parser.tab.h
  HOSTCC  scripts/kconfig/lexer.lex.o
  YACC    scripts/kconfig/parser.tab.c
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTCC  scripts/kconfig/preprocess.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTLD  scripts/kconfig/conf
*** Default configuration is based on 'multi_v7_defconfig'
scripts/Kconfig.include:35: linker 'arm-linux-gnueabihf-ld' not found
make[2]: *** [scripts/kconfig/Makefile;82: defconfig] Error 1
make[1]: *** [Makefile;557: defconfig] Error 2
make: *** [Makefile;325: __build_one_by_one] Error 2




The presence $(CC) and $(LD) are checked in the Kconfig stage.
If it is not enough, it is OK to add $(AS) check.

$(error-if,$(failure,command -v $(AS)),assembler '$(AS)' not found)


But, I do not want to add all of $(AR), $(NM), ... etc.


Thanks.



> After:
> $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang
> $CROSS_COMPILE set to arm-linux-gnueabihf-, but unable to find
> arm-linux-gnueabihf-as.
> Makefile:522: recipe for target 'outputmakefile' failed
> make: *** [outputmakefile] Error 1
>
> Signed-off-by: Nick Desaulniers <ndesaulniers@...gle.com>
> ---
> Note: this is probably more generally useful, but after a few minutes
> wrestling with Make errors related to "recipe commences before first
> target" and "missing separator," I came to understand my hatred of GNU
> Make. Open to sugguestions for where better to invoke this from the top
> level Makefile.
>
>  Makefile                      |  1 +
>  scripts/check_crosscompile.sh | 18 ++++++++++++++++++
>  2 files changed, 19 insertions(+)
>  create mode 100755 scripts/check_crosscompile.sh
>
> diff --git a/Makefile b/Makefile
> index a61a95b6b38f..774339674b59 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -519,6 +519,7 @@ endif
>
>  ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
>  ifneq ($(CROSS_COMPILE),)
> +       $(Q)$(CONFIG_SHELL) $(srctree)/scripts/check_crosscompile.sh
>  CLANG_FLAGS    := --target=$(notdir $(CROSS_COMPILE:%-=%))
>  GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit))
>  CLANG_FLAGS    += --prefix=$(GCC_TOOLCHAIN_DIR)
> diff --git a/scripts/check_crosscompile.sh b/scripts/check_crosscompile.sh
> new file mode 100755
> index 000000000000..f4586fbfee18
> --- /dev/null
> +++ b/scripts/check_crosscompile.sh
> @@ -0,0 +1,18 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# (c) 2019, Nick Desaulniers <ndesaulniers@...gle.com>
> +function check () {
> +  # Remove trailing commands, for example arch/arm/Makefile may add `-EL`.
> +  utility=$(echo ${1} | awk '{print $1;}')
> +  command -v "${utility}" &> /dev/null
> +  if [[ $? != 0 ]]; then
> +    echo "\$CROSS_COMPILE set to ${CROSS_COMPILE}," \
> +      "but unable to find ${utility}."
> +    exit 1
> +  fi
> +}
> +utilities=("${AS}" "${LD}" "${CC}" "${AR}" "${NM}" "${STRIP}" "${OBJCOPY}"
> +  "${OBJDUMP}")
> +for utility in "${utilities[@]}"; do
> +  check "${utility}"
> +done
> --
> 2.21.0.1020.gf2820cf01a-goog
>


--
Best Regards
Masahiro Yamada