linux-kernel - Re: linux-next: build failure after merge of the arm64 tree

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAMn1gO6P_VfDRjGZb67ZS4Kh0wjTEQi0cbOkmibTokHQOgP7qw@mail.gmail.com>
Date:   Tue, 6 Aug 2019 19:34:36 -0700
From:   Peter Collingbourne <pcc@...gle.com>
To:     Stephen Rothwell <sfr@...b.auug.org.au>
Cc:     Catalin Marinas <catalin.marinas@....com>,
        Will Deacon <will@...nel.org>,
        Linux Next Mailing List <linux-next@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Masahiro Yamada <yamada.masahiro@...ionext.com>,
        benh@...nel.crashing.org, paulus@...ba.org, mpe@...erman.id.au
Subject: Re: linux-next: build failure after merge of the arm64 tree

On Tue, Aug 6, 2019 at 4:50 PM Stephen Rothwell <sfr@...b.auug.org.au> wrote:
>
> Hi all,
>
> After merging the arm64 tree, today's linux-next build (powerpc
> ppc64_defconfig) was just spinning in make - it executing some scripts,
> but it was hard to catch just what.
>
> Apparently caused by commit
>
>   5cf896fb6be3 ("arm64: Add support for relocating the kernel with RELR relocations")
>
> I have not idea why, but reverting the above commit allows to build
> to finish.

Okay, I can reproduce with:

$ make ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- defconfig
*** Default configuration is based on 'ppc64_defconfig'
#
# No change to .config
#
$ make ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- -j72
scripts/kconfig/conf  --syncconfig Kconfig
scripts/kconfig/conf  --syncconfig Kconfig
scripts/kconfig/conf  --syncconfig Kconfig
[...]

And confirmed that backing out my patch fixes it.

The problem seems to come from the use of $(NM) in the Kconfig file.
If I apply this diff:

diff --git a/init/Kconfig b/init/Kconfig
index d96127ebc44e0..177a95b323230 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -31,7 +31,7 @@ config CC_HAS_ASM_GOTO
        def_bool $(success,$(srctree)/scripts/gcc-goto.sh $(CC))

 config TOOLS_SUPPORT_RELR
-       def_bool $(success,env "CC=$(CC)" "LD=$(LD)" "NM=$(NM)"
"OBJCOPY=$(OBJCOPY)" $(srctree)/scripts/tools-support-relr.sh)
+       def_bool $(success,$(NM))

 config CC_HAS_WARN_MAYBE_UNINITIALIZED
        def_bool $(cc-option,-Wmaybe-uninitialized)

I still see the hang. Replacing $(NM) with something else makes it go away.

That leads me to ask what is special about $(NM) + powerpc? It turns
out to be this fragment of arch/powerpc/Makefile:

ifdef CONFIG_PPC64
new_nm := $(shell if $(NM) --help 2>&1 | grep -- '--synthetic' >
/dev/null; then echo y; else echo n; fi)

ifeq ($(new_nm),y)
NM              := $(NM) --synthetic
endif
endif

We're setting NM to something else based on a config option, which I
presume sets up some sort of circular dependency that confuses
Kconfig. Removing this fragment of the makefile (or appending
--synthetic unconditionally) also makes the problem go away.

We should at least able to remove the test for a new-enough binutils.
According to changes.rst we require binutils 2.21 which was released
in 2011, and support for --synthetic was added to binutils in 2004:
https://github.com/bminor/binutils-gdb/commit/0873df2aec48685715d2c5041c1f6f4ed43976c1

But why are we passing --synthetic at all on ppc64? This behaviour
seems to come from this commit from 2004:
https://github.com/mpe/linux-fullhistory/commit/0e32679a4ea48a634d94e97355d47512ef14d71f
which states: "On new toolchains we need to use nm --synthetic or we
miss code symbols."

But I only see a couple of missing symbols if I compare the output of
nm with and without --synthetic on a powerpc64 kernel:

$ diff -u <(powerpc-linux-gnu-nm --synthetic vmlinux)
<(powerpc-linux-gnu-nm  vmlinux)
--- /dev/fd/63 2019-08-06 18:48:56.127020621 -0700
+++ /dev/fd/62 2019-08-06 18:48:56.131020636 -0700
@@ -74840,7 +74840,6 @@
 c000000001901b10 D LZ4_decompress_fast
 c0000000007819a0 T .LZ4_decompress_fast_continue
 c000000001901b70 D LZ4_decompress_fast_continue
-c0000000007811e0 t .LZ4_decompress_fast_extDict
 c000000001901b40 d LZ4_decompress_fast_extDict
 c000000000781960 T .LZ4_decompress_fast_usingDict
 c000000001901b58 D LZ4_decompress_fast_usingDict
@@ -74856,7 +74855,6 @@
 c000000001901bd0 D LZ4_decompress_safe_usingDict
 c0000000007822b0 T .LZ4_decompress_safe_withPrefix64k
 c000000001901b88 D LZ4_decompress_safe_withPrefix64k
-c000000000780c60 t .LZ4_decompress_safe_withSmallPrefix
 c000000001901b28 d LZ4_decompress_safe_withSmallPrefix
 c00000000077fbe0 T .LZ4_setStreamDecode
 c000000001901ac8 D LZ4_setStreamDecode

I guess the problem was worse back in 2004. It almost certainly didn't
involve these particular symbols because they were added in commit
2209fda323e2fd2a2d0885595fd5097717f8d2aa from 2018. So I guess we have
a couple of possible quick fixes (assuming that the Kconfig issue
can't be solved somehow): either stop passing --synthetic on powerpc
and lose a couple of symbols in 64-bit kernels, or start passing it
unconditionally on powerpc (it doesn't seem to make a difference to
the nm output on a ppc64_defconfig kernel with CONFIG_PPC64=n). I'm
cc'ing the powerpc maintainers for their opinion on what to do. While
this is being resolved we should probably back out my patch from
-next.

Peter