linux-kernel - Re: [PATCH v8 00/16] Add support for Clang LTO

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAK8P3a1p8yKMKXB1cUpy-5PjehJGPX0SPNEx4VsMmqgOeR6fZg@mail.gmail.com>
Date:   Wed, 9 Dec 2020 10:19:44 +0100
From:   Arnd Bergmann <arnd@...nel.org>
To:     Fangrui Song <maskray@...gle.com>
Cc:     Sami Tolvanen <samitolvanen@...gle.com>,
        Masahiro Yamada <masahiroy@...nel.org>,
        Steven Rostedt <rostedt@...dmis.org>,
        Will Deacon <will@...nel.org>,
        Josh Poimboeuf <jpoimboe@...hat.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        "Paul E. McKenney" <paulmck@...nel.org>,
        Kees Cook <keescook@...omium.org>,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        clang-built-linux <clang-built-linux@...glegroups.com>,
        Kernel Hardening <kernel-hardening@...ts.openwall.com>,
        linux-arch <linux-arch@...r.kernel.org>,
        Linux ARM <linux-arm-kernel@...ts.infradead.org>,
        Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        linux-pci <linux-pci@...r.kernel.org>
Subject: Re: [PATCH v8 00/16] Add support for Clang LTO

On Wed, Dec 9, 2020 at 5:56 AM 'Fangrui Song' via Clang Built Linux
<clang-built-linux@...glegroups.com> wrote:
> On 2020-12-08, 'Sami Tolvanen' via Clang Built Linux wrote:
> >On Tue, Dec 8, 2020 at 4:15 AM Arnd Bergmann <arnd@...nel.org> wrote:
> >> So far I have
> >> not managed to get a working build out of it, the main problem so far being
> >> that it is really slow to build because the link stage only uses one CPU.
> >> These are the other issues I've seen so far:
>
> ld.lld ThinLTO uses the number of (physical cores enabled by affinity) by default.

Ah, I see.  Do you know if it's also possible to do something like
-flto=jobserver
to integrate better with the kernel build system?

I tend to run multiple builds under a top-level makefile with 'make
-j30' in order
to use 30 of the 32 threads and leave the scheduling to jobserver instead of
the kernel. If the linker itself is multithreaded but the jobserver
thinks it is a
single thread, could end up with 30 concurrent linkers each trying to use
16 cores.

> >> - CONFIG_CPU_BIG_ENDIAN doesn't seem to work with lld, and LTO
> >>   doesn't work with ld.bfd.
> >>   I've added a CPU_LITTLE_ENDIAN dependency to
> >>   ARCH_SUPPORTS_LTO_CLANG{,THIN}
> >
> >Ah, good point. I'll fix this in v9.
>
> Full/Thin LTO should work with GNU ld and gold with LLVMgold.so built from
> llvm-project (https://llvm.org/docs/GoldPlugin.html ). You'll need to make sure
> that LLVMgold.so is newer than clang. (Newer clang may introduce bitcode
> attributes which are unrecognizable by older LLVMgold.so/ld.lld)

The current patch series requires LLD:

config HAS_LTO_CLANG
       def_bool y
       depends on CC_IS_CLANG && CLANG_VERSION >= 110000 && LD_IS_LLD

Is this something we should change then, or try to keep it simple with the
current approach, leaving LTO disabled for big-endian builds and hosts without
a working lld?

       Arnd