linux-kernel - Re: Very slow clang kernel config ..

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20210501233754.4he27eoteqvplywu@glandium.org>
Date:   Sun, 2 May 2021 08:37:54 +0900
From:   Mike Hommey <mh@...ndium.org>
To:     Nick Desaulniers <ndesaulniers@...gle.com>
Cc:     Linus Torvalds <torvalds@...ux-foundation.org>,
        Tom Stellard <tstellar@...hat.com>,
        Masahiro Yamada <masahiroy@...nel.org>,
        Nathan Chancellor <nathan@...nel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        clang-built-linux <clang-built-linux@...glegroups.com>,
        Fangrui Song <maskray@...gle.com>,
        Serge Guelton <sguelton@...hat.com>,
        Sylvestre Ledru <sylvestre@...illa.com>
Subject: Re: Very slow clang kernel config ..

On Fri, Apr 30, 2021 at 06:48:11PM -0700, Nick Desaulniers wrote:
> On Fri, Apr 30, 2021 at 6:22 PM Linus Torvalds
> <torvalds@...ux-foundation.org> wrote:
> >
> > On Fri, Apr 30, 2021 at 5:25 PM Nick Desaulniers
> > <ndesaulniers@...gle.com> wrote:
> > >
> > > Ah, no, sorry, these are the runtime link editor/loader. So probably
> > > spending quite some time resolving symbols in large binaries.
> >
> > Yeah. Appended is the profile I see when I profile that "make
> > oldconfig", so about 45% of all time seems to be spent in just symbol
> > lookup and relocation.
> >
> > And a fair amount of time just creating and tearing down that huge
> > executable (with a lot of copy-on-write overhead too), with the kernel
> > side of that being another 15%. The cost of that is likely also fairly
> > directly linked to all the dynamic linking costs, which brings in all
> > that data.
> >
> > Just to compare, btw, this is the symbol lookup overhead for the gcc case:
> >
> >    1.43%  ld-2.33.so             do_lookup_x
> >    0.96%  ld-2.33.so             _dl_relocate_object
> >    0.69%  ld-2.33.so             _dl_lookup_symbol_x
> >
> > so it really does seem to be something very odd going on with the clang binary.
> >
> > Maybe the Fedora binary is built some odd way, but it's likely just
> > the default clang build.
> >
> >              Linus
> >
> > ----
> >   23.59%  ld-2.33.so          _dl_lookup_symbol_x
> >   11.41%  ld-2.33.so          _dl_relocate_object
> >    9.95%  ld-2.33.so          do_lookup_x
> >    4.00%  [kernel.vmlinux]    copy_page
> >    3.98%  [kernel.vmlinux]    next_uptodate_page
> >    3.05%  [kernel.vmlinux]    zap_pte_range
> >    1.81%  [kernel.vmlinux]    clear_page_rep
> >    1.68%  [kernel.vmlinux]    asm_exc_page_fault
> >    1.33%  ld-2.33.so          strcmp
> >    1.33%  ld-2.33.so          check_match
> 
> 47.61% spent in symbol table lookup. Nice. (Not counting probably a
> fair amount of the libc calls below).
> 
> >    0.92%  libLLVM-12.so       llvm::StringMapImpl::LookupBucketFor
> 
> ^ wait a minute; notice how in your profile the `Shared Object` is
> attributed to `libLLVM-12.so` while mine is `clang-13`?  Clang can be
> built as either having libllvm statically linked or dynamically; see
> the cmake variables
> LLVM_BUILD_LLVM_DYLIB:BOOL
> LLVM_LINK_LLVM_DYLIB:BOOL
> BUILD_SHARED_LIBS:BOOL
> https://llvm.org/docs/CMake.html
> 
> I think those are frowned upon; useful for cutting down on developers
> iteration speed due to not having to relink llvm when developing
> clang. But shipping that in production? I just checked and it doesn't
> look like we do that for AOSP's build of LLVM.
> 
> Tom, is one of the above intentionally set for clang builds on Fedora?
> I'm guessing it's intentional that there are packages for
> libLLVM-12.so and libclang-cpp.so.12, perhaps they have other
> dependents?

Have you tried building clang/llvm with -Bsymbolic-functions?

Mike