linux-kernel - Re: Very slow clang kernel config ..

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 3 May 2021 03:03:31 +0200 (CEST)
From:   "Maciej W. Rozycki" <macro@...am.me.uk>
To:     Linus Torvalds <torvalds@...ux-foundation.org>
cc:     Tom Stellard <tstellar@...hat.com>,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        Masahiro Yamada <masahiroy@...nel.org>,
        Nathan Chancellor <nathan@...nel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        clang-built-linux <clang-built-linux@...glegroups.com>,
        Fangrui Song <maskray@...gle.com>,
        Serge Guelton <sguelton@...hat.com>,
        Sylvestre Ledru <sylvestre@...illa.com>
Subject: Re: Very slow clang kernel config ..

On Sat, 1 May 2021, Linus Torvalds wrote:

> > Yes, it's intentional.  Dynamic linking libraries from other packages is
> > the Fedora policy[1], and clang and llvm are separate packages (in Fedora).
> 
> Side note: I really wish Fedora stopped doing that.

 I wish they never stop.

> Shared libraries are not a good thing in general. They add a lot of
> overhead in this case, but more importantly they also add lots of
> unnecessary dependencies and complexity, and almost no shared
> libraries are actually version-safe, so it adds absolutely zero
> upside.

 I agree shared libraries are a tough compromise, but there is an 
important upside.  Let me quote myself from a recent discussion 
<https://lore.kernel.org/linux-mips/alpine.DEB.2.21.2103191500040.21463@angie.orcam.me.uk/>:

"Dynamic shared objects (libraries) were invented in early 1990s for two
reasons:

1. To limit the use of virtual memory.  Memory conservation may not be as
   important nowadays in many applications where vast amounts of RAM are
   available, though of course this does not apply everywhere, and still
   it has to be weighed up whether any waste of resources is justified and
   compensated by a gain elsewhere.

2. To make it easy to replace a piece of code shared among many programs,
   so that you don't have to relink them all (or recompile if sources are
   available) when say an issue is found or a feature is added that is
   transparent to applications (for instance a new protocol or a better
   algorithm).  This still stands very much nowadays.

People went through great efforts to support shared libraries, sacrificed
performance for it even back then when the computing power was much lower
than nowadays.  Support was implemented in Linux for the a.out binary
format even, despite the need to go through horrible hoops to get a.out
shared libraries built.  Some COFF environments were adapted for shared
library support too."

 And the context here is a bug in the linker caused all programs built by 
Golang to be broken WRT FPU handling for the 32-bit MIPS configuration, 
due to a bad ABI annotation causing the wrong per-process FPU mode being 
set up at run time (Golang seemed to have got stuck in early 2000s as far 
the MIPS ABI goes and chose to produce what has been considered legacy 
objects for some 10 years now, and nobody noticed in 10 years or so that 
the GNU linker does not handle legacy MIPS objects correctly anymore).  
This could have been fixed easily by rebuilding the Go runtime, but as it 
happens Google chose not to create a shared Go runtime and all programs 
are linked statically except for libc.

 This had led to a desperate attempt to work the issue around crudely in 
the kernel (which cannot be done in a completely foolproof way, because 
there's missing information) so that Debian does not have to rebuild 2000+ 
packages in a stable distribution, which OTOH is a no-no for them.

 Whether distributions package shared libraries in a reasonable manner is 
another matter, and I've lost hope it will ever happen, at least widely 
(there has been an attempt to address that with a distribution called PLD, 
where the policy used to have it that shared libraries coming from a given 
source package need to go into a separate binary package on their own, so 
that several versions of different SONAMEs each of the same shared library 
package can safely coexist in a single system, but I haven't checked it in 
many years whether the policy has been retained, nor actually ever used 
PLD myself).

  Maciej