linux-kernel - Re: [PATCH v5 21/21] kbuild: implement CONFIG_TRIM_UNUSED

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <93s3n008-7oon-30rq-5219-5r244919r38q@syhkavp.arg>
Date:   Mon, 15 May 2023 17:31:33 -0400 (EDT)
From:   Nicolas Pitre <nico@...xnic.net>
To:     Masahiro Yamada <masahiroy@...nel.org>
cc:     linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
        Nathan Chancellor <nathan@...nel.org>,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        Nicolas Schier <nicolas@...sle.eu>
Subject: Re: [PATCH v5 21/21] kbuild: implement CONFIG_TRIM_UNUSED_KSYMS
 without recursion

On Mon, 15 May 2023, Masahiro Yamada wrote:

> When CONFIG_TRIM_UNUSED_KSYMS is enabled, Kbuild recursively traverses
> the directory tree to determine which EXPORT_SYMBOL to trim. If an
> EXPORT_SYMBOL turns out to be unused by anyone, Kbuild begins the
> second traverse, where some source files are recompiled with their
> EXPORT_SYMBOL() tuned into a no-op.
> 
> Linus stated negative opinions about this slowness in commits:
> 
>  - 5cf0fd591f2e ("Kbuild: disable TRIM_UNUSED_KSYMS option")
>  - a555bdd0c58c ("Kbuild: enable TRIM_UNUSED_KSYMS again, with some guarding")
> 
> We can do this better now. The final data structures of EXPORT_SYMBOL
> are generated by the modpost stage, so modpost can selectively emit
> KSYMTAB entries that are really used by modules.
> 
> Commit 2cce989f8461 ("kbuild: unify two modpost invocations") is another
> ground-work to do this in a one-pass algorithm. With the list of modules,
> modpost sets sym->used if it is used by a module. modpost emits KSYMTAB
> only for symbols with sym->used==true.
> 
> BTW, Nicolas explained why the trimming was implemented with recursion:
> 
>   https://lore.kernel.org/all/2o2rpn97-79nq-p7s2-nq5-8p83391473r@syhkavp.arg/
> 
> Actually, we never achieved that level of optimization where the chain
> reaction of trimming comes into play because:
> 
>  - CONFIG_LTO_CLANG cannot remove any unused symbols
>  - CONFIG_LD_DEAD_CODE_DATA_ELIMINATION is enabled only for vmlinux,
>    but not modules

I did achieve it using LTO with gcc back then. See the section called 
"The tree that hides the forest" of https://lwn.net/Articles/746780/ for 
example results.

> If deeper trimming is required, we need to revisit this, but I guess
> that is unlikely to happen.

Would have been nicer to keep this possibility as an option. The code is 
already there and working as intended. The build cost is intrinsic to 
the approach of course. The actual bug is to impose that cost onto 
people who didn't explicitly ask for it.

But I'm no longer fighting this battle.


Nicolas