linux-kernel - Re: [PATCH v3] kbuild: distributed build support for Clang ThinLTO

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAF1bQ=QHFJmqGOSOk-=SDh84+s-+qEcPiybFzXMOPpzanUiJZQ@mail.gmail.com>
Date: Tue, 27 May 2025 12:57:18 -0700
From: Rong Xu <xur@...gle.com>
To: Masahiro Yamada <masahiroy@...nel.org>
Cc: Nathan Chancellor <nathan@...nel.org>, Eric Naim <dnaim@...hyos.org>, 
	Nicolas Schier <nicolas.schier@...ux.dev>, 
	Nick Desaulniers <nick.desaulniers+lkml@...il.com>, Bill Wendling <morbo@...gle.com>, 
	Justin Stitt <justinstitt@...gle.com>, Miguel Ojeda <ojeda@...nel.org>, 
	Thomas Gleixner <tglx@...utronix.de>, Alice Ryhl <aliceryhl@...gle.com>, 
	Sami Tolvanen <samitolvanen@...gle.com>, "Mike Rapoport (Microsoft)" <rppt@...nel.org>, 
	Rafael Aquini <aquini@...hat.com>, Michael Ellerman <mpe@...erman.id.au>, 
	Stafford Horne <shorne@...il.com>, Christophe Leroy <christophe.leroy@...roup.eu>, 
	Teresa Johnson <tejohnson@...gle.com>, linux-kernel@...r.kernel.org, 
	linux-kbuild@...r.kernel.org, llvm@...ts.linux.dev
Subject: Re: [PATCH v3] kbuild: distributed build support for Clang ThinLTO

On Mon, May 26, 2025 at 2:05 AM Masahiro Yamada <masahiroy@...nel.org> wrote:
>
> On Thu, May 22, 2025 at 6:35 AM <xur@...gle.com> wrote:
>
> > +ifdef CONFIG_LTO_CLANG_THIN_DIST
> > +# Generate .o_thinlto_native (obj) from .o (bitcode) file
> > +# ---------------------------------------------------------------------------
> > +quiet_cmd_cc_o_bc = CC $(quiet_modtag) $@
> > +
> > +cmd_cc_o_bc      = $(if $(filter bitcode, $(shell file -b $<)),$(CC) \
> > +                  $(filter-out -Wp% $(LINUXINCLUDE) %.h.gch %.h -D% \
> > +                  -flto=thin, $(c_flags)) \
>
> I think this filter-out is fragile.
>
> I think it is better to use $(_c_flags), like this:
>
>     $(filter-out -flto=thin, $(_c_flags))
Ack.
>
>
>
>
>
>
>
>
>
>
>
>
> > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > index 2fe73cda0bddb..9cfd23590334d 100644
> > --- a/scripts/Makefile.lib
> > +++ b/scripts/Makefile.lib
> > @@ -34,8 +34,13 @@ else
> >  obj-m := $(filter-out %/, $(obj-m))
> >  endif
> >
> > +builtin_suffix := $(if $(filter %.a_thinlto_native, $(MAKECMDGOALS)),.a_thinlto_native,.a)
> > +ifeq ($(thinlto_final_pass),1)
> > +        builtin_suffix :=.a_thinlto_native
> > +endif
> > +
> >  ifdef need-builtin
> > -obj-y          := $(patsubst %/, %/built-in.a, $(obj-y))
> > +obj-y          := $(patsubst %/, %/built-in$(builtin_suffix), $(obj-y))
> >  else
> >  obj-y          := $(filter-out %/, $(obj-y))
> >  endif
> > diff --git a/scripts/Makefile.vmlinux_o b/scripts/Makefile.vmlinux_o
> > index b024ffb3e2018..f9abc45a68b36 100644
> > --- a/scripts/Makefile.vmlinux_o
> > +++ b/scripts/Makefile.vmlinux_o
> > @@ -9,6 +9,14 @@ include $(srctree)/scripts/Kbuild.include
> >  # for objtool
> >  include $(srctree)/scripts/Makefile.lib
> >
> > +ifeq ($(thinlto_final_pass),1)
> > +vmlinux_a := vmlinux.a_thinlto_native
> > +vmlinux_libs := $(patsubst %.a,%.a_thinlto_native,$(KBUILD_VMLINUX_LIBS))
> > +else
> > +vmlinux_a := vmlinux.a
> > +vmlinux_libs := $(KBUILD_VMLINUX_LIBS)
> > +endif
> > +
> >  # Generate a linker script to ensure correct ordering of initcalls for Clang LTO
> >  # ---------------------------------------------------------------------------
> >
> > @@ -18,7 +26,7 @@ quiet_cmd_gen_initcalls_lds = GEN     $@
> >         $(PERL) $(real-prereqs) > $@
> >
> >  .tmp_initcalls.lds: $(srctree)/scripts/generate_initcall_order.pl \
> > -               vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE
> > +               $(vmlinux_a) $(vmlinux_libs) FORCE
> >         $(call if_changed,gen_initcalls_lds)
> >
> >  targets := .tmp_initcalls.lds
> > @@ -59,8 +67,8 @@ quiet_cmd_ld_vmlinux.o = LD      $@
> >         $(LD) ${KBUILD_LDFLAGS} -r -o $@ \
> >         $(vmlinux-o-ld-args-y) \
> >         $(addprefix -T , $(initcalls-lds)) \
> > -       --whole-archive vmlinux.a --no-whole-archive \
> > -       --start-group $(KBUILD_VMLINUX_LIBS) --end-group \
> > +       --whole-archive $(vmlinux_a) --no-whole-archive \
> > +       --start-group $(vmlinux_libs) --end-group \
> >         $(cmd_objtool)
> >
> >  define rule_ld_vmlinux.o
> > @@ -68,7 +76,7 @@ define rule_ld_vmlinux.o
> >         $(call cmd,gen_objtooldep)
> >  endef
> >
> > -vmlinux.o: $(initcalls-lds) vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE
> > +vmlinux.o: $(initcalls-lds) $(vmlinux_a) $(vmlinux_libs) FORCE
> >         $(call if_changed_rule,ld_vmlinux.o)
> >
> >  targets += vmlinux.o
> > diff --git a/scripts/Makefile.vmlinux_thinlink b/scripts/Makefile.vmlinux_thinlink
> > new file mode 100644
> > index 0000000000000..13e4026c7d45b
> > --- /dev/null
> > +++ b/scripts/Makefile.vmlinux_thinlink
> > @@ -0,0 +1,53 @@
> > +# SPDX-License-Identifier: GPL-2.0-only
> > +
> > +PHONY := __default
> > +__default: vmlinux.thinlink
> > +
> > +include include/config/auto.conf
> > +include $(srctree)/scripts/Kbuild.include
> > +
> > +
> > +# Generate a linker script to ensure correct ordering of initcalls for Clang LTO
> > +# ---------------------------------------------------------------------------
> > +
> > +quiet_cmd_gen_initcalls_lds = GEN     $@
> > +      cmd_gen_initcalls_lds = \
> > +       $(PYTHON3) $(srctree)/scripts/jobserver-exec \
> > +       $(PERL) $(real-prereqs) > $@
>
>
> This is a copy-paste from scripts/Makefile.vmlinux_o.
>
> Is this linker script critical when generating the index file?

Our usual guideline for the ThinLTO indexing command is to use the
same arguments
as the final link. For this particular case, it probably does not
matter (i.e we can skip the linker script option).

>
>
>
> --
> Best Regards
> Masahiro Yamada