linux-kernel - Re: [PATCH] kbuild: revive parallel execution for .tmp

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAK7LNARCM=rUm8mA8GRQ7ufeyfneGf4OEvHmESKt=zuxs2KrHw@mail.gmail.com>
Date:   Tue, 15 Nov 2022 21:16:09 +0900
From:   Masahiro Yamada <masahiroy@...nel.org>
To:     Nathan Chancellor <nathan@...nel.org>
Cc:     linux-kbuild@...r.kernel.org,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        Nicolas Schier <nicolas@...sle.eu>,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH] kbuild: revive parallel execution for .tmp_initcalls.lds rule

On Tue, Nov 15, 2022 at 9:18 AM Nathan Chancellor <nathan@...nel.org> wrote:
>
> On Tue, Nov 15, 2022 at 02:46:17AM +0900, Masahiro Yamada wrote:
> > Prior to commit 5d45950dfbb1 ("kbuild: move vmlinux.o link to
> > scripts/Makefile.vmlinux_o"), jobserver-exec was invoked from the shell
> > script, link-vmlinux.sh. It can get access to the jobserver because
> > Makefile adds '+' prefix, as in:
> >
> >     +$(call if_changed_dep,link_vmlinux)
> >
> > Since 5d45950dfbb1, jobserver-exec is invoked from Makefile, but the
> > '+' prefix is missing, hence jobserver-exec has no access to the
> > jobserver.
> >
> > Fixes: 5d45950dfbb1 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o")
> > Signed-off-by: Masahiro Yamada <masahiroy@...nel.org>
>
> Reviewed-by: Nathan Chancellor <nathan@...nel.org>
>
> At least it doesn't seem like compile times were majorly affected. I
> benchmarked arm64 allmodconfig (worst case scenario with ThinLTO):


Ah, sorry.

./scripts/generate_initcall_order.pl always runs in two threads
(the parent and a single child) because it is given only
one argument (vmlinux.a).

So, using jobserver-exec is meaningless.
Also, the code for parallel execution control in
./scripts/generate_initcall_order.pl is meaningless.



BTW, allmodconfig is not the worst case because most of the objects
go into modules instead of vmlinux.

allyesconfig would add more symbols to vmlinux although I did not test it.







>
> Benchmark 1: 094226ad94f4 ("Linux 6.1-rc5")
>   Time (mean ± σ):     899.036 s ±  1.133 s    [User: 49314.495 s, System: 3840.796 s]
>   Range (min … max):   898.118 s … 900.302 s    3 runs
>
> Benchmark 2: 0f45cbb5399b ("kbuild: revive parallel execution for .tmp_initcalls.lds rule")
>   Time (mean ± σ):     898.482 s ±  0.152 s    [User: 49329.703 s, System: 3836.408 s]
>   Range (min … max):   898.306 s … 898.584 s    3 runs
>
> Summary
>   '0f45cbb5399b ("kbuild: revive parallel execution for .tmp_initcalls.lds rule")' ran
>     1.00 ± 0.00 times faster than '094226ad94f4 ("Linux 6.1-rc5")'
>
> > ---
> >
> >  scripts/Makefile.vmlinux_o | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/scripts/Makefile.vmlinux_o b/scripts/Makefile.vmlinux_o
> > index 0edfdb40364b..ae52d3b3f063 100644
> > --- a/scripts/Makefile.vmlinux_o
> > +++ b/scripts/Makefile.vmlinux_o
> > @@ -19,7 +19,7 @@ quiet_cmd_gen_initcalls_lds = GEN     $@
> >
> >  .tmp_initcalls.lds: $(srctree)/scripts/generate_initcall_order.pl \
> >               vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE
> > -     $(call if_changed,gen_initcalls_lds)
> > +     +$(call if_changed,gen_initcalls_lds)
> >
> >  targets := .tmp_initcalls.lds
> >
> > --
> > 2.34.1
> >



--
Best Regards

Masahiro Yamada