linux-kernel - Re: [GIT PULL] Kbuild fixes for v6.15-rc7

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAK7LNARY48483tOLUf01-d_VnQh+K=-e+SLEeAqMaZfZC0YjLA@mail.gmail.com>
Date: Tue, 27 May 2025 15:27:24 +0900
From: Masahiro Yamada <masahiroy@...nel.org>
To: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Linux Kernel Mailing List <linux-kernel@...r.kernel.org>, 
	Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>
Subject: Re: [GIT PULL] Kbuild fixes for v6.15-rc7

Hi Linus,

On Tue, May 27, 2025 at 3:16 AM Linus Torvalds
<torvalds@...ux-foundation.org> wrote:
>
> Masahiro,
>
> On Wed, 14 May 2025 at 22:15, Masahiro Yamada <masahiroy@...nel.org> wrote:
> >
> > Please pull some Kbuild fixes.
>
> So this isn't actually about your latest kbuild fixes, but more a
> generic kbuild thing now that the merge window has opened..
>
> My build times have gone up by a *lot* lately (and by "lately" I don't
> mean very recently, but in the last month or so), to the point that
> I'm now finding it to be a real annoyance due to it slowing down the
> merge window for me.
>
> I did a profile on the load just to check, and it's all just gcc
> (well, cc1 mainly), so it's not that something else like ccache got
> enabled.
>
> (Things like ccache can speed up builds in other scenarios, but I've
> also seen it slow things down a lot for the worst-case sitation when
> header files change, so I wanted to verify that it's not just that)
>
> So it's just that we're spending a lot more time in the compiler.
>
> I haven't tried to bisect it, and now I'm too busy to, but the two
> main suspects are either that KASAN is now enabled and that just makes
> the builds a lot slower (if so, it's my own fault - introduced by me
> in commit 6f110a5e4f99 "Disable SLUB_TINY for build testing").
>
> Or it's just that I upgraded to F42 and gcc-15 itself is perhaps just
> that much slower.
>
> Anyway, I'm emailing you exactly because I'm busy with the merge
> window and this is causing issues for me but I don't have time to
> figure it out - and so I'm going "maybe Masahiro is aware of this
> already"

I am not aware of the slowness you mentioned.

I compared v6.15-rc1 and v6.15-1972-g914873bc7df9
(today's latest), but I did not see noticable slowness.

masahiro@...ar:~/ref/linux((HEAD detached at v6.15-rc1))$ git describe
v6.15-rc1
masahiro@...ar:~/ref/linux((HEAD detached at v6.15-rc1))$ git clean
-fdx && time make -j24 defconfig all
 [ snip ]
real 1m46.444s
user 27m59.363s
sys 4m47.092s


masahiro@...ar:~/ref/linux(master)$ git describe
v6.15-1972-g914873bc7df9
masahiro@...ar:~/ref/linux(master)$ git clean -fdx && time make -j24
defconfig all
[snip]
real 1m45.410s
user 28m11.197s
sys 4m45.550s


However, building the kernel in Fedora 42 is 30% slower
than in Fedora 41.

So, maybe, this is because you recently upgraded your OS to Fedora 42.




[In Fedora 41 Docker container]

[masahiro@...de5a516cc linux]$ gcc --version | head -n1
gcc (GCC) 14.3.1 20250523 (Red Hat 14.3.1-1)
[masahiro@...de5a516cc linux]$ git clean -fdx && time make -j24 defconfig all
  [snip]
 real 2m25.794s
user 25m18.131s
sys 5m56.074s



[In Fedora 42 Docker container]

[masahiro@...a26dda811 linux]$ gcc --version | head -n1
gcc (GCC) 15.1.1 20250521 (Red Hat 15.1.1-2)
[masahiro@...a26dda811 linux]$ git clean -fdx && time make -j24 defconfig all
  [snip]
real 3m12.714s
user 35m59.299s
sys 9m10.699s







> I also did notice while timing things that the final module link time
> seems to have limited threading
>
> For the rest of the build, I have load averages in the 100+ range
> (because I build with "-j128"), but then the last few minutes it
> starts dropping a lot when it's in the
>
>   CC [M] ...
>
> stage, and when it hits that Makefile.modfinal stage and we get
>
>   LD [M] ...
>
> stage it seems to have serialized completely and just spends 100% of
> time in one single 'make' instance.
>
> But that may be an effect of the link being so fast that 'ps' and
> 'top' just don't tend to pick them out at all, of course. But I do
> note that the LD [M] phase seems to be entirely serialized with the
> 'CC [M]' phase. Does it really have to be?
>
>                Linus

The last CC[M] and LD[M] stages are very fast, because
the auto-generated *.mod.c files are tiny.



CC [M]     foo.mod.o
CC [M]     bar.mod.o

can be compiled in parallel.


LD [M]     foo.o
LD [M]     bar.o

can be linked in parallel.


So, I believe we can benefit from parallel building.



-- 
Best Regards
Masahiro Yamada