linux-kernel - Re: [PATCH 0/7] minmax: reduce compilation time

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <36aa2cad-1db1-4abf-8dd2-fb20484aabc3@lucifer.local>
Date: Fri, 26 Jul 2024 20:21:31 +0100
From: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>
To: Linus Torvalds <torvalds@...uxfoundation.org>
Cc: David Laight <David.Laight@...lab.com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "Matthew Wilcox (Oracle)" <willy@...radead.org>,
        Christoph Hellwig <hch@...radead.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Andy Shevchenko <andriy.shevchenko@...ux.intel.com>,
        Dan Carpenter <dan.carpenter@...aro.org>,
        Arnd Bergmann <arnd@...nel.org>, "Jason@...c4.com" <Jason@...c4.com>,
        "pedro.falcato@...il.com" <pedro.falcato@...il.com>,
        Mateusz Guzik <mjguzik@...il.com>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>
Subject: Re: [PATCH 0/7] minmax: reduce compilation time

On Fri, Jul 26, 2024 at 07:56:31PM GMT, Lorenzo Stoakes wrote:
> On Fri, Jul 26, 2024 at 11:24:09AM GMT, Linus Torvalds wrote:
> > On Fri, 26 Jul 2024 at 11:13, Lorenzo Stoakes
> > <lorenzo.stoakes@...cle.com> wrote:
> > >
> > > 5,447,539       ./arch/x86/xen/setup.o.pre
> >
> > Can you perhaps do some kind of "max expansion" on all the
> > preprocessor files (you seem to have done it by changing the ".c.o"
> > rule to just spit it out as "o.pre", which sounds fine).
>
> Yeah I simply hacked in a gcc -E, ugly but effective, e.g. prepending
> cmd_cc_o_c in scripts/Makefile.build with:
>
> cmd_cc_o_c = $(CC) $(c_flags) -E $< > $@...e; \
>
> (with credit to Arnd, who came up with a timing patch using a similar
> technique, I just super trivially adapted to do this instead).
>
> I therefore generate a bunch of .pre files, + run this to get total
> numbers:
>
> find . -iname '*.pre' -exec du -cb {} + | grep total$ | cut -f1 | paste -sd+ - | bc
>
> So I have the numbers for all of the files. Did you want me to post the
> numbers somewhere, or were you after someting cleverer, not sure what you
> mean by "max expansion"?
>
> >
> > For example, this trivial patch seems to fix the setup.c expansion by
> > about an order of magnitude (ie 50M -> 5M).
> >
> > Entirely untested, but looks ObviouslyCorrect(tm) to me.
> >
> >                      Linus
> >
> >                 Linus
>
> Yeah this has a big impact, with just this patch we go from 51,407,944 to
> 4,351,747 locally for me, so match your results.
>
> Patch looks obviously correct to me, I wonder if we should just apply that
> now and take out one of the worst offenders while we figure out how to fix
> the broader problem?
>
> Reviewed-by: Lorenzo Stoakes <lorenzo.stoakes@...cle.com> fwiw on that :)

A simple comparison pre-revert vs. post-revert gives some ideas for other
low-hanging fruit:

  389180./drivers/block/loop.o.pre
  409741./net/ipv4/tcp_output.o.pre
  462230./net/ipv4/tcp_input.o.pre
  470714./mm/page_alloc.o.pre
  570698./block/bio.o.pre
  600206./net/ipv6/proc.o.pre
  761333./arch/x86/kernel/cpu/cacheinfo.o.pre
  1315265./drivers/gpu/drm/i915/display/intel_pmdemand.o.pre
  1334256./mm/compaction.o.pre
  45960405./arch/x86/xen/setup.o.pre

I might try an allmodconfig with David's patches to see if that pulls out any
other really obvious cases.