linux-kernel - Re: [PATCH 0/7] minmax: reduce compilation time

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <137646a7-7017-490d-be78-5bd5627609c3@lucifer.local>
Date: Fri, 26 Jul 2024 19:56:31 +0100
From: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>
To: Linus Torvalds <torvalds@...uxfoundation.org>
Cc: David Laight <David.Laight@...lab.com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "Matthew Wilcox (Oracle)" <willy@...radead.org>,
        Christoph Hellwig <hch@...radead.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Andy Shevchenko <andriy.shevchenko@...ux.intel.com>,
        Dan Carpenter <dan.carpenter@...aro.org>,
        Arnd Bergmann <arnd@...nel.org>, "Jason@...c4.com" <Jason@...c4.com>,
        "pedro.falcato@...il.com" <pedro.falcato@...il.com>,
        Mateusz Guzik <mjguzik@...il.com>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>
Subject: Re: [PATCH 0/7] minmax: reduce compilation time

On Fri, Jul 26, 2024 at 11:24:09AM GMT, Linus Torvalds wrote:
> On Fri, 26 Jul 2024 at 11:13, Lorenzo Stoakes
> <lorenzo.stoakes@...cle.com> wrote:
> >
> > 5,447,539       ./arch/x86/xen/setup.o.pre
>
> Can you perhaps do some kind of "max expansion" on all the
> preprocessor files (you seem to have done it by changing the ".c.o"
> rule to just spit it out as "o.pre", which sounds fine).

Yeah I simply hacked in a gcc -E, ugly but effective, e.g. prepending
cmd_cc_o_c in scripts/Makefile.build with:

cmd_cc_o_c = $(CC) $(c_flags) -E $< > $@...e; \

(with credit to Arnd, who came up with a timing patch using a similar
technique, I just super trivially adapted to do this instead).

I therefore generate a bunch of .pre files, + run this to get total
numbers:

find . -iname '*.pre' -exec du -cb {} + | grep total$ | cut -f1 | paste -sd+ - | bc

So I have the numbers for all of the files. Did you want me to post the
numbers somewhere, or were you after someting cleverer, not sure what you
mean by "max expansion"?

>
> For example, this trivial patch seems to fix the setup.c expansion by
> about an order of magnitude (ie 50M -> 5M).
>
> Entirely untested, but looks ObviouslyCorrect(tm) to me.
>
>                      Linus
>
>                 Linus

Yeah this has a big impact, with just this patch we go from 51,407,944 to
4,351,747 locally for me, so match your results.

Patch looks obviously correct to me, I wonder if we should just apply that
now and take out one of the worst offenders while we figure out how to fix
the broader problem?

Reviewed-by: Lorenzo Stoakes <lorenzo.stoakes@...cle.com> fwiw on that :)