linux-kernel - Re: [PATCH 6/6] Add Propeller configuration for kernel build.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAKwvOdm0iZspjpuueBV1=eFt+Bf4edWBZsDsj10kEvTGZRye2w@mail.gmail.com>
Date: Fri, 27 Sep 2024 15:45:39 -0700
From: Nick Desaulniers <ndesaulniers@...gle.com>
To: Maksim Panchenko <max4bolt@...il.com>, Rong Xu <xur@...gle.com>
Cc: Han Shen <shenhan@...gle.com>, Sriraman Tallam <tmsriram@...gle.com>, 
	David Li <davidxl@...gle.com>, Jonathan Corbet <corbet@....net>, 
	Masahiro Yamada <masahiroy@...nel.org>, Nathan Chancellor <nathan@...nel.org>, 
	Nicolas Schier <nicolas@...sle.eu>, Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar <mingo@...hat.com>, 
	Borislav Petkov <bp@...en8.de>, Dave Hansen <dave.hansen@...ux.intel.com>, x86@...nel.org, 
	"H . Peter Anvin" <hpa@...or.com>, Ard Biesheuvel <ardb@...nel.org>, Arnd Bergmann <arnd@...db.de>, 
	Josh Poimboeuf <jpoimboe@...nel.org>, Peter Zijlstra <peterz@...radead.org>, 
	Bill Wendling <morbo@...gle.com>, Justin Stitt <justinstitt@...gle.com>, 
	Vegard Nossum <vegard.nossum@...cle.com>, John Moon <john@...on.dev>, 
	Andrew Morton <akpm@...ux-foundation.org>, Heiko Carstens <hca@...ux.ibm.com>, 
	Luis Chamberlain <mcgrof@...nel.org>, Samuel Holland <samuel.holland@...ive.com>, 
	Mike Rapoport <rppt@...nel.org>, "Paul E . McKenney" <paulmck@...nel.org>, Rafael Aquini <aquini@...hat.com>, 
	Petr Pavlu <petr.pavlu@...e.com>, Eric DeVolder <eric.devolder@...cle.com>, 
	Bjorn Helgaas <bhelgaas@...gle.com>, Randy Dunlap <rdunlap@...radead.org>, 
	Benjamin Segall <bsegall@...gle.com>, Breno Leitao <leitao@...ian.org>, 
	Wei Yang <richard.weiyang@...il.com>, Brian Gerst <brgerst@...il.com>, 
	Juergen Gross <jgross@...e.com>, Palmer Dabbelt <palmer@...osinc.com>, 
	Alexandre Ghiti <alexghiti@...osinc.com>, Kees Cook <kees@...nel.org>, 
	Sami Tolvanen <samitolvanen@...gle.com>, Xiao Wang <xiao.w.wang@...el.com>, 
	Jan Kiszka <jan.kiszka@...mens.com>, linux-doc@...r.kernel.org, 
	linux-kernel@...r.kernel.org, linux-kbuild@...r.kernel.org, 
	linux-efi@...r.kernel.org, linux-arch@...r.kernel.org, llvm@...ts.linux.dev, 
	Krzysztof Pszeniczny <kpszeniczny@...gle.com>, Stephane Eranian <eranian@...gle.com>, 
	Maksim Panchenko <maks@...a.com>
Subject: Re: [PATCH 6/6] Add Propeller configuration for kernel build.

On Thu, Sep 19, 2024 at 4:52 AM Maksim Panchenko <max4bolt@...il.com> wrote:
>
> On Sun, Jul 28, 2024 at 01:29:56PM -0700, Rong Xu wrote:
> > Add the build support for using Clang's Propeller optimizer. Like
> > AutoFDO, Propeller uses hardware sampling to gather information
> > about the frequency of execution of different code paths within a
> > binary. This information is then used to guide the compiler's
> > optimization decisions, resulting in a more efficient binary.
>
> Thank you for submitting the patches with the latest compiler features.
>
> Regarding Propeller, I want to quickly mention that I plan to send a
> patch to include BOLT as a profile-based post-link optimizer for the
> kernel. I'd like it to be considered an alternative that is selectable
> at build time.
>
> BOLT also uses sampling, and the profile can be collected on virtually
> any kernel (with some caveats).  There are no constraints on the
> compiler (i.e., any version of GCC or Clang is acceptable), while Linux
> perf is the only external dependency used for profile collection and
> conversion. BOLT works on top of AutoFDO and LTO but can be used without
> them if the user desires. The build overhead is a few seconds.
>
> As you've heard from the LLVM discussion
> (https://discourse.llvm.org/t/optimizing-the-linux-kernel-with-autofdo-including-thinlto-and-propeller)
> and LPC talk (https://lpc.events/event/18/contributions/1921/), at Meta,
> we've also successfully optimized the kernel and got similar results.
>
> Again, this is a heads-up before the patch, and I would like to hear
> what people think about having a binary optimizer as a user-selectable
> alternative to Propeller.

I'd imagine that folks would be interested in running Propeller, or
BOLT, but perhaps not both.

In that sense, Kconfig has the means to express mutual exclusion.
It's perhaps worth working together to get the kconfig selection
working such that folks can play with enabling these newer toolchain
related technologies.

The next instance of the bi-weekly public Clang Built Linux meeting is
next Wednesday. (Links from https://clangbuiltlinux.github.io/)

Perhaps it's worth Rong (and Sriraman and Han) and Maksim to stop by and chat?
-- 
Thanks,
~Nick Desaulniers