linux-kernel - Re: [PATCH 6/6] Add Propeller configuration for kernel build.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20240928173530.GC430964@thelio-3990X>
Date: Sat, 28 Sep 2024 10:35:30 -0700
From: Nathan Chancellor <nathan@...nel.org>
To: Nick Desaulniers <ndesaulniers@...gle.com>
Cc: Maksim Panchenko <max4bolt@...il.com>, Rong Xu <xur@...gle.com>,
	Han Shen <shenhan@...gle.com>,
	Sriraman Tallam <tmsriram@...gle.com>,
	David Li <davidxl@...gle.com>, Jonathan Corbet <corbet@....net>,
	Masahiro Yamada <masahiroy@...nel.org>,
	Nicolas Schier <nicolas@...sle.eu>,
	Thomas Gleixner <tglx@...utronix.de>,
	Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>,
	Dave Hansen <dave.hansen@...ux.intel.com>, x86@...nel.org,
	"H . Peter Anvin" <hpa@...or.com>, Ard Biesheuvel <ardb@...nel.org>,
	Arnd Bergmann <arnd@...db.de>, Josh Poimboeuf <jpoimboe@...nel.org>,
	Peter Zijlstra <peterz@...radead.org>,
	Bill Wendling <morbo@...gle.com>,
	Justin Stitt <justinstitt@...gle.com>,
	Vegard Nossum <vegard.nossum@...cle.com>,
	John Moon <john@...on.dev>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Heiko Carstens <hca@...ux.ibm.com>,
	Luis Chamberlain <mcgrof@...nel.org>,
	Samuel Holland <samuel.holland@...ive.com>,
	Mike Rapoport <rppt@...nel.org>,
	"Paul E . McKenney" <paulmck@...nel.org>,
	Rafael Aquini <aquini@...hat.com>, Petr Pavlu <petr.pavlu@...e.com>,
	Eric DeVolder <eric.devolder@...cle.com>,
	Bjorn Helgaas <bhelgaas@...gle.com>,
	Randy Dunlap <rdunlap@...radead.org>,
	Benjamin Segall <bsegall@...gle.com>,
	Breno Leitao <leitao@...ian.org>,
	Wei Yang <richard.weiyang@...il.com>,
	Brian Gerst <brgerst@...il.com>, Juergen Gross <jgross@...e.com>,
	Palmer Dabbelt <palmer@...osinc.com>,
	Alexandre Ghiti <alexghiti@...osinc.com>,
	Kees Cook <kees@...nel.org>,
	Sami Tolvanen <samitolvanen@...gle.com>,
	Xiao Wang <xiao.w.wang@...el.com>,
	Jan Kiszka <jan.kiszka@...mens.com>, linux-doc@...r.kernel.org,
	linux-kernel@...r.kernel.org, linux-kbuild@...r.kernel.org,
	linux-efi@...r.kernel.org, linux-arch@...r.kernel.org,
	llvm@...ts.linux.dev, Krzysztof Pszeniczny <kpszeniczny@...gle.com>,
	Stephane Eranian <eranian@...gle.com>,
	Maksim Panchenko <maks@...a.com>
Subject: Re: [PATCH 6/6] Add Propeller configuration for kernel build.

On Fri, Sep 27, 2024 at 03:45:39PM -0700, Nick Desaulniers wrote:
> On Thu, Sep 19, 2024 at 4:52 AM Maksim Panchenko <max4bolt@...il.com> wrote:
> >
> > On Sun, Jul 28, 2024 at 01:29:56PM -0700, Rong Xu wrote:
> > > Add the build support for using Clang's Propeller optimizer. Like
> > > AutoFDO, Propeller uses hardware sampling to gather information
> > > about the frequency of execution of different code paths within a
> > > binary. This information is then used to guide the compiler's
> > > optimization decisions, resulting in a more efficient binary.
> >
> > Thank you for submitting the patches with the latest compiler features.
> >
> > Regarding Propeller, I want to quickly mention that I plan to send a
> > patch to include BOLT as a profile-based post-link optimizer for the
> > kernel. I'd like it to be considered an alternative that is selectable
> > at build time.
> >
> > BOLT also uses sampling, and the profile can be collected on virtually
> > any kernel (with some caveats).  There are no constraints on the
> > compiler (i.e., any version of GCC or Clang is acceptable), while Linux
> > perf is the only external dependency used for profile collection and
> > conversion. BOLT works on top of AutoFDO and LTO but can be used without
> > them if the user desires. The build overhead is a few seconds.
> >
> > As you've heard from the LLVM discussion
> > (https://discourse.llvm.org/t/optimizing-the-linux-kernel-with-autofdo-including-thinlto-and-propeller)
> > and LPC talk (https://lpc.events/event/18/contributions/1921/), at Meta,
> > we've also successfully optimized the kernel and got similar results.
> >
> > Again, this is a heads-up before the patch, and I would like to hear
> > what people think about having a binary optimizer as a user-selectable
> > alternative to Propeller.
> 
> I'd imagine that folks would be interested in running Propeller, or
> BOLT, but perhaps not both.
> 
> In that sense, Kconfig has the means to express mutual exclusion.
> It's perhaps worth working together to get the kconfig selection
> working such that folks can play with enabling these newer toolchain
> related technologies.

Right, I would expect this to just be a Kconfig choice with a
description like "Post link optimization" or something of the sort, like
the RANDSTRUCT or DEBUG_INFO ones. If it does make sense to do them at
the same time, they can obviously be separate.

> The next instance of the bi-weekly public Clang Built Linux meeting is
> next Wednesday. (Links from https://clangbuiltlinux.github.io/)
> 
> Perhaps it's worth Rong (and Sriraman and Han) and Maksim to stop by and chat?

I would certainly be open to discussing the plans for upstreaming these
in the meeting. I think the sessions went well in the Toolchains Track.
There were no major objections from what I could tell.

Cheers,
Nathan