linux-kernel - Re: [PATCH 6/6] Add Propeller configuration for kernel build.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAF1bQ=Ts1yoStfvsG-Q2BQmuRttTw03Yd4PnAU5MGQMCRRtbvw@mail.gmail.com>
Date: Mon, 29 Jul 2024 12:04:22 -0700
From: Rong Xu <xur@...gle.com>
To: Peter Zijlstra <peterz@...radead.org>
Cc: Han Shen <shenhan@...gle.com>, Sriraman Tallam <tmsriram@...gle.com>, 
	David Li <davidxl@...gle.com>, Jonathan Corbet <corbet@....net>, 
	Masahiro Yamada <masahiroy@...nel.org>, Nathan Chancellor <nathan@...nel.org>, 
	Nicolas Schier <nicolas@...sle.eu>, Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar <mingo@...hat.com>, 
	Borislav Petkov <bp@...en8.de>, Dave Hansen <dave.hansen@...ux.intel.com>, x86@...nel.org, 
	"H . Peter Anvin" <hpa@...or.com>, Ard Biesheuvel <ardb@...nel.org>, Arnd Bergmann <arnd@...db.de>, 
	Josh Poimboeuf <jpoimboe@...nel.org>, Nick Desaulniers <ndesaulniers@...gle.com>, 
	Bill Wendling <morbo@...gle.com>, Justin Stitt <justinstitt@...gle.com>, 
	Vegard Nossum <vegard.nossum@...cle.com>, John Moon <john@...on.dev>, 
	Andrew Morton <akpm@...ux-foundation.org>, Heiko Carstens <hca@...ux.ibm.com>, 
	Luis Chamberlain <mcgrof@...nel.org>, Samuel Holland <samuel.holland@...ive.com>, 
	Mike Rapoport <rppt@...nel.org>, "Paul E . McKenney" <paulmck@...nel.org>, Rafael Aquini <aquini@...hat.com>, 
	Petr Pavlu <petr.pavlu@...e.com>, Eric DeVolder <eric.devolder@...cle.com>, 
	Bjorn Helgaas <bhelgaas@...gle.com>, Randy Dunlap <rdunlap@...radead.org>, 
	Benjamin Segall <bsegall@...gle.com>, Breno Leitao <leitao@...ian.org>, 
	Wei Yang <richard.weiyang@...il.com>, Brian Gerst <brgerst@...il.com>, 
	Juergen Gross <jgross@...e.com>, Palmer Dabbelt <palmer@...osinc.com>, 
	Alexandre Ghiti <alexghiti@...osinc.com>, Kees Cook <kees@...nel.org>, 
	Sami Tolvanen <samitolvanen@...gle.com>, Xiao Wang <xiao.w.wang@...el.com>, 
	Jan Kiszka <jan.kiszka@...mens.com>, linux-doc@...r.kernel.org, 
	linux-kernel@...r.kernel.org, linux-kbuild@...r.kernel.org, 
	linux-efi@...r.kernel.org, linux-arch@...r.kernel.org, llvm@...ts.linux.dev, 
	Krzysztof Pszeniczny <kpszeniczny@...gle.com>, Stephane Eranian <eranian@...gle.com>
Subject: Re: [PATCH 6/6] Add Propeller configuration for kernel build.

On Mon, Jul 29, 2024 at 2:53 AM Peter Zijlstra <peterz@...radead.org> wrote:
>
> On Sun, Jul 28, 2024 at 01:29:59PM -0700, Rong Xu wrote:
> > Add the build support for using Clang's Propeller optimizer. Like
> > AutoFDO, Propeller uses hardware sampling to gather information
> > about the frequency of execution of different code paths within a
> > binary. This information is then used to guide the compiler's
> > optimization decisions, resulting in a more efficient binary.
> >
> > The support requires a Clang compiler LLVM 19 or later, and the
> > create_llvm_prof tool
> > (https://github.com/google/autofdo/releases/tag/v0.30.1). This
>
> What's the relation between this and llvm-profgen? Is the above simply
> a google 'internal' proof of concept thing that will eventually make its
> way into llvm-profgen?
>
> It seems a bit weird LLVM landed propeller without the required profile
> generation tool.

AutoFDO historically required a third-party tool
(create_llvm_prof, mentioned above) in Clang.
AutoFDO in GCC also requires this tool (same source, different name).

llvm-profgen is a relatively new tool that was integrated directly into LLVM
(similar purpose, but different implementation).

AutoFDO and create_llvm_prof are not "proof of concept". They have been widely
used in Google for many years. Propeller currently uses create_llvm_prof as the
profile converting tool. But there is an effort to move to LLVM.