linux-kernel - Re: [PATCH v6 7/7] Add Propeller configuration for kernel build

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <202410281714.20A7BE8@keescook>
Date: Mon, 28 Oct 2024 17:14:59 -0700
From: Kees Cook <kees@...nel.org>
To: Rong Xu <xur@...gle.com>
Cc: Alice Ryhl <aliceryhl@...gle.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Arnd Bergmann <arnd@...db.de>, Bill Wendling <morbo@...gle.com>,
	Borislav Petkov <bp@...en8.de>, Breno Leitao <leitao@...ian.org>,
	Brian Gerst <brgerst@...il.com>,
	Dave Hansen <dave.hansen@...ux.intel.com>,
	David Li <davidxl@...gle.com>, Han Shen <shenhan@...gle.com>,
	Heiko Carstens <hca@...ux.ibm.com>,
	"H. Peter Anvin" <hpa@...or.com>, Ingo Molnar <mingo@...hat.com>,
	Jann Horn <jannh@...gle.com>, Jonathan Corbet <corbet@....net>,
	Josh Poimboeuf <jpoimboe@...nel.org>,
	Juergen Gross <jgross@...e.com>,
	Justin Stitt <justinstitt@...gle.com>,
	Masahiro Yamada <masahiroy@...nel.org>,
	"Mike Rapoport (IBM)" <rppt@...nel.org>,
	Nathan Chancellor <nathan@...nel.org>,
	Nick Desaulniers <ndesaulniers@...gle.com>,
	Nicolas Schier <nicolas@...sle.eu>,
	"Paul E. McKenney" <paulmck@...nel.org>,
	Peter Zijlstra <peterz@...radead.org>,
	Sami Tolvanen <samitolvanen@...gle.com>,
	Thomas Gleixner <tglx@...utronix.de>,
	Wei Yang <richard.weiyang@...il.com>, workflows@...r.kernel.org,
	Miguel Ojeda <miguel.ojeda.sandonis@...il.com>,
	Maksim Panchenko <max4bolt@...il.com>,
	"David S. Miller" <davem@...emloft.net>,
	Andreas Larsson <andreas@...sler.com>,
	Yonghong Song <yonghong.song@...ux.dev>,
	Yabin Cui <yabinc@...gle.com>,
	Krzysztof Pszeniczny <kpszeniczny@...gle.com>,
	Sriraman Tallam <tmsriram@...gle.com>,
	Stephane Eranian <eranian@...gle.com>, x86@...nel.org,
	linux-arch@...r.kernel.org, sparclinux@...r.kernel.org,
	linux-doc@...r.kernel.org, linux-kbuild@...r.kernel.org,
	linux-kernel@...r.kernel.org, llvm@...ts.linux.dev
Subject: Re: [PATCH v6 7/7] Add Propeller configuration for kernel build

On Fri, Oct 25, 2024 at 10:14:09PM -0700, Rong Xu wrote:
> Add the build support for using Clang's Propeller optimizer. Like
> AutoFDO, Propeller uses hardware sampling to gather information
> about the frequency of execution of different code paths within a
> binary. This information is then used to guide the compiler's
> optimization decisions, resulting in a more efficient binary.
> 
> The support requires a Clang compiler LLVM 19 or later, and the
> create_llvm_prof tool
> (https://github.com/google/autofdo/releases/tag/v0.30.1). This
> commit is limited to x86 platforms that support PMU features
> like LBR on Intel machines and AMD Zen3 BRS.
> 
> Here is an example workflow for building an AutoFDO+Propeller
> optimized kernel:
> 
> 1) Build the kernel on the host machine, with AutoFDO and Propeller
>    build config
>       CONFIG_AUTOFDO_CLANG=y
>       CONFIG_PROPELLER_CLANG=y
>    then
>       $ make LLVM=1 CLANG_AUTOFDO_PROFILE=<autofdo_profile>
> 
> “<autofdo_profile>” is the profile collected when doing a non-Propeller
> AutoFDO build. This step builds a kernel that has the same optimization
> level as AutoFDO, plus a metadata section that records basic block
> information. This kernel image runs as fast as an AutoFDO optimized
> kernel.
> 
> 2) Install the kernel on test/production machines.
> 
> 3) Run the load tests. The '-c' option in perf specifies the sample
>    event period. We suggest using a suitable prime number,
>    like 500009, for this purpose.
>    For Intel platforms:
>       $ perf record -e BR_INST_RETIRED.NEAR_TAKEN:k -a -N -b -c <count> \
>         -o <perf_file> -- <loadtest>
>    For AMD platforms:
>       The supported system are: Zen3 with BRS, or Zen4 with amd_lbr_v2
>       # To see if Zen3 support LBR:
>       $ cat proc/cpuinfo | grep " brs"
>       # To see if Zen4 support LBR:
>       $ cat proc/cpuinfo | grep amd_lbr_v2
>       # If the result is yes, then collect the profile using:
>       $ perf record --pfm-events RETIRED_TAKEN_BRANCH_INSTRUCTIONS:k -a \
>         -N -b -c <count> -o <perf_file> -- <loadtest>
> 
> 4) (Optional) Download the raw perf file to the host machine.
> 
> 5) Generate Propeller profile:
>    $ create_llvm_prof --binary=<vmlinux> --profile=<perf_file> \
>      --format=propeller --propeller_output_module_name \
>      --out=<propeller_profile_prefix>_cc_profile.txt \
>      --propeller_symorder=<propeller_profile_prefix>_ld_profile.txt
> 
>    “create_llvm_prof” is the profile conversion tool, and a prebuilt
>    binary for linux can be found on
>    https://github.com/google/autofdo/releases/tag/v0.30.1 (can also build
>    from source).
> 
>    "<propeller_profile_prefix>" can be something like
>    "/home/user/dir/any_string".
> 
>    This command generates a pair of Propeller profiles:
>    "<propeller_profile_prefix>_cc_profile.txt" and
>    "<propeller_profile_prefix>_ld_profile.txt".
> 
> 6) Rebuild the kernel using the AutoFDO and Propeller profile files.
>       CONFIG_AUTOFDO_CLANG=y
>       CONFIG_PROPELLER_CLANG=y
>    and
>       $ make LLVM=1 CLANG_AUTOFDO_PROFILE=<autofdo_profile> \
>         CLANG_PROPELLER_PROFILE_PREFIX=<propeller_profile_prefix>
> 
> Co-developed-by: Han Shen <shenhan@...gle.com>
> Signed-off-by: Han Shen <shenhan@...gle.com>

Looks good. Similarly isolated like FDO. :)

Reviewed-by: Kees Cook <kees@...nel.org>

-- 
Kees Cook