[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <3183ab86-8f1f-4624-9175-31e77d773699@cachyos.org>
Date: Sat, 2 Nov 2024 20:53:43 +0100
From: Peter Jung <ptr1337@...hyos.org>
To: Rong Xu <xur@...gle.com>, Alice Ryhl <aliceryhl@...gle.com>,
Andrew Morton <akpm@...ux-foundation.org>, Arnd Bergmann <arnd@...db.de>,
Bill Wendling <morbo@...gle.com>, Borislav Petkov <bp@...en8.de>,
Breno Leitao <leitao@...ian.org>, Brian Gerst <brgerst@...il.com>,
Dave Hansen <dave.hansen@...ux.intel.com>, David Li <davidxl@...gle.com>,
Han Shen <shenhan@...gle.com>, Heiko Carstens <hca@...ux.ibm.com>,
"H. Peter Anvin" <hpa@...or.com>, Ingo Molnar <mingo@...hat.com>,
Jann Horn <jannh@...gle.com>, Jonathan Corbet <corbet@....net>,
Josh Poimboeuf <jpoimboe@...nel.org>, Juergen Gross <jgross@...e.com>,
Justin Stitt <justinstitt@...gle.com>, Kees Cook <kees@...nel.org>,
Masahiro Yamada <masahiroy@...nel.org>, "Mike Rapoport (IBM)"
<rppt@...nel.org>, Nathan Chancellor <nathan@...nel.org>,
Nick Desaulniers <ndesaulniers@...gle.com>,
Nicolas Schier <nicolas@...sle.eu>, "Paul E. McKenney" <paulmck@...nel.org>,
Peter Zijlstra <peterz@...radead.org>,
Sami Tolvanen <samitolvanen@...gle.com>, Thomas Gleixner
<tglx@...utronix.de>, Wei Yang <richard.weiyang@...il.com>,
workflows@...r.kernel.org, Miguel Ojeda <miguel.ojeda.sandonis@...il.com>,
Maksim Panchenko <max4bolt@...il.com>, "David S. Miller"
<davem@...emloft.net>, Andreas Larsson <andreas@...sler.com>,
Yonghong Song <yonghong.song@...ux.dev>, Yabin Cui <yabinc@...gle.com>,
Krzysztof Pszeniczny <kpszeniczny@...gle.com>,
Sriraman Tallam <tmsriram@...gle.com>, Stephane Eranian <eranian@...gle.com>
Cc: x86@...nel.org, linux-arch@...r.kernel.org, sparclinux@...r.kernel.org,
linux-doc@...r.kernel.org, linux-kbuild@...r.kernel.org,
linux-kernel@...r.kernel.org, llvm@...ts.linux.dev
Subject: Re: [PATCH v7 1/7] Add AutoFDO support for Clang build
On 02.11.24 20:46, Peter Jung wrote:
>
>
> On 02.11.24 18:51, Rong Xu wrote:
>> Add the build support for using Clang's AutoFDO. Building the kernel
>> with AutoFDO does not reduce the optimization level from the
>> compiler. AutoFDO uses hardware sampling to gather information about
>> the frequency of execution of different code paths within a binary.
>> This information is then used to guide the compiler's optimization
>> decisions, resulting in a more efficient binary. Experiments
>> showed that the kernel can improve up to 10% in latency.
>>
>> The support requires a Clang compiler after LLVM 17. This submission
>> is limited to x86 platforms that support PMU features like LBR on
>> Intel machines and AMD Zen3 BRS. Support for SPE on ARM 1,
>> and BRBE on ARM 1 is part of planned future work.
>>
>> Here is an example workflow for AutoFDO kernel:
>>
>> 1) Build the kernel on the host machine with LLVM enabled, for example,
>> $ make menuconfig LLVM=1
>> Turn on AutoFDO build config:
>> CONFIG_AUTOFDO_CLANG=y
>> With a configuration that has LLVM enabled, use the following
>> command:
>> scripts/config -e AUTOFDO_CLANG
>> After getting the config, build with
>> $ make LLVM=1
>>
>> 2) Install the kernel on the test machine.
>>
>> 3) Run the load tests. The '-c' option in perf specifies the sample
>> event period. We suggest using a suitable prime number,
>> like 500009, for this purpose.
>> For Intel platforms:
>> $ perf record -e BR_INST_RETIRED.NEAR_TAKEN:k -a -N -b -c
>> <count> \
>> -o <perf_file> -- <loadtest>
>> For AMD platforms:
>> The supported system are: Zen3 with BRS, or Zen4 with amd_lbr_v2
>> For Zen3:
>> $ cat proc/cpuinfo | grep " brs"
>> For Zen4:
>> $ cat proc/cpuinfo | grep amd_lbr_v2
>> $ perf record --pfm-events RETIRED_TAKEN_BRANCH_INSTRUCTIONS:k
>> -a \
>> -N -b -c <count> -o <perf_file> -- <loadtest>
>>
>> 4) (Optional) Download the raw perf file to the host machine.
>>
>> 5) To generate an AutoFDO profile, two offline tools are available:
>> create_llvm_prof and llvm_profgen. The create_llvm_prof tool is part
>> of the AutoFDO project and can be found on GitHub
>> (https://github.com/google/autofdo), version v0.30.1 or later. The
>> llvm_profgen tool is included in the LLVM compiler itself. It's
>> important to note that the version of llvm_profgen doesn't need to
>> match the version of Clang. It needs to be the LLVM 19 release or
>> later, or from the LLVM trunk.
>> $ llvm-profgen --kernel --binary=<vmlinux> --
>> perfdata=<perf_file> \
>> -o <profile_file>
>> or
>> $ create_llvm_prof --binary=<vmlinux> --profile=<perf_file> \
>> --format=extbinary --out=<profile_file>
>>
>> Note that multiple AutoFDO profile files can be merged into one via:
>> $ llvm-profdata merge -o <profile_file> <profile_1> ...
>> <profile_n>
>>
>> 6) Rebuild the kernel using the AutoFDO profile file with the same config
>> as step 1, (Note CONFIG_AUTOFDO_CLANG needs to be enabled):
>> $ make LLVM=1 CLANG_AUTOFDO_PROFILE=<profile_file>
>>
>> Co-developed-by: Han Shen<shenhan@...gle.com>
>> Signed-off-by: Han Shen<shenhan@...gle.com>
>> Signed-off-by: Rong Xu<xur@...gle.com>
>> Suggested-by: Sriraman Tallam<tmsriram@...gle.com>
>> Suggested-by: Krzysztof Pszeniczny<kpszeniczny@...gle.com>
>> Suggested-by: Nick Desaulniers<ndesaulniers@...gle.com>
>> Suggested-by: Stephane Eranian<eranian@...gle.com>
>> Tested-by: Yonghong Song<yonghong.song@...ux.dev>
>> Tested-by: Yabin Cui<yabinc@...gle.com>
>> Tested-by: Nathan Chancellor<nathan@...nel.org>
>> Reviewed-by: Kees Cook<kees@...nel.org>
>
> Tested-by: Peter Jung <ptr1337@...hyos.org>
>
The compilations and testing with the "make pacman-pkg" function from
the kernel worked fine.
One problem I do face:
When I apply a AutoFDO profile together with the PKGBUILD [1] from
archlinux im running into issues at "module_install" at the packaging.
See following log:
```
make[2]: *** [scripts/Makefile.modinst:125:
/tmp/makepkg/linux-cachyos-rc-autofdo/pkg/linux-cachyos-rc-autofdo/usr/lib/modules/6.12.0-rc5-5-cachyos-rc-autofdo/kernel/arch/x86/kvm/kvm.ko]
Error 1
make[2]: *** Deleting file
'/tmp/makepkg/linux-cachyos-rc-autofdo/pkg/linux-cachyos-rc-autofdo/usr/lib/modules/6.12.0-rc5-5-cachyos-rc-autofdo/kernel/arch/x86/kvm/kvm.ko'
INSTALL
/tmp/makepkg/linux-cachyos-rc-autofdo/pkg/linux-cachyos-rc-autofdo/usr/lib/modules/6.12.0-rc5-5-cachyos-rc-autofdo/kernel/crypto/cryptd.ko
make[2]: *** Waiting for unfinished jobs....
```
This can be fixed with removed "INSTALL_MOD_STRIP=1" to the passed
parameters of module_install.
This explicitly only happens, if a profile is passed - otherwise the
packaging works without problems.
Regards,
Peter Jung
Powered by blists - more mailing lists