| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CACkGtrgJHtG5pXR1z=6G4XR6ffT5jEi3jZQo=UhYj091naBhsA@mail.gmail.com>
Date: Mon, 4 Nov 2024 12:24:29 -0800
From: Han Shen <shenhan@...gle.com>
To: Peter Jung <ptr1337@...hyos.org>
Cc: Rong Xu <xur@...gle.com>, Alice Ryhl <aliceryhl@...gle.com>,
Andrew Morton <akpm@...ux-foundation.org>, Arnd Bergmann <arnd@...db.de>,
Bill Wendling <morbo@...gle.com>, Borislav Petkov <bp@...en8.de>, Breno Leitao <leitao@...ian.org>,
Brian Gerst <brgerst@...il.com>, Dave Hansen <dave.hansen@...ux.intel.com>,
David Li <davidxl@...gle.com>, Heiko Carstens <hca@...ux.ibm.com>, "H. Peter Anvin" <hpa@...or.com>,
Ingo Molnar <mingo@...hat.com>, Jann Horn <jannh@...gle.com>, Jonathan Corbet <corbet@....net>,
Josh Poimboeuf <jpoimboe@...nel.org>, Juergen Gross <jgross@...e.com>,
Justin Stitt <justinstitt@...gle.com>, Kees Cook <kees@...nel.org>,
Masahiro Yamada <masahiroy@...nel.org>, "Mike Rapoport (IBM)" <rppt@...nel.org>,
Nathan Chancellor <nathan@...nel.org>, Nick Desaulniers <ndesaulniers@...gle.com>,
Nicolas Schier <nicolas@...sle.eu>, "Paul E. McKenney" <paulmck@...nel.org>,
Peter Zijlstra <peterz@...radead.org>, Sami Tolvanen <samitolvanen@...gle.com>,
Thomas Gleixner <tglx@...utronix.de>, Wei Yang <richard.weiyang@...il.com>,
workflows@...r.kernel.org, Miguel Ojeda <miguel.ojeda.sandonis@...il.com>,
Maksim Panchenko <max4bolt@...il.com>, "David S. Miller" <davem@...emloft.net>,
Andreas Larsson <andreas@...sler.com>, Yonghong Song <yonghong.song@...ux.dev>,
Yabin Cui <yabinc@...gle.com>, Krzysztof Pszeniczny <kpszeniczny@...gle.com>,
Sriraman Tallam <tmsriram@...gle.com>, Stephane Eranian <eranian@...gle.com>, x86@...nel.org,
linux-arch@...r.kernel.org, sparclinux@...r.kernel.org,
linux-doc@...r.kernel.org, linux-kbuild@...r.kernel.org,
linux-kernel@...r.kernel.org, llvm@...ts.linux.dev
Subject: Re: [PATCH v7 1/7] Add AutoFDO support for Clang build
Hi Peter,
Thanks for providing the detailed reproduce.
Now I can see the error (after I synced to 6.12.0-rc6, I was using rc5).
I'll look into that and report back.
> I have tested your provided method, but the AutoFDO profile (lld does
not get lto-sample-profile=$pathtoprofile passed)
I see. You also turned on ThinLTO, which I didn't, so the profile was
only used during compilation, not passed to lld.
Thanks,
Han
On Mon, Nov 4, 2024 at 9:31 AM Peter Jung <ptr1337@...hyos.org> wrote:
>
> Hi Han,
>
> I have tested your provided method, but the AutoFDO profile (lld does
> not get lto-sample-profile=$pathtoprofile passed) nor Clang as compiler
> gets used.
> Please replace following PKGBUILD and config from linux-mainline with
> the provided one in the gist. The patch is also included there.
>
> https://gist.github.com/ptr1337/c92728bb273f7dbc2817db75eedec9ed
>
> The main change I am doing here, is passing following to the build array
> and replacing "make all":
>
> make LLVM=1 LLVM_IAS=1 CLANG_AUTOFDO_PROFILE=${srcdir}/perf.afdo all
>
> When compiling the kernel with makepkg, this results at the packaging to
> following issue and can be reliable reproduced.
>
> Regards,
>
> Peter
>
>
> On 04.11.24 05:50, Han Shen wrote:
> > Hi Peter, thanks for reporting the issue. I am trying to reproduce it
> > in the up-to-date archlinux environment. Below is what I have:
> > 0. pacman -Syu
> > 1. cloned archlinux build files from
> > https://aur.archlinux.org/linux-mainline.git the newest mainline
> > version is 6.12rc5-1.
> > 2. changed the PKGBUILD file to include the patches series
> > 3. changed the "config" to turn on clang autofdo
> > 4. collected afdo profiles
> > 5. MAKEFLAGS="-j48 V=1 LLVM=1 CLANG_AUTOFDO_PROFILE=$(pwd)/perf.afdo" \
> > makepkg -s --skipinteg --skippgp
> > 6. install and reboot
> > The above steps succeeded.
> > You mentioned the error happens at "module_install", can you instruct
> > me how to execute the "module_install" step?
> >
> > Thanks,
> > Han
> >
> > On Sat, Nov 2, 2024 at 12:53 PM Peter Jung<ptr1337@...hyos.org> wrote:
> >>
> >>
> >> On 02.11.24 20:46, Peter Jung wrote:
> >>>
> >>> On 02.11.24 18:51, Rong Xu wrote:
> >>>> Add the build support for using Clang's AutoFDO. Building the kernel
> >>>> with AutoFDO does not reduce the optimization level from the
> >>>> compiler. AutoFDO uses hardware sampling to gather information about
> >>>> the frequency of execution of different code paths within a binary.
> >>>> This information is then used to guide the compiler's optimization
> >>>> decisions, resulting in a more efficient binary. Experiments
> >>>> showed that the kernel can improve up to 10% in latency.
> >>>>
> >>>> The support requires a Clang compiler after LLVM 17. This submission
> >>>> is limited to x86 platforms that support PMU features like LBR on
> >>>> Intel machines and AMD Zen3 BRS. Support for SPE on ARM 1,
> >>>> and BRBE on ARM 1 is part of planned future work.
> >>>>
> >>>> Here is an example workflow for AutoFDO kernel:
> >>>>
> >>>> 1) Build the kernel on the host machine with LLVM enabled, for example,
> >>>> $ make menuconfig LLVM=1
> >>>> Turn on AutoFDO build config:
> >>>> CONFIG_AUTOFDO_CLANG=y
> >>>> With a configuration that has LLVM enabled, use the following
> >>>> command:
> >>>> scripts/config -e AUTOFDO_CLANG
> >>>> After getting the config, build with
> >>>> $ make LLVM=1
> >>>>
> >>>> 2) Install the kernel on the test machine.
> >>>>
> >>>> 3) Run the load tests. The '-c' option in perf specifies the sample
> >>>> event period. We suggest using a suitable prime number,
> >>>> like 500009, for this purpose.
> >>>> For Intel platforms:
> >>>> $ perf record -e BR_INST_RETIRED.NEAR_TAKEN:k -a -N -b -c
> >>>> <count> \
> >>>> -o <perf_file> -- <loadtest>
> >>>> For AMD platforms:
> >>>> The supported system are: Zen3 with BRS, or Zen4 with amd_lbr_v2
> >>>> For Zen3:
> >>>> $ cat proc/cpuinfo | grep " brs"
> >>>> For Zen4:
> >>>> $ cat proc/cpuinfo | grep amd_lbr_v2
> >>>> $ perf record --pfm-events RETIRED_TAKEN_BRANCH_INSTRUCTIONS:k
> >>>> -a \
> >>>> -N -b -c <count> -o <perf_file> -- <loadtest>
> >>>>
> >>>> 4) (Optional) Download the raw perf file to the host machine.
> >>>>
> >>>> 5) To generate an AutoFDO profile, two offline tools are available:
> >>>> create_llvm_prof and llvm_profgen. The create_llvm_prof tool is part
> >>>> of the AutoFDO project and can be found on GitHub
> >>>> (https://github.com/google/autofdo), version v0.30.1 or later. The
> >>>> llvm_profgen tool is included in the LLVM compiler itself. It's
> >>>> important to note that the version of llvm_profgen doesn't need to
> >>>> match the version of Clang. It needs to be the LLVM 19 release or
> >>>> later, or from the LLVM trunk.
> >>>> $ llvm-profgen --kernel --binary=<vmlinux> --
> >>>> perfdata=<perf_file> \
> >>>> -o <profile_file>
> >>>> or
> >>>> $ create_llvm_prof --binary=<vmlinux> --profile=<perf_file> \
> >>>> --format=extbinary --out=<profile_file>
> >>>>
> >>>> Note that multiple AutoFDO profile files can be merged into one via:
> >>>> $ llvm-profdata merge -o <profile_file> <profile_1> ...
> >>>> <profile_n>
> >>>>
> >>>> 6) Rebuild the kernel using the AutoFDO profile file with the same config
> >>>> as step 1, (Note CONFIG_AUTOFDO_CLANG needs to be enabled):
> >>>> $ make LLVM=1 CLANG_AUTOFDO_PROFILE=<profile_file>
> >>>>
> >>>> Co-developed-by: Han Shen<shenhan@...gle.com>
> >>>> Signed-off-by: Han Shen<shenhan@...gle.com>
> >>>> Signed-off-by: Rong Xu<xur@...gle.com>
> >>>> Suggested-by: Sriraman Tallam<tmsriram@...gle.com>
> >>>> Suggested-by: Krzysztof Pszeniczny<kpszeniczny@...gle.com>
> >>>> Suggested-by: Nick Desaulniers<ndesaulniers@...gle.com>
> >>>> Suggested-by: Stephane Eranian<eranian@...gle.com>
> >>>> Tested-by: Yonghong Song<yonghong.song@...ux.dev>
> >>>> Tested-by: Yabin Cui<yabinc@...gle.com>
> >>>> Tested-by: Nathan Chancellor<nathan@...nel.org>
> >>>> Reviewed-by: Kees Cook<kees@...nel.org>
> >>> Tested-by: Peter Jung<ptr1337@...hyos.org>
> >>>
> >> The compilations and testing with the "make pacman-pkg" function from
> >> the kernel worked fine.
> >>
> >> One problem I do face:
> >> When I apply a AutoFDO profile together with the PKGBUILD [1] from
> >> archlinux im running into issues at "module_install" at the packaging.
> >>
> >> See following log:
> >> ```
> >> make[2]: *** [scripts/Makefile.modinst:125:
> >> /tmp/makepkg/linux-cachyos-rc-autofdo/pkg/linux-cachyos-rc-autofdo/usr/lib/modules/6.12.0-rc5-5-cachyos-rc-autofdo/kernel/arch/x86/kvm/kvm.ko]
> >> Error 1
> >> make[2]: *** Deleting file
> >> '/tmp/makepkg/linux-cachyos-rc-autofdo/pkg/linux-cachyos-rc-autofdo/usr/lib/modules/6.12.0-rc5-5-cachyos-rc-autofdo/kernel/arch/x86/kvm/kvm.ko'
> >> INSTALL
> >> /tmp/makepkg/linux-cachyos-rc-autofdo/pkg/linux-cachyos-rc-autofdo/usr/lib/modules/6.12.0-rc5-5-cachyos-rc-autofdo/kernel/crypto/cryptd.ko
> >> make[2]: *** Waiting for unfinished jobs....
> >> ```
> >>
> >>
> >> This can be fixed with removed "INSTALL_MOD_STRIP=1" to the passed
> >> parameters of module_install.
> >>
> >> This explicitly only happens, if a profile is passed - otherwise the
> >> packaging works without problems.
> >>
> >> Regards,
> >>
> >> Peter Jung
> >>
>
Powered by blists - more mailing lists