Message-ID: <CAF1bQ=SRnSP9mgnyRw+Hg=0-CX-uOwKmsiwHf6b2bFXKnWxPHw@mail.gmail.com>
Date: Thu, 7 Nov 2024 10:44:21 -0800
From: Rong Xu <xur@...gle.com>
To: Masahiro Yamada <masahiroy@...nel.org>
Cc: Alice Ryhl <aliceryhl@...gle.com>, Andrew Morton <akpm@...ux-foundation.org>, 
	Arnd Bergmann <arnd@...db.de>, Bill Wendling <morbo@...gle.com>, Borislav Petkov <bp@...en8.de>, 
	Breno Leitao <leitao@...ian.org>, Brian Gerst <brgerst@...il.com>, 
	Dave Hansen <dave.hansen@...ux.intel.com>, David Li <davidxl@...gle.com>, 
	Han Shen <shenhan@...gle.com>, Heiko Carstens <hca@...ux.ibm.com>, "H. Peter Anvin" <hpa@...or.com>, 
	Ingo Molnar <mingo@...hat.com>, Jann Horn <jannh@...gle.com>, Jonathan Corbet <corbet@....net>, 
	Josh Poimboeuf <jpoimboe@...nel.org>, Juergen Gross <jgross@...e.com>, 
	Justin Stitt <justinstitt@...gle.com>, Kees Cook <kees@...nel.org>, 
	"Mike Rapoport (IBM)" <rppt@...nel.org>, Nathan Chancellor <nathan@...nel.org>, 
	Nick Desaulniers <ndesaulniers@...gle.com>, Nicolas Schier <nicolas@...sle.eu>, 
	"Paul E. McKenney" <paulmck@...nel.org>, Peter Zijlstra <peterz@...radead.org>, 
	Sami Tolvanen <samitolvanen@...gle.com>, Thomas Gleixner <tglx@...utronix.de>, 
	Wei Yang <richard.weiyang@...il.com>, workflows@...r.kernel.org, 
	Miguel Ojeda <miguel.ojeda.sandonis@...il.com>, Maksim Panchenko <max4bolt@...il.com>, 
	"David S. Miller" <davem@...emloft.net>, Andreas Larsson <andreas@...sler.com>, 
	Yonghong Song <yonghong.song@...ux.dev>, Yabin Cui <yabinc@...gle.com>, 
	Krzysztof Pszeniczny <kpszeniczny@...gle.com>, Sriraman Tallam <tmsriram@...gle.com>, 
	Stephane Eranian <eranian@...gle.com>, x86@...nel.org, linux-arch@...r.kernel.org, 
	sparclinux@...r.kernel.org, linux-doc@...r.kernel.org, 
	linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org, 
	llvm@...ts.linux.dev
Subject: Re: [PATCH v7 0/7] Add AutoFDO and Propeller support for Clang build

Thanks for the explanation.

On Thu, Nov 7, 2024 at 6:58 AM Masahiro Yamada <masahiroy@...nel.org> wrote:
>
> On Thu, Nov 7, 2024 at 4:00 AM Rong Xu <xur@...gle.com> wrote:
> >
> > On Wed, Nov 6, 2024 at 8:09 AM Masahiro Yamada <masahiroy@...nel.org> wrote:
> > >
> > > On Sun, Nov 3, 2024 at 2:51 AM Rong Xu <xur@...gle.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > This patch series is to integrate AutoFDO and Propeller support into
> > > > the Linux kernel. AutoFDO is a profile-guided optimization technique
> > > > that leverages hardware sampling to enhance binary performance.
> > > > Unlike Instrumentation-based FDO (iFDO), AutoFDO offers a user-friendly
> > > > and straightforward application process. While iFDO generally yields
> > > > superior profile quality and performance, our findings reveal that
> > > > AutoFDO achieves remarkable effectiveness, bringing performance close
> > > > to iFDO for benchmark applications.
> > > >
> > > > Propeller is a profile-guided, post-link optimizer that improves
> > > > the performance of large-scale applications compiled with LLVM. It
> > > > operates by relinking the binary based on an additional round of runtime
> > > > profiles, enabling precise optimizations that are not possible at
> > > > compile time.  Similar to AutoFDO, Propeller too utilizes hardware
> > > > sampling to collect profiles and apply post-link optimizations to improve
> > > > the benchmark’s performance over and above AutoFDO.
> > > >
> > > > Our empirical data demonstrates significant performance improvements
> > > > with AutoFDO and Propeller, up to 10% on microbenchmarks and up to 5%
> > > > on large warehouse-scale benchmarks. This makes a strong case for their
> > > > inclusion as supported features in the upstream kernel.
> > > >
> > > > Background
> > > >
> > > > A significant fraction of fleet processing cycles (excluding idle time)
> > > > from data center workloads are attributable to the kernel. Ware-house
> > > > scale workloads maximize performance by optimizing the production kernel
> > > > using iFDO (a.k.a instrumented PGO, Profile Guided Optimization).
> > > >
> > > > iFDO can significantly enhance application performance but its use
> > > > within the kernel has raised concerns. AutoFDO is a variant of FDO that
> > > > uses the hardware’s Performance Monitoring Unit (PMU) to collect
> > > > profiling data. While AutoFDO typically yields smaller performance
> > > > gains than iFDO, it presents unique benefits for optimizing kernels.
> > > >
> > > > AutoFDO eliminates the need for instrumented kernels, allowing a single
> > > > optimized kernel to serve both execution and profile collection. It also
> > > > minimizes slowdown during profile collection, potentially yielding
> > > > higher-fidelity profiling, especially for time-sensitive code, compared
> > > > to iFDO. Additionally, AutoFDO profiles can be obtained from production
> > > > environments via the hardware’s PMU whereas iFDO profiles require
> > > > carefully curated load tests that are representative of real-world
> > > > traffic.
> > > >
> > > > AutoFDO facilitates profile collection across diverse targets.
> > > > Preliminary studies indicate significant variation in kernel hot spots
> > > > within Google’s infrastructure, suggesting potential performance gains
> > > > through target-specific kernel customization.
> > > >
> > > > Furthermore, other advanced compiler optimization techniques, including
> > > > ThinLTO and Propeller can be stacked on top of AutoFDO, similar to iFDO.
> > > > ThinLTO achieves better runtime performance through whole-program
> > > > analysis and cross module optimizations. The main difference between
> > > > traditional LTO and ThinLTO is that the latter is scalable in time and
> > > > memory.
> > > >
> > > > This patch series adds AutoFDO and Propeller support to the kernel. The
> > > > actual solution comes in six parts:
> > > >
> > > > [P 1] Add the build support for using AutoFDO in Clang
> > > >
> > > >       Add the basic support for AutoFDO build and provide the
> > > >       instructions for using AutoFDO.
> > > >
> > > > [P 2] Fix objtool for bogus warnings when -ffunction-sections is enabled
> > > >
> > > > [P 3] Adjust symbol ordering in text output sections
> > > >
> > > > [P 4] Add markers for text_unlikely and text_hot sections
> > > >
> > > > [P 5] Enable –ffunction-sections for the AutoFDO build
> > > >
> > > > [P 6] Enable Machine Function Split (MFS) optimization for AutoFDO
> > > >
> > > > [P 7] Add Propeller configuration to the kernel build
> > > >
> > > > Patch 1 provides basic AutoFDO build support. Patches 2 to 6 further
> > > > enhance the performance of AutoFDO builds and are functionally dependent
> > > > on Patch 1. Patch 7 enables support for Propeller and is dependent on
> > > > patch 2 to patch 4.
> > > >
> > > > Caveats
> > > >
> > > > AutoFDO is compatible with both GCC and Clang, but the patches in this
> > > > series are exclusively applicable to LLVM 17 or newer for AutoFDO and
> > > > LLVM 19 or newer for Propeller. For profile conversion, two different
> > > > tools could be used, llvm_profgen or create_llvm_prof. llvm_profgen
> > > > needs to be the LLVM 19 or newer, or just the LLVM trunk. Alternatively,
> > > > create_llvm_prof v0.30.1 or newer can be used instead of llvm-profgen.
> > > >
> > > > Additionally, the build is only supported on x86 platforms equipped
> > > > with PMU capabilities, such as LBR on Intel machines. More
> > > > specifically:
> > > >  * Intel platforms: works on every platform that supports LBR;
> > > >    we have tested on Skylake.
> > > >  * AMD platforms: tested on AMD Zen3 with the BRS feature. The kernel
> > > >    needs to be configured with “CONFIG_PERF_EVENTS_AMD_BRS=y", To
> > > >    check, use
> > > >    $ cat /proc/cpuinfo | grep “ brs”
> > > >    For the AMD Zen4, AMD LBRV2 is supported, but we suspect a bug with
> > > >    AMD LBRv2 implementation in Genoa which blocks the usage.
> > > >
> > > > For ARM, we plan to send patches for SPE-based Propeller when
> > > > AutoFDO for Arm is ready.
> > > >
> > > > Experiments and Results
> > > >
> > > > Experiments were conducted to compare the performance of AutoFDO-optimized
> > > > kernel images (version 6.9.x) against default builds.. The evaluation
> > > > encompassed both open source microbenchmarks and real-world production
> > > > services from Google and Meta. The selected microbenchmarks included Neper,
> > > > a network subsystem benchmark, and UnixBench which is a comprehensive suite
> > > > for assessing various kernel operations.
> > > >
> > > > For Neper, AutoFDO optimization resulted in a 6.1% increase in throughput
> > > > and a 10.6% reduction in latency. UnixBench saw a 2.2% improvement in its
> > > > index score under low system load and a 2.6% improvement under high system
> > > > load.
> > > >
> > > > For further details on the improvements observed in Google and Meta's
> > > > production services, please refer to the LLVM discourse post:
> > > > https://discourse.llvm.org/t/optimizing-the-linux-kernel-with-autofdo-including-thinlto-and-propeller/79108
> > > >
> > > > Thanks,
> > > >
> > > > Rong Xu and Han Shen
> > >
> > >
> > > I applied this series to linux-kbuild.
> > >
> >
> > Thanks for taking the patch!
> >
> > > As I mentioned before, I do not like #ifdef because
> > > it hides (not fixes) issues only for default cases.
> >
> > We followed the suggestion and removed most of the #if (or #ifdef) in
> > the linker script.
> > I just checked: there are two #ifdef remaining:
> > (1) in the propeller patch for .llvm_bb_addr_map
> > (2) in linker script patch for arch/sparc/kernel/vmlinux.lds.S.
> >
> > I think it's likely safe to remove the checks for head_64.o in
> > non-SPARC64 builds and .llvm_bb_addr_map symbols in non-propeller builds.
> >
> > SPARC64 builds should always produce head_64.o, and non-SPARC64
> > builds shouldn't.
> >
> > Propeller builds always generate .llvm_bb_addr_map symbols, and the
> > linker will omit the section if it's empty in non-propeller builds.
> >
> > Keeping the checks is harmless and might slightly reduce linker
> > workload for matching.
> > But If you'd prefer to remove them, I'm happy to provide a patch.
>
>
> I am talking about the #ifdef in include/asm-generic/vmlinux.lds.h
>
>
> Yeah, it is me who (reluctantly) accepted cb87481ee89d.
>
> Now, the #ifdef has become a little more complicated.
> The default case is safe, but there are hidden issues.
>
> Some issues are easy to fix, so I sent some patches.
> https://lore.kernel.org/linux-kbuild/20241106161445.189399-1-masahiroy@kernel.org/T/#t
> https://lore.kernel.org/linux-kbuild/20241106161445.189399-1-masahiroy@kernel.org/T/#m4e4fa70386696e903b68d3fe1d7277e9a63fbefe
> https://lore.kernel.org/linux-kbuild/20241107111519.GA15424@willie-the-truck/T/#mccf6d49ddd11c90dcc583d7a68934bb3311da880

I did notice the issues for .data.* -- that is one of the reasons we
separated text from data in our patch.

>
> For example, see e41f501d3912.
>
> When CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y or
> CONFIG_LTO_CLANG=y or CONFIG_AUTOFDO_CLANG=y or
> CONFIG_PROPELLER_CLANG=y, the .text.startup sections
> will go to TEXT_MAIN instead of INIT_TEXT.
> This is not a fatal issue, but we cannot reuse memory for .text.startup
> sections.
>
> Removing the #ifdef (i.e. reverting cb87481ee89d) is more difficult
> because we need to take a closer look at potential impacts for all
> architectures.

I'm not sure if there is a naming convention for section names in the kernel.
For special sections, we should avoid using .text.* or .data.*,
instead, using "..', or use
other prefixes.

The compiler can generate sections names like .text.hot.*", ".text.unknown.*",
  ".text.unlikely.*", ".text.split.*", ".text.startup." or
".text.exit. It seems we've
addressed most of them except .text.startup and .text.exit.

For text.startup and .text.exit, have you considered renaming the
sections within
the linker script -- they are fixed strings and should be able to be renamed.

>
> I understood you did not want to take a risk to break random architectures,
> so I decided to postpone the #ifdef issue and accept your patch set.

Thanks for the understanding!

>
> --
> Best Regards
> Masahiro Yamada

Powered by blists - more mailing lists