linux-kernel - Re: [RFC PATCH 00/11] Add a percpu subsection for hot data

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAMzpN2geum53f3nRn0MNBtA3trU5-350XhDeDLs+kx_GC7fKGg@mail.gmail.com>
Date: Sun, 23 Feb 2025 09:44:51 -0500
From: Brian Gerst <brgerst@...il.com>
To: Ard Biesheuvel <ardb@...nel.org>
Cc: Ingo Molnar <mingo@...nel.org>, linux-kernel@...r.kernel.org, x86@...nel.org, 
	"H . Peter Anvin" <hpa@...or.com>, Thomas Gleixner <tglx@...utronix.de>, Borislav Petkov <bp@...en8.de>, 
	Uros Bizjak <ubizjak@...il.com>, Linus Torvalds <torvalds@...ux-foundation.org>, 
	Andy Lutomirski <luto@...nel.org>, Peter Zijlstra <peterz@...radead.org>, 
	Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: [RFC PATCH 00/11] Add a percpu subsection for hot data

On Sun, Feb 23, 2025 at 5:20 AM Ard Biesheuvel <ardb@...nel.org> wrote:
>
> On Sun, 23 Feb 2025 at 10:37, Ingo Molnar <mingo@...nel.org> wrote:
> >
> >
> > * Brian Gerst <brgerst@...il.com> wrote:
> >
> > > Add a new percpu subsection for data that is frequently accessed and
> > > exclusive to each processor.  This is intended to replace the pcpu_hot
> > > struct on X86, and is available to all architectures.
> > >
> > > The one caveat with this approach is that it depends on the linker to
> > > effeciently pack data that is smaller than machine word size.  The
> > > binutils linker does this properly:
> > >
> > > ffffffff842f6000 D __per_cpu_hot_start
> > > ffffffff842f6000 D softirq_pending
> > > ffffffff842f6002 D hardirq_stack_inuse
> > > ffffffff842f6008 D hardirq_stack_ptr
> > > ffffffff842f6010 D __ref_stack_chk_guard
> > > ffffffff842f6010 D __stack_chk_guard
> > > ffffffff842f6018 D const_cpu_current_top_of_stack
> > > ffffffff842f6018 D cpu_current_top_of_stack
> > > ffffffff842f6020 D const_current_task
> > > ffffffff842f6020 D current_task
> > > ffffffff842f6028 D __preempt_count
> > > ffffffff842f602c D cpu_number
> > > ffffffff842f6030 D this_cpu_off
> > > ffffffff842f6038 D __x86_call_depth
> > > ffffffff842f6040 D __per_cpu_hot_end
> > >
> > > The LLVM linker doesn't do as well with packing smaller data objects,
> > > causing it to spill over into a second cacheline.
> >
> > ... now it's linker-ordered. But on the other hand that can be an
> > advantage too: the linker will try to (or at least has a chance to)
> > order the fields optimally for cache density, while the hand-packing
> > always has the potential to bitrot without much of an outside,
> > actionable indicator for the bitrot.
> >
>
> The linker will need some help here - by default, it just emits these
> variables in the order they appear in the input.
>
> If we emit each such variable 'foo' into .data..hot.foo, and define
> the contents of the section as
>
> *(SORT_BY_ALIGNMENT(.data..hot.*))
>
> we should get optimal packing as long as the alignment of these
> variables does not exceed their size.

Thanks for the tip on SORT_BY_ALIGNMENT().  That got the LLVM linker
to pack the data correctly.


Brian Gerst