Message-ID: <29b154c679adeab912f8f5770344126264a698b9.camel@surriel.com>
Date: Tue, 06 May 2025 12:00:04 -0400
From: Rik van Riel <riel@...riel.com>
To: Nadav Amit <nadav.amit@...il.com>
Cc: Linux Kernel Mailing List <linux-kernel@...r.kernel.org>, "open
 list:MEMORY MANAGEMENT"	 <linux-mm@...ck.org>, the arch/x86 maintainers
 <x86@...nel.org>, 	kernel-team@...a.com, Dave Hansen
 <dave.hansen@...ux.intel.com>, luto@...nel.org, 	peterz@...radead.org,
 Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar	 <mingo@...hat.com>,
 Borislav Petkov <bp@...en8.de>, "H. Peter Anvin"	 <hpa@...or.com>, Yu-cheng
 Yu <yu-cheng.yu@...el.com>
Subject: Re: [RFC PATCH 7/9] x86/mm: Introduce Remote Action Request

On Tue, 2025-05-06 at 18:50 +0300, Nadav Amit wrote:
> 
> 
> > On 6 May 2025, at 18:16, Rik van Riel <riel@...riel.com> wrote:
> > 
> > It gets better. Page 8 of the RAR whitepaper tells
> > us that we can simply use RAR to have a CPU send
> > itself TLB flush instructions, and the microcode
> > will do the flush at the same time the other CPUs
> > handle theirs.
> > 
> > "At this point, the ILP may invalidate its own TLB by 
> > signaling RAR to itself in order to invoke the RAR handler
> > locally as well"
> > 
> > I tried this, but things blew up very early in
> > boot, presumably due to the CPU trying to send
> > itself a RAR before it was fully configured to
> > handle them.
> > 
> > The code may need a better decision point than
> > cpu_feature_enabled(X86_FEATURE_RAR) to decide
> > whether or not to use RAR.
> > 
> > Probably something that indicates RAR is actually
> > ready to use on all CPUs.
> > 
> 
> Once you get something working (perhaps with a branch for
> now) you can take the static-key/static-call path, presumably.
> I would first try to get something working properly.
> 
The static-key code is implemented with alternatives,
which call flush_tlb_mm_range.

I've not spent the time digging into whether that
creates any chicken-egg scenarios yet :)

> > I think we have 3 cases here:
> > 
> > 1) Only the local TLB needs to be flushed.
> >   In this case we can INVPCID locally, and skip any
> >   potential contention on the RAR payload table.
> 
> More like INVLPG (and INVPCID to the user PTI). AFAIK, Andy said
> INVLPG performs better than INVPCID for a single entry. But yes,
> this is a simple and hot scenario that should have a separate
> code-path.

I think this can probably be handled in flush_tlb_mm_range(),
so the RAR code is only called for cases (2) and (3) to
begin with.

> 
> > 
> > 2) Only one remote CPU needs to be flushed (no local).
> >   This can use the arch_rar_send_single_ipi() thing.
> > 
> > 3) Multiple CPUs need to be flushed. This could include
> >   the local CPU, or be only multiple remote CPUs.
> >   For this case we could just use arch_send_rar_ipi_mask(),
> >   including sending a RAR request to the local CPU, which
> >   should handle it concurrently with the other CPUs.
> > 
> > Does that seem like a reasonable way to handle things?
> 
> It it. It is just that code-wise, I think the 2nd and 3rd cases
> are similar, and it can be better to distinguish the differences
> between them without creating two completely separate code-paths.
> This makes maintenance and reasoning more simple, I think.
> 
> Consider having a look at smp_call_function_many_cond(). I think
> it handles the 2nd and 3rd cases nicely in the manner I just
> described. Admittedly, I am a bit biased…

I need to use smp_call_function_many_cond() anyway,
to prevent sending RARs to CPUs that are in lazy
TLB mode (and possibly in a power saving idle state).

IPI TLB flushing and RAR can probably both use the
same should_flush_tlb() helper function.

-- 
All Rights Reversed.

Powered by blists - more mailing lists