linux-kernel - [RFC PATCH v4 8/8] x86/tlb: flush the local TLB twice (DEBUG)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250619200442.1694583-9-riel@surriel.com>
Date: Thu, 19 Jun 2025 16:04:00 -0400
From: Rik van Riel <riel@...riel.com>
To: linux-kernel@...r.kernel.org
Cc: kernel-team@...a.com,
	dave.hansen@...ux.intel.com,
	luto@...nel.org,
	peterz@...radead.org,
	bp@...en8.de,
	x86@...nel.org,
	nadav.amit@...il.com,
	seanjc@...gle.com,
	tglx@...utronix.de,
	mingo@...nel.org,
	Rik van Riel <riel@...a.com>,
	Rik van Riel <riel@...riel.com>
Subject: [RFC PATCH v4 8/8] x86/tlb: flush the local TLB twice (DEBUG)

From: Rik van Riel <riel@...a.com>

The RAR code attempts to flush the local TLB in addition to remote TLBs,
if the local TLB is in the cpumask. This can be seen in that the status
for the local CPU is changed from RAR_PENDING to RAR_SUCCESS.

However, it appears that the local TLB is not actually getting flushed
when the microcode flips the status to RAR_SUCCESS!

The RAR white paper suggests it should work:

"At this point, the ILP may invalidate its own TLB by signaling RAR
 to itself in order to invoke the RAR handler locally as well."

I would really appreciate some guidance from Intel on how to move
forward here.

Is the RAR code doing something wrong?

Is the CPU not behaving quite as documented?

What is the best way forward?

Not-signed-off-by: Rik van Riel <riel@...riel.com>
---
 arch/x86/mm/tlb.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 590742838e43..f12eff2dbcc8 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -1469,22 +1469,22 @@ static void rar_tlb_flush(struct flush_tlb_info *info)
 {
 	unsigned long asid = mm_global_asid(info->mm);
 	cpumask_t *cpumask = mm_cpumask(info->mm);
+	int cpu = raw_smp_processor_id();
 	u16 pcid = kern_pcid(asid);
 
 	if (info->trim_cpumask)
 		trim_cpumask(info);
 
 	/* Only the local CPU needs to be flushed? */
-	if (cpumask_equal(cpumask, cpumask_of(raw_smp_processor_id()))) {
+	if (cpumask_test_cpu(cpu, cpumask)) {
 		lockdep_assert_irqs_enabled();
 		local_irq_disable();
 		flush_tlb_func(info);
 		local_irq_enable();
-		return;
 	}
 
 	/* Flush all the CPUs at once with RAR. */
-	if (cpumask_weight(cpumask)) {
+	if (cpumask_any_but(cpumask, cpu)) {
 		smp_call_rar_many(mm_cpumask(info->mm), pcid, info->start, info->end);
 		if (cpu_feature_enabled(X86_FEATURE_PTI))
 			smp_call_rar_many(mm_cpumask(info->mm), user_pcid(asid), info->start, info->end);
-- 
2.49.0