lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Tue,  2 Jan 2024 14:00:40 -0800
From: Samuel Holland <samuel.holland@...ive.com>
To: Palmer Dabbelt <palmer@...belt.com>,
	linux-riscv@...ts.infradead.org
Cc: linux-kernel@...r.kernel.org,
	linux-mm@...ck.org,
	Alexandre Ghiti <alexghiti@...osinc.com>,
	Samuel Holland <samuel.holland@...ive.com>
Subject: [PATCH v4 03/12] riscv: mm: Broadcast kernel TLB flushes only when needed

__flush_tlb_range() avoids broadcasting TLB flushes when an mm context
is only active on the local CPU. Apply this same optimization to TLB
flushes of kernel memory when only one CPU is online. This check can be
constant-folded when SMP is disabled.

Signed-off-by: Samuel Holland <samuel.holland@...ive.com>
---

Changes in v4:
 - New patch for v4

 arch/riscv/mm/tlbflush.c | 17 ++++++-----------
 1 file changed, 6 insertions(+), 11 deletions(-)

diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c
index 09b03bf71e6a..2f18fe6fc4f3 100644
--- a/arch/riscv/mm/tlbflush.c
+++ b/arch/riscv/mm/tlbflush.c
@@ -98,27 +98,23 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start,
 {
 	const struct cpumask *cmask;
 	unsigned long asid = FLUSH_TLB_NO_ASID;
-	bool broadcast;
+	unsigned int cpu;
 
 	if (mm) {
-		unsigned int cpuid;
-
 		cmask = mm_cpumask(mm);
 		if (cpumask_empty(cmask))
 			return;
 
-		cpuid = get_cpu();
-		/* check if the tlbflush needs to be sent to other CPUs */
-		broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids;
-
 		if (static_branch_unlikely(&use_asid_allocator))
 			asid = atomic_long_read(&mm->context.id) & asid_mask;
 	} else {
 		cmask = cpu_online_mask;
-		broadcast = true;
 	}
 
-	if (!broadcast) {
+	cpu = get_cpu();
+
+	/* Check if the TLB flush needs to be sent to other CPUs. */
+	if (cpumask_any_but(cmask, cpu) >= nr_cpu_ids) {
 		local_flush_tlb_range_asid(start, size, stride, asid);
 	} else if (riscv_use_sbi_for_rfence()) {
 		sbi_remote_sfence_vma_asid(cmask, start, size, asid);
@@ -132,8 +128,7 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start,
 		on_each_cpu_mask(cmask, __ipi_flush_tlb_range_asid, &ftd, 1);
 	}
 
-	if (mm)
-		put_cpu();
+	put_cpu();
 }
 
 void flush_tlb_mm(struct mm_struct *mm)
-- 
2.42.0


Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ