Message-ID: <alpine.DEB.2.21.2511122032400.25436@angie.orcam.me.uk>
Date: Wed, 12 Nov 2025 23:42:11 +0000 (GMT)
From: "Maciej W. Rozycki" <macro@...am.me.uk>
To: Nick Bowler <nbowler@...conx.ca>, 
    Thomas Bogendoerfer <tsbogend@...ha.franken.de>, 
    Jiaxun Yang <jiaxun.yang@...goat.com>
cc: linux-mips@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: [PATCH v2] MIPS: mm: Prevent a TLB shutdown on initial
 uniquification

Depending on the particular CPU implementation a TLB shutdown may occur 
if multiple matching entries are detected upon the execution of a TLBP 
or the TLBWI/TLBWR instructions.  Given that we don't know what entries 
we have been handed we need to be very careful with the initial TLB 
setup and avoid all these instructions.

Therefore read all the TLB entries one by one with the TLBR instruction, 
bypassing the content addressing logic, and truncate any large pages in 
place so as to avoid a case in the second step where an incoming entry 
for a large page at a lower address overlaps with a replacement entry 
chosen at another index.  Then preinitialize the TLB using addresses 
outside our usual unique range and avoiding clashes with any entries 
received, before making the usual call to local_flush_tlb_all().

This fixes (at least) R4x00 cores if TLBP hits multiple matching TLB 
entries (SGI IP22 PROM for examples sets up all TLBs to the same virtual 
address).

Signed-off-by: Maciej W. Rozycki <macro@...am.me.uk>
Fixes: 35ad7e181541 ("MIPS: mm: tlb-r4k: Uniquify TLB entries on init")
Cc: stable@...r.kernel.org # v6.17+
---
Hi,

 Verified the same way as before, also with some diagnostics added so as 
to make sure things get set up correctly, with my Malta/74Kf system for a 
32-bit configuration and with my SWARM/BCM1250 system for a 64-bit one.

 In addition to the Wired register setup discussed with v1 I have realised 
the incoming entries may include large pages, possibly exceeding the size 
of KSEG0 even.  Such entries may overlap with our temporary entries added 
in the second step, so truncate any large pages in place as this ensures 
no clash happens with the received contents of the TLB.

 NB this doesn't handle incoming PageGrain.ESP having been set, but it's 
an unrelated preexisting issue that would have to be handled elsewhere.  
Possibly it doesn't matter in reality.

 Additionally PageMask is left set at what has been retrieved from the 
last incoming TLB entry in the first step and has to be reset to our page 
size before proceeding with the second step.

 And last but not least the comparator function returned 0 incorrectly 
when the difference between 64-bit elements was positive but with none of 
the high-order 32 bits set.  Fixed with a branchless sequence of 3 machine 
instructions, which I think is the minimum here (only the sign and zero 
matter here, but this sequence actually produces -1/0/1, because why not).
No change for the 32-bit case, the difference is returned as is.

  Maciej

Changes from v1 (at 
<https://lore.kernel.org/r/alpine.DEB.2.21.2511110547430.25436@angie.orcam.me.uk/>):

- Also include wired entries while reading original contents of TLB.

- Truncate any large pages in place while reading original TLB entries.

- Reset PageMask to PM_DEFAULT_MASK after reading in TLB entries.

- Fix the 64-bit case for the sort comparator.
---
 arch/mips/mm/tlb-r4k.c |  102 +++++++++++++++++++++++++++++++------------------
 1 file changed, 65 insertions(+), 37 deletions(-)

linux-mips-tlb-r4k-uniquify-fix.diff
Index: linux-macro/arch/mips/mm/tlb-r4k.c
===================================================================
--- linux-macro.orig/arch/mips/mm/tlb-r4k.c
+++ linux-macro/arch/mips/mm/tlb-r4k.c
@@ -15,6 +15,7 @@
 #include <linux/mm.h>
 #include <linux/hugetlb.h>
 #include <linux/export.h>
+#include <linux/sort.h>
 
 #include <asm/cpu.h>
 #include <asm/cpu-type.h>
@@ -508,54 +509,80 @@ static int __init set_ntlb(char *str)
 
 __setup("ntlb=", set_ntlb);
 
-/* Initialise all TLB entries with unique values */
+
+/* Comparison function for EntryHi VPN fields.  */
+static int r4k_vpn_cmp(const void *a, const void *b)
+{
+	long v = *(unsigned long *)a - *(unsigned long *)b;
+	int s = sizeof(long) > sizeof(int) ? sizeof(long) * 8 - 1: 0;
+	return s ? (v != 0) | v >> s : v;
+}
+
+/*
+ * Initialise all TLB entries with unique values that do not clash with
+ * what we have been handed over and what we'll be using ourselves.
+ */
 static void r4k_tlb_uniquify(void)
 {
-	int entry = num_wired_entries();
+	unsigned long tlb_vpns[1 << MIPS_CONF1_TLBS_SIZE];
+	int tlbsize = current_cpu_data.tlbsize;
+	int start = num_wired_entries();
+	unsigned long vpn_mask;
+	int ent, idx, i;
+
+	vpn_mask = GENMASK(cpu_vmbits - 1, 13);
+	vpn_mask |= IS_ENABLED(CONFIG_64BIT) ? 3ULL << 62 : 1 << 31;
 
 	htw_stop();
+
+	for (i = 0; i < tlbsize; i++) {
+		unsigned long vpn;
+
+		write_c0_index(i);
+		mtc0_tlbr_hazard();
+		tlb_read();
+		tlb_read_hazard();
+		vpn = read_c0_entryhi();
+		vpn &= vpn_mask & PAGE_MASK;
+		tlb_vpns[i] = vpn;
+		if (i < start)
+			continue;
+
+		/* Prevent any large pages from overlapping regular ones.  */
+		write_c0_pagemask(read_c0_pagemask() & PM_DEFAULT_MASK);
+		mtc0_tlbw_hazard();
+		tlb_write_indexed();
+		tlbw_use_hazard();
+	}
+
+	sort(tlb_vpns, tlbsize, sizeof(tlb_vpns[0]), r4k_vpn_cmp, NULL);
+
+	write_c0_pagemask(PM_DEFAULT_MASK);
 	write_c0_entrylo0(0);
 	write_c0_entrylo1(0);
 
-	while (entry < current_cpu_data.tlbsize) {
-		unsigned long asid_mask = cpu_asid_mask(&current_cpu_data);
-		unsigned long asid = 0;
-		int idx;
+	idx = 0;
+	ent = tlbsize;
+	for (i = start; i < tlbsize; i++)
+		while (1) {
+			unsigned long entryhi, vpn;
 
-		/* Skip wired MMID to make ginvt_mmid work */
-		if (cpu_has_mmid)
-			asid = MMID_KERNEL_WIRED + 1;
+			entryhi = UNIQUE_ENTRYHI(ent);
+			vpn = entryhi & vpn_mask & PAGE_MASK;
 
-		/* Check for match before using UNIQUE_ENTRYHI */
-		do {
-			if (cpu_has_mmid) {
-				write_c0_memorymapid(asid);
-				write_c0_entryhi(UNIQUE_ENTRYHI(entry));
+			if (idx >= tlbsize || vpn < tlb_vpns[idx]) {
+				write_c0_entryhi(entryhi);
+				write_c0_index(i);
+				mtc0_tlbw_hazard();
+				tlb_write_indexed();
+				ent++;
+				break;
+			} else if (vpn == tlb_vpns[idx]) {
+				ent++;
 			} else {
-				write_c0_entryhi(UNIQUE_ENTRYHI(entry) | asid);
+				idx++;
 			}
-			mtc0_tlbw_hazard();
-			tlb_probe();
-			tlb_probe_hazard();
-			idx = read_c0_index();
-			/* No match or match is on current entry */
-			if (idx < 0 || idx == entry)
-				break;
-			/*
-			 * If we hit a match, we need to try again with
-			 * a different ASID.
-			 */
-			asid++;
-		} while (asid < asid_mask);
-
-		if (idx >= 0 && idx != entry)
-			panic("Unable to uniquify TLB entry %d", idx);
-
-		write_c0_index(entry);
-		mtc0_tlbw_hazard();
-		tlb_write_indexed();
-		entry++;
-	}
+		}
 
 	tlbw_use_hazard();
 	htw_start();
@@ -602,6 +629,7 @@ static void r4k_tlb_configure(void)
 
 	/* From this point on the ARC firmware is dead.	 */
 	r4k_tlb_uniquify();
+	local_flush_tlb_all();
 
 	/* Did I tell you that ARC SUCKS?  */
 }

Powered by blists - more mailing lists