linux-kernel - [RFC PATCH v2 17/21] riscv: mm: Apply Svnapot for base page mapping if possible

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20241205103729.14798-18-luxu.kernel@bytedance.com>
Date: Thu,  5 Dec 2024 18:37:25 +0800
From: Xu Lu <luxu.kernel@...edance.com>
To: paul.walmsley@...ive.com,
	palmer@...belt.com,
	aou@...s.berkeley.edu,
	ardb@...nel.org,
	anup@...infault.org,
	atishp@...shpatra.org
Cc: xieyongji@...edance.com,
	lihangjing@...edance.com,
	punit.agrawal@...edance.com,
	linux-kernel@...r.kernel.org,
	linux-riscv@...ts.infradead.org,
	Xu Lu <luxu.kernel@...edance.com>
Subject: [RFC PATCH v2 17/21] riscv: mm: Apply Svnapot for base page mapping if possible

All hardware pages in the same software page point to the same contiguous
memory region (the region size is equal to the software page size) and
has same prots. Thus this commit uses Svnapot extension to optimize the
mapping to software page to reduce tlb pressure.

Signed-off-by: Xu Lu <luxu.kernel@...edance.com>
---
 arch/riscv/include/asm/pgtable.h | 16 +++++++++++++++-
 arch/riscv/mm/pgtable.c          |  6 ++++++
 2 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 5b2ca92ad833..9f347e5eefeb 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -483,13 +483,27 @@ static inline unsigned long __pte_mknapot(unsigned long pteval,
 	return pteval;
 }
 
+static inline unsigned long __pte_denapot(unsigned long pteval)
+{
+	unsigned long prot_mask = ~(_PAGE_HW_PFN_MASK | _PAGE_NAPOT);
+	unsigned long res;
+
+	if (!__pte_napot(pteval))
+		return pteval;
+	res = __page_val_to_hwpfn(pteval);
+	res = res & (res - 1UL);
+	pteval = (res << _PAGE_HWPFN_SHIFT) | (pteval & prot_mask);
+
+	return pteval;
+}
+
 #ifdef CONFIG_RISCV_USE_SW_PAGE
 static inline pte_t pte_mknapot(pte_t pte, unsigned int order)
 {
 	unsigned long pteval = pte_val(pte);
 	unsigned int i;
 
-	pteval = __pte_mknapot(pteval, order);
+	pteval = __pte_denapot(pteval);
 	for (i = 0; i < HW_PAGES_PER_PAGE; i++)
 		pte.ptes[i] = pteval;
 
diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c
index 150aea8e2d7a..0bcaffe798d5 100644
--- a/arch/riscv/mm/pgtable.c
+++ b/arch/riscv/mm/pgtable.c
@@ -11,6 +11,12 @@ pte_t __pte(unsigned long pteval)
 {
 	pte_t pte;
 	unsigned int i;
+	unsigned int order;
+
+	if (has_svnapot() && __pte_present(pteval) && !__pte_napot(pteval))
+		for_each_napot_order(order)
+			if (napot_cont_shift(order) == PAGE_SHIFT)
+				pteval = __pte_mknapot(pteval, order);
 
 	for (i = 0; i < HW_PAGES_PER_PAGE; i++) {
 		pte.ptes[i] = pteval;
-- 
2.20.1