linux-kernel - Re: [PATCH v14 04/13] x86/mm: use INVLPGB for kernel TLB flushes

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250228214328.GGZ8IuAMKL4FeSTBER@fat_crate.local>
Date: Fri, 28 Feb 2025 22:43:28 +0100
From: Borislav Petkov <bp@...en8.de>
To: Rik van Riel <riel@...riel.com>
Cc: x86@...nel.org, linux-kernel@...r.kernel.org, peterz@...radead.org,
	dave.hansen@...ux.intel.com, zhengqi.arch@...edance.com,
	nadav.amit@...il.com, thomas.lendacky@....com, kernel-team@...a.com,
	linux-mm@...ck.org, akpm@...ux-foundation.org, jackmanb@...gle.com,
	jannh@...gle.com, mhklinux@...look.com, andrew.cooper3@...rix.com,
	Manali.Shukla@....com, mingo@...nel.org
Subject: Re: [PATCH v14 04/13] x86/mm: use INVLPGB for kernel TLB flushes

On Tue, Feb 25, 2025 at 10:00:39PM -0500, Rik van Riel wrote:
> Use broadcast TLB invalidation for kernel addresses when available.
> 
> Remove the need to send IPIs for kernel TLB flushes.
> 
> Signed-off-by: Rik van Riel <riel@...riel.com>
> Tested-by: Manali Shukla <Manali.Shukla@....com>
> Tested-by: Brendan Jackman <jackmanb@...gle.com>
> Tested-by: Michael Kelley <mhklinux@...look.com>
> ---
>  arch/x86/mm/tlb.c | 32 ++++++++++++++++++++++++++++++--
>  1 file changed, 30 insertions(+), 2 deletions(-)

Changes ontop:

--- /tmp/current.patch	2025-02-28 22:39:33.236465716 +0100
+++ /tmp/0001-x86-mm-Use-INVLPGB-for-kernel-TLB-flushes.patch	2025-02-28 22:39:59.432310072 +0100
@@ -1,36 +1,43 @@
+From b97ae5e31069cd536b563df185de52d33a565077 Mon Sep 17 00:00:00 2001
 From: Rik van Riel <riel@...riel.com>
 Date: Tue, 25 Feb 2025 22:00:39 -0500
-Subject: x86/mm: Use INVLPGB for kernel TLB flushes
+Subject: [PATCH] x86/mm: Use INVLPGB for kernel TLB flushes
 
 Use broadcast TLB invalidation for kernel addresses when available.
-
 Remove the need to send IPIs for kernel TLB flushes.
 
+   [ bp: Integrate dhansen's comments additions. ]
+
 Signed-off-by: Rik van Riel <riel@...riel.com>
 Signed-off-by: Borislav Petkov (AMD) <bp@...en8.de>
-Tested-by: Manali Shukla <Manali.Shukla@....com>
-Tested-by: Brendan Jackman <jackmanb@...gle.com>
-Tested-by: Michael Kelley <mhklinux@...look.com>
+Acked-by: Dave Hansen <dave.hansen@...el.com>
 Link: https://lore.kernel.org/r/20250226030129.530345-5-riel@surriel.com
 ---
- arch/x86/mm/tlb.c | 32 ++++++++++++++++++++++++++++++--
- 1 file changed, 30 insertions(+), 2 deletions(-)
+ arch/x86/mm/tlb.c | 39 +++++++++++++++++++++++++++++++++++++--
+ 1 file changed, 37 insertions(+), 2 deletions(-)
 
 diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
-index dbcb5c968ff9..f44a03bca41c 100644
+index dbcb5c968ff9..5c44b94ad5af 100644
 --- a/arch/x86/mm/tlb.c
 +++ b/arch/x86/mm/tlb.c
-@@ -1077,6 +1077,18 @@ void flush_tlb_all(void)
+@@ -1077,6 +1077,25 @@ void flush_tlb_all(void)
  	on_each_cpu(do_flush_tlb_all, NULL, 1);
  }
  
++/* Flush an arbitrarily large range of memory with INVLPGB. */
 +static void invlpgb_kernel_range_flush(struct flush_tlb_info *info)
 +{
 +	unsigned long addr, nr;
 +
 +	for (addr = info->start; addr < info->end; addr += nr << PAGE_SHIFT) {
 +		nr = (info->end - addr) >> PAGE_SHIFT;
++
++		/*
++		 * INVLPGB has a limit on the size of ranges it can
++		 * flush. Break up large flushes.
++		 */
 +		nr = clamp_val(nr, 1, invlpgb_count_max);
++
 +		invlpgb_flush_addr_nosync(addr, nr);
 +	}
 +	__tlbsync();
 

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette