linux-kernel - Re: [PATCH 06/10] x86,mm: use broadcast TLB flushing for page reclaim TLB flushing

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20241222111849.GW11133@noisy.programming.kicks-ass.net>
Date: Sun, 22 Dec 2024 12:18:49 +0100
From: Peter Zijlstra <peterz@...radead.org>
To: Rik van Riel <riel@...riel.com>
Cc: x86@...nel.org, linux-kernel@...r.kernel.org, kernel-team@...a.com,
	dave.hansen@...ux.intel.com, luto@...nel.org, tglx@...utronix.de,
	mingo@...hat.com, bp@...en8.de, hpa@...or.com,
	akpm@...ux-foundation.org
Subject: Re: [PATCH 06/10] x86,mm: use broadcast TLB flushing for page
 reclaim TLB flushing

On Sat, Dec 21, 2024 at 11:06:38PM -0500, Rik van Riel wrote:
> In the page reclaim code, we only track the CPU(s) where the TLB needs
> to be flushed, rather than all the individual mappings that may be getting
> invalidated.
> 
> Use broadcast TLB flushing when that is available.
> 
> Signed-off-by: Rik van Riel <riel@...riel.com>
> ---
>  arch/x86/mm/tlb.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
> index bf85cd0590d5..9422b10edec1 100644
> --- a/arch/x86/mm/tlb.c
> +++ b/arch/x86/mm/tlb.c
> @@ -1313,6 +1313,12 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch)
>  
>  	int cpu = get_cpu();
>  
> +	if (static_cpu_has(X86_FEATURE_INVLPGB)) {
> +		invlpgb_flush_all_nonglobals();
> +		tlbsync();
> +		goto out_put_cpu;
> +	}

Urgh, move this before the get_cpu(), and write it like:

	if (static_cpu_has(X86_FEATURE_INVLPGB)) {
		guard(preempt)();
		invlpgb_flush_all_nonglobals();
		tlbsync();
		return;	
	}

?


> +
>  	info = get_flush_tlb_info(NULL, 0, TLB_FLUSH_ALL, 0, false,
>  				  TLB_GENERATION_INVALID);
>  	/*
> @@ -1332,6 +1338,7 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch)
>  	cpumask_clear(&batch->cpumask);
>  
>  	put_flush_tlb_info();
> +out_put_cpu:
>  	put_cpu();
>  }
>  
> -- 
> 2.47.1
>