linux-kernel - Re: [PATCH v10 09/12] x86/mm: enable broadcast TLB invalidation for multi-threaded processes

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250212104239.GF19118@noisy.programming.kicks-ass.net>
Date: Wed, 12 Feb 2025 11:42:39 +0100
From: Peter Zijlstra <peterz@...radead.org>
To: Rik van Riel <riel@...riel.com>
Cc: x86@...nel.org, linux-kernel@...r.kernel.org, bp@...en8.de,
	dave.hansen@...ux.intel.com, zhengqi.arch@...edance.com,
	nadav.amit@...il.com, thomas.lendacky@....com, kernel-team@...a.com,
	linux-mm@...ck.org, akpm@...ux-foundation.org, jackmanb@...gle.com,
	jannh@...gle.com, mhklinux@...look.com, andrew.cooper3@...rix.com,
	Manali Shukla <Manali.Shukla@....com>
Subject: Re: [PATCH v10 09/12] x86/mm: enable broadcast TLB invalidation for
 multi-threaded processes

On Tue, Feb 11, 2025 at 04:08:04PM -0500, Rik van Riel wrote:

I poked around at this function a little, and ended up with the below.

As to your question if any INVLPGB capable hardware needs PTI; the
answer is no. No AMD machine needs PTI; but it should still work, just
in case someone needs to test something.

static void broadcast_tlb_flush(struct flush_tlb_info *info)
{
	unsigned long asid = info->mm->context.global_asid;
	bool pmd = info->stride_shift == PMD_SHIFT;
	unsigned long addr = info->start;

	/*
	 * TLB flushes with INVLPGB are kicked off asynchronously.
	 * The inc_mm_tlb_gen() guarantees page table updates are done
	 * before these TLB flushes happen.
	 */
	if (info->end == TLB_FLUSH_ALL) {
		invlpgb_flush_single_pcid_nosync(kern_pcid(asid));
		if (static_cpu_has(X86_FEATURE_PTI))
			invlpgb_flush_single_pcid_nosync(user_pcid(asid));

	} else do {
		unsigned long nr = 1;

		if (info->stride_shift <= PMD_SHIFT) {
			/*
			 * Calculate how many pages can be flushed at once; if the
			 * remainder of the range is less than one page, flush one.
			 */
			nr = (info->end - addr) >> info->stride_shift);
			nr = clamp_val(nr, 1, invlpgb_count_max);
		}

		invlpgb_flush_user_nr_nosync(kern_pcid(asid), addr, nr, pmd);
		if (static_cpu_has(X86_FEATURE_PTI))
			invlpgb_flush_user_nr_nosync(user_pcid(asid), addr, nr, pmd);

		addr += nr << info->stride_shift;
	} while (addr < info->end);

	finish_asid_transition(info);

	/* Wait for the INVLPGBs kicked off above to finish. */
	tlbsync();
}