linux-kernel - [PATCh 0/3] x86,tlb: context switch optimizations

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20241109003727.3958374-1-riel@surriel.com>
Date: Fri,  8 Nov 2024 19:27:47 -0500
From: Rik van Riel <riel@...riel.com>
To: linux-kernel@...r.kernel.org
Cc: dave.hansen@...ux.intel.com,
	luto@...nel.org,
	peterz@...radead.org,
	tglx@...utronix.de,
	mingo@...hat.com,
	bp@...en8.de,
	x86@...nel.org,
	kernel-team@...a.com,
	hpa@...or.com
Subject: [PATCh 0/3] x86,tlb: context switch optimizations

While profiling switch_mm_irqs_off with several workloads,
it appears there are two hot spots that probably don't need
to be there.

The first is the atomic clearing and setting of the current
CPU in prev's and next's mm_cpumask. This can create a large
amount of cache line contention. On a web server, these two
together take about 17% of the CPU time spent in switch_mm_irqs_off.

We should be able to avoid much of the cache line thrashing
by only clearing bits in mm_cpumask lazily from the first
TLB flush to a process, after which the other TLB flushes can
be more narrowly targeted.

A second cause of overhead seems to be the cpumask_test_cpu
inside the WARN_ON_ONCE in the prev == next branch of
switch_mm_irqs_off.

This warning never ever seems to fire, even on a very large
fleet, so it may be best to hide that behind CONFIG_DEBUG_VM.
With the web server workload, this is also about 17% of
switch_mm_irqs_off.