linux-kernel - [PATCH 08/11] x86/tsc: Avoid synchronizing TSCs with multiple CPUs in parallel

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20211209150938.3518-9-dwmw2@infradead.org>
Date:   Thu,  9 Dec 2021 15:09:35 +0000
From:   David Woodhouse <dwmw2@...radead.org>
To:     Thomas Gleixner <tglx@...utronix.de>
Cc:     Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>,
        Dave Hansen <dave.hansen@...ux.intel.com>, x86@...nel.org,
        "H. Peter Anvin" <hpa@...or.com>,
        Paolo Bonzini <pbonzini@...hat.com>,
        "Paul E. McKenney" <paulmck@...nel.org>,
        linux-kernel@...r.kernel.org, kvm@...r.kernel.org,
        rcu@...r.kernel.org, mimoja@...oja.de, hewenliang4@...wei.com,
        hushiyuan@...wei.com, luolongjun@...wei.com, hejingxian@...wei.com
Subject: [PATCH 08/11] x86/tsc: Avoid synchronizing TSCs with multiple CPUs in parallel

From: David Woodhouse <dwmw@...zon.co.uk>

The TSC sync algorithm is only designed to do a 1:1 sync between the
source and target CPUs.

In order to enable parallel CPU bringup, serialize it by using an
atomic_t containing the number of the target CPU whose turn it is.

In future we should look at inventing a 1:many TSC synchronization
algorithm, perhaps falling back to 1:1 if a warp is observed but
doing them all in parallel for the common case where no adjustment
is needed. Or just avoiding the sync completely for cases like kexec
where we trust that they were in sync already.

This is perfectly sufficient for the short term though, until we get
those further optimisations.

Signed-off-by: David Woodhouse <dwmw@...zon.co.uk>
---
 arch/x86/kernel/tsc_sync.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/x86/kernel/tsc_sync.c b/arch/x86/kernel/tsc_sync.c
index 50a4515fe0ad..4ee247d89a49 100644
--- a/arch/x86/kernel/tsc_sync.c
+++ b/arch/x86/kernel/tsc_sync.c
@@ -202,6 +202,7 @@ bool tsc_store_and_check_tsc_adjust(bool bootcpu)
  * Entry/exit counters that make sure that both CPUs
  * run the measurement code at once:
  */
+static atomic_t tsc_sync_cpu = ATOMIC_INIT(-1);
 static atomic_t start_count;
 static atomic_t stop_count;
 static atomic_t skip_test;
@@ -326,6 +327,8 @@ void check_tsc_sync_source(int cpu)
 		atomic_set(&test_runs, 1);
 	else
 		atomic_set(&test_runs, 3);
+
+	atomic_set(&tsc_sync_cpu, cpu);
 retry:
 	/*
 	 * Wait for the target to start or to skip the test:
@@ -407,6 +410,10 @@ void check_tsc_sync_target(void)
 	if (unsynchronized_tsc())
 		return;
 
+	/* Wait for this CPU's turn */
+	while (atomic_read(&tsc_sync_cpu) != cpu)
+		cpu_relax();
+
 	/*
 	 * Store, verify and sanitize the TSC adjust register. If
 	 * successful skip the test.
-- 
2.31.1