linux-kernel - [PATCH v2 6/6] x86/asm: Change sync_core() to use MOV to CR2 to serialize

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu,  1 Dec 2016 16:35:02 -0800
From:   Andy Lutomirski <luto@...nel.org>
To:     x86@...nel.org
Cc:     One Thousand Gnomes <gnomes@...rguk.ukuu.org.uk>,
        Borislav Petkov <bp@...en8.de>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        Brian Gerst <brgerst@...il.com>,
        Matthew Whitehead <tedheadster@...il.com>,
        Henrique de Moraes Holschuh <hmh@....eng.br>,
        Peter Zijlstra <peterz@...radead.org>,
        Andy Lutomirski <luto@...nel.org>
Subject: [PATCH v2 6/6] x86/asm: Change sync_core() to use MOV to CR2 to serialize

Aside from being excessively slow, CPUID is problematic: Linux runs
on a handful of CPUs that don't have CPUID.  MOV to CR2 is always
available, so use it instead.

On my laptop, CPUID(eax=1, ecx=0) is ~83ns and MOV-to-CR2 is ~42ns,
so this should be a nice speedup.

Signed-off-by: Andy Lutomirski <luto@...nel.org>
---
 arch/x86/include/asm/processor.h | 31 ++++++++-----------------------
 1 file changed, 8 insertions(+), 23 deletions(-)

diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index c4402053c663..6727ed1c0ca0 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -594,31 +594,16 @@ static __always_inline void cpu_relax(void)
 /* Stop speculative execution and prefetching of modified code. */
 static inline void native_sync_core(void)
 {
-	int tmp;
-
-#ifdef CONFIG_X86_32
-	/*
-	 * Do a CPUID if available, otherwise do a jump.  The jump
-	 * can conveniently enough be the jump around CPUID.
-	 */
-	asm volatile("cmpl %2,%1\n\t"
-		     "jl 1f\n\t"
-		     "cpuid\n"
-		     "1:"
-		     : "=a" (tmp)
-		     : "rm" (boot_cpu_data.cpuid_level), "ri" (0), "0" (1)
-		     : "ebx", "ecx", "edx", "memory");
-#else
 	/*
-	 * CPUID is a barrier to speculative execution.
-	 * Prefetched instructions are automatically
-	 * invalidated when modified.
+	 * MOV to CR2 is architecturally defined as a serializing
+	 * instruction.  It's nice because it works on all CPUs, it
+	 * doesn't clobber registers, and (unlike CPUID) it won't force
+	 * a VM exit.
+	 *
+	 * 0xbf172b23 is random poison just in case something ends up
+	 * caring about this value.
 	 */
-	asm volatile("cpuid"
-		     : "=a" (tmp)
-		     : "0" (1)
-		     : "ebx", "ecx", "edx", "memory");
-#endif
+	native_write_cr2(0xbf172b23);
 }
 
 extern void select_idle_routine(const struct cpuinfo_x86 *c);
-- 
2.9.3