linux-kernel - [tip:x86/pti] x86/speculation: Avoid __switch_to

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <tip-5635d99953f04b550738f6f4c1c532667c3fd872@git.kernel.org>
Date:   Wed, 28 Nov 2018 06:31:22 -0800
From:   tip-bot for Thomas Gleixner <tipbot@...or.com>
To:     linux-tip-commits@...r.kernel.org
Cc:     aarcange@...hat.com, longman9394@...il.com, mingo@...nel.org,
        torvalds@...ux-foundation.org, dave.hansen@...el.com,
        peterz@...radead.org, tglx@...utronix.de, ak@...ux.intel.com,
        luto@...nel.org, thomas.lendacky@....com, jcm@...hat.com,
        jpoimboe@...hat.com, hpa@...or.com, arjan@...ux.intel.com,
        asit.k.mallick@...el.com, casey.schaufler@...el.com,
        keescook@...omium.org, jkosina@...e.cz,
        linux-kernel@...r.kernel.org, gregkh@...uxfoundation.org,
        dwmw@...zon.co.uk, tim.c.chen@...ux.intel.com,
        david.c.stewart@...el.com
Subject: [tip:x86/pti] x86/speculation: Avoid __switch_to_xtra() calls

Commit-ID:  5635d99953f04b550738f6f4c1c532667c3fd872
Gitweb:     https://git.kernel.org/tip/5635d99953f04b550738f6f4c1c532667c3fd872
Author:     Thomas Gleixner <tglx@...utronix.de>
AuthorDate: Sun, 25 Nov 2018 19:33:48 +0100
Committer:  Thomas Gleixner <tglx@...utronix.de>
CommitDate: Wed, 28 Nov 2018 11:57:11 +0100

x86/speculation: Avoid __switch_to_xtra() calls

The TIF_SPEC_IB bit does not need to be evaluated in the decision to invoke
__switch_to_xtra() when:

 - CONFIG_SMP is disabled

 - The conditional STIPB mode is disabled

The TIF_SPEC_IB bit still controls IBPB in both cases so the TIF work mask
checks might invoke __switch_to_xtra() for nothing if TIF_SPEC_IB is the
only set bit in the work masks.

Optimize it out by masking the bit at compile time for CONFIG_SMP=n and at
run time when the static key controlling the conditional STIBP mode is
disabled.

Signed-off-by: Thomas Gleixner <tglx@...utronix.de>
Reviewed-by: Ingo Molnar <mingo@...nel.org>
Cc: Peter Zijlstra <peterz@...radead.org>
Cc: Andy Lutomirski <luto@...nel.org>
Cc: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Jiri Kosina <jkosina@...e.cz>
Cc: Tom Lendacky <thomas.lendacky@....com>
Cc: Josh Poimboeuf <jpoimboe@...hat.com>
Cc: Andrea Arcangeli <aarcange@...hat.com>
Cc: David Woodhouse <dwmw@...zon.co.uk>
Cc: Tim Chen <tim.c.chen@...ux.intel.com>
Cc: Andi Kleen <ak@...ux.intel.com>
Cc: Dave Hansen <dave.hansen@...el.com>
Cc: Casey Schaufler <casey.schaufler@...el.com>
Cc: Asit Mallick <asit.k.mallick@...el.com>
Cc: Arjan van de Ven <arjan@...ux.intel.com>
Cc: Jon Masters <jcm@...hat.com>
Cc: Waiman Long <longman9394@...il.com>
Cc: Greg KH <gregkh@...uxfoundation.org>
Cc: Dave Stewart <david.c.stewart@...el.com>
Cc: Kees Cook <keescook@...omium.org>
Cc: stable@...r.kernel.org
Link: https://lkml.kernel.org/r/20181125185005.374062201@linutronix.de


---
 arch/x86/include/asm/thread_info.h | 13 +++++++++++--
 arch/x86/kernel/process.h          | 15 +++++++++++++++
 2 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h
index fa583ec99e3e..6d201699c651 100644
--- a/arch/x86/include/asm/thread_info.h
+++ b/arch/x86/include/asm/thread_info.h
@@ -147,9 +147,18 @@ struct thread_info {
 	 _TIF_FSCHECK)
 
 /* flags to check in __switch_to() */
-#define _TIF_WORK_CTXSW							\
+#define _TIF_WORK_CTXSW_BASE						\
 	(_TIF_IO_BITMAP|_TIF_NOCPUID|_TIF_NOTSC|_TIF_BLOCKSTEP|		\
-	 _TIF_SSBD|_TIF_SPEC_IB)
+	 _TIF_SSBD)
+
+/*
+ * Avoid calls to __switch_to_xtra() on UP as STIBP is not evaluated.
+ */
+#ifdef CONFIG_SMP
+# define _TIF_WORK_CTXSW	(_TIF_WORK_CTXSW_BASE | _TIF_SPEC_IB)
+#else
+# define _TIF_WORK_CTXSW	(_TIF_WORK_CTXSW_BASE)
+#endif
 
 #define _TIF_WORK_CTXSW_PREV (_TIF_WORK_CTXSW|_TIF_USER_RETURN_NOTIFY)
 #define _TIF_WORK_CTXSW_NEXT (_TIF_WORK_CTXSW)
diff --git a/arch/x86/kernel/process.h b/arch/x86/kernel/process.h
index 020fbfac3a27..898e97cf6629 100644
--- a/arch/x86/kernel/process.h
+++ b/arch/x86/kernel/process.h
@@ -2,6 +2,8 @@
 //
 // Code shared between 32 and 64 bit
 
+#include <asm/spec-ctrl.h>
+
 void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p);
 
 /*
@@ -14,6 +16,19 @@ static inline void switch_to_extra(struct task_struct *prev,
 	unsigned long next_tif = task_thread_info(next)->flags;
 	unsigned long prev_tif = task_thread_info(prev)->flags;
 
+	if (IS_ENABLED(CONFIG_SMP)) {
+		/*
+		 * Avoid __switch_to_xtra() invocation when conditional
+		 * STIPB is disabled and the only different bit is
+		 * TIF_SPEC_IB. For CONFIG_SMP=n TIF_SPEC_IB is not
+		 * in the TIF_WORK_CTXSW masks.
+		 */
+		if (!static_branch_likely(&switch_to_cond_stibp)) {
+			prev_tif &= ~_TIF_SPEC_IB;
+			next_tif &= ~_TIF_SPEC_IB;
+		}
+	}
+
 	/*
 	 * __switch_to_xtra() handles debug registers, i/o bitmaps,
 	 * speculation mitigations etc.