linux-kernel - [RFC PATCH v3 8/8] x86/entry/pti: don't switch PGD when TIF_DISABLE_PTI

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 10 Jan 2018 20:28:20 +0100
From:   Willy Tarreau <w@....eu>
To:     linux-kernel@...r.kernel.org, x86@...nel.org
Cc:     Willy Tarreau <w@....eu>, Andy Lutomirski <luto@...nel.org>,
        Borislav Petkov <bp@...en8.de>,
        Brian Gerst <brgerst@...il.com>,
        Dave Hansen <dave.hansen@...ux.intel.com>,
        Ingo Molnar <mingo@...nel.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Josh Poimboeuf <jpoimboe@...hat.com>,
        "H. Peter Anvin" <hpa@...or.com>, Kees Cook <keescook@...omium.org>
Subject: [RFC PATCH v3 8/8] x86/entry/pti: don't switch PGD when TIF_DISABLE_PTI_NOW is set

When a syscall returns to userspace with TIF_DISABLE_PTI_NOW set on
the task, it means it is configured to disable page table isolation
(PTI). In this case, returns from kernel to user will not switch the
CR3, leaving it to the kernel one which already maps both user and
kernel pages. This avoids a TLB flush, and saves another one on next
entry.

Thanks to these changes, haproxy running under KVM went back from
12700 conn/s (without PCID) or 19700 (with PCID) to 23100 once loaded
after calling prctl(), indicating that PTI has no measurable impact on
this workload.

Signed-off-by: Willy Tarreau <w@....eu>
Cc: Andy Lutomirski <luto@...nel.org>
Cc: Borislav Petkov <bp@...en8.de>
Cc: Brian Gerst <brgerst@...il.com>
Cc: Dave Hansen <dave.hansen@...ux.intel.com>
Cc: Ingo Molnar <mingo@...nel.org>
Cc: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Peter Zijlstra <peterz@...radead.org>
Cc: Thomas Gleixner <tglx@...utronix.de>
Cc: Josh Poimboeuf <jpoimboe@...hat.com>
Cc: "H. Peter Anvin" <hpa@...or.com>
Cc: Kees Cook <keescook@...omium.org>

v3:
  - switched back to using a task flag

v2:
  - use pti_disable instead of task flag
---
 arch/x86/entry/calling.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h
index 19c6790..563478d 100644
--- a/arch/x86/entry/calling.h
+++ b/arch/x86/entry/calling.h
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: GPL-2.0 */
 #include <linux/jump_label.h>
+#include <asm/thread_info.h>
 #include <asm/unwind_hints.h>
 #include <asm/cpufeatures.h>
 #include <asm/page_types.h>
@@ -229,6 +230,12 @@

 .macro SWITCH_TO_USER_CR3_NOSTACK scratch_reg:req scratch_reg2:req
 	ALTERNATIVE "jmp .Lend_\@", "", X86_FEATURE_PTI
+
+	/* This task may be exempt from PTI */
+	movq    PER_CPU_VAR(current_task), \scratch_reg
+	btq     $TIF_DISABLE_PTI_NOW, TASK_TI_flags(\scratch_reg)
+	jc      .Lend_\@
+
 	mov	%cr3, \scratch_reg

 	ALTERNATIVE "jmp .Lwrcr3_\@", "", X86_FEATURE_PCID
-- 
1.7.12.1