linux-kernel - [RFC PATCH v2 6/6] x86/entry/pti: don't switch PGD on when pti

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1515502580-12261-7-git-send-email-w@1wt.eu>
Date:   Tue,  9 Jan 2018 13:56:20 +0100
From:   Willy Tarreau <w@....eu>
To:     linux-kernel@...r.kernel.org, x86@...nel.org
Cc:     Willy Tarreau <w@....eu>, Andy Lutomirski <luto@...nel.org>,
        Borislav Petkov <bp@...en8.de>,
        Brian Gerst <brgerst@...il.com>,
        Dave Hansen <dave.hansen@...ux.intel.com>,
        Ingo Molnar <mingo@...nel.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Josh Poimboeuf <jpoimboe@...hat.com>,
        "H. Peter Anvin" <hpa@...or.com>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Kees Cook <keescook@...omium.org>
Subject: [RFC PATCH v2 6/6] x86/entry/pti: don't switch PGD on when pti_disable is set

When a syscall returns to userspace with pti_disable set, it means the
current mm is configured to disable page table isolation (PTI). In this
case, returns from kernel to user will not switch the CR3, leaving it
to the kernel one which already maps both user and kernel pages. This
avoids a TLB flush, and saves another one on next entry.

Thanks to these changes, haproxy running under KVM went back from
12700 conn/s (without PCID) or 19700 (with PCID) to 23100 once loaded
after calling prctl(), indicating that PTI has no measurable impact on
this workload.

Signed-off-by: Willy Tarreau <w@....eu>
Cc: Andy Lutomirski <luto@...nel.org>
Cc: Borislav Petkov <bp@...en8.de>
Cc: Brian Gerst <brgerst@...il.com>
Cc: Dave Hansen <dave.hansen@...ux.intel.com>
Cc: Ingo Molnar <mingo@...nel.org>
Cc: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Peter Zijlstra <peterz@...radead.org>
Cc: Thomas Gleixner <tglx@...utronix.de>
Cc: Josh Poimboeuf <jpoimboe@...hat.com>
Cc: "H. Peter Anvin" <hpa@...or.com>
Cc: Greg Kroah-Hartman <gregkh@...uxfoundation.org>
Cc: Kees Cook <keescook@...omium.org>

v2:
  - use pti_disable instead of task flag
---
 arch/x86/entry/calling.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h
index 2c0d3b5..5361a10 100644
--- a/arch/x86/entry/calling.h
+++ b/arch/x86/entry/calling.h
@@ -229,6 +229,11 @@

 .macro SWITCH_TO_USER_CR3_NOSTACK scratch_reg:req scratch_reg2:req
 	ALTERNATIVE "jmp .Lend_\@", "", X86_FEATURE_PTI
+
+	/* The "pti_disable" mm attribute is mirrored into this per-cpu var */
+	cmpb	$0, PER_CPU_VAR(pti_disable)
+	jne	.Lend_\@
+
 	mov	%cr3, \scratch_reg

 	ALTERNATIVE "jmp .Lwrcr3_\@", "", X86_FEATURE_PCID
-- 
1.7.12.1