linux-kernel - [PATCH 9/9] bpf,x86: Respect X86_FEATURE

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 13 Oct 2021 14:22:26 +0200
From:   Peter Zijlstra <peterz@...radead.org>
To:     x86@...nel.org, jpoimboe@...hat.com, andrew.cooper3@...rix.com
Cc:     linux-kernel@...r.kernel.org, peterz@...radead.org,
        alexei.starovoitov@...il.com, ndesaulniers@...gle.com
Subject: [PATCH 9/9] bpf,x86: Respect X86_FEATURE_RETPOLINE*

Current BPF codegen doesn't respect X86_FEATURE_RETPOLINE* flags and
unconditionally emits a thunk call, this is sub-optimal and doesn't
match the regular, compiler generated, code.

Signed-off-by: Peter Zijlstra (Intel) <peterz@...radead.org>
---
 arch/x86/net/bpf_jit_comp.c |   18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -2123,14 +2123,18 @@ static int emit_fallback_jump(u8 **pprog
 	int err = 0;
 
 #ifdef CONFIG_RETPOLINE
-	/* Note that this assumes the the compiler uses external
-	 * thunks for indirect calls. Both clang and GCC use the same
-	 * naming convention for external thunks.
-	 */
-	err = emit_jump(&prog, __x86_indirect_thunk_rdx, prog);
-#else
-	EMIT2(0xFF, 0xE2);	/* jmp rdx */
+	if (cpu_feature_enabled(X86_FEATURE_RETPOLINE)) {
+		if (cpu_feature_enabled(X86_FEATURE_RETPOLINE_AMD)) {
+			/* The AMD retpoline can be easily emitted inline. */
+			EMIT3(0x0F, 0xAE, 0xE8);	/* lfence */
+			EMIT2(0xFF, 0xE2);		/* jmp rdx */
+		} else {
+			/* Call the retpoline thunk */
+			err = emit_jump(&prog, __x86_indirect_thunk_rdx, prog);
+		}
+	} else
 #endif
+	EMIT2(0xFF, 0xE2);	/* jmp rdx */
 	*pprog = prog;
 	return err;
 }