linux-kernel - Re: [PATCH v2] x86: add hintable NOPs emulation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <9EDED468-AB68-4558-8D94-C3756170C364@zytor.com>
Date: Wed, 20 Aug 2025 08:31:48 -0700
From: "H. Peter Anvin" <hpa@...or.com>
To: Marcos Del Sol Vives <marcos@...a.pet>, linux-kernel@...r.kernel.org
CC: marcos@...a.pet, Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>,
        Dave Hansen <dave.hansen@...ux.intel.com>, x86@...nel.org,
        Peter Zijlstra <peterz@...radead.org>, Kees Cook <kees@...nel.org>,
        "Xin Li (Intel)" <xin@...or.com>,
        Sabyrzhan Tasbolatov <snovitoll@...il.com>
Subject: Re: [PATCH v2] x86: add hintable NOPs emulation

On August 20, 2025 4:04:35 AM PDT, Marcos Del Sol Vives <marcos@...a.pet> wrote:
>Hintable NOPs are a series of instructions introduced by Intel with the
>Pentium Pro (i686), and described in US patent US5701442A.
>
>These instructions were reserved to allow backwards-compatible changes
>in the instruction set possible, by having old processors treat them as
>variable-length NOPs, while having other semantics in modern processors.
>
>Some modern uses are:
> - Multi-byte/long NOPs
> - Indirect Branch Tracking (ENDBR32)
> - Shadow Stack (part of CET)
>
>Some processors advertising i686 compatibility lack full support for
>them, which may cause #UD to be incorrectly triggered, crashing software
>that uses then with an unexpected SIGILL.
>
>One such software is sudo in Debian bookworm, which is compiled with
>GCC -fcf-protection=branch and contains ENDBR32 instructions. It crashes
>on my Vortex86DX3 processor and VIA C3 Nehalem processors [1].
>
>This patch is a much simplified version of my previous patch for x86
>instruction emulation [2], that only emulates hintable NOPs.
>
>When #UD is raised, it checks if the opcode corresponds to a hintable NOP
>in user space. If true, it warns the user via the dmesg and advances the
>instruction pointer, thus emulating its expected NOP behaviour.
>
>[1]: https://lists.debian.org/debian-devel/2023/10/msg00118.html
>[2]: https://lore.kernel.org/all/20210626130313.1283485-1-marcos@orca.pet/
>
>Signed-off-by: Marcos Del Sol Vives <marcos@...a.pet>
>---
> arch/x86/Kconfig        | 29 +++++++++++++++++++++++++++++
> arch/x86/kernel/traps.c | 33 +++++++++++++++++++++++++++++++++
> 2 files changed, 62 insertions(+)
>
>diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
>index 58d890fe2100..a6daebdc2573 100644
>--- a/arch/x86/Kconfig
>+++ b/arch/x86/Kconfig
>@@ -1286,6 +1286,35 @@ config X86_IOPL_IOPERM
> 	  ability to disable interrupts from user space which would be
> 	  granted if the hardware IOPL mechanism would be used.
> 
>+config X86_HNOP_EMU
>+	bool "Hintable NOPs emulation"
>+	depends on X86_32
>+	default y
>+	help
>+	  Hintable NOPs are a series of instructions introduced by Intel with
>+	  the Pentium Pro (i686), and described in US patent US5701442A.
>+
>+	  These instructions were reserved to allow backwards-compatible
>+	  changes in the instruction set possible, by having old processors
>+	  treat them as variable-length NOPs, while having other semantics in
>+	  modern processors.
>+
>+	  Some modern uses are:
>+	   - Multi-byte/long NOPs
>+	   - Indirect Branch Tracking (ENDBR32)
>+	   - Shadow Stack (part of CET)
>+
>+	  Some processors advertising i686 compatibility (such as Cyrix MII,
>+	  VIA C3 Nehalem or DM&P Vortex86DX3) lack full support for them,
>+	  which may cause SIGILL to be incorrectly raised in user space when
>+	  a hintable NOP is encountered.
>+
>+	  Say Y here if you want the kernel to emulate them, allowing programs
>+	  that make use of them to run transparently on such processors.
>+
>+	  This emulation has no performance penalty for processors that
>+	  properly support them, so if unsure, enable it.
>+
> config TOSHIBA
> 	tristate "Toshiba Laptop support"
> 	depends on X86_32
>diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
>index 36354b470590..22b51c4186e7 100644
>--- a/arch/x86/kernel/traps.c
>+++ b/arch/x86/kernel/traps.c
>@@ -295,12 +295,45 @@ DEFINE_IDTENTRY(exc_overflow)
> 	do_error_trap(regs, 0, "overflow", X86_TRAP_OF, SIGSEGV, 0, NULL);
> }
> 
>+static bool handle_hnop(struct pt_regs *regs)
>+{
>+	unsigned char buf[MAX_INSN_SIZE];
>+	unsigned long nr_copied;
>+	struct insn insn;
>+
>+	if (!IS_ENABLED(CONFIG_X86_HNOP_EMU))
>+		return false;
>+
>+	nr_copied = insn_fetch_from_user(regs, buf);
>+	if (nr_copied <= 0)
>+		return false;
>+
>+	if (!insn_decode_from_regs(&insn, regs, buf, nr_copied))
>+		return false;
>+
>+	/* Hintable NOPs cover 0F 18 to 0F 1F */
>+	if (insn.opcode.bytes[0] != 0x0F ||
>+	    insn.opcode.bytes[1] < 0x18 || insn.opcode.bytes[1] > 0x1F)
>+		return false;
>+
>+	pr_warn_once("%s[%d] uses hintable NOPs that your processor does not support.\n"
>+		     "The kernel is emulating them; the performance of this "
>+		     "and other executables using them will be impacted.\n",
>+		     current->comm, task_pid_nr(current));
>+
>+	regs->ip += insn.length;
>+	return true;
>+}
>+
> #ifdef CONFIG_X86_F00F_BUG
> void handle_invalid_op(struct pt_regs *regs)
> #else
> static inline void handle_invalid_op(struct pt_regs *regs)
> #endif
> {
>+	if (user_mode(regs) && handle_hnop(regs))
>+		return;
>+
> 	do_error_trap(regs, 0, "invalid opcode", X86_TRAP_UD, SIGILL,
> 		      ILL_ILLOPN, error_get_trap_addr(regs));
> }

Interesting that they build with endbr32 support, since the kernel vdso isn't endbr32 compliant, as I found...