linux-kernel - [RFC][PATCH 0/2] x86: Unify/extend insn_is

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250929081853.230968966@infradead.org>
Date: Mon, 29 Sep 2025 10:18:53 +0200
From: Peter Zijlstra <peterz@...radead.org>
To: x86@...nel.org,
 hpa@...or.com,
 ludloff@...il.com,
 andrew.cooper3@...rix.com,
 mhiramat@...nel.org
Cc: linux-kernel@...r.kernel.org,
 peterz@...radead.org,
 andrii@...nel.org,
 olsajiri@...il.com,
 jpoimboe@...nel.org
Subject: [RFC][PATCH 0/2] x86: Unify/extend insn_is_nop()

Hi all, welcome to an x86 instruction encoding quest.

I realized I implemented insn_is_nop() twice, obviously they have different bugs.
Additionally, I was made aware of another uprobe use-case.

As such, unify and extend insn_is_nop().

While doing this, I went down the rabbit hole of REX2/*VEX* encoded
instructions (because REX2.b4 NOP). The Intel APX document is clear as mud on
what exactly VEX/EVEX Map0 is, but the way I read it, it is the normal single
byte opcode map, no exceptions. Similarly Map1 is the 0f second byte opcode
map. So VEX prefixed NOPs should just work.

REX2 has a similar Map0/Map1 scheme, except it excludes a number of opcode
ranges -- notable not including NOP/NOPL. So REX2 prefixed NOPs are fine too.

Because I had to figure it all out anyway, I've made the function deal with
them, but perhaps the more prudent option is to simply reject all REX2/VEX
prefixed instructions entirely.

(although EVEX NOPL can hit 11 bytes without additional prefixes, so maybe
someone will start using them at some point).