linux-kernel - [PATCH RFC v1 16/20] KVM: x86: Decode REX2 prefix in the emulator

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20251110180131.28264-17-chang.seok.bae@intel.com>
Date: Mon, 10 Nov 2025 18:01:27 +0000
From: "Chang S. Bae" <chang.seok.bae@...el.com>
To: kvm@...r.kernel.org,
	linux-kernel@...r.kernel.org
Cc: pbonzini@...hat.com,
	seanjc@...gle.com,
	chao.gao@...el.com,
	zhao1.liu@...el.com,
	chang.seok.bae@...el.com
Subject: [PATCH RFC v1 16/20] KVM: x86: Decode REX2 prefix in the emulator

Extend the instruction emulator to recognize and interpret the REX2
prefix byte. Also, detect and flag invalid prefix sequences after a REX2
prefix.

In the existing prefix-decoding loop,
  * The loop exits when the first non-prefix byte is encountered.
  * Any non-REX prefix clears previously recorded REX information.

For REX2, however, once a REX2 prefix is encountered, most subsequent
prefixes are invalid. So, each subsequent prefix needs to be validated
before continuing the loop.

Signed-off-by: Chang S. Bae <chang.seok.bae@...el.com>
---
RFC note:
The REX2 decoding itself is straightforward. The additional logic is
mainly to detect and handle invalid prefix sequences. If this seems
excessive, there is a chance to cut off this check since VMX would raise
'#UD' on such cases anyway.
---
 arch/x86/kvm/emulate.c | 24 +++++++++++++++++++++++-
 1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
index 9bd61ea496e5..f9381a4055d6 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
@@ -4844,7 +4844,7 @@ int x86_decode_insn(struct x86_emulate_ctxt *ctxt, void *insn, int insn_len, int
 	ctxt->op_bytes = def_op_bytes;
 	ctxt->ad_bytes = def_ad_bytes;
 
-	/* Legacy prefixes. */
+	/* Legacy and REX/REX2 prefixes. */
 	for (;;) {
 		switch (ctxt->b = insn_fetch(u8, ctxt)) {
 		case 0x66:	/* operand-size override */
@@ -4887,9 +4887,20 @@ int x86_decode_insn(struct x86_emulate_ctxt *ctxt, void *insn, int insn_len, int
 		case 0x40 ... 0x4f: /* REX */
 			if (mode != X86EMUL_MODE_PROT64)
 				goto done_prefixes;
+			if (ctxt->rex_prefix == REX2_PREFIX)
+				break;
 			ctxt->rex_prefix = REX_PREFIX;
 			ctxt->rex.raw    = 0x0f & ctxt->b;
 			continue;
+		case 0xd5: /* REX2 */
+			if (mode != X86EMUL_MODE_PROT64)
+				goto done_prefixes;
+			if (ctxt->rex_prefix == REX2_PREFIX &&
+			    ctxt->rex.bits.m0 == 0)
+				break;
+			ctxt->rex_prefix = REX2_PREFIX;
+			ctxt->rex.raw    = insn_fetch(u8, ctxt);
+			continue;
 		case 0xf0:	/* LOCK */
 			ctxt->lock_prefix = 1;
 			break;
@@ -4901,6 +4912,17 @@ int x86_decode_insn(struct x86_emulate_ctxt *ctxt, void *insn, int insn_len, int
 			goto done_prefixes;
 		}
 
+		if (ctxt->rex_prefix == REX2_PREFIX) {
+			/*
+			 * A legacy or REX prefix following a REX2 prefix
+			 * forms an invalid byte sequences. Likewise,
+			 * a second REX2 prefix following a REX2 prefix
+			 * with M0=0 is invalid.
+			 */
+			ctxt->rex_prefix = REX2_INVALID;
+			goto done_prefixes;
+		}
+
 		/* Any legacy prefix after a REX prefix nullifies its effect. */
 		ctxt->rex_prefix = REX_NONE;
 		ctxt->rex.raw = 0;
-- 
2.51.0