linux-kernel - Re: [PATCH 3/3] objtool/x86: Fix NOP decode

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <a2f6775f-0be5-401b-bc01-effd3f428482@citrix.com>
Date: Thu, 25 Sep 2025 12:29:18 +0100
From: Andrew Cooper <andrew.cooper3@...rix.com>
To: peterz@...radead.org
Cc: alexandre.chartre@...cle.com, jpoimboe@...nel.org,
 linux-kernel@...r.kernel.org, x86@...nel.org
Subject: Re: [PATCH 3/3] objtool/x86: Fix NOP decode

> I've made that:
>
> 	case 0x90:                                                                                                                                                                                                                        
> 		if (rex_b) /* XCHG %r8, %rax */ 
> 			break; 
>
> 		if (prefix == 0xf3) /* REP NOP := PAUSE */ 
> 			break; 
>
> 		insn->type = INSN_NOP; 
> 		break;

Legacy prefixes can come in any order.  What is F3 66 90 ?

Also, VEX/EVEX/REX2 want excluding too, all of which can encode rex_b
differently.

Is it really only rex_b which prevents NOP becoming a pause, or is it
any REX prefix?  I would have thought it was any REX prefix.

~Andrew