linux-kernel - Re: [PATCH 3/3] objtool/x86: Fix NOP decode

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250925124332.GA3245006@noisy.programming.kicks-ass.net>
Date: Thu, 25 Sep 2025 14:43:32 +0200
From: Peter Zijlstra <peterz@...radead.org>
To: Andrew Cooper <andrew.cooper3@...rix.com>
Cc: alexandre.chartre@...cle.com, jpoimboe@...nel.org,
	linux-kernel@...r.kernel.org, x86@...nel.org
Subject: Re: [PATCH 3/3] objtool/x86: Fix NOP decode

On Thu, Sep 25, 2025 at 12:29:18PM +0100, Andrew Cooper wrote:
> > I've made that:
> >
> > 	case 0x90:                                                                                                                                                                                                                        
> > 		if (rex_b) /* XCHG %r8, %rax */ 
> > 			break; 
> >
> > 		if (prefix == 0xf3) /* REP NOP := PAUSE */ 
> > 			break; 
> >
> > 		insn->type = INSN_NOP; 
> > 		break;
> 
> Legacy prefixes can come in any order.  What is F3 66 90 ?
> 
> Also, VEX/EVEX/REX2 want excluding too, all of which can encode rex_b
> differently.

So luckily objtool only really cares about instructions as found in the
kernel text. Neither f3 66 90 nor VEX/EVEX/REX2 prefixes are of much
concern.

But yes.. I happen to have an insn_is_nop() function that can be used on
userspace, and that certainly wants to be taught about these... x86 is
such a pain :/

> Is it really only rex_b which prevents NOP becoming a pause, or is it
> any REX prefix?  I would have thought it was any REX prefix.

SDM opcode table and instruction reference seems consistent with f3
only.