linux-kernel - Re: [PATCH v2 1/1] x86/fred: Clear WFE in missing-ENDBRANCH #CPs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <05745387-d391-4709-8bd8-6d82650e052f@intel.com>
Date: Thu, 7 Nov 2024 12:51:06 -0800
From: Dave Hansen <dave.hansen@...el.com>
To: "Xin Li (Intel)" <xin@...or.com>, linux-kernel@...r.kernel.org
Cc: tglx@...utronix.de, mingo@...hat.com, bp@...en8.de,
 dave.hansen@...ux.intel.com, x86@...nel.org, hpa@...or.com,
 peterz@...radead.org, andrew.cooper3@...rix.com
Subject: Re: [PATCH v2 1/1] x86/fred: Clear WFE in missing-ENDBRANCH #CPs

On 9/16/24 11:10, Xin Li (Intel) wrote:
> The WFE, i.e., WAIT_FOR_ENDBRANCH, bit in the augmented CS of FRED
> stack frame is set to 1 in missing-ENDBRANCH #CP exceptions.
> 
> The CPU will generate another missing-ENDBRANCH #CP if the WFE bit
> is left set, because the CPU IBT will be set in the WFE state upon
> completion of the following ERETS instruction and then the CPU will
> resume from the instruction that just caused the previous #CP.
> 
> Clear WFE to avoid dead looping in missing-ENDBRANCH #CPs.
> 
> Describe the IBT story in the comment of ibt_clear_fred_wfe() using
> Andrew Cooper's write-up.

I should have responded to this earlier.  I do see why Andrew thought my
earlier description was off base.  Let me see if I can try for a better
changelog:

The kernel can enable Indirect Branch Tracking (IBT) for itself.
Hardware generates a #CP exception if a kernel indirect branch lands
somewhere other than an ENDBR instruction.  The kernel #CP handler then
decides if the it should warn or do a fatal BUG().

The BUG() case works fine with or without FRED.  But the warning mode is
broken with FRED.

In short, the pre-FRED architecture clobbers the kernel IBT state of an
interrupted context.  That includes clobbering the state of IBT when the
#CP went off, suppressing future #CP's.  This is bad architecture, but
handy for a #CP handler that wants to suppress those future #CP's.

FRED, on the other hand, provides space on the entry stack (in an
expanded CS area) to save and restore IBT state. Since the hardware
doesn't clobber the IBT state, software must do it instead.

Aside:
	Why does without-FRED case work? There is only one CET WFE bit
	per privilege level.  The #CP handler itself has an ENDBR
	instruction.  That ENDBR clears WFE on the way to handling the
	#CP. Consider what would happen if a kernel indirect call landed
	on an XOR instead of an ENDBR:

	  CALL (*%rax)  // sets WFE
	  XOR %rax,%rax // uh oh, not an ENDBR
	  #CP
	  ENDBR	// first instruction in CP handler, clears WFE
	  ... handle #CP here
	  IRET
	  XOR %rax,%rax // No problem, WFE still clear!

	See? The handler clears WFE and lets the XOR run.

--

Is that a more complete (and accurate) story for folks?