linux-kernel - Re: [PATCH v1 1/1] x86/fred: Clear the WFE bit in missing-ENDBRANCH #CP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <676eb52a-5d9f-4204-9e79-15c1d7dc2b08@zytor.com>
Date: Thu, 12 Sep 2024 01:53:41 -0700
From: Xin Li <xin@...or.com>
To: Dave Hansen <dave.hansen@...el.com>, linux-kernel@...r.kernel.org
Cc: tglx@...utronix.de, mingo@...hat.com, bp@...en8.de,
        dave.hansen@...ux.intel.com, x86@...nel.org, hpa@...or.com,
        peterz@...radead.org, andrew.cooper3@...rix.com
Subject: Re: [PATCH v1 1/1] x86/fred: Clear the WFE bit in missing-ENDBRANCH
 #CP

On 9/11/2024 5:22 PM, Dave Hansen wrote:
> On 9/11/24 16:19, Xin Li (Intel) wrote:
>> +/*
>> + * The WFE (WAIT_FOR_ENDBRANCH) bit in the augmented CS of FRED stack frame is
>> + * set to 1 in missing-ENDBRANCH #CP exceptions.
> 
> I think there's a bit of relatively irrelevant info in there.  For
> instance, I don't think it's super important to mention that FRED is
> involved and where the WFE bit is in memory.
> 
> FRED's involvement is kinda a no-brainer from the whole X86_FEATURE_FRED
> thing, and if you're reading exception handler code and don't know that
> 'regs' is on the stack, this probably isn't the place to explain that.

I often find myself in a dilemma, should I mention some technical
background which sometimes could also be distracting :(

Based on your feedback, maybe the following is better?

static void ibt_clear_fred_wfe(struct pt_regs *regs)
{
	if (regs->fred_cs.wfe)
		regs->fred_cs.wfe = 0;
}

And we know only FRED hardware will set the WFE bit.

> 
>> + * If the WFE bit is left as 1, the CPU will generate another missing-ENDBRANCH
>> + * #CP because the indirect branch tracker will be set in the WAIT_FOR_ENDBRANCH
>> + * state upon completion of the following ERETS instruction and the CPU will
>> + * restart from the IP that just caused a previous missing-ENDBRANCH #CP.
>> + *
>> + * Clear the WFE bit to avoid dead looping due to the above reason.
>> + */
>> +static void ibt_clear_fred_wfe(struct pt_regs *regs)
>> +{
>> +	if (cpu_feature_enabled(X86_FEATURE_FRED))
>> +		regs->fred_cs.wfe = 0;
>> +}
> 
> Can I suggest a slightly different comment?
> 
> /*
>   * WFE==1 (WAIT_FOR_ENDBRANCH) means that the CPU expects the next ERETS
>   * to jump to an ENDBR instruction. If the ENDBR is missing, the CPU
>   * raises a #CP.
>   *
>   * Clear WFE to avoid that #CP.
>   *
>   * Use this function in a #CP handler to effectively give the next
>   * ERETS a free pass to ignore IBT for a single instruction.
>   */
> 
> I think original comment really needs a "How do I use this?" sentence or
> two.
> 
> A comment at the call site also wouldn't hurt:
> 
>   	if (unlikely(regs->ip == (unsigned long)&ibt_selftest_noendbr)){
>   		regs->ax = 0;
> +		/* Disable IBT enforcement for one exception return: */
> +		ibt_clear_fred_wfe(regs);
>   		return;
>   	}
> 
> I'm finding it kinda hard to concisely differentiate between the
> "disable IBT at one ERETS" and "disable IBT forever", but I hope this
> sounds good to folks.
> 

My understanding is that a missing-ENDBRANCH #CP is triggered in two
steps:

     1) Upon completion of an indirect call/jmp, or an event return
        instruction, the CPU indirect branch tracker is put in the
        WAIT_FOR_ENDBRANCH state.

     2) As the CPU is in WAIT_FOR_ENDBRANCH state, if the instruction to
        be executed is ENDBR, the CPU indirect branch tracker exits
        WAIT_FOR_ENDBRANCH state, otherwise a #CP is generated.

So this is more of "preserve WAIT_FOR_ENDBRANCH state" or not.

IDT is unable to preserve WAIT_FOR_ENDBRANCH state when returning from
event handling, which as Andrew mentioned is a security hole.