linux-kernel - Re: [RFC PATCH] x86/split_lock: Disable SLD if an unaware (out-of-tree) module enables VMX

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200403172018.GD2701@linux.intel.com>
Date:   Fri, 3 Apr 2020 10:20:18 -0700
From:   Sean Christopherson <sean.j.christopherson@...el.com>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>,
        x86@...nel.org, "H. Peter Anvin" <hpa@...or.com>,
        linux-kernel@...r.kernel.org,
        "Kenneth R. Crudup" <kenny@...ix.com>,
        Jessica Yu <jeyu@...nel.org>,
        Rasmus Villemoes <rasmus.villemoes@...vas.dk>,
        Paolo Bonzini <pbonzini@...hat.com>,
        Fenghua Yu <fenghua.yu@...el.com>,
        Xiaoyao Li <xiaoyao.li@...el.com>,
        Nadav Amit <nadav.amit@...il.com>,
        Thomas Hellstrom <thellstrom@...are.com>,
        Tony Luck <tony.luck@...el.com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Jann Horn <jannh@...gle.com>,
        Kees Cook <keescook@...omium.org>,
        David Laight <David.Laight@...LAB.COM>,
        Doug Covelli <dcovelli@...are.com>
Subject: Re: [RFC PATCH] x86/split_lock: Disable SLD if an unaware
 (out-of-tree) module enables VMX

On Fri, Apr 03, 2020 at 06:42:44PM +0200, Peter Zijlstra wrote:
> On Fri, Apr 03, 2020 at 09:30:07AM -0700, Sean Christopherson wrote:
> > Hook into native CR4 writes to disable split-lock detection if CR4.VMXE
> > is toggled on by an SDL-unaware entity, e.g. an out-of-tree hypervisor
> > module.  Most/all VMX-based hypervisors blindly reflect #AC exceptions
> > into the guest, or don't intercept #AC in the first place.  With SLD
> > enabled, this results in unexpected #AC faults in the guest, leading to
> > crashes in the guest and other undesirable behavior.
> > 
> > Reported-by: "Kenneth R. Crudup" <kenny@...ix.com>
> > Cc: Peter Zijlstra <peterz@...radead.org>
> > Cc: Thomas Gleixner <tglx@...utronix.de>
> > Cc: Jessica Yu <jeyu@...nel.org>
> > Cc: Rasmus Villemoes <rasmus.villemoes@...vas.dk>
> > Cc: Kenneth R. Crudup <kenny@...ix.com>
> > Cc: Paolo Bonzini <pbonzini@...hat.com>
> > Cc: Fenghua Yu <fenghua.yu@...el.com>
> > Cc: Xiaoyao Li <xiaoyao.li@...el.com>
> > Cc: Nadav Amit <namit@...are.com>
> > Cc: Thomas Hellstrom <thellstrom@...are.com>
> > Cc: Tony Luck <tony.luck@...el.com>
> > Cc: Steven Rostedt <rostedt@...dmis.org>
> > Cc: Greg Kroah-Hartman <gregkh@...uxfoundation.org>
> > Cc: Jann Horn <jannh@...gle.com>
> > Cc: Kees Cook <keescook@...omium.org>
> > Cc: David Laight <David.Laight@...LAB.COM>
> > Cc: Doug Covelli <dcovelli@...are.com>
> > Signed-off-by: Sean Christopherson <sean.j.christopherson@...el.com>
> > ---
> > 
> > A bit ugly, but on the plus side the code is largely contained to intel.c.
> > I think forgoing the on_all_cpus() remote kill is safe? 
> 
> How would it be safe? You can't control where the module text will be
> ran, or how quickly.

Ugh, I forgot about the stupid core scope behavior.

CR4.VMXE needs to be set on every logical CPU before that CPU can do VMXON
and enter a guest, so every CPU will come through this code and locally
disable SLD.

But, a SMT sibling could race on the WRMSR and re-enable SLD on the CPU
that just killed SLD.  Waiting until other CPUs stop enabling SLD should
work.  Something like this?  Disclaimer, memory ordering isn't my forte.

static atomic_t enabling_sld = ATOMIC_INIT(0);

static void sld_update_msr(bool on)
{
	u64 test_ctrl_val = msr_test_ctrl_cache;

	if (on && !sld_killed)
		test_ctrl_val |= MSR_TEST_CTRL_SPLIT_LOCK_DETECT;

	if (test_ctrl_val & MSR_TEST_CTRL_SPLIT_LOCK_DETECT)
		atomic_inc(&enabling_sld);

	wrmsrl(MSR_TEST_CTRL, test_ctrl_val);

	if (test_ctrl_val & MSR_TEST_CTRL_SPLIT_LOCK_DETECT)
		atomic_dec(&enabling_sld);
}

void split_lock_cr4_write(unsigned long val)
{
	u64 ctrl;

	/*
	 * Out-of-tree hypervisors that aren't aware of split-lock will blindly
	 * reflect split-lock #AC into their guests.  Kill split-lock detection
	 * if an unaware entity enables VMX.
	 */
	if (!static_cpu_has(X86_FEATURE_VMX) ||
	    !static_cpu_has(X86_FEATURE_SPLIT_LOCK_DETECT) ||
	    !(val & X86_CR4_VMXE) || atomic_read(&cr4_vmxe_split_lock_safe) ||
	    (native_read_cr4() & X86_CR4_VMXE))
		return;

	WARN_ON_ONCE(1);

	/*
	 * Set the global kill flag to prevent re-enabling SLD, e.g. via
	 * switch_to_sld().
	 */
	WRITE_ONCE(sld_killed, true);

	/*
	 * No need to forcefully disable SLD on other CPUs, they'll come here
	 * if/when they set CR4.VMXE.  But, wait until no other threads are
	 * enabling SLD, i.e. have seen sld_killed, as the MSR may be shared
	 * by SMT siblings.
	 */
	while (atomic_read(&enabling_sld));
	sld_update_msr(false);
}