linux-kernel - KVM vs AMD: Re: [PATCH v3 48/59] x86/retbleed: Add SKL return thunk

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <08bbd7ab-049e-3cc3-f814-636669b856be@citrix.com>
Date:   Thu, 3 Nov 2022 22:53:54 +0000
From:   Andrew Cooper <Andrew.Cooper3@...rix.com>
To:     Nathan Chancellor <nathan@...nel.org>,
        Peter Zijlstra <peterz@...radead.org>
CC:     Thomas Gleixner <tglx@...utronix.de>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "x86@...nel.org" <x86@...nel.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Tim Chen <tim.c.chen@...ux.intel.com>,
        Josh Poimboeuf <jpoimboe@...nel.org>,
        Pawan Gupta <pawan.kumar.gupta@...ux.intel.com>,
        Johannes Wikner <kwikner@...z.ch>,
        Alyssa Milburn <alyssa.milburn@...ux.intel.com>,
        Jann Horn <jannh@...gle.com>, "H.J. Lu" <hjl.tools@...il.com>,
        Joao Moreira <joao.moreira@...el.com>,
        Joseph Nuzman <joseph.nuzman@...el.com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Juergen Gross <jgross@...e.com>,
        Masami Hiramatsu <mhiramat@...nel.org>,
        Alexei Starovoitov <ast@...nel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        K Prateek Nayak <kprateek.nayak@....com>,
        Eric Dumazet <edumazet@...gle.com>,
        Sean Christopherson <sean.j.christopherson@...el.com>,
        Paolo Bonzini <pbonzini@...hat.com>,
        kvm list <kvm@...r.kernel.org>
Subject: KVM vs AMD: Re: [PATCH v3 48/59] x86/retbleed: Add SKL return thunk

On 21/10/2022 16:21, Nathan Chancellor wrote:
> On Fri, Oct 21, 2022 at 11:53:09AM +0200, Peter Zijlstra wrote:
>> On Thu, Oct 20, 2022 at 04:10:28PM -0700, Nathan Chancellor wrote:
>>> This commit is now in -next as commit 5d8213864ade ("x86/retbleed: Add
>>> SKL return thunk"). I just bisected an immediate reboot on my AMD test
>>> system when starting a virtual machine with QEMU + KVM to it (see the
>>> bisect log below). My Intel test systems do not show this.
>>> Unfortunately, I do not have much more information, as there are no logs
>>> in journalctl, which makes sense as the reboot occurs immediately after
>>> I hit the enter key for the QEMU command.
>>>
>>> If there is any further information I can provide or patches I can test
>>> for further debugging, I am more than happy to do so.
>> Moo :-(
>>
>> you happen to have a .config for me?
> Sure thing, sorry I did not provide it in the first place! Attached. It
> has been run through localmodconfig for the particular machine but I
> assume the core pieces should still be present.

Following up from some debugging on IRC.

The problem is that FILL_RETURN_BUFFER now has a per-cpu variable
access, and AMD SVM has a fun optimisation where the VMRUN instruction
doesn't swap, amongst other things, %gs.

per-cpu variables only become safe following
vmload(__sme_page_pa(sd->save_area)); in svm_vcpu_enter_exit().

Given that retbleed=force ought to work on non-skylake hardware, the
appropriate fix is to move the VMLOAD/VMSAVE's down into asm and put
them adjacent to VMRUN.

This also addresses an undocumented dependency where its only the memory
clobber in vmload() which stops the compiler moving
svm_vcpu_enter_exit()'s calculation of sd into an unsafe position.

~Andrew