linux-kernel - Re: [PATCH v3 6/6] KVM: selftests: test KVM_GUESTDBG

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87fsseo7iu.fsf@vitty.brq.redhat.com>
Date:   Tue, 02 Nov 2021 17:18:49 +0100
From:   Vitaly Kuznetsov <vkuznets@...hat.com>
To:     Sean Christopherson <seanjc@...gle.com>
Cc:     Maxim Levitsky <mlevitsk@...hat.com>,
        Paolo Bonzini <pbonzini@...hat.com>, kvm@...r.kernel.org,
        Kieran Bingham <kbingham@...nel.org>,
        Jan Kiszka <jan.kiszka@...mens.com>,
        Andrew Jones <drjones@...hat.com>,
        Jonathan Corbet <corbet@....net>,
        Ingo Molnar <mingo@...hat.com>,
        Thomas Gleixner <tglx@...utronix.de>,
        "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" <x86@...nel.org>,
        Johannes Berg <johannes.berg@...el.com>,
        Wanpeng Li <wanpengli@...cent.com>,
        "H. Peter Anvin" <hpa@...or.com>, Jessica Yu <jeyu@...nel.org>,
        Jim Mattson <jmattson@...gle.com>,
        Joerg Roedel <joro@...tes.org>,
        Yang Weijiang <weijiang.yang@...el.com>,
        linux-kernel@...r.kernel.org, Borislav Petkov <bp@...en8.de>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>,
        "open list:DOCUMENTATION" <linux-doc@...r.kernel.org>,
        Shuah Khan <shuah@...nel.org>,
        Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: [PATCH v3 6/6] KVM: selftests: test KVM_GUESTDBG_BLOCKIRQ

Sean Christopherson <seanjc@...gle.com> writes:

> On Tue, Nov 02, 2021, Vitaly Kuznetsov wrote:
>> Sean Christopherson <seanjc@...gle.com> writes:
>> > I haven't verified on hardware, but my guess is that this code in vmx_vcpu_run()
>> >
>> > 	/* When single-stepping over STI and MOV SS, we must clear the
>> > 	 * corresponding interruptibility bits in the guest state. Otherwise
>> > 	 * vmentry fails as it then expects bit 14 (BS) in pending debug
>> > 	 * exceptions being set, but that's not correct for the guest debugging
>> > 	 * case. */
>> > 	if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP)
>> > 		vmx_set_interrupt_shadow(vcpu, 0);
>> >
>> > interacts badly with APICv=1.  It will kill the STI shadow and cause the IRQ in
>> > vmcs.GUEST_RVI to be recognized when it (micro-)architecturally should not.  My
>> > head is going in circles trying to sort out what would actually happen.  Maybe
>> > comment out that and/or disable APICv to see if either one makes the test pass?
>> >
>> 
>> Interestingly,
>> 
>> loading 'kvm-intel' with 'enable_apicv=0' makes the test pass, however,
>> commenting out "vmx_set_interrupt_shadow()" as suggested gives a
>> different result (with enable_apicv=1):
>> 
>> # ./x86_64/debug_regs 
>> ==== Test Assertion Failure ====
>>   x86_64/debug_regs.c:179: run->exit_reason == KVM_EXIT_DEBUG && run->debug.arch.exception == DB_VECTOR && run->debug.arch.pc == target_rip && run->debug.arch.dr6 == target_dr6
>>   pid=16352 tid=16352 errno=0 - Success
>>      1	0x0000000000402b33: main at debug_regs.c:179 (discriminator 10)
>>      2	0x00007f36401bd554: ?? ??:0
>>      3	0x00000000004023a9: _start at ??:?
>>   SINGLE_STEP[1]: exit 9 exception -2147483615 rip 0x1 (should be 0x4024d9) dr6 0xffff4ff0 (should be 0xffff4ff0)
>
> Exit 9 is KVM_EXIT_FAIL_ENTRY, which in this case VM-Entry likely failed due to
> invalid guest state because there was STI blocking with single-step enabled but
> no pending BS #DB:
>
>   Bit 14 (BS) must be 1 if the TF flag (bit 8) in the RFLAGS field is 1 and the
>   BTF flag (bit 1) in the IA32_DEBUGCTL field is 0.
>
> Which is precisely what that hack-a-fix avoids.  There isn't really a clean
> solution for legacy single-step, AFAIK the only way to avoid this would be to
> switch KVM_GUESTDBG_SINGLESTEP to use MTF.
>
> But that mess is a red herring, the test fails with the same signature with APICv=1
> if the STI is replaced by PUSHF+BTS+POPFD (to avoid the STI shadow).  We all missed
> this key detail from Vitaly's report:
>
> SINGLE_STEP[1]: exit 8 exception 1 rip 0x402a25 (should be 0x402a27) dr6 0xffff4ff0 (should be 0xffff4ff0)
>                 ^^^^^^
>
> Exit '8' is KVM_EXIT_SHUTDOWN, i.e. the arrival of the IRQ hosed the guest because
> the test doesn't invoke vm_init_descriptor_tables() to install event handlers.
> The "exception 1" shows up because the run page isn't sanitized by the test, i.e.
> it's stale data that happens to match.
>
> So I would fully expect this test to fail with AVIC=1.  The problem is that
> KVM_GUESTDBG_BLOCKIRQ does absolutely nothing to handle APICv interrupts.  And
> even if KVM does something to fudge that behavior in the emulated local APIC, the
> test will then fail miserably virtual IPIs (currently AVIC only).

FWIW, the test doesn't seem to fail on my AMD EPYC system even with "avic=1" ...

>
> I stand by my original comment that "Deviating this far from architectural behavior
> will end in tears at some point."  Rather than try to "fix" APICv, I vote to instead
> either reject KVM_GUESTDBG_BLOCKIRQ if APICv=1, or log a debug message saying that
> KVM_GUESTDBG_BLOCKIRQ is ineffective with APICv=1.
>

-- 
Vitaly