Message-ID: <2ca5ce5b-039b-4477-8648-e1a292361eda@linux.intel.com>
Date: Fri, 17 Jan 2025 08:49:36 +0800
From: Binbin Wu <binbin.wu@...ux.intel.com>
To: Sean Christopherson <seanjc@...gle.com>, Kai Huang <kai.huang@...el.com>
Cc: "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
 "pbonzini@...hat.com" <pbonzini@...hat.com>, Chao Gao <chao.gao@...el.com>,
 Rick P Edgecombe <rick.p.edgecombe@...el.com>,
 Xiaoyao Li <xiaoyao.li@...el.com>,
 Reinette Chatre <reinette.chatre@...el.com>,
 Yan Y Zhao <yan.y.zhao@...el.com>, Adrian Hunter <adrian.hunter@...el.com>,
 "tony.lindgren@...ux.intel.com" <tony.lindgren@...ux.intel.com>,
 Isaku Yamahata <isaku.yamahata@...el.com>,
 "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 12/16] KVM: TDX: Inhibit APICv for TDX guest




On 1/16/2025 10:50 PM, Sean Christopherson wrote:
> On Thu, Jan 16, 2025, Kai Huang wrote:
>> On Mon, 2025-01-13 at 10:09 +0800, Binbin Wu wrote:
>>> Lazy check for pending APIC EOI when In-kernel IOAPIC
>>> -----------------------------------------------------
>>> In-kernel IOAPIC does not receive EOI with AMD SVM AVIC since the processor
>>> accelerates write to APIC EOI register and does not trap if the interrupt
>>> is edge-triggered. So there is a workaround by lazy check for pending APIC
>>> EOI at the time when setting new IOAPIC irq, and update IOAPIC EOI if no
>>> pending APIC EOI.
>>> KVM is also not be able to intercept EOI for TDX guests.
>>> - When APICv is enabled
>>>     The code of lazy check for pending APIC EOI doesn't work for TDX because
>>>     KVM can't get the status of real IRR and ISR, and the values are 0s in
>>>     vIRR and vISR in apic->regs[], kvm_apic_pending_eoi() will always return
>>>     false. So the RTC pending EOI will always be cleared when ioapic_set_irq()
>>>     is called for RTC. Then userspace may miss the coalesced RTC interrupts.
>>> - When When APICv is disabled
>>>     ioapic_lazy_update_eoi() will not be called，then pending EOI status for
>>>     RTC will not be cleared after setting and this will mislead userspace to
>>>     see coalesced RTC interrupts.
>>> Options:
>>> - Force irqchip split for TDX guests to eliminate the use of in-kernel IOAPIC.
>>> - Leave it as it is, but the use of RTC may not be accurate.
>> Looking at the code, it seems KVM only traps EOI for level-triggered interrupt
>> for in-kernel IOAPIC chip, but IIUC IOAPIC in userspace also needs to be told
>> upon EOI for level-triggered interrupt.  I don't know how does KVM works with
>> userspace IOAPIC w/o trapping EOI for level-triggered interrupt, but "force
>> irqchip split for TDX guest" seems not right.
> Forcing a "split" IRQ chip is correct, in the sense that TDX doesn't support an
> I/O APIC and the "split" model is the way to concoct such a setup.  With a "full"
> IRQ chip, KVM is responsible for emulating the I/O APIC, which is more or less
> nonsensical on TDX because it's fully virtual world, i.e. there's no reason to
> emulate legacy devices that only know how to talk to the I/O APIC (or PIC, etc.).
> Disallowing an in-kernel I/O APIC is ideal from KVM's perspective, because
> level-triggered interrupts and thus the I/O APIC as a whole can't be faithfully
> emulated (see below).
>
>> I think the problem is level-triggered interrupt,
> Yes, because the TDX Module doesn't allow the hypervisor to modify the EOI-bitmap,
> i.e. all EOIs are accelerated and never trigger exits.

Yes, and I think it needs to add the description about it and the
level-trigger interrupt in the commit message of some patch.


>
>> so I think another option is to reject level-triggered interrupt for TDX guest.
> This is a "don't do that, it will hurt" situation.  With a sane VMM, the level-ness
> of GSIs is controlled by the guest.  For GSIs that are routed through the I/O APIC,
> the level-ness is determined by the corresponding Redirection Table entry.  For
> "GSIs" that are actually MSIs (KVM piggybacks legacy GSI routing to let userspace
> wire up MSIs), and for direct MSIs injection (KVM_SIGNAL_MSI), the level-ness is
> dictated by the MSI itself, which again is guest controlled.
>
> If the guest induces generation of a level-triggered interrupt, the VMM is left
> with the choice of dropping the interrupt, sending it as-is, or converting it to
> an edge-triggered interrupt.  Ditto for KVM.  All of those options will make the
> guest unhappy.
>
> So while it _might_ make debugging broken guests either, I don't think it's worth
> the complexity to try and prevent the VMM/guest from sending level-triggered
> GSI-routed interrupts.  It'd be a bit of a whack-a-mole and there's no architectural
> behavior KVM can provide that's better than sending the interrupt and hoping for
> the best.
Currently, KVM doesn't do anything special if the guest send level-triggered
interrupts for TDX guests.
QEMU has a patch to set the eoi_intercept_unsupported to true for tdx guests.
https://lore.kernel.org/kvm/20241105062408.3533704-41-xiaoyao.li@intel.com/

And it seems that the level_trigger_unsupported info will be passed to guest
via ACPI table. I didn't dig deep into it, I suppose with the information,
guests will not send level-triggered GSI interrupts?

Powered by blists - more mailing lists