Message-ID: <ZvqXBY12_53HLV9k@google.com>
Date: Mon, 30 Sep 2024 12:18:13 +0000
From: Mostafa Saleh <smostafa@...gle.com>
To: Yi Liu <yi.l.liu@...el.com>
Cc: kvm@...r.kernel.org, open list <linux-kernel@...r.kernel.org>,
	Eric Auger <eric.auger@...hat.com>,
	Alex Williamson <alex.williamson@...hat.com>, kwankhede@...dia.com,
	Marc Zyngier <maz@...nel.org>, Will Deacon <will@...nel.org>,
	Quentin Perret <qperret@...gle.com>
Subject: Re: [RFC] Simple device assignment with VFIO platform

Hi Yi,

Thanks a lot for the feedback.

On Mon, Sep 30, 2024 at 03:19:05PM +0800, Yi Liu wrote:
> On 2024/9/28 00:17, Mostafa Saleh wrote:
> > Hi All,
> > 
> > Background
> > ==========
> > I have been looking into assigning simple devices which are not DMA
> > capable to VMs on Android using VFIO platform.
> > 
> > I have been mainly looking with respect to Protected KVM (pKVM), which
> > would need some extra modifications mostly to KVM-VFIO, that is quite
> > early under prototyping at the moment, which have core pending pKVM
> > dependencies upstream as guest memfd[1] and IOMMUs support[2].
> > 
> > However, this problem is not pKVM(or KVM) specific, and about the
> > design of VFIO.
> > 
> > [1] https://lore.kernel.org/kvm/20240801090117.3841080-1-tabba@google.com/
> > [2] https://lore.kernel.org/kvmarm/20230201125328.2186498-1-jean-philippe@linaro.org/
> > 
> > Problem
> > =======
> > At the moment, VFIO platform will deny a device from probing (through
> > vfio_group_find_or_alloc()), if it’s not part of an IOMMU group,
> > unless (CONFIG_VFIO_NOIOMMU is configured)
> 
> so your device does not have an IOMMU and also it does not do DMA at all?

Exactly.

> 
> > As far as I understand the current solutions to pass through platform
> > devices that are not DMA capable are:
> > - Use VFIO platform + (CONFIG_VFIO_NOIOMMU): The problem with that, it
> > taints the kernel and this doesn’t actually fit the device description
> > as the device doesn’t only have an IOMMU, but it’s not DMA capable at
> > all, so the kernel should be safe with assigning the device without
> > DMA isolation.
> 
> you need to set the vfio_noiommu parameter as well. yes, this would give
> your device a fake iommu group. But the kernel would say this taints it.

Yes, that would be the main problem with this approach.

> 
> > 
> > - Use VFIO mdev with an emulated IOMMU, this seems it could work. But
> > many of the code would be duplicate with the VFIO platform code as the
> > device is a platform device.
> > 
> > - Use UIO: Can map MMIO to userspace which seems to be focused for
> > userspace drivers rather than VM passthrough and I can’t find its
> > support in Qemu.
> 
> QEMU is for device passthrough, it makes sense it needs to use the VFIO
> without noiommu instead of UIO. The below link has more explanations.
> 

I agree, the reason I considered UIO, is that it allows mmap of the device
MMIO, so in theory those can be passed to KVM, but that might be abuse of
the UAPI? Specially I can’t find any VMM support for that.

> https://wiki.qemu.org/Features/VT-d
> 
> As the introduction of vfio cdev, you may compile the vfio group out
> by CONFIG_VFIO_GROUP==n. Supposedly, you will not be blocked by the
> vfio_group_find_or_alloc(). But you might be blocked due to no present
> iommu. You may have a try though.

That fails at probe also, because of the device has no IOMMU as
vfio_platform_probe() ends up calling device_iommu_capable().

Also, AFAIU, using cdev must be tied with IOMMUFD, which assumes
existence of IOMMU and would failt at bind.

> 
> > One other benefit from supporting this in VFIO platform, that we can
> > use the existing UAPI for platform devices (and support in VMMs)
> > 
> > Proposal
> > ========
> > Extend VFIO platform to allow assigning devices without an IOMMU, this
> > can be possibly done by
> > - Checking device capability from the platform bus (would be something
> > ACPI/OF specific similar to how it configures DMA from
> > platform_dma_configure(), we can add a new function something like
> > platfrom_dma_capable())
> > 
> > - Using emulated IOMMU for such devices
> > (vfio_register_emulated_iommu_dev()), instead of having intrusive
> > changes about IOMMUs existence.
> 
> is it the mdev approach listed in the above?

No, I meant to extended VFIO-platform to understand that, so in case the
device has no IOMMU group, it can check with the platform bus if the
device is DMA capable or not and it can allow those devices (maybe
gated behind a command line) to be probed with emulated group.

Having an new mdev just for platform devices with no DMA capabilities might
be good if we don’t want to change VFIO platform, but my main concern is that
might need more code + duplication with VFIO-platform.

Thanks,
Mostafa

> -- 
> Regards,
> Yi Liu

Powered by blists - more mailing lists