Message-ID: <dec21a7dd4ba0f095009f9bf4657c5e8c0a4d9da.camel@infradead.org>
Date: Mon, 07 Apr 2025 15:59:03 +0100
From: David Woodhouse <dwmw2@...radead.org>
To: Christoph Hellwig <hch@...radead.org>
Cc: "Michael S. Tsirkin" <mst@...hat.com>, virtio-comment@...ts.linux.dev, 
 Claire Chang <tientzu@...omium.org>, linux-devicetree
 <devicetree@...r.kernel.org>, Rob Herring <robh+dt@...nel.org>,
 Jörg Roedel <joro@...tes.org>, 
 iommu@...ts.linux-foundation.org, linux-kernel@...r.kernel.org,
 graf@...zon.de
Subject: Re: [RFC PATCH 1/3] content: Add VIRTIO_F_SWIOTLB to negotiate use
 of SWIOTLB bounce buffers

On Mon, 2025-04-07 at 07:06 -0700, Christoph Hellwig wrote:
> On Mon, Apr 07, 2025 at 11:09:54AM +0100, David Woodhouse wrote:
> > > should be usable for all devices bar actual addressing limits that are
> > > handled in the dma layer already.  The only things you need is:
> > > 
> > >  a) a way to declare one or more pools
> > >  b) a way to destinguish between devices behind a two stage IOMMU vs not
> > >     to figure out if they need to use a pool
> > 
> > I'm not averse to that, but it's different to the `restricted-dma-pool`
> > model that's defined today which has explicit matching. So I'd like to
> > reconcile them — and preferably literally use PRP0001 to expose
> > `restricted-dma-pool` even under ACPI.
> 
> A per-device flag is probably fine and easier for some things like
> pool sizing.  It also would be worse for other things like eventually
> implementing percpu pools.

What exactly are you thinking of when you say a 'per-device' flag?

> > Maybe it's as simple as a flag/property on the `restricted-dma-pool`
> > node which declares that it's a 'catch-all', and that *all* devices
> > which aren't explicitly bound to an IOMMU or other DMA operations (e.g.
> > explicitly bound to a different restricted-dma-pool) should use it?
> 
> Yeah.  Similar what we do with the generic shared-dma-pool.

Yeah, I was trying to work that one out, and it looks like I'm almost
describing the 'linux,dma-default' property. Except that seems to be
only for coherent allocations on the `shared-dma-pool`.

Maybe it makes sense for 'linux,dma-default' on a restricted-dma-pool
to mean that it should be used as a swiotlb for *all* DMA?

I don't much like the "linux," prefix but can live with that if that's
the precedent.

> > Either way, we'd still ideally want a virtio feature bit to say "don't
> > touch me if you don't understand my DMA restrictions", to prevent older
> > drivers (on older operating systems) from failing.
> 
> As said before they really need system level awareness of a coco host.
> That's not something to be hacked into virtio drivers.

Nah. As long as they use the IOMMU they're offered, guests don't need
any awareness at all that they're running in an environment where the
VMM can't access all their memory. There is precisely *zero* enablement
required in the guest for that.

The only problem is virtual devices which are provided by that VMM.

If the guests *do* have system level awareness of the particular
variant of CoCo host they're running on, then sure, they can explicitly
manage sharing/encryption/whatever with a system-specific set of DMA
operations and the problem goes away. But that's a long way off.

How does this sound as a plan...

 1. Define 'linux,dma-default' on a 'restricted-dma-pool' object to
    mean that it should be used as a fallback SWIOTLB for *all* devices
    (basically, as if each device explicitly referenced it as they do
    today).

 2. Provide a way to represent `restricted-dma-pool` in ACPI (which
    ought to work naturally as PRP0001 and it's mostly a matter of
    untangling it from OF-specific code on the guest side).

 3. Add a VIRTIO_F_RESTRICTED_DMA feature to virtio, so that devices
    which *know* they have such a limitation can gracefully decline
    to operate with a driver which doesn't support it.

 4. Add a way for a virtio-pci device to explicitly reference a given
    `restricted-dma-pool` like virtio-mmio devices can today. And it
    probably makes sense for a virtio-pci device which requires the
    VIRTIO_F_RESTRICTED_DMA feature to explicitly reference such a
    pool even if that pool has the `linux,dma-default' property?

With #1 and #2 we provide this facility even for passthrough PCI
devices. And with #3 and #4 we make it work *better* for virtio, which
is the more important use case (since those two-stage IOMMUs *do* exist
in the cases of CoCo-supporting hardware; it's just that they don't
enable *virtual* devices).


Download attachment "smime.p7s" of type "application/pkcs7-signature" (5069 bytes)

Powered by blists - more mailing lists