Message-ID: <f2f82341-3799-4379-a0e7-6e9d56a7eda1@amd.com>
Date: Mon, 12 Jan 2026 17:12:36 +0100
From: Christian König <christian.koenig@....com>
To: Jason Gunthorpe <jgg@...pe.ca>
Cc: Simona Vetter <simona.vetter@...ll.ch>, Leon Romanovsky
 <leon@...nel.org>, Sumit Semwal <sumit.semwal@...aro.org>,
 Alex Williamson <alex@...zbot.org>, Kevin Tian <kevin.tian@...el.com>,
 Joerg Roedel <joro@...tes.org>, Will Deacon <will@...nel.org>,
 Robin Murphy <robin.murphy@....com>, linux-rdma@...r.kernel.org,
 linux-kernel@...r.kernel.org, linux-media@...r.kernel.org,
 dri-devel@...ts.freedesktop.org, linaro-mm-sig@...ts.linaro.org,
 kvm@...r.kernel.org, iommu@...ts.linux.dev
Subject: Re: [PATCH 0/4] dma-buf: add revoke mechanism to invalidate shared
 buffers

On 1/12/26 16:35, Jason Gunthorpe wrote:
> On Mon, Jan 12, 2026 at 03:56:32PM +0100, Christian König wrote:
>>> The problem revoke is designed to solve is that many importers have
>>> hardware that can either be DMA'ing or failing. There is no fault
>>> mechanims that can be used to implement the full "move around for no
>>> reason" semantics that are implied by move_notify.
>>
>> In this case just call dma_buf_pin(). We already support that
>> approach for RDMA devices which can't do ODP.
> 
> That alone isn't good enough - the patch adding the non-ODP support
> also contained this:
> 
> static void
> ib_umem_dmabuf_unsupported_move_notify(struct dma_buf_attachment *attach)
> {
> 	struct ib_umem_dmabuf *umem_dmabuf = attach->importer_priv;
> 
> 	ibdev_warn_ratelimited(umem_dmabuf->umem.ibdev,
> 			       "Invalidate callback should not be called when memory is pinned\n");
> }

Yeah, I know. That's what I meant we have to better document this.

> 
> static struct dma_buf_attach_ops ib_umem_dmabuf_attach_pinned_ops = {
> 	.allow_peer2peer = true,
> 	.move_notify = ib_umem_dmabuf_unsupported_move_notify,
> };
> 
> So we can't just allow it to attach to exporters that are going to
> start calling move_notify while pinned.

The point is exporters are already doing this.

> Looking around I don't see anyone else doing something like this, and
> reading your remarks I think EFA guys got it wrong. So I'm wondering
> if this should not have been allowed. Unfortunately 5 years later I'm
> pretty sure it is being used in places where we don't have HW support
> to invalidate at all, and it is now well established uAPI that we
> can't just break.
> 
> Which is why we are coming to negotiation because at least the above
> isn't going to work if move_notify is called for revoke reasons, and
> we'd like to block attaching exporters that need revoke for the above.

Ah, yes that makes sense. This is clearly a new requirement.

So basically for PCIe hotplug was a rare event were we said we have some problems with non-ODP but we can live with that, but for this use case here it's more like a perfectly normal condition that userspace can trigger.

So the exporter wants to reject importers which can't handle a mapping invalidation while the BO is pinned, correct?

> 
> So, would you be happier with this if we documented that move_notify
> can be called for pinned importers for revoke purposes and figure out
> something to mark the above as special so exporters can fail pin if
> they are going to call move_notify?

That would work for me. I mean it is already current practice, we just never fully documented it.

> 
> Then this series would transform into documentation, making VFIO
> accept pin and continue to call move_notify as it does right now, and
> some logic to reject the RDMA non-ODP importer.

I think we just need to expose this with flags or similar from the importer side. As far as I know RDMA without ODP is currently the only one really needing this (except for cross device scanout, but that is special anyway).

Christian.

> 
> Jason

Powered by blists - more mailing lists