linux-kernel - Re: [PATCH v5 3/4] mm/gup: allow FOLL_LONGTERM & FOLL_PCI

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20240815163811.GN3468552@ziepe.ca>
Date: Thu, 15 Aug 2024 13:38:11 -0300
From: Jason Gunthorpe <jgg@...pe.ca>
To: Christoph Hellwig <hch@...radead.org>
Cc: Martin Oliveira <martin.oliveira@...eticom.com>,
	linux-rdma@...r.kernel.org, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org, Andrew Morton <akpm@...ux-foundation.org>,
	Artemy Kovalyov <artemyko@...dia.com>,
	Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
	Leon Romanovsky <leon@...nel.org>,
	Logan Gunthorpe <logang@...tatee.com>,
	Michael Guralnik <michaelgur@...dia.com>,
	Mike Marciniszyn <mike.marciniszyn@...el.com>,
	Shiraz Saleem <shiraz.saleem@...el.com>, Tejun Heo <tj@...nel.org>,
	John Hubbard <jhubbard@...dia.com>,
	Dan Williams <dan.j.williams@...el.com>,
	David Sloan <david.sloan@...eticom.com>
Subject: Re: [PATCH v5 3/4] mm/gup: allow FOLL_LONGTERM & FOLL_PCI_P2PDMA

On Tue, Aug 13, 2024 at 09:26:55PM -0700, Christoph Hellwig wrote:
> On Tue, Aug 13, 2024 at 01:05:02PM -0300, Jason Gunthorpe wrote:
> > > Where do we block driver unbind with an open resource?  
> > 
> > I keep seeing it in different subsystems, safe driver unbind is really
> > hard. :\ eg I think VFIO has some waits in it
> 
> Well, waits for what? 

Looks like it waits for the fds to close at least. It has weird
handshake where it notifies userspace that the device is closing and
the userspace needs to close the fd. See vfio_unregister_group_dev()

In the past VFIO couldn't do anything else because it had VMAs open
that point to the device's mmio and it would be wrong to unbind the
driver and leave those uncleaned. VFIO now knows how to zap VMA so
maybe this could be improved, but it looks like a sizable uplift.

> Usuaully an unbind has to wait for something, but waiting for
> unbound references is always a bug.  And I can't think of any
> sensible subsystem doing this.

I've seen several cases over the years.. It can be hard in many cases
to deal with the issue. Like the VMA's above for instance. rdma also
had to learn how to do zap before it could do fast unbind.

Some places just have bugs where they don't wait and Weird Stuff
happens. There is a strange misconception that module refcounts
protect against unbind :\

Not saying this is good design, or desirable, just pointing out we
seem to tolerate this in enough other places.

In this case the only resolution I could think of would be to have the
rdma stack somehow register a notifier on the pgmap. Then we could
revoke the RDMA access synchronously during destruction and return
those refcounts.

That does seems a bit complicated though..

Jason