[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <f7c39317-92a4-520e-8e69-a8606cd40e9a@linux.ibm.com>
Date: Thu, 12 Jan 2023 12:21:17 -0500
From: Matthew Rosato <mjrosato@...ux.ibm.com>
To: Jason Gunthorpe <jgg@...dia.com>,
Sean Christopherson <seanjc@...gle.com>
Cc: alex.williamson@...hat.com, pbonzini@...hat.com, cohuck@...hat.com,
farman@...ux.ibm.com, pmorel@...ux.ibm.com,
borntraeger@...ux.ibm.com, frankja@...ux.ibm.com,
imbrenda@...ux.ibm.com, david@...hat.com, akrowiak@...ux.ibm.com,
jjherne@...ux.ibm.com, pasic@...ux.ibm.com,
zhenyuw@...ux.intel.com, zhi.a.wang@...el.com,
linux-s390@...r.kernel.org, kvm@...r.kernel.org,
intel-gvt-dev@...ts.freedesktop.org,
intel-gfx@...ts.freedesktop.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH 1/2] KVM: async kvm_destroy_vm for vfio devices
On 1/12/23 7:45 AM, Jason Gunthorpe wrote:
> On Wed, Jan 11, 2023 at 08:53:34PM +0000, Sean Christopherson wrote:
>> On Wed, Jan 11, 2023, Jason Gunthorpe wrote:
>>> On Wed, Jan 11, 2023 at 07:54:51PM +0000, Sean Christopherson wrote:
>>>
>>>> Something feels off. If KVM's refcount is 0, then accessing device->group->kvm
>>>> in vfio_device_open() can't happen unless there's a refcounting bug somewhere.
>>>
>>> The problem is in close, not open.
>>
>> The deadlock problem is, yes. My point is that if group_lock needs to be taken
>> when nullifying group->kvm during kvm_vfio_destroy(), then there is also a refcounting
>> prolem with respect to open(). If there is no refcounting problem, then nullifying
>> group->kvm during kvm_vfio_destroy() is unnecessary (but again, I doubt this is
>> the case).
>
> IIRC the drivers are supposed to use one of the refcount not zero
> incrs to counteract this, but I never checked they do..
>
> Yi is working on a patch to change things so vfio drops the kvm
> pointer when the kvm file closes, not when the reference goes to 0
> to avoid a refcount cycle problem which should also solve that.
>
>> diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
>> index 6e8804fe0095..b3a84d65baa6 100644
>> --- a/drivers/vfio/vfio_main.c
>> +++ b/drivers/vfio/vfio_main.c
>> @@ -772,7 +772,12 @@ static struct file *vfio_device_open(struct vfio_device *device)
>> * reference and release it during close_device.
>> */
>> mutex_lock(&device->group->group_lock);
>> - device->kvm = device->group->kvm;
>> +
>> + if (device->kvm_ops && device->group->kvm) {
>> + ret = device->kvm_ops->get_kvm(device->group->kvm);
>
> At this point I'd rather just use the symbol get stuff like kvm does
> and call the proper functions.
>
So should I work up a v2 that does symbol gets for kvm_get_kvm_safe and kvm_put_kvm from vfio_main and drop kvm_put_kvm_async? Or is the patch Yi is working on changing things such that will also address the deadlock issue?
If so, something like the following (where vfio_kvm_get symbol gets kvm_get_kvm_safe and vfio_kvm_put symbol gets kvm_put_kvm):
diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
index 5177bb061b17..a49bf1080f0a 100644
--- a/drivers/vfio/vfio_main.c
+++ b/drivers/vfio/vfio_main.c
@@ -361,16 +361,22 @@ static int vfio_device_first_open(struct vfio_device *device,
if (ret)
goto err_module_put;
+ if (kvm && !vfio_kvm_get(kvm)) {
+ ret = -ENOENT;
+ goto err_unuse_iommu;
+ }
device->kvm = kvm;
if (device->ops->open_device) {
ret = device->ops->open_device(device);
if (ret)
- goto err_unuse_iommu;
+ goto err_put_kvm;
}
return 0;
-err_unuse_iommu:
+err_put_kvm:
+ vfio_put_kvm(kvm);
device->kvm = NULL;
+err_unuse_iommu:
if (iommufd)
vfio_iommufd_unbind(device);
else
@@ -465,6 +471,9 @@ static int vfio_device_fops_release(struct inode *inode, struct file *filep)
vfio_device_group_close(device);
+ if (device->open_count == 0 && device->group->kvm)
+ vfio_kvm_put(device->group->kvm);
+
vfio_device_put_registration(device);
return 0;
Powered by blists - more mailing lists