linux-kernel - Re: [PATCH v6 00/20] Add VFIO mediated device support and DEV-MSI support for the idxd driver

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20210611182945.GZ1002214@nvidia.com>
Date:   Fri, 11 Jun 2021 15:29:45 -0300
From:   Jason Gunthorpe <jgg@...dia.com>
To:     Dave Jiang <dave.jiang@...el.com>
Cc:     "Tian, Kevin" <kevin.tian@...el.com>,
        Alex Williamson <alex.williamson@...hat.com>,
        Cornelia Huck <cohuck@...hat.com>,
        Kirti Wankhede <kwankhede@...dia.com>,
        "tglx@...utronix.de" <tglx@...utronix.de>,
        "vkoul@...nel.org" <vkoul@...nel.org>,
        "Raj, Ashok" <ashok.raj@...el.com>,
        "Dey, Megha" <megha.dey@...el.com>,
        "Pan, Jacob jun" <jacob.jun.pan@...el.com>,
        "Liu, Yi L" <yi.l.liu@...el.com>, "Lu, Baolu" <baolu.lu@...el.com>,
        "Kumar, Sanjay K" <sanjay.k.kumar@...el.com>,
        "Luck, Tony" <tony.luck@...el.com>,
        "Williams, Dan J" <dan.j.williams@...el.com>,
        "eric.auger@...hat.com" <eric.auger@...hat.com>,
        "pbonzini@...hat.com" <pbonzini@...hat.com>,
        "dmaengine@...r.kernel.org" <dmaengine@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "kvm@...r.kernel.org" <kvm@...r.kernel.org>
Subject: Re: [PATCH v6 00/20] Add VFIO mediated device support and DEV-MSI
 support for the idxd driver

On Fri, Jun 11, 2021 at 11:21:42AM -0700, Dave Jiang wrote:
> 
> On 6/8/2021 9:02 AM, Dave Jiang wrote:
> > 
> > On 6/7/2021 12:11 PM, Jason Gunthorpe wrote:
> > > On Mon, Jun 07, 2021 at 11:13:04AM -0700, Dave Jiang wrote:
> > > 
> > > > So in step 1, we 'tag' the wq to be dedicated to guest usage and
> > > > put the
> > > > hardware wq into enable state. For a dedicated mode wq, we can
> > > > definitely
> > > > just register directly and skip the mdev step. For a shared wq
> > > > mode, we can
> > > > have multiple mdev running on top of a single wq. So we need
> > > > some way to
> > > > create more mdevs. We can either go with the existing
> > > > established creation
> > > > path by mdev, or invent something custom for the driver as Jason
> > > > suggested
> > > > to accomodate additional virtual devices for guests. We
> > > > implemented the mdev
> > > > path originally with consideration of mdev is established and
> > > > has a known
> > > > interface already.
> > > It sounds like you could just as easially have a 'create new vfio'
> > > file under the idxd sysfs.. Especially since you already have a bus
> > > and dynamic vfio specific things being created on this bus.
> > 
> > Will explore this and using of 'struct vfio_device' without mdev.
> > 
> Hi Jason. I hacked the idxd driver to remove mdev association and use
> vfio_device directly. Ran into some issues. Specifically mdev does some
> special handling when it comes to iommu domain.

Yes, I know of this, it this needs fixing.

> effect of special handling for iommu_attach_group. And in addition, it ends
> up switching the bus to pci_bus_type before iommu_domain_alloc() is called. 
> Do we need to provide similar type of handling for vfio_device that are not
> backed by an entire PCI device like vfio_pci? Not sure it's the right thing
> to do to attach these devices to pci_bus_type directly.

Yes, type1 needs to be changed so it somehow knows that the struct
device is 'sw only', for instance because it has no direct HW IOMMU
connection, then all the mdev hackery should be deleted from type1.

I haven't investigated closely exactly how to do this.

Jason