linux-kernel - RE: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <MWHPR11MB1886C5A82E4846A86F4D31988C529@MWHPR11MB1886.namprd11.prod.outlook.com>
Date:   Wed, 12 May 2021 00:40:04 +0000
From:   "Tian, Kevin" <kevin.tian@...el.com>
To:     Jason Gunthorpe <jgg@...dia.com>
CC:     Jean-Philippe Brucker <jean-philippe@...aro.org>,
        Li Zefan <lizefan@...wei.com>,
        "Jiang, Dave" <dave.jiang@...el.com>,
        "Raj, Ashok" <ashok.raj@...el.com>,
        Jonathan Corbet <corbet@....net>,
        "Jean-Philippe Brucker" <jean-philippe@...aro.com>,
        LKML <linux-kernel@...r.kernel.org>,
        "iommu@...ts.linux-foundation.org" <iommu@...ts.linux-foundation.org>,
        "Alex Williamson" <alex.williamson@...hat.com>,
        Johannes Weiner <hannes@...xchg.org>,
        Tejun Heo <tj@...nel.org>,
        "cgroups@...r.kernel.org" <cgroups@...r.kernel.org>,
        "Wu, Hao" <hao.wu@...el.com>, David Woodhouse <dwmw2@...radead.org>
Subject: RE: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation
 APIs

> From: Jason Gunthorpe <jgg@...dia.com>
> Sent: Wednesday, May 12, 2021 8:25 AM
> 
> On Wed, May 12, 2021 at 12:21:24AM +0000, Tian, Kevin wrote:
> 
> > > Basically each RID knows based on its kernel drivers if it is a local
> > > or global RID and the ioasid knob can further fine tune this for any
> > > other specialty cases.
> >
> > It's fine if you insist on this way. Then we leave it to userspace to
> > ensure same split range is used across devices when vIOMMU is
> > concerned.
> 
> I'm still confused why there is a split range needed.

a device could support both ENQCMD and non-ENQCMD submissions.

for ENQCMD path, CPU provides a PASID translation mechanism (from
guest PASID to host PASID)

for non-ENQCMD path, guest driver directly programs untranslated 
guest PASID to the device MMIO register.

the host kernel only setups host PASID entry which is hwid for a said
ioasid page table.

If we don't split range, we have to assume guest PASID == host PASID
otherwise non-ENQCMD path will fail. But expose host PASID to guest
breaks migration.

If we want to allow migration, then need support guest PASID != host
PASID and make sure both entries point to the same page table so
ENQCMD (host PASID) and non-ENQCMD (guest PASID) can both work.
It requires range split to avoid conflict between host/guest PASIDs
in the same space.

> 
> > Please note such range split has to be enforced through
> > vIOMMU which (e.g. on VT-d) includes a register to report available
> > PASID space size (applying to all devices behind this vIOMMU) to
> > the guest. The kernel just follows per-RID split info. If anything broken,
> > the userspace just shoots its own foot.
> 
> Is it because this specific vIOMMU protocol is limiting things?

When range split is enabled, we need a way to tell the guest about the
local range size so guest PASID is allocated only within this range. Then
we use vIOMMU to expose such information.

> 
> > > > > It does need some user visible difference because SIOV/mdev is not
> > > > > migratable. Only the kernel can select a PASID, userspace (and hence
> > > > > the guest) shouldn't have the option to force a specific PASID as the
> > > > > PASID space is shared across the entire RID to all VMs using the mdev.
> > > >
> > > > not migratable only when you choose exposing host-allocated PASID
> > > > into guest. However in the entire this proposal we actually virtualize
> > > > PASIDs, letting the guest manage its own PASID space in all
> > > > scenarios
> > >
> > > PASID cannot be virtualized without also using ENQCMD.
> > >
> > > A mdev that is using PASID without ENQCMD is non-migratable and this
> > > needs to be make visiable in the uAPI.
> >
> > No. without ENQCMD the PASID must be programmed to a mdev MMIO
> > register. This operation is mediated then mdev driver can translate the
> > PASID from virtual to real.
> 
> That is probably unworkable with real devices, but if you do this you
> need to explicitly expose the vPASID to the mdev API somehow, and still
> the device needs to declare if it supports this, and devices that
> don't should still work in a non-migratable mode.
> 

It's not necessary. For real devices we use alias mapping for both guest/host
PASID as explained above. Then we can have the guest to always register its 
vPASID with ioasid (just like map/unmap GPA to HVA), and then let host drivers 
to figure out whether that vPASID can be used as a real hwid. When it's considered
virtual and a real hwid is allocated by the host, the mapping is saved under this
ioasid to be queried by device drivers if translation is required. 
 
>From this angle, the previous IOASID_SET_HWID possibly should be renamed 
to IOASID_SET_USER_HWID.

Thanks
Kevin