Message-ID: <BN9PR11MB52760F8EE835B4D06BAF0C248C91A@BN9PR11MB5276.namprd11.prod.outlook.com>
Date: Wed, 28 Jan 2026 00:57:59 +0000
From: "Tian, Kevin" <kevin.tian@...el.com>
To: Jason Gunthorpe <jgg@...dia.com>
CC: "Williams, Dan J" <dan.j.williams@...el.com>, Jonathan Cameron
	<jonathan.cameron@...wei.com>, Nicolin Chen <nicolinc@...dia.com>,
	"will@...nel.org" <will@...nel.org>, "robin.murphy@....com"
	<robin.murphy@....com>, "bhelgaas@...gle.com" <bhelgaas@...gle.com>,
	"joro@...tes.org" <joro@...tes.org>, "praan@...gle.com" <praan@...gle.com>,
	"baolu.lu@...ux.intel.com" <baolu.lu@...ux.intel.com>,
	"miko.lenczewski@....com" <miko.lenczewski@....com>,
	"linux-arm-kernel@...ts.infradead.org"
	<linux-arm-kernel@...ts.infradead.org>, "iommu@...ts.linux.dev"
	<iommu@...ts.linux.dev>, "linux-kernel@...r.kernel.org"
	<linux-kernel@...r.kernel.org>, "linux-pci@...r.kernel.org"
	<linux-pci@...r.kernel.org>, "linux-cxl@...r.kernel.org"
	<linux-cxl@...r.kernel.org>
Subject: RE: [PATCH RFCv1 1/3] PCI: Allow ATS to be always on for CXL.cache
 capable devices

> From: Jason Gunthorpe <jgg@...dia.com>
> Sent: Tuesday, January 27, 2026 11:05 PM
> 
> On Tue, Jan 27, 2026 at 08:10:06AM +0000, Tian, Kevin wrote:
> > > From: Williams, Dan J <dan.j.williams@...el.com>
> > > Sent: Friday, January 23, 2026 3:46 AM
> > >
> > > Jason Gunthorpe wrote:
> > > > On Wed, Jan 21, 2026 at 09:44:32PM -0800, dan.j.williams@...el.com
> > > wrote:
> > > > > I do not immediately see what is wrong with requiring userspace
> policy
> > > > > opt-in. That naturally gets replaced by installing the device's
> > > > > certificate (for native PCI CMA), authenticating the device with the
> > > > > TSM (for PCI IDE), or obviated by secure-ATS if that arrives.
> > > >
> > > > I think that goes back to the discussion about not loading drivers
> > > > before validating the device.
> > > >
> > > > It would also make alot of sense to leave the IOMMU blocking until the
> > > > driver is loaded for these secure situations. The blocking translation
> > > > should block ATS too.
> > > >
> > > > Then the flow you are describing will work well:
> > > >
> > > > 1) At pre-boot the IOMMU will block all DMA including Translated.
> > > > 2) The OS activates the IOMMU driver and keeps blocking.
> > > > 3) Instead of immediately binding a default domain the IOMMU core
> > > >    leaves the translation blocking.
> > > > 4) The OS defers loading the driver to userspace.
> > > > 5) Userspace measures the device and "accepts" it by loading the
> > > >    driver
> > > > 6) IOMMU core attaches a non-blocking default domain and activates
> ATS
> > >
> > > That works for me. Give the paranoid the ability to have a point where
> they
> > > can
> > > be assured that the shields were not lowered prematurely.
> >
> > Jason described the flow as "for these secure situations", i.e. not a general
> > requirement for cxl.cache, but iiuc Dan may instead want userspace policy
> > opt-in to be default (and with CMA/TSM etc. it gets easier)?
> 
> I think the general strategy has been to push userspace to do security
> decisions before binding drivers. So we have a plan for confidential
> compute VMs, and if there is interest then we can probably re-use that
> plan in all other cases.

make sense

> 
> > At a glance cxl.cache devices have gained ATS enabled automatically in
> > most cases (same as for all other ats-capable PCI devices):
> 
> Yes.
> 
> > - ARM: ATS is enabled automatically when attaching the default domain
> >   to the device in certain configurations, and this series tries to auto
> >   enable it in a missing configuration
> 
> Yes, ARM took the position that ATS should be left disabled for
> IDENTITY both because of SMMU constraints and also because it made
> some sense that you wouldn't want ATS overhead just to get a 1:1
> translation.
> 
> > - AMD: ATS is enabled at domain attach time
> 
> I'd argue this is an error and it should work like ARM
> 
> > - Intel: ATS is enabled when a device is probed by intel-iommu driver
> >   (incompatible with the suggested flow)
> 
> This is definately not a good choice :)
> 
> IMHO it is security required that the IOMMU driver block Translated
> requests while a BLOCKED domain is attached, and while the IOMMU is
> refusing ATS then device's ATS enable should be disabled.

It was made that way by commit 5518f239aff1 ("iommu/vt-d: Move
scalable mode ATS enablement to probe path "). The old policy was
same as AMD side, and changed to current way so domain change 
in RID won't affect the ATS requirement for PASIDs.

But I agree BLOCKED is special. Ideally there is no reason to have
RID attached to a BLOCKED domain while PASIDs are not
blocked... oh, wait

there is one scenario, e.g. VFIO allows domains attached to RID and
PASIDs being changed independently. It's a sane situation to have
userspace change RID domain via attach/detach/re-attach, while
PASID domains are still active. 'detach' will attach RID to a BLOCKED
domain, then disabling ATS in that window may break PASIDs.

How does ARM address this scenario? Is it more suitable to have
a new interface specific for driver bind/unbind to enable/disable
ATS instead of toggling it based on BLOCKED?

> 
> > Given above already shipped in distributions, probably we have to keep
> > them for compatibility (implying this series makes sense to fix a gap
> > in existing policy), then treat the suggested flow as an enhancement
> > for future?
> 
> I don't think we have a compatability issue here, just a security
> one.

'compatibility issue' if auto-enabling is completely removed with
userspace opt-in as the only way. But since it's not the case, then
yes it's more a security one.

> 
> Drivers need to ensure that ATS is disabled at PCI and Translated
> requestes blocked in IOMMU HW while a BLOCKED domain is attached.
> 
> Drivers can choose if they want to enable ATS for IDENTITY or not,
> (recommend not for performance and consistency).
> 
> Jason

Powered by blists - more mailing lists