Message-ID: <20250915123515.GE1024672@nvidia.com>
Date: Mon, 15 Sep 2025 09:35:15 -0300
From: Jason Gunthorpe <jgg@...dia.com>
To: "Tian, Kevin" <kevin.tian@...el.com>
Cc: Nicolin Chen <nicolinc@...dia.com>, "joro@...tes.org" <joro@...tes.org>,
	"bhelgaas@...gle.com" <bhelgaas@...gle.com>,
	"suravee.suthikulpanit@....com" <suravee.suthikulpanit@....com>,
	"will@...nel.org" <will@...nel.org>,
	"robin.murphy@....com" <robin.murphy@....com>,
	"sven@...nel.org" <sven@...nel.org>, "j@...nau.net" <j@...nau.net>,
	"alyssa@...enzweig.io" <alyssa@...enzweig.io>,
	"neal@...pa.dev" <neal@...pa.dev>,
	"robin.clark@....qualcomm.com" <robin.clark@....qualcomm.com>,
	"m.szyprowski@...sung.com" <m.szyprowski@...sung.com>,
	"krzk@...nel.org" <krzk@...nel.org>,
	"alim.akhtar@...sung.com" <alim.akhtar@...sung.com>,
	"dwmw2@...radead.org" <dwmw2@...radead.org>,
	"baolu.lu@...ux.intel.com" <baolu.lu@...ux.intel.com>,
	"yong.wu@...iatek.com" <yong.wu@...iatek.com>,
	"matthias.bgg@...il.com" <matthias.bgg@...il.com>,
	"angelogioacchino.delregno@...labora.com" <angelogioacchino.delregno@...labora.com>,
	"tjeznach@...osinc.com" <tjeznach@...osinc.com>,
	"paul.walmsley@...ive.com" <paul.walmsley@...ive.com>,
	"palmer@...belt.com" <palmer@...belt.com>,
	"aou@...s.berkeley.edu" <aou@...s.berkeley.edu>,
	"alex@...ti.fr" <alex@...ti.fr>,
	"heiko@...ech.de" <heiko@...ech.de>,
	"schnelle@...ux.ibm.com" <schnelle@...ux.ibm.com>,
	"mjrosato@...ux.ibm.com" <mjrosato@...ux.ibm.com>,
	"gerald.schaefer@...ux.ibm.com" <gerald.schaefer@...ux.ibm.com>,
	"orsonzhai@...il.com" <orsonzhai@...il.com>,
	"baolin.wang@...ux.alibaba.com" <baolin.wang@...ux.alibaba.com>,
	"zhang.lyra@...il.com" <zhang.lyra@...il.com>,
	"wens@...e.org" <wens@...e.org>,
	"jernej.skrabec@...il.com" <jernej.skrabec@...il.com>,
	"samuel@...lland.org" <samuel@...lland.org>,
	"jean-philippe@...aro.org" <jean-philippe@...aro.org>,
	"rafael@...nel.org" <rafael@...nel.org>,
	"lenb@...nel.org" <lenb@...nel.org>,
	"Liu, Yi L" <yi.l.liu@...el.com>,
	"cwabbott0@...il.com" <cwabbott0@...il.com>,
	"quic_pbrahma@...cinc.com" <quic_pbrahma@...cinc.com>,
	"iommu@...ts.linux.dev" <iommu@...ts.linux.dev>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"asahi@...ts.linux.dev" <asahi@...ts.linux.dev>,
	"linux-arm-kernel@...ts.infradead.org" <linux-arm-kernel@...ts.infradead.org>,
	"linux-arm-msm@...r.kernel.org" <linux-arm-msm@...r.kernel.org>,
	"linux-samsung-soc@...r.kernel.org" <linux-samsung-soc@...r.kernel.org>,
	"linux-mediatek@...ts.infradead.org" <linux-mediatek@...ts.infradead.org>,
	"linux-riscv@...ts.infradead.org" <linux-riscv@...ts.infradead.org>,
	"linux-rockchip@...ts.infradead.org" <linux-rockchip@...ts.infradead.org>,
	"linux-s390@...r.kernel.org" <linux-s390@...r.kernel.org>,
	"linux-sunxi@...ts.linux.dev" <linux-sunxi@...ts.linux.dev>,
	"linux-tegra@...r.kernel.org" <linux-tegra@...r.kernel.org>,
	"virtualization@...ts.linux.dev" <virtualization@...ts.linux.dev>,
	"linux-acpi@...r.kernel.org" <linux-acpi@...r.kernel.org>,
	"linux-pci@...r.kernel.org" <linux-pci@...r.kernel.org>,
	"patches@...ts.linux.dev" <patches@...ts.linux.dev>,
	"Sethi, Vikram" <vsethi@...dia.com>,
	"helgaas@...nel.org" <helgaas@...nel.org>,
	"etzhao1900@...il.com" <etzhao1900@...il.com>
Subject: Re: [PATCH v4 1/7] iommu/arm-smmu-v3: Add release_domain to attach
 prior to release_dev()

On Fri, Sep 12, 2025 at 09:33:06AM +0000, Tian, Kevin wrote:
> > From: Nicolin Chen <nicolinc@...dia.com>
> > Sent: Monday, September 1, 2025 7:32 AM
> > 
> > +static int arm_smmu_attach_dev_release(struct iommu_domain *domain,
> > +				       struct device *dev)
> > +{
> > +	struct arm_smmu_master *master = dev_iommu_priv_get(dev);
> > +
> > +	WARN_ON(master->iopf_refcount);

This doesn't look right anymore..

Now that iopf is managed automatically it technically doesn't go to
zero until the attaches below:

> > +
> > +	/* Put the STE back to what arm_smmu_init_strtab() sets */
> > +	if (dev->iommu->require_direct)
> > +
> > 	arm_smmu_attach_dev_identity(&arm_smmu_identity_domain,
> > dev);
> > +	else
> > +
> > 	arm_smmu_attach_dev_blocked(&arm_smmu_blocked_domain,
> > dev);

And I'd argue the attaches internally should have the assertion. If no
pasids and blocked/identity the iopf == 0.

> it's a bit confusing that a BLOCKED domain type could turn to the
> identity mode, though this movement doesn't change the original
> behavior.

That isn't what is happening here..

If dev->iommu->require_direct is set we prevent attaching BLOCKING
domains entirely:

	if (dev->iommu->require_direct &&
	    (new_domain->type == IOMMU_DOMAIN_BLOCKED ||
	     new_domain == group->blocking_domain)) {
		dev_warn(dev,
			 "Firmware has requested this device have a 1:1 IOMMU mapping, rejecting configuring the device without a 1:1 mapping. Contact your platform vendor.\n");
		return -EINVAL;
	}

So in most sane cases the above will never convert BLOCKING to
IDENTITY. What it is doing is preserving the RMRs...

Also, I don't think this should be in the smmu driver, every driver
should have this same logic, it is part of the definition of RMR
Let's put it in the core code:

	if (!dev->iommu->attach_deferred && ops->release_domain) {
		struct iommu_domain *release_domain = ops->release_domain;

		/*
		 * If the device requires direct mappings then it should not 
		 * be parked on a BLOCKED domain during release as that would
		 * break the direct mappings.
		 */
		if (dev->iommu->require_direct && ops->identity_domain &&
		    release_domain == ops->blocked_domain)
			release_domain = ops->identity_domain;

		release_domain->ops->attach_dev(release_domain, dev);
	}

Jason

Powered by blists - more mailing lists