linux-kernel - Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20240603171921.GA685838@bhelgaas>
Date: Mon, 3 Jun 2024 12:19:21 -0500
From: Bjorn Helgaas <helgaas@...nel.org>
To: Robin Murphy <robin.murphy@....com>
Cc: Frank Li <Frank.Li@....com>, Richard Zhu <hongxing.zhu@....com>,
	Lucas Stach <l.stach@...gutronix.de>,
	Lorenzo Pieralisi <lpieralisi@...nel.org>,
	Krzysztof Wilczyński <kw@...ux.com>,
	Rob Herring <robh@...nel.org>, Bjorn Helgaas <bhelgaas@...gle.com>,
	Shawn Guo <shawnguo@...nel.org>,
	Sascha Hauer <s.hauer@...gutronix.de>,
	Pengutronix Kernel Team <kernel@...gutronix.de>,
	Fabio Estevam <festevam@...il.com>,
	NXP Linux Team <linux-imx@....com>,
	Philipp Zabel <p.zabel@...gutronix.de>,
	Liam Girdwood <lgirdwood@...il.com>,
	Mark Brown <broonie@...nel.org>,
	Manivannan Sadhasivam <manivannan.sadhasivam@...aro.org>,
	Krzysztof Kozlowski <krzysztof.kozlowski+dt@...aro.org>,
	Conor Dooley <conor+dt@...nel.org>, linux-pci@...r.kernel.org,
	imx@...ts.linux.dev, linux-arm-kernel@...ts.infradead.org,
	linux-kernel@...r.kernel.org, bpf@...r.kernel.org,
	devicetree@...r.kernel.org, Will Deacon <will@...nel.org>,
	Joerg Roedel <joro@...tes.org>, Jason Gunthorpe <jgg@...pe.ca>,
	Alyssa Rosenzweig <alyssa@...enzweig.io>,
	Marc Zyngier <maz@...nel.org>
Subject: Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support
 MSI ITS and IOMMU for i.MX95

On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
> On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > [+cc IOMMU and pcie-apple.c folks for comment]
> > 
> > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > This involves examining the msi-map and smmu-map to ensure consistent
> > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > controller is utilized as a fallback.
> > > 
> > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > controller. This function configures the correct LUT based on Device Tree
> > > Settings (DTS).
> > 
> > This scheme is pretty similar to apple_pcie_bus_notifier().  If we
> > have to do this, I wish it were *more* similar, i.e., copy the
> > function names, bitmap tracking, code structure, etc.
> > 
> > I don't really know how stream IDs work, but I assume they are used on
> > most or all arm64 platforms, so I'm a little surprised that of all the
> > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > this notifier.
> 
> This is one of those things that's mostly at the mercy of the PCIe root
> complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
> is derived directly from the PCI RID, sometimes with additional high-order
> bits hard-wired to disambiguate PCI segments. I believe this RID-translation
> LUT is a particular feature of the the Synopsys IP - I know there's also one
> on the NXP Layerscape platforms, but on those it's programmed by the
> bootloader, which also generates the appropriate "msi-map" and "iommu-map"
> properties to match. Ideally that's what i.MX should do as well, but hey.

Maybe this RID-translation is a feature of i.MX, not of Synopsys?  I
see that the LUT CSR accesses use IMX95_* definitions.

> If it's really necessary to do this programming from Linux, then there's
> still no point in it being dynamic - the mappings cannot ever change, since
> the rest of the kernel believes that what the DT said at boot time was
> already a property of the hardware. It would be a lot more logical, and
> likely simpler, for the driver to just read the relevant map property and
> program the entire LUT to match, all in one go at controller probe time.
> Rather like what's already commonly done with the parsing of "dma-ranges" to
> program address-translation LUTs for inbound windows.
> 
> Plus that would also give a chance of safely dealing with bad DTs specifying
> invalid ID mappings (by refusing to probe at all). As it is, returning an
> error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
> further notifiers from running at that point - the device will still be
> added, allowed to bind a driver, and able to start sending DMA/MSI traffic
> without the controller being correctly programmed, which at best won't work
> and at worst may break the whole system.

Frank, could the imx LUT be programmed once at boot-time instead of at
device-add time?  I'm guessing maybe not because apparently there is a
risk of running out of LUT entries?

It sounds like the consequences of running out of LUT entries are
catastrophic, e.g., memory corruption from mis-directed DMA?  If
that's possible, I think we need to figure out how to prevent the
device from being used, not just dev_warn() about it.

Bjorn