Message-ID: <3aq4qqx6ip2f567f6vjraojqcgvoo6t4impyhlbovb2zo5ptxq@5x3g5zdbhxgo>
Date: Wed, 9 Jul 2025 16:03:57 +0530
From: Manivannan Sadhasivam <mani@...nel.org>
To: Frank Li <Frank.li@....com>
Cc: Kishon Vijay Abraham I <kishon@...nel.org>, 
	"Rafael J. Wysocki" <rafael@...nel.org>, Thomas Gleixner <tglx@...utronix.de>, 
	Anup Patel <apatel@...tanamicro.com>, Marc Zyngier <maz@...nel.org>, 
	Greg Kroah-Hartman <gregkh@...uxfoundation.org>, Danilo Krummrich <dakr@...nel.org>, 
	Bjorn Helgaas <bhelgaas@...gle.com>, Arnd Bergmann <arnd@...db.de>, Shuah Khan <shuah@...nel.org>, 
	Richard Zhu <hongxing.zhu@....com>, Lucas Stach <l.stach@...gutronix.de>, 
	Lorenzo Pieralisi <lpieralisi@...nel.org>, Rob Herring <robh@...nel.org>, Shawn Guo <shawnguo@...nel.org>, 
	Sascha Hauer <s.hauer@...gutronix.de>, Pengutronix Kernel Team <kernel@...gutronix.de>, 
	Fabio Estevam <festevam@...il.com>, Krzysztof Kozlowski <krzk+dt@...nel.org>, 
	Conor Dooley <conor+dt@...nel.org>, Krzysztof Wilczyński <kwilczynski@...nel.org>, 
	Niklas Cassel <cassel@...nel.org>, dlemoal@...nel.org, jdmason@...zu.us, 
	linux-kernel@...r.kernel.org, linux-arm-kernel@...ts.infradead.org, linux-pci@...r.kernel.org, 
	linux-kselftest@...r.kernel.org, imx@...ts.linux.dev, devicetree@...r.kernel.org
Subject: Re: [PATCH v19 01/10] PCI: endpoint: Set ID and of_node for function
 driver

On Tue, Jul 08, 2025 at 03:06:02PM GMT, Frank Li wrote:
> On Tue, Jul 08, 2025 at 04:51:55PM +0530, Manivannan Sadhasivam wrote:
> > On Mon, Jul 07, 2025 at 12:41:22PM GMT, Frank Li wrote:
> > > On Wed, Jul 02, 2025 at 11:19:36AM -0400, Frank Li wrote:
> > > > On Wed, Jul 02, 2025 at 08:25:17PM +0530, Manivannan Sadhasivam wrote:
> > > > > On Wed, Jul 02, 2025 at 10:40:53AM GMT, Frank Li wrote:
> > > > > > On Wed, Jul 02, 2025 at 04:30:48PM +0530, Manivannan Sadhasivam wrote:
> > > > > > > On Mon, Jun 09, 2025 at 12:34:13PM GMT, Frank Li wrote:
> > > > > > > > Set device ID as 'vfunc_no << 3 | func_no' and use
> > > > > > > > 'device_set_of_node_from_dev()' to set 'of_node' the same as the EPC parent
> > > > > > > > device.
> > > > > > > >
> > > > > > > > Currently, EPF 'of_node' is NULL, but many functions depend on 'of_node'
> > > > > > > > settings, such as DMA, IOMMU, and MSI. At present, all DMA allocation
> > > > > > > > functions use the EPC's device node, but they should use the EPF one.
> > > > > > > > For multiple function drivers, IOMMU/MSI should be different for each
> > > > > > > > function driver.
> > > > > > > >
> > > > > > >
> > > > > > > We don't define OF node for any function, so device_set_of_node_from_dev() also
> > > > > > > ends up reusing the EPC node. So how can you make use of it in multi EPF setup?
> > > > > >
> > > > > > In mfd devices, children devices reuse parent's of_node
> > > > > > drivers/gpio/gpio-adp5585.c
> > > > > > drivers/input/keyboard/adp5589-keys.c
> > > > > > drivers/pwm/pwm-adp5585.c
> > > > > >
> > > > > > multi EPF should be similar to create multi children devices of mfd.
> > > > > >
> > > > >
> > > > > No, they are not similar. MFD are real physical devices, but EPFs are (so far)
> > > > > software based entities.
> > > > >
> > > > > > > I don't understand.
> > > > > >
> > > > > > >
> > > > > > > > If multiple function devices share the same EPC device, there will be
> > > > > > > > no isolation between them. Setting the ID and 'of_node' prepares for
> > > > > > > > proper support.
> > > > > >
> > > > > > Only share the same of_node.
> > > > > >
> > > > > > Actually pci host bridge have similar situation, all pci ep devices reuse
> > > > > > bridge's of node. framework use rid to distringuish it. EPF can use device::id
> > > > > > to do similar things.
> > > > > >
> > > > > > Actually iommu face the similar problem. So far, there are not EP device enable
> > > > > > iommu yet, because it needs special mapping.
> > > > > >
> > > > > > Prevously, I consider create dymatic of_node for each EPF and copy iommu/msi
> > > > > > information to each children. But when I see adp5585 case, I think direct
> > > > > > use parent's of_node should be simple and good enough.
> > > > > >
> > > > > > In future, I suggest add children dt binding for it. For example: EPF provide
> > > > > > a mailbox interface. how other dts node to refer to this mailbox's phandle?
> > > > > >
> > > > >
> > > > > As I said above, EPFs are not real devices. There is currently only one
> > > > > exception, MHI, which is backed by a hardware entity. So we cannot add
> > > > > devicetree nodes for EPF, unless each EPF is a hardware entity.
> > > >
> > > > But how resolve this problem, if a DT device need phandle to a EPF? anyway
> > > > this is off topic. let go back this doorbell.
> > > >
> > > > It needs an of_node for EPF device, I tried many method before.
> > > >
> > > > Create dymatic of_node for it? MSI framework still go through to parent
> > > > of_node to get such information. not big differnece as my view.
> > >
> > > Actually, DMA have simular issues, just 'workaround' it now.
> > >
> > > pci_epf_test_read() {
> > > 	...
> > > 	struct device *dma_dev = epf->epc->dev.parent;
> > > 	...
> > > 	dst_phys_addr = dma_map_single(dma_dev, buf, map_size,
> > >                                                        DMA_FROM_DEVICE);
> > > 					^^^ [1]
> > > 	...
> > > }
> > >
> > > [1] here direct use epc->dev.parent's of node implicy. If IOMMU enable,
> > > two EPF will share one IOMMU space without isolation. If add of_node(may
> > > dyamatic create one). we should resolve this problem by use epf device
> > > here. Difference EPF will use difference IOMMU space like MSI.
> > >
> >
> > Unless your platform comes up with a hardware based EPF, we are not going to
> > have DT node for any EPF. So all EPFs have to share the same DT node of the EPC.
> > So right now, it doesn't make a difference if you use a dynamic of_node or copy
> > the EPC node.
> >
> > Just reuse the EPC node for now.
> 
> It is show-stop issue. The closest version like
> https://lore.kernel.org/all/20241204-ep-msi-v10-2-87c378dbcd6d@nxp.com/
> 
> Or we just support one EPF. There are not good way to pass down epf ID to MSI
> controller.
> 
> [1]: Add DOMAIN_BUS_DEVICE_PCI_EP_MSI (like PCI RC bus),
> https://lore.kernel.org/all/20250211-ep-msi-v15-0-bcacc1f2b1a9@nxp.com/
> rejected by irq mantainer, they think it is too similar with platform msi.
> 
> The key problem is MSI controller need known both EPF's ID and EPC's MSI
> domain information.
> 
> If use EPC, there are no way to pass down EPF's ID. as above dma example,
> use EPC devices, dma_map_single() can't distringiush difference EPF. It is
> not big issue all EPF share a IO space. but can't do that for MSI. the
> different devices can't share the MSI space.
> 
> software managed dt property already used in many devices.
> 

You need not just a property, but a whole new DT node. This won't fly with the
DT maintainers, so it is not my call.

I'd suggest that we merge this initial implementation that supports only one EPF
for doorbell and tackle the multi-EPF implementation in future patches. I do not
want this series to be in a limbo for another 5-6 releases.

- Mani

-- 
மணிவண்ணன் சதாசிவம்

Powered by blists - more mailing lists