Message-ID: <381abcc3-d6c3-ba57-4e69-1e65083f1b6c@linux.intel.com>
Date: Thu, 8 Jan 2026 15:13:18 +0200 (EET)
From: Ilpo Järvinen <ilpo.jarvinen@...ux.intel.com>
To: Bjorn Helgaas <helgaas@...nel.org>
cc: Rob Herring <robh@...nel.org>, linux-pci@...r.kernel.org, 
    "David S. Miller" <davem@...emloft.net>, 
    Andreas Larsson <andreas@...sler.com>, Bjorn Helgaas <bhelgaas@...gle.com>, 
    sparclinux@...r.kernel.org, LKML <linux-kernel@...r.kernel.org>, 
    Nathaniel Roach <nroach44@...il.com>, 
    John Paul Adrian Glaubitz <glaubitz@...sik.fu-berlin.de>
Subject: Re: [PATCH RESEND 1/1] SPARC/PCI: Correct 64-bit non-pref -> pref
 BAR resources

On Wed, 7 Jan 2026, Bjorn Helgaas wrote:

> [+cc Adrian]
> 
> On Mon, Nov 24, 2025 at 07:04:11PM +0200, Ilpo Järvinen wrote:
> > SPARC T5-2 dts describes some PCI BARs as 64-bit resources without
> > pref(etchable) bit (0x83... vs 0xc3... in assigned-addresses) for
> > address ranges above the 4G threshold. Such resources cannot be placed
> > into a non-prefetchable PCI bridge window that is capable only to
> > 32-bit addressing. As such, it looks the platform is improperly
> > describe by dts.
> > 
> > Kernel detect this problem (see the IORESOURCE_PREFETCH check in
> > pci_find_parent_resource()) and fails to assign these BAR resources to
> > the resource tree due to lack of a compatible bridge window.
> > 
> > Prior to the commit 754babaaf333 ("sparc/PCI: Remove
> > pcibios_enable_device() as they do nothing extra") SPARC arch code did
> > not test whether device resources were successfully in the resource
> > tree when enabling a device, effectively hiding the problem. After
> > removing the arch specific enable code, PCI core's
> > pci_enable_resources() refuses to enable the device when it finds not
> > all mem resources are assigned, and therefore mpt3sas can't be enabled:
> > 
> > pci 0001:04:00.0: reg 0x14: [mem 0x801110000000-0x80111000ffff 64bit]
> > pci 0001:04:00.0: reg 0x1c: [mem 0x801110040000-0x80111007ffff 64bit]
> > pci 0001:04:00.0: BAR 1 [mem 0x801110000000-0x80111000ffff 64bit]: can't claim; no compatible bridge window
> > pci 0001:04:00.0: BAR 3 [mem 0x801110040000-0x80111007ffff 64bit]: can't claim; no compatible bridge window
> > mpt3sas 0001:04:00.0: BAR 1 [mem size 0x00010000 64bit]: not assigned; can't enable device
> > 
> > For clarity, this filtered log only shows failures for one mpt3sas
> > device but other devices fail similarly. In the reported case, the end
> > result with all the failures is an unbootable system.
> > 
> > Things appeared to "work" before the commit 754babaaf333 ("sparc/PCI:
> > Remove pcibios_enable_device() as they do nothing extra") because the
> > resource tree is agnostic to whether PCI BAR resources are properly in
> > the tree or not. So as long as there was a parent resource (e.g. a root
> > bus resource) that contains the address range, the resource tree code
> > just places resource request underneath it without any consideration to
> > the intermediate BAR resource. While it worked, it's incorrect setup
> > still.
> > 
> > Add of fixup to set IORESOURCE_PREFETCH flag for a 64-bit PCI resource
> > that has the end address above 4G requiring placement into the
> > prefetchable window. Also log the issue.
> > 
> > Fixes: 754babaaf333 ("sparc/PCI: Remove pcibios_enable_device() as they do nothing extra")
> > Reported-by: Nathaniel Roach <nroach44@...il.com>
> > Tested-by: Nathaniel Roach <nroach44@...il.com>
> > Closes: https://github.com/sparclinux/issues/issues/22
> > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@...ux.intel.com>
> 
> Originally posted as attachment in response to Adrian's pointer to
> Nathaniel's regression report:
> 
>   https://lore.kernel.org/r/d221be13-f652-cc75-90d1-92cf38e0110e@linux.intel.com
> 
> This seems reasonable to me and addresses a v6.18 regression, so I put
> this on pci/for-linus for v6.19.
> 
> > ---
> > 
> > Resending with linux-pci@ ML.
> > 
> > Any comments on the approach are welcome. E.g., is the fixup done at a
> > correct level? Should it be targeted specifically to the known failures
> > (how?) to avoid hiding more platform description problems?
> > 
> > It seems VF BARs still have 64-bit non-pref despite this change, AFAICT,
> > those are read directly from the device's config space so would require
> > ordinary quirks. None of them result in device enable failing though so the
> > issue is orthogonal to the one being fixed here.
> 
> The VF BARs that don't have PCI_BASE_ADDRESS_MEM_PREFETCH set is a
> generic question, not anything sparc-specific, right?

What I tried to say is that on this platform, we have such cases, but that 
being said they do not cause regression (and they cannot be "fixed" by the 
same means either).

None of these BARs (non-VF and VF ones) would not get assigned because of 
the IORESOURCE_PREFETCH check in pci_find_parent_resource() but 
post-754babaaf333 ("sparc/PCI: Remove pcibios_enable_device() as they do 
nothing extra"), enable device just doesn't like that. But because VF 
BARs are treated "optional", only non-VF BARs cause enable fail 
(unrelated to this issue, I got asked this week why VF resources are 
optional and realized I don't know the answer myself).

...

> As of the "Removing Prefetchable Terminology" ECN
> (https://members.pcisig.com/wg/PCI-SIG/document/20839) that was
> incorporated in PCIe r6.3, PCI_BASE_ADDRESS_MEM_PREFETCH (bit 3) is
> deprecated.  There's a recommendation that it be set for 64-bit BARs
> and cleared for 32-bit BARs, but I suppose we'll see 64-bit VF BARs
> with it cleared as in this case.

I'd missed this change (IIRC, I only read CBs only up to r6.2 and then 
r6.4 to r7.0).

...If we look VF BAR flags on general level, could we now actually set 
IORESOURCE_PREFETCH in decode_bar() if it's a 64-bit BAR on a PCIe 
device? That should make those VF BARs and other non-pref 64-bit "right" 
and it would be rather straightforward way to differentiate PCI vs PCIe 
use cases.

> We might need to rework the PCI_BASE_ADDRESS_MEM_PREFETCH handling to
> accommodate this.  I suspect we'll still need to pay attention to it
> at least for conventional PCI, but there might be some wiggle room for
> PCIe.

Abolish 64-bit non-pref goes to non-prefetchable window rule, or more like 
replace it with some rule that depends on conventional PCI being present 
or not, doesn't look that complicated change thanks to the addition of 
pbus_select_window_for_type() which consolidates these decision into a 
central place (but it would require finding out if there's conventional 
PCI involved which requires more work such as walking the device hierarchy 
or maintaining flag which tells that information). Only the assignment 
still run on its own fallback rules independent of 
pbus_select_window_for_type() and of course that 
pci_find_parent_resource() (this should probably be renamed to _for_addr 
or something along those lines to better align with 
pbus_select_window_for_type()).

Out of those two approaches, to keep conventional PCI compability, it 
would be probably more straightforward to just make any PCIe 64-bit BAR 
IORESOURCE_PREFETCH but that isn't as clearly connected to the intended 
logic decision which is "conventional PCI or not?" but I guess a comment 
in decode_bar() and perhaps in ioport.h would be good enough the explain 
it.

Related to this, the root bus resource flags are a mess, IORESOURCE_MEM_64 
is not always there even if addresses are clearly 64-bit and parent 
resource selection there should probably be agnostic to 
IORESOURCE_PREFETCH (which is currently not done as cleanly as it could be 
but just as artifacts the fallbacks have).

-- 
 i.

> > If suggesting a different approach, please do realize my knowledge
> > about OF code is generally very limited (and I'm not sure how directly
> > the fixup code in other archs, mainly ppc, can be used as an example
> > how to do fixups with sparc).
> > ---
> >  arch/sparc/kernel/pci.c | 23 +++++++++++++++++++++++
> >  1 file changed, 23 insertions(+)
> > 
> > diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c
> > index a9448088e762..b290107170e9 100644
> > --- a/arch/sparc/kernel/pci.c
> > +++ b/arch/sparc/kernel/pci.c
> > @@ -181,6 +181,28 @@ static int __init ofpci_debug(char *str)
> >  
> >  __setup("ofpci_debug=", ofpci_debug);
> >  
> > +static void of_fixup_pci_pref(struct pci_dev *dev, int index,
> > +			      struct resource *res)
> > +{
> > +	struct pci_bus_region region;
> > +
> > +	if (!(res->flags & IORESOURCE_MEM_64))
> > +		return;
> > +
> > +	if (!resource_size(res))
> > +		return;
> > +
> > +	pcibios_resource_to_bus(dev->bus, &region, res);
> > +	if (region.end <= ~((u32)0))
> > +		return;
> > +
> > +	if (!(res->flags & IORESOURCE_PREFETCH)) {
> > +		res->flags |= IORESOURCE_PREFETCH;
> > +		pci_info(dev, "reg 0x%x: fixup: pref added to 64-bit resource\n",
> > +			 index);
> > +	}
> > +}
> > +
> >  static unsigned long pci_parse_of_flags(u32 addr0)
> >  {
> >  	unsigned long flags = 0;
> > @@ -244,6 +266,7 @@ static void pci_parse_of_addrs(struct platform_device *op,
> >  		res->end = op_res->end;
> >  		res->flags = flags;
> >  		res->name = pci_name(dev);
> > +		of_fixup_pci_pref(dev, i, res);
> >  
> >  		pci_info(dev, "reg 0x%x: %pR\n", i, res);
> >  	}
> > 
> > base-commit: 3a8660878839faadb4f1a6dd72c3179c1df56787
> > -- 
> > 2.39.5
> > 
> 

Powered by blists - more mailing lists