linux-kernel - Re: [PATCH] PCI/PM: Ensure power-up succeeded before restoring MMIO state

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <aPptf2gLpoWL3Ics@google.com>
Date: Thu, 23 Oct 2025 11:01:35 -0700
From: Brian Norris <briannorris@...omium.org>
To: Bjorn Helgaas <helgaas@...nel.org>
Cc: Bjorn Helgaas <bhelgaas@...gle.com>, linux-kernel@...r.kernel.org,
	linux-pci@...r.kernel.org,
	Mario Limonciello <mario.limonciello@....com>,
	"Rafael J. Wysocki" <rafael@...nel.org>
Subject: Re: [PATCH] PCI/PM: Ensure power-up succeeded before restoring MMIO
 state

Hi Bjorn,

On Thu, Oct 23, 2025 at 12:25:47PM -0500, Bjorn Helgaas wrote:
> [+cc Mario, Rafael]
> 
> On Thu, Aug 21, 2025 at 07:58:12AM -0700, Brian Norris wrote:
> > From: Brian Norris <briannorris@...gle.com>
> > 
> > As the comments in pci_pm_thaw_noirq() suggest, pci_restore_state() may
> > need to restore MSI-X state in MMIO space. This is only possible if we
> > reach D0; if we failed to power up, this might produce a fatal error
> > when touching memory space.
> > 
> > Check for errors (as the "verify" in "pci_pm_power_up_and_verify_state"
> > implies), and skip restoring if it fails.
> > 
> > This mitigates errors seen during resume_noirq, for example, when the
> > platform did not resume the link properly.
> > 
> > Cc: stable@...r.kernel.org
> > Signed-off-by: Brian Norris <briannorris@...gle.com>
> > Signed-off-by: Brian Norris <briannorris@...omium.org>
> > ---
> > 
> >  drivers/pci/pci-driver.c | 12 +++++++++---
> >  drivers/pci/pci.c        | 13 +++++++++++--
> >  drivers/pci/pci.h        |  2 +-
> >  3 files changed, 21 insertions(+), 6 deletions(-)
> > 
> > diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
> > index 302d61783f6c..d66d95bd0ca2 100644
> > --- a/drivers/pci/pci-driver.c
> > +++ b/drivers/pci/pci-driver.c
> > @@ -557,7 +557,13 @@ static void pci_pm_default_resume(struct pci_dev *pci_dev)
> >  
> >  static void pci_pm_default_resume_early(struct pci_dev *pci_dev)
> >  {
> > -	pci_pm_power_up_and_verify_state(pci_dev);
> > +	/*
> > +	 * If we failed to reach D0, we'd better not touch MSI-X state in MMIO
> > +	 * space.
> > +	 */
> > +	if (pci_pm_power_up_and_verify_state(pci_dev))
> > +		return;
> 
> The MSI-X comment here seems oddly specific.

It's just as "oddly specific" as the existing comment in
pci_pm_thaw_noirq(), as mentioned in the commit message :)

The key point for MSI-X is that unlike the rest of pci_restore_state(),
it requires touching memory space. While config registers are OK to
touch in D3, memory space is not.

> On most platforms, config/mem/io accesses to a device not in D0 result
> in an error being logged, writes being dropped, and reads returning ~0
> data.

On my arm64 / pcie-designware-based platforms, that is mostly similar,
but there are some cases that are different. See below:

> I don't know the details, but I assume the fatal error is a problem
> specific to arm64.

Maybe. See my response here also:

  Re: [PATCH] PCI/sysfs: Ensure devices are powered for config reads
  https://lore.kernel.org/all/aNMoMY17CTR2_jQz@google.com/

In particular, when resuming the system in a case where the link was in
L2 and failed to resume properly, the PCIe controller may not be alive
enough even to emit completion timeouts. So it might hit case (a):

  "PCIe HW is not powered [...] and this tends to be SError, and a
  crash."

Memory space is unique, because while config accesses can be
intercepted/avoided by driver software, memory accesses cannot.

> If the device is not in D0, we can avoid the problem here, but it
> seems like we're just leaving a landmine for somebody else to hit
> later.  The driver will surely access the device after resume, won't
> it?

It's a possible landmine, yes. Although in my case, the link can go
through error recovery and restore itself later in the resume process.

> Is it better to wait for a fatal error there?
> 
> Even if we avoid errors here, aren't we effectively claiming to have
> restored the device state, which is now a lie?

I'm not sure we claim that. The device will stay in PCI_D3cold, and
pdev->state_saved will remain true.

But yes, it's a tricky situation to decide what to do next. My basic
assertion is that it's not OK to continue to restore state though.

Alternatives: pci_dev_set_disconnected()? pcie_do_recovery() /
pci_channel_io_frozen?

> Even on other platforms, if the writes that are supposed to restore
> the state are dropped because the device isn't in D0, the result is
> also not what we expect, and something is probably broken.

Sure. IMO, that's even more reason not to run pci_restore_state(),
because that will erroneously drop the state, and we'll have zero chance
of restoring it later.

Brian