linux-kernel - Re: [PATCH v3 1/2] PCI/AER: Fix missing uevent on recovery when a reset is requested

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <f496fc0f-64d7-46a4-8562-dba74e31a956@linux.intel.com>
Date: Fri, 1 Aug 2025 10:20:43 -0700
From: Sathyanarayanan Kuppuswamy <sathyanarayanan.kuppuswamy@...ux.intel.com>
To: Lukas Wunner <lukas@...ner.de>
Cc: Niklas Schnelle <schnelle@...ux.ibm.com>,
 Bjorn Helgaas <bhelgaas@...gle.com>,
 Mahesh J Salgaonkar <mahesh@...ux.ibm.com>,
 Linas Vepstas <linasvepstas@...il.com>,
 Ilpo Järvinen <ilpo.jarvinen@...ux.intel.com>,
 Manivannan Sadhasivam <mani@...nel.org>,
 Gerald Schaefer <gerald.schaefer@...ux.ibm.com>,
 Heiko Carstens <hca@...ux.ibm.com>, Vasily Gorbik <gor@...ux.ibm.com>,
 Alexander Gordeev <agordeev@...ux.ibm.com>,
 Christian Borntraeger <borntraeger@...ux.ibm.com>,
 Sven Schnelle <svens@...ux.ibm.com>,
 Peter Oberparleiter <oberpar@...ux.ibm.com>,
 Matthew Rosato <mjrosato@...ux.ibm.com>, Oliver O'Halloran
 <oohall@...il.com>, Sinan Kaya <okaya@...nel.org>,
 linuxppc-dev@...ts.ozlabs.org, linux-s390@...r.kernel.org,
 linux-kernel@...r.kernel.org, linux-pci@...r.kernel.org,
 Keith Busch <kbusch@...nel.org>
Subject: Re: [PATCH v3 1/2] PCI/AER: Fix missing uevent on recovery when a
 reset is requested

Hi Lukas,

On 7/31/25 10:44 PM, Lukas Wunner wrote:
> On Thu, Jul 31, 2025 at 10:04:38AM -0700, Sathyanarayanan Kuppuswamy wrote:
>> On 7/31/25 6:01 AM, Lukas Wunner wrote:
>>> +++ b/drivers/pci/pcie/err.c
>>> @@ -165,6 +165,12 @@ static int report_resume(struct pci_dev *dev, void *data)
>>>    	return 0;
>>>    }
>>> +static int report_disconnect(struct pci_dev *dev, void *data)
>>> +{
>>> +	pci_uevent_ers(dev, PCI_ERS_RESULT_DISCONNECT);
>>> +	return 0;
>>> +}
>> Since you are notifying the user space, I am wondering whether the drivers
>> should be notified about the recovery failure?
> The drivers are usually *causing* the recovery failure by returning
> PCI_ERS_RESULT_DISCONNECT from their pci_error_handlers callbacks
> (or by lacking pci_error_handlers, in particular ->error_detected()).
>
> So in principle the drivers should be aware of recovery failure.
>
> There are cases where multiple drivers are involved.  E.g. on GPUs,
> there's often a PCIe switch with a graphics device and various sound
> or telemetry devices.  Typically errors are reported by the Upstream
> Port, so the Secondary Bus Reset occurs at the Root or Downstream Port
> above the Upstream Port and affects the switch and all subordinate
> devices.  In cases like this, recovery failure may be caused by a
> single driver (e.g. GPU) and the other drivers (e.g. telemetry) may
> be unaware of it.

Yes, my comment was referring to the scenario mentioned above. If one of the
subordinate devices fails recovery, then recovery effectively fails for all devices
under that downstream port (or root port). Notifying all devices under that port
would allow their drivers to perform the necessary cleanup

>
> The recovery flow documented in Documentation/PCI/pci-error-recovery.rst
> was originally conceived for EEH and indeed EEH does notify all drivers
> of recovery failures by invoking the ->error_detected() callback with
> channel_state pci_channel_io_perm_failure.  See this call ...
>
> 	eeh_pe_report("error_detected(permanent failure)", pe,
> 		      eeh_report_failure, NULL);
>
> ... in arch/powerpc/kernel/eeh_driver.c below the recover_failed label
> in eeh_handle_normal_event().

Agree. The current implementation does not seem to follow the steps
mentioned in the Documentation/PCI/pci-error-recovery.rst.

STEP 6: Permanent Failure
-------------------------
A "permanent failure" has occurred, and the platform cannot recover
the device.  The platform will call error_detected() with a
pci_channel_state_t value of pci_channel_io_perm_failure.

The device driver should, at this point, assume the worst. It should
cancel all pending I/O, refuse all new I/O, returning -EIO to
higher layers. The device driver should then clean up all of its
memory and remove itself from kernel operations, much as it would
during system shutdown.


>
> I don't know why pcie_do_recovery() doesn't do the same on recovery
> failure.  This is one of several annoying deviations between AER and
> EEH.  Ideally the behavior should be the same across all platforms
> so that drivers don't have to cope with platform-specific quirks.
>
> However I think that's orthogonal to the pci_uevent_ers() invocation
> in pcie_do_recovery().

Agree. My thought is, since there is an attempt to fix the user notification
side of things, may be the driver side should also be fixed together .

> Thanks,
>
> Lukas

-- 
Sathyanarayanan Kuppuswamy
Linux Kernel Developer