linux-kernel - Re: [PATCH] PCI/DOE: Poll DOE Busy bit for up to 1 second in pci_doe_send

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20241013155834.GA607803@bhelgaas>
Date: Sun, 13 Oct 2024 10:58:34 -0500
From: Bjorn Helgaas <helgaas@...nel.org>
To: Gregory Price <gourry@...rry.net>
Cc: linux-pci@...r.kernel.org, linux-cxl@...r.kernel.org,
	linux-kernel@...r.kernel.org, lukas@...ner.de,
	dan.j.williams@...el.com, bhelgaas@...gle.com, dave@...olabs.net,
	dave.jiang@...el.com, vishal.l.verma@...el.com,
	Jonathan.Cameron@...wei.com
Subject: Re: [PATCH] PCI/DOE: Poll DOE Busy bit for up to 1 second in
 pci_doe_send_req

On Fri, Oct 11, 2024 at 09:59:12AM -0400, Gregory Price wrote:
> On Thu, Oct 10, 2024 at 05:16:28PM -0500, Bjorn Helgaas wrote:
> > On Fri, Oct 04, 2024 at 12:28:28PM -0400, Gregory Price wrote:
> > > During initial device probe, the PCI DOE busy bit for some CXL
> > > devices may be left set for a longer period than expected by the
> > > current driver logic. Despite local comments stating DOE Busy is
> > > unlikely to be detected, it appears commonly specifically during
> > > boot when CXL devices are being probed.
> > > 
> > > This was observed on a single socket AMD platform with 2 CXL memory
> > > expanders attached to the single socket. It was not the case that
> > > concurrent accesses were being made, as validated by monitoring
> > > mailbox commands on the device side.
> > > 
> > > This behavior has been observed with multiple CXL memory expanders
> > > from different vendors - so it appears unrelated to the model.
> > > 
> > > In all observed tests, only a small period of the retry window is
> > > actually used - typically only a handful of loop iterations.
> > > 
> > > Polling on the PCI DOE Busy Bit for (at max) one PCI DOE timeout
> > > interval (1 second), resolves this issues cleanly.
> > > 
> > > Per PCIe r6.2 sec 6.30.3, the DOE Busy Bit being cleared does not
> > > raise an interrupt, so polling is the best option in this scenario.
> > > 
> > > Subsqeuent code in doe_statemachine_work and abort paths also wait
> > > for up to 1 PCI DOE timeout interval, so this order of (potential)
> > > additional delay is presumed acceptable.
> > 
> > I provisionally applied this to pci/doe for v6.13 with Lukas and
> > Jonathan's reviewed-by.  
> > 
> > Can we include a sample of any dmesg logging or other errors users
> > would see because of this problem?  I'll update the commit log with
> > any of this information to help users connect an issue with this fix.
> >
> 
> The only indication in dmesg you will see is a line like
> 
> [   24.542625] endpoint6: DOE failed -EBUSY
> 
> produced by cxl_cdat_get_length or cxl_cdat_read_table
> 
> 
> Do you want an updated patch with the nits fixed?

No need, I fixed the nits and added the dmesg line to the commit log.
Thank you!

Bjorn