linux-kernel - Re: [PATCH v13 02/25] PCI/CXL: Introduce pcie_is

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20251208183356.GA3416294@bhelgaas>
Date: Mon, 8 Dec 2025 12:33:56 -0600
From: Bjorn Helgaas <helgaas@...nel.org>
To: "Bowman, Terry" <terry.bowman@....com>
Cc: dave@...olabs.net, jonathan.cameron@...wei.com, dave.jiang@...el.com,
	alison.schofield@...el.com, dan.j.williams@...el.com,
	bhelgaas@...gle.com, shiju.jose@...wei.com, ming.li@...omail.com,
	Smita.KoralahalliChannabasappa@....com, rrichter@....com,
	dan.carpenter@...aro.org, PradeepVineshReddy.Kodamati@....com,
	lukas@...ner.de, Benjamin.Cheatham@....com,
	sathyanarayanan.kuppuswamy@...ux.intel.com,
	linux-cxl@...r.kernel.org, alucerop@....com, ira.weiny@...el.com,
	linux-kernel@...r.kernel.org, linux-pci@...r.kernel.org
Subject: Re: [PATCH v13 02/25] PCI/CXL: Introduce pcie_is_cxl()

On Mon, Dec 08, 2025 at 09:26:29AM -0600, Bowman, Terry wrote:
> On 12/5/2025 6:45 PM, Bjorn Helgaas wrote:
> > On Mon, Nov 03, 2025 at 06:09:38PM -0600, Terry Bowman wrote:
> >> CXL and AER drivers need the ability to identify CXL devices.
> >>
> >> Introduce set_pcie_cxl() with logic checking for CXL.mem or CXL.cache
> >> status in the CXL Flexbus DVSEC status register. The CXL Flexbus DVSEC
> >> presence is used because it is required for all the CXL PCIe devices.[1]

Spec usage seems to be "Flex Bus", which would help searches.

> >> Add boolean 'struct pci_dev::is_cxl' with the purpose to cache the CXL
> >> CXL.cache and CXl.mem status.
> >>
> >> In the case the device is an EP or USP, call set_pcie_cxl() on behalf of
> >> the parent downstream device. Once a device is created there is
> >> possibilty the parent training or CXL state was updated as well. This
> >> will make certain the correct parent CXL state is cached.
> ...

> >> +		pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_FLEXBUS_STATUS_OFFSET, &cap);
> >> +
> >> +		dev->is_cxl = FIELD_GET(PCI_DVSEC_CXL_FLEXBUS_STATUS_CACHE_MASK, cap) ||
> >> +			FIELD_GET(PCI_DVSEC_CXL_FLEXBUS_STATUS_MEM_MASK, cap);

> > Not sure the "_MASK" and "_OFFSET" on the end of all these #defines is
> > really needed.  Other items in pci_regs.h typically don't include
> > them, and these names get really long.  
> 
> These were moved over from local CXL header. As a result they are not 
> consistent in the usage of offset and mask. Would you like for this to 
> be made the same ? This would be to change all points where used.

I think it's worth dropping "_MASK" and "_OFFSET" to reduce the
length.

> >> +	 * Update parent's CXL state because alternate protocol training
> >> +	 * may have changed
> > 
> > What is the event that changes alternate protocol training?  The
> > commit log says "once a device is created", but I don't know what that
> > means in terms of hardware.
> 
> There is potential an upstream device (switch) was hotpluged and in
> the case of the alt training retries may not be correctly cached in
> pci_dev::is_cxl.

I assume this refers to the Alternate Protocol Negotiation (PCIe r7.0,
seg 4.2.5.2), which happens during link training.

IIUC, the problem here is that we may enumerate a bridge before its
secondary link has trained.  In that case, the Cache_Enabled and
Mem_Enabled bits in the bridge's Flex Bus Port Status may be zero.
After the secondary link has trained, those bits may be set based on
the alternate protocol negotiation, so we need to re-read the bridges
Port Status.  Annoying that these are documented as RO when they don't
really seem to be read-only.

Maybe there's no point in reading the bridge Flex Bus Port Status at
all until we enumerate a device *below* the bridge, e.g, something
like this:

  if (!pci_is_pcie(dev))
    return;

  if (!(pci_pcie_type(dev) == PCI_EXP_TYPE_ENDPOINT ||
	pci_pcie_type(dev) == PCI_EXP_TYPE_UPSTREAM))
    return;

  /* Parent's CXL status only valid when link to child has trained */
  bridge = pci_upstream_bridge(dev);
  dvsec = pci_find_dvsec_capability(bridge, PCI_VENDOR_ID_CXL,
                                    PCI_DVSEC_CXL_FLEXBUS_PORT);
  if (!dvsec)
    return;

  pci_read_config_word(bridge, dvsec + PCI_DVSEC_CXL_FLEXBUS_STATUS_OFFSET, &cap);
  ...