netdev - Re: Q: Usage of pci_enable_atomic_ops_to

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <9c7c4217171fb56c505dc90b8c73b2ce079207a9.camel@linux.ibm.com>
Date: Mon, 03 Nov 2025 13:16:40 +0100
From: Gerd Bayer <gbayer@...ux.ibm.com>
To: Bjorn Helgaas <bhelgaas@...gle.com>, Jay Cornwall
 <Jay.Cornwall@....com>,
        Felix Kuehling <Felix.Kuehling@....com>
Cc: Niklas Schnelle <schnelle@...ux.ibm.com>,
        Alexander Schmidt	
 <alexs@...ux.ibm.com>,
        netdev <netdev@...r.kernel.org>,
        linux-rdma	
 <linux-rdma@...r.kernel.org>,
        linux-pci <linux-pci@...r.kernel.org>,
        Gerd
 Bayer	 <gbayer@...ux.ibm.com>
Subject: Re: Q: Usage of pci_enable_atomic_ops_to_root()

On Fri, 2025-10-24 at 19:18 +0200, Gerd Bayer wrote:
> Hi all,
> 
> I stumbled over mlx5's usage of pci_enable_atomic_ops_to_root() at
> 
> https://elixir.bootlin.com/linux/v6.18-rc2/source/drivers/net/ethernet/mellanox/mlx5/core/main.c#L937
> 
> and was wondering if its repeated calls with the 3 available sizes gave
> it the intended result.
> 
> I assume the intent was to enable requesting AtomicOps only if all
> three sizes 32/64/128-bit were supported at the root-complex. However,
> pci_enable_atomic_ops_to_root() would enable the request at the PCIe
> level, even if just 32-bit sized Ops was supported at the root-complex.

Looks like I might just send out an RFC patch for review by the
Mellanox/NVidia folks? Not sure if I can find a test-setup for this,
though...

> So I checked other users in the kernel and found an inconclusive
> picture:
> The AMD GPU that this was originally introduced for [0] checks for a
> combination of two sizes, while a few infiniband/ethernet and the vfio-
> pci driver do variations of sequential checks (potentially enabling
> requests that they don't want to)
> 
> Now the PCIe Spec Rev. 7.0 has also a mixed bag. Section 6.15.3.1
> mandates for Root Ports:
> 
> > If a Root Port implements any AtomicOp Completer capability for host
> > memory access, it must implement all 32-bit and 64-bit AtomicOp
> > Completer capabilities. Implementing 128-bit CAS Completer capability
> > is optional.
> 
> While this is specific, marking the CAS Op Completions in the 128-bit
> variant optional, the Capability bits just specify 128-bit AtomicOps
> (all AtomicOps: FetchAdd, Swap, CAS). Strictly interpreted, this would
> require root port implementors to announce all-or-nothing of 32/64/128-
> bit AtomicOps - which kind of makes the size-granularity of the
> capability bits useless - and leave the endpoint device (and its
> driver) attempting to use 128-bit CAS in the dark...

I guess I need to ask the folks at PCI SIG?

> [0]: https://lore.kernel.org/linux-pci/1515113100-4718-1-git-send-email-Felix.Kuehling@amd.com/
> 
> Can anybody shed some light on this?
> Thank you,
> Gerd