linux-kernel - Re: [PATCH V4 00/12] PCIe TPH and cache direct injection support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <04354274-d198-4162-badc-1078d1293517@amd.com>
Date: Tue, 3 Sep 2024 17:42:09 -0500
From: Wei Huang <wei.huang2@....com>
To: linux-pci@...r.kernel.org, linux-kernel@...r.kernel.org,
 linux-doc@...r.kernel.org, netdev@...r.kernel.org,
 Bjorn Helgaas <helgaas@...nel.org>
Cc: Jonathan.Cameron@...wei.com, corbet@....net, davem@...emloft.net,
 edumazet@...gle.com, kuba@...nel.org, pabeni@...hat.com,
 alex.williamson@...hat.com, gospo@...adcom.com, michael.chan@...adcom.com,
 ajit.khaparde@...adcom.com, somnath.kotur@...adcom.com,
 andrew.gospodarek@...adcom.com, manoj.panicker2@....com,
 Eric.VanTassell@....com, vadim.fedorenko@...ux.dev, horms@...nel.org,
 bagasdotme@...il.com, bhelgaas@...gle.com, lukas@...ner.de,
 paul.e.luse@...el.com, jing2.liu@...el.com
Subject: Re: [PATCH V4 00/12] PCIe TPH and cache direct injection support

Hi Bjorn,

I have incorporated Jakub's feedback on patch 11 (i.e. bnxt.c) in my new 
revision. Do you have additional comments or suggestions on V4 to 
include in the next spin?

Thanks,
-Wei

On 8/22/24 3:41 PM, Wei Huang wrote:
> Hi All,
> 
> TPH (TLP Processing Hints) is a PCIe feature that allows endpoint
> devices to provide optimization hints for requests that target memory
> space. These hints, in a format called steering tag (ST), are provided
> in the requester's TLP headers and allow the system hardware, including
> the Root Complex, to optimize the utilization of platform resources
> for the requests.
> 
> Upcoming AMD hardware implement a new Cache Injection feature that
> leverages TPH. Cache Injection allows PCIe endpoints to inject I/O
> Coherent DMA writes directly into an L2 within the CCX (core complex)
> closest to the CPU core that will consume it. This technology is aimed
> at applications requiring high performance and low latency, such as
> networking and storage applications.
> 
> This series introduces generic TPH support in Linux, allowing STs to be
> retrieved and used by PCIe endpoint drivers as needed. As a
> demonstration, it includes an example usage in the Broadcom BNXT driver.
> When running on Broadcom NICs with the appropriate firmware, it shows
> substantial memory bandwidth savings and better network bandwidth using
> real-world benchmarks. This solution is vendor-neutral and implemented
> based on industry standards (PCIe Spec and PCI FW Spec).
> 
> V3->V4:
>   * Rebase on top of the latest pci/next tree (tag: 6.11-rc1)
>   * Add new API functioins to query/enable/disable TPH support
>   * Make pcie_tph_set_st() completely independent from pcie_tph_get_cpu_st()
>   * Rewrite bnxt.c based on new APIs
>   * Remove documentation for now due to constantly changing API
>   * Remove pci=notph, but keep pci=nostmode with better flow (Bjorn)
>   * Lots of code rewrite in tph.c & pci-tph.h with cleaner interface (Bjorn)
>   * Add TPH save/restore support (Paul Luse and Lukas Wunner)
> 
> V2->V3:
>   * Rebase on top of pci/next tree (tag: pci-v6.11-changes)
>   * Redefine PCI TPH registers (pci_regs.h) without breaking uapi
>   * Fix commit subjects/messages for kernel options (Jonathan and Bjorn)
>   * Break API functions into three individual patches for easy review
>   * Rewrite lots of code in tph.c/tph.h based (Jonathan and Bjorn)
> 
> V1->V2:
>   * Rebase on top of pci.git/for-linus (6.10-rc1)
>   * Address mismatched data types reported by Sparse (Sparse check passed)
>   * Add pcie_tph_intr_vec_supported() for checking IRQ mode support
>   * Skip bnxt affinity notifier registration if
>     pcie_tph_intr_vec_supported()=false
>   * Minor fixes in bnxt driver (i.e. warning messages)
> 
> Manoj Panicker (1):
>    bnxt_en: Add TPH support in BNXT driver
> 
> Michael Chan (1):
>    bnxt_en: Pass NQ ID to the FW when allocating RX/RX AGG rings
> 
> Paul Luse (1):
>    PCI/TPH: Add save/restore support for TPH
> 
> Wei Huang (9):
>    PCI: Introduce PCIe TPH support framework
>    PCI: Add TPH related register definition
>    PCI/TPH: Add pcie_tph_modes() to query TPH modes
>    PCI/TPH: Add pcie_enable_tph() to enable TPH
>    PCI/TPH: Add pcie_disable_tph() to disable TPH
>    PCI/TPH: Add pcie_tph_enabled() to check TPH state
>    PCI/TPH: Add pcie_tph_set_st_entry() to set ST tag
>    PCI/TPH: Add pcie_tph_get_cpu_st() to get ST tag
>    PCI/TPH: Add pci=nostmode to force TPH No ST Mode
> 
>   .../admin-guide/kernel-parameters.txt         |   3 +
>   drivers/net/ethernet/broadcom/bnxt/bnxt.c     |  86 ++-
>   drivers/net/ethernet/broadcom/bnxt/bnxt.h     |   4 +
>   drivers/pci/pci.c                             |   4 +
>   drivers/pci/pci.h                             |  12 +
>   drivers/pci/pcie/Kconfig                      |  11 +
>   drivers/pci/pcie/Makefile                     |   1 +
>   drivers/pci/pcie/tph.c                        | 563 ++++++++++++++++++
>   drivers/pci/probe.c                           |   1 +
>   include/linux/pci-tph.h                       |  48 ++
>   include/linux/pci.h                           |   7 +
>   include/uapi/linux/pci_regs.h                 |  38 +-
>   12 files changed, 768 insertions(+), 10 deletions(-)
>   create mode 100644 drivers/pci/pcie/tph.c
>   create mode 100644 include/linux/pci-tph.h
>