| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20260112111707.794526-3-rrichter@amd.com>
Date: Mon, 12 Jan 2026 12:16:45 +0100
From: Robert Richter <rrichter@....com>
To: Alison Schofield <alison.schofield@...el.com>, Vishal Verma
<vishal.l.verma@...el.com>, Ira Weiny <ira.weiny@...el.com>, Dan Williams
<dan.j.williams@...el.com>, Jonathan Cameron <jonathan.cameron@...wei.com>,
Dave Jiang <dave.jiang@...el.com>, Davidlohr Bueso <dave@...olabs.net>,
Jonathan Corbet <corbet@....net>
CC: <linux-cxl@...r.kernel.org>, <linux-kernel@...r.kernel.org>, Gregory Price
<gourry@...rry.net>, "Fabio M. De Francesco"
<fabio.m.de.francesco@...ux.intel.com>, Terry Bowman <terry.bowman@....com>,
Joshua Hahn <joshua.hahnjy@...il.com>, Robert Richter <rrichter@....com>,
<linux-doc@...r.kernel.org>
Subject: [PATCH v3 3/3] Documentation/driver-api/cxl: ACPI PRM Address Translation Support and AMD Zen5 enablement
This adds a convention document for the following patch series:
cxl: ACPI PRM Address Translation Support and AMD Zen5 enablement
Version 7 and later:
https://lore.kernel.org/linux-cxl/20251114213931.30754-1-rrichter@amd.com/
Link: https://lore.kernel.org/linux-cxl/20251114213931.30754-1-rrichter@amd.com/
Reviewed-by: Gregory Price <gourry@...rry.net>
Signed-off-by: Robert Richter <rrichter@....com>
---
v3:
* updated link in descriptions and add Link: tag (Jonathan Cameron),
* clarified CFMWS and HPA range description (Jonathan Cameron),
* use 80 chars line limit (Jonathan Cameron),
* added block diagram to illustrate the example address mappings (Dave),
v2:
* updated sob-chain,
* spell fix in patch description (Randy),
* made small changes as suggested by Randy,
* Removed include:: <isonum.txt> line (Jon).
Signed-off-by: Robert Richter <rrichter@....com>
---
Documentation/driver-api/cxl/conventions.rst | 1 +
.../driver-api/cxl/conventions/cxl-atl.rst | 205 ++++++++++++++++++
2 files changed, 206 insertions(+)
create mode 100644 Documentation/driver-api/cxl/conventions/cxl-atl.rst
diff --git a/Documentation/driver-api/cxl/conventions.rst b/Documentation/driver-api/cxl/conventions.rst
index 9267a697b2fe..0d2e07279ad9 100644
--- a/Documentation/driver-api/cxl/conventions.rst
+++ b/Documentation/driver-api/cxl/conventions.rst
@@ -14,4 +14,5 @@ same convention.
:caption: Contents
conventions/cxl-lmh.rst
+ conventions/cxl-atl.rst
conventions/template.rst
diff --git a/Documentation/driver-api/cxl/conventions/cxl-atl.rst b/Documentation/driver-api/cxl/conventions/cxl-atl.rst
new file mode 100644
index 000000000000..69fed0044b52
--- /dev/null
+++ b/Documentation/driver-api/cxl/conventions/cxl-atl.rst
@@ -0,0 +1,205 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+ACPI PRM CXL Address Translation
+================================
+
+Document
+--------
+
+CXL Revision 3.2, Version 1.0
+
+License
+-------
+
+SPDX-License Identifier: CC-BY-4.0
+
+Creator/Contributors
+--------------------
+
+- Robert Richter, AMD
+
+Summary of the Change
+---------------------
+
+The CXL Fixed Memory Window Structures (CFMWS) describe zero or more Host
+Physical Address (HPA) windows associated with one or more CXL Host Bridges.
+Each HPA range of a CXL Host Bridge is represented by a CFMWS entry. An HPA
+range may include addresses currently assigned to CXL.mem devices, or an OS may
+assign ranges from an address window to a device.
+
+Host-managed Device Memory is Device-attached memory that is mapped to system
+coherent address space and accessible to the Host using standard write-back
+semantics. The managed address range is configured in the CXL HDM Decoder
+registers of the device. An HDM Decoder in a device is responsible for
+converting HPA into DPA by stripping off specific address bits.
+
+CXL devices and CXL bridges use the same HPA space. It is common across all
+components that belong to the same host domain. The view of the address region
+must be consistent on the CXL.mem path between the Host and the Device.
+
+This is described in the current CXL specification (Table 1-1, 3.3.1, 8.2.4.20,
+9.13.1, 9.18.1.3). [#cxl-spec-3.2]_
+
+Depending on the interconnect architecture of the platform, components attached
+to a host may not share the same host physical address space. Those platforms
+need address translation to convert an HPA between the host and the attached
+component, such as a CXL device. The translation mechanism is host-specific and
+implementation dependent.
+
+E.g., x86 AMD platforms use a Data Fabric that manages access to physical
+memory. Devices have their own memory space and can be configured to use
+'Normalized addresses' different to System Physical Addresses (SPA). Address
+translation is needed then. Details are described also under x86 AMD
+Documentation/admin-guide/RAS/address-translation.rst.
+
+Those AMD platforms provide PRM handlers in firmware to perform various types of
+address translation, including for CXL endpoints. AMD Zen5 systems implement the
+ACPI PRM CXL Address Translation firmware call. The ACPI PRM handler has a
+specific GUID to uniquely identify platforms with support of Normalized
+addressing. This is documented in the ACPI v6.5 Porting Guide, Address
+Translation - CXL DPA to System Physical Address. [#amd-ppr-58088]_
+
+When in Normalized address mode, HDM decoder address ranges must be configured
+and handled differently. Hardware addresses used in the HDM decoder
+configurations of an endpoint are not SPA and need to be translated from the
+endpoint's to its CXL host bridge's address range. This is especially important
+to find an endpoint's associated CXL Host Bridge and HPA window described in the
+CFMWS. Also, the interleave decoding is done by the Data Fabric and the endpoint
+does not perform decoding when converting HPA to DPA. Instead, interleaving is
+switched off for the endpoint (1 way). Finally, address translation might also
+be needed to inspect the Endpoint's hardware addresses, such as during
+profiling, tracing or error handling.
+
+For example, with Normalized addressing the HDM decoders could look as
+following::
+
+ -------------------------------
+ | Root Decoder (CFMWS) |
+ | SPA Range: 0x850000000 |
+ | Size: 0x8000000000 (512 GB) |
+ | Interleave Ways: 1 |
+ -------------------------------
+ |
+ v
+ -------------------------------
+ | Host Bridge Decoder (HDM) |
+ | SPA Range: 0x850000000 |
+ | Size: 0x8000000000 (512 GB) |
+ | Interleave Ways: 4 |
+ | Targets: endpoint5,8,11,13 |
+ | Granularity: 256 |
+ -------------------------------
+ |
+ -----------------------------+------------------------------
+ | | | |
+ v v v v
+ ------------------- ------------------- ------------------- -------------------
+ | endpoint5 | | endpoint8 | | endpoint11 | | endpoint13 |
+ | decoder5.0 | | decoder8.0 | | decoder11.0 | | decoder13.0 |
+ | PCIe: | | PCIe: | | PCIe: | | PCIe: |
+ | 0000:e2:00.0 | | 0000:e3:00.0 | | 0000:e4:00.0 | | 0000:e1:00.0 |
+ | DPA: | | DPA: | | DPA: | | DPA: |
+ | Start: 0x0 | | Start: 0x0 | | Start: 0x0 | | Start: 0x0 |
+ | Size: | | Size: | | Size: | | Size: |
+ | 0x2000000000 | | 0x2000000000 | | 0x2000000000 | | 0x2000000000 |
+ | (128 GB) | | (128 GB) | | (128 GB) | | (128 GB) |
+ | Interleaving: | | Interleaving: | | Interleaving: | | Interleaving: |
+ | Ways: 1 | | Ways: 1 | | Ways: 1 | | Ways: 1 |
+ | Gran: 256 | | Gran: 256 | | Gran: 256 | | Gran: 256 |
+ ------------------- ------------------- ------------------- -------------------
+ | | | |
+ v v v v
+ DPA DPA DPA DPA
+
+This shows the representation in sysfs:
+
+.. code-block:: none
+
+ /sys/bus/cxl/devices/endpoint5/decoder5.0/interleave_granularity:256
+ /sys/bus/cxl/devices/endpoint5/decoder5.0/interleave_ways:1
+ /sys/bus/cxl/devices/endpoint5/decoder5.0/size:0x2000000000
+ /sys/bus/cxl/devices/endpoint5/decoder5.0/start:0x0
+ /sys/bus/cxl/devices/endpoint8/decoder8.0/interleave_granularity:256
+ /sys/bus/cxl/devices/endpoint8/decoder8.0/interleave_ways:1
+ /sys/bus/cxl/devices/endpoint8/decoder8.0/size:0x2000000000
+ /sys/bus/cxl/devices/endpoint8/decoder8.0/start:0x0
+ /sys/bus/cxl/devices/endpoint11/decoder11.0/interleave_granularity:256
+ /sys/bus/cxl/devices/endpoint11/decoder11.0/interleave_ways:1
+ /sys/bus/cxl/devices/endpoint11/decoder11.0/size:0x2000000000
+ /sys/bus/cxl/devices/endpoint11/decoder11.0/start:0x0
+ /sys/bus/cxl/devices/endpoint13/decoder13.0/interleave_granularity:256
+ /sys/bus/cxl/devices/endpoint13/decoder13.0/interleave_ways:1
+ /sys/bus/cxl/devices/endpoint13/decoder13.0/size:0x2000000000
+ /sys/bus/cxl/devices/endpoint13/decoder13.0/start:0x0
+
+Note the endpoint interleaving configurations with a direct mapping (1-way).
+
+With PRM calls, the kernel can determine the following mappings:
+
+.. code-block:: none
+
+ cxl decoder5.0: address mapping found for 0000:e2:00.0 (hpa -> spa):
+ 0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
+ cxl decoder8.0: address mapping found for 0000:e3:00.0 (hpa -> spa):
+ 0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
+ cxl decoder11.0: address mapping found for 0000:e4:00.0 (hpa -> spa):
+ 0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
+ cxl decoder13.0: address mapping found for 0000:e1:00.0 (hpa -> spa):
+ 0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
+
+The corresponding CXL host bridge (HDM) decoders and root decoder (CFMWS) show
+and match with the calculated endpoint mappings:
+
+.. code-block:: none
+
+ /sys/bus/cxl/devices/port1/decoder1.0/interleave_granularity:256
+ /sys/bus/cxl/devices/port1/decoder1.0/interleave_ways:4
+ /sys/bus/cxl/devices/port1/decoder1.0/size:0x8000000000
+ /sys/bus/cxl/devices/port1/decoder1.0/start:0x850000000
+ /sys/bus/cxl/devices/port1/decoder1.0/target_list:0,1,2,3
+ /sys/bus/cxl/devices/port1/decoder1.0/target_type:expander
+ /sys/bus/cxl/devices/root0/decoder0.0/interleave_granularity:256
+ /sys/bus/cxl/devices/root0/decoder0.0/interleave_ways:1
+ /sys/bus/cxl/devices/root0/decoder0.0/size:0x8000000000
+ /sys/bus/cxl/devices/root0/decoder0.0/start:0x850000000
+ /sys/bus/cxl/devices/root0/decoder0.0/target_list:7
+
+The following changes of the specification are needed:
+
+* Allow a CXL device to be in a different HPA space other than the host's space.
+
+* The platform can use implementation-specific address translation when crossing
+ memory domains on the CXL.mem path between the Host and the Device.
+
+* The kernel (OSPM) determines Endpoint SPA range and interleaving configuration
+ using platform-specific address translation methods.
+
+Benefits of the Change
+----------------------
+
+Without the change, the OSPM may not determine the memory region and Root
+Decoder of an Endpoint and its corresponding HDM decoder. Region creation would
+fail. Platforms with a different interconnect architecture would fail to setup
+and use CXL.
+
+References
+----------
+
+.. [#cxl-spec-3.2] Compute Express Link Specification, Revision 3.2, Version 1.0,
+ https://www.computeexpresslink.org/
+
+.. [#amd-ppr-58088] AMD Family 1Ah Models 00h–0Fh and Models 10h–1Fh,
+ ACPI v6.5 Porting Guide, Publication # 58088,
+ https://www.amd.com/en/search/documentation/hub.html
+
+Detailed Description of the Change
+----------------------------------
+
+Add the following paragraph in 8.2.4.20 CXL HDM Decoder Capability Structure of
+the specification [#cxl-spec-3.2]_ to the end:
+
+"A device may use a different HPA space that is not common to other components
+of the host domain. The platform is responsible for address translation when
+crossing HPA spaces. The OSPM must determine the interleaving configuration and
+perform address translation to HPA ranges of the HDM decoders as needed. The
+translation mechanism is host-specific and implementation dependent."
--
2.47.3
Powered by blists - more mailing lists