| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <2296acdf-de5d-4591-a165-d59f9f432dee@amd.com>
Date: Thu, 10 Jul 2025 16:15:32 -0500
From: Nathan Fontenot <nathan.fontenot@....com>
To: "Zhijian Li (Fujitsu)" <lizhijian@...itsu.com>,
"Koralahalli Channabasappa, Smita" <Smita.KoralahalliChannabasappa@....com>,
"Bowman, Terry" <terry.bowman@....com>,
Alison Schofield <alison.schofield@...el.com>,
"linux-cxl@...r.kernel.org" <linux-cxl@...r.kernel.org>,
Jonathan Cameron <Jonathan.Cameron@...wei.com>,
"dan.j.williams@...el.com" <dan.j.williams@...el.com>,
"dave.jiang@...el.com" <dave.jiang@...el.com>,
"vishal.l.verma@...el.com" <vishal.l.verma@...el.com>,
Robert Richter <rrichter@....com>, Gregory Price <gourry@...rry.net>,
Fan Ni <fan.ni@...sung.com>, Davidlohr Bueso <dave@...olabs.net>
Cc: "Yasunori Gotou (Fujitsu)" <y-goto@...itsu.com>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [RFC] CXL/ACPI: Remove overlapping Soft Reserved regions before
adding CFMW resources
On 7/9/2025 9:30 PM, Zhijian Li (Fujitsu) wrote:
> Hi all,
>
> This RFC proposes a new approach to handle the resource conflict between CXL Fixed Memory Windows (CFMW) and "Soft Reserved" memory regions.
>
> I've been thinking about the "Soft Reserved" issue lately and went through the previous proposals [0]. The existing solutions are generally centered around CXL region management. For instance, an early proposal [1] suggested removing the "Soft Reserved" resource during region teardown. The current approach [2] has evolved to remove it after the CXL driver automatically constructs a CXL region.
>
> I haven't found prior discussions centered on removing the "Soft Reserved" region *before* the CFMW resource is even added. If this has been discussed before, please point me to the thread.
>
> My proposal is to remove the "Soft Reserved" resource at the moment we add the root decoder resource (the CFMW) from `CXL_ACPI`.
A couple of thoughts on this approach.
If the device_hmem driver loads prior to the cxl_acpi driver this could be an issue. The device
hmem driver init routine makes a copy of all soft reserve resources into hmem_active. It is
this hmem_active list that is used by the hmem driver to create dax devices, this could possibly
create a dax device for a soft reserve that no longer exists, or has been split up.
Perhaps moving the MODULE_SOFTDEP("pre: cxl_acpi") from dax/hmem/hmem.c to dax/hmem/device.c
could solve this to ensure the cxl_acpi code runs prior to building the hmem_active list.
One other aspect that this approach would prevent is the ability to use the memory covered by
a cxl region if there is an error in creating that region. In previous approaches for instance,
if during cxl region discovery we get an error and fail to create the region, the soft reserve
intersecting that failed region would still exist and be available for the dax driver to consume.
-Nathan
>
> Here’s why I believe this is feasible and more effective.
>
> #### Background
>
> Currently, CXL memory can be exposed to the users in several ways(1)(2)(3), depending on firmware and driver configurations. The diagram below illustrates the flow:
>
> ```
> +-----------------------+
> | |
> | CXL memory |
> | |
> +----------+------------+ +----------------------+
> | | |
> | | |
> +firmware---------------------+ | +---------v---------+ +--------------+
> | v | | | | NO | |
> | +----------+-------------+ | | | enable CXL_ACPI? +---------> HMEM |
> | | | | | | | | |
> | | expose to E820? | | | +-------------------+ +-------+------+
> | | | | | |YES |
> | +----------+-------------+ | | | |
> | | | | +---------v----------+ |
> | | YES | | | iomem | |
> | v | | | CXL WindowN | |
> +-(1)-----------+ | +-----------+-------------+ | | | (4) | |
> | iomem | NO | | set | | | +---------+----------+ |
> | System Ram +<--------+ EFI_MEMORY_SP attr? | | | | |
> | | | | | | | | |
> +---------------+ | +-------------------------+ | | +---------v----------+ +-------v--------+
> +-----------------------------+ | | | | |
> | YES | | CXL region +--------> (2) dax/kmem |
> v | | | | (3) device_dax |
> +-----------+-------------+ | +--------------------+ +----------------+
> | iomem | |
> | Soft Reserved +--------+
> | |
> +-------------------------+
> ```
>
> The problem we are facing occurs in path **(4)**, where the ACPI-defined `CXL WindowN` (CFMW) overlaps with a `Soft Reserved` region. In this scenario, if we try to destroy the driver-created CXL region to create a new one, the operation fails because the underlying memory range is still claimed by "Soft Reserved".
>
> Here is an example from ` /proc/iomem `:
>
> ```
> c070000000-fcffffffff : CXL Window 0
> c070000000-fcffffffff : Soft Reserved
> c070000000-fcffffffff : region0 ### Deleting this will not free the range for a new region
> c070000000-fcffffffff : dax0.0
> c070000000-fcffffffff : System RAM (kmem)
> ```
>
> #### Proposal
>
> The fundamental assumption of the CXL driver design appears to be (and I believe this is correct) that once the kernel successfully parses `CEDT.CFMWS` via `CXL_ACPI`, the CXL subsystem is designated to take full control of the corresponding CXL memory device.
>
> If this holds true, it means that the "Soft Reserved" region, which serves as a firmware-level hint to prevent the OS from arbitrarily using this memory, is no longer necessary after the CFMW is identified. The CFMW itself becomes the authoritative definition for this memory range.
>
> Therefore, we can safely remove the "Soft Reserved" resource from the `iomem_resource` tree right before inserting the new CFMW resource.
>
> This approach is simple and highly effective. It decouples the "Soft Reserved" problem from CXL region management entirely. By cleaning up the resource conflict at the earliest possible stage, this solution should be compatible with all CXL region patterns, including scenarios with misconfigured or unconfigured HDM Decoders.
>
> I haven't spent much time working out all the implementation details yet, but a quick proof-of-concept (PoC) shows that this approach appears to work. A rough sketch of the code is below. If this direction is ACKed, I will prepare and send out a complete implementation for review.
> ```diff
> --- a/drivers/cxl/acpi.c
> +++ b/drivers/cxl/acpi.c
> @@ -780,7 +783,9 @@ static int add_cxl_resources(struct resource *cxl_res)
> */
> cxl_set_public_resource(res, new);
>
> - insert_resource_expand_to_fit(&iomem_resource, new);
> + /* Remove Soft Reserved that is fully covered by this window */
> + claim_and_punch_out_soft_reserved(&iomem_resource, new);
>
> next = res->sibling;
> while (next && resource_overlaps(new, next)) {
> ```
>
> And the new helper function in `kernel/resource.c`:
>
> ```diff
> --- a/kernel/resource.c
> +++ b/kernel/resource.c
> @@ -364,6 +355,7 @@ int release_resource(struct resource *old)
>
> EXPORT_SYMBOL(release_resource);
>
> +/**
> + * claim_and_punch_out_soft_reserved - Claim a resource region, punching out
> + * any overlapping "Soft Reserved" areas.
> + * @root: The root of the resource tree (e.g., &iomem_resource).
> + * @new: The new resource to insert.
> + *
> + * Description:
> + * This function prepares for the insertion of a new resource (@new) by
> + * resolving conflicts with existing "Soft Reserved" regions. It works as
> + * follows:
> + *
> + * 1. It iterates through the children of @root, searching for any resource
> + * that both overlaps with @new and is identified as "Soft Reserved"
> + * (by its name and the IORES_DESC_SOFT_RESERVED descriptor).
> + * 2. For each conflicting "Soft Reserved" resource found, it removes the
> + * original conflicting resource from the tree.
> + * 3. It then calculates the remaining parts of the original resource that
> + * lie outside the range of @new. This may result in one or two smaller,
> + * non-overlapping parts.
> + * 4. These remaining parts are re-inserted into the resource tree as new,
> + * smaller "Soft Reserved" resources. This action is akin to "punching a
> + * hole" in the original reserved region.
> + * 5. After all conflicts are resolved, the @new resource is inserted into
> + * the tree, claiming the now-available space.
> + *
> + * Return: 0 on success, or a negative error code on failure.
> + */
> +int claim_and_punch_out_soft_reserved(struct resource *root,
> + struct resource *new);
> +
> ```
>
> Looking forward to your feedback and thoughts on this approach.
>
> Thanks,
> Zhijian
>
> -----
>
> **References:**
>
> [0]
> [PATCH 0/2] cxl/region: Improve Soft Reserved resource handling: [https://lore.kernel.org/linux-cxl/cover.1687568084.git.alison.schofield@intel.com/](https://lore.kernel.org/linux-cxl/cover.1687568084.git.alison.schofield@intel.com/)
> [PATCH v2 0/2] cxl/region: Improve Soft Reserved resource handling: [https://lore.kernel.org/linux-cxl/cover.1691176651.git.alison.schofield@intel.com/](https://lore.kernel.org/linux-cxl/cover.1691176651.git.alison.schofield@intel.com/)
> [PATCH v3 0/2] cxl/region: Improve Soft Reserved resource handling: [https://lore.kernel.org/linux-cxl/cover.1692638817.git.alison.schofield@intel.com/](https://lore.kernel.org/linux-cxl/cover.1692638817.git.alison.schofield@intel.com/)
> [RFC] cxl: Update Soft Reserved resources upon region creation: [https://lore.kernel.org/linux-cxl/20241004181754.8960-1-nathan.fontenot@amd.com/](https://lore.kernel.org/linux-cxl/20241004181754.8960-1-nathan.fontenot@amd.com/)
> [RFC v2] cxl: Update Soft Reserved resources upon region creation: [https://lore.kernel.org/linux-cxl/20241030172751.81392-1-nathan.fontenot@amd.com/](https://lore.kernel.org/linux-cxl/20241030172751.81392-1-nathan.fontenot@amd.com/)
> [PATCH] cxl: Update Soft Reserved resources upon region creation: [https://lore.kernel.org/linux-cxl/20241202155542.22111-1-nathan.fontenot@amd.com/](https://lore.kernel.org/linux-cxl/20241202155542.22111-1-nathan.fontenot@amd.com/)
> [PATCH v2 0/4] Add managed SOFT RESERVE resource handling: [https://lore.kernel.org/linux-cxl/cover.1737046620.git.nathan.fontenot@amd.com/](https://lore.kernel.org/linux-cxl/cover.1737046620.git.nathan.fontenot@amd.com/)
> [PATCH v3 0/4] Add managed SOFT RESERVE resource handling: [https://lore.kernel.org/linux-cxl/20250403183315.286710-1-terry.bowman@amd.com/](https://lore.kernel.org/linux-cxl/20250403183315.286710-1-terry.bowman@amd.com/)
> [PATCH v4 0/7] Add managed SOFT RESERVE resource handling: [https://lore.kernel.org/linux-cxl/20250603221949.53272-1-Smita.KoralahalliChannabasappa@amd.com/#r](https://lore.kernel.org/linux-cxl/20250603221949.53272-1-Smita.KoralahalliChannabasappa@amd.com/#r)
>
> [1] [https://lore.kernel.org/linux-cxl/cover.1691176651.git.alison.schofield@intel.com/](https://lore.kernel.org/linux-cxl/cover.1691176651.git.alison.schofield@intel.com/)
> [2] [https://lore.kernel.org/linux-cxl/20241004181754.8960-1-nathan.fontenot@amd.com/](https://lore.kernel.org/linux-cxl/20241004181754.8960-1-nathan.fontenot@amd.com/)
Powered by blists - more mailing lists