| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <cb4ebb27-62fb-48fc-87b8-d2c55b3c4b6d@fujitsu.com>
Date: Fri, 11 Jul 2025 01:30:05 +0000
From: "Zhijian Li (Fujitsu)" <lizhijian@...itsu.com>
To: "Koralahalli Channabasappa, Smita"
<Smita.KoralahalliChannabasappa@....com>, "Bowman, Terry"
<terry.bowman@....com>, Alison Schofield <alison.schofield@...el.com>,
"linux-cxl@...r.kernel.org" <linux-cxl@...r.kernel.org>, Jonathan Cameron
<Jonathan.Cameron@...wei.com>, "dan.j.williams@...el.com"
<dan.j.williams@...el.com>, "dave.jiang@...el.com" <dave.jiang@...el.com>,
"vishal.l.verma@...el.com" <vishal.l.verma@...el.com>, Robert Richter
<rrichter@....com>, "Fontenot, Nathan" <Nathan.Fontenot@....com>, Gregory
Price <gourry@...rry.net>, Fan Ni <fan.ni@...sung.com>, Davidlohr Bueso
<dave@...olabs.net>
CC: "Yasunori Gotou (Fujitsu)" <y-goto@...itsu.com>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [RFC] CXL/ACPI: Remove overlapping Soft Reserved regions before
adding CFMW resources
On 11/07/2025 04:39, Koralahalli Channabasappa, Smita wrote:
> Hi Zhijian
>
> On 7/9/2025 7:30 PM, Zhijian Li (Fujitsu) wrote:
>> Hi all,
>>
>> This RFC proposes a new approach to handle the resource conflict between CXL Fixed Memory Windows (CFMW) and "Soft Reserved" memory regions.
>>
>> I've been thinking about the "Soft Reserved" issue lately and went through the previous proposals [0]. The existing solutions are generally centered around CXL region management. For instance, an early proposal [1] suggested removing the "Soft Reserved" resource during region teardown. The current approach [2] has evolved to remove it after the CXL driver automatically constructs a CXL region.
>>
>> I haven't found prior discussions centered on removing the "Soft Reserved" region *before* the CFMW resource is even added. If this has been discussed before, please point me to the thread.
>>
>> My proposal is to remove the "Soft Reserved" resource at the moment we add the root decoder resource (the CFMW) from `CXL_ACPI`.
>>
>> Here’s why I believe this is feasible and more effective.
>>
>> #### Background
>>
>> Currently, CXL memory can be exposed to the users in several ways(1)(2)(3), depending on firmware and driver configurations. The diagram below illustrates the flow:
>>
>> ```
>> +-----------------------+
>> | |
>> | CXL memory |
>> | |
>> +----------+------------+ +----------------------+
>> | | |
>> | | |
>> +firmware---------------------+ | +---------v---------+ +--------------+
>> | v | | | | NO | |
>> | +----------+-------------+ | | | enable CXL_ACPI? +---------> HMEM |
>> | | | | | | | | |
>> | | expose to E820? | | | +-------------------+ +-------+------+
>> | | | | | |YES |
>> | +----------+-------------+ | | | |
>> | | | | +---------v----------+ |
>> | | YES | | | iomem | |
>> | v | | | CXL WindowN | |
>> +-(1)-----------+ | +-----------+-------------+ | | | (4) | |
>> | iomem | NO | | set | | | +---------+----------+ |
>> | System Ram +<--------+ EFI_MEMORY_SP attr? | | | | |
>> | | | | | | | | |
>> +---------------+ | +-------------------------+ | | +---------v----------+ +-------v--------+
>> +-----------------------------+ | | | | |
>> | YES | | CXL region +--------> (2) dax/kmem |
>> v | | | | (3) device_dax |
>> +-----------+-------------+ | +--------------------+ +----------------+
>> | iomem | |
>> | Soft Reserved +--------+
>> | |
>> +-------------------------+
>> ```
>>
>> The problem we are facing occurs in path **(4)**, where the ACPI-defined `CXL WindowN` (CFMW) overlaps with a `Soft Reserved` region. In this scenario, if we try to destroy the driver-created CXL region to create a new one, the operation fails because the underlying memory range is still claimed by "Soft Reserved".
>>
>> Here is an example from ` /proc/iomem `:
>>
>> ```
>> c070000000-fcffffffff : CXL Window 0
>> c070000000-fcffffffff : Soft Reserved
>> c070000000-fcffffffff : region0 ### Deleting this will not free the range for a new region
>> c070000000-fcffffffff : dax0.0
>> c070000000-fcffffffff : System RAM (kmem)
>> ```
>>
>> #### Proposal
>>
>> The fundamental assumption of the CXL driver design appears to be (and I believe this is correct) that once the kernel successfully parses `CEDT.CFMWS` via `CXL_ACPI`, the CXL subsystem is designated to take full control of the corresponding CXL memory device.
>>
>> If this holds true, it means that the "Soft Reserved" region, which serves as a firmware-level hint to prevent the OS from arbitrarily using this memory, is no longer necessary after the CFMW is identified. The CFMW itself becomes the authoritative definition for this memory range.
>
> I may be wrong, but shouldn't we also guarantee that cxl_region instances be always populated after the CXL Windows (CFMWS) are setup through cxl_acpi?
However, my point was that once the CFMWS is correctly parsed (and inserted into iomem_resource), the CXL memory should be handed over to the CXL subsystem.
If the CXL subsystem encounters obstacles in setting up the cxl region, the following actions should be taken either:
- Fix the firmware
- Allow the CXL subsystem to reprogram misconfigured HDM decoders
- Disable the cxl_acpi driver to fall back to the hmem path.
To my understanding, this aligns with the behavior of the current upstream kernel.
Thanks
Zhijian
>
> Dan's comments here:
> https://lore.kernel.org/all/65e0fcae989d6_1138c7294c2@dwillia2-xfh.jf.intel.com.notmuch/
>
> "After cxl_acpi is loaded Linux knows where all the CXL windows are, but
> it does not know if there are cxl_region instances populated into those
> windows. That process involves waiting for PCI probe to complete and the
> CXL autoassembly process to run its course."
>
> This patchset https://lore.kernel.org/all/20250603221949.53272-1-Smita.KoralahalliChannabasappa@amd.com/ does wait for cxl_mem (cxl_pci) to complete but I agree it doesn't guarantee region readiness. I was thinking to wait on cxl_region_probe to complete along with cxl_mem in v5.
>
> Thanks
> Smita
>>
>> Therefore, we can safely remove the "Soft Reserved" resource from the `iomem_resource` tree right before inserting the new CFMW resource.
>>
>> This approach is simple and highly effective. It decouples the "Soft Reserved" problem from CXL region management entirely. By cleaning up the resource conflict at the earliest possible stage, this solution should be compatible with all CXL region patterns, including scenarios with misconfigured or unconfigured HDM Decoders.
>>
>> I haven't spent much time working out all the implementation details yet, but a quick proof-of-concept (PoC) shows that this approach appears to work. A rough sketch of the code is below. If this direction is ACKed, I will prepare and send out a complete implementation for review.
>> ```diff
>> --- a/drivers/cxl/acpi.c
>> +++ b/drivers/cxl/acpi.c
>> @@ -780,7 +783,9 @@ static int add_cxl_resources(struct resource *cxl_res)
>> */
>> cxl_set_public_resource(res, new);
>> - insert_resource_expand_to_fit(&iomem_resource, new);
>> + /* Remove Soft Reserved that is fully covered by this window */
>> + claim_and_punch_out_soft_reserved(&iomem_resource, new);
>> next = res->sibling;
>> while (next && resource_overlaps(new, next)) {
>> ```
>>
>> And the new helper function in `kernel/resource.c`:
>>
>> ```diff
>> --- a/kernel/resource.c
>> +++ b/kernel/resource.c
>> @@ -364,6 +355,7 @@ int release_resource(struct resource *old)
>> EXPORT_SYMBOL(release_resource);
>> +/**
>> + * claim_and_punch_out_soft_reserved - Claim a resource region, punching out
>> + * any overlapping "Soft Reserved" areas.
>> + * @root: The root of the resource tree (e.g., &iomem_resource).
>> + * @new: The new resource to insert.
>> + *
>> + * Description:
>> + * This function prepares for the insertion of a new resource (@new) by
>> + * resolving conflicts with existing "Soft Reserved" regions. It works as
>> + * follows:
>> + *
>> + * 1. It iterates through the children of @root, searching for any resource
>> + * that both overlaps with @new and is identified as "Soft Reserved"
>> + * (by its name and the IORES_DESC_SOFT_RESERVED descriptor).
>> + * 2. For each conflicting "Soft Reserved" resource found, it removes the
>> + * original conflicting resource from the tree.
>> + * 3. It then calculates the remaining parts of the original resource that
>> + * lie outside the range of @new. This may result in one or two smaller,
>> + * non-overlapping parts.
>> + * 4. These remaining parts are re-inserted into the resource tree as new,
>> + * smaller "Soft Reserved" resources. This action is akin to "punching a
>> + * hole" in the original reserved region.
>> + * 5. After all conflicts are resolved, the @new resource is inserted into
>> + * the tree, claiming the now-available space.
>> + *
>> + * Return: 0 on success, or a negative error code on failure.
>> + */
>> +int claim_and_punch_out_soft_reserved(struct resource *root,
>> + struct resource *new);
>> +
>> ```
>>
>> Looking forward to your feedback and thoughts on this approach.
>>
>> Thanks,
>> Zhijian
>>
>> -----
>>
>> **References:**
>>
>> [0]
>> [PATCH 0/2] cxl/region: Improve Soft Reserved resource handling: [https://lore.kernel.org/linux-cxl/cover.1687568084.git.alison.schofield@intel.com/](https://lore.kernel.org/linux-cxl/cover.1687568084.git.alison.schofield@intel.com/)
>> [PATCH v2 0/2] cxl/region: Improve Soft Reserved resource handling: [https://lore.kernel.org/linux-cxl/cover.1691176651.git.alison.schofield@intel.com/](https://lore.kernel.org/linux-cxl/cover.1691176651.git.alison.schofield@intel.com/)
>> [PATCH v3 0/2] cxl/region: Improve Soft Reserved resource handling: [https://lore.kernel.org/linux-cxl/cover.1692638817.git.alison.schofield@intel.com/](https://lore.kernel.org/linux-cxl/cover.1692638817.git.alison.schofield@intel.com/)
>> [RFC] cxl: Update Soft Reserved resources upon region creation: [https://lore.kernel.org/linux-cxl/20241004181754.8960-1-nathan.fontenot@amd.com/](https://lore.kernel.org/linux-cxl/20241004181754.8960-1-nathan.fontenot@amd.com/)
>> [RFC v2] cxl: Update Soft Reserved resources upon region creation: [https://lore.kernel.org/linux-cxl/20241030172751.81392-1-nathan.fontenot@amd.com/](https://lore.kernel.org/linux-cxl/20241030172751.81392-1-nathan.fontenot@amd.com/)
>> [PATCH] cxl: Update Soft Reserved resources upon region creation: [https://lore.kernel.org/linux-cxl/20241202155542.22111-1-nathan.fontenot@amd.com/](https://lore.kernel.org/linux-cxl/20241202155542.22111-1-nathan.fontenot@amd.com/)
>> [PATCH v2 0/4] Add managed SOFT RESERVE resource handling: [https://lore.kernel.org/linux-cxl/cover.1737046620.git.nathan.fontenot@amd.com/](https://lore.kernel.org/linux-cxl/cover.1737046620.git.nathan.fontenot@amd.com/)
>> [PATCH v3 0/4] Add managed SOFT RESERVE resource handling: [https://lore.kernel.org/linux-cxl/20250403183315.286710-1-terry.bowman@amd.com/](https://lore.kernel.org/linux-cxl/20250403183315.286710-1-terry.bowman@amd.com/)
>> [PATCH v4 0/7] Add managed SOFT RESERVE resource handling: [https://lore.kernel.org/linux-cxl/20250603221949.53272-1-Smita.KoralahalliChannabasappa@amd.com/#r](https://lore.kernel.org/linux-cxl/20250603221949.53272-1-Smita.KoralahalliChannabasappa@amd.com/#r)
>>
>> [1] [https://lore.kernel.org/linux-cxl/cover.1691176651.git.alison.schofield@intel.com/](https://lore.kernel.org/linux-cxl/cover.1691176651.git.alison.schofield@intel.com/)
>> [2] [https://lore.kernel.org/linux-cxl/20241004181754.8960-1-nathan.fontenot@amd.com/](https://lore.kernel.org/linux-cxl/20241004181754.8960-1-nathan.fontenot@amd.com/)
>
Powered by blists - more mailing lists