| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <46b8e026-78c5-46de-97b7-074c1e75fd08@fujitsu.com>
Date: Wed, 27 Aug 2025 06:30:05 +0000
From: "Zhijian Li (Fujitsu)" <lizhijian@...itsu.com>
To: "Koralahalli Channabasappa, Smita"
<Smita.KoralahalliChannabasappa@....com>, Alison Schofield
<alison.schofield@...el.com>
CC: "dan.j.williams@...el.com" <dan.j.williams@...el.com>,
"linux-cxl@...r.kernel.org" <linux-cxl@...r.kernel.org>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
"nvdimm@...ts.linux.dev" <nvdimm@...ts.linux.dev>,
"linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
"linux-pm@...r.kernel.org" <linux-pm@...r.kernel.org>, Davidlohr Bueso
<dave@...olabs.net>, Jonathan Cameron <jonathan.cameron@...wei.com>, Dave
Jiang <dave.jiang@...el.com>, Vishal Verma <vishal.l.verma@...el.com>, Ira
Weiny <ira.weiny@...el.com>, Matthew Wilcox <willy@...radead.org>, Jan Kara
<jack@...e.cz>, "Rafael J . Wysocki" <rafael@...nel.org>, Len Brown
<len.brown@...el.com>, Pavel Machek <pavel@...nel.org>, Li Ming
<ming.li@...omail.com>, Jeff Johnson <jeff.johnson@....qualcomm.com>, Ying
Huang <huang.ying.caritas@...il.com>, "Xingtao Yao (Fujitsu)"
<yaoxt.fnst@...itsu.com>, Peter Zijlstra <peterz@...radead.org>, Greg KH
<gregkh@...uxfoundation.org>, Nathan Fontenot <nathan.fontenot@....com>,
Terry Bowman <terry.bowman@....com>, Robert Richter <rrichter@....com>,
Benjamin Cheatham <benjamin.cheatham@....com>, PradeepVineshReddy Kodamati
<PradeepVineshReddy.Kodamati@....com>, "Yasunori Gotou (Fujitsu)"
<y-goto@...itsu.com>
Subject: Re: [PATCH v5 3/7] cxl/acpi: Add background worker to coordinate with
cxl_mem probe completion
All,
I have confirmed that in the !CXL_REGION configuration, the same environment may fail to fall back to hmem.(Your new patch cannot resolve this issue)
In my environment:
- There are two CXL memory devices corresponding to:
```
5d0000000-6cffffff : CXL Window 0
6d0000000-7cffffff : CXL Window 1
```
- E820 table contains a 'soft reserved' entry:
```
[ 0.000000] BIOS-e820: [mem 0x00000005d0000000-0x00000007cfffffff] soft reserved
```
However, since my ACPI SRAT doesn't describe the CXL memory devices (the point), `acpi/hmat.c` won't allocate memory targets for them. This prevents the call chain:
```c
hmat_register_target_devices() // for each SRAT-described target
-> hmem_register_resource()
-> insert entry into "HMEM devices" resource
```
Therefore, for successful fallback to hmem in this environment: `dax_hmem.ko` and `kmem.ko` must request resources BEFORE `cxl_acpi.ko` inserts 'CXL Window X'
However the kernel cannot guarantee this initialization order.
When cxl_acpi runs before dax_kmem/kmem:
```
(built-in) CXL_REGION=n
driver/dax/hmem/device.c cxl_acpi.ko dax_hmem.ko kmem.ko
(1) Add entry '15d0000000-7cfffffff'
(2) Traverse "HMEM devices"
Insert to iomem:
5d0000000-7cffffff : Soft Reserved
(3) Insert CXL Window 0/1
/proc/iomem shows:
5d0000000-7cffffff : Soft Reserved
5d0000000-6cffffff : CXL Window 0
6d0000000-7cffffff : CXL Window 1
(4) Create dax device
(5) request_mem_region() fails
for 5d0000000-7cffffff
Reason: Children of 'Soft Reserved'
(CXL Windows 0/1) don't cover full range
```
---------------------
In my another environment where ACPI SRAT has separate entries per CXL device:
1. `acpi/hmat.c` inserts two entries into "HMEM devices":
- 5d0000000-6cffffff
- 6d0000000-7cffffff
2. Regardless of module order, dax/kmem requests per-device resources, resulting in:
```
5d0000000-7cffffff : Soft Reserved
5d0000000-6cffffff : CXL Window 0
5d0000000-6cffffff : dax0.0
5d0000000-6cffffff : System RAM (kmem)
6d0000000-7cffffff : CXL Window 1
6d0000000-7cffffff : dax1.0
6d0000000-7cffffff : System RAM (kmem)
```
Thanks,
Zhijian
On 25/08/2025 15:50, Li Zhijian wrote:
>
>
> On 22/08/2025 11:56, Koralahalli Channabasappa, Smita wrote:
>>>
>>>>
>>>>> ```
>>>>>
>>>>> 3. When CXL_REGION is disabled, there is a failure to fallback to dax_hmem, in which case only CXL Window X is visible.
>>>>
>>>> Haven't tested !CXL_REGION yet.
>>
>> When CXL_REGION is disabled, DEV_DAX_CXL will also be disabled. So dax_hmem should handle it.
>
> Yes, falling back to dax_hmem/kmem is the result we expect.
> I haven't figured out the root cause of the issue yet, but I can tell you that in my QEMU environment,
> there is currently a certain probability that it cannot fall back to dax_hmem/kmem.
>
> Upon its failure, I observed the following warnings and errors (with my local fixup kernel).
> [ 12.203254] kmem dax0.0: mapping0: 0x5d0000000-0x7cfffffff could not reserve region
> [ 12.203437] kmem dax0.0: probe with driver kmem failed with error -16
>
>
>
>> I was able to fallback to dax_hmem. But let me know if I'm missing something.
>>
>> config DEV_DAX_CXL
>> tristate "CXL DAX: direct access to CXL RAM regions"
>> depends on CXL_BUS && CXL_REGION && DEV_DAX
>> ..
>>
>>>>
>>>>> On failure:
>>>>> ```
>>>>> 100000000-27ffffff : System RAM
>>>>> 5c0001128-5c00011b7 : port1
>>>>> 5c0011128-5c00111b7 : port2
>>>>> 5d0000000-6cffffff : CXL Window 0
>>>>> 6d0000000-7cffffff : CXL Window 1
>>>>> 7000000000-700000ffff : PCI Bus 0000:0c
>>>>> 7000000000-700000ffff : 0000:0c:00.0
>>>>> 7000001080-70000010d7 : mem1
>>>>> ```
>>>>>
>>>>> On success:
>>>>> ```
>>>>> 5d0000000-7cffffff : dax0.0
>>>>> 5d0000000-7cffffff : System RAM (kmem)
>>>>> 5d0000000-6cffffff : CXL Window 0
>>>>> 6d0000000-7cffffff : CXL Window 1
>>>>> ```
Powered by blists - more mailing lists