linux-kernel - RE: [PATCH v4 0/9] dax/hmem, cxl: Coordinate Soft Reserved handling with CXL and HMEM

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID:
 <OS9PR01MB124214C25B1A4A4FA1075CADA90A7A@OS9PR01MB12421.jpnprd01.prod.outlook.com>
Date: Fri, 5 Dec 2025 02:54:13 +0000
From: "Yasunori Gotou (Fujitsu)" <y-goto@...itsu.com>
To: "'dan.j.williams@...el.com'" <dan.j.williams@...el.com>, "Tomasz Wolski
 (Fujitsu)" <tomasz.wolski@...itsu.com>, "alison.schofield@...el.com"
	<alison.schofield@...el.com>
CC: "Smita.KoralahalliChannabasappa@....com"
	<Smita.KoralahalliChannabasappa@....com>, "ardb@...nel.org"
	<ardb@...nel.org>, "benjamin.cheatham@....com" <benjamin.cheatham@....com>,
	"bp@...en8.de" <bp@...en8.de>, "dave.jiang@...el.com" <dave.jiang@...el.com>,
	"dave@...olabs.net" <dave@...olabs.net>, "gregkh@...uxfoundation.org"
	<gregkh@...uxfoundation.org>, "huang.ying.caritas@...il.com"
	<huang.ying.caritas@...il.com>, "ira.weiny@...el.com" <ira.weiny@...el.com>,
	"jack@...e.cz" <jack@...e.cz>, "jeff.johnson@....qualcomm.com"
	<jeff.johnson@....qualcomm.com>, "jonathan.cameron@...wei.com"
	<jonathan.cameron@...wei.com>, "len.brown@...el.com" <len.brown@...el.com>,
	"linux-cxl@...r.kernel.org" <linux-cxl@...r.kernel.org>,
	"linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-pm@...r.kernel.org" <linux-pm@...r.kernel.org>, "Zhijian Li (Fujitsu)"
	<lizhijian@...itsu.com>, "ming.li@...omail.com" <ming.li@...omail.com>,
	"nathan.fontenot@....com" <nathan.fontenot@....com>, "nvdimm@...ts.linux.dev"
	<nvdimm@...ts.linux.dev>, "pavel@...nel.org" <pavel@...nel.org>,
	"peterz@...radead.org" <peterz@...radead.org>, "rafael@...nel.org"
	<rafael@...nel.org>, "rrichter@....com" <rrichter@....com>,
	"terry.bowman@....com" <terry.bowman@....com>, "vishal.l.verma@...el.com"
	<vishal.l.verma@...el.com>, "willy@...radead.org" <willy@...radead.org>,
	"Xingtao Yao (Fujitsu)" <yaoxt.fnst@...itsu.com>, "yazen.ghannam@....com"
	<yazen.ghannam@....com>
Subject: RE: [PATCH v4 0/9] dax/hmem, cxl: Coordinate Soft Reserved handling
 with CXL and HMEM

Hi,
Just one comment.


> > [4] Physical machine: 2 CFMWS + Host-bridge + 2 CXL devices
> >
> > kernel: BIOS-e820: [mem 0x0000002070000000-0x000000a06fffffff] soft
> > reserved
> >
> > 2070000000-606fffffff : CXL Window 0
> >   2070000000-606fffffff : region0
> >     2070000000-606fffffff : dax0.0
> >       2070000000-606fffffff : System RAM (kmem) 6070000000-a06fffffff
> > : CXL Window 1
> >   6070000000-a06fffffff : region1
> >     6070000000-a06fffffff : dax1.0
> >       6070000000-a06fffffff : System RAM (kmem)
> 
> Ok, so a real world maching that creates a merged
> 0x0000002070000000-0x000000a06fffffff range. Can you confirm that the SRAT
> has separate entries for those ranges? Otherwise, need to rethink how to keep
> this fallback algorithm simple and predictable.
> 
> > kernel: BIOS-e820: [mem 0x0000002070000000-0x000000a06fffffff] soft
> > reserved
> >
> > == region 1 teardown and unplug (the unplug was done via ubind/remove
> > in /sys/bus/pci/devices)
> 
> Note that you need to explicitly destroy the region for the physical removal case.
> Otherwise, decoders stay committed throughout the hierarchy. Simple unbind /
> PCI device removal does not manage CXL decoders.
> 
> >
> > 2070000000-606fffffff : CXL Window 0
> >   2070000000-606fffffff : region0
> >     2070000000-606fffffff : dax0.0
> >       2070000000-606fffffff : System RAM (kmem) 6070000000-a06fffffff
> > : CXL Window 1
> >
> > == plug - after PCI rescan cannot create hmem 6070000000-a06fffffff :
> > CXL Window 1
> >   6070000000-a06fffffff : region1
> >
> > kernel: cxl_region region1: config state: 0
> > kernel: cxl_acpi ACPI0017:00: decoder0.1: created region1
> > kernel: cxl_pci 0000:04:00.0: mem1:decoder10.0: __construct_region
> > region1 res: [mem 0x6070000000-0xa06fffffff flags 0x200] iw: 1 ig:
> > 4096
> > kernel: cxl_mem mem1: decoder:decoder10.0 parent:0000:04:00.0
> > port:endpoint10 range:0x6070000000-0xa06fffffff pos:0
> > kernel: cxl region1: region sort successful
> > kernel: cxl region1: mem1:endpoint10 decoder10.0 add: mem1:decoder10.0
> > @ 0 next: none nr_eps: 1 nr_targets: 1
> > kernel: cxl region1: pci0000:00:port2 decoder2.1 add: mem1:decoder10.0
> > @ 0 next: mem1 nr_eps: 1 nr_targets: 1
> > kernel: cxl region1: pci0000:00:port2 cxl_port_setup_targets expected
> > iw: 1 ig: 4096 [mem 0x6070000000-0xa06fffffff flags 0x200]
> > kernel: cxl region1: pci0000:00:port2 cxl_port_setup_targets got iw: 1
> > ig: 256 state: disabled 0x6070000000:0xa06fffffff
> 
> Did the device get reset in the process? This looks like decoders bounced in an
> inconsistent fashion from unplug to replug and autodiscovery.

You are correct.
This environment does not support actual PCIe hotplug.
Even if we perform PCIe hotplug emulation by manipulating sysfs, some CXL Decoder registers,
which have read-only attributes, are not initialized.
I confirmed about a month and a half ago that this was causing the hot-add process to fail.
I suspect that such registers must be initialized by the hardware when a hot-add occurs.

I should have informed Wolski-san about this in advance. My apologies.

Thank you,
---
Yasunori Goto