linux-kernel - Re: [RFC PATCH] resource: Fix CXL node not populated issue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <67594f90a7b87_25073294f0@dwillia2-xfh.jf.intel.com.notmuch>
Date: Wed, 11 Dec 2024 00:38:40 -0800
From: Dan Williams <dan.j.williams@...el.com>
To: Bharata B Rao <bharata@....com>, Dan Williams <dan.j.williams@...el.com>,
	Raghavendra K T <raghavendra.kt@....com>, <linux-kernel@...r.kernel.org>,
	<linux-cxl@...r.kernel.org>
CC: Huang Ying <ying.huang@...el.com>, Andrew Morton
	<akpm@...ux-foundation.org>, David Hildenbrand <david@...hat.com>, "Davidlohr
 Bueso" <dave@...olabs.net>, Jonathan Cameron <jonathan.cameron@...wei.com>,
	Dave Jiang <dave.jiang@...el.com>, Alison Schofield
	<alison.schofield@...el.com>, Vishal Verma <vishal.l.verma@...el.com>, "Ira
 Weiny" <ira.weiny@...el.com>, Alistair Popple <apopple@...dia.com>, "Andy
 Shevchenko" <andriy.shevchenko@...ux.intel.com>, Bjorn Helgaas
	<bhelgaas@...gle.com>, Baoquan He <bhe@...hat.com>,
	<ilpo.jarvinen@...ux.intel.com>, Mika Westerberg
	<mika.westerberg@...ux.intel.com>, Fontenot Nathan <Nathan.Fontenot@....com>,
	Wei Huang <wei.huang2@....com>, <regressions@...ts.linux.dev>
Subject: Re: [RFC PATCH] resource: Fix CXL node not populated issue

Bharata B Rao wrote:
> On 11-Dec-24 10:14 AM, Dan Williams wrote:
> > 
> > I was able to reproduce a similar set of conditions with the cxl_test
> > environment:
> > 
> > f010000000-f04fffffff : Soft Reserved
> >    f010000000-f04fffffff : CXL Window 0
> >      f020000000-f03fffffff : region3
> >        f020000000-f03fffffff : dax3.0
> >          f020000000-f03fffffff : System RAM (kmem)
> > 
> > 
> > ...but that did not result in the bug. So there are some other details
> > missing. Can you proceed with providing the dmesg from the good and the
> > bad cases?
> > 
> > gist.github.com is useful for this.
> 
> Sorry for the delay in providing the data. The system wasn't available 
> for a while. Now I have put the good(6.11.0-rc6) and bad(6.13.0-rc1) 
> dmesg and iomem logs at
> 
> https://gist.github.com/bharata/4a57db11e044fd1d313035f3dd5f763b
> 
> What I see is that in the bad case, we hit the below shown dev_dbg 
> message (not seen in the logs that are provided)
> 
> static int hmem_register_device(struct device *host, int target_nid,
>                                  const struct resource *res)
> {
>          struct platform_device *pdev;
>          struct memregion_info info;
>          long id;
>          int rc;
> 
>          if (IS_ENABLED(CONFIG_CXL_REGION) &&
>              region_intersects(res->start, resource_size(res), 
> IORESOURCE_MEM,
>                                IORES_DESC_CXL) != REGION_DISJOINT) {
>                  dev_dbg(host, "deferring range to CXL: %pr\n", res); <--
>                  return 0;
>          }
> 
> With this, it appears that dev_dax_kmem_probe() won't happen for the CXL 
> range and hence CXL memory doesn't get detected.
> 
> Let us know if you need more debug data.

Ah, thanks for that!

So, it turns out Ying's patch is working as advertised. It is traversing
past the top-level entry of the the iomem_resource topology to discover that a
Soft Reserved range is described by CXL. Then the expectation is that
the CXL subsystem parses the topology and registers a dax device.

I missed that detail in my repro because I was not testing the HMEM
handoff.

Now, the dmesg shows that the CXL subsystem gives up early on the
CXL devices as they do not appear to be meeting the expecations of a "CXL
Memory Device" as described by the "PCI Header - Class Code Register
(Offset 09h)" of the CXL spec.

Specifically these messages:

    [    4.449072] cxl_pci 0000:9f:00.0: registers not found: status mbox memdev 

I think this situation is increasingly showing that the pace of
non-standard quirks being deployed is higher than CXL subsystem is able
to keep pace.

The immediate workaround to this problem that Linux discovered is to
disable the cxl_acpi driver. Can you confirm that preventing the
cxl_acpi driver from loading restores that missing node?

Longer term the urgency of Nathan's patch [1] needs to be escalated.

[1]: http://lore.kernel.org/20241202155542.22111-1-nathan.fontenot@amd.com