[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <692e7201b9ba6_261c1100b3@dwillia2-mobl4.notmuch>
Date: Mon, 1 Dec 2025 20:58:41 -0800
From: <dan.j.williams@...el.com>
To: <dan.j.williams@...el.com>, <alejandro.lucero-palau@....com>,
<linux-cxl@...r.kernel.org>, <netdev@...r.kernel.org>,
<dan.j.williams@...el.com>, <edward.cree@....com>, <davem@...emloft.net>,
<kuba@...nel.org>, <pabeni@...hat.com>, <edumazet@...gle.com>,
<dave.jiang@...el.com>
CC: Alejandro Lucero <alucerop@....com>
Subject: Re: [PATCH v21 01/23] cxl/mem: refactor memdev allocation
dan.j.williams@ wrote:
[..]
> > diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c
> > index e370d733e440..8de19807ac7b 100644
> > --- a/drivers/cxl/core/memdev.c
> > +++ b/drivers/cxl/core/memdev.c
> > @@ -8,6 +8,7 @@
> > #include <linux/idr.h>
> > #include <linux/pci.h>
> > #include <cxlmem.h>
> > +#include "private.h"
> > #include "trace.h"
> > #include "core.h"
> >
> > @@ -648,42 +649,25 @@ static void detach_memdev(struct work_struct *work)
> >
> > static struct lock_class_key cxl_memdev_key;
> >
> > -static struct cxl_memdev *cxl_memdev_alloc(struct cxl_dev_state *cxlds,
> > - const struct file_operations *fops)
> > +int devm_cxl_memdev_add_or_reset(struct device *host, struct cxl_memdev *cxlmd)
>
> Can you say more why Type-2 drivers need an "_or_reset()" export? If a
> Type-2 driver is calling devm_cxl_add_memdev() from its ->probe()
> routine, then just return on failure. Confused.
Oh is this replacing my "cxl/mem: Arrange for always-synchronous memdev
attach"? I see an _or_reset method in there too. So the description in
my changelog, that did not get carried over to this replacement patch,
is a Type-2 driver may want to fallback PCIe only operation and not rely
on probe failure to cleanup the aborted memdev setup.
...but really that quick and dirty patch from me was poor quality and
introduced bugs. Here is a much smaller version that achieves the same
result and drops the opportunity for new bugs. I will send this out
after rebasing that whole probe-order branch.
-- >8 --
>From c41c535392ed3fcf203d754b8468a5ca91d83438 Mon Sep 17 00:00:00 2001
From: Dan Williams <dan.j.williams@...el.com>
Date: Thu, 31 Jul 2025 14:15:05 -0700
Subject: [PATCH] cxl/mem: Arrange for always-synchronous memdev attach
In preparation for CXL accelerator drivers that have a hard dependency on
CXL capability initialization, arrange for the endpoint probe result to be
conveyed to the caller of devm_cxl_add_memdev().
As it stands cxl_pci does not care about the attach state of the cxl_memdev
because all generic memory expansion functionality can be handled by the
cxl_core. For accelerators, that driver needs to know perform driver
specific initialization if CXL is available, or exectute a fallback to PCIe
only operation.
By moving devm_cxl_add_memdev() to cxl_mem.ko it removes async module
loading as one reason that a memdev may not be attached upon return from
devm_cxl_add_memdev().
Signed-off-by: Dan Williams <dan.j.williams@...el.com>
---
drivers/cxl/Kconfig | 2 +-
drivers/cxl/cxlmem.h | 2 ++
drivers/cxl/core/memdev.c | 10 +++++++---
drivers/cxl/mem.c | 17 +++++++++++++++++
4 files changed, 27 insertions(+), 4 deletions(-)
diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
index 48b7314afdb8..f1361ed6a0d4 100644
--- a/drivers/cxl/Kconfig
+++ b/drivers/cxl/Kconfig
@@ -22,6 +22,7 @@ if CXL_BUS
config CXL_PCI
tristate "PCI manageability"
default CXL_BUS
+ select CXL_MEM
help
The CXL specification defines a "CXL memory device" sub-class in the
PCI "memory controller" base class of devices. Device's identified by
@@ -89,7 +90,6 @@ config CXL_PMEM
config CXL_MEM
tristate "CXL: Memory Expansion"
- depends on CXL_PCI
default CXL_BUS
help
The CXL.mem protocol allows a device to act as a provider of "System
diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
index c12ab4fc9512..012e68acad34 100644
--- a/drivers/cxl/cxlmem.h
+++ b/drivers/cxl/cxlmem.h
@@ -95,6 +95,8 @@ static inline bool is_cxl_endpoint(struct cxl_port *port)
return is_cxl_memdev(port->uport_dev);
}
+struct cxl_memdev *__devm_cxl_add_memdev(struct device *host,
+ struct cxl_dev_state *cxlds);
struct cxl_memdev *devm_cxl_add_memdev(struct device *host,
struct cxl_dev_state *cxlds);
int devm_cxl_sanitize_setup_notifier(struct device *host,
diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c
index 4dff7f44d908..7a4153e1c6a7 100644
--- a/drivers/cxl/core/memdev.c
+++ b/drivers/cxl/core/memdev.c
@@ -1050,8 +1050,12 @@ static const struct file_operations cxl_memdev_fops = {
.llseek = noop_llseek,
};
-struct cxl_memdev *devm_cxl_add_memdev(struct device *host,
- struct cxl_dev_state *cxlds)
+/*
+ * Core helper for devm_cxl_add_memdev() that wants to both create a device and
+ * assert to the caller that upon return cxl_mem::probe() has been invoked.
+ */
+struct cxl_memdev *__devm_cxl_add_memdev(struct device *host,
+ struct cxl_dev_state *cxlds)
{
struct cxl_memdev *cxlmd;
struct device *dev;
@@ -1093,7 +1097,7 @@ struct cxl_memdev *devm_cxl_add_memdev(struct device *host,
put_device(dev);
return ERR_PTR(rc);
}
-EXPORT_SYMBOL_NS_GPL(devm_cxl_add_memdev, "CXL");
+EXPORT_SYMBOL_FOR_MODULES(__devm_cxl_add_memdev, "cxl_mem");
static void sanitize_teardown_notifier(void *data)
{
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
index 6e6777b7bafb..55883797ab2d 100644
--- a/drivers/cxl/mem.c
+++ b/drivers/cxl/mem.c
@@ -201,6 +201,22 @@ static int cxl_mem_probe(struct device *dev)
return devm_add_action_or_reset(dev, enable_suspend, NULL);
}
+/**
+ * devm_cxl_add_memdev - Add a CXL memory device
+ * @host: devres alloc/release context and parent for the memdev
+ * @cxlds: CXL device state to associate with the memdev
+ *
+ * Upon return the device will have had a chance to attach to the
+ * cxl_mem driver, but may fail if the CXL topology is not ready
+ * (hardware CXL link down, or software platform CXL root not attached)
+ */
+struct cxl_memdev *devm_cxl_add_memdev(struct device *host,
+ struct cxl_dev_state *cxlds)
+{
+ return __devm_cxl_add_memdev(host, cxlds);
+}
+EXPORT_SYMBOL_NS_GPL(devm_cxl_add_memdev, "CXL");
+
static ssize_t trigger_poison_list_store(struct device *dev,
struct device_attribute *attr,
const char *buf, size_t len)
@@ -248,6 +264,7 @@ static struct cxl_driver cxl_mem_driver = {
.probe = cxl_mem_probe,
.id = CXL_DEVICE_MEMORY_EXPANDER,
.drv = {
+ .probe_type = PROBE_FORCE_SYNCHRONOUS,
.dev_groups = cxl_mem_groups,
},
};
--
2.51.1
Powered by blists - more mailing lists