[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <ae90c23c-290e-4bc5-9608-6d56d4e83c91@intel.com>
Date: Thu, 4 Dec 2025 09:58:25 -0700
From: Dave Jiang <dave.jiang@...el.com>
To: Dan Williams <dan.j.williams@...el.com>
Cc: linux-cxl@...r.kernel.org, linux-kernel@...r.kernel.org,
Smita.KoralahalliChannabasappa@....com, alison.schofield@...el.com,
terry.bowman@....com, alejandro.lucero-palau@....com,
linux-pci@...r.kernel.org, Jonathan.Cameron@...wei.com,
Alejandro Lucero <alucerop@....com>
Subject: Re: [PATCH 2/6] cxl/mem: Arrange for always-synchronous memdev attach
On 12/3/25 7:21 PM, Dan Williams wrote:
> In preparation for CXL accelerator drivers that have a hard dependency on
> CXL capability initialization, arrange for cxl_mem_probe() to always run
> synchronous with the device_add() of cxl_memdev instances. I.e.
> cxl_mem_driver registration is always complete before the first memdev
> creation event.
>
> At present, cxl_pci does not care about the attach state of the cxl_memdev
> because all generic memory expansion functionality can be handled by the
> cxl_core. For accelerators, however, that driver needs to perform driver
> specific initialization if CXL is available, or execute a fallback to PCIe
> only operation.
>
> This synchronous attach guarantee is also needed for Soft Reserve Recovery,
> which is an effort that needs to assert that devices have had a chance to
> attach before making a go / no-go decision on proceeding with CXL subsystem
> initialization.
>
> By moving devm_cxl_add_memdev() to cxl_mem.ko it removes async module
> loading as one reason that a memdev may not be attached upon return from
> devm_cxl_add_memdev().
>
> Cc: Smita Koralahalli <Smita.KoralahalliChannabasappa@....com>
> Cc: Alejandro Lucero <alucerop@....com>
> Signed-off-by: Dan Williams <dan.j.williams@...el.com>
Reviewed-by: Dave Jiang <dave.jiang@...el.com>
> ---
> drivers/cxl/Kconfig | 2 +-
> drivers/cxl/cxlmem.h | 2 ++
> drivers/cxl/core/memdev.c | 10 +++++++---
> drivers/cxl/mem.c | 17 +++++++++++++++++
> 4 files changed, 27 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
> index 48b7314afdb8..f1361ed6a0d4 100644
> --- a/drivers/cxl/Kconfig
> +++ b/drivers/cxl/Kconfig
> @@ -22,6 +22,7 @@ if CXL_BUS
> config CXL_PCI
> tristate "PCI manageability"
> default CXL_BUS
> + select CXL_MEM
> help
> The CXL specification defines a "CXL memory device" sub-class in the
> PCI "memory controller" base class of devices. Device's identified by
> @@ -89,7 +90,6 @@ config CXL_PMEM
>
> config CXL_MEM
> tristate "CXL: Memory Expansion"
> - depends on CXL_PCI
> default CXL_BUS
> help
> The CXL.mem protocol allows a device to act as a provider of "System
> diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
> index c12ab4fc9512..012e68acad34 100644
> --- a/drivers/cxl/cxlmem.h
> +++ b/drivers/cxl/cxlmem.h
> @@ -95,6 +95,8 @@ static inline bool is_cxl_endpoint(struct cxl_port *port)
> return is_cxl_memdev(port->uport_dev);
> }
>
> +struct cxl_memdev *__devm_cxl_add_memdev(struct device *host,
> + struct cxl_dev_state *cxlds);
> struct cxl_memdev *devm_cxl_add_memdev(struct device *host,
> struct cxl_dev_state *cxlds);
> int devm_cxl_sanitize_setup_notifier(struct device *host,
> diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c
> index 4dff7f44d908..7a4153e1c6a7 100644
> --- a/drivers/cxl/core/memdev.c
> +++ b/drivers/cxl/core/memdev.c
> @@ -1050,8 +1050,12 @@ static const struct file_operations cxl_memdev_fops = {
> .llseek = noop_llseek,
> };
>
> -struct cxl_memdev *devm_cxl_add_memdev(struct device *host,
> - struct cxl_dev_state *cxlds)
> +/*
> + * Core helper for devm_cxl_add_memdev() that wants to both create a device and
> + * assert to the caller that upon return cxl_mem::probe() has been invoked.
> + */
> +struct cxl_memdev *__devm_cxl_add_memdev(struct device *host,
> + struct cxl_dev_state *cxlds)
> {
> struct cxl_memdev *cxlmd;
> struct device *dev;
> @@ -1093,7 +1097,7 @@ struct cxl_memdev *devm_cxl_add_memdev(struct device *host,
> put_device(dev);
> return ERR_PTR(rc);
> }
> -EXPORT_SYMBOL_NS_GPL(devm_cxl_add_memdev, "CXL");
> +EXPORT_SYMBOL_FOR_MODULES(__devm_cxl_add_memdev, "cxl_mem");
>
> static void sanitize_teardown_notifier(void *data)
> {
> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> index 6e6777b7bafb..55883797ab2d 100644
> --- a/drivers/cxl/mem.c
> +++ b/drivers/cxl/mem.c
> @@ -201,6 +201,22 @@ static int cxl_mem_probe(struct device *dev)
> return devm_add_action_or_reset(dev, enable_suspend, NULL);
> }
>
> +/**
> + * devm_cxl_add_memdev - Add a CXL memory device
> + * @host: devres alloc/release context and parent for the memdev
> + * @cxlds: CXL device state to associate with the memdev
> + *
> + * Upon return the device will have had a chance to attach to the
> + * cxl_mem driver, but may fail if the CXL topology is not ready
> + * (hardware CXL link down, or software platform CXL root not attached)
> + */
> +struct cxl_memdev *devm_cxl_add_memdev(struct device *host,
> + struct cxl_dev_state *cxlds)
> +{
> + return __devm_cxl_add_memdev(host, cxlds);
> +}
> +EXPORT_SYMBOL_NS_GPL(devm_cxl_add_memdev, "CXL");
> +
> static ssize_t trigger_poison_list_store(struct device *dev,
> struct device_attribute *attr,
> const char *buf, size_t len)
> @@ -248,6 +264,7 @@ static struct cxl_driver cxl_mem_driver = {
> .probe = cxl_mem_probe,
> .id = CXL_DEVICE_MEMORY_EXPANDER,
> .drv = {
> + .probe_type = PROBE_FORCE_SYNCHRONOUS,
> .dev_groups = cxl_mem_groups,
> },
> };
Powered by blists - more mailing lists