[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <67119a9189428_3f14294b4@dwillia2-xfh.jf.intel.com.notmuch>
Date: Thu, 17 Oct 2024 16:15:29 -0700
From: Dan Williams <dan.j.williams@...el.com>
To: Huang Ying <ying.huang@...el.com>, Dan Williams
<dan.j.williams@...el.com>, Dave Jiang <dave.jiang@...el.com>
CC: <linux-cxl@...r.kernel.org>, <linux-kernel@...r.kernel.org>, Huang Ying
<ying.huang@...el.com>, Gregory Price <gourry@...rry.net>, Davidlohr Bueso
<dave@...olabs.net>, Jonathan Cameron <jonathan.cameron@...wei.com>, "Alison
Schofield" <alison.schofield@...el.com>, Vishal Verma
<vishal.l.verma@...el.com>, Ira Weiny <ira.weiny@...el.com>, Alejandro Lucero
<alucerop@....com>, Ben Cheatham <benjamin.cheatham@....com>
Subject: Re: [PATCH 5/5] cxl: Avoid to create dax regions for type2
accelerators
Huang Ying wrote:
> The memory range of a type2 accelerator should be managed by the type2
> accelerator specific driver instead of the common dax region drivers,
> as discussed in [1].
>
> [1] https://lore.kernel.org/linux-cxl/66469ff1b8fbc_2c2629427@dwillia2-xfh.jf.intel.com.notmuch/
>
> So, the patch skips dax regions creation for type2 accelerator device
> memory regions.
>
> Based on: https://lore.kernel.org/linux-cxl/168592159835.1948938.1647215579839222774.stgit@dwillia2-xfh.jf.intel.com/
>
> Signed-off-by: "Huang, Ying" <ying.huang@...el.com>
> Co-developed-by: Dan Williams <dan.j.williams@...el.com>
> Signed-off-by: Dan Williams <dan.j.williams@...el.com>
> Reviewed-by: Gregory Price <gourry@...rry.net>
> Cc: Davidlohr Bueso <dave@...olabs.net>
> Cc: Jonathan Cameron <jonathan.cameron@...wei.com>
> Cc: Dave Jiang <dave.jiang@...el.com>
> Cc: Alison Schofield <alison.schofield@...el.com>
> Cc: Vishal Verma <vishal.l.verma@...el.com>
> Cc: Ira Weiny <ira.weiny@...el.com>
> Cc: Alejandro Lucero <alucerop@....com>
> Cc: Ben Cheatham <benjamin.cheatham@....com>
> ---
> drivers/cxl/core/region.c | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index d709738ada61..708be236c9a2 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c
> @@ -3473,6 +3473,14 @@ static int cxl_region_probe(struct device *dev)
> p->res->start, p->res->end, cxlr,
> is_system_ram) > 0)
> return 0;
> + /*
> + * Accelerator regions have specific usage, skip
> + * device-dax registration.
> + */
> + if (cxlr->type == CXL_DECODER_ACCEL)
> + return 0;
> +
> + /* Expander routes to device-dax */
So Linux is this weird career choice where you get to argue with
yourself months later. I understand why I took this shortcut of assuming
that the coherence mode determines device-dax routing, but that is not
sufficient.
An HDM-DB region could want the device-dax uAPI, and an HDM-H range
could want to do its own uAPI besides device-dax.
So, in retrospect, I think it is a mistake to assume uAPI from coherence
mode. It really is a property of the region decided by the region
creator independent of the coherence mode or the device type.
I am thinking that 'struct cxl_region' grows something like:
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index 5406e3ab3d4a..4cf1d030404d 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -511,12 +511,25 @@ struct cxl_region_params {
*/
#define CXL_REGION_F_NEEDS_RESET 1
+/*
+ * enum cxl_mem_api - where to route this cxl region
+ * @CXL_MEM_API_DAX: application specific / soft-reserved memory
+ * @CXL_MEM_API_PMEM: direct region to the NVDIMM subsystem
+ * @CXL_MEM_API_NONE: accelerator specific / hard-reserved memory
+ */
+enum cxl_mem_api {
+ CXL_MEM_API_DAX,
+ CXL_MEM_API_PMEM,
+ CXL_MEM_API_NONE,
+};
+
/**
* struct cxl_region - CXL region
* @dev: This region's device
* @id: This region's id. Id is globally unique across all regions
* @mode: Endpoint decoder allocation / access mode
* @type: Endpoint decoder target type
+ * @api: What if any subsystem will present this region to consumers
* @cxl_nvb: nvdimm bridge for coordinating @cxlr_pmem setup / shutdown
* @cxlr_pmem: (for pmem regions) cached copy of the nvdimm bridge
* @flags: Region state flags
@@ -530,6 +543,7 @@ struct cxl_region {
int id;
enum cxl_decoder_mode mode;
enum cxl_decoder_type type;
+ enum cxl_mem_api api;
struct cxl_nvdimm_bridge *cxl_nvb;
struct cxl_pmem_region *cxlr_pmem;
unsigned long flags;
Now, I have not seen how Alejandro's series handles this, but this
type-2 series was shorter so I started here first.
Powered by blists - more mailing lists