| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <aBzqf92hI-CIfBGG@aschofie-mobl2.lan>
Date: Thu, 8 May 2025 10:31:43 -0700
From: Alison Schofield <alison.schofield@...el.com>
To: Alejandro Lucero Palau <alucerop@....com>
CC: <alejandro.lucero-palau@....com>, <linux-cxl@...r.kernel.org>,
<netdev@...r.kernel.org>, <dan.j.williams@...el.com>, <edward.cree@....com>,
<davem@...emloft.net>, <kuba@...nel.org>, <pabeni@...hat.com>,
<edumazet@...gle.com>, <dave.jiang@...el.com>, Jonathan Cameron
<Jonathan.Cameron@...wei.com>
Subject: Re: [PATCH v14 11/22] cxl: define a driver interface for HPA free
space enumeration
On Thu, May 08, 2025 at 03:09:03PM +0100, Alejandro Lucero Palau wrote:
>
> On 5/8/25 02:13, Alison Schofield wrote:
> > On Thu, Apr 17, 2025 at 10:29:14PM +0100, alejandro.lucero-palau@....com wrote:
> > > From: Alejandro Lucero <alucerop@....com>
> > >
> > > CXL region creation involves allocating capacity from device DPA
> > > (device-physical-address space) and assigning it to decode a given HPA
> > > (host-physical-address space). Before determining how much DPA to
> > > allocate the amount of available HPA must be determined. Also, not all
> > > HPA is created equal, some specifically targets RAM, some target PMEM,
> > > some is prepared for device-memory flows like HDM-D and HDM-DB, and some
> > > is host-only (HDM-H).
> > >
> > > Wrap all of those concerns into an API that retrieves a root decoder
> > > (platform CXL window) that fits the specified constraints and the
> > > capacity available for a new region.
> > >
> > > Add a complementary function for releasing the reference to such root
> > > decoder.
> > This commit message lacks a why.
> >
> > It would be useful to state whether or not it makes any functional
> > changes to the existing cxl driver hpa handling. Seems not.
> >
>
> I have had to think about the why and I'm not sure I have the right answer,
> so maybe other should comment on this.
>
>
> I think with Type2 support, regions can be created from the drivers now,
> what requires more awareness and to find the proper HPA/cxl root port. Until
> now regions are created from user space, and those sysfs files hide the
> already established link to the cxl root port ... but I can not tell now for
> sure how this is being performed at decoder init time where a region is
> created for committed decoders (by the BIOS).
>
>
> A comment from Dan will be helpful here.
My question on this patch is just specific to this patch. I think I have
my answer - that these new functions have no affect on the behavior of
the cxl region driver when used for Type3.
>
>
> > > Based on https://lore.kernel.org/linux-cxl/168592159290.1948938.13522227102445462976.stgit@dwillia2-xfh.jf.intel.com/
> > >
> > > Signed-off-by: Alejandro Lucero <alucerop@....com>
> > > Reviewed-by: Jonathan Cameron <Jonathan.Cameron@...wei.com>
> > > ---
> > > drivers/cxl/core/region.c | 164 ++++++++++++++++++++++++++++++++++++++
> > > drivers/cxl/cxl.h | 3 +
> > > include/cxl/cxl.h | 11 +++
> > > 3 files changed, 178 insertions(+)
> > >
> > > diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> > > index 80caaf14d08a..0a9eab4f8e2e 100644
> > > --- a/drivers/cxl/core/region.c
> > > +++ b/drivers/cxl/core/region.c
> > > @@ -695,6 +695,170 @@ static int free_hpa(struct cxl_region *cxlr)
> > > return 0;
> > > }
> > > +struct cxlrd_max_context {
> > > + struct device * const *host_bridges;
> > > + int interleave_ways;
> > > + unsigned long flags;
> > > + resource_size_t max_hpa;
> > > + struct cxl_root_decoder *cxlrd;
> > > +};
> > > +
> > > +static int find_max_hpa(struct device *dev, void *data)
> > > +{
> > > + struct cxlrd_max_context *ctx = data;
> > > + struct cxl_switch_decoder *cxlsd;
> > > + struct cxl_root_decoder *cxlrd;
> > > + struct resource *res, *prev;
> > > + struct cxl_decoder *cxld;
> > > + resource_size_t max;
> > > + int found = 0;
> > > +
> > > + if (!is_root_decoder(dev))
> > > + return 0;
> > > +
> > > + cxlrd = to_cxl_root_decoder(dev);
> > > + cxlsd = &cxlrd->cxlsd;
> > > + cxld = &cxlsd->cxld;
> > > +
> > > + /*
> > > + * None flags are declared as bitmaps but for the sake of better code
> > > + * used here as such, restricting the bitmap size to those bits used by
> > > + * any Type2 device driver requester.
> > > + */
> > > + if (!bitmap_subset(&ctx->flags, &cxld->flags, CXL_DECODER_F_MAX)) {
> > > + dev_dbg(dev, "flags not matching: %08lx vs %08lx\n",
> > > + cxld->flags, ctx->flags);
> > > + return 0;
> > > + }
> > > +
> > > + for (int i = 0; i < ctx->interleave_ways; i++) {
> > > + for (int j = 0; j < ctx->interleave_ways; j++) {
> > > + if (ctx->host_bridges[i] == cxlsd->target[j]->dport_dev) {
> > > + found++;
> > > + break;
> > > + }
> > > + }
> > > + }
> > > +
> > > + if (found != ctx->interleave_ways) {
> > > + dev_dbg(dev, "Not enough host bridges found(%d) for interleave ways requested (%d)\n",
> > > + found, ctx->interleave_ways);
> > > + return 0;
> > > + }
> > > +
> > > + /*
> > > + * Walk the root decoder resource range relying on cxl_region_rwsem to
> > > + * preclude sibling arrival/departure and find the largest free space
> > > + * gap.
> > > + */
> > > + lockdep_assert_held_read(&cxl_region_rwsem);
> > > + res = cxlrd->res->child;
> > > +
> > > + /* With no resource child the whole parent resource is available */
> > > + if (!res)
> > > + max = resource_size(cxlrd->res);
> > > + else
> > > + max = 0;
> > > +
> > > + for (prev = NULL; res; prev = res, res = res->sibling) {
> > > + struct resource *next = res->sibling;
> > > + resource_size_t free = 0;
> > > +
> > > + /*
> > > + * Sanity check for preventing arithmetic problems below as a
> > > + * resource with size 0 could imply using the end field below
> > > + * when set to unsigned zero - 1 or all f in hex.
> > > + */
> > > + if (prev && !resource_size(prev))
> > > + continue;
> > > +
> > > + if (!prev && res->start > cxlrd->res->start) {
> > > + free = res->start - cxlrd->res->start;
> > > + max = max(free, max);
> > > + }
> > > + if (prev && res->start > prev->end + 1) {
> > > + free = res->start - prev->end + 1;
> > > + max = max(free, max);
> > > + }
> > > + if (next && res->end + 1 < next->start) {
> > > + free = next->start - res->end + 1;
> > > + max = max(free, max);
> > > + }
> > > + if (!next && res->end + 1 < cxlrd->res->end + 1) {
> > > + free = cxlrd->res->end + 1 - res->end + 1;
> > > + max = max(free, max);
> > > + }
> > > + }
> > > +
> > > + dev_dbg(CXLRD_DEV(cxlrd), "found %pa bytes of free space\n", &max);
> > > + if (max > ctx->max_hpa) {
> > > + if (ctx->cxlrd)
> > > + put_device(CXLRD_DEV(ctx->cxlrd));
> > > + get_device(CXLRD_DEV(cxlrd));
> > > + ctx->cxlrd = cxlrd;
> > > + ctx->max_hpa = max;
> > > + }
> > > + return 0;
> > > +}
> > > +
> > > +/**
> > > + * cxl_get_hpa_freespace - find a root decoder with free capacity per constraints
> > > + * @cxlmd: the CXL memory device with an endpoint that is mapped by the returned
> > > + * decoder
> > > + * @interleave_ways: number of entries in @host_bridges
> > > + * @flags: CXL_DECODER_F flags for selecting RAM vs PMEM, and Type2 device
> > > + * @max_avail_contig: output parameter of max contiguous bytes available in the
> > > + * returned decoder
> > > + *
> > > + * The return tuple of a 'struct cxl_root_decoder' and 'bytes available given
> > > + * in (@max_avail_contig))' is a point in time snapshot. If by the time the
> > > + * caller goes to use this root decoder's capacity the capacity is reduced then
> > > + * caller needs to loop and retry.
> > > + *
> > > + * The returned root decoder has an elevated reference count that needs to be
> > > + * put with cxl_put_root_decoder(cxlrd).
> > > + */
> > > +struct cxl_root_decoder *cxl_get_hpa_freespace(struct cxl_memdev *cxlmd,
> > > + int interleave_ways,
> > > + unsigned long flags,
> > > + resource_size_t *max_avail_contig)
> > > +{
> > > + struct cxl_port *endpoint = cxlmd->endpoint;
> > > + struct cxlrd_max_context ctx = {
> > > + .host_bridges = &endpoint->host_bridge,
> > > + .flags = flags,
> > > + };
> > > + struct cxl_port *root_port;
> > > + struct cxl_root *root __free(put_cxl_root) = find_cxl_root(endpoint);
> > > +
> > > + if (!is_cxl_endpoint(endpoint)) {
> > > + dev_dbg(&endpoint->dev, "hpa requestor is not an endpoint\n");
> > > + return ERR_PTR(-EINVAL);
> > > + }
> > > +
> > > + if (!root) {
> > > + dev_dbg(&endpoint->dev, "endpoint can not be related to a root port\n");
> > > + return ERR_PTR(-ENXIO);
> > > + }
> > > +
> > > + root_port = &root->port;
> > > + scoped_guard(rwsem_read, &cxl_region_rwsem)
> > > + device_for_each_child(&root_port->dev, &ctx, find_max_hpa);
> > > +
> > > + if (!ctx.cxlrd)
> > > + return ERR_PTR(-ENOMEM);
> > > +
> > > + *max_avail_contig = ctx.max_hpa;
> > > + return ctx.cxlrd;
> > > +}
> > > +EXPORT_SYMBOL_NS_GPL(cxl_get_hpa_freespace, "CXL");
> > > +
> > > +void cxl_put_root_decoder(struct cxl_root_decoder *cxlrd)
> > > +{
> > > + put_device(CXLRD_DEV(cxlrd));
> > > +}
> > > +EXPORT_SYMBOL_NS_GPL(cxl_put_root_decoder, "CXL");
> > > +
> > > static ssize_t size_store(struct device *dev, struct device_attribute *attr,
> > > const char *buf, size_t len)
> > > {
> > > diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> > > index 4523864eebd2..c35620c24c8f 100644
> > > --- a/drivers/cxl/cxl.h
> > > +++ b/drivers/cxl/cxl.h
> > > @@ -672,6 +672,9 @@ struct cxl_root_decoder *to_cxl_root_decoder(struct device *dev);
> > > struct cxl_switch_decoder *to_cxl_switch_decoder(struct device *dev);
> > > struct cxl_endpoint_decoder *to_cxl_endpoint_decoder(struct device *dev);
> > > bool is_root_decoder(struct device *dev);
> > > +
> > > +#define CXLRD_DEV(cxlrd) (&(cxlrd)->cxlsd.cxld.dev)
> > > +
> > > bool is_switch_decoder(struct device *dev);
> > > bool is_endpoint_decoder(struct device *dev);
> > > struct cxl_root_decoder *cxl_root_decoder_alloc(struct cxl_port *port,
> > > diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h
> > > index 9c0f097ca6be..e9ae7eff2393 100644
> > > --- a/include/cxl/cxl.h
> > > +++ b/include/cxl/cxl.h
> > > @@ -26,6 +26,11 @@ enum cxl_devtype {
> > > struct device;
> > > +#define CXL_DECODER_F_RAM BIT(0)
> > > +#define CXL_DECODER_F_PMEM BIT(1)
> > > +#define CXL_DECODER_F_TYPE2 BIT(2)
> > > +#define CXL_DECODER_F_MAX 3
> > > +
> > > /*
> > > * Capabilities as defined for:
> > > *
> > > @@ -250,4 +255,10 @@ void cxl_mem_dpa_init(struct cxl_dpa_info *info, u64 volatile_bytes,
> > > int cxl_dpa_setup(struct cxl_dev_state *cxlds, const struct cxl_dpa_info *info);
> > > struct cxl_memdev *devm_cxl_add_memdev(struct device *host,
> > > struct cxl_dev_state *cxlmds);
> > > +struct cxl_port;
> > > +struct cxl_root_decoder *cxl_get_hpa_freespace(struct cxl_memdev *cxlmd,
> > > + int interleave_ways,
> > > + unsigned long flags,
> > > + resource_size_t *max);
> > > +void cxl_put_root_decoder(struct cxl_root_decoder *cxlrd);
> > > #endif /* __CXL_CXL_H__ */
> > > --
> > > 2.34.1
> > >
> > >
Powered by blists - more mailing lists