Message-ID: <1a788b8b-48b8-4853-906f-97af5952ce21@intel.com>
Date: Wed, 20 Nov 2024 10:15:59 -0700
From: Dave Jiang <dave.jiang@...el.com>
To: Alejandro Lucero Palau <alucerop@....com>, Zhi Wang <zhiw@...dia.com>
Cc: alejandro.lucero-palau@....com, linux-cxl@...r.kernel.org,
 netdev@...r.kernel.org, dan.j.williams@...el.com, martin.habets@...inx.com,
 edward.cree@....com, davem@...emloft.net, kuba@...nel.org,
 pabeni@...hat.com, edumazet@...gle.com
Subject: Re: [PATCH v5 13/27] cxl: prepare memdev creation for type2



On 11/20/24 6:57 AM, Alejandro Lucero Palau wrote:
> 
> On 11/19/24 21:27, Dave Jiang wrote:
>>
>> On 11/19/24 1:06 PM, Zhi Wang wrote:
>>> On Tue, 19 Nov 2024 11:24:44 -0700
>>> Dave Jiang <dave.jiang@...el.com> wrote:
>>>
>>>>
>>>> On 11/18/24 9:44 AM, alejandro.lucero-palau@....com wrote:
>>>>> From: Alejandro Lucero <alucerop@....com>
>>>>>
>>>>> Current cxl core is relying on a CXL_DEVTYPE_CLASSMEM type device
>>>>> when creating a memdev leading to problems when obtaining
>>>>> cxl_memdev_state references from a CXL_DEVTYPE_DEVMEM type. This
>>>>> last device type is managed by a specific vendor driver and does
>>>>> not need same sysfs files since not userspace intervention is
>>>>> expected.
>>>>>
>>>>> Create a new cxl_mem device type with no attributes for Type2.
>>>>>
>>>>> Avoid debugfs files relying on existence of clx_memdev_state.
>>>>>
>>>>> Make devm_cxl_add_memdev accesible from a accel driver.
>>>>>
>>>>> Signed-off-by: Alejandro Lucero <alucerop@....com>
>>>>> ---
>>>>>   drivers/cxl/core/cdat.c   |  3 +++
>>>>>   drivers/cxl/core/memdev.c | 15 +++++++++++++--
>>>>>   drivers/cxl/core/region.c |  3 ++-
>>>>>   drivers/cxl/mem.c         | 25 +++++++++++++++++++------
>>>>>   include/cxl/cxl.h         |  2 ++
>>>>>   5 files changed, 39 insertions(+), 9 deletions(-)
>>>>>
>>>>> diff --git a/drivers/cxl/core/cdat.c b/drivers/cxl/core/cdat.c
>>>>> index e9cd7939c407..192cff18ea25 100644
>>>>> --- a/drivers/cxl/core/cdat.c
>>>>> +++ b/drivers/cxl/core/cdat.c
>>>>> @@ -577,6 +577,9 @@ static struct cxl_dpa_perf
>>>>> *cxled_get_dpa_perf(struct cxl_endpoint_decoder *cxle struct
>>>>> cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds); struct
>>>>> cxl_dpa_perf *perf;
>>>>> +    if (!mds)
>>>>> +        return ERR_PTR(-EINVAL);
>>>>> +
>>>>>       switch (mode) {
>>>>>       case CXL_DECODER_RAM:
>>>>>           perf = &mds->ram_perf;
>>>>> diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c
>>>>> index d746c8a1021c..df31eea0c06b 100644
>>>>> --- a/drivers/cxl/core/memdev.c
>>>>> +++ b/drivers/cxl/core/memdev.c
>>>>> @@ -547,9 +547,17 @@ static const struct device_type
>>>>> cxl_memdev_type = { .groups = cxl_memdev_attribute_groups,
>>>>>   };
>>>>>   +static const struct device_type cxl_accel_memdev_type = {
>>>>> +    .name = "cxl_memdev",
>>>>> +    .release = cxl_memdev_release,
>>>>> +    .devnode = cxl_memdev_devnode,
>>>>> +};
>>>>> +
>>>>>   bool is_cxl_memdev(const struct device *dev)
>>>>>   {
>>>>> -    return dev->type == &cxl_memdev_type;
>>>>> +    return (dev->type == &cxl_memdev_type ||
>>>>> +        dev->type == &cxl_accel_memdev_type);
>>>>> +
>>>>>   }
>>>>>   EXPORT_SYMBOL_NS_GPL(is_cxl_memdev, CXL);
>>>> Does type2 device also exports a CDAT?
>>>>
>>> Yes. Type2 can also export a CDAT.
>> Thanks! Probably should have the split out helpers regardless.
> 
> 
> Maybe, but should not we wait until that is required? I did not see the need for adding them with this patchset.

Sure. I think my concern is with paths that apply only to one type but not the other. If you have not encountered any then we can wait.

DJ

> 
> 
>>>> I'm also wondering if we should have distinctive helpers:
>>>> is_cxl_type3_memdev()
>>>> is_cxl_type2_memdev()
>>>>
>>>> and is_cxl_memdev() is just calling those two helpers above.
>>>>
>>>> And if no CDAT is exported, we should change the is_cxl_memdev() to
>>>> is_cxl_type3_memdev() in read_cdat_data().
>>>>
>>>> DJ
>>>>
>>>>>   @@ -660,7 +668,10 @@ static struct cxl_memdev
>>>>> *cxl_memdev_alloc(struct cxl_dev_state *cxlds, dev->parent =
>>>>> cxlds->dev; dev->bus = &cxl_bus_type;
>>>>>       dev->devt = MKDEV(cxl_mem_major, cxlmd->id);
>>>>> -    dev->type = &cxl_memdev_type;
>>>>> +    if (cxlds->type == CXL_DEVTYPE_DEVMEM)
>>>>> +        dev->type = &cxl_accel_memdev_type;
>>>>> +    else
>>>>> +        dev->type = &cxl_memdev_type;
>>>>>       device_set_pm_not_required(dev);
>>>>>       INIT_WORK(&cxlmd->detach_work, detach_memdev);
>>>>>   diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
>>>>> index dff618c708dc..622e3bb2e04b 100644
>>>>> --- a/drivers/cxl/core/region.c
>>>>> +++ b/drivers/cxl/core/region.c
>>>>> @@ -1948,7 +1948,8 @@ static int cxl_region_attach(struct
>>>>> cxl_region *cxlr, return -EINVAL;
>>>>>       }
>>>>>   -    cxl_region_perf_data_calculate(cxlr, cxled);
>>>>> +    if (cxlr->type == CXL_DECODER_HOSTONLYMEM)
>>>>> +        cxl_region_perf_data_calculate(cxlr, cxled);
>>>>>         if (test_bit(CXL_REGION_F_AUTO, &cxlr->flags)) {
>>>>>           int i;
>>>>> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
>>>>> index a9fd5cd5a0d2..cb771bf196cd 100644
>>>>> --- a/drivers/cxl/mem.c
>>>>> +++ b/drivers/cxl/mem.c
>>>>> @@ -130,12 +130,18 @@ static int cxl_mem_probe(struct device *dev)
>>>>>       dentry = cxl_debugfs_create_dir(dev_name(dev));
>>>>>       debugfs_create_devm_seqfile(dev, "dpamem", dentry,
>>>>> cxl_mem_dpa_show);
>>>>> -    if (test_bit(CXL_POISON_ENABLED_INJECT,
>>>>> mds->poison.enabled_cmds))
>>>>> -        debugfs_create_file("inject_poison", 0200, dentry,
>>>>> cxlmd,
>>>>> -                    &cxl_poison_inject_fops);
>>>>> -    if (test_bit(CXL_POISON_ENABLED_CLEAR,
>>>>> mds->poison.enabled_cmds))
>>>>> -        debugfs_create_file("clear_poison", 0200, dentry,
>>>>> cxlmd,
>>>>> -                    &cxl_poison_clear_fops);
>>>>> +    /*
>>>>> +     * Avoid poison debugfs files for Type2 devices as they
>>>>> rely on
>>>>> +     * cxl_memdev_state.
>>>>> +     */
>>>>> +    if (mds) {
>>>>> +        if (test_bit(CXL_POISON_ENABLED_INJECT,
>>>>> mds->poison.enabled_cmds))
>>>>> +            debugfs_create_file("inject_poison", 0200,
>>>>> dentry, cxlmd,
>>>>> +
>>>>> &cxl_poison_inject_fops);
>>>>> +        if (test_bit(CXL_POISON_ENABLED_CLEAR,
>>>>> mds->poison.enabled_cmds))
>>>>> +            debugfs_create_file("clear_poison", 0200,
>>>>> dentry, cxlmd,
>>>>> +
>>>>> &cxl_poison_clear_fops);
>>>>> +    }
>>>>>         rc = devm_add_action_or_reset(dev, remove_debugfs, dentry);
>>>>>       if (rc)
>>>>> @@ -219,6 +225,13 @@ static umode_t cxl_mem_visible(struct kobject
>>>>> *kobj, struct attribute *a, int n) struct cxl_memdev *cxlmd =
>>>>> to_cxl_memdev(dev); struct cxl_memdev_state *mds =
>>>>> to_cxl_memdev_state(cxlmd->cxlds);
>>>>> +    /*
>>>>> +     * Avoid poison sysfs files for Type2 devices as they rely
>>>>> on
>>>>> +     * cxl_memdev_state.
>>>>> +     */
>>>>> +    if (!mds)
>>>>> +        return 0;
>>>>> +
>>>>>       if (a == &dev_attr_trigger_poison_list.attr)
>>>>>           if (!test_bit(CXL_POISON_ENABLED_LIST,
>>>>>                     mds->poison.enabled_cmds))
>>>>> diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h
>>>>> index 6033ce84b3d3..5608ed0f5f15 100644
>>>>> --- a/include/cxl/cxl.h
>>>>> +++ b/include/cxl/cxl.h
>>>>> @@ -57,4 +57,6 @@ int cxl_pci_accel_setup_regs(struct pci_dev
>>>>> *pdev, struct cxl_dev_state *cxlds); int
>>>>> cxl_request_resource(struct cxl_dev_state *cxlds, enum cxl_resource
>>>>> type); int cxl_release_resource(struct cxl_dev_state *cxlds, enum
>>>>> cxl_resource type); void cxl_set_media_ready(struct cxl_dev_state
>>>>> *cxlds); +struct cxl_memdev *devm_cxl_add_memdev(struct device
>>>>> *host,
>>>>> +                       struct cxl_dev_state
>>>>> *cxlds); #endif
>>>>

Powered by blists - more mailing lists