[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20241107113205.00007f53@Huawei.com>
Date: Thu, 7 Nov 2024 11:32:05 +0000
From: Jonathan Cameron <Jonathan.Cameron@...wei.com>
To: <ira.weiny@...el.com>
CC: Dave Jiang <dave.jiang@...el.com>, Fan Ni <fan.ni@...sung.com>, "Navneet
Singh" <navneet.singh@...el.com>, Jonathan Corbet <corbet@....net>, "Andrew
Morton" <akpm@...ux-foundation.org>, Dan Williams <dan.j.williams@...el.com>,
Davidlohr Bueso <dave@...olabs.net>, "Alison Schofield"
<alison.schofield@...el.com>, Vishal Verma <vishal.l.verma@...el.com>,
<linux-cxl@...r.kernel.org>, <linux-doc@...r.kernel.org>,
<nvdimm@...ts.linux.dev>, <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v6 23/27] dax/region: Create resources on sparse DAX
regions
On Tue, 05 Nov 2024 12:38:45 -0600
ira.weiny@...el.com wrote:
> From: Navneet Singh <navneet.singh@...el.com>
>
> DAX regions which map dynamic capacity partitions require that memory be
> allowed to come and go. Recall sparse regions were created for this
> purpose. Now that extents can be realized within DAX regions the DAX
> region driver can start tracking sub-resource information.
>
> The tight relationship between DAX region operations and extent
> operations require memory changes to be controlled synchronously with
> the user of the region. Synchronize through the dax_region_rwsem and by
> having the region driver drive both the region device as well as the
> extent sub-devices.
>
> Recall requests to remove extents can happen at any time and that a host
> is not obligated to release the memory until it is not being used. If
> an extent is not used allow a release response.
>
> When extents are eligible for release. No mappings exist but data may
> reside in caches not yet written to the device. Call
> cxl_region_invalidate_memregion() to write back data to the device prior
> to signaling the release complete.
>
> Speculative writes after a release may dirty the cache such that a read
> from a newly surfaced extent may not come from the device. Call
> cxl_region_invalidate_memregion() prior to bringing a new extent online
> to ensure the cache is marked invalid.
>
> While these invalidate calls are inefficient they are the best we can do
> to ensure cache consistency without back invalidate. Furthermore this
> should occur infrequently with sufficiently large extents and work loads
> to not be too bad of an impact.
>
> The DAX layer has no need for the details of the CXL memory extent
> devices. Expose extents to the DAX layer as device children of the DAX
> region device. A single callback from the driver aids the DAX layer to
> determine if the child device is an extent. The DAX layer also
> registers a devres function to automatically clean up when the device is
> removed from the region.
>
> There is a race between extents being surfaced and the dax_cxl driver
> being loaded. The driver must therefore scan for any existing extents
> while still under the device lock.
>
> Respond to extent notifications. Manage the DAX region resource tree
> based on the extents lifetime. Return the status of remove
> notifications to lower layers such that it can manage the hardware
> appropriately.
>
> Signed-off-by: Navneet Singh <navneet.singh@...el.com>
> Co-developed-by: Ira Weiny <ira.weiny@...el.com>
> Signed-off-by: Ira Weiny <ira.weiny@...el.com>
Same again.
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@...wei.com>
Powered by blists - more mailing lists