linux-kernel - Re: [EXT] Re: [RFC PATCH v2 0/2] Node migration between memory tiers

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ZZ2DoPT8LzNzXyme@memverge.com>
Date: Tue, 9 Jan 2024 12:34:24 -0500
From: Gregory Price <gregory.price@...verge.com>
To: "Huang, Ying" <ying.huang@...el.com>
Cc: Srinivasulu Thanneeru <sthanneeru@...ron.com>,
	Srinivasulu Opensrc <sthanneeru.opensrc@...ron.com>,
	"linux-cxl@...r.kernel.org" <linux-cxl@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"aneesh.kumar@...ux.ibm.com" <aneesh.kumar@...ux.ibm.com>,
	"dan.j.williams@...el.com" <dan.j.williams@...el.com>,
	"mhocko@...e.com" <mhocko@...e.com>,
	"tj@...nel.org" <tj@...nel.org>,
	"john@...alactic.com" <john@...alactic.com>,
	Eishan Mirakhur <emirakhur@...ron.com>,
	Vinicius Tavares Petrucci <vtavarespetr@...ron.com>,
	Ravis OpenSrc <Ravis.OpenSrc@...ron.com>,
	"Jonathan.Cameron@...wei.com" <Jonathan.Cameron@...wei.com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	Johannes Weiner <hannes@...xchg.org>, Wei Xu <weixugc@...gle.com>,
	Hao Xiang <hao.xiang@...edance.com>,
	"Ho-Ren (Jack) Chuang" <horenchuang@...edance.com>
Subject: Re: [EXT] Re: [RFC PATCH v2 0/2] Node migration between memory tiers

On Tue, Jan 09, 2024 at 11:41:11AM +0800, Huang, Ying wrote:
> Gregory Price <gregory.price@...verge.com> writes:
> 
> > On Thu, Jan 04, 2024 at 02:05:01PM +0800, Huang, Ying wrote:
> >> >
> >> > From  https://lpc.events/event/16/contributions/1209/attachments/1042/1995/Live%20In%20a%20World%20With%20Multiple%20Memory%20Types.pdf
> >> > abstract_distance_offset: override by users to deal with firmware issue.
> >> >
> >> > say firmware can configure the cxl node into wrong tiers, similar to
> >> > that it may also configure all cxl nodes into single memtype, hence
> >> > all these nodes can fall into a single wrong tier.
> >> > In this case, per node adistance_offset would be good to have ?
> >> 
> >> I think that it's better to fix the error firmware if possible.  And
> >> these are only theoretical, not practical issues.  Do you have some
> >> practical issues?
> >> 
> >> I understand that users may want to move nodes between memory tiers for
> >> different policy choices.  For that, memory_type based adistance_offset
> >> should be good.
> >> 
> >
> > There's actually an affirmative case to change memory tiering to allow
> > either movement of nodes between tiers, or at least base placement on
> > HMAT information. Preferably, membership would be changable to allow
> > hotplug/DCD to be managed (there's no guarantee that the memory passed
> > through will always be what HMAT says on initial boot).
> 
> IIUC, from Jonathan Cameron as below, the performance of memory
> shouldn't change even for DCD devices.
> 
> https://lore.kernel.org/linux-mm/20231103141636.000007e4@Huawei.com/
> 
> It's possible to change the performance of a NUMA node changed, if we
> hot-remove a memory device, then hot-add another different memory
> device.  It's hoped that the CDAT changes too.
> 
> So, all in all, HMAT + CDAT can help us to put the memory device in
> appropriate memory tiers.  Now, we have HMAT support in upstream.  We
> will working on CDAT support.

That should be sufficient assuming the `-numa hmat-lb` setting in QEMU
does the right thing.  I suppose we also need to figure out a way to set
CDAT information for a memory device that isn't related to CXL (from the
perspective of the guest).  I'll take a look if I get cycles.

~Gregory