linux-kernel - Re: CXL Boot to Bash - Section 3: Memory (block) Hotplug

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Z7TswQbpPV590ADr@gourry-fedora-PF4VCD3F>
Date: Tue, 18 Feb 2025 15:25:37 -0500
From: Gregory Price <gourry@...rry.net>
To: David Hildenbrand <david@...hat.com>
Cc: Yang Shi <shy828301@...il.com>, lsf-pc@...ts.linux-foundation.org,
	linux-mm@...ck.org, linux-cxl@...r.kernel.org,
	linux-kernel@...r.kernel.org
Subject: Re: CXL Boot to Bash - Section 3: Memory (block) Hotplug

On Tue, Feb 18, 2025 at 08:25:59PM +0100, David Hildenbrand wrote:
> On 18.02.25 19:04, Gregory Price wrote:
> 
> Hm?
> 
> If you enable memmap_on_memory, we will place the memmap on that carved-out
> region, independent of ZONE_NORMAL/ZONE_MOVABLE etc. It's the "altmap".
>
> Reason that we can place the memmap on a ZONE_MOVABLE is because, although
> it is "unmovable", we told memory offlining code that it doesn't have to
> care about offlining that memmap carveout, there is no migration to be done.
> Just offline the block (memmap gets stale) and remove that block (memmap
> gets removed).
> 
> If there is a reason where we carve out the memmap and *not* use it, that
> case must be fixed.
> 

Hm, I managed to trace down the wrong path on this particular code.

I will go back and redo my tests to sanity check, but here's what I
would expect to see:

1) if memmap_on_memory is off, and hotplug capacity (node1) is
   zone_movable - then zone_normal (node0) should have N pages
   accounted in nr_memmap_pages

   1a) when dropping these memory blocks, I should see node0 memory
       use drop by 4GB - since this is just GFP_KERNEL pages.

2) if memmap_on_memory is on, and hotplug capacity (node1) is
   zone_movable - then each memory block (256MB) should appear
   as 252MB (-4MB of 64-byte page structs).  For 256GB (my system)
   I should see a total of 252GB of onlined memory (-4GB of page struct)

   2a) when dropping these memory blocks, I should see node0 memory use
       stay the same - since it was vmemmap usage.

I will double check that this isn't working as expected, and i'll double
check for a build option as well.

stupid question - it sorta seems like you'd want this as the default
setting for driver-managed hotplug memory blocks, but I suppose for
very small blocks there's problems (as described in the docs).

:thinking: - is it silly to suggest maybe a per-driver memmap_on_memory
setting rather than just a global setting?  For CXL capacity, this seems
like a no-brainer since blocks can't be smaller than 256MB (per spec).

~Gregory