Message-ID: <4bab9000-0b49-a852-d574-1c8b2fe10de1@redhat.com>
Date:   Wed, 6 Oct 2021 10:01:39 +0200
From:   David Hildenbrand <david@...hat.com>
To:     Mike Rapoport <rppt@...nel.org>
Cc:     linux-kernel@...r.kernel.org,
        Andrew Morton <akpm@...ux-foundation.org>,
        Jonathan Corbet <corbet@....net>,
        Michal Hocko <mhocko@...e.com>,
        Oscar Salvador <osalvador@...e.de>, linux-doc@...r.kernel.org,
        linux-mm@...ck.org
Subject: Re: [PATCH v1 3/3] memory-hotplug.rst: document the "auto-movable"
 online policy

On 06.10.21 02:35, Mike Rapoport wrote:
> On Thu, Sep 30, 2021 at 04:41:17PM +0200, David Hildenbrand wrote:
>> In commit e83a437faa62 ("mm/memory_hotplug: introduce "auto-movable" online
>> policy") we introduced a new memory online policy to automatically
>> select a zone for memory blocks to be onlined. We added a way to
>> set the active online policy and tunables for the auto-movable online
>> policy. In follow-up commits we tweaked the "auto-movable" policy to also
>> consider memory device details when selecting zones for memory blocks to
>> be onlined.
>>
>> Let's document the new toggles and how the two online policies we have
>> work.
>>
>> Signed-off-by: David Hildenbrand <david@...hat.com>
>> ---
>>   .../admin-guide/mm/memory-hotplug.rst         | 128 +++++++++++++++---
>>   1 file changed, 108 insertions(+), 20 deletions(-)
>>
>> diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst b/Documentation/admin-guide/mm/memory-hotplug.rst
>> index ee00b70dedde..c20a2c0031cf 100644
>> --- a/Documentation/admin-guide/mm/memory-hotplug.rst
>> +++ b/Documentation/admin-guide/mm/memory-hotplug.rst
>> @@ -165,9 +165,8 @@ Or alternatively::
>>   
>>   	% echo 1 > /sys/devices/system/memory/memoryXXX/online
>>   
>> -The kernel will select the target zone automatically, usually defaulting to
>> -``ZONE_NORMAL`` unless ``movable_node`` has been specified on the kernel
>> -command line or if the memory block would intersect the ZONE_MOVABLE already.
>> +The kernel will select the target zone automatically, depending on the
>> +configured ``online_policy``.
>>   
>>   One can explicitly request to associate an offline memory block with
>>   ZONE_MOVABLE by::
>> @@ -198,6 +197,9 @@ Auto-onlining can be enabled by writing ``online``, ``online_kernel`` or
>>   
>>   	% echo online > /sys/devices/system/memory/auto_online_blocks
>>   
>> +Similarly to manual onlining, with ``online`` the kernel will select the
>> +target zone automatically, depending on the configured ``online_policy``.
>> +
>>   Modifying the auto-online behavior will only affect all subsequently added
>>   memory blocks only.
>>   
>> @@ -393,9 +395,11 @@ command line parameters are relevant:
>>   ======================== =======================================================
>>   ``memhp_default_state``	 configure auto-onlining by essentially setting
>>                            ``/sys/devices/system/memory/auto_online_blocks``.
>> -``movable_node``	 configure automatic zone selection in the kernel. When
>> -			 set, the kernel will default to ZONE_MOVABLE, unless
>> -			 other zones can be kept contiguous.
>> +``movable_node``	 configure automatic zone selection in the kernel when
>> +			 using the ``contig-zones`` online policy. When
>> +			 set, the kernel will default to ZONE_MOVABLE when
>> +			 onlining a memory block, unless other zones can be kept
>> +			 contiguous.
> 
> The movable_node main purpose is to allow unplugging an entire node. Zone
> selection is a consequence of this. You may want to cite the description of
> movable_node in kernel-paramenters.txt here.

Right, I only document the effects of these parameters on memory 
hot(un)plug.

What about:

diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst 
b/Documentation/admin-guide/mm/memory-hotplug.rst
index c20a2c0031cf..f8976ded0863 100644
--- a/Documentation/admin-guide/mm/memory-hotplug.rst
+++ b/Documentation/admin-guide/mm/memory-hotplug.rst
@@ -402,6 +402,9 @@ command line parameters are relevant:
                          contiguous.
  ======================== 
=======================================================

+See Documentation/admin-guide/kernel-parameters.txt for a more generic
+description of these command line parameters.
+
  Module Parameters
  ------------------


> 
> And, pardon my ignorance, how movable_node will play with auto-movable
> policy?

It's essentially ignored with the auto-movable policy for memory 
hotplugged after boot (!MEMBLOCK_HOTPLUG). That's why only the 
description of "contig-zones" below describes how it interacts with the 
``movable_node``, and we make it clear here that it's restricted to the 
"contig-zones" policy as well.

<details>
Bare metal, where we care about reliably unplugging hotplugged memory 
usually configures auto-onlining to "online_movable": for example, 
that's the case on RHEL. auto-movable doesn't make too much sense for 
bare metal: the nature of "movable_node" is to essentially online 
anything that might get hotunplugged MOVABLE, especially after 
hotplugging memory and rebooting: that is highly dangerous especially in 
virtualized environments.

"auto-movable" is valuable in virtualized environments, where we add 
memory via:
* add_memory_driver_managed() like virtio-mem, whereby such memory is
   never part of the firmware provided memory-map, so the policy is
   always in control even after a reboot.
* Hotplugged virtual DIMMs, such as provided by x86-64/arm64, whereby we
   don't include these DIMMs in the firmware-provided memory map, but
   ACPI code adds them after early boot, making it behave similar to
   add_memory_driver_managed() -- the policy is always in control even
   after a reboot.
</details>


Thanks!

-- 
Thanks,

David / dhildenb

Powered by blists - more mailing lists