linux-kernel - Re: vmemmap alloc failure in hot_add

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <ff7f3fd0-dfd9-25c8-ef01-8fe29d5af9f5@kernel.org>
Date:   Fri, 18 Jun 2021 14:49:22 -0700
From:   Nathan Chancellor <nathan@...nel.org>
To:     Michael Kelley <mikelley@...rosoft.com>,
        David Hildenbrand <david@...hat.com>
Cc:     KY Srinivasan <kys@...rosoft.com>,
        Haiyang Zhang <haiyangz@...rosoft.com>,
        Stephen Hemminger <sthemmin@...rosoft.com>,
        Wei Liu <wei.liu@...nel.org>, Dexuan Cui <decui@...rosoft.com>,
        "linux-hyperv@...r.kernel.org" <linux-hyperv@...r.kernel.org>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        Oscar Salvador <osalvador@...e.de>,
        Hillf Danton <hdanton@...a.com>
Subject: Re: vmemmap alloc failure in hot_add_req()

Hi Michael,

On 6/17/2021 5:16 PM, Michael Kelley wrote:
> From: David Hildenbrand <david@...hat.com> Sent: Thursday, June 17, 2021 1:43 AM
>>
>>> It does look like this kernel configuration has
>>> CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE=y.
>>
>> Okay, so then it's most likely really more of an issue with fragmented
>> physical memory -- which is suboptimal but not a show blocker in your setup.
>>
>> (there are still cases where memory onlining can fail, especially with
>> kasan running, but these are rather corner cases)
>>
>>>
>>>> If it's not getting onlined, you easily sport after hotplug e.g., via
>>>> "lsmem" that there are quite some offline memory blocks.
>>>>
>>>> Note that x86_64 code will fallback from populating huge pages to
>>>> populating base pages for the vmemmap; this can happen easily when under
>>>> memory pressure.
>>>
>>> Not sure if it is relevant or not but this warning can show up within a
>>> minute of startup without me doing anything in particular.
>>
>> I remember that Hyper-V will start with a certain (configured) boot VM
>> memory size and once the guest is up and running, use memory stats of
>> the guest to decide whether to add (hotplug) or remove (balloon inflate)
>> memory from the VM.
>>
>> So this could just be Hyper-V trying to apply its heuristics.
> 
> Nathan --
> 
> Could you clarify if your VM is running in the context of the Windows
> Subsystem for Linux (WSL) v2 feature in Windows 10?  Or are you
> running a "traditional" VM created using the Hyper-V Manager UI
> or Powershell?

This is a traditional VM created using the Hyper-V Manager.

> If the latter, how do you have the memory configuration set up?  In
> the UI, first you can specify the RAM allocated to the VM.  Then
> separately, you can enable the "Dynamic Memory" feature, in which
> case you also specify a "Minimum RAM" and "Maximum RAM".  It
> looks like you must have the "Dynamic Memory" feature enabled
> since the original stack trace includes the hot_add_req() function
> from the hv_balloon driver.

That is correct. I believe Dynamic Memory is the default setting so I 
just left that as it was. The startup memory for this virtual machine is 
2GB as it is a lightweight Arch Linux Xfce4 configuration and aside from 
occasionally compiling software, it will just be sitting there because 
it is mainly there for testing kernels.

> The Dynamic Memory feature is generally used only when you
> need to allow Hyper-V to manage the allocation of physical memory
> across multiple VMs.  Dynamic Memory is essentially Hyper-V's way of
> allowing memory overcommit.  If you don't need that capability,
> turning off Dynamic Memory and just specifying the amount of
> memory you want to assign to the VM is the best course of action.

Ack. My workstation was occasionally memory constrained so I figured 
relying on the Dynamic Memory feature would make sense. I upgraded the 
amount of RAM that I had today so I will probably just end up disabling 
the Dynamic Memory feature and allocating the amount of memory up front.

> With Dynamic Memory enabled, you may have encountered a
> situation where the memory needs of the VM grew very quickly,
> and Hyper-V balloon driver got into a situation where it needed
> to allocate memory in order to add memory, and it couldn't.  If
> you want to continue to use the Dynamic Memory feature, then
> you probably need to increase the initial amount of RAM assigned
> to the VM (the "RAM" setting in the Hyper-V Manager UI).

I will keep that in mind and see if I can find a good number.

Thanks for the reply!

Cheers,
Nathan