Message-ID: <97413239-2bdc-456f-8511-65fb9f1e301c@kernel.org>
Date: Wed, 14 Jan 2026 12:55:11 +0100
From: "David Hildenbrand (Red Hat)" <david@...nel.org>
To: Li Zhe <lizhe.67@...edance.com>
Cc: akpm@...ux-foundation.org, ankur.a.arora@...cle.com, fvdl@...gle.com,
 joao.m.martins@...cle.com, linux-kernel@...r.kernel.org, linux-mm@...ck.org,
 mhocko@...e.com, mjguzik@...il.com, muchun.song@...ux.dev,
 osalvador@...e.de, raghavendra.kt@....com
Subject: Re: [PATCH v2 0/8] Introduce a huge-page pre-zeroing mechanism

On 1/14/26 12:36, Li Zhe wrote:
> On Wed, 14 Jan 2026 11:41:48 +0100, david@...nel.org wrote:
>   
>> On 1/13/26 13:41, Li Zhe wrote:
>>> On Tue, 13 Jan 2026 11:15:29 +0100, david@...nel.org wrote:
>>>
>>>> On 1/13/26 07:37, Li Zhe wrote:
>>>>> On Mon, 12 Jan 2026 20:52:12 +0100, david@...nel.org wrote:
>>>>>
>>>>>>> As for concern (4), I believe it is orthogonal to this patchset, and
>>>>>>> the cover letter already contains a performance comparison that
>>>>>>> demonstrates the additional benefit.
>>>>>>>
>>>>>>>> I did see some comments in [1] about QEMU supporting user-mode
>>>>>>>> parallel zero-page operations; I'm just not sure what the current
>>>>>>>> state of that support looks like, or what the corresponding benchmark
>>>>>>>> numbers are.
>>>>>>>
>>>>>>> As noted above, QEMU already employs a parallel page-touch mechanism,
>>>>>>> yet the elapsed time remains noticeable. I am not deeply familiar with
>>>>>>> QEMU; please correct me if I am mistaken.
>>>>>>
>>>>>> I implemented some part of the parallel preallocation support in QEMU.
>>>>>>
>>>>>> With QEMU, you can specify the number of threads and even specify the
>>>>>> NUMA-placement of these threads. So you can pretty much fine-tune that
>>>>>> for an environment.
>>>>>>
>>>>>> You still pre-zero all hugetlb pages at VM startup time, just in
>>>>>> parallel though. So you pay some price at APP startup time.
>>>>>
>>>>> Hi David,
>>>>>
>>>>> Thank you for the comprehensive explanation.
>>>>>
>>>>> You are absolutely correct: QEMU's parallel preallocation is performed
>>>>> only during VM start-up. We submitted this patch series mainly
>>>>> because we observed that, even with the existing parallel mechanism,
>>>>> launching large-size VMs still incurs prohibitive delays. (Bringing up
>>>>> a 2 TB VM still requires more than 40 seconds for zeroing)
>>>>>
>>>>>> If you know that you will run such a VM (or something else) later, you
>>>>>> could pre-zero the memory from user space by using a hugetlb-backed file
>>>>>> and supplying that to QEMU as memory backend for the VM. Then, you can
>>>>>> start your VM without any pre-zeroing.
>>>>>>
>>>>>> I guess that approach should work universally. Of course, there are
>>>>>> limitations, as you would have to know how much memory an app needs, and
>>>>>> have a way to supply that memory in form of a file to that app.
>>>>>
>>>>> Regarding user-space pre-zeroing, I agree that it is feasible once the
>>>>> VM's memory footprint is known. We evaluated this approach internally;
>>>>> however, in production environments, it is almost impossible to predict
>>>>> the exact amount of memory a VM will require.
>>>>
>>>> Of course, you could preallocate to the expected maximum and then
>>>> truncate the file to the size you need :)
>>>
>>> The solution you described seems similar to delegating hugepage
>>> management to a userspace daemon. I haven't explored this approach
>>> before, but it appears quite complex. Beyond ensuring secure memory
>>> isolation between VMs, we would also need to handle scenarios where
>>> the management daemon or the QEMU process crashes, which implies
>>> implementing robust recovery and memory reclamation mechanisms.
>>
>> Yes, but I don't think that's particularly complicated. You have to
>> remove the backing file, yes.
>>
>>> Do
>>> you happen to have any documentation or references regarding
>>> userspace hugepage management that I could look into?
>>
>> Not really any documentation. I pretty much only know how QEMU+libvirt
>> ends up using it :)
>>
>>> Compared to
>>> the userspace approach, I wonder if implementing hugepage
>>> pre-zeroing directly within the kernel would be a simpler and more
>>> direct way to accelerate VM creation.
>>
>> I mean, yes. I don't particularly enjoy user-space having to poll for
>> pre-zeroing of pages ... it feels like an odd interface for something
>> that is supposed to be simple.
>>
>> I do understand the reasoning that "zeroing must be charged to
>> somebody", and that using a kthread is a bit suboptimal as well.
> 
> My previous explanation may have caused misunderstanding. This
> patchset merely exports an interface that allows users to initiate
> and halt page zeroing on demand; the CPU cost is borne by the user,
> and no kernel thread is introduced.

You said "I wonder if implementing hugepage pre-zeroing directly within 
the kernel would be a simpler and more direct way to accelerate VM 
creation".

And I agree. But to make that fly (no user space polling interface), I 
was wondering whether we could do it like "init_on_free" and let whoever 
frees a hugetlb folio just reinitialize it with 0.

No kernel thread, no user space thread involved.

-- 
Cheers

David

Powered by blists - more mailing lists