Message-Id: <20260113124147.48460-1-lizhe.67@bytedance.com>
Date: Tue, 13 Jan 2026 20:41:47 +0800
From: "Li Zhe" <lizhe.67@...edance.com>
To: <david@...nel.org>
Cc: <akpm@...ux-foundation.org>, <ankur.a.arora@...cle.com>, 
	<fvdl@...gle.com>, <joao.m.martins@...cle.com>, 
	<linux-kernel@...r.kernel.org>, <linux-mm@...ck.org>, 
	<lizhe.67@...edance.com>, <mhocko@...e.com>, <mjguzik@...il.com>, 
	<muchun.song@...ux.dev>, <osalvador@...e.de>, <raghavendra.kt@....com>
Subject: Re: [PATCH v2 0/8] Introduce a huge-page pre-zeroing mechanism

On Tue, 13 Jan 2026 11:15:29 +0100, david@...nel.org wrote:

> On 1/13/26 07:37, Li Zhe wrote:
> > On Mon, 12 Jan 2026 20:52:12 +0100, david@...nel.org wrote:
> > 
> >>> As for concern (4), I believe it is orthogonal to this patchset, and
> >>> the cover letter already contains a performance comparison that
> >>> demonstrates the additional benefit.
> >>>
> >>>> I did see some comments in [1] about QEMU supporting user-mode
> >>>> parallel zero-page operations; I'm just not sure what the current
> >>>> state of that support looks like, or what the corresponding benchmark
> >>>> numbers are.
> >>>
> >>> As noted above, QEMU already employs a parallel page-touch mechanism,
> >>> yet the elapsed time remains noticeable. I am not deeply familiar with
> >>> QEMU; please correct me if I am mistaken.
> >>
> >> I implemented some part of the parallel preallocation support in QEMU.
> >>
> >> With QEMU, you can specify the number of threads and even specify the
> >> NUMA-placement of these threads. So you can pretty much fine-tune that
> >> for an environment.
> >>
> >> You still pre-zero all hugetlb pages at VM startup time, just in
> >> parallel though. So you pay some price at APP startup time.
> > 
> > Hi David,
> > 
> > Thank you for the comprehensive explanation.
> > 
> > You are absolutely correct: QEMU's parallel preallocation is performed
> > only during VM start-up. We submitted this patch series mainly
> > because we observed that, even with the existing parallel mechanism,
> > launching large-size VMs still incurs prohibitive delays. (Bringing up
> > a 2 TB VM still requires more than 40 seconds for zeroing)
> > 
> >> If you know that you will run such a VM (or something else) later, you
> >> could pre-zero the memory from user space by using a hugetlb-backed file
> >> and supplying that to QEMU as memory backend for the VM. Then, you can
> >> start your VM without any pre-zeroing.
> >>
> >> I guess that approach should work universally. Of course, there are
> >> limitations, as you would have to know how much memory an app needs, and
> >> have a way to supply that memory in form of a file to that app.
> > 
> > Regarding user-space pre-zeroing, I agree that it is feasible once the
> > VM's memory footprint is known. We evaluated this approach internally;
> > however, in production environments, it is almost impossible to predict
> > the exact amount of memory a VM will require.
> 
> Of course, you could preallocate to the expected maximum and then 
> truncate the file to the size you need :)

The solution you described seems similar to delegating hugepage
management to a userspace daemon. I haven't explored this approach
before, but it appears quite complex. Beyond ensuring secure memory
isolation between VMs, we would also need to handle scenarios where
the management daemon or the QEMU process crashes, which implies
implementing robust recovery and memory reclamation mechanisms. Do
you happen to have any documentation or references regarding
userspace hugepage management that I could look into? Compared to
the userspace approach, I wonder if implementing hugepage
pre-zeroing directly within the kernel would be a simpler and more
direct way to accelerate VM creation.

Thanks,
Zhe

Powered by blists - more mailing lists