Message-Id: <20260112112728.94590-1-lizhe.67@bytedance.com>
Date: Mon, 12 Jan 2026 19:27:26 +0800
From: "Li Zhe" <lizhe.67@...edance.com>
To: <muchun.song@...ux.dev>
Cc: <akpm@...ux-foundation.org>, <david@...nel.org>, <fvdl@...gle.com>, 
	<linux-kernel@...r.kernel.org>, <linux-mm@...ck.org>, 
	<lizhe.67@...edance.com>, <osalvador@...e.de>, <mjguzik@...il.com>, 
	<mhocko@...e.com>, <joao.m.martins@...cle.com>, 
	<ankur.a.arora@...cle.com>, <raghavendra.kt@....com>
Subject: Re: [PATCH v2 0/8] Introduce a huge-page pre-zeroing mechanism

On Fri, 9 Jan 2026 14:05:01 +0800, muchun.song@...ux.dev wrote:

> > On Jan 7, 2026, at 19:31, Li Zhe <lizhe.67@...edance.com> wrote:
> > 
> > This patchset is based on this commit[1]("mm/hugetlb: optionally
> > pre-zero hugetlb pages").
> 
> I’d like you to add a brief summary here that roughly explains
> what concerns the previous attempts raised and whether the
> current proposal has already addressed those concerns, so more
> people can quickly grasp the context.

In my opinion, the main concerns raised in the preceding discussion[1]
may be summarized as follows:

(1): The CPU cost of background zeroing is not attributable to the
task that consumes the pages, breaking fairness and cgroup accounting.

(2) Policy (when, how many threads) is hard-coded in the kernel. User
space lacks adequate means of control.

(3) Comparable functionality is already available in user space. (QEMU
support parallel preallocation)

(4) Faster zeroing method is provied in kernel[2].

In my view, these concerns have already been addressed by this patchset.

It merely supplies the tools and leaves all policy decisions to user
space; the kernel just performs the zeroing on behalf of the user,
thereby resolving concerns (1) and (2).

Regarding concern (3), I am aware that QEMU has implemented a parallel
page-touch mechanism, which does reduce VM creation time; nevertheless,
in our measurements it still consumes a non-trivial amount of time.
(According to feedback from QEMU colleagues, bringing up a 2 TB VM
still requires more than 40 seconds for zeroing)

> > Fresh hugetlb pages are zeroed out when they are faulted in,
> > just like with all other page types. This can take up a good
> > amount of time for larger page sizes (e.g. around 250
> > milliseconds for a 1G page on a Skylake machine).
> > 
> > This normally isn't a problem, since hugetlb pages are typically
> > mapped by the application for a long time, and the initial
> > delay when touching them isn't much of an issue.
> > 
> > However, there are some use cases where a large number of hugetlb
> > pages are touched when an application starts (such as a VM backed
> > by these pages), rendering the launch noticeably slow.
> > 
> > On an Skylake platform running v6.19-rc2, faulting in 64 × 1 GB huge
> > pages takes about 16 seconds, roughly 250 ms per page. Even with
> > Ankur’s optimizations[2], the time drops only to ~13 seconds,
> > ~200 ms per page, still a noticeable delay.

As for concern (4), I believe it is orthogonal to this patchset, and
the cover letter already contains a performance comparison that
demonstrates the additional benefit.

> I did see some comments in [1] about QEMU supporting user-mode
> parallel zero-page operations; I’m just not sure what the current
> state of that support looks like, or what the corresponding benchmark
> numbers are.

As noted above, QEMU already employs a parallel page-touch mechanism,
yet the elapsed time remains noticeable. I am not deeply familiar with
QEMU; please correct me if I am mistaken.

> > To accelerate the above scenario, this patchset exports a per-node,
> > read-write "zeroable_hugepages" sysfs interface for every hugepage size.
> > This interface reports how many hugepages on that node can currently
> > be pre-zeroed and allows user space to request that any integer number
> > in the range [0, max] be zeroed in a single operation.
> > 
> > This mechanism offers the following advantages:
> > 
> > (1) User space gains full control over when zeroing is triggered,
> > enabling it to minimize the impact on both CPU and cache utilization.
> > 
> > (2) Applications can spawn as many zeroing processes as they need,
> > enabling concurrent background zeroing.
> > 
> > (3) By binding the process to specific CPUs, users can confine zeroing
> > threads to cores that do not run latency-critical tasks, eliminating
> > interference.
> > 
> > (4) A zeroing process can be interrupted at any time through standard
> > signal mechanisms, allowing immediate cancellation.
> > 
> > (5) The CPU consumption incurred by zeroing can be throttled and contained
> > with cgroups, ensuring that the cost is not borne system-wide.
> > 
> > Tested on the same Skylake platform as above, when the 64 GiB of memory
> > was pre-zeroed in advance by the pre-zeroing mechanism, the faulting
> > latency test completed in negligible time.

[1]: https://lore.kernel.org/linux-mm/202412030519.W14yll4e-lkp@intel.com/T/#t
[2]: https://lore.kernel.org/all/20251215204922.475324-1-ankur.a.arora@oracle.com/T/#u

Thanks,
Zhe

Powered by blists - more mailing lists