linux-kernel - Re: [PATCH] mm/hugetlb: two-phase hugepage allocation when reservation is high

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <f8940d8d-0cb8-4b10-854d-6097d1a2c39c@redhat.com>
Date: Tue, 26 Aug 2025 15:25:17 +0200
From: David Hildenbrand <david@...hat.com>
To: lirongqing <lirongqing@...du.com>, muchun.song@...ux.dev,
 osalvador@...e.de, akpm@...ux-foundation.org, linux-mm@...ck.org,
 linux-kernel@...r.kernel.org, giorgitchankvetadze1997@...il.com
Cc: Wenjie Xu <xuwenjie04@...du.com>
Subject: Re: [PATCH] mm/hugetlb: two-phase hugepage allocation when
 reservation is high

On 26.08.25 12:18, lirongqing wrote:
> From: Li RongQing <lirongqing@...du.com>
> 
> When the total reserved hugepages account for 95% or more of system RAM
> (common in cloud computing on physical servers), allocating them all in one
> go can lead to OOM or fail to allocate huge page during early boot.
> 
> The commit 91f386bf0772 ("hugetlb: batch freeing of vmemmap pages") can
> worsen peak memory pressure under these conditions by deferring page frees,
> exacerbating allocation failures. To prevent this, split the allocation
> into two equal batches whenever
> 	huge_reserved_pages >= totalram_pages() * 90 / 100.
> 
> This change does not alter the number of padata worker threads per batch;
> it merely introduces a second round of padata_do_multithreaded(). The added
> overhead of restarting the worker threads is minimal.
> 
> The result on a 256G memory machine as below:
> Before:
> [    4.350400] HugeTLB: allocation took 706ms with hugepage_allocation_threads=32
> [    4.351577] HugeTLB: allocating 128512 of page size 2.00 MiB failed.  Only allocated 128074 hugepages.
> [    4.355608] HugeTLB: registered 2.00 MiB page size, pre-allocated 128074 pages
> After:
> [    3.561088] HugeTLB: two-phase hugepage allocation is used
> [    4.280300] HugeTLB: allocation took 712ms with hugepage_allocation_threads=32
> [    4.281054] HugeTLB: registered 2.00 MiB page size, pre-allocated 128512 pages
> 
> Fixes: 91f386bf0772 ("hugetlb: batch freeing of vmemmap pages")
> 
> Co-developed-by: Wenjie Xu <xuwenjie04@...du.com>
> Signed-off-by: Wenjie Xu <xuwenjie04@...du.com>
> Signed-off-by: Li RongQing <lirongqing@...du.com>
> ---
> Diff with v1: add log if two-phase hugepage allocation is triggered
>                add the knod to control split ratio
> 
>   Documentation/admin-guide/mm/hugetlbpage.rst | 12 +++++++++
>   mm/hugetlb.c                                 | 39 ++++++++++++++++++++++++++--
>   2 files changed, 49 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/admin-guide/mm/hugetlbpage.rst b/Documentation/admin-guide/mm/hugetlbpage.rst
> index 67a9419..5cfb6e3 100644
> --- a/Documentation/admin-guide/mm/hugetlbpage.rst
> +++ b/Documentation/admin-guide/mm/hugetlbpage.rst
> @@ -156,6 +156,18 @@ hugepage_alloc_threads
>   		hugepage_alloc_threads=8
>   
>   	Note that this parameter only applies to non-gigantic huge pages.
> +
> +hugepage_split_ratio
> +    Controls the threshold for two-phase hugepage allocation.
> +    When the total number of reserved hugepages (huge_reserved_pages) exceeds
> +    (totalram_pages * hugepage_split_ratio / 100), the hugepage allocation process
> +    during boot is split into two batches.
> +
> +    Default value is 90, meaning the two-phase allocation is triggered when
> +    reserved hugepages exceed 90% of total system RAM.
> +    The value can be adjusted via the kernel command line parameter
> +    "hugepage_split_ratio=". Valid range is 1 to 99.

Can we just do something reasonable here and not introduce toggles where 
nobody knows how to really set a reasonable value?

This really sounds like something we should not be exporting to users.

Also, can't we fail lightly during the first attempt and dynamically 
decide if we should do a second pase?

-- 
Cheers

David / dhildenb