[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <cjlxwzcx57oss5rpmbufywbnz7pha5vueu3vnythp4rvn6bcf2@m7yhkifpqsio>
Date: Mon, 4 Aug 2025 12:41:09 +0200
From: "Pankaj Raghav (Samsung)" <kernel@...kajraghav.com>
To: David Hildenbrand <david@...hat.com>
Cc: Suren Baghdasaryan <surenb@...gle.com>,
Ryan Roberts <ryan.roberts@....com>, Baolin Wang <baolin.wang@...ux.alibaba.com>,
Borislav Petkov <bp@...en8.de>, Ingo Molnar <mingo@...hat.com>,
"H . Peter Anvin" <hpa@...or.com>, Vlastimil Babka <vbabka@...e.cz>, Zi Yan <ziy@...dia.com>,
Mike Rapoport <rppt@...nel.org>, Dave Hansen <dave.hansen@...ux.intel.com>,
Michal Hocko <mhocko@...e.com>, Lorenzo Stoakes <lorenzo.stoakes@...cle.com>,
Andrew Morton <akpm@...ux-foundation.org>, Thomas Gleixner <tglx@...utronix.de>,
Nico Pache <npache@...hat.com>, Dev Jain <dev.jain@....com>,
"Liam R . Howlett" <Liam.Howlett@...cle.com>, Jens Axboe <axboe@...nel.dk>, linux-kernel@...r.kernel.org,
willy@...radead.org, linux-mm@...ck.org, x86@...nel.org, linux-block@...r.kernel.org,
linux-fsdevel@...r.kernel.org, "Darrick J . Wong" <djwong@...nel.org>, mcgrof@...nel.org,
gost.dev@...sung.com, hch@....de, Pankaj Raghav <p.raghav@...sung.com>
Subject: Re: [RFC v2 2/4] mm: add static huge zero folio
On Fri, Aug 01, 2025 at 05:49:10PM +0200, David Hildenbrand wrote:
> On 24.07.25 16:49, Pankaj Raghav (Samsung) wrote:
> > From: Pankaj Raghav <p.raghav@...sung.com>
> >
> > There are many places in the kernel where we need to zeroout larger
> > chunks but the maximum segment we can zeroout at a time by ZERO_PAGE
> > is limited by PAGE_SIZE.
> >
> > This is especially annoying in block devices and filesystems where we
> > attach multiple ZERO_PAGEs to the bio in different bvecs. With multipage
> > bvec support in block layer, it is much more efficient to send out
> > larger zero pages as a part of single bvec.
> >
> > This concern was raised during the review of adding LBS support to
> > XFS[1][2].
> >
> > Usually huge_zero_folio is allocated on demand, and it will be
> > deallocated by the shrinker if there are no users of it left. At moment,
> > huge_zero_folio infrastructure refcount is tied to the process lifetime
> > that created it. This might not work for bio layer as the completions
> > can be async and the process that created the huge_zero_folio might no
> > longer be alive. And, one of the main point that came during discussion
> > is to have something bigger than zero page as a drop-in replacement.
> >
> > Add a config option STATIC_HUGE_ZERO_FOLIO that will always allocate
>
> "... will result in allocating the huge zero folio on first request, if not already allocated, and turn it static such that it can never get freed."
Sounds good.
>
> > the huge_zero_folio, and it will never drop the reference. This makes
> > using the huge_zero_folio without having to pass any mm struct and does
> > not tie the lifetime of the zero folio to anything, making it a drop-in
> > replacement for ZERO_PAGE.
> >
> > If STATIC_HUGE_ZERO_FOLIO config option is enabled, then
> > mm_get_huge_zero_folio() will simply return this page instead of
> > dynamically allocating a new PMD page.
> >
> > This option can waste memory in small systems or systems with 64k base
> > page size. So make it an opt-in and also add an option from individual
> > architecture so that we don't enable this feature for larger base page
> > size systems.
> > > [1] https://lore.kernel.org/linux-xfs/20231027051847.GA7885@lst.de/
> > [2] https://lore.kernel.org/linux-xfs/ZitIK5OnR7ZNY0IG@infradead.org/
> >
> > Co-developed-by: David Hildenbrand <david@...hat.com>
> > Signed-off-by: David Hildenbrand <david@...hat.com>
> > Signed-off-by: Pankaj Raghav <p.raghav@...sung.com>
> > ---
> > arch/x86/Kconfig | 1 +
> > include/linux/huge_mm.h | 18 ++++++++++++++++++
> > mm/Kconfig | 21 +++++++++++++++++++++
> > mm/huge_memory.c | 42 +++++++++++++++++++++++++++++++++++++++++
> > 4 files changed, 82 insertions(+)
> >
> > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> > index 0ce86e14ab5e..8e2aa1887309 100644
> > --- a/arch/x86/Kconfig
> > +++ b/arch/x86/Kconfig
> > @@ -153,6 +153,7 @@ config X86
> > select ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP if X86_64
> > select ARCH_WANT_HUGETLB_VMEMMAP_PREINIT if X86_64
> > select ARCH_WANTS_THP_SWAP if X86_64
> > + select ARCH_WANTS_STATIC_HUGE_ZERO_FOLIO if X86_64
> > select ARCH_HAS_PARANOID_L1D_FLUSH
> > select ARCH_WANT_IRQS_OFF_ACTIVATE_MM
> > select BUILDTIME_TABLE_SORT
> > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> > index 7748489fde1b..78ebceb61d0e 100644
> > --- a/include/linux/huge_mm.h
> > +++ b/include/linux/huge_mm.h
> > @@ -476,6 +476,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf);
> > extern struct folio *huge_zero_folio;
> > extern unsigned long huge_zero_pfn;
> > +extern atomic_t huge_zero_folio_is_static;
> > static inline bool is_huge_zero_folio(const struct folio *folio)
> > {
> > @@ -494,6 +495,18 @@ static inline bool is_huge_zero_pmd(pmd_t pmd)
> > struct folio *mm_get_huge_zero_folio(struct mm_struct *mm);
> > void mm_put_huge_zero_folio(struct mm_struct *mm);
> > +struct folio *__get_static_huge_zero_folio(void);
> > +
> > +static inline struct folio *get_static_huge_zero_folio(void)
> > +{
> > + if (!IS_ENABLED(CONFIG_STATIC_HUGE_ZERO_FOLIO))
> > + return NULL;
> > +
> > + if (likely(atomic_read(&huge_zero_folio_is_static)))
> > + return huge_zero_folio;
> > +
> > + return __get_static_huge_zero_folio();
> > +}
> > static inline bool thp_migration_supported(void)
> > {
> > @@ -685,6 +698,11 @@ static inline int change_huge_pud(struct mmu_gather *tlb,
> > {
> > return 0;
> > }
> > +
> > +static inline struct folio *get_static_huge_zero_folio(void)
> > +{
> > + return NULL;
> > +}
> > #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
> > static inline int split_folio_to_list_to_order(struct folio *folio,
> > diff --git a/mm/Kconfig b/mm/Kconfig
> > index 0287e8d94aea..e2132fcf2ccb 100644
> > --- a/mm/Kconfig
> > +++ b/mm/Kconfig
> > @@ -835,6 +835,27 @@ config ARCH_WANT_GENERAL_HUGETLB
> > config ARCH_WANTS_THP_SWAP
> > def_bool n
> > +config ARCH_WANTS_STATIC_HUGE_ZERO_FOLIO
> > + def_bool n
> > +
> > +config STATIC_HUGE_ZERO_FOLIO
> > + bool "Allocate a PMD sized folio for zeroing"
> > + depends on ARCH_WANTS_STATIC_HUGE_ZERO_FOLIO && TRANSPARENT_HUGEPAGE
> > + help
> > + Without this config enabled, the huge zero folio is allocated on
> > + demand and freed under memory pressure once no longer in use.
> > + To detect remaining users reliably, references to the huge zero folio
> > + must be tracked precisely, so it is commonly only available for mapping
> > + it into user page tables.
> > +
> > + With this config enabled, the huge zero folio can also be used
> > + for other purposes that do not implement precise reference counting:
> > + it is still allocated on demand, but never freed, allowing for more
> > + wide-spread use, for example, when performing I/O similar to the
> > + traditional shared zeropage.
> > +
> > + Not suitable for memory constrained systems.
> > +
> > config MM_ID
> > def_bool n
> > diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> > index 5d8365d1d3e9..c160c37f4d31 100644
> > --- a/mm/huge_memory.c
> > +++ b/mm/huge_memory.c
> > @@ -75,6 +75,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
> > static bool split_underused_thp = true;
> > static atomic_t huge_zero_refcount;
> > +atomic_t huge_zero_folio_is_static __read_mostly;
> > struct folio *huge_zero_folio __read_mostly;
> > unsigned long huge_zero_pfn __read_mostly = ~0UL;
> > unsigned long huge_anon_orders_always __read_mostly;
> > @@ -266,6 +267,47 @@ void mm_put_huge_zero_folio(struct mm_struct *mm)
> > put_huge_zero_page();
> > }
> > +#ifdef CONFIG_STATIC_HUGE_ZERO_FOLIO
> > +#define FAIL_COUNT_LIMIT 2
> > +
> > +struct folio *__get_static_huge_zero_folio(void)
> > +{
> > + static unsigned long fail_count_clear_timer;
> > + static atomic_t huge_zero_static_fail_count __read_mostly;
> > +
> > + if (unlikely(!slab_is_available()))
> > + return NULL;
> > +
> > + /*
> > + * If we failed to allocate a huge zero folio multiple times,
> > + * just refrain from trying for one minute before retrying to get
> > + * a reference again.
> > + */
>
> Is this "try twice" really worth it? Just try once, and if it fails, try only again in the future.
>
Yeah, that makes sense. Let's go with try it once for now.
> I guess we'll learn how that will behave in practice, and how we'll have to fine-tune it :)
>
>
> In shrink_huge_zero_page_scan(), should we probably warn if something buggy happens?
Yeah, I can fold this in the next version. I guess WARN_ON_ONCE already
adds an unlikely to the conditition which is appropriate.
>
> Something like
>
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index 2b4ea5a2ce7d2..b1109f8699a24 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -277,7 +277,11 @@ static unsigned long shrink_huge_zero_page_scan(struct shrinker *shrink,
> struct shrink_control *sc)
> {
> if (atomic_cmpxchg(&huge_zero_refcount, 1, 0) == 1) {
> - struct folio *zero_folio = xchg(&huge_zero_folio, NULL);
> + struct folio *zero_folio;
> +
> + if (WARN_ON_ONCE(atomic_read(&huge_zero_folio_is_static)))
> + return 0;
> + zero_folio = xchg(&huge_zero_folio, NULL);
> BUG_ON(zero_folio == NULL);
> WRITE_ONCE(huge_zero_pfn, ~0UL);
> folio_put(zero_folio);
>
>
> --
> Cheers,
>
--
Pankaj
Powered by blists - more mailing lists