Message-ID: <CAA1CXcAP6bX2=g-S-zqPrAQq6kJbQVOvx2cYYrXSCCL4R8UPpQ@mail.gmail.com>
Date: Tue, 13 Jan 2026 14:58:24 -0700
From: Nico Pache <npache@...hat.com>
To: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>
Cc: linux-kernel@...r.kernel.org, linux-trace-kernel@...r.kernel.org, 
	linux-mm@...ck.org, linux-doc@...r.kernel.org, david@...hat.com, 
	ziy@...dia.com, baolin.wang@...ux.alibaba.com, Liam.Howlett@...cle.com, 
	ryan.roberts@....com, dev.jain@....com, corbet@....net, rostedt@...dmis.org, 
	mhiramat@...nel.org, mathieu.desnoyers@...icios.com, 
	akpm@...ux-foundation.org, baohua@...nel.org, willy@...radead.org, 
	peterx@...hat.com, wangkefeng.wang@...wei.com, usamaarif642@...il.com, 
	sunnanyong@...wei.com, vishal.moola@...il.com, 
	thomas.hellstrom@...ux.intel.com, yang@...amperecomputing.com, kas@...nel.org, 
	aarcange@...hat.com, raquini@...hat.com, anshuman.khandual@....com, 
	catalin.marinas@....com, tiwai@...e.de, will@...nel.org, 
	dave.hansen@...ux.intel.com, jack@...e.cz, cl@...two.org, jglisse@...gle.com, 
	surenb@...gle.com, zokeefe@...gle.com, hannes@...xchg.org, 
	rientjes@...gle.com, mhocko@...e.com, rdunlap@...radead.org, hughd@...gle.com, 
	richard.weiyang@...il.com, lance.yang@...ux.dev, vbabka@...e.cz, 
	rppt@...nel.org, jannh@...gle.com, pfalcato@...e.de
Subject: Re: [PATCH v13 mm-new 00/16] khugepaged: mTHP support

On Thu, Jan 8, 2026 at 10:20 AM Lorenzo Stoakes
<lorenzo.stoakes@...cle.com> wrote:
>
> (Sorry for multiple mails replying to same, lei/lore are broken again so my
> setup isn't working properly).
>
> I tried to fixup the conflicts here to run tests locally but there's too many
> and I messed it up.
>
> Could you please resend this series rebased on mm-unstable please?

Yes ofc! fixed all the conflicts yesterday, just finalizing my
testing. Should be out tomorrow or Thursday.

>
> Thanks, Lorenzo
>
> On Mon, Dec 01, 2025 at 10:46:11AM -0700, Nico Pache wrote:
> > The following series provides khugepaged with the capability to collapse
> > anonymous memory regions to mTHPs.
> >
> > To achieve this we generalize the khugepaged functions to no longer depend
> > on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual
> > pages that are occupied (!none/zero). After the PMD scan is done, we use
> > the bitmap to find the optimal mTHP sizes for the PMD range. The
> > restriction on max_ptes_none is removed during the scan, to make sure we
> > account for the whole PMD range in the bitmap. When no mTHP size is
> > enabled, the legacy behavior of khugepaged is maintained.
> >
> > We currently only support max_ptes_none values of 0 or HPAGE_PMD_NR - 1
> > (ie 511). If any other value is specified, the kernel will emit a warning
> > and no mTHP collapse will be attempted. If a mTHP collapse is attempted,
> > but contains swapped out, or shared pages, we don't perform the collapse.
> > It is now also possible to collapse to mTHPs without requiring the PMD THP
> > size to be enabled. These limitiations are to prevent collapse "creep"
> > behavior. This prevents constantly promoting mTHPs to the next available
> > size, which would occur because a collapse introduces more non-zero pages
> > that would satisfy the promotion condition on subsequent scans.
> >
> > Patch 1:     Refactor/rename hpage_collapse
> > Patch 2:     Refactoring to combine madvise_collapse and khugepaged
> > Patch 3-8:   Generalize khugepaged functions for arbitrary orders and
> >            introduce some helper functions
> > Patch 9:     skip collapsing mTHP to smaller orders
> > Patch 10-11: Add per-order mTHP statistics and tracepoints
> > Patch 12:    Introduce collapse_allowable_orders
> > Patch 13-15: Introduce bitmap and mTHP collapse support, fully enabled
> > Patch 16:    Documentation
> >
> > ---------
> >  Testing
> > ---------
> > - Built for x86_64, aarch64, ppc64le, and s390x
> > - selftests mm
> > - I created a test script that I used to push khugepaged to its limits
> >    while monitoring a number of stats and tracepoints. The code is
> >    available here[1] (Run in legacy mode for these changes and set mthp
> >    sizes to inherit)
> >    The summary from my testings was that there was no significant
> >    regression noticed through this test. In some cases my changes had
> >    better collapse latencies, and was able to scan more pages in the same
> >    amount of time/work, but for the most part the results were consistent.
> > - redis testing. I tested these changes along with my defer changes
> >   (see followup [2] post for more details). We've decided to get the mTHP
> >   changes merged first before attempting the defer series.
> > - some basic testing on 64k page size.
> > - lots of general use.
> >
> > V13 Changes:
> > - Lots of minor nits, cleanups, comments, and renames
> > - Bitmap function simplification and more helpers (Wei, Lorenzo)
> > - Max_ptes_none (0 or 511) restriction
> > - commit description expansion
> > - list all reachable enum values in mthp_collapse()
> > - Fix ppc64 compile error due to using HPAGE_PMD_ORDER (replace with
> >   ilog2(MAX_PTRS_PER_PTE))
> >
> > V12: https://lore.kernel.org/lkml/20251022183717.70829-1-npache@redhat.com/
> > V11: https://lore.kernel.org/lkml/20250912032810.197475-1-npache@redhat.com/
> > V10: https://lore.kernel.org/lkml/20250819134205.622806-1-npache@redhat.com/
> > V9 : https://lore.kernel.org/lkml/20250714003207.113275-1-npache@redhat.com/
> > V8 : https://lore.kernel.org/lkml/20250702055742.102808-1-npache@redhat.com/
> > V7 : https://lore.kernel.org/lkml/20250515032226.128900-1-npache@redhat.com/
> > V6 : https://lore.kernel.org/lkml/20250515030312.125567-1-npache@redhat.com/
> > V5 : https://lore.kernel.org/lkml/20250428181218.85925-1-npache@redhat.com/
> > V4 : https://lore.kernel.org/lkml/20250417000238.74567-1-npache@redhat.com/
> > V3 : https://lore.kernel.org/lkml/20250414220557.35388-1-npache@redhat.com/
> > V2 : https://lore.kernel.org/lkml/20250211003028.213461-1-npache@redhat.com/
> > V1 : https://lore.kernel.org/lkml/20250108233128.14484-1-npache@redhat.com/
> >
> > A big thanks to everyone that has reviewed, tested, and participated in
> > the development process. Its been a great experience working with all of
> > you on this endeavour.
> >
> > [1] - https://gitlab.com/npache/khugepaged_mthp_test
> > [2] - https://lore.kernel.org/lkml/20250515033857.132535-1-npache@redhat.com/
> >
> > Baolin Wang (1):
> >   khugepaged: run khugepaged for all orders
> >
> > Dev Jain (1):
> >   khugepaged: generalize alloc_charge_folio()
> >
> > Nico Pache (14):
> >   khugepaged: rename hpage_collapse_* to collapse_*
> >   introduce collapse_single_pmd to unify khugepaged and madvise_collapse
> >   khugepaged: generalize hugepage_vma_revalidate for mTHP support
> >   khugepaged: introduce is_mthp_order helper
> >   khugepaged: generalize __collapse_huge_page_* for mTHP support
> >   khugepaged: introduce collapse_max_ptes_none helper function
> >   khugepaged: generalize collapse_huge_page for mTHP collapse
> >   khugepaged: skip collapsing mTHP to smaller orders
> >   khugepaged: add per-order mTHP collapse failure statistics
> >   khugepaged: improve tracepoints for mTHP orders
> >   khugepaged: introduce collapse_allowable_orders helper function
> >   khugepaged: Introduce mTHP collapse support
> >   khugepaged: avoid unnecessary mTHP collapse attempts
> >   Documentation: mm: update the admin guide for mTHP collapse
> >
> >  Documentation/admin-guide/mm/transhuge.rst |  80 ++-
> >  include/linux/huge_mm.h                    |   5 +
> >  include/trace/events/huge_memory.h         |  34 +-
> >  mm/huge_memory.c                           |  11 +
> >  mm/khugepaged.c                            | 696 +++++++++++++++------
> >  mm/mremap.c                                |   2 +-
> >  6 files changed, 618 insertions(+), 210 deletions(-)
> >
> > --
> > 2.51.1
> >
>

Powered by blists - more mailing lists