linux-kernel - [PATCH v3 0/3] mm/hugetlb: fixes for PMD table sharing (incl. using mmu

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20251223205046.565162-1-david@kernel.org>
Date: Tue, 23 Dec 2025 21:50:43 +0100
From: "David Hildenbrand (Red Hat)" <david@...nel.org>
To: linux-kernel@...r.kernel.org
Cc: linux-arch@...r.kernel.org,
	linux-mm@...ck.org,
	"David Hildenbrand (Red Hat)" <david@...nel.org>,
	Will Deacon <will@...nel.org>,
	"Aneesh Kumar K.V" <aneesh.kumar@...nel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Nick Piggin <npiggin@...il.com>,
	Peter Zijlstra <peterz@...radead.org>,
	Arnd Bergmann <arnd@...db.de>,
	Muchun Song <muchun.song@...ux.dev>,
	Oscar Salvador <osalvador@...e.de>,
	"Liam R. Howlett" <Liam.Howlett@...cle.com>,
	Lorenzo Stoakes <lorenzo.stoakes@...cle.com>,
	Vlastimil Babka <vbabka@...e.cz>,
	Jann Horn <jannh@...gle.com>,
	Pedro Falcato <pfalcato@...e.de>,
	Rik van Riel <riel@...riel.com>,
	Harry Yoo <harry.yoo@...cle.com>,
	Laurence Oberman <loberman@...hat.com>,
	Prakash Sangappa <prakash.sangappa@...cle.com>,
	Nadav Amit <nadav.amit@...il.com>
Subject: [PATCH v3 0/3] mm/hugetlb: fixes for PMD table sharing (incl. using mmu_gather)

One functional fix, one performance regression fix, and two related
comment fixes.

I cleaned up my prototype I recently shared [1] for the performance fix,
deferring most of the cleanups I had in the prototype to a later point.
While doing that I identified the other things.

The goal of this patch set is to be backported to stable trees "fairly"
easily. At least patch #1 and #4.

Patch #1 fixes hugetlb_pmd_shared() not detecting any sharing
Patch #2 + #3 are simple comment fixes that patch #4 interacts with.
Patch #4 is a fix for the reported performance regression due to excessive
IPI broadcasts during fork()+exit().

The last patch is all about TLB flushes, IPIs and mmu_gather.
Read: complicated

I added as much comments + description that I possibly could, and I am
hoping for review from Jann.

There are plenty of cleanups in the future to be had + one reasonable
optimization on x86. But that's all out of scope for this series.

Compile tested on plenty of architectures.

Runtime tested, with a focus on fixing the performance regression using
the original reproducer [2] on x86.

[1] https://lore.kernel.org/all/8cab934d-4a56-44aa-b641-bfd7e23bd673@kernel.org/
[2] https://lore.kernel.org/all/8cab934d-4a56-44aa-b641-bfd7e23bd673@kernel.org/

--

v2 -> v3:
* Rebased to 6.19-rc2 and retested on x86
* Changes on last patch:
 * Introduce and use tlb_gather_mmu_vma() for properly setting up mmu_gather
   for hugetlb -- thanks to Harry for pointing me once again at the nasty
   hugetlb integration in mmu_gather
 * Move tlb_remove_huge_tlb_entry() after move_huge_pte()
 * For consistency, always call tlb_gather_mmu_vma() after
   flush_cache_range()
 * Don't pass mmu_gather to hugetlb_change_protection(), simply use
   a local one for now. (avoids messing with tlb_start_vma() /
   tlb_start_end())
 * Dropped Lorenzo's RB due to the changes

v1 -> v2:
* Picked RB's/ACK's, hopefully I didn't miss any
* Added the initialization of fully_unshared_tables in __tlb_gather_mmu()
  (Thanks Nadav!)
* Refined some comments based on Lorenzo's feedback.

Cc: Will Deacon <will@...nel.org>
Cc: "Aneesh Kumar K.V" <aneesh.kumar@...nel.org>
Cc: Andrew Morton <akpm@...ux-foundation.org>
Cc: Nick Piggin <npiggin@...il.com>
Cc: Peter Zijlstra <peterz@...radead.org>
Cc: Arnd Bergmann <arnd@...db.de>
Cc: Muchun Song <muchun.song@...ux.dev>
Cc: Oscar Salvador <osalvador@...e.de>
Cc: "Liam R. Howlett" <Liam.Howlett@...cle.com>
Cc: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>
Cc: Vlastimil Babka <vbabka@...e.cz>
Cc: Jann Horn <jannh@...gle.com>
Cc: Pedro Falcato <pfalcato@...e.de>
Cc: Rik van Riel <riel@...riel.com>
Cc: Harry Yoo <harry.yoo@...cle.com>
Cc: Uschakow, Stanislav" <suschako@...zon.de>
Cc: Laurence Oberman <loberman@...hat.com>
Cc: Prakash Sangappa <prakash.sangappa@...cle.com>
Cc: Nadav Amit <nadav.amit@...il.com>

David Hildenbrand (Red Hat) (3):
  mm/hugetlb: fix two comments related to huge_pmd_unshare()
  mm/rmap: fix two comments related to huge_pmd_unshare()
  mm/hugetlb: fix excessive IPI broadcasts when unsharing PMD tables
    using mmu_gather

 include/asm-generic/tlb.h |  77 +++++++++++++++++++++-
 include/linux/hugetlb.h   |  15 +++--
 include/linux/mm_types.h  |   1 +
 mm/hugetlb.c              | 131 +++++++++++++++++++++-----------------
 mm/mmu_gather.c           |  33 ++++++++++
 mm/rmap.c                 |  45 ++++++-------
 6 files changed, 212 insertions(+), 90 deletions(-)

-- 
2.52.0