linux-kernel - [PATCH v2 0/2] Map MAP_STACK to VM_NOHUGEPAGE only if THP is enabled

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <20250506-map-map_stack-to-vm_nohugepage-only-if-thp-is-enabled-v2-0-f11f0c794872@kuka.com>
Date: Tue, 06 May 2025 15:44:31 +0200
From: Ignacio Moreno Gonzalez via B4 Relay <devnull+Ignacio.MorenoGonzalez.kuka.com@...nel.org>
To: lorenzo.stoakes@...cle.com
Cc: Liam.Howlett@...cle.com, akpm@...ux-foundation.org, 
 yang@...amperecomputing.com, linux-mm@...ck.org, 
 linux-kernel@...r.kernel.org, 
 Ignacio Moreno Gonzalez <Ignacio.MorenoGonzalez@...a.com>, 
 stable@...r.kernel.org, Matthew Wilcox <willy@...radead.org>
Subject: [PATCH v2 0/2] Map MAP_STACK to VM_NOHUGEPAGE only if THP is
 enabled

... and make setting MADV_NOHUGEPAGE with madvise() into a no-op if THP
is not enabled.

I discovered this issue when trying to use the tool CRIU to checkpoint
and restore a container. Our running kernel is compiled without
CONFIG_TRANSPARENT_HUGETABLES. CRIU parses the output of
/proc/<pid>/smaps and saves the "nh" flag. When trying to restore the
container, CRIU fails to restore the "nh" mappings, since madvise()
MADV_NOHUGEPAGE always returns an error because
CONFIG_TRANSPARENT_HUGETABLES is not defined.

These patches:
- Avoid mapping MAP_STACK to VM_NOHUGEPAGE if !THP
- Avoid returning an error when calling madvise() with MADV_NOHUGEPAGE
  if !THP

Signed-off-by: Ignacio Moreno Gonzalez <Ignacio.MorenoGonzalez@...a.com>
---
Changes in v2:
- [Patch 1/2] Use '#ifdef' instead of '#if defined(...)'
- [Patch 1/2] Add 'Fixes: c4608d1bf7c6...'
- Create [Patch 2/2]

- Link to v1: https://lore.kernel.org/r/20250502-map-map_stack-to-vm_nohugepage-only-if-thp-is-enabled-v1-1-113cc634cd51@kuka.com

---
Ignacio Moreno Gonzalez (2):
      mm: mmap: map MAP_STACK to VM_NOHUGEPAGE only if THP is enabled
      mm: madvise: no-op for MADV_NOHUGEPAGE if THP is disabled

 include/linux/huge_mm.h | 6 ++++++
 include/linux/mman.h    | 2 ++
 2 files changed, 8 insertions(+)
---
base-commit: fc96b232f8e7c0a6c282f47726b2ff6a5fb341d2
change-id: 20250428-map-map_stack-to-vm_nohugepage-only-if-thp-is-enabled-ce40a1de095d

Best regards,
-- 
Ignacio Moreno Gonzalez <Ignacio.MorenoGonzalez@...a.com>