linux-kernel - Re: [PATCH v3] mm: mmap: map MAP_STACK to VM

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <0ec57b2e-0a1c-443f-89e1-608d5bccbf26@redhat.com>
Date: Wed, 7 May 2025 11:50:15 +0200
From: David Hildenbrand <david@...hat.com>
To: Ignacio.MorenoGonzalez@...a.com, lorenzo.stoakes@...cle.com
Cc: Liam.Howlett@...cle.com, akpm@...ux-foundation.org,
 yang@...amperecomputing.com, linux-mm@...ck.org,
 linux-kernel@...r.kernel.org, stable@...r.kernel.org
Subject: Re: [PATCH v3] mm: mmap: map MAP_STACK to VM_NOHUGEPAGE only if THP
 is enabled

On 07.05.25 11:11, Ignacio Moreno Gonzalez via B4 Relay wrote:
> From: Ignacio Moreno Gonzalez <Ignacio.MorenoGonzalez@...a.com>
> 
> commit c4608d1bf7c6 ("mm: mmap: map MAP_STACK to VM_NOHUGEPAGE") maps
> the mmap option MAP_STACK to VM_NOHUGEPAGE. This is also done if
> CONFIG_TRANSPARENT_HUGETABLES is not defined. But in that case, the
> VM_NOHUGEPAGE does not make sense.
> 
> Fixes: c4608d1bf7c6 ("mm: mmap: map MAP_STACK to VM_NOHUGEPAGE")
> Cc: stable@...r.kernel.org
> Reviewed-by: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>
> Reviewed-by: Yang Shi <yang@...amperecomputing.com>
> Reviewed-by: Liam R. Howlett <Liam.Howlett@...cle.com>
> Signed-off-by: Ignacio Moreno Gonzalez <Ignacio.MorenoGonzalez@...a.com>
> ---
> I discovered this issue when trying to use the tool CRIU to checkpoint
> and restore a container. Our running kernel is compiled without
> CONFIG_TRANSPARENT_HUGETABLES. CRIU parses the output of

I'm sure you mean CONFIG_TRANSPARENT_HUGEPAGE ?

> /proc/<pid>/smaps and saves the "nh" flag. When trying to restore the
> container, CRIU fails to restore the "nh" mappings, since madvise()
> MADV_NOHUGEPAGE always returns an error because
> CONFIG_TRANSPARENT_HUGETABLES is not defined.

This should go into the patch description; it explains how this is 
actually causing issues.

> 
> The mapping MAP_STACK -> VM_NOHUGEPAGE was introduced by commit
> c4608d1bf7c6 ("mm: mmap: map MAP_STACK to VM_NOHUGEPAGE") in order to
> fix a regression introduced by commit efa7df3e3bb5 ("mm: align larger
> anonymous mappings on THP boundaries"). The change introducing the
> regression (efa7df3e3bb5) was limited to THP kernels, but its fix
> (c4608d1bf7c6) is applied without checking if THP is set.
> 
> The mapping MAP_STACK -> VM_NOHUGEPAGE should only be applied if THP is
> enabled.
> ---
> Changes in v3:
> - Exclude non-stable patch (for huge_mm.h) from this series to avoid mixing stable and non-stable patches, as suggested by Andrew.
> - Extend description in cover letter.
> - Link to v2: https://lore.kernel.org/r/20250506-map-map_stack-to-vm_nohugepage-only-if-thp-is-enabled-v2-0-f11f0c794872@kuka.com
> 
> Changes in v2:
> - [Patch 1/2] Use '#ifdef' instead of '#if defined(...)'
> - [Patch 1/2] Add 'Fixes: c4608d1bf7c6...'
> - Create [Patch 2/2]
> 
> - Link to v1: https://lore.kernel.org/r/20250502-map-map_stack-to-vm_nohugepage-only-if-thp-is-enabled-v1-1-113cc634cd51@kuka.com
> ---
>   include/linux/mman.h | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/include/linux/mman.h b/include/linux/mman.h
> index bce214fece16b9af3791a2baaecd6063d0481938..f4c6346a8fcd29b08d43f7cd9158c3eddc3383e1 100644
> --- a/include/linux/mman.h
> +++ b/include/linux/mman.h
> @@ -155,7 +155,9 @@ calc_vm_flag_bits(struct file *file, unsigned long flags)
>   	return _calc_vm_trans(flags, MAP_GROWSDOWN,  VM_GROWSDOWN ) |
>   	       _calc_vm_trans(flags, MAP_LOCKED,     VM_LOCKED    ) |
>   	       _calc_vm_trans(flags, MAP_SYNC,	     VM_SYNC      ) |
> +#ifdef CONFIG_TRANSPARENT_HUGEPAGE
>   	       _calc_vm_trans(flags, MAP_STACK,	     VM_NOHUGEPAGE) |
> +#endif
>   	       arch_calc_vm_flag_bits(file, flags);
>   }

LGTM. I'm surprise we even have VM_NOHUGEPAGE on kernels ... without 
THP. I mean, as you say, even setting MADV_NOHUGEPAGE does not wrok.

-- 
Cheers,

David / dhildenb