linux-kernel - Re: [PATCH] mm: mmap: map MAP_STACK to VM

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <8d859d39-e0d2-445c-a5a9-7f2bc99ddfb6@lucifer.local>
Date: Fri, 2 May 2025 15:24:49 +0100
From: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>
To: Matthew Wilcox <willy@...radead.org>
Cc: Ignacio.MorenoGonzalez@...a.com, Liam.Howlett@...cle.com,
        linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] mm: mmap: map MAP_STACK to VM_NOHUGEPAGE only if THP is
 enabled

On Fri, May 02, 2025 at 03:16:52PM +0100, Matthew Wilcox wrote:
> On Fri, May 02, 2025 at 02:12:16PM +0100, Lorenzo Stoakes wrote:
> > On Fri, May 02, 2025 at 02:03:48PM +0100, Matthew Wilcox wrote:
> > > On Fri, May 02, 2025 at 11:31:41AM +0200, Ignacio Moreno Gonzalez via B4 Relay wrote:
> > > > From: Ignacio Moreno Gonzalez <Ignacio.MorenoGonzalez@...a.com>
> > > >
> > > > commit c4608d1bf7c6 ("mm: mmap: map MAP_STACK to VM_NOHUGEPAGE") maps
> > > > the mmap option MAP_STACK to VM_NOHUGEPAGE. This is also done if
> > > > CONFIG_TRANSPARENT_HUGETABLES is not defined. But in that case, the
> > > > VM_NOHUGEPAGE does not make sense. For instance, when calling madvise()
> > > > with MADV_NOHUGEPAGE, an error is always returned.
> > >
> > > Isn't that the real problem though?
> >
> > Hmm, but wouldn't we want users who are trying to set MADV_[NO]HUGEPAGE to
> > be made aware that it isn't going to do anything?
>
> ... I thought the patch was clear.  Only setting NOHUGEPAGE becomes a
> no-op.  Setting HUGEPAGE remains EINVAL.

Sorry I"m an idiot, glossed over the 'if (advice == MADV_NOHUGEPAGE)'
bit...

>
> > And wouldn't changing this be a possibly 'breaking userspace' thing if
> > somebody somewhere relies on this?
>
> I don't see what userspace could rely on it returning EINVAL, since it
> won't on a kernel which has THP enabled.

I mean for the purposes of detecting THP being disabled, but I mean it's
far-fetched.

>
> > Also this will make this inconsistent with e.g. MADV_COLLAPSE also?
>
> Not sure how ...
>

Yup this was based on my misreading...

But I'm still not a fan of us setting the VM_NOHUGEPAGE flag like this when
it's meaningless and this is (probably) the only way you can do it on a
non-THP kernel.

It's user-visible in /proc/$pid/smaps (I mean, that's the whole problem
here right?) and impacts merging (though MAP_STACK you'd think it'd
probably not be a big deal for...)

Or ifdef the VM_[NO]HUGEPAGE on CONFIG_TRANSPARENT_HUGEPAGE?

So maybe, let's do both things?