linux-kernel - Re: [PATCH 0/1] prctl: allow overriding system THP policy to always

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <913bdc9b-a3c2-401c-99d0-18799850db9e@redhat.com>
Date: Sun, 11 May 2025 10:15:53 +0200
From: David Hildenbrand <david@...hat.com>
To: Zi Yan <ziy@...dia.com>
Cc: Johannes Weiner <hannes@...xchg.org>, Yafang Shao <laoar.shao@...il.com>,
 Usama Arif <usamaarif642@...il.com>,
 Andrew Morton <akpm@...ux-foundation.org>, linux-mm@...ck.org,
 shakeel.butt@...ux.dev, riel@...riel.com, baolin.wang@...ux.alibaba.com,
 lorenzo.stoakes@...cle.com, Liam.Howlett@...cle.com, npache@...hat.com,
 ryan.roberts@....com, linux-kernel@...r.kernel.org, kernel-team@...a.com
Subject: Re: [PATCH 0/1] prctl: allow overriding system THP policy to always

On 10.05.25 01:34, Zi Yan wrote:
> On 9 May 2025, at 18:42, David Hildenbrand wrote:
> 
>>>>>> - madvise
>>>>>>     The sysadmin gently encourages the use of THP, but it is only
>>>>>> enabled when explicitly requested by the application.
>>>
>>> And this "user mode" or "manual mode", where applications self-manage
>>> which parts of userspace they want to enroll.
>>>
>>> Both madvise() and unprivileged prctl() should work here as well,
>>> IMO. There is no policy or security difference between them, it's just
>>> about granularity and usability.
>>>
>>>>>> - never
>>>>>>      The sysadmin discourages the use of THP, and "its use is only permitted
>>>>>> with explicit approval" .
>>>
>>> This one I don't quite agree with, and IMO conflicts with what David
>>> is saying as well.
>>
>> Yeah ... "never" does not mean "sometimes" in my reality :)
>>
>>>
>>>>> "never" so far means "no thps, no exceptions". We've had serious THP
>>>>> issues in the past, where our workaround until we sorted out the issue
>>>>> for affected customers was to force-disable THPs on that system during boot.
>>>>
>>>> Right, that reflects the current behavior. What we aim to enhance is
>>>> by adding the requirement that "its use is only permitted with
>>>> explicit approval."
>>>
>>> I think you're conflating a safety issue with a security issue.
>>>
>>> David is saying there can be cases where the kernel is broken, and
>>> "never" is a production escape hatch to disable the feature until a
>>> kernel upgrade for the fix is possible. In such a case, it doesn't
>>> make sense to override this decision based on any sort of workload
>>> policy, privileged or not.
>>>
>>> The way I understand you is that you want enrollment (and/or
>>> self-management) only for blessed applications. Because you don't
>>> generally trust workloads in the wild enough to switch the global
>>> default away from "never", given the semantics of always/madvise.
>>
>> Assuming "never" means "never" and "always" means "always" ( crazy, right? :) ), could be make use of "madvise" mode, which essentially means "VM_HUGEPAGE" takes control?
>>
>> We'd need
>>
>> a) A way to enable THP for a process. Changing the default/vma settings to VM_HUGEPAGE as discussed using a prctl could work.
>>
>> b) A way to ignore VM_HUGEPAGE for processes. Maybe the existing prctl to force-disable THPs could work?
> 
> This means process level control overrides VMA level control, which
> overrides global control, right?
> 
> Intuitively, it should be that VMA level control overrides process level
> control, which overrides global control, namely finer granularly control
> precedes coarse one. But some apps might not use VMA level control
> (e.g., madvise) carefully, we want to override that. Maybe ignoring VMA
> level control is what we want?

Let's take a step back:

Current behavior is

1) If anybody (global / process / VM) says "never" 
(never/PR_SET_THP_DISABLE/VM_NOHUGEPAGE), the behavior is "never".

2) In "madvise" mode, only "VM_HUGEPAGE" gets THP unless 
PR_SET_THP_DISABLE is set. So per-process overrides per-VMA.

3) In "always" mode, everything gets THP unless per-VMA (VM_NOHUGEPAGE) 
or per-process (PR_SET_THP_DISABLE) disables it.


Interestingly, PR_SET_THP_DISABLE used to mimic exactly what I proposed 
for the other direction (change default of VM_HUGEPAGE), except that it 
wouldn't modify already existing mappings. Worth looking at 
1860033237d4. Not sure if that commit was the right call, but it's the 
semantics we have today.

That commit notes:

"It should be noted, that the new implementation makes 
PR_SET_THP_DISABLE master override to any per-VMA setting, which was not 
the case previously."


Whatever we do, we have to be careful to not create more mess or 
inconsistency.

Especially, if anybody sets VM_NOHUGEPAGE or PR_SET_THP_DISABLE, we must 
not use THPs, ever.

-- 
Cheers,

David / dhildenb