linux-kernel - Re: [PATCH POC] prctl: extend PR_SET_THP_DISABLE to optionally exclude VM

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <2655eea8-e598-4c26-a7dc-a8c6b494a68b@gmail.com>
Date: Fri, 25 Jul 2025 17:26:06 +0100
From: Usama Arif <usamaarif642@...il.com>
To: David Hildenbrand <david@...hat.com>, linux-kernel@...r.kernel.org
Cc: linux-mm@...ck.org, linux-fsdevel@...r.kernel.org,
 linux-doc@...r.kernel.org, Jonathan Corbet <corbet@....net>,
 Andrew Morton <akpm@...ux-foundation.org>,
 Lorenzo Stoakes <lorenzo.stoakes@...cle.com>, Zi Yan <ziy@...dia.com>,
 Baolin Wang <baolin.wang@...ux.alibaba.com>,
 "Liam R. Howlett" <Liam.Howlett@...cle.com>, Nico Pache <npache@...hat.com>,
 Ryan Roberts <ryan.roberts@....com>, Dev Jain <dev.jain@....com>,
 Barry Song <baohua@...nel.org>, Vlastimil Babka <vbabka@...e.cz>,
 Mike Rapoport <rppt@...nel.org>, Suren Baghdasaryan <surenb@...gle.com>,
 Michal Hocko <mhocko@...e.com>, SeongJae Park <sj@...nel.org>,
 Jann Horn <jannh@...gle.com>, Yafang Shao <laoar.shao@...il.com>,
 Matthew Wilcox <willy@...radead.org>, Johannes Weiner <hannes@...xchg.org>
Subject: Re: [PATCH POC] prctl: extend PR_SET_THP_DISABLE to optionally
 exclude VM_HUGEPAGE



On 25/07/2025 14:08, David Hildenbrand wrote:
> On 25.07.25 00:27, Usama Arif wrote:
>>
>>> Hi!
>>>
>>>>
>>>> Over here, with MMF_DISABLE_THP_EXCEPT_ADVISED, MADV_HUGEPAGE will succeed as vm_flags has
>>>> VM_HUGEPAGE set, but MADV_COLLAPSE will fail to give a hugepage (as VM_HUGEPAGE is not set
>>>> and MMF_DISABLE_THP_EXCEPT_ADVISED is set) which I feel might not be the right behaviour
>>>> as MADV_COLLAPSE is "advise" and the prctl flag is PR_THP_DISABLE_EXCEPT_ADVISED?
>>>
>>> THPs are disabled for these regions, so it's at least consistent with the "disable all", but ...
>>>
>>>>
>>>> This will be checked in multiple places in madvise_collapse: thp_vma_allowable_order,
>>>> hugepage_vma_revalidate which calls thp_vma_allowable_order and hpage_collapse_scan_pmd
>>>> which also ends up calling hugepage_vma_revalidate.
>>>>> A hacky way would be to save and overwrite vma->vm_flags with VM_HUGEPAGE at the start of madvise_collapse
>>>> if VM_NOHUGEPAGE is not set, and reset vma->vm_flags to its original value at the end of madvise_collapse
>>>> (Not something I am recommending, just throwing it out there).
>>>
>>> Gah.
>>>
>>>>
>>>> Another possibility is to pass the fact that you are in madvise_collapse to these functions
>>>> as an argument, this might look ugly, although maybe not as ugly as hugepage_vma_revalidate
>>>> already has collapse control arg, so just need to take care of thp_vma_allowable_orders.
>>>
>>> Likely this.
>>>
>>>>
>>>> Any preference or better suggestions?
>>>
>>> What you are asking for is not MMF_DISABLE_THP_EXCEPT_ADVISED as I planned it, but MMF_DISABLE_THP_EXCEPT_ADVISED_OR_MADV_COLLAPSE.
>>>
>>> Now, one could consider MADV_COLLAPSE an "advise". (I am not opposed to that change)
>>>
>>
>> lol yeah I always think of MADV_COLLAPSE as an extreme version of MADV_HUGE (more of a demand
>> than an advice :)), eventhough its not persistant.
>> Which is why I think might be unexpected if MADV_HUGE gives hugepages but MADV_COLLAPSE doesn't
>> (But could just be my opinion).
>>
>>> Indeed, the right way might be telling vma_thp_disabled() whether we are in collapse.
>>>
>>> Can you try implementing that on top of my patch to see how it looks?
>>>
>>
>> My reasoning is that a process that is running with system policy always but with
>> PR_THP_DISABLE_EXCEPT_ADVISED gets THPs in exactly the same behaviour as a process that is running
>> with system policy madvise. This will help us achieve (3) that you mentioned in the
>> commit message:
>> (3) Switch from THP=madvise to THP=always, but keep the old behavior
>>       (THP only when advised) for selected workloads.
>>
>>
>> I have written quite a few selftests now for prctl SET_THP_DISABLE, both with and without
>> PR_THP_DISABLE_EXCEPT_ADVISED set incorporating your feedback on it. I have all of them passing
>> with the below diff. The diff is slightly ugly, but very simple and hopefully acceptable. If it
>> looks good, I can send a series with everything. Probably make the below diff as a separate patch
>> on top of this patch as its mostly adding an extra arg to functions and would keep the review easier?
> 
> Yes, we should do it as a separate patch, makes our life easier, because that requires more work.
> 
> We require a cleanup first, the boolean parameter for __thp_vma_allowable_orders() is no good.
> 
> I just pushed something untested to my branch (slightly adjusted patch#1 + 2 more patches), can you have a look at that? (untested ... :) )
> 

Thanks for this!
I tested it and its good, have sent it for review.