linux-kernel - Re: [PATCH] mm: huge_memory: fix the check for allowed huge orders in shmem

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <2d175a55-84e3-489f-8c93-66bedaa859a6@linux.alibaba.com>
Date: Sun, 15 Jun 2025 19:14:20 +0800
From: Baolin Wang <baolin.wang@...ux.alibaba.com>
To: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>
Cc: akpm@...ux-foundation.org, david@...hat.com, ziy@...dia.com,
 Liam.Howlett@...cle.com, npache@...hat.com, ryan.roberts@....com,
 dev.jain@....com, baohua@...nel.org, linux-mm@...ck.org,
 linux-kernel@...r.kernel.org
Subject: Re: [PATCH] mm: huge_memory: fix the check for allowed huge orders in
 shmem



On 2025/6/13 19:16, Lorenzo Stoakes wrote:
> On Fri, Jun 13, 2025 at 05:12:19PM +0800, Baolin Wang wrote:
>> Shmem already supports mTHP, and shmem_allowable_huge_orders() will return
>> the huge orders allowed by shmem. However, there is no check against the
>> 'orders' parameter passed by __thp_vma_allowable_orders(), which can lead
>> to incorrect check results for __thp_vma_allowable_orders().
>>
>> For example, when a user wants to check if shmem supports PMD-sized THP
>> by thp_vma_allowable_order(), if shmem only enables 64K mTHP, the current
>> logic would cause thp_vma_allowable_order() to return true, implying that
>> shmem allows PMD-sized THP allocation, which it actually does not.
>>
>> I don't think this will cause a significant impact on users, and this will
>> only have some impact on the shmem THP collapse. That is to say, even though
>> the shmem sysfs setting does not enable the PMD-sized THP, the
>> thp_vma_allowable_order() still indicates that shmem allows PMD-sized collapse,
>> meaning it might successfully collapse into THP, or it might not (for example,
>> thp_vma_suitable_order() check failed in the collapse process). However, this
>> still does not align with the shmem sysfs configuration, fix it.
> 
> Can you explain why?
> 
> It's a bit painful to trace through the code paths, but why do you think only
> MADV_COLLAPSE will be impacted? Surely everywhere that checks this is?

For shmem, thp_vma_allowable_order() and its wrapper are only used in 
show_smap() and shmem collapse (which includes khugepaged and 
madvise_collapse()). For shmem collapse, as I mentioned, the impact 
might not be very significant. For show_smap(), since it will use the 
'THP_ORDERS_ALL', it will not affect the results of show_smap().

>> Fixes: 26c7d8413aaf ("mm: thp: support "THPeligible" semantics for mTHP with anonymous shmem")
>> Signed-off-by: Baolin Wang <baolin.wang@...ux.alibaba.com>
> 
> I can't see how this can be incorrect, as we really should be restricting
> ourselves to the orders requested.
> 
> So:
> 
> Reviewed-by: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>

Thanks.

> 
>> ---
>> Note: this general change is suitable to be split out as a bugfix patch
>> based on the discussions in the previous thread[1].
>>
>> [1] https://lore.kernel.org/all/86bf2dcd-4be9-4fd9-98cc-da55aea52be0@lucifer.local/
>> ---
>>   mm/huge_memory.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
>> index d3e66136e41a..a8cfa37cae72 100644
>> --- a/mm/huge_memory.c
>> +++ b/mm/huge_memory.c
>> @@ -166,7 +166,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
>>   	 * own flags.
>>   	 */
>>   	if (!in_pf && shmem_file(vma->vm_file))
>> -		return shmem_allowable_huge_orders(file_inode(vma->vm_file),
>> +		return orders & shmem_allowable_huge_orders(file_inode(vma->vm_file),
>>   						   vma, vma->vm_pgoff, 0,
> 
> I mean this seems correct to me, but what a massive oversight.
> 
> I wish we had a sensible way of testing this...

It might not be easy to write test cases because it requires dynamically 
toggling the mTHP sysfs setting for shmem. However, as khugepaged 
supports mTHP collapse in the future, we can try to add more tests.

>>   						   !enforce_sysfs
> This whole code path is entirely indicative of what a complete mess this whole
> thing is.
> 
> The fact shmem separately calls this function is just ugh. I'm talking myself
> into some mega refactoring here :)

Yes, Shmem has its own separate mTHP sysfs interfaces, with more complex 
logic :)