linux-kernel - Re: [PATCH] mm: huge_memory: fix the check for allowed huge orders in shmem

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <c7e64411-1c07-401e-8503-928184ca22f6@lucifer.local>
Date: Fri, 13 Jun 2025 12:16:57 +0100
From: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>
To: Baolin Wang <baolin.wang@...ux.alibaba.com>
Cc: akpm@...ux-foundation.org, david@...hat.com, ziy@...dia.com,
        Liam.Howlett@...cle.com, npache@...hat.com, ryan.roberts@....com,
        dev.jain@....com, baohua@...nel.org, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH] mm: huge_memory: fix the check for allowed huge orders
 in shmem

On Fri, Jun 13, 2025 at 05:12:19PM +0800, Baolin Wang wrote:
> Shmem already supports mTHP, and shmem_allowable_huge_orders() will return
> the huge orders allowed by shmem. However, there is no check against the
> 'orders' parameter passed by __thp_vma_allowable_orders(), which can lead
> to incorrect check results for __thp_vma_allowable_orders().
>
> For example, when a user wants to check if shmem supports PMD-sized THP
> by thp_vma_allowable_order(), if shmem only enables 64K mTHP, the current
> logic would cause thp_vma_allowable_order() to return true, implying that
> shmem allows PMD-sized THP allocation, which it actually does not.
>
> I don't think this will cause a significant impact on users, and this will
> only have some impact on the shmem THP collapse. That is to say, even though
> the shmem sysfs setting does not enable the PMD-sized THP, the
> thp_vma_allowable_order() still indicates that shmem allows PMD-sized collapse,
> meaning it might successfully collapse into THP, or it might not (for example,
> thp_vma_suitable_order() check failed in the collapse process). However, this
> still does not align with the shmem sysfs configuration, fix it.

Can you explain why?

It's a bit painful to trace through the code paths, but why do you think only
MADV_COLLAPSE will be impacted? Surely everywhere that checks this is?

>
> Fixes: 26c7d8413aaf ("mm: thp: support "THPeligible" semantics for mTHP with anonymous shmem")
> Signed-off-by: Baolin Wang <baolin.wang@...ux.alibaba.com>

I can't see how this can be incorrect, as we really should be restricting
ourselves to the orders requested.

So:

Reviewed-by: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>

> ---
> Note: this general change is suitable to be split out as a bugfix patch
> based on the discussions in the previous thread[1].
>
> [1] https://lore.kernel.org/all/86bf2dcd-4be9-4fd9-98cc-da55aea52be0@lucifer.local/
> ---
>  mm/huge_memory.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index d3e66136e41a..a8cfa37cae72 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -166,7 +166,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
>  	 * own flags.
>  	 */
>  	if (!in_pf && shmem_file(vma->vm_file))
> -		return shmem_allowable_huge_orders(file_inode(vma->vm_file),
> +		return orders & shmem_allowable_huge_orders(file_inode(vma->vm_file),
>  						   vma, vma->vm_pgoff, 0,

I mean this seems correct to me, but what a massive oversight.

I wish we had a sensible way of testing this...

>  						   !enforce_sysfs);

This whole code path is entirely indicative of what a complete mess this whole
thing is.

The fact shmem separately calls this function is just ugh. I'm talking myself
into some mega refactoring here :)

>
> --
> 2.43.5
>