linux-kernel - Re: [PATCH] mm: thp: Deny THP for guest_memfd and secretmem in file_thp

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAEvNRgH6yfFtOK3E3__hCjkNTQr55y-_N=SpBBr0i9_q6E2_8g@mail.gmail.com>
Date: Wed, 11 Feb 2026 07:38:46 -0800
From: Ackerley Tng <ackerleytng@...gle.com>
To: "David Hildenbrand (Arm)" <david@...nel.org>, Deepanshu Kartikey <kartikey406@...il.com>
Cc: akpm@...ux-foundation.org, lorenzo.stoakes@...cle.com, 
	baolin.wang@...ux.alibaba.com, Liam.Howlett@...cle.com, npache@...hat.com, 
	ryan.roberts@....com, dev.jain@....com, baohua@...nel.org, seanjc@...gle.com, 
	pbonzini@...hat.com, michael.roth@....com, vannapurve@...gle.com, 
	ziy@...dia.com, linux-mm@...ck.org, linux-kernel@...r.kernel.org, 
	syzbot+33a04338019ac7e43a44@...kaller.appspotmail.com, 
	Fangrui Song <i@...kray.me>
Subject: Re: [PATCH] mm: thp: Deny THP for guest_memfd and secretmem in file_thp_enabled()

"David Hildenbrand (Arm)" <david@...nel.org> writes:

> On 2/11/26 00:00, Ackerley Tng wrote:
>> "David Hildenbrand (Arm)" <david@...nel.org> writes:
>>
>>>>
>>>> I could give this a shot. 5.15.199 doesn't have AS_INACCESSIBLE. Should
>>>> we backport AS_INACCESSIBLE there or could the fix for 5.15.199 just be
>>>> special-casing secretmem like you suggested below?
>>>
>>> Yes. If there is no guest_memfd we wouldn't need it.
>>>
>>
>> Seems like on 5.15.199 there's a hugepage_vma_check(), which will return
>> false since secretmem has vma->vm_ops defined [1], so secretmem VMAs are
>> skipped.
>
> Are you sure? We check for CONFIG_READ_ONLY_THP_FOR_FS before that:
>

Ah... I was working on a reproducer then I realized 5.15 doesn't have
MADV_COLLAPSE, then I tried to hack in an ioctl to trigger
khugepaged. That turned out to be awkward but it got me to look at
hugepage_vma_check(), and then I went down the rabbit hole to keep
looking for the similar check function throughout the other stable
kernels... and amongst all of that forgot that
CONFIG_READ_ONLY_THP_FOR_FS was unset :(

You're probably right about VM_EXEC.

Here's the reproducer for 6.12, I put this in
tools/testing/selftests/mm/memfd_secret.c and called repro() from
main(). This time I enabled CONFIG_READ_ONLY_THP_FOR_FS :).

void repro(void)
{
	uint8_t *mem;
	int ret;
	int fd;
	int i;

	printf("%d triggering secretmem\n", __LINE__);

	fd = memfd_secret(0);
	if (fd < 0) {
		if (errno == ENOSYS)
			ksft_exit_skip("memfd_secret is not supported\n");
		else
			ksft_exit_fail_msg("memfd_secret failed: %s\n",
					   strerror(errno));
	}

	if (ftruncate(fd, SZ_2M))
		ksft_exit_fail_msg("ftruncate failed: %s\n", strerror(errno));

#define ALIGNED_ADDRESS ((void*)0x400000000UL)

	mem = mmap(ALIGNED_ADDRESS, SZ_2M, PROT_READ | PROT_WRITE, MAP_FIXED
| MAP_SHARED, fd, 0);
	if (mem != ALIGNED_ADDRESS)
		ksft_exit_fail_msg("Couldn't allocate memory\n");

	ret = madvise(mem, SZ_2M, MADV_HUGEPAGE);
	if (ret)
		ksft_exit_fail_msg("MADV_HUGEPAGE failed mem=%p ret=%d errno=%d\n",
mem, ret, errno);

#define READ_ONCE(x) (*(volatile typeof(x) *) &(x))
	for (i = 0; i < SZ_2M; i += getpagesize())
		READ_ONCE(mem[i]);

	ret = madvise(mem, SZ_2M, MADV_COLLAPSE);
	if (ret)
		ksft_exit_fail_msg("MADV_COLLAPSE failed ret=%d errno=%d\n", ret, errno);

	munmap(mem, SZ_2M);
	close(fd);
}

This reproducer gets us to madvise_collapse() ->
hpage_collapse_scan_file() -> collapse_file(), and copy_mc_highpage()
fails because copy_mc_to_kernel() returns 4096.

memory_failure_queue() causes this to be printed on the console

[ 1068.322578] Memory failure: 0x106d96f: recovery action for clean
unevictable LRU page: Recovered

No crash :) Is a crash the requirement for a backport to stable kernels?

> 	/* Only regular file is valid */
> 	if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && vma->vm_file &&
> 	    (vm_flags & VM_EXEC)) {
> 		struct inode *inode = vma->vm_file->f_inode;
>
> 		return !inode_is_open_for_write(inode) &&
> 			S_ISREG(inode->i_mode);
> 	}
>
>
> So if you have VM_EXEC on the VMA (mmaped with PROT_EXEC), it would work.
> I think secretmem sets SB_I_NOEXEC, which prevents that. Same for guest_memfd.
>
> v6.6.123 still has that VM_EXEC check in file_thp_enabled().
>
> The check was dropped in commit:
>
> commit 7fbb5e188248c50f737720825da1864ce42536d1
> Author: Fangrui Song <i@...kray.me>
> Date:   Tue Dec 19 21:41:23 2023 -0800
>
>      mm: remove VM_EXEC requirement for THP eligibility
>
>      Commit e6be37b2e7bd ("mm/huge_memory.c: add missing read-only THP checking
>      in transparent_hugepage_enabled()") introduced the VM_EXEC requirement,
>      which is not strictly needed.
>
>      lld's default --rosegment option and GNU ld's -z separate-code option
>      (default on Linux/x86 since binutils 2.31) create a read-only PT_LOAD
>      segment without the PF_X flag, which should be eligible for THP.
>
>
> So that one broke secretmem.
>
>
> So when we fix it, we should
>
> Fixes: 7fbb5e188248 ("mm: remove VM_EXEC requirement for THP eligibility")
>
>
> What about the following:
>
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index 44ff8a648afd..9fbe5c28a6bc 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -94,6 +94,9 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma)
>
>          inode = file_inode(vma->vm_file);
>
> +       if (IS_ANON_FILE(inode))
> +               return false;
> +
>          return !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode);
>   }
>
>
>
> --
> Cheers,
>
> David