linux-kernel - Re: [PATCH] KVM: guest_memfd: Disable VMA merging with VM

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CADhLXY6-LwV+O9557w+J6N0yWcGm9PQGoZUyj5BZ+LPPDC+DDg@mail.gmail.com>
Date: Mon, 9 Feb 2026 09:10:54 +0530
From: Deepanshu Kartikey <kartikey406@...il.com>
To: Ackerley Tng <ackerleytng@...gle.com>
Cc: "David Hildenbrand (arm)" <david@...nel.org>, Sean Christopherson <seanjc@...gle.com>, 
	syzbot+33a04338019ac7e43a44@...kaller.appspotmail.com, kvm@...r.kernel.org, 
	linux-kernel@...r.kernel.org, pbonzini@...hat.com, 
	syzkaller-bugs@...glegroups.com, michael.roth@....com, vannapurve@...gle.com
Subject: Re: [PATCH] KVM: guest_memfd: Disable VMA merging with VM_DONTEXPAND

On Sun, Feb 8, 2026 at 11:04 PM Ackerley Tng <ackerleytng@...gle.com> wrote:
>

> Since this also affects secretmem, I think thp_vma_allowable_order() is
> the best place to intercept the collapsing flow for both secretmem and
> guest_memfd.
>
> Let me know if you have any ideas!
>

Hi David, Ackerley,

I have been looking into this bug and I think the root cause is in
file_thp_enabled(). When CONFIG_READ_ONLY_THP_FOR_FS is enabled,
guest_memfd and secretmem inodes pass the S_ISREG() and
!inode_is_open_for_write() checks, so file_thp_enabled() incorrectly
returns true. This allows khugepaged and MADV_COLLAPSE to create large
folios in the page cache.

I sent a patch that fixes this at the source by explicitly rejecting
GUEST_MEMFD_MAGIC and SECRETMEM_MAGIC in file_thp_enabled():

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 40cf59301c21..4f57c78b57dd 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -93,6 +93,9 @@ static inline bool file_thp_enabled(struct
vm_area_struct *vma)
  return false;

  inode = file_inode(vma->vm_file);
+ if (inode->i_sb->s_magic == GUEST_MEMFD_MAGIC ||
+     inode->i_sb->s_magic == SECRETMEM_MAGIC)
+ return false;

  return !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode);
 }

I have tested this and confirmed the warning no longer triggers. This
approach covers both guest_memfd and secretmem in one place without
needing separate VMA flag changes in each subsystem. I have sent the
patch.

Please have a look and let me know your thoughts.

Thanks,
Deepanshu