Message-ID: <aRtTKpltYqEAbxHw@google.com>
Date: Mon, 17 Nov 2025 08:54:02 -0800
From: Sean Christopherson <seanjc@...gle.com>
To: Alexander Potapenko <glider@...gle.com>
Cc: syzbot <syzbot+5c566b850d6ab6f0427a@...kaller.appspotmail.com>, 
	kvm@...r.kernel.org, linux-kernel@...r.kernel.org, pbonzini@...hat.com, 
	syzkaller-bugs@...glegroups.com
Subject: Re: [syzbot] [kvm?] INFO: task hung in kvm_swap_active_memslots (2)

On Mon, Nov 17, 2025, Alexander Potapenko wrote:
> On Mon, Nov 17, 2025 at 11:44 AM syzbot
> <syzbot+5c566b850d6ab6f0427a@...kaller.appspotmail.com> wrote:
> >
> > Hello,
> >
> > syzbot found the following issue on:
> >
> > HEAD commit:    3a8660878839 Linux 6.18-rc1
> > git tree:       upstream
> > console output: https://syzkaller.appspot.com/x/log.txt?x=160a05e2580000
> > kernel config:  https://syzkaller.appspot.com/x/.config?x=e854293d7f44b5a5
> > dashboard link: https://syzkaller.appspot.com/bug?extid=5c566b850d6ab6f0427a
> > compiler:       gcc (Debian 12.2.0-14+deb12u1) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40
> >
> > Unfortunately, I don't have any reproducer for this issue yet.
> >
> > Downloadable assets:
> > disk image: https://storage.googleapis.com/syzbot-assets/87a66406ce1a/disk-3a866087.raw.xz
> > vmlinux: https://storage.googleapis.com/syzbot-assets/7c3300da5269/vmlinux-3a866087.xz
> > kernel image: https://storage.googleapis.com/syzbot-assets/b4fcefdaf57b/bzImage-3a866087.xz
> >
> > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > Reported-by: syzbot+5c566b850d6ab6f0427a@...kaller.appspotmail.com
> >
> > INFO: task syz.2.1185:11790 blocked for more than 143 seconds.
> >       Not tainted syzkaller #0
> > "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > task:syz.2.1185      state:D stack:25976 pid:11790 tgid:11789 ppid:5836   task_flags:0x400140 flags:0x00080002
> > Call Trace:
> >  <TASK>
> >  context_switch kernel/sched/core.c:5325 [inline]
> >  __schedule+0x1190/0x5de0 kernel/sched/core.c:6929
> >  __schedule_loop kernel/sched/core.c:7011 [inline]
> >  schedule+0xe7/0x3a0 kernel/sched/core.c:7026
> >  kvm_swap_active_memslots+0x2ea/0x7d0 virt/kvm/kvm_main.c:1642
> >  kvm_activate_memslot virt/kvm/kvm_main.c:1786 [inline]
> >  kvm_create_memslot virt/kvm/kvm_main.c:1852 [inline]
> >  kvm_set_memslot+0xd3b/0x1380 virt/kvm/kvm_main.c:1964
> >  kvm_set_memory_region+0xe53/0x1610 virt/kvm/kvm_main.c:2120
> >  kvm_set_internal_memslot+0x9f/0xe0 virt/kvm/kvm_main.c:2143
> >  __x86_set_memory_region+0x2f6/0x740 arch/x86/kvm/x86.c:13242
> >  kvm_alloc_apic_access_page+0xc5/0x140 arch/x86/kvm/lapic.c:2788
> >  vmx_vcpu_create+0x503/0xbd0 arch/x86/kvm/vmx/vmx.c:7599
> >  kvm_arch_vcpu_create+0x688/0xb20 arch/x86/kvm/x86.c:12706
> >  kvm_vm_ioctl_create_vcpu virt/kvm/kvm_main.c:4207 [inline]
> >  kvm_vm_ioctl+0xfec/0x3fd0 virt/kvm/kvm_main.c:5158
> >  vfs_ioctl fs/ioctl.c:51 [inline]
> >  __do_sys_ioctl fs/ioctl.c:597 [inline]
> >  __se_sys_ioctl fs/ioctl.c:583 [inline]
> >  __x64_sys_ioctl+0x18e/0x210 fs/ioctl.c:583
> >  do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
> >  do_syscall_64+0xcd/0xfa0 arch/x86/entry/syscall_64.c:94
> >  entry_SYSCALL_64_after_hwframe+0x77/0x7f
> > RIP: 0033:0x7f3c9978eec9
> > RSP: 002b:00007f3c9a676038 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
> > RAX: ffffffffffffffda RBX: 00007f3c999e5fa0 RCX: 00007f3c9978eec9
> > RDX: 0000000000000000 RSI: 000000000000ae41 RDI: 0000000000000003
> > RBP: 00007f3c99811f91 R08: 0000000000000000 R09: 0000000000000000
> > R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
> > R13: 00007f3c999e6038 R14: 00007f3c999e5fa0 R15: 00007ffda33577e8
> >  </TASK>
> >
> > Showing all locks held in the system:
> > 1 lock held by khungtaskd/31:
> >  #0: ffffffff8e3c42e0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire include/linux/rcupdate.h:331 [inline]
> >  #0: ffffffff8e3c42e0 (rcu_read_lock){....}-{1:3}, at: rcu_read_lock include/linux/rcupdate.h:867 [inline]
> >  #0: ffffffff8e3c42e0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x1c0 kernel/locking/lockdep.c:6775
> > 2 locks held by getty/8058:
> >  #0: ffff88803440d0a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x24/0x80 drivers/tty/tty_ldisc.c:243
> >  #1: ffffc9000e0cd2f0 (&ldata->atomic_read_lock){+.+.}-{4:4}, at: n_tty_read+0x41b/0x14f0 drivers/tty/n_tty.c:2222
> > 2 locks held by syz.2.1185/11790:
> >  #0: ffff888032d640a8 (&kvm->slots_lock){+.+.}-{4:4}, at: class_mutex_constructor include/linux/mutex.h:228 [inline]
> >  #0: ffff888032d640a8 (&kvm->slots_lock){+.+.}-{4:4}, at: kvm_alloc_apic_access_page+0x27/0x140 arch/x86/kvm/lapic.c:2782
> >  #1: ffff888032d64138 (&kvm->slots_arch_lock){+.+.}-{4:4}, at: kvm_set_memslot+0x34/0x1380 virt/kvm/kvm_main.c:1915
> 
> It's worth noting that in addition to upstream this bug has been
> reported by syzkaller on several other kernel branches.
> In each report, the task was shown to be holding the same pair of
> locks: &kvm->slots_arch_lock and &kvm->slots_lock.

Ya, though that's not terribly interesting because kvm_swap_active_memslots()
holds those locks, and the issue is specifically that kvm_swap_active_memslots()
is waiting on kvm->mn_active_invalidate_count to go to zero.

Paolo even called out this possibility in commit 52ac8b358b0c ("KVM: Block memslot
updates across range_start() and range_end()"):

 : Losing the rwsem fairness does theoretically allow MMU notifiers to
 : block install_new_memslots forever.  Note that mm/mmu_notifier.c's own
 : retry scheme in mmu_interval_read_begin also uses wait/wake_up
 : and is likewise not fair.

In every reproducer, the "VMM" process is either getting thrashed by reclaim, or
the process itself is generating a constant stream of mmu_notifier invalidations.

I don't see an easy, or even decent, solution for this.  Forcing new invalidations
to wait isn't really an option because in-flight invalidations may be sleepable
(and KVM has zero visibility into the the behavior of invalidator), while new
invalidations may not be sleepable.

And on the KVM side, bailing from kvm_activate_memslot() on a pending signal
isn't an option, because kvm_activate_memslot() must not fail.  Hmm, at least,
not without terminating the VM.  I guess maybe that's an option?  Add a timeout
(maybe with a module param) to the kvm_swap_active_memslots() loop, and then WARN
and kill the VM if the timeout is hit.

Powered by blists - more mailing lists