| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <1f8dd706-0eb0-4e09-a0fe-fc48fb24005e@kzalloc.com>
Date: Mon, 11 Aug 2025 00:05:49 +0900
From: Yunseong Kim <ysk@...lloc.com>
To: linux-usb@...r.kernel.org
Cc: linux-rt-users@...r.kernel.org, gregkh@...uxfoundation.org,
stern@...land.harvard.edu, Thomas Gleixner <tglx@...utronix.de>,
Sebastian Andrzej Siewior <bigeasy@...utronix.de>,
Clark Williams <clrkwllms@...nel.org>, Steven Rostedt <rostedt@...dmis.org>,
linux-kernel@...r.kernel.org, syzkaller@...glegroups.com
Subject: [BUG] usb: sleepable spinlock used in USB bh_worker softirq on
PREEMPT_RT
While running a PREEMPT_RT-enabled kernel I observed a sleepable
spinlock (rt_spin_lock) being taken from a softirq context within
the USB core framework. On PREEMPT_RT, spin_lock() may sleep when
contended. This is unsafe in softirq context and can cause hangs or
deadlocks.
Below is the relevant call path, reformatted for clarity:
(1) softirq context: workqueue_softirq_action
--> bh_worker()
--> process_scheduled_works()
--> process_one_work()
--> usb_giveback_urb_bh()
--> __usb_hcd_giveback_urb()
--> hub_irq()
--> hub_resubmit_irq_urb()
--> spin_lock()
|
--> rt_spin_lock()
** <-- sleepable spinlock
used in softirq
context **
Stack trace excerpt:
| BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48
| in_atomic(): 0, irqs_disabled(): 1, non_block: 0, pid: 15, name: ksoftirqd/0
| preempt_count: 0, expected: 0
| RCU nest depth: 2, expected: 2
| CPU: 0 UID: 0 PID: 15 Comm: ksoftirqd/0 Not tainted 6.16.0-11245-gc87c2e4c1589 #35 PREEMPT_RT
| Hardware name: QEMU KVM Virtual Machine, BIOS 2025.02-8ubuntu1 06/11/2025
| Call trace:
| show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:499 (C)
| __dump_stack+0x30/0x40 lib/dump_stack.c:94
| dump_stack_lvl+0x148/0x1d8 lib/dump_stack.c:120
| dump_stack+0x1c/0x3c lib/dump_stack.c:129
| __might_resched+0x2e4/0x52c kernel/sched/core.c:8957
| __rt_spin_lock kernel/locking/spinlock_rt.c:48 [inline]
| rt_spin_lock+0xa8/0x1bc kernel/locking/spinlock_rt.c:57
| spin_lock include/linux/spinlock_rt.h:44 [inline]
| hub_resubmit_irq_urb+0x2c/0x168 drivers/usb/core/hub.c:687
| hub_irq+0x2ac/0x3d0 drivers/usb/core/hub.c:814
| __usb_hcd_giveback_urb+0x2f0/0x5e8 drivers/usb/core/hcd.c:1663
| usb_giveback_urb_bh+0x234/0x3c4 drivers/usb/core/hcd.c:1697
| process_one_work kernel/workqueue.c:3236 [inline]
| process_scheduled_works+0x678/0xd18 kernel/workqueue.c:3319
| bh_worker+0x2f0/0x59c kernel/workqueue.c:3579
| workqueue_softirq_action+0x104/0x14c kernel/workqueue.c:3606
| tasklet_hi_action+0x18/0x8c kernel/softirq.c:860
| handle_softirqs+0x208/0x63c kernel/softirq.c:579
| run_ksoftirqd+0x64/0x264 kernel/softirq.c:968
| smpboot_thread_fn+0x4ac/0x908 kernel/smpboot.c:160
| kthread+0x5e8/0x734 kernel/kthread.c:464
| ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:844
(2) softirq context: run_ksoftirqd()
--> handle_softirqs()
--> tasklet_action()
--> workqueue_softirq_action()
--> bh_worker()
--> process_scheduled_works()
--> process_one_work()
--> usb_giveback_urb_bh()
--> __usb_hcd_giveback_urb()
--> async_completed()
--> spin_lock()
|
--> rt_spin_lock()
** <-- BUG: sleeping function
called from invalid
context **
Stack trace excerpt:
| BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48
| in_atomic(): 0, irqs_disabled(): 1, non_block: 0, pid: 15, name: ksoftirqd/0
| preempt_count: 0, expected: 0
| RCU nest depth: 2, expected: 2
| CPU: 0 UID: 0 PID: 15 Comm: ksoftirqd/0 Not tainted 6.16.0-10393-g3267b1c07fff-dirty #43 PREEMPT_RT
| Hardware name: QEMU KVM Virtual Machine, BIOS 2025.02-8ubuntu1 06/11/2025
| Call trace:
| show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:499 (C)
| __dump_stack+0x30/0x40 lib/dump_stack.c:94
| dump_stack_lvl+0x148/0x1d8 lib/dump_stack.c:120
| dump_stack+0x1c/0x3c lib/dump_stack.c:129
| __might_resched+0x2e4/0x52c kernel/sched/core.c:8957
| __rt_spin_lock kernel/locking/spinlock_rt.c:48 [inline]
| rt_spin_lock+0xa8/0x1bc kernel/locking/spinlock_rt.c:57
| spin_lock include/linux/spinlock_rt.h:44 [inline]
| async_completed+0x70/0xabc drivers/usb/core/devio.c:633
| __usb_hcd_giveback_urb+0x2f0/0x5e8 drivers/usb/core/hcd.c:1663
| usb_giveback_urb_bh+0x234/0x3c4 drivers/usb/core/hcd.c:1697
| process_one_work kernel/workqueue.c:3236 [inline]
| process_scheduled_works+0x678/0xd18 kernel/workqueue.c:3319
| bh_worker+0x2f0/0x59c kernel/workqueue.c:3579
| workqueue_softirq_action+0x104/0x14c kernel/workqueue.c:3606
| tasklet_action+0x18/0x8c kernel/softirq.c:854
| handle_softirqs+0x208/0x63c kernel/softirq.c:579
| run_ksoftirqd+0x64/0x264 kernel/softirq.c:968
| smpboot_thread_fn+0x4ac/0x908 kernel/smpboot.c:160
| kthread+0x5e8/0x734 kernel/kthread.c:464
| ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:844
Softirq context is not allowed to sleep. PREEMPT_RT replaces spinlock
implementations with sleepable variants to improve RT performance.
This is fine in process context but unsafe in atomic contexts such as
softirq. This specific instance is in the USB core bh path, so a
framework-level fix seems appropriate.
Because USB activity frequently triggers hub interrupts and URB
givebacks "urb->complete(urb)", this issue can be hit easily on
PREEMPT_RT systems. This means that using USB devices under PREEMPT_RT
may be unstable in its current state, especially under high I/O load or
when multiple devices are active. The likelihood of encountering this
problem increases with frequent plug/unplug events, isochronous transfers,
or heavy USB traffic.
At the moment, I think the following solutions might be possible:
1. Use raw spinlocks (raw_spin_lock_irqsave/restore) in bh paths.
2. Move sleepable work into process context (regular workqueues).
3. Avoid locking entirely in these atomic paths if possible.
I believe this requires a change in the USB core framework rather than
just in individual drivers.
Kernel config, full logs, and reproduction steps can be provided on
request.
This bug was uncovered during my work to fixing KCOV for PREEMPT_RT awareness.
Link: https://lore.kernel.org/all/ee26e7b2-80dd-49b1-bca2-61e460f73c2d@kzalloc.com/t/#u
Best Regards,
Yunseong Kim
Powered by blists - more mailing lists