| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID:
<TYUPR06MB621712E6A082791B397088BFD2F72@TYUPR06MB6217.apcprd06.prod.outlook.com>
Date: Wed, 5 Feb 2025 06:38:48 +0000
From: 胡连勤 <hulianqin@...o.com>
To: "gregkh@...uxfoundation.org" <gregkh@...uxfoundation.org>, Dan Carpenter
<dan.carpenter@...aro.org>, Jon Hunter <jonathanh@...dia.com>
CC: Prashanth K <quic_prashk@...cinc.com>, "mwalle@...nel.org"
<mwalle@...nel.org>, "quic_jjohnson@...cinc.com" <quic_jjohnson@...cinc.com>,
David Brownell <dbrownell@...rs.sourceforge.net>, "linux-usb@...r.kernel.org"
<linux-usb@...r.kernel.org>, "linux-kernel@...r.kernel.org"
<linux-kernel@...r.kernel.org>, opensource.kernel
<opensource.kernel@...o.com>, "linux-tegra@...r.kernel.org"
<linux-tegra@...r.kernel.org>, Brad Griffis <bgriffis@...dia.com>, Harshit
Mogalapalli <harshit.m.mogalapalli@...il.com>
Subject:
答复: [PATCH v3] usb: gadget: u_serial: Disable ep before setting port to null to fix the crash caused by port being null
Hi Greg, Dan,Jon, other kernel experts:
> On Tue, Jan 21, 2025 at 12:19:37PM +0000, Jon Hunter wrote:
> > > > > [ 59.918177] tegra-xudc 3550000.usb: ep 3 disabled [
> > > > > 59.923097] tegra-xudc 3550000.usb: ep 2 disabled [ 59.927955]
> > > > > tegra-xudc 3550000.usb: ep 5 disabled [ 80.911432] rcu: INFO:
> > > > > rcu_preempt self-detected stall on CPU [ 80.917354] rcu:
> > > > > 6-....: (5248 ticks this GP)
> > > > > idle=ec24/1/0x4000000000000000 softirq=1213/1213 fqs=2623 [
> > > > > 80.927146] rcu: (t=5253 jiffies g=3781 q=1490 ncpus=12) [
> > > > > 80.932704] Sending NMI from CPU 6 to CPUs 2:
> > > > > [ 90.981555] CPU: 6 UID: 0 PID: 18 Comm: rcu_exp_gp_kthr Not
> > > > > tainted 6.13.0-rc7-00043-g619f0b6fad52 #1 [ 90.981558]
> > > > > Hardware name: NVIDIA NVIDIA Jetson AGX Orin Developer
> > > > > Kit/Jetson, BIOS 00.0.0-dev-main_92e5ae_88fd1_296de
> > > > > 12/16/2024
> > > > > [ 90.981559] pstate: 80400009 (Nzcv daif +PAN -UAO -TCO -DIT
> > > > > -SSBS BTYPE=--) [ 90.981562] pc :
> > > > > smp_call_function_single+0xdc/0x1a0
> > > > > [ 90.981574] lr : __sync_rcu_exp_select_node_cpus+0x228/0x3c0
> > > > > [ 90.981578] sp : ffff800082eb3cd0 [ 90.981579] x29:
> > > > > ffff800082eb3cd0 x28: 0000000000000010 x27:
> > > > > ffff0000802933c0
> > > > > [ 90.981582] x26: ffff0007a8a1d700 x25: ffff800082895500 x24:
> > > > > ffff800080132018
> > > > > [ 90.981584] x23: 0000000000000014 x22: ffff800081fb7700 x21:
> > > > > ffff80008280d970
> > > > > [ 90.981586] x20: 0000000000000feb x19: ffff800082eb3d00 x18:
> > > > > 0000000000000000
> > > > > [ 90.981588] x17: 0000000000000000 x16: 0000000000000000 x15:
> > > > > 0000000000000000
> > > > > [ 90.981590] x14: ffff000080293440 x13: 0000000000000001 x12:
> > > > > 0000000000000000
> > > > > [ 90.981591] x11: ffff800081fb2388 x10: ffff0000802933c0 x9 :
> > > > > 0000000000000001
> > > > > [ 90.981593] x8 : 0000000000000040 x7 : 0000000000017068 x6 :
> > > > > ffff800080132018
> > > > > [ 90.981595] x5 : 0000000000000000 x4 : ffff0007a8a4f9c8 x3 :
> > > > > 0000000000000001
> > > > > [ 90.981597] x2 : 0000000000000000 x1 : ffff0007a8a4f9c0 x0 :
> > > > > 0000000000000004
> > > > > [ 90.981599] Call trace:
> > > > > [ 90.981601] smp_call_function_single+0xdc/0x1a0 (P) [
> > > > > 90.981605] __sync_rcu_exp_select_node_cpus+0x228/0x3c0
> > > > > [ 90.981607] sync_rcu_exp_select_cpus+0x13c/0x2a0
> > > > > [ 90.981609] wait_rcu_exp_gp+0x18/0x30 [ 90.981611]
> > > > > kthread_worker_fn+0xd0/0x188 [ 90.981614] kthread+0x118/0x11c
> > > > > [ 90.981619] ret_from_fork+0x10/0x20 [ 101.416347] sched: DL
> > > > > replenish lagged too much
> > > > >
> > > >
> > > > Odd, you have a usb-serial gadget device in this system that is
> > > > disconnecting somehow? That oops doesn't point to anything in the
> > > > usb gadget codebase, "all" we have done is move the call to
> > > > shutdown the endpoints to earlier in the disconnect function.
> > >
> > > Yes the board starts usb-serial and usb-ethernet gadget and on
> > > reboot when tearing it down I am seeing the above. As soon as it
> > > disables the tegra-xudc endpoints (as seen above) the board appears to
> stall.
> > >
> > > > I'm glad to revert this, but it feels really odd that this is
> > > > causing you an rcu stall issue.
> > >
> > > Thanks. I can't say I understand it either, but I am certain it is
> > > caused by this change.
> > >
> > > Happy to run any tests to narrow this down a bit.
> >
> >
> > I did a bit more looking at this and I see that we setup a USB gadget
> > device via the configfs as described in this doc [0]. The RCU stall
> > occurs when we attempt to disable the gadget on shutdown by ...
> >
> > $ echo "" > /path/to/UDC
> >
> > Jon
> >
> > [0] https://docs.kernel.org/usb/gadget_configfs.html
>
> It's an RCU stall so it's probably a locking bug. The original code dropped the
> port->port_lock, called usb_ep_disable() then took the lock again. There
> must have been a reason to drop the lock although I haven't found it yet.
> The new code moved the usb_ep_disable() under the lock.
>
> 1431 port->suspended = false;
> 1432 spin_unlock(&port->port_lock);
> 1433 spin_unlock_irqrestore(&serial_port_lock, flags);
> 1434
> 1435 /* disable endpoints, aborting down any active I/O */
> 1436 usb_ep_disable(gser->out);
> 1437 usb_ep_disable(gser->in);
> 1438
> 1439 /* finally, free any unused/unusable I/O buffers */
> 1440 spin_lock_irqsave(&port->port_lock, flags);
> 1441 if (port->port.count == 0)
> 1442 kfifo_free(&port->port_write_buf);
>
>
> It might help to try testing again with lockdep enabled.
>
To solve this problem, I plan to refer to the methods in gserial_suspend and gserial_resume functions
and use spin lock to solve the competition between gs_read_complete and gserial_disconnect.
The patch is as follows:
Considering that in some extreme cases, when the unbind operation
is being executed, gserial_disconnect has already cleared gser->ioport,
triggering a gadget reconfiguration at this time and gs_read_complete
gets called afterwards, which results in accessing null pointer,
add a null pointer check to prevent this situation.
Added a static spinlock to prevent gser->ioport from becoming
null after the newly added check.
Call trace:
gs_read_complete+0x58/0x240
usb_gadget_giveback_request+0x40/0x160
dwc3_remove_requests+0x170/0x484
dwc3_ep0_out_start+0xb0/0x1d4
__dwc3_gadget_start+0x25c/0x720
kretprobe_trampoline.cfi_jt+0x0/0x8
kretprobe_trampoline.cfi_jt+0x0/0x8
udc_bind_to_driver+0x1d8/0x300
usb_gadget_probe_driver+0xa8/0x1dc
gadget_dev_desc_UDC_store+0x13c/0x188
configfs_write_iter+0x160/0x1f4
vfs_write+0x2d0/0x40c
ksys_write+0x7c/0xf0
__arm64_sys_write+0x20/0x30
invoke_syscall+0x60/0x150
el0_svc_common+0x8c/0xf8
do_el0_svc+0x28/0xa0
el0_svc+0x24/0x84
Fixes: c1dca562be8a ("usb gadget: split out serial core")
Cc: stable@...r.kernel.org
Suggested-by: Prashanth K <quic_prashk@...cinc.com>
Signed-off-by: Lianqin Hu <hulianqin@...o.com>
drivers/usb/gadget/function/u_serial.c | 17 ++++++++++++++---
1 file changed, 14 insertions(+), 3 deletions(-)
diff --git a/drivers/usb/gadget/function/u_serial.c b/drivers/usb/gadget/function/u_serial.c
index 0a8c05b2746b..6f2c9715e8df 100644
--- a/drivers/usb/gadget/function/u_serial.c
+++ b/drivers/usb/gadget/function/u_serial.c
@@ -456,13 +456,24 @@ static void gs_rx_push(struct work_struct *work)
static void gs_read_complete(struct usb_ep *ep, struct usb_request *req)
{
- struct gs_port *port = ep->driver_data;
+ struct gs_port *port;
+ unsigned long flags;
+
+ spin_lock_irqsave(&serial_port_lock, flags);
+ port = ep->driver_data;
+
+ if (!port) {
+ spin_unlock_irqrestore(&serial_port_lock, flags);
+ return;
+ }
- /* Queue all received data until the tty layer is ready for it. */
spin_lock(&port->port_lock);
+ spin_unlock(&serial_port_lock);
+
+ /* Queue all received data until the tty layer is ready for it. */
list_add_tail(&req->list, &port->read_queue);
schedule_delayed_work(&port->push, 0);
- spin_unlock(&port->port_lock);
+ spin_unlock_irqrestore(&port->port_lock, flags);
}
static void gs_write_complete(struct usb_ep *ep, struct usb_request *req)
--
2.39.0
Please ask Linux kernel experts to help evaluate whether it is feasible.
Thanks
Powered by blists - more mailing lists