linux-kernel - Re: [PATCH v3] usb: gadget: u_serial: Disable ep before setting port to null to fix the crash caused by port being null

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <0fa6c6db-fd75-4a09-b4fa-d6a98bb8afac@stanley.mountain>
Date: Thu, 23 Jan 2025 17:56:44 +0300
From: Dan Carpenter <dan.carpenter@...aro.org>
To: Jon Hunter <jonathanh@...dia.com>
Cc: "gregkh@...uxfoundation.org" <gregkh@...uxfoundation.org>,
	胡连勤 <hulianqin@...o.com>,
	Prashanth K <quic_prashk@...cinc.com>,
	"mwalle@...nel.org" <mwalle@...nel.org>,
	"quic_jjohnson@...cinc.com" <quic_jjohnson@...cinc.com>,
	David Brownell <dbrownell@...rs.sourceforge.net>,
	"linux-usb@...r.kernel.org" <linux-usb@...r.kernel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"opensource.kernel" <opensource.kernel@...o.com>,
	"linux-tegra@...r.kernel.org" <linux-tegra@...r.kernel.org>,
	Brad Griffis <bgriffis@...dia.com>,
	Harshit Mogalapalli <harshit.m.mogalapalli@...il.com>
Subject: Re: [PATCH v3] usb: gadget: u_serial: Disable ep before setting port
 to null to fix the crash caused by port being null

On Tue, Jan 21, 2025 at 12:19:37PM +0000, Jon Hunter wrote:
> > > > [   59.918177] tegra-xudc 3550000.usb: ep 3 disabled
> > > > [   59.923097] tegra-xudc 3550000.usb: ep 2 disabled
> > > > [   59.927955] tegra-xudc 3550000.usb: ep 5 disabled
> > > > [   80.911432] rcu: INFO: rcu_preempt self-detected stall on CPU
> > > > [   80.917354] rcu:     6-....: (5248 ticks this GP)
> > > > idle=ec24/1/0x4000000000000000 softirq=1213/1213 fqs=2623
> > > > [   80.927146] rcu:     (t=5253 jiffies g=3781 q=1490 ncpus=12)
> > > > [   80.932704] Sending NMI from CPU 6 to CPUs 2:
> > > > [   90.981555] CPU: 6 UID: 0 PID: 18 Comm: rcu_exp_gp_kthr Not
> > > > tainted 6.13.0-rc7-00043-g619f0b6fad52 #1
> > > > [   90.981558] Hardware name: NVIDIA NVIDIA Jetson AGX Orin
> > > > Developer Kit/Jetson, BIOS 00.0.0-dev-main_92e5ae_88fd1_296de
> > > > 12/16/2024
> > > > [   90.981559] pstate: 80400009 (Nzcv daif +PAN -UAO -TCO -DIT
> > > > -SSBS BTYPE=--)
> > > > [   90.981562] pc : smp_call_function_single+0xdc/0x1a0
> > > > [   90.981574] lr : __sync_rcu_exp_select_node_cpus+0x228/0x3c0
> > > > [   90.981578] sp : ffff800082eb3cd0
> > > > [   90.981579] x29: ffff800082eb3cd0 x28: 0000000000000010 x27:
> > > > ffff0000802933c0
> > > > [   90.981582] x26: ffff0007a8a1d700 x25: ffff800082895500 x24:
> > > > ffff800080132018
> > > > [   90.981584] x23: 0000000000000014 x22: ffff800081fb7700 x21:
> > > > ffff80008280d970
> > > > [   90.981586] x20: 0000000000000feb x19: ffff800082eb3d00 x18:
> > > > 0000000000000000
> > > > [   90.981588] x17: 0000000000000000 x16: 0000000000000000 x15:
> > > > 0000000000000000
> > > > [   90.981590] x14: ffff000080293440 x13: 0000000000000001 x12:
> > > > 0000000000000000
> > > > [   90.981591] x11: ffff800081fb2388 x10: ffff0000802933c0 x9 :
> > > > 0000000000000001
> > > > [   90.981593] x8 : 0000000000000040 x7 : 0000000000017068 x6 :
> > > > ffff800080132018
> > > > [   90.981595] x5 : 0000000000000000 x4 : ffff0007a8a4f9c8 x3 :
> > > > 0000000000000001
> > > > [   90.981597] x2 : 0000000000000000 x1 : ffff0007a8a4f9c0 x0 :
> > > > 0000000000000004
> > > > [   90.981599] Call trace:
> > > > [   90.981601]  smp_call_function_single+0xdc/0x1a0 (P)
> > > > [   90.981605]  __sync_rcu_exp_select_node_cpus+0x228/0x3c0
> > > > [   90.981607]  sync_rcu_exp_select_cpus+0x13c/0x2a0
> > > > [   90.981609]  wait_rcu_exp_gp+0x18/0x30
> > > > [   90.981611]  kthread_worker_fn+0xd0/0x188
> > > > [   90.981614]  kthread+0x118/0x11c
> > > > [   90.981619]  ret_from_fork+0x10/0x20
> > > > [  101.416347] sched: DL replenish lagged too much
> > > > 
> > > 
> > > Odd, you have a usb-serial gadget device in this system that is
> > > disconnecting somehow?  That oops doesn't point to anything in the usb
> > > gadget codebase, "all" we have done is move the call to shutdown the
> > > endpoints to earlier in the disconnect function.
> > 
> > Yes the board starts usb-serial and usb-ethernet gadget and on reboot
> > when tearing it down I am seeing the above. As soon as it disables the
> > tegra-xudc endpoints (as seen above) the board appears to stall.
> > 
> > > I'm glad to revert this, but it feels really odd that this is causing
> > > you an rcu stall issue.
> > 
> > Thanks. I can't say I understand it either, but I am certain it is
> > caused by this change.
> > 
> > Happy to run any tests to narrow this down a bit.
> 
> 
> I did a bit more looking at this and I see that we setup a USB gadget device
> via the configfs as described in this doc [0]. The RCU stall occurs when we
> attempt to disable the gadget on shutdown by ...
> 
>  $ echo "" > /path/to/UDC
> 
> Jon
> 
> [0] https://docs.kernel.org/usb/gadget_configfs.html

It's an RCU stall so it's probably a locking bug.  The original code
dropped the port->port_lock, called usb_ep_disable() then took the
lock again.  There must have been a reason to drop the lock although
I haven't found it yet.  The new code moved the usb_ep_disable() under
the lock.

  1431          port->suspended = false;
  1432          spin_unlock(&port->port_lock);
  1433          spin_unlock_irqrestore(&serial_port_lock, flags);
  1434  
  1435          /* disable endpoints, aborting down any active I/O */
  1436          usb_ep_disable(gser->out);
  1437          usb_ep_disable(gser->in);
  1438  
  1439          /* finally, free any unused/unusable I/O buffers */
  1440          spin_lock_irqsave(&port->port_lock, flags);
  1441          if (port->port.count == 0)
  1442                  kfifo_free(&port->port_write_buf);


It might help to try testing again with lockdep enabled.

regards,
dan carpenter