[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <CA+V-a8s6TmgM4=J-3=zt3ZbNdLtwn5sfBP6FdZVNg09t634P_w@mail.gmail.com>
Date: Mon, 24 Jun 2024 10:25:38 +0100
From: "Lad, Prabhakar" <prabhakar.csengg@...il.com>
To: Alexandre Torgue <alexandre.torgue@...s.st.com>, Jose Abreu <joabreu@...opsys.com>
Cc: netdev <netdev@...r.kernel.org>
Subject: STMMAC driver CPU stall warning
Hi Alexandre and Jose,
While integrating the STMMAC driver for the Renesas RZ/V2H SoC I am
seeing the CPU stall warning when the locking config options are
enabled. Has anyone seen this issue?
Platform and Kernel details :
-------------------------------------
- Renesas RZ/V2H(P) EVK (ARM64)
- 6.10.0-rc5
- IP version, Synopsys ID: 0x52
With the below locking configs enabled:
----------------------------------------------------
# CONFIG_PER_VMA_LOCK_STATS is not set
# CONFIG_SOFTLOCKUP_DETECTOR is not set
CONFIG_HAVE_HARDLOCKUP_DETECTOR_BUDDY=y
# CONFIG_HARDLOCKUP_DETECTOR is not set
CONFIG_LOCK_DEBUGGING_SUPPORT=y
CONFIG_PROVE_LOCKING=y
# CONFIG_PROVE_RAW_LOCK_NESTING is not set
# CONFIG_LOCK_STAT is not set
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_LOCKDEP=y
CONFIG_LOCKDEP_BITS=15
CONFIG_LOCKDEP_CHAINS_BITS=16
CONFIG_LOCKDEP_STACK_TRACE_BITS=19
CONFIG_LOCKDEP_STACK_TRACE_HASH_BITS=14
CONFIG_LOCKDEP_CIRCULAR_QUEUE_BITS=12
# CONFIG_DEBUG_LOCKDEP is not set
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
# CONFIG_LOCK_TORTURE_TEST is not set
# CONFIG_CSD_LOCK_WAIT_DEBUG is not set
CONFIG_TRACE_CLOCK=y
[ 33.701954] rcu: INFO: rcu_preempt self-detected stall on CPU
[ 33.707776] rcu: 0-....: (1 GPs behind)
idle=7054/1/0x4000000000000000 softirq=413/414 fqs=2228
[ 33.716693] rcu: (t=6500 jiffies g=-691 q=65 ncpus=4)
[ 33.721856] CPU: 0 PID: 37 Comm: kworker/u16:1 Not tainted
6.10.0-rc5-arm64-renesas-00383-g34de6e9c5801 #495
[ 33.731717] Hardware name: Renesas based on r9a09g057h44 (DT)
[ 33.737833] Workqueue: rpciod rpc_async_schedule
[ 33.742488] pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 33.749474] pc : _raw_spin_unlock_irqrestore+0x34/0x68
[ 33.754640] lr : _raw_spin_unlock_irqrestore+0x64/0x68
[ 33.759800] sp : ffff8000828133b0
[ 33.763128] x29: ffff8000828133b0 x28: 0000000000000080 x27: 0000000000000000
[ 33.770304] x26: ffff8000818faa00 x25: ffff0003ee18b200 x24: 0000000000000001
[ 33.777478] x23: ffff0003ee18b180 x22: ffff0003ee18b180 x21: ffff0003ee18b180
[ 33.784653] x20: ffff0003ee18b180 x19: 0000000000000000 x18: 00000000000020e8
[ 33.791827] x17: 344617ab1e4d0a08 x16: 0101000003d71d0f x15: 0000000000000028
[ 33.799002] x14: 000000000000000a x13: 0000000000004d7c x12: 0000000000000000
[ 33.806176] x11: ffff800082540a7c x10: ffff8000814e3cd4 x9 : 0000000000000168
[ 33.813351] x8 : 0000000000000001 x7 : 00000000000000c8 x6 : ffff8000821e6000
[ 33.820524] x5 : 0000000000000000 x4 : 0000000000000009 x3 : 0000000000000000
[ 33.827698] x2 : 0000000000000001 x1 : ffff8000814e3cd0 x0 : 0000000000009384
[ 33.834872] Call trace:
[ 33.837331] _raw_spin_unlock_irqrestore+0x34/0x68
[ 33.842145] hrtimer_start_range_ns+0x138/0x314
[ 33.846700] stmmac_tx_timer_arm+0x88/0x8c
[ 33.850819] stmmac_xmit+0x778/0xac4
[ 33.854413] dev_hard_start_xmit+0xcc/0x2d0
[ 33.858618] sch_direct_xmit+0x88/0x378
[ 33.862477] __dev_queue_xmit+0x784/0xfc0
[ 33.866506] ip_finish_output2+0x794/0xac8
[ 33.870627] __ip_finish_output+0x110/0x27c
[ 33.874831] ip_output+0x6c/0x278
[ 33.878165] __ip_queue_xmit+0x194/0x5a0
[ 33.882106] ip_queue_xmit+0x10/0x18
[ 33.885698] __tcp_transmit_skb+0x4a8/0xba4
[ 33.889903] __tcp_send_ack.part.0+0xc8/0x144
[ 33.894280] tcp_release_cb+0x1ec/0x208
[ 33.898133] release_sock+0x48/0x9c
[ 33.901642] tcp_sendmsg+0x44/0x58
[ 33.905063] inet_sendmsg+0x40/0x64
[ 33.908575] sock_sendmsg+0x7c/0xe0
[ 33.912085] xprt_sock_sendmsg+0xdc/0x300
[ 33.916115] xs_tcp_send_request+0xec/0x240
[ 33.920318] xprt_transmit+0x1e4/0x5e8
[ 33.924084] call_transmit+0x7c/0x90
[ 33.927680] __rpc_execute+0xdc/0x6f8
[ 33.931361] rpc_async_schedule+0x28/0x40
[ 33.935390] process_one_work+0x21c/0x620
[ 33.939422] worker_thread+0x1a8/0x374
[ 33.943190] kthread+0x10c/0x110
[ 33.946436] ret_from_fork+0x10/0x20
[ 33.950033] Sending NMI from CPU 0 to CPUs 3:
[ 33.954415] NMI backtrace for cpu 3
[ 33.957929] CPU: 3 PID: 67 Comm: kworker/u16:6 Not tainted
6.10.0-rc5-arm64-renesas-00383-g34de6e9c5801 #495
[ 33.967788] Hardware name: Renesas based on r9a09g057h44 (DT)
[ 33.973902] Workqueue: xprtiod xs_stream_data_receive_workfn
[ 33.979591] pstate: 00400005 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 33.986576] pc : queued_spin_lock_slowpath+0x68/0x514
[ 33.991650] lr : do_raw_spin_lock+0xb8/0x104
[ 33.995941] sp : ffff800082fcba70
[ 33.999268] x29: ffff800082fcba70 x28: 0000000000000000 x27: 0000000000000000
[ 34.006441] x26: ffff0000c2966840 x25: 0000000000000000 x24: ffff800081863000
[ 34.013614] x23: ffff0000c0578000 x22: 000000000000000c x21: 0000000000000040
[ 34.020786] x20: ffff8000814e1008 x19: ffff0000c29f0190 x18: 0000000000093930
[ 34.027958] x17: 0019001a00400009 x16: 0038004000000000 x15: 0000000000000000
[ 34.035130] x14: 0000000000000001 x13: ffff0000c2a20080 x12: 0000000000000000
[ 34.042302] x11: ffff800082540a7c x10: ffff8000814e3cd4 x9 : 00000000000000a0
[ 34.049473] x8 : 0000000000000001 x7 : 00000000000000c8 x6 : ffff8000821e6000
[ 34.056645] x5 : f647258a66f2c9bc x4 : ffff8000821ad9e0 x3 : 0000000000000000
[ 34.063815] x2 : ffff80036cd00000 x1 : 0000000000000000 x0 : 0000000000000001
[ 34.070987] Call trace:
[ 34.073444] queued_spin_lock_slowpath+0x68/0x514
[ 34.078169] do_raw_spin_lock+0xb8/0x104
[ 34.082110] _raw_spin_lock_bh+0x54/0x60
[ 34.086051] lock_sock_nested+0x3c/0x84
[ 34.089905] tcp_recvmsg+0x5c/0x1c0
[ 34.093412] inet_recvmsg+0x60/0x244
[ 34.097007] sock_recvmsg+0x2c/0x6c
[ 34.100515] xs_read_stream.constprop.0+0xb0/0x4e0
[ 34.105327] xs_stream_data_receive_workfn+0x54/0x1b4
[ 34.110398] process_one_work+0x21c/0x620
[ 34.114427] worker_thread+0x1a8/0x374
[ 34.118194] kthread+0x10c/0x110
[ 34.121438] ret_from_fork+0x10/0x20
[ 68.470538] random: crng init done
Below is the dmesg logs for gbeth:
----------------------------------------------
[ 1.611307] renesas-gbeth 15c30000.ethernet: IRQ sfty not found
[ 1.626475] renesas-gbeth 15c30000.ethernet: User ID: 0x1, Synopsys ID: 0x52
[ 1.633615] renesas-gbeth 15c30000.ethernet: DWMAC4/5
[ 1.638809] renesas-gbeth 15c30000.ethernet: DMA HW capability
register supported
[ 1.646351] renesas-gbeth 15c30000.ethernet: RX Checksum Offload
Engine supported
[ 1.653904] renesas-gbeth 15c30000.ethernet: Wake-Up On Lan supported
[ 1.660516] renesas-gbeth 15c30000.ethernet: Enable RX Mitigation
via HW Watchdog Timer
[ 1.668584] renesas-gbeth 15c30000.ethernet: Enabled L3L4 Flow TC (entries=8)
[ 1.675782] renesas-gbeth 15c30000.ethernet: Enabled RFS Flow TC (entries=10)
[ 1.682979] renesas-gbeth 15c30000.ethernet: Using 32/32 bits DMA
host/device width
[ 1.698226] renesas-gbeth 15c40000.ethernet: IRQ sfty not found
[ 1.713295] renesas-gbeth 15c40000.ethernet: User ID: 0x0, Synopsys ID: 0x52
[ 1.720432] renesas-gbeth 15c40000.ethernet: DWMAC4/5
[ 1.725637] renesas-gbeth 15c40000.ethernet: DMA HW capability
register supported
[ 1.733179] renesas-gbeth 15c40000.ethernet: RX Checksum Offload
Engine supported
[ 1.740720] renesas-gbeth 15c40000.ethernet: Wake-Up On Lan supported
[ 1.747327] renesas-gbeth 15c40000.ethernet: Enable RX Mitigation
via HW Watchdog Timer
[ 1.755408] renesas-gbeth 15c40000.ethernet: device MAC address
ba:91:02:27:30:8f
[ 1.762977] renesas-gbeth 15c40000.ethernet: Enabled L3L4 Flow TC (entries=8)
[ 1.770185] renesas-gbeth 15c40000.ethernet: Enabled RFS Flow TC (entries=10)
[ 1.777382] renesas-gbeth 15c40000.ethernet: Using 32/32 bits DMA
host/device width
[ 2.240115] renesas-gbeth 15c30000.ethernet eth0: Register
MEM_TYPE_PAGE_POOL RxQ-0
[ 2.248621] renesas-gbeth 15c30000.ethernet eth0: Register
MEM_TYPE_PAGE_POOL RxQ-1
[ 2.257069] renesas-gbeth 15c30000.ethernet eth0: Register
MEM_TYPE_PAGE_POOL RxQ-2
[ 2.265151] renesas-gbeth 15c30000.ethernet eth0: Register
MEM_TYPE_PAGE_POOL RxQ-3
[ 2.341611] renesas-gbeth 15c30000.ethernet eth0: PHY [stmmac-0:00]
driver [Microchip KSZ9131 Gigabit PHY] (irq=POLL)
[ 2.357928] renesas-gbeth 15c30000.ethernet eth0: No Safety
Features support found
[ 2.365581] renesas-gbeth 15c30000.ethernet eth0: IEEE 1588-2008
Advanced Timestamp supported
[ 2.375086] renesas-gbeth 15c30000.ethernet eth0: registered PTP clock
[ 2.382575] renesas-gbeth 15c30000.ethernet eth0: configuring for
phy/rgmii-id link mode
[ 2.400349] renesas-gbeth 15c40000.ethernet eth1: Register
MEM_TYPE_PAGE_POOL RxQ-0
[ 2.409133] renesas-gbeth 15c40000.ethernet eth1: Register
MEM_TYPE_PAGE_POOL RxQ-1
[ 2.417244] renesas-gbeth 15c40000.ethernet eth1: Register
MEM_TYPE_PAGE_POOL RxQ-2
[ 2.425629] renesas-gbeth 15c40000.ethernet eth1: Register
MEM_TYPE_PAGE_POOL RxQ-3
[ 2.466041] renesas-gbeth 15c40000.ethernet eth1: PHY [stmmac-1:00]
driver [Microchip KSZ9131 Gigabit PHY] (irq=POLL)
[ 2.482310] renesas-gbeth 15c40000.ethernet eth1: No Safety
Features support found
[ 2.489977] renesas-gbeth 15c40000.ethernet eth1: IEEE 1588-2008
Advanced Timestamp supported
[ 2.499184] renesas-gbeth 15c40000.ethernet eth1: registered PTP clock
[ 2.506662] renesas-gbeth 15c40000.ethernet eth1: configuring for
phy/rgmii-id link mode
[ 6.498736] renesas-gbeth 15c30000.ethernet eth0: Link is Up -
1Gbps/Full - flow control rx/tx
[ 73.941601] renesas-gbeth 15c40000.ethernet eth1: Register
MEM_TYPE_PAGE_POOL RxQ-0
[ 73.954665] renesas-gbeth 15c40000.ethernet eth1: Register
MEM_TYPE_PAGE_POOL RxQ-1
[ 73.963942] renesas-gbeth 15c40000.ethernet eth1: Register
MEM_TYPE_PAGE_POOL RxQ-2
[ 73.976780] renesas-gbeth 15c40000.ethernet eth1: Register
MEM_TYPE_PAGE_POOL RxQ-3
[ 74.066033] renesas-gbeth 15c40000.ethernet eth1: PHY [stmmac-1:00]
driver [Microchip KSZ9131 Gigabit PHY] (irq=POLL)
[ 74.086068] renesas-gbeth 15c40000.ethernet eth1: No Safety
Features support found
[ 74.098223] renesas-gbeth 15c40000.ethernet eth1: IEEE 1588-2008
Advanced Timestamp supported
[ 74.102485] renesas-gbeth 15c40000.ethernet eth1: registered PTP clock
[ 74.122445] renesas-gbeth 15c40000.ethernet eth1: configuring for
phy/rgmii-id link mode
Please let me know if you need me to try anything or provide more
debug information.
Cheers,
Prabhakar
Powered by blists - more mailing lists