Date:   Fri, 12 Apr 2019 14:13:52 +0300
From:   "Leonidas P. Papadakos" <papadakospan@...il.com>
To:     Jose Abreu <jose.abreu@...opsys.com>
Cc:     Robin Murphy <robin.murphy@....com>,
        Philipp Tomsich <philipp.tomsich@...obroma-systems.com>,
        Heiko Stübner <heiko@...ech.de>,
        Christoph Müllner 
        <christoph.muellner@...obroma-systems.com>,
        Maxime Coquelin <mcoquelin.stm32@...il.com>,
        Alexandre Torgue <alexandre.torgue@...com>,
        netdev@...r.kernel.org, LKML <linux-kernel@...r.kernel.org>,
        Linux ARM <linux-arm-kernel@...ts.infradead.org>,
        Klaus Goger <klaus.goger@...obroma-systems.com>
Subject: RE: [PATCH 1/2] stmmac: introduce flag to dynamically disable TX
 offload for rockchip devices


> Can you please share the stacktrace here ? I can't access pastebin 
> due to
> corporate policy.

This happens seconds after I change the MTU. I remember it used to 
happen with 1500 as well, before the gmac tweaks. It makes sense

[  111.111639] NETDEV WATCHDOG: eth0 (rk_gmac-dwmac): transmit queue 0 
timed out
[  111.112374] WARNING: CPU: 0 PID: 0 at net/sched/sch_generic.c:461 
dev_watchdog+0x2b8/0x2c0
[  111.113112] Modules linked in: lz4 lz4_compress zram rockchipdrm 
realtek analogix_dp dw_mipi_dsi dw_hdmi cec rc_core dwmac_rk 
stmmac_platform drm_kms_helper stmmac rtc_rk808 drm rockchip_thermal 
drm_panel_orientation_quirks dw_wdt syscopyarea sysfillrect sysimgblt 
fb_sys_fops ip6t_REJECT nf_reject_ipv6 xt_hl ip6t_rt ipt_REJECT 
xt_recent xt_multiport xt_comment xt_limit xt_addrtype xt_conntrack 
ip6table_filter ip6_tables nf_conntrack_netbios_ns 
nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp nf_conntrack 
nf_defrag_ipv6 nf_defrag_ipv4 iptable_filter bpfilter tcp_bbr tcp_lp 
nfsd
[  111.117725] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 
5.1.0-rc4-1-ARCH #1
[  111.118326] Hardware name: Firefly roc-rk3328-cc (DT)
[  111.118778] pstate: 20000005 (nzCv daif -PAN -UAO)
[  111.119204] pc : dev_watchdog+0x2b8/0x2c0
[  111.119568] lr : dev_watchdog+0x2b8/0x2c0
[  111.119928] sp : ffff000010003d50
[  111.120223] x29: ffff000010003d50 x28: ffff00001166a000
[  111.120695] x27: 0000000000000140 x26: 00000000ffffffff
[  111.121166] x25: 0000000000000000 x24: 0000000000000000
[  111.121635] x23: ffff8000383c4480 x22: 0000000000000001
[  111.122104] x21: ffff000011687000 x20: ffff8000383c4000
[  111.122579] x19: 0000000000000000 x18: 0000000000000010
[  111.123052] x17: 0000000000000000 x16: 0000000000000000
[  111.123520] x15: ffffffffffffffff x14: ffff00001168d6c8
[  111.123995] x13: ffff000090003a67 x12: ffff000010003a6f
[  111.124470] x11: ffff0000116b7000 x10: ffff0000100039f0
[  111.124938] x9 : 00000000ffffffd0 x8 : ffff0000108395d0
[  111.125411] x7 : 0000000000000179 x6 : 0000000000000001
[  111.125885] x5 : 0000000000000000 x4 : 0000000000000001
[  111.126359] x3 : 0000000000000000 x2 : ffff00001168ea58
[  111.126828] x1 : 576938cacdaf1700 x0 : 0000000000000000
[  111.127309] Call trace:
[  111.127533]  dev_watchdog+0x2b8/0x2c0
[  111.127864]  call_timer_fn+0x34/0x170
[  111.128195]  expire_timers.part.5+0xc8/0x158
[  111.128578]  run_timer_softirq+0xd4/0x218
[  111.128940]  __do_softirq+0x130/0x330
[  111.129273]  irq_exit+0xc0/0xd0
[  111.129560]  __handle_domain_irq+0x70/0xc0
[  111.129924]  gic_handle_irq+0x58/0xa8
[  111.130255]  el1_irq+0xb8/0x140
[  111.130542]  arch_cpu_idle+0x3c/0x1c8
[  111.130874]  default_idle_call+0x38/0x40
[  111.131226]  do_idle+0x23c/0x2b8
[  111.131513]  cpu_startup_entry+0x2c/0x30
[  111.131868]  rest_init+0xb8/0xc4
[  111.132161]  arch_call_rest_init+0x14/0x1c
[  111.132529]  start_kernel+0x484/0x4b0
[  111.132862] ---[ end trace 7efe323f823d5938 ]---
[  111.133352] rk_gmac-dwmac ff540000.ethernet eth0: Reset adapter.
[  111.134637] rk_gmac-dwmac ff540000.ethernet eth0: Link is Down
[  111.138412] RTL8211E Gigabit Ethernet stmmac-0:00: attached PHY 
driver [RTL8211E Gigabit Ethernet] (mii_bus:phy_addr=stmmac-0:00, 
irq=POLL)
[  111.151252] rk_gmac-dwmac ff540000.ethernet eth0: No Safety Features 
support found
[  111.151962] rk_gmac-dwmac ff540000.ethernet eth0: PTP not supported 
by HW
[  115.352150] rk_gmac-dwmac ff540000.ethernet eth0: Link is Up - 
1Gbps/Full - flow control rx/tx
[  115.352965] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

I also have another one that happened while wireguard was loaded:
I brought down eth0. changed the mtu, and brought ot up again.

Απρ 11 16:06:11.411210 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: Link is Down
Απρ 11 16:06:38.621145 renegade kernel: RTL8211E Gigabit Ethernet 
stmmac-0:00: attached PHY driver [RTL8211E Gigabit Ethernet] 
(mii_bus:phy_addr=stmmac-0:00, irq=POLL)
Απρ 11 16:06:38.641228 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: No Safety Features support found
Απρ 11 16:06:38.642602 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: PTP not supported by HW
Απρ 11 16:06:45.901192 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
Απρ 11 16:06:45.902237 renegade kernel: IPv6: 
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Απρ 11 16:08:41.781166 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: Link is Down
Απρ 11 16:08:54.791163 renegade kernel: RTL8211E Gigabit Ethernet 
stmmac-0:00: attached PHY driver [RTL8211E Gigabit Ethernet] 
(mii_bus:phy_addr=stmmac-0:00, irq=POLL)
Απρ 11 16:08:54.801388 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: No Safety Features support found
Απρ 11 16:08:54.802780 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: PTP not supported by HW
Απρ 11 16:09:02.061182 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
Απρ 11 16:09:02.062327 renegade kernel: IPv6: 
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Απρ 11 16:09:03.214823 renegade kernel: ------------[ cut here 
]------------
Απρ 11 16:09:03.215300 renegade kernel: refcount_t: underflow; 
use-after-free.
Απρ 11 16:09:03.215472 renegade kernel: WARNING: CPU: 2 PID: 0 at 
lib/refcount.c:190 refcount_sub_and_test_checked+0xc8/0xd8
Απρ 11 16:03:25.240754 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet: TX Checksum insertion supported
Απρ 11 16:03:25.242650 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet: Wake-Up On Lan supported
Απρ 11 16:03:25.244155 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet: Normal descriptors
Απρ 11 16:03:25.247336 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet: Ring mode enabled
Απρ 11 16:03:25.248837 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet: Enable RX Mitigation via HW Watchdog Timer
Απρ 11 16:03:25.351198 renegade kernel: rockchip-drm 
display-subsystem: [drm:rockchip_drm_platform_probe [rockchipdrm]] 
*ERROR* No available vop found for display-subsy>
Απρ 11 16:03:25.391228 renegade kernel: libphy: stmmac: probed
Απρ 11 16:03:25.391708 renegade kernel: RTL8211E Gigabit Ethernet 
stmmac-0:00: attached PHY driver [RTL8211E Gigabit Ethernet] 
(mii_bus:phy_addr=stmmac-0:00, irq=POLL)
Απρ 11 16:03:25.393255 renegade kernel: RTL8211E Gigabit Ethernet 
stmmac-0:01: attached PHY driver [RTL8211E Gigabit Ethernet] 
(mii_bus:phy_addr=stmmac-0:01, irq=POLL)
Απρ 11 16:03:25.541243 renegade kernel: RTL8211E Gigabit Ethernet 
stmmac-0:00: attached PHY driver [RTL8211E Gigabit Ethernet] 
(mii_bus:phy_addr=stmmac-0:00, irq=POLL)
Απρ 11 16:03:25.561202 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: No Safety Features support found
Απρ 11 16:03:25.562458 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: PTP not supported by HW
Απρ 11 16:03:25.563674 renegade kernel: A link change request failed 
with some changes committed already. Interface eth0 may have been left 
with an inconsistent configu>
Απρ 11 16:03:26.841235 renegade kernel: zram: Added device: zram0
Απρ 11 16:03:26.941263 renegade kernel: random: crng init done
Απρ 11 16:03:26.941801 renegade kernel: random: 7 urandom warning(s) 
missed due to ratelimiting
Απρ 11 16:03:27.571191 renegade kernel: phy 
phy-ff450000.syscon:usb2-phy@....0: charger = USB_DCP_CHARGER
Απρ 11 16:03:27.921230 renegade kernel: zram0: detected capacity 
change from 0 to 1025568768
Απρ 11 16:03:28.813838 renegade kernel: Adding 1001528k swap on 
/dev/zram0.  Priority:32767 extents:1 across:1001528k SSDscFS
Απρ 11 16:03:32.861210 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
Απρ 11 16:03:32.862527 renegade kernel: IPv6: 
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Απρ 11 16:05:25.932408 renegade kernel: wireguard: loading 
out-of-tree module taints kernel.
Απρ 11 16:05:25.936144 renegade kernel: wireguard: WireGuard 
0.0.20190406 loaded. See www.wireguard.com for information.
Απρ 11 16:05:25.936467 renegade kernel: wireguard: Copyright (C) 
2015-2019 Jason A. Donenfeld <Jason@...c4.com>. All Rights Reserved.
Απρ 11 16:05:26.451886 renegade kernel: NFSD: starting 90-second 
grace period (net f0000041)
Απρ 11 16:05:27.252030 renegade kernel: nf_conntrack: default 
automatic helper assignment has been turned off for security reasons 
and CT-based  firewall rule not found>
Απρ 11 16:06:11.411210 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: Link is Down
Απρ 11 16:06:38.621145 renegade kernel: RTL8211E Gigabit Ethernet 
stmmac-0:00: attached PHY driver [RTL8211E Gigabit Ethernet] 
(mii_bus:phy_addr=stmmac-0:00, irq=POLL)
Απρ 11 16:06:38.641228 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: No Safety Features support found
Απρ 11 16:06:38.642602 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: PTP not supported by HW
Απρ 11 16:06:45.901192 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
Απρ 11 16:06:45.902237 renegade kernel: IPv6: 
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Απρ 11 16:08:41.781166 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: Link is Down
Απρ 11 16:08:54.791163 renegade kernel: RTL8211E Gigabit Ethernet 
stmmac-0:00: attached PHY driver [RTL8211E Gigabit Ethernet] 
(mii_bus:phy_addr=stmmac-0:00, irq=POLL)
Απρ 11 16:08:54.801388 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: No Safety Features support found
Απρ 11 16:08:54.802780 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: PTP not supported by HW
Απρ 11 16:09:02.061182 renegade kernel: rk_gmac-dwmac 
ff540000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
Απρ 11 16:09:02.062327 renegade kernel: IPv6: 
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Απρ 11 16:09:03.214823 renegade kernel: ------------[ cut here 
]------------
Απρ 11 16:09:03.215300 renegade kernel: refcount_t: underflow; 
use-after-free.
Απρ 11 16:09:03.215472 renegade kernel: WARNING: CPU: 2 PID: 0 at 
lib/refcount.c:190 refcount_sub_and_test_checked+0xc8/0xd8
Απρ 11 16:09:03.215603 renegade kernel: Modules linked in: 
iptable_nat ipt_MASQUERADE wireguard(O) ip6_udp_tunnel udp_tunnel lz4 
lz4_compress zram realtek rockchipdrm a>
Απρ 11 16:09:03.215845 renegade kernel: CPU: 2 PID: 0 Comm: 
swapper/2 Tainted: G           O      5.1.0-rc4-1-ARCH #1
Απρ 11 16:09:03.215959 renegade kernel: Hardware name: Firefly 
roc-rk3328-cc (DT)
Απρ 11 16:09:03.216060 renegade kernel: pstate: 80000005 (Nzcv daif 
-PAN -UAO)
Απρ 11 16:09:03.216160 renegade kernel: pc : 
refcount_sub_and_test_checked+0xc8/0xd8
Απρ 11 16:09:03.216256 renegade kernel: lr : 
refcount_sub_and_test_checked+0xc8/0xd8
Απρ 11 16:09:03.216344 renegade kernel: sp : ffff000010013ca0
Απρ 11 16:09:03.216434 renegade kernel: x29: ffff000010013ca0 x28: 
ffff80003406c8c0
Απρ 11 16:09:03.216550 renegade kernel: x27: ffff80002792d100 x26: 
ffff80003406c8c0
Απρ 11 16:09:03.216648 renegade kernel: x25: 0000000000000060 x24: 
0000000000000000
Απρ 11 16:09:03.216734 renegade kernel: x23: 000000000000014b x22: 
0000000000000001
Απρ 11 16:09:03.216826 renegade kernel: x21: 0000000000000004 x20: 
0000000000000000
Απρ 11 16:09:03.216922 renegade kernel: x19: ffff80002792d100 x18: 
0000000000000010
Απρ 11 16:09:03.217014 renegade kernel: x17: 0000000000000000 x16: 
0000000000000000
Απρ 11 16:09:03.217109 renegade kernel: x15: ffffffffffffffff x14: 
0720072007200720
Απρ 11 16:09:03.217197 renegade kernel: x13: 0720072007200720 x12: 
0720072007200720
Απρ 11 16:09:03.217283 renegade kernel: x11: 0720072007200720 x10: 
0720072007200720
Απρ 11 16:09:03.217427 renegade kernel: x9 : 0720072007200720 x8 : 
072007200720072e
Απρ 11 16:09:03.217537 renegade kernel: x7 : 0000000000000184 x6 : 
0000000000000001
Απρ 11 16:09:03.217637 renegade kernel: x5 : 0000000000000000 x4 : 
0000000000000001
Απρ 11 16:09:03.217740 renegade kernel: x3 : 0000000000000007 x2 : 
0000000000000007
Απρ 11 16:09:03.217833 renegade kernel: x1 : 576938cacdaf1700 x0 : 
0000000000000000
Απρ 11 16:09:03.217928 renegade kernel: Call trace:
Απρ 11 16:09:03.218016 renegade kernel:  
refcount_sub_and_test_checked+0xc8/0xd8

> If it's a queue timeout then please share your "dmesg | grep -i 
> stmmac"
> since boot.

The only time I see it mentioned is basically when the link is brought 
up

$ dmesg | grep -i stmmac
[  +0,134955] libphy: stmmac: probed
[  +0,000352] RTL8211E Gigabit Ethernet stmmac-0:00: attached PHY 
driver [RTL8211E Gigabit Ethernet] (mii_bus:phy_addr=stmmac-0:00, 
irq=POLL)
[  +0,001253] RTL8211E Gigabit Ethernet stmmac-0:01: attached PHY 
driver [RTL8211E Gigabit Ethernet] (mii_bus:phy_addr=stmmac-0:01, 
irq=POLL)
[  +0,256702] RTL8211E Gigabit Ethernet stmmac-0:00: attached PHY 
driver [RTL8211E Gigabit Ethernet] (mii_bus:phy_addr=stmmac-0:00, 
irq=POLL)

> This is not a workaround neither an issue. It's well stablished that 
> PBL
> setting interferes with COE so one must choose a setting that depends 
> on
> FIFO size. I would like to make it automatic in the driver but I 
> didn't
> have the time to submit a patch yet, sorry.
> 
> Thanks,
> Jose Miguel Abreu

When you say FIFO size, is that related to the tx/rx_delay?

Powered by blists - more mailing lists