Date:   Mon, 20 May 2019 07:42:40 +0000
From:   Yoshihiro Shimoda <yoshihiro.shimoda.uh@...esas.com>
To:     Sergei Shtylyov <sergei.shtylyov@...entembedded.com>,
        "davem@...emloft.net" <davem@...emloft.net>
CC:     "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "linux-renesas-soc@...r.kernel.org" 
        <linux-renesas-soc@...r.kernel.org>
Subject: RE: [PATCH] net: sh_eth: fix mdio access in sh_eth_close() for some
 SoCs

Hello Sergei-san,

Thank you for the reply! And, I'm sorry for the delayed response because I had days off in last week.

> From: Sergei Shtylyov, Sent: Thursday, May 16, 2019 12:41 AM
> 
> Hello!
> 
>    It's not "some SoCs", it's only R-Car gen2 and RZ/G1 SoCs.

Do you prefer the subject "net: sh_eth: fix mdio access in sh_eth_close() for R-Car gen2 and RZ/G1 SoCs"
instead of "... some SoCs"?

> On 05/15/2019 08:36 AM, Yoshihiro Shimoda wrote:
> 
> > The sh_eth_close() resets the MAC and then calls phy_stop()
> > so that mdio read access result is incorrect without any error
> > according to kernel trace like below:
> >
> > ifconfig-216   [003] .n..   109.133124: mdio_access: ee700000.ethernet-ffffffff read  phy:0x01 reg:0x00 val:0xffff
> 
>    Not sure how RMII mode affects the MDIO transfers...

I should have described what happen on the previous code.
According to the hardware manual, the RMII mode should be set to 1 before
operation the Ethernet MAC. The MDIO transferring is one of the Ethernet MAC operation.
So, I assumed this RMII mode affects the MDIO transfers.

By the way, I also should have described what happen on the previous code.
If I input the following command on non-NFS rootfs environment (e.g. USB memory rootfs),
sometimes, a lot of FCD interruption happened and then the following "rcu "error happened.
---
# ifconfig eth0 down# ifconfig eth0 up
[  794.545823] Micrel KSZ8041RNLI ee700000.ethernet-ffffffff:01: attached PHY driver [Micrel KSZ8041RNLI] (mii_bus:phy_addr=ee700000.ethernet-ffffffff:01, irq=157)
# ifconfig eth0 up# ifconfig eth0 down
# ifconfig eth0 down# ifconfig eth0 up
[  797.662138] Micrel KSZ8041RNLI ee700000.ethernet-ffffffff:01: attached PHY driver [Micrel KSZ8041RNLI] (mii_bus:phy_addr=ee700000.ethernet-ffffffff:01, irq=157)
[  797.689158] sh-eth ee700000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
[  797.702631] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
# [  818.759628] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[  818.770827] rcu:     0-....: (2 GPs behind) idle=d12/0/0x3 softirq=388/388 fqs=1051
[  818.783635] rcu:     (detected by 1, t=2104 jiffies, g=377, q=18)
[  818.794848] Sending NMI from CPU 1 to CPUs 0:
[  818.804733] NMI backtrace for cpu 0
[  818.804736] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.1.0-00002-g192664e-dirty #27
[  818.804738] Hardware name: Generic R-Car Gen2 (Flattened Device Tree)
[  818.804740] PC is at __do_softirq+0x7c/0x344
[  818.804742] LR is at irq_exit+0x6c/0xcc
[  818.804744] pc : [<c01021e4>]    lr : [<c0126aa0>]    psr: 60030113
[  818.804746] sp : c0d01eb8  ip : 00200102  fp : c0d03080
[  818.804748] r10: 00000001  r9 : 0000000a  r8 : c0ca2940
[  818.804750] r7 : 00000004  r6 : c0ca2934  r5 : 00000000  r4 : ffffe000
[  818.804752] r3 : c0ca2940  r2 : 2aaa8000  r1 : 00000000  r0 : 2aaa8000
[  818.804754] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[  818.804756] Control: 10c5387d  Table: 6a5f006a  DAC: 00000051
[  818.804758] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.1.0-00002-g192664e-dirty #27
[  818.804760] Hardware name: Generic R-Car Gen2 (Flattened Device Tree)
[  818.804762] [<c011049c>] (unwind_backtrace) from [<c010bc0c>] (show_stack+0x18/0x1c)
[  818.804764] [<c010bc0c>] (show_stack) from [<c07f7458>] (dump_stack+0x7c/0x9c)
[  818.804766] [<c07f7458>] (dump_stack) from [<c07fd358>] (nmi_cpu_backtrace+0x98/0xb8)
[  818.804768] [<c07fd358>] (nmi_cpu_backtrace) from [<c010e6bc>] (handle_IPI+0x2bc/0x33c)
[  818.804770] [<c010e6bc>] (handle_IPI) from [<c03fa084>] (gic_handle_irq+0x8c/0x98)
[  818.804773] [<c03fa084>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0x90)
[  818.804774] Exception stack(0xc0d01e68 to 0xc0d01eb0)
[  818.804777] 1e60:                   2aaa8000 00000000 2aaa8000 c0ca2940 ffffe000 00000000
[  818.804779] 1e80: c0ca2934 00000004 c0ca2940 0000000a 00000001 c0d03080 00200102 c0d01eb8
[  818.804781] 1ea0: c0126aa0 c01021e4 60030113 ffffffff
[  818.804783] [<c0101a8c>] (__irq_svc) from [<c01021e4>] (__do_softirq+0x7c/0x344)
[  818.804785] [<c01021e4>] (__do_softirq) from [<c0126aa0>] (irq_exit+0x6c/0xcc)
[  818.804787] [<c0126aa0>] (irq_exit) from [<c016c344>] (__handle_domain_irq+0x88/0xbc)
[  818.804789] [<c016c344>] (__handle_domain_irq) from [<c03fa058>] (gic_handle_irq+0x60/0x98)
[  818.804791] [<c03fa058>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0x90)
[  818.804793] Exception stack(0xc0d01f38 to 0xc0d01f80)
[  818.804795] 1f20:                                                       00000000 0008bbb8
[  818.804798] 1f40: eb74b520 c011aee0 00000001 ffffe000 c0d08c30 c0d8e5be c0d08c74 c0d980c0
[  818.804800] 1f60: 00000001 c0c5ea40 00000000 c0d01f88 c01084ec c014c85c 60030013 ffffffff
[  818.804802] [<c0101a8c>] (__irq_svc) from [<c014c85c>] (do_idle+0xf0/0x13c)
[  818.804804] [<c014c85c>] (do_idle) from [<c014cb10>] (cpu_startup_entry+0x20/0x24)
[  818.804806] [<c014cb10>] (cpu_startup_entry) from [<c0c00fac>] (start_kernel+0x3e0/0x488)

Note:
 - After I input the second ifconfig eth0 up, I cannot input any command.
 - If I disconnect an ethernet cable, I can input command again.
---

> > To fix the issue, this patch adds a condition and set the RMII mode
> > regiseter in sh_eth_dev_exit() for some SoCs.
> >
> > Note that when I have tried to move the sh_eth_dev_exit() calling
> > after phy_stop() on sh_eth_close(), but it gets worse.
> 
>    Ah, I was going to suggest changing the call order... what happens then?

Sometimes, the following error happened and then I cannot recover the system
even if I disconnected an ethernet cable. Do you prefer this way, instead of
adding RMII mode setting on sh_eth_dev_exit()?
---
# ifconfig eth0 up;
[   64.896021] Micrel KSZ8041RNLI ee700000.ethernet-ffffffff:01: attached PHY driver [Micrel KSZ8041RNLI] (mii_bus:phy_addr=ee700000.ethernet-ffffffff:01, irq=157)
# [   66.868241] sh-eth ee700000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
[   66.881777] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
# ifconfig eth0 down
[   71.164900] sh-eth ee700000.ethernet eth0: Link is Down
[   92.169767] rcu: INFO: rcu_sched self-detected stall on CPU
[   92.180654] rcu: 	0-....: (2100 ticks this GP) idle=9d6/1/0x40000004 softirq=513/513 fqs=1050 
[   92.194720] rcu: 	 (t=2102 jiffies g=-367 q=46)
[   92.204723] NMI backtrace for cpu 0
[   92.213705] CPU: 0 PID: 228 Comm: ifconfig Not tainted 5.1.0-00002-g192664e-dirty #28
[   92.227149] Hardware name: Generic R-Car Gen2 (Flattened Device Tree)
[   92.239258] [<c011049c>] (unwind_backtrace) from [<c010bc0c>] (show_stack+0x18/0x1c)
[   92.252683] [<c010bc0c>] (show_stack) from [<c07f7458>] (dump_stack+0x7c/0x9c)
[   92.265548] [<c07f7458>] (dump_stack) from [<c07fd360>] (nmi_cpu_backtrace+0xa0/0xb8)
[   92.279039] [<c07fd360>] (nmi_cpu_backtrace) from [<c07fd3fc>] (nmi_trigger_cpumask_backtrace+0x84/0x114)
[   92.294384] [<c07fd3fc>] (nmi_trigger_cpumask_backtrace) from [<c01827e4>] (rcu_dump_cpu_stacks+0xb4/0xd0)
[   92.309893] [<c01827e4>] (rcu_dump_cpu_stacks) from [<c0180cc8>] (rcu_sched_clock_irq+0x2cc/0x6f0)
[   92.324744] [<c0180cc8>] (rcu_sched_clock_irq) from [<c0189170>] (update_process_times+0x38/0x64)
[   92.339528] [<c0189170>] (update_process_times) from [<c019a5d0>] (tick_nohz_handler+0xd4/0x128)
[   92.354261] [<c019a5d0>] (tick_nohz_handler) from [<c06573a4>] (arch_timer_handler_virt+0x30/0x38)
[   92.369207] [<c06573a4>] (arch_timer_handler_virt) from [<c0171548>] (handle_percpu_devid_irq+0xf0/0x224)
[   92.384817] [<c0171548>] (handle_percpu_devid_irq) from [<c016c2ac>] (generic_handle_irq+0x20/0x30)
[   92.399910] [<c016c2ac>] (generic_handle_irq) from [<c016c364>] (__handle_domain_irq+0xa8/0xbc)
[   92.414678] [<c016c364>] (__handle_domain_irq) from [<c03fa058>] (gic_handle_irq+0x60/0x98)
[   92.429128] [<c03fa058>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0x90)
[   92.442748] Exception stack(0xea1f9b50 to 0xea1f9b98)
[   92.453924] 9b40:                                     2aaa8000 00000000 2aaa8000 c0ca2940
[   92.468345] 9b60: ffffe000 00000000 c0ca2934 00000002 c0ca2940 0000000a 00000000 c0d03080
[   92.482764] 9b80: 00404100 ea1f9ba0 c0126aa0 c01021e4 60030113 ffffffff
[   92.495519] [<c0101a8c>] (__irq_svc) from [<c01021e4>] (__do_softirq+0x7c/0x344)
[   92.508973] [<c01021e4>] (__do_softirq) from [<c0126aa0>] (irq_exit+0x6c/0xcc)
[   92.522215] [<c0126aa0>] (irq_exit) from [<c016c344>] (__handle_domain_irq+0x88/0xbc)
[   92.536092] [<c016c344>] (__handle_domain_irq) from [<c03fa058>] (gic_handle_irq+0x60/0x98)
[   92.550513] [<c03fa058>] (gic_handle_irq) from [<c0101a8c>] (__irq_svc+0x6c/0x90)
[   92.564095] Exception stack(0xea1f9c20 to 0xea1f9c68)
[   92.575129] 9c20: f0911320 00000001 00000000 f0911320 00000003 eabb7844 00000001 00000000
[   92.589271] 9c40: 00000000 00003800 00000000 ea1f9e50 ea1f9d24 ea1f9c70 c0555d80 c055c590
[   92.603319] 9c60: 60030013 ffffffff
[   92.612529] [<c0101a8c>] (__irq_svc) from [<c055c590>] (sh_mdio_ctrl+0x3c/0x58)
[   92.625531] [<c055c590>] (sh_mdio_ctrl) from [<c0555d80>] (mdiobb_send_num+0x28/0x34)
[   92.638971] [<c0555d80>] (mdiobb_send_num) from [<c0555f10>] (mdiobb_write+0x70/0x94)
[   92.652337] [<c0555f10>] (mdiobb_write) from [<c05555a4>] (__mdiobus_write+0x64/0x10c)
[   92.665688] [<c05555a4>] (__mdiobus_write) from [<c05526e8>] (__phy_modify_changed+0x58/0x6c)
[   92.679587] [<c05526e8>] (__phy_modify_changed) from [<c0552760>] (__phy_modify+0x10/0x18)
[   92.693127] [<c0552760>] (__phy_modify) from [<c05527a4>] (phy_modify+0x3c/0x54)
[   92.705749] [<c05527a4>] (phy_modify) from [<c0553b14>] (phy_suspend+0xb0/0xd0)
[   92.718274] [<c0553b14>] (phy_suspend) from [<c0551118>] (phy_state_machine+0x1a4/0x1a8)
[   92.731589] [<c0551118>] (phy_state_machine) from [<c05511a8>] (phy_stop+0x8c/0xb8)
[   92.744411] [<c05511a8>] (phy_stop) from [<c055e2d4>] (sh_eth_close+0x58/0xa0)
[   92.756668] [<c055e2d4>] (sh_eth_close) from [<c06ce740>] (__dev_close_many+0xc0/0xdc)
[   92.769620] [<c06ce740>] (__dev_close_many) from [<c06d4f9c>] (__dev_change_flags+0xdc/0x1b0)
[   92.783186] [<c06d4f9c>] (__dev_change_flags) from [<c06d5090>] (dev_change_flags+0x20/0x50)
[   92.796660] [<c06d5090>] (dev_change_flags) from [<c074d740>] (devinet_ioctl+0x2d8/0x5dc)
[   92.809892] [<c074d740>] (devinet_ioctl) from [<c074f8f8>] (inet_ioctl+0x260/0x324)
[   92.822560] [<c074f8f8>] (inet_ioctl) from [<c06b2ea4>] (sock_ioctl+0x148/0x424)
[   92.834968] [<c06b2ea4>] (sock_ioctl) from [<c0252e00>] (vfs_ioctl+0x28/0x3c)
[   92.847110] [<c0252e00>] (vfs_ioctl) from [<c0253614>] (do_vfs_ioctl+0x98/0x844)
[   92.859518] [<c0253614>] (do_vfs_ioctl) from [<c0253e00>] (ksys_ioctl+0x40/0x5c)
[   92.871875] [<c0253e00>] (ksys_ioctl) from [<c0101000>] (ret_fast_syscall+0x0/0x54)
[   92.884498] Exception stack(0xea1f9fa8 to 0xea1f9ff0)
[   92.894477] 9fa0:                   000ab853 bec4fc90 00000003 00008914 bec4fc90 000ab853
[   92.907648] 9fc0: 000ab853 bec4fc90 bec4ff52 00000036 bec4fe48 000000b8 00096710 00000000
[   92.920805] 9fe0: 000ca1e4 bec4fc2c 0001aa8c b6e72f3c
---

> > Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@...esas.com>
> > ---
> >  drivers/net/ethernet/renesas/sh_eth.c | 4 ++++
> >  1 file changed, 4 insertions(+)
> >
> > diff --git a/drivers/net/ethernet/renesas/sh_eth.c b/drivers/net/ethernet/renesas/sh_eth.c
> > index e33af37..106ae90 100644
> > --- a/drivers/net/ethernet/renesas/sh_eth.c
> > +++ b/drivers/net/ethernet/renesas/sh_eth.c
> > @@ -1596,6 +1596,10 @@ static void sh_eth_dev_exit(struct net_device *ndev)
> >
> >  	/* Set MAC address again */
> >  	update_mac_address(ndev);
> > +
> > +	/* Set the mode again if required */
> 
>    Should be "RMII mode", n ot just "Mode". We prolly need more detailed explanation...

I got it.

Best regards,
Yoshihiro Shimoda

> > +	if (mdp->cd->rmiimode)
> > +		sh_eth_write(ndev, 0x1, RMIIMODE);
> >  }
> >
> >  static void sh_eth_rx_csum(struct sk_buff *skb)
> 
> MBR, Sergei

Powered by blists - more mailing lists