Message-ID:
 <TY3PR01MB1134624A76189BF079F1CE82186C82@TY3PR01MB11346.jpnprd01.prod.outlook.com>
Date: Tue, 4 Mar 2025 10:56:39 +0000
From: Biju Das <biju.das.jz@...renesas.com>
To: Russell King <linux@...linux.org.uk>
CC: "Lad, Prabhakar" <prabhakar.csengg@...il.com>, Andrew Lunn
	<andrew+netdev@...n.ch>, "David S. Miller" <davem@...emloft.net>, Eric
 Dumazet <edumazet@...gle.com>, Jakub Kicinski <kuba@...nel.org>, Paolo Abeni
	<pabeni@...hat.com>, Rob Herring <robh@...nel.org>, Krzysztof Kozlowski
	<krzk+dt@...nel.org>, Conor Dooley <conor+dt@...nel.org>, Philipp Zabel
	<p.zabel@...gutronix.de>, Geert Uytterhoeven <geert+renesas@...der.be>,
	Giuseppe Cavallaro <peppe.cavallaro@...com>, Jose Abreu
	<joabreu@...opsys.com>, Alexandre Torgue <alexandre.torgue@...s.st.com>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	"devicetree@...r.kernel.org" <devicetree@...r.kernel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-renesas-soc@...r.kernel.org" <linux-renesas-soc@...r.kernel.org>,
	Fabrizio Castro <fabrizio.castro.jz@...esas.com>, Prabhakar Mahadev Lad
	<prabhakar.mahadev-lad.rj@...renesas.com>
Subject: RE: [PATCH 3/3] net: stmmac: Add DWMAC glue layer for Renesas GBETH



> -----Original Message-----
> From: Russell King <linux@...linux.org.uk>
> Sent: 04 March 2025 10:00
> To: Biju Das <biju.das.jz@...renesas.com>
> Cc: Lad, Prabhakar <prabhakar.csengg@...il.com>; Andrew Lunn <andrew+netdev@...n.ch>; David S. Miller
> <davem@...emloft.net>; Eric Dumazet <edumazet@...gle.com>; Jakub Kicinski <kuba@...nel.org>; Paolo
> Abeni <pabeni@...hat.com>; Rob Herring <robh@...nel.org>; Krzysztof Kozlowski <krzk+dt@...nel.org>;
> Conor Dooley <conor+dt@...nel.org>; Philipp Zabel <p.zabel@...gutronix.de>; Geert Uytterhoeven
> <geert+renesas@...der.be>; Giuseppe Cavallaro <peppe.cavallaro@...com>; Jose Abreu
> <joabreu@...opsys.com>; Alexandre Torgue <alexandre.torgue@...s.st.com>; netdev@...r.kernel.org;
> devicetree@...r.kernel.org; linux-kernel@...r.kernel.org; linux-renesas-soc@...r.kernel.org; Fabrizio
> Castro <fabrizio.castro.jz@...esas.com>; Prabhakar Mahadev Lad <prabhakar.mahadev-
> lad.rj@...renesas.com>
> Subject: Re: [PATCH 3/3] net: stmmac: Add DWMAC glue layer for Renesas GBETH
> 
> On Tue, Mar 04, 2025 at 06:58:44AM +0000, Biju Das wrote:
> > Hi Russel king,
> >
> > > -----Original Message-----
> > > From: Russell King <linux@...linux.org.uk>
> > > Sent: 03 March 2025 11:19
> > > Subject: Re: [PATCH 3/3] net: stmmac: Add DWMAC glue layer for
> > > Renesas GBETH
> > >
> > > On Sun, Mar 02, 2025 at 10:02:15PM +0000, Lad, Prabhakar wrote:
> > > > Hi Russell,
> > > >
> > > > On Sun, Mar 2, 2025 at 9:44 PM Russell King (Oracle)
> > > > <linux@...linux.org.uk> wrote:
> > > > >
> > > > > On Sun, Mar 02, 2025 at 09:20:49PM +0000, Lad, Prabhakar wrote:
> > > > > > Hi Russell,
> > > > > > > What is the reason for setting this flag? If it's because of
> > > > > > > suspend/ resume failures, does my "net: stmmac: fix resume
> > > > > > > failures due to RX clock" series solve this for you without requiring this flag?
> > > > > > >
> > > > > > Ive set this flag based on the configuration supported by this IP.
> > > > > > Unfortunately the platform which I am working on doesn't
> > > > > > support s2r yet so I cannot test suspend/resume path yet. But
> > > > > > I do see an issue when I unload and load just the glue module the DMA reset fails.
> > > > >
> > > > > Thanks for that feedback - that's a scenario I hadn't considered.
> > > > >
> > > > > I was trying to avoid having to disable LPI RX clock-stop on
> > > > > suspend by ensuring that it was enabled at resume time. I think
> > > > > that's valid, but you've brought up another similar scenario:
> > > > >
> > > > > - device is brought up, configures RX clock stop
> > > > > - links with media, negotiates EEE
> > > > > - driver is unloaded, link doesn't go down, but due to no traffic goes
> > > > >   into idle, so RX clock is stopped
> > > > > - driver reloaded, RX clock still stopped, reset fails
> > > > >
> > > > > I would like to solve that so we can get the power savings from
> > > > > stopping the clock, but still have reset work when necessary.
> > > > >
> > > > I would be happy to test the patches ;)
> > > >
> > > > > I'm guessing that the "DMA reset fails" refers to this path:
> > > > >
> > > > > stmmac_open() -> __stmmac_open() -> stmmac_hw_setup() ->
> > > > > stmmac_init_dma_engine() -> stmmac_reset() ?
> > > > >
> > > > Yes.
> > > >
> > > > > In other words, when the device is being brought back up
> > > > > adminsitratively?
> > > > >
> > > > > What happens if you (replace $if):
> > > > >
> > > > > # ip li set dev $if down
> > > > > # ip li set dev $if up
> > > > >
> > > > > Does that also fail without STMMAC_FLAG_RX_CLK_RUNS_IN_LPI set?
> > > > >
> > > > Logs without STMMAC_FLAG_RX_CLK_RUNS_IN_LPI flag set:
> > > > --------------------------------------------------------------
> > > > root@...2h-evk-alpha:~# ip li set dev eth1 down
> > > > [   33.606549] renesas-gbeth 15c40000.ethernet eth1: Link is Down
> > > > root@...2h-evk-alpha:~#
> > > > root@...2h-evk-alpha:~# ip li set dev eth0 down
> > > > [   37.356992] renesas-gbeth 15c30000.ethernet eth0: Link is Down
> > > > root@...2h-evk-alpha:~#
> > > > root@...2h-evk-alpha:~# ip li set dev eth1 up
> > > > [   43.974803] renesas-gbeth 15c40000.ethernet eth1: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-0
> > > > [   43.983189] renesas-gbeth 15c40000.ethernet eth1: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-1
> > > > [   43.991155] renesas-gbeth 15c40000.ethernet eth1: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-2
> > > > [   43.999128] renesas-gbeth 15c40000.ethernet eth1: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-3
> > > > [   44.072079] renesas-gbeth 15c40000.ethernet eth1: PHY [stmmac-1:00]
> > > > driver [Microchip KSZ9131 Gigabit PHY] (irq=POLL)
> > > > [   44.094605] dwmac4: Master AXI performs fixed burst length
> > > > [   44.100138] renesas-gbeth 15c40000.ethernet eth1: No Safety
> > > > Features support found
> > > > [   44.107748] renesas-gbeth 15c40000.ethernet eth1: IEEE 1588-2008
> > > > Advanced Timestamp supported
> > > > [   44.116725] renesas-gbeth 15c40000.ethernet eth1: registered PTP clock
> > > > [   44.123352] renesas-gbeth 15c40000.ethernet eth1: configuring for
> > > > phy/rgmii-id link mode
> > > > root@...2h-evk-alpha:~#
> > > > root@...2h-evk-alpha:~# ip li set dev eth1[   47.207761] renesas-gbeth
> > > > 15c40000.ethernet eth1: Link is Up - 1Gbps/Full - flow control off
> > > > ^C root@...2h-evk-alpha:~# ^C root@...2h-evk-alpha:~# ip li set
> > > > dev eth0 up
> > > > [   55.636722] renesas-gbeth 15c30000.ethernet eth0: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-0
> > > > [   55.645139] renesas-gbeth 15c30000.ethernet eth0: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-1
> > > > [   55.653111] renesas-gbeth 15c30000.ethernet eth0: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-2
> > > > [   55.661073] renesas-gbeth 15c30000.ethernet eth0: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-3
> > > > [   55.732087] renesas-gbeth 15c30000.ethernet eth0: PHY [stmmac-0:00]
> > > > driver [Microchip KSZ9131 Gigabit PHY] (irq=POLL)
> > > > [   55.754612] dwmac4: Master AXI performs fixed burst length
> > > > [   55.760143] renesas-gbeth 15c30000.ethernet eth0: No Safety
> > > > Features support found
> > > > [   55.767740] renesas-gbeth 15c30000.ethernet eth0: IEEE 1588-2008
> > > > Advanced Timestamp supported
> > > > [   55.776705] renesas-gbeth 15c30000.ethernet eth0: registered PTP clock
> > > > [   55.783333] renesas-gbeth 15c30000.ethernet eth0: configuring for
> > > > phy/rgmii-id link mode
> > > > root@...2h-evk-alpha:~#
> > > > root@...2h-evk-alpha:~# [   58.855844] renesas-gbeth 15c30000.ethernet
> > > > eth0: tx_clk_stop=1
> > > > [   58.861989] renesas-gbeth 15c30000.ethernet eth0: Link is Up -
> > > > 1Gbps/Full - flow control rx/tx
> > > >
> > > > root@...2h-evk-alpha:~#
> > > > root@...2h-evk-alpha:~#
> > > >
> > > > Logs with STMMAC_FLAG_RX_CLK_RUNS_IN_LPI flag set:
> > > > --------------------------------------------------------------
> > > > root@...2h-evk-alpha:~# ip li set dev eth1 down
> > > > [   30.053790] renesas-gbeth 15c40000.ethernet eth1: Link is Down
> > > > root@...2h-evk-alpha:~# ip li set dev eth0 down
> > > > [   35.366935] renesas-gbeth 15c30000.ethernet eth0: Link is Down
> > > > root@...2h-evk-alpha:~# ip li set dev eth1 up
> > > > [   40.448563] renesas-gbeth 15c40000.ethernet eth1: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-0
> > > > [   40.456725] renesas-gbeth 15c40000.ethernet eth1: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-1
> > > > [   40.464893] renesas-gbeth 15c40000.ethernet eth1: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-2
> > > > [   40.472840] renesas-gbeth 15c40000.ethernet eth1: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-3
> > > > [   40.543895] renesas-gbeth 15c40000.ethernet eth1: PHY [stmmac-1:00]
> > > > driver [Microchip KSZ9131 Gigabit PHY] (irq=POLL)
> > > > [   40.566419] dwmac4: Master AXI performs fixed burst length
> > > > [   40.571949] renesas-gbeth 15c40000.ethernet eth1: No Safety
> > > > Features support found
> > > > [   40.579550] renesas-gbeth 15c40000.ethernet eth1: IEEE 1588-2008
> > > > Advanced Timestamp supported
> > > > [   40.588505] renesas-gbeth 15c40000.ethernet eth1: registered PTP clock
> > > > [   40.595135] renesas-gbeth 15c40000.ethernet eth1: configuring for
> > > > phy/rgmii-id link mode
> > > > root@...2h-evk-alpha:~#
> > > > root@...2h-evk-alpha:~# [   43.687551] renesas-gbeth 15c40000.ethernet
> > > > eth1: Link is Up - 1Gbps/Full - flow control off
> > > >
> > > > root@...2h-evk-alpha:~# ip li set dev eth0 up
> > > > [   49.644479] renesas-gbeth 15c30000.ethernet eth0: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-0
> > > > [   49.652719] renesas-gbeth 15c30000.ethernet eth0: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-1
> > > > [   49.660681] renesas-gbeth 15c30000.ethernet eth0: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-2
> > > > [   49.669059] renesas-gbeth 15c30000.ethernet eth0: Register
> > > > MEM_TYPE_PAGE_POOL RxQ-3
> > > > [   49.740011] renesas-gbeth 15c30000.ethernet eth0: PHY [stmmac-0:00]
> > > > driver [Microchip KSZ9131 Gigabit PHY] (irq=POLL)
> > > > [   49.762518] dwmac4: Master AXI performs fixed burst length
> > > > [   49.768057] renesas-gbeth 15c30000.ethernet eth0: No Safety
> > > > Features support found
> > > > [   49.775655] renesas-gbeth 15c30000.ethernet eth0: IEEE 1588-2008
> > > > Advanced Timestamp supported
> > > > [   49.784609] renesas-gbeth 15c30000.ethernet eth0: registered PTP clock
> > > > [   49.791236] renesas-gbeth 15c30000.ethernet eth0: configuring for
> > > > phy/rgmii-id link mode
> > > > root@...2h-evk-alpha:~#
> > > > root@...2h-evk-alpha:~# [   52.871635] renesas-gbeth 15c30000.ethernet
> > > > eth0: tx_clk_stop=1
> > > > [   52.877777] renesas-gbeth 15c30000.ethernet eth0: Link is Up -
> > > > 1Gbps/Full - flow control rx/tx
> > >
> > down/upping the interface but it seems you get different behaviour.
> > >
> > > I'd like to understand why that is, because at the moment I'm
> > > wondering whether my patches that address the suspend/resume need
> > > further work before I send them - but in order to assess that, I
> > > need to work out why your issue only seems to occur in the module removal/insertion and not
> down/up as well as I'd expect.
> >
> > FYI, With linux next, On RZ/G3E SoC which has similar IP as RZ/V2H,i
> > ethernet works during suspend entry/exit Even though STR is not fully
> > functional yet.
> 
> For the failure to happen, you need to check whether EEE is being used:
> 
> # ethtool --show-eee ethX
> 
> and check whether it states that EEE is enabled and active, and Tx LPI also shows the timer value.
> 
> You need a PHY that does stop it's receive clock when the link enters low-power mode. PHYs are not
> required to have this ability implemented, and there's no way for software to know whether it is or
> not.
> 
> Then, you need to be certain that your link partner does actually support EEE and signals LPI from its
> side, rather than just advertising EEE. Lastly, you need to ensure that there is no traffic over the
> cable when you're resuming for the period of the reset timeout for the failure to occur. If the link
> wakes up, the clock will be started and reset will complete.
> 
> One can rule out some of the above by checking the LPI status bits, either in the DWMAC or PHY which
> indicates whether transmit and/or receive seeing LPI signalled.
> 
> If the link doesn't enter low power, then the receive clock won't be stopped, and reset will complete.
> If the link wakes up during reset, then the clock will be restarted, and reset will complete before
> the timeout expires.
> 
> So, the possibility for a successful test is quite high.


This what I get on next. It is showing enabled , but inactive.

root@...rc-rzg3e:~# ethtool --show-eee eth0
EEE settings for eth0:
        EEE status: enabled - inactive
        Tx LPI: 1000000 (us)
        Supported EEE link modes:  100baseT/Full
                                   1000baseT/Full
        Advertised EEE link modes:  100baseT/Full
                                    1000baseT/Full
        Link partner advertised EEE link modes:  Not reported
root@...rc-rzg3e:~# [ 4255.568190] PM: suspend entry (s2idle)
[ 4255.572235] Filesystems sync: 0.000 seconds
[ 4255.577786] Freezing user space processes
[ 4255.585187] Freezing user space processes completed (elapsed 0.003 seconds)
[ 4255.592143] OOM killer disabled.
[ 4255.595375] Freezing remaining freezable tasks
[ 4255.600907] Freezing remaining freezable tasks completed (elapsed 0.001 seconds)
[ 4255.608297] printk: Suspending console(s) (use no_console_suspend to debug)
ÿ[ 4255.657045] renesas-gbeth 15c30000.ethernet eth0: Link is Down

...
...
[ 4255.659896] PM: Some devices failed to suspend, or early wake event detected
[ 4255.661252] renesas-gbeth 15c30000.ethernet eth0: configuring for phy/rgmii-id link mode
[ 4255.665681] dwmac4: Master AXI performs fixed burst length
[ 4255.665763] renesas-gbeth 15c30000.ethernet eth0: No Safety Features support found
[ 4255.665823] renesas-gbeth 15c30000.ethernet eth0: IEEE 1588-2008 Advanced Timestamp supported
[ 4255.776289] OOM killer enabled.
[ 4255.779431] Restarting tasks ... done.
[ 4255.783936] random: crng reseeded on system resumption
[ 4255.789595] PM: suspend exit
[ 4255.793070] PM: suspend entry (s2idle)
[ 4255.797174] Filesystems sync: 0.000 seconds
[ 4255.801734] Freezing user space processes
[ 4255.806464] Freezing user space processes completed (elapsed 0.004 seconds)
[ 4255.813431] OOM killer disabled.
[ 4255.816644] Freezing remaining freezable tasks
[ 4256.055857] Freezing remaining freezable tasks completed (elapsed 0.234 seconds)
[ 4256.063427] printk: Suspending console(s) (use no_console_suspend to debug)
[ 4256.101633] gpio-keys keys: failed to configure IRQ 51 as wakeup source: -38
[ 4256.101683] gpio-keys keys: PM: dpm_run_callback(): platform_pm_suspend returns -38
[ 4256.101755] gpio-keys keys: PM: failed to suspend: error -38
[ 4256.101792] PM: Some devices failed to suspend, or early wake event detected
[ 4256.103095] renesas-gbeth 15c30000.ethernet eth0: configuring for phy/rgmii-id link mode
[ 4256.107110] dwmac4: Master AXI performs fixed burst length
[ 4256.107192] renesas-gbeth 15c30000.ethernet eth0: No Safety Features support found
[ 4256.107254] renesas-gbeth 15c30000.ethernet eth0: IEEE 1588-2008 Advanced Timestamp supported
[ 4256.215317] OOM killer enabled.
[ 4256.218475] Restarting tasks ... done.
[ 4256.222866] random: crng reseeded on system resumption
[ 4256.228101] PM: suspend exit
[ 4259.113502] renesas-gbeth 15c30000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx

root@...rc-rzg3e:~# [ 4260.514428] 8021q: 802.1Q VLAN Support v1.8
[ 4260.518689] 8021q: adding VLAN 0 to HW filter on device eth0

root@...rc-rzg3e:~#

Cheers,
Biju

Powered by blists - more mailing lists