Message-ID: <aQuTldBf4CPTzS77@shell.armlinux.org.uk>
Date: Wed, 5 Nov 2025 18:12:37 +0000
From: "Russell King (Oracle)" <linux@...linux.org.uk>
To: Mohd Ayaan Anwar <mohd.anwar@....qualcomm.com>
Cc: Andrew Lunn <andrew@...n.ch>, Heiner Kallweit <hkallweit1@...il.com>,
	Alexandre Torgue <alexandre.torgue@...s.st.com>,
	Alexis Lothoré <alexis.lothore@...tlin.com>,
	Andrew Lunn <andrew+netdev@...n.ch>,
	Boon Khai Ng <boon.khai.ng@...era.com>,
	Daniel Machon <daniel.machon@...rochip.com>,
	"David S. Miller" <davem@...emloft.net>,
	Eric Dumazet <edumazet@...gle.com>, Furong Xu <0x1207@...il.com>,
	Jacob Keller <jacob.e.keller@...el.com>,
	Jakub Kicinski <kuba@...nel.org>,
	"Jan Petrous (OSS)" <jan.petrous@....nxp.com>,
	linux-arm-kernel@...ts.infradead.org,
	linux-stm32@...md-mailman.stormreply.com,
	Maxime Chevallier <maxime.chevallier@...tlin.com>,
	Maxime Coquelin <mcoquelin.stm32@...il.com>, netdev@...r.kernel.org,
	Paolo Abeni <pabeni@...hat.com>, Simon Horman <horms@...nel.org>,
	Vladimir Oltean <olteanv@...il.com>,
	Yu-Chun Lin <eleanor15x@...il.com>
Subject: Re: [PATCH net-next 0/3] net: stmmac: phylink PCS conversion part 3
 (dodgy stuff)

On Wed, Nov 05, 2025 at 09:16:10PM +0530, Mohd Ayaan Anwar wrote:
> Hello Russell,
> 
> I finally got some time to test out 100M/1G/2.5G with all these changes
> on the QCS9100 Ride R3 board (AQR115C PHY, qcom-ethqos).
> 
> Apart from this patch series, I incorporated the following changes:
> 1. Hacks suggested to have the PCS code work for 2500Base-X.
> 2. Removed snps,ps-speed from DT.
> 3. Picked [PATCH net-next v2] net: stmmac: qcom-ethqos: remove
> MAC_CTRL_REG modification.
> 
> Apologies for the lengthy email- I’ve included phylink logs for all
> five points for completeness.

That's fine, thanks.

> *Observations:*
> 
> 1. 2.5G Link up - no warning about the PCS configuration getting changed
> by glue. Data path works fine.
> 
> 	[   10.342908] qcom-ethqos 23000000.ethernet eth0: PHY stmmac-0:00 uses interfaces 4,23,27, validating 23
> 	[   10.352486] qcom-ethqos 23000000.ethernet eth0:  interface 23 (2500base-x) rate match pause supports 0-7,9,13-14,47
> 	[   10.363215] qcom-ethqos 23000000.ethernet eth0: PHY [stmmac-0:00] driver [Aquantia AQR115C] (irq=365)
> 	[   10.372690] qcom-ethqos 23000000.ethernet eth0: phy: 2500base-x setting supported 0000000,00000000,00008000,000062ff advertising 0000000,00000000,00008000,000062ff
> 	[   10.428389] qcom-ethqos 23000000.ethernet eth0: configuring for phy/2500base-x link mode
> 	[   10.436717] qcom-ethqos 23000000.ethernet eth0: major config, requested phy/2500base-x
> 	[   10.444870] qcom-ethqos 23000000.ethernet eth0: interface 2500base-x inband modes: pcs=01 phy=00
> 	[   10.453913] qcom-ethqos 23000000.ethernet eth0: major config, active phy/outband/2500base-x
> 	[   10.462506] qcom-ethqos 23000000.ethernet eth0: phylink_mac_config: mode=phy/2500base-x/none adv=0000000,00000000,00000000,00000000 pause=00
> 	[   10.485700] qcom-ethqos 23000000.ethernet eth0: phy link down 2500base-x/Unknown/Unknown/none/off/nolpi
> 	[   15.131941] qcom-ethqos 23000000.ethernet eth0: phy link up 2500base-x/2.5Gbps/Full/none/rx/tx/nolpi
> 	[   15.143632] qcom-ethqos 23000000.ethernet eth0: ethqos_configure_sgmii : Speed = 2500
> 	[   15.153226] stmmac_pcs: Link Up
> 	[   15.153274] qcom-ethqos 23000000.ethernet eth0: Link is Up - 2.5Gbps/Full - flow control rx/tx
> 
> 
> 2. 1G Link up - Warning (PCS configuration changed from phylink by glue,
> please report: 0x00040000 -> 0x00041200). Data path works fine.
> 
...
> 	[   10.532328] qcom-ethqos 23000000.ethernet eth0: major config, active phy/outband/2500base-x
> 	[   10.540919] qcom-ethqos 23000000.ethernet eth0: phylink_mac_config: mode=phy/2500base-x/none adv=0000000,00000000,00000000,00000000 pause=00
> 	[   10.563437] qcom-ethqos 23000000.ethernet eth0: phy link down 2500base-x/Unknown/Unknown/none/off/nolpi
> 	[   13.912074] qcom-ethqos 23000000.ethernet eth0: phy link up sgmii/1Gbps/Full/none/rx/tx/nolpi
> 	[   13.919074] stmmac_pcs: Link Up
> 	< a *bunch* of "stmmac_pcs: Link Down" prints, more details in 4.>
> 	[   14.948996] stmmac_pcs: Link Up
> 	[   14.949149] stmmac_pcs: Link Down
> 	[   14.949169] stmmac_pcs: Link Up

This is showing that the link state is flapping, which I guess is caused
by the PHY having switched to 1.25Mbaud SGMII but still being programmed
for 3.125Mbaud for 2.5G rate.

> 	[   14.949301] qcom-ethqos 23000000.ethernet eth0: major config, requested phy/sgmii
> 	[   14.949317] stmmac_pcs: Link Down
> 	[   14.949326] qcom-ethqos 23000000.ethernet eth0: interface sgmii inband modes: pcs=03 phy=03
> 	[   14.949331] qcom-ethqos 23000000.ethernet eth0: major config, active phy/outband/sgmii
> 	[   14.949335] qcom-ethqos 23000000.ethernet eth0: phylink_mac_config: mode=phy/sgmii/none adv=0000000,00000000,00000000,00000000 pause=03
> 	[   14.952026] qcom-ethqos 23000000.ethernet eth0: ethqos_configure_sgmii : Speed = 1000
> 	[   14.952033] dwmac: PCS configuration changed from phylink by glue, please report: 0x00040000 -> 0x00041200
> 	[   14.952057] qcom-ethqos 23000000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
> 	[   15.035977] stmmac_pcs: ANE process completed
> 	[   15.035978] stmmac_pcs: Link Down
> 	[   15.036004] stmmac_pcs: Link Up

Since we are using phy/outband mode (see "major config, active") this
means we're not expecting to use in-band, so the PCS has in-band
disabled by the PCS code, but then re-enabled by the glue code.

One thing which occurs to me though is that dwmac_ctrl_ane() doesn't
clear GMAC_AN_CTRL_SGMRAL when srgmi_ral is false, which means its
taking the speed information from the PS and FES bits in the MAC
control register.

Now, the question is, what happens if the call to
ethqos_pcs_set_inband() / stmmac_pcs_ctrl_ane() are omitted? Does
traffic still flow over the interface?

> 3. 100M Link up - Warning (PCS configuration changed from phylink by
> glue, please report: 0x00040000 -> 0x00041200). Data path works fine.
> 
...
> 	[   20.390144] qcom-ethqos 23000000.ethernet eth0: major config, active phy/outband/2500base-x
> 	[   20.398720] qcom-ethqos 23000000.ethernet eth0: phylink_mac_config: mode=phy/2500base-x/none adv=0000000,00000000,00000000,00000000 pause=00
> 	[   20.418477] stmmac_pcs: Link Down
> 	[   20.421912] stmmac_pcs: Link Down
> 	[   20.426255] qcom-ethqos 23000000.ethernet eth0: phy link down 2500base-x/100Mbps/Full/none/rx/tx/nolpi
> 	[   20.440795] stmmac_pcs: Link Down
> 	[   23.095229] qcom-ethqos 23000000.ethernet eth0: phy link up sgmii/100Mbps/Full/none/rx/tx/nolpi
> 	[   23.101362] stmmac_pcs: Link Down
> 	[   23.106527] qcom-ethqos 23000000.ethernet eth0: major config, requested phy/sgmii
> 	[   23.107624] stmmac_pcs: Link Down
> 	[   23.118707] stmmac_pcs: Link Down
> 	[   23.124703] qcom-ethqos 23000000.ethernet eth0: interface sgmii inband modes: pcs=03 phy=03
> 	[   23.128141] stmmac_pcs: Link Down
> 	[   23.136699] qcom-ethqos 23000000.ethernet eth0: major config, active phy/outband/sgmii
> 	[   23.140126] stmmac_pcs: Link Down
> 	[   23.148232] qcom-ethqos 23000000.ethernet eth0: phylink_mac_config: mode=phy/sgmii/none adv=0000000,00000000,00000000,00000000 pause=03
> 	[   23.151657] stmmac_pcs: Link Down
> 	[   23.166924] qcom-ethqos 23000000.ethernet eth0: ethqos_configure_sgmii : Speed = 100
> 	[   23.167584] stmmac_pcs: Link Down
> 	[   23.175511] dwmac: PCS configuration changed from phylink by glue, please report: 0x00040000 -> 0x00041200
> 	[   23.178944] stmmac_pcs: Link Up
> 	[   23.188862] qcom-ethqos 23000000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
> 	[   23.192097] stmmac_pcs: Link Down
> 	[   23.204346] stmmac_pcs: ANE process completed
> 	[   23.208828] stmmac_pcs: Link Up

It would be interesting to see how this behaves as well without the
call to ethqos_pcs_set_inband() / stmmac_pcs_ctrl_ane().

> 4. Sometimes after toggling the interface or even during boot-up, the
> console gets flooded with "stmmac_pcs: Link Down" prints. For e.g.,
> 
> 	// Interface toggled
> 	< a bunch of "stmmac_pcs: Link Down" prints>
> 	[  549.898750] stmmac_pcs: Link Down
> 	[  549.902186] stmmac_pcs: Link Down
> 	[  549.905628] stmmac_pcs: Link Down
> 	[  549.909069] stmmac_pcs: Link Down
> 	[  549.912509] stmmac_pcs: Link Down
> 	[  549.915948] stmmac_pcs: Link Down
> 	[  549.919391] stmmac_pcs: Link Down
> 	[  549.922858] stmmac_pcs: Link Down
> 	[  549.924140] qcom-ethqos 23000000.ethernet eth0: ethqos_configure_sgmii : Speed = 2500
> 	[  549.926304] stmmac_pcs: Link Down
> 	[  549.934349] qcom-ethqos 23000000.ethernet eth0: Link is Up - 2.5Gbps/Full - flow control rx/tx
> 	[  549.937746] stmmac_pcs: Link Up
> 
>    ethtool stats reveal an unusually high number of interrupts (I have
>    seen this number go as high as about 16000 when booting up with a 1G
>    link)
> 	     irq_pcs_ane_n: 0
> 	     irq_pcs_link_n: 1998	

Some PCS have this, caused by spuriously detecting what they think is a
valid signal but isn't. We don't enable comma detection for SGMII, but
maybe that would help avoid this? It depends how the serdes is
implemented. The alternative is that we may need to disable the IRQ
(we're not really using it for anything in this mode, as we're using
outband mode.)

> 5. Switching between 100M/1G/2.5G link is a bit of a mixed bag.
> Sometimes it works, sometimes the data path breaks and needs an
> interface toggle to be functional again. I don't necessarily think that
> it's due to the speed specific configurations done by configure_sgmii as
> that shouldn't impact switching between 1G and 2.5G, or even the switch
> from 1G/2.5G to 100M.

Maybe this is the lack of comma detect being enabled - what I read in
the databook, dwmac expects the external serdes to do comman detection
when enabled. It appears that the ECD bit just toggles an output to
the implementation's serdes to tell it to enable this, but it's
entirely possible that an implementation doesn't need this control.
It's something worth trying though.

> While this is *broken* on net-next as well, the current patch series
> allowed me to notice a peculiar behavior - it looks like sometimes the
> PCS link doesn't come up:
> 	
> 	[   55.491996] qcom-ethqos 23000000.ethernet eth0: phy link down 2500base-x/2.5Gbps/Full/none/rx/tx/nolpi
> 	[   55.501622] qcom-ethqos 23000000.ethernet eth0: Link is Down
> 	[   58.907705] qcom-ethqos 23000000.ethernet eth0: phy link up sgmii/1Gbps/Full/none/rx/tx/nolpi
> 	[   58.913724] stmmac_pcs: Link Down
> 	[   58.919947] qcom-ethqos 23000000.ethernet eth0: major config, requested phy/sgmii
> 	[   58.927656] qcom-ethqos 23000000.ethernet eth0: interface sgmii inband modes: pcs=03 phy=03
> 	[   58.936256] qcom-ethqos 23000000.ethernet eth0: major config, active phy/outband/sgmii
> 	[   58.944409] qcom-ethqos 23000000.ethernet eth0: phylink_mac_config: mode=phy/sgmii/none adv=0000000,00000000,00000000,00000000 pause=03
> 	[   58.958298] qcom-ethqos 23000000.ethernet eth0: ethqos_configure_sgmii : Speed = 1000
> 	[   58.967448] dwmac: PCS configuration changed from phylink by glue, please report: 0x00040000 -> 0x00041200
> 	[   58.977392] qcom-ethqos 23000000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
> 
> 
> Since 5 is unrelated to this series (I'll try to debug it separately),
> let me know if you'd like me to run any other experiments for 2, 3, and
> 4.

As mentioned above, ECD may help. If the serdes needs this signal
enabled to perform command detection and word synchronisation, not
enabling it is likely to result in the PCS receiving unaligned words
that it can't make sense of.

> > The "invalid port speed" warning that results if this property is
> > not set to 10, 100 or 1000 is another bug - only if this warning
> > is printed will the "normal" mode be selected.
> > 
> > Since the PCS series 1 and 2 have been merged into net-next, it
> > will be safe to submit patches removing these properties from your
> > DT files, without fear of this warning appearing.
> > 
> 
> Thanks for the explanation. I see the incorrect use of snps,ps-speed in
> the DT of a couple of more boards that use the same MAC core. Would it
> be okay to add your Suggested-by when submitting the fix patches?

Yes please, and definitely! Thanks!

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

Powered by blists - more mailing lists