Message-ID: <20250804134115.cf4vzzopf5yvglxk@skbuf>
Date: Mon, 4 Aug 2025 16:41:15 +0300
From: Vladimir Oltean <vladimir.oltean@....com>
To: Alexander Wilhelm <alexander.wilhelm@...termo.com>
Cc: "Russell King (Oracle)" <linux@...linux.org.uk>,
	Andrew Lunn <andrew@...n.ch>,
	Heiner Kallweit <hkallweit1@...il.com>,
	"David S. Miller" <davem@...emloft.net>,
	Eric Dumazet <edumazet@...gle.com>,
	Jakub Kicinski <kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>,
	netdev@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: Aquantia PHY in OCSGMII mode?

Hi Alexander,

On Mon, Aug 04, 2025 at 03:01:44PM +0200, Alexander Wilhelm wrote:
> > Please find the attached patch.
> [...]
> 
> Hi Vladimir,
> 
> I’ve applied the patch you provided, but it doesn’t seem to fully resolve the
> issue -- or perhaps I’ve misconfigured something. I’m encountering the following
> error during initialization:
> 
>     mdio_bus 0x0000000ffe4e7000:00: AN not supported on 3.125GHz SerDes lane
>     fsl_dpaa_mac ffe4e6000.ethernet eth0: pcs_config failed: -EOPNOTSUPP
> 
> The relevant code is located in `drivers/net/pcs/pcs-lynx.c`, within the
> `lynx_pcs_config(...)` function. In the case of 2500BASE-X with in-band
> autonegotiation enabled, the function logs an error and returns -EOPNOTSUPP.

Once I saw this I immediately realized my mistake. More details at the end.

> From what I can tell, autonegotiation isn’t supported on a 3.125GHz SerDes lane
> when using 2500BASE-X. What I’m unclear about is how this setup is supposed to
> work in practice. My understanding is that on the host side, communication
> always uses OCSGMII with flow control, allowing speed pacing via pause frames.
> But what about the line side -- should it be configurable, or is it expected to
> operate in a fixed mode?

So there are two "auto-negotiation" processes involved.


 +-----+ internal +-----+          2500base-x       +-----------+  2.5GBase-T  +------------+
 | MAC |==========| PCS |===========================| Local PHY |==============| Remote PHY | ...
 +-----+ GMII not +-----+   in-band autonegotiation +-----------+   clause 28  +------------+
    represented in the                                           autonegotiation
        device tree                  (1)                              (2)

In the context of this error, it is about the in-band auto-negotiation (1).
This is what managed = "in-band-status" describes.

Actually "in-band autonegotation" is more of an umbrella term whose
exact meaning depends on the phy-interface-mode, i.e. the host-side
interface of the phylib PHY.

For 2500base-x, it refers to the state machines from IEEE 802.3 clause 37,
through which the two ends of this link segment exchange their support
of pause frames and duplex, through special 8b/10b code words.

Actually there is another form of "in-band autonegotiation" commonly in
use, where Cisco took the 802.3 clause 37 mechanism but modified just
the content and purpose of the exchanged messages. This is notably used
for SGMII and USXGMII. In Cisco's reinterpretation, the in-band code
words sent by the PHY on side (1) contain info about the link speed and
duplex negotiated by this device on side (2). And the in-band code words
sent by the MAC-side PCS on side (1) just contain an ACK that it
received the message and is going to reconfigure itself to the line-side
speed.

Whereas the "normal" form of in-band auto-negotiation for 2500base-x is
used over optical links and is truly a symmetric capability exchange,
the Cisco modified form for SGMII/USXGMII only carries useful information
one way (from PHY to MAC), and nothing is really "negotiated". A generic
mechanism was made domain-specific.

The auto-negotiation process which you are concerned about, the one
which dictates the line-side link mode to be used, is process (2) in the
diagram above, and happens independently of process (1).

The exchange (1) of code words is what the Lynx PCS doesn't support when
operating at 2.5G. It has no implications on process (2). It just means
that the PCS doesn't support being told in-band (over the SerDes lanes)
what speed, duplex and flow control settings to use. But it only supports
2.5G for speed anyway, full duplex, and the flow control needs to be
resolved out of band (by reading PHY registers over MDIO) and written
into PCS registers.

The limitation is more relevant for a fibre optic link than for the
Aquantia PHY case. I'm not even sure whether Aquantia PHYs send in-band
code words over OCSGMII anyway (I only tested in combination with the
Lynx PCS which wouldn't see them anyway), and if it does, what format do
they have.

> > For debugging, I recommend dumping /proc/device-tree/soc/fman@...0000/ethernet@...00/
> > (node may change for different MAC) to make sure that all the required
> > properties are there, i.e. phy-handle, phy-connection-type, pcsphy-handle.
> [...]
> 
> I decompiled the running device tree. Below are the excerpt from the resulting file:
> 
>     /dts-v1/;
> 
>     / {
>         soc@...000000 {
>             fman@...000 {
>                 ethernet@...00 {
>                     phy-handle = <0x0f>;
>                     compatible = "fsl,fman-memac";
>                     mac-address = [00 00 5b 05 a2 cb];
>                     local-mac-address = [00 00 5b 05 a2 cb];
>                     fsl,fman-ports = <0x0b 0x0c>;
>                     ptp-timer = <0x0a>;
>                     status = "okay";
>                     pcsphy-handle = <0x0d 0x0e>;
>                     reg = <0xe6000 0x1000>;
>                     phy-connection-type = "2500base-x";
>                     sleep = <0x10 0x10000000>;
>                     pcs-handle-names = "sgmii", "qsgmii";
>                     cell-index = <0x03>;
>                 };
> 
>                 mdio@...00 {
>                     fsl,erratum-a009885;
>                     compatible = "fsl,fman-memac-mdio", "fsl,fman-xmdio";
>                     status = "okay";
>                     #address-cells = <0x01>;
>                     #size-cells = <0x00>;
>                     reg = <0xfd000 0x1000>;
> 
>                     ethernet-phy@7 {
>                         compatible = "ethernet-phy-id31c3.1c63";
>                         phandle = <0x0f>;
>                         reg = <0x07>;
>                     };
>                 };
>             };
>         };
>     };
> 
> Let me know how I can assist further -- do you need any additional information from my side?

The device tree dump looks ok.

I said there should be no managed = "in-band-status" property in the
device tree, and indeed there is none.

What I failed to consider is that the FMan mEMAC driver sets phylink's
"default_an_inband" property to true, making it as if the device tree
node had this property anyway.

The driver needs to be further patched to prevent that from happening.
Here's a line that needs to be squashed into the previous change, could
you please retest with it?

--- a/drivers/net/ethernet/freescale/fman/fman_memac.c
+++ b/drivers/net/ethernet/freescale/fman/fman_memac.c
@@ -1229,6 +1229,7 @@ int memac_initialization(struct mac_device *mac_dev,
 	 * those configurations modes don't use in-band autonegotiation.
 	 */
 	if (!of_property_present(mac_node, "managed") &&
+	    mac_dev->phy_if != PHY_INTERFACE_MODE_2500BASEX &&
 	    mac_dev->phy_if != PHY_INTERFACE_MODE_MII &&
 	    !phy_interface_mode_is_rgmii(mac_dev->phy_if))
 		mac_dev->phylink_config.default_an_inband = true;

Powered by blists - more mailing lists