Message-ID:
 <DM3PR11MB87365B3AD3C360B0EF0432F3ECE92@DM3PR11MB8736.namprd11.prod.outlook.com>
Date: Thu, 30 Jan 2025 04:50:14 +0000
From: <Tristram.Ha@...rochip.com>
To: <olteanv@...il.com>
CC: <Woojung.Huh@...rochip.com>, <andrew@...n.ch>, <hkallweit1@...il.com>,
	<maxime.chevallier@...tlin.com>, <davem@...emloft.net>,
	<edumazet@...gle.com>, <kuba@...nel.org>, <pabeni@...hat.com>,
	<linux@...linux.org.uk>, <UNGLinuxDriver@...rochip.com>,
	<netdev@...r.kernel.org>, <linux-kernel@...r.kernel.org>
Subject: RE: [PATCH RFC net-next 1/2] net: pcs: xpcs: Add special code to
 operate in Microchip KSZ9477 switch

> On Wed, Jan 29, 2025 at 12:31:09AM +0000, Tristram.Ha@...rochip.com wrote:
> > The default value of DW_VR_MII_AN_CTRL is DW_VR_MII_PCS_MODE_C37_SGMII
> > (0x04).  When a SGMII SFP is used the SGMII port works without any
> > programming.  So for example network communication can be done in U-Boot
> > through the SGMII port.  When a 1000BaseX SFP is used that register needs
> > to be programmed (DW_VR_MII_SGMII_LINK_STS |
> > DW_VR_MII_TX_CONFIG_PHY_SIDE_SGMII |
> DW_VR_MII_PCS_MODE_C37_1000BASEX)
> > (0x18) for it to work.
> 
> Can it be that DW_VR_MII_PCS_MODE_C37_1000BASEX is the important setting
> when writing 0x18, and the rest is just irrelevant and bogus? If not,
> could you please explain what is the role of DW_VR_MII_SGMII_LINK_STS |
> DW_VR_MII_TX_CONFIG_PHY_SIDE_SGMII for 1000Base-X? The XPCS data book
> does not suggest they would be considered for 1000Base-X operation. Are
> you suggesting for KSZ9477 that is different? If so, please back that
> statement up.

As I mentioned before, the IP used in KSZ9477 is old and so may not match
the behavior in current DesignWare specifications.  I experimented with
different settings and observed these behaviors.

DW_VR_MII_DIG_CTRL1 has default value 0x2200.  Setting MAC_AUTO_SW (9)
has no effect as the value read back does not retain the bit.  Setting
PHY_MODE_CTRL (0) retains the bit but does not seem to have any effect
and it is not required for operation.  So we can ignore this register
for KSZ9477.

DW_VR_MII_AN_CTRL has default value 0x0004, which means C37_SGMII is
enabled.  Plugging in a 10/100/1000Base-T SFP will work without doing
anything.

Setting TX_CONFIG_PHY_SIDE_SGMII (3) requires auto-negotiation to be
disabled in MII_BMCR for the port to work.

SGMII_LINK_STS (4) depends on TX_CONFIG_PHY_SIDE_SGMII.  If that bit is
not set then this bit has no effect.  The result of setting this bit is
auto-negotiation can be enabled in MII_BMCR.

So C37_SGMII mode can still work with TX_CONFIG_PHY_SIDE_SGMII on and
auto-negotiation disabled.  But the problem is when the cable is
unplugged and plugged the port does not work as the module cannot detect
the link.  Enabling auto-negotiation and then disabling it will cause
the port to work again.

Now for 1000BaseX mode C37_1000BASEX is used and when auto-negotiation
is disabled the port works.  For the XPCS driver this can be done by
setting neg_mode to false at the beginning.  Problem is this flag can
only be set once at driver initialization.  When auto-negotiation is not
used then SFP using SGMII mode does not work properly.

So for 1000BaseX mode TX_CONFIG_PHY_SIDE_SGMII can be turned on and then
SGMII_LINK_STS allows auto-negotiation to be enabled all the time for
both SGMII and 1000BaseX modes to work.

C37_SGMII working:

Auto-negotiation enabled

Auto-negotiation disabled
TX_CONFIG_PHY_SIDE_SGMII on
(stop working after cable is unplugged and re-plugged)

C37_1000BASEX working:

Auto-negotiation disabled

Auto-negotiation disabled
TX_CONFIG_PHY_SIDE_SGMII on

Auto-negotiation enabled
TX_CONFIG_PHY_SIDE_SGMII on
SGMII_LINK_STS on

Note this behavior for 1000BaseX mode only occurs in KSZ9477, so we can
stop finding the reasons with current specs.

Microchip has another chip with newer IP version that does not have this
behavior for 1000BaseX mode.  That is, it does not require
auto-negotiation to be disabled for the port to work.  However, that chip
has major issues when using 2.5G mode so I do not know how reliable it is
when using 1G mode.

> > (DW_VR_MII_TX_CONFIG_PHY_SIDE_SGMII has to be used together with
> > DW_VR_MII_TX_CONFIG_MASK to mean 0x08.  Likewise for
> > DW_VR_MII_PCS_MODE_C37_SGMII and DW_VR_MII_PCS_MODE_MASK to mean
> 0x04.
> > It is a little difficult to just use those names to indicate the actual
> > value.)
> >
> > DW_VR_MII_DIG_CTRL1 is never touched.
> DW_VR_MII_DIG_CTRL1_MAC_AUTO_SW
> > does not exist in KSZ9477 implementation.  As setting that bit does not
> > have any effect I did not do anything about it.
> 
> Never touched by whom? xpcs_config_aneg_c37_sgmii() surely tries to
> touch it... Don't you think that the absence of this bit from the
> KSZ9477 implementation might have something to do with KSZ9477's unique
> need to force the link speed when in in-band mode?
> 
> Here's a paragraph about this from the data book:
> 
> DWC_xpcs supports automatic reconfiguration of SGMII speed/duplex mode
> based on the outcome of auto-negotiation. This feature can be enabled by
> programming bit[9] (MAC_AUTO_SW) of VR_MII_DIG_CTRL1 when operating in
> SGMII MAC mode. DWC_xpcs is initially configured in the speed/duplex
> mode as programmed in the SR_MII_CTRL. If MAC_AUTO_SW bit is enabled,
> DWC_xpcs automatically switches to the negotiated speed mode after the
> completion of CL37 Auto-negotiation. This eliminates the software
> overhead of reading CL37 AN SGMII Status from VR_MII_AN_INTR_STS and
> then programming SS13 and SS6 speed-selection bits of SR_MII_CTRL
> appropriately.
> 
> > It does have the intended effect of separating SGMII and 1000BaseX modes
> > in later versions.  And DW_VR_MII_DIG_CTRL1_PHY_MODE_CTRL is used along
> > with it.  They are mutually exclusive.  For SGMII SFP
> > DW_VR_MII_DIG_CTRL1_MAC_AUTO_SW is set; for 1000BaseX SFP
> > DW_VR_MII_DIG_CTRL1_PHY_MODE_CTRL is set.
> 
> It's difficult for me to understand what you are trying to communicate here.

The new chip has MAC_AUTO_SW bit in DIG_CTRL1 register and PHY_MODE_CTRL
is required to operate the port.  However I am not sure about the
MAC_AUTO_SW bit as it is not required for SGMII operation.  When it is on
PHY_MODE_CTRL becomes don't care and can be turned off.  It is primarily
used to detect the type of SFPs at the beginning.  Note this chip does
not use Linux so I cannot verify its function with the XPCS driver, but
the 2.5G code in that driver is so simple I wonder if it is complete.

One major difference is the 2G5_EN bit, which seems to be just to enable
2.5G mode, but it is not so in Microchip implementation as the bit means
2.5G mode is being used and so hardware needs to do something special to
manage the bandwidth.  This bit is not required to be turned off for 1G
mode.  There are other registers to set to change to 2.5G or 1G mode.
This is a little out of topic.

> > Note the evaluation board never has SFP cage logic so I never knew there
> > is a PHY inside the SGMII SFP.
> 
> What kind of SFP cage logic does the evaluation board have?

The original KSZ9477 evaluation board never has this SFP cage logic and
during development we never used that to verify SGMII function.  There is
a way to detect which type of SFP is used so reading its EEPROM is never
required.  There may be exceptions.

Microchip had to build a new board to submit this DSA driver patch for
SGMII support.

Powered by blists - more mailing lists