Date:   Mon, 30 May 2022 15:33:56 +0800
From:   Tan Tee Min <tee.min.tan@...ux.intel.com>
To:     Andrew Lunn <andrew@...n.ch>
Cc:     Heiner Kallweit <hkallweit1@...il.com>,
        Russell King <linux@...linux.org.uk>,
        "David S . Miller" <davem@...emloft.net>,
        Eric Dumazet <edumazet@...gle.com>,
        Jakub Kicinski <kuba@...nel.org>,
        Paolo Abeni <pabeni@...hat.com>, Dan Murphy <dmurphy@...com>,
        netdev@...r.kernel.org, linux-kernel@...r.kernel.org,
        stable@...r.kernel.org, Voon Wei Feng <weifeng.voon@...el.com>,
        Sit Michael Wei Hong <michael.wei.hong.sit@...el.com>,
        Ling Pei Lee <pei.lee.ling@...el.com>,
        Looi Hong Aun <hong.aun.looi@...el.com>,
        Wong Vee Khee <vee.khee.wong@...el.com>,
        Tan Tee Min <tee.min.tan@...el.com>
Subject: Re: [PATCH net-next v2 1/1] net: phy: dp83867: retrigger SGMII AN
 when link change

On Fri, May 27, 2022 at 02:43:04PM +0200, Andrew Lunn wrote:
> On Fri, May 27, 2022 at 09:47:09AM +0800, Tan Tee Min wrote:
> > On Thu, May 26, 2022 at 02:32:14PM +0200, Andrew Lunn wrote:
> > > On Thu, May 26, 2022 at 05:03:47PM +0800, Tan Tee Min wrote:
> > > > This could cause an issue during power up, when PHY is up prior to MAC.
> > > > At this condition, once MAC side SGMII is up, MAC side SGMII wouldn`t
> > > > receive new in-band message from TI PHY with correct link status, speed
> > > > and duplex info.
> > > > 
> > > > As suggested by TI, implemented a SW solution here to retrigger SGMII
> > > > Auto-Neg whenever there is a link change.
> > > 
> > > Is there a bit in the PHY which reports host side link? There is no
> > > point triggering an AN if there is already link.
> > > 
> > >       Andrew
> > 
> > Thanks for your comment.
> > 
> > There is no register bit in TI PHY which reports the SGMII AN link status.
> > But, there is a bit that only reports the SGMII AN completion status.
> > 
> > In this case, the PHY side SGMII AN has been already completed prior to MAC is up.
> > So, once MAC side SGMII is up, MAC side SGMII wouldn`t receive any new
> > in-band message from TI PHY.
> 
> That does not make any sense for how i understand how this should
> work.
> 
> Say the bootloader brings the MAC up, the SERDES gets sync and AN is
> performed between the MAC and the PHY.
> 
> Linux takes over, downs the MAC and so the SERDES link is lost. The
> PHY should notice this. Later Linux configures the MAC up, the SERDES
> link should establish and AN should be performed.
> 
> Are you saying that the SERDES link is established, and stays
> established, even when the MAC is down?
> 
> What is the structure of the host? Does it have a MAC block and a
> SERDES block? It could be, the SERDES block is running independent of
> the MAC block, and the link is established all the time, even when the
> MAC is down. What you are missing is the MAC asking the SERDES block
> for the results of the AN when the MAC comes up. So this is actually
> an Ethernet driver bug, and you are working around it in the PHY
> driver.
> 
> Are there registers in the MAC for the SERDES? Can you read the SERDES
> link and AN state?
> 
> I have seen some MAC/SERDES combinations where you have to manually
> move the AN results from the SERDES into the MAC. So could be, your
> host will do it automatically is the MAC is up, but it won't do it if
> the MAC is down when SERDES AN completes.
> 
> I just want to fully understand the issue, because if this is just a
> workaround in the PHY, and you change the PHY, you are going to need
> the same workaround in the next PHY driver.
> 
>     Andrew

Below is the HW structure for Intel mGbE controller with external PHY.
The SERDES is located in the PHY IF in the diagram below and the EQoS
MAC uses pcs-xpcs driver for SGMII interface.

    <-----------------GBE Controller---------->|<---External PHY chip--->
    +----------+         +----+            +---+           +------------+
    |   EQoS   | <-GMII->| DW | < ------ > |PHY| <-SGMII-> |External PHY|
    |   MAC    |         |xPCS|            |IF |           |(TI DP83867)|
    +----------+         +----+            +---+           +------------+
           ^               ^                 ^                ^
           |               |                 |                |
           +---------------------MDIO-------------------------+

There are registers in the DW XPCS to read the SGMII AN status and
it's showing the SGMII AN has not completed and link status is down.
But TI PHY is showing SGMII AN is completed and the copper link is
established.

FYI, the current pcs-xpcs driver is configuring C37 SGMII as MAC-side
SGMII, so it's expecting to receive AN Tx Config from PHY about the
link state change after C28 AN is completed between PHY and Link Partner.
Here is the pcs-xpcs code for your reference:
https://elixir.bootlin.com/linux/latest/source/drivers/net/pcs/pcs-xpcs.c#L725

We faced a similar issue on MaxLinear GPY PHY in the past.
And, MaxLinear folks admitted the issue and implemented fixes/improvements
in the GPY PHY Firmware to overcome the SGMII AN issue.
Besides, they have also implemented this similar SW Workaround in their
PHY driver code to cater for the old Firmware.
Feel free to refer GPY driver code here:
https://elixir.bootlin.com/linux/latest/source/drivers/net/phy/mxl-gpy.c#L222

Apart from TI and MaxLinear PHY, we've also tested the Marvell 88E2110 and
88E1512 PHY with the MAC/SERDES combination above, Marvell PHY is working
fine without any issue.

Thanks,
Tee Min

Powered by blists - more mailing lists