Date:   Mon, 23 Dec 2019 09:57:51 +0000
From:   "Madalin Bucur (OSS)" <madalin.bucur@....nxp.com>
To:     Russell King - ARM Linux admin <linux@...linux.org.uk>,
        "Madalin Bucur (OSS)" <madalin.bucur@....nxp.com>
CC:     Andrew Lunn <andrew@...n.ch>,
        "antoine.tenart@...e-electrons.com" 
        <antoine.tenart@...e-electrons.com>,
        "jaz@...ihalf.com" <jaz@...ihalf.com>,
        "baruch@...s.co.il" <baruch@...s.co.il>,
        "davem@...emloft.net" <davem@...emloft.net>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "f.fainelli@...il.com" <f.fainelli@...il.com>,
        "hkallweit1@...il.com" <hkallweit1@...il.com>,
        "shawnguo@...nel.org" <shawnguo@...nel.org>,
        "devicetree@...r.kernel.org" <devicetree@...r.kernel.org>
Subject: RE: [PATCH 1/6] net: phy: add interface modes for XFI, SFI

> -----Original Message-----
> From: devicetree-owner@...r.kernel.org <devicetree-owner@...r.kernel.org>
> On Behalf Of Russell King - ARM Linux admin
> Sent: Monday, December 23, 2019 10:27 AM
> To: Madalin Bucur (OSS) <madalin.bucur@....nxp.com>
> Subject: Re: [PATCH 1/6] net: phy: add interface modes for XFI, SFI
> 
> On Mon, Dec 23, 2019 at 07:50:08AM +0000, Madalin Bucur (OSS) wrote:
> > > -----Original Message-----
> > > From: Andrew Lunn <andrew@...n.ch>
> > > Sent: Friday, December 20, 2019 12:06 PM
> > > To: Madalin Bucur (OSS) <madalin.bucur@....nxp.com>
> > > Cc: Russell King - ARM Linux admin <linux@...linux.org.uk>;
> > > antoine.tenart@...e-electrons.com; jaz@...ihalf.com;
> baruch@...s.co.il;
> > > davem@...emloft.net; netdev@...r.kernel.org; f.fainelli@...il.com;
> > > hkallweit1@...il.com; shawnguo@...nel.org; devicetree@...r.kernel.org
> > > Subject: Re: [PATCH 1/6] net: phy: add interface modes for XFI, SFI
> > >
> > > On Fri, Dec 20, 2019 at 09:39:08AM +0000, Madalin Bucur (OSS) wrote:
> > > > > -----Original Message-----
> > > > > From: Andrew Lunn <andrew@...n.ch>
> > > > > Sent: Friday, December 20, 2019 11:29 AM
> > > > > To: Russell King - ARM Linux admin <linux@...linux.org.uk>
> > > > > Cc: Madalin Bucur (OSS) <madalin.bucur@....nxp.com>;
> > > antoine.tenart@...e-
> > > > > electrons.com; jaz@...ihalf.com; baruch@...s.co.il;
> > > davem@...emloft.net;
> > > > > netdev@...r.kernel.org; f.fainelli@...il.com;
> hkallweit1@...il.com;
> > > > > shawnguo@...nel.org; devicetree@...r.kernel.org
> > > > > Subject: Re: [PATCH 1/6] net: phy: add interface modes for XFI,
> SFI
> > > > >
> > > > > > How does this help us when we can't simply change the existing
> > > usage?
> > > > > > We can update the DT but we can't free up the usage of "10gbase-
> kr".
> > > > >
> > > > > Agreed. Code needs to keep on interpreting "10gbase-kr" as any 10G
> > > > > link. If we ever have a true 10gbase-kr, 802.3ap, one meter of
> copper
> > > > > and two connectors, we are going to have to add a new mode to
> > > > > represent true 10gbase-kr.
> > > > >
> > > > > 	Andrew
> > > >
> > > > Hi, actually we do have that. What would be the name of the new mode
> > > > representing true 10GBase-KR that we will need to add when we
> upstream
> > > > support for that?
> > >
> > > Ah!
> > >
> > > This is going to be messy.
> > >
> > > Do you really need to differentiate? What seems to make 802.3ap
> > > different is the FEC, autoneg and link training. Does you hardware
> > > support this? Do you need to know you are supposed to be using 802.3ap
> > > in order to configure these features?
> >
> > Yes, it does.
> >
> > > What are we going to report to user space? 10gbase-kr, or
> > > 10gbase-kr-true? How do we handle the mess this makes with firmware
> > > based cards which correctly report
> > > ETHTOOL_LINK_MODE_10000baseKR_Full_BIT to user space?
> > >
> > > What do we currently report to user space? Is it possible for us to
> > > split DT from user space? DT says 10gbase-kr-true but to user space we
> > > say ETHTOOL_LINK_MODE_10000baseKR_Full_BIT?
> > >
> > > I think in order to work through these issues, somebody probably needs
> > > the hardware, and the desire to see it working. So it might actually
> > > be you who makes a proposal how we sort this out, with help from
> > > Russell and I.
> > >
> > > 	Andrew
> >
> > We're overcomplicating the fix. As far as I can see only some Marvell
> boards
> > declared 10GBase-KR as PHY interface type. These either support 10GBase-
> KR or
> > they don't. When we learn this, we'll need to set things straight in the
> device
> > trees and code. Until then it will remain as is, there is no trouble
> with that.
> 
> No we aren't.
> 
> You think we can just change the existing DT, switching them to use
> XFI/SFI and free up the "10gbase-kr" definition.  Yes, we can change
> the existing DT.  What we *can't* do is free up the existing definition
> for "10gbase-kr" because old device trees must continue to work with
> new kernels.  That is one of the rules we abide by with the kernel.

We do not need to "free up" the definition, if that particular device
does use 10GBase-KR then it does not need changing. Please note 10GBase-KR
is quite well established in the Ethernet nomenclature (802.3ap, 2007),
it's clear what it does so there is no need to "free it up" somewhere.

The phy-connection-type was introduced back in the day when this
connection was more aligned to the MAC-PHY MII. That resulted in
XGMII being used for all 10G interfaces when they were added.
But the clean MII separation does not align with today's HW, where
the asic/SoC to external PHY chip interface has evolved towards
high speed serial interfaces with obvious benefits. We're now seeing
the PCS, PMA blocks that are part of the PHY layer moved together with
the MAC.

I've recently seen how a certain PHY driver started using USXGMII and
rejected XGMII (please note the MII is still XGMII but it's buried
somewhere in the peer SoC) as a phy-conection-type. Some HW platforms
may use that PHY in USXGMII mode, others use it in one of the other
supported modes - XFI. Why is USXGMII a valid phy-connection-type
but XFI is not? Is it correct that I describe that HW as using
10GBase-KR instead of XFI, something that has a clearly different
meaning for anyone familiar with IEEE 802.3ap just because there was
a typo in a patch, be it a device tree one? I'm puzzled.

> Now, looking at the Armada 8040 data, it only mentions XFI.  It's
> described as "10GBASE-R (10GbE on single SERDES, including XFI)"
> and goes on to say that it is compliant with "IEEE 802.3 standard".
> However, there is no mention of a CDR, except for XAUI/XSGS mode,
> not for 10GBASE-R/XFI mode.

Is "10GBASE-R" in wide use? Although you can extract that from the
Ethernet nomenclature to mean 10G with 64B/66B encoding, I did not
see it used much by vendors. I do see more of the XFI, SFI terminology
in use.
 
> So, it really isn't clear whether Marvell uses "XFI" to refer to a
> port with a CDR or not.
> 
> Marvell's original MVPP2.2 and comphy drivers did used to distinguish
> between XFI and SFI, but there was absolutely no difference in the
> way the hardware was programmed.
> 
> Then there's the matter that (I believe with some firmware) it can
> also support 10GBASE-KR (with clause 73 AN) after all.

Then there's not a real need to change that device tree, it's correct.
The HW can be used in 10GBASE-KR with appropriate SW so the HW description
is fine.
 
> So, we can't just replace the existing usage in DT with "SFI" just
> because it's connected to a SFP cage, or "XFI" if it's connected
> to a PHY which also mentiones "XFI" in its datasheet. We can't tell
> whether the hardware _actually_ supports 1EEE 10GBASE-R, XFI or SFI.

We don't need to replace it. But if a certain HW is certain to use XFI
(or SFI), we just need to describe it as such in the device tree.

> Given that XFI and SFI are not actually documented (afaics) this is
> really not that surprising - and I would not be surprised if these
> three terms were used interchangably by some device manufacturers
> as a result.

XFI (and SFI?) are the result of MSA, multi-source agreements so they
must be documented somewhere (see http://www.xfpmsa.org/). Whether that
document is available for free or behind a paywall/membership it's
another discussion.

> --
> RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
> FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps
> up
> According to speedtest.net: 11.9Mbps down 500kbps up
OT: FTTB 1Gbps link, my phone reports 279Mbps down, 169Mbps up, limited by WiFi
Internet provider services vary wildly across the globe

Best regards,
Madalin

Powered by blists - more mailing lists