Message-ID: <aE_oue0q4QhoYggH@shell.armlinux.org.uk>
Date: Mon, 16 Jun 2025 10:49:45 +0100
From: "Russell King (Oracle)" <linux@...linux.org.uk>
To: Alexander Duyck <alexander.duyck@...il.com>
Cc: Leon Romanovsky <leon@...nel.org>, Andrew Lunn <andrew@...n.ch>,
	netdev@...r.kernel.org, hkallweit1@...il.com, davem@...emloft.net,
	pabeni@...hat.com, kuba@...nel.org, Jiri Pirko <jiri@...dia.com>
Subject: Re: [net-next PATCH 0/6] Add support for 25G, 50G, and 100G to fbnic

On Sat, Jun 14, 2025 at 09:26:00AM -0700, Alexander Duyck wrote:
> On Fri, Jun 13, 2025 at 3:44 PM Russell King (Oracle)
> <linux@...linux.org.uk> wrote:
> >
> > On Fri, Jun 13, 2025 at 07:00:24PM +0300, Leon Romanovsky wrote:
> > > Excellent, like you said, no one needs this code except fbnic, which is
> > > exactly as was agreed - no core in/out API changes special for fbnic.
> >
> > Rather than getting all religious about this, I'd prefer to ask a
> > different question.
> >
> > Is it useful to add 50GBASER, LAUI and 100GBASEP PHY interface modes,
> > and would anyone else use them? That's the real question here, and
> > *not* whomever is submitting the patches or who is the first user.
> >
> > So, let's look into this. According to the proposed code and comments,
> > PHY_INTERFACE_MODE_50GBASER is a single lane for 50G with clause 134
> > FEC.
> >
> > LAUI seems to also be a single lane 50G, no mention about FEC (so one
> > assumes it has none) and the comment states it's an attachment unit
> > interface. It doesn't mention anything else about it.
> >
> > Lastly, we have PHY_INTERFACE_MDOE_100GBASEP, which is again a single
> > lane running at 100G with clause 134 FEC.
> >
> > I assume these are *all* IEEE 802.3 defined protocols, sadly my 2018
> > version of 802.3 doesn't cover them. If they are, then someday, it is
> > probable that someone will want these definitions.
> 
> Yes, they are all 802.3 protocols. The definition for these all start
> with clause 131-140 in the IEEE spec.

Good.

> > Now, looking at the SFP code:
> > - We already have SFF8024_ECC_100GBASE_CR4 which is value 0x0b.
> >   SFF-8024 says that this is "100GBASE-CR4, 25GBASE-CR, CA-25G-L,
> >   50GBASE-CR2 with RS (Clause 91) FEC".
> >
> >   We have a linkmode for 100GBASE-CR4 which we already use, and the
> >   code adds the LAUI interface.
> 
> The LAUI interface is based on the definition in clause 135 of the
> IEEE spec. Basically the spec calls it out as LAUI-2 which is used for
> cases where the FEC is either not present or implemented past the PCS
> PMA.

Please name things as per the 802.3 spec, although "based on" suggests
it isn't strictly to the 802.3 spec... I don't have a recent enough spec
to be able to check yet.

> >   Well, "50GBASE-CR2" is 50GBASE-R over two lanes over a copper cable.
> >   So, this doesn't fit as LAUI is as per the definition above
> >   extracted from the code.
> 
> In the 50G spec LAUI is a 2 lane setup that is running over NRZ, the
> PAM4 variant is 50GAUI and can run over 2 lanes or 1.

I guess what you're saying here is that 802.3 LAUI-2 is NRZ, whereas
your LAUI is PAM4 ? Please nail this down properly, is your LAUI
specific to your implementation, or is it defined by 802.3?

> > - Adding SFF8024_ECC_200GBASE_CR4 which has value 0x40. SFF-8024
> >   says this is "50GBASE-CR, 100GBASE-CR2, 200GBASE-CR4". No other
> >   information, e.g. whether FEC is supported or not.
> 
> Yeah, it makes it about as clear as mud. Especially since they use "R"
> in the naming, but then in the IEEE spec they explain that the
> 100GbaseP is essentially the R form for 2 lanes or less but they
> rename it with "P" to indicate PAM4 instead of NRZ.

So, 100GBASE-R is four lanes, 100GBASE-P is two or one lane using PAM4,
right?

> >   We do have ETHTOOL_LINK_MODE_50000baseCR_Full_BIT, which is added.
> >   This is added with PHY_INTERFACE_MODE_50GBASER
> >
> >   Similar for ETHTOOL_LINK_MODE_100000baseCR2_Full_BIT, but with
> >   PHY_INTERFACE_MDOE_100GBASEP. BASE-P doesn't sound like it's
> >   compatible with BASE-R, but I have no information on this.
> >
> >   Finally, we have ETHTOOL_LINK_MODE_200000baseCR4_Full_BIT which
> >   has not been added.
> 
> I was only adding what I could test. Basically in our case we have the
> CR4 cable that is running 2 links at CR2 based on the division of the
> cable.

Much of these translations are based on documentation rather than
testing, especially for speeds >10G. Adding the faster speeds helps
avoid a stream of patches as people test other modules, unless the
spec is actually wrong.

> > So, it looks to me like some of these additions could be useful one
> > day, but I'm not convinced that their use with SFPs is correct.
> 
> Arguably this isn't SPF, this QSFP which is divisible. For QSFP and
> QSFP+ they didn't do as good a job of explaining it as they did with
> QSFP-DD where the CMIS provides an explanation on how to advertise the
> ability to split up the connection into mulitple links.



> 
> > Now, the next question is whether we have anyone else who could
> > possibly use this.
> >
> > Well, we have the LX2160A SoC in mainline, used on SolidRun boards
> > that are available. These support 25GBASE-R, what could be called
> > 50GBASE-R2 (CAUI-2), and what could be called 100GBASE-R4 (CAUI-4).
> >
> > This is currently as far as my analysis has gone, and I'm starting
> > to fall asleep, so it's time to stop trying to comment further on
> > this right now. Some of what I've written may not be entirely
> > accurate either. I'm unlikely to have time to provide any further
> > comment until after the weekend.
> >
> > However, I think a summary would be... the additions could be useful
> > but currently seem to me wrongly used.
> 
> If needed I can probably drop the 200G QSFP support for now until we
> can get around to actually adding QSFP support instead of just 200G
> support. I am pretty sure only the 50G could be supported by a SFP as
> it would be a single lane setup, I don't know if SFP can support
> multiple lanes anyway.

Electrically, a SFP cage only has a fixed number of pins, and only has
sufficient to support one lane. As such, I'm not sure it makes sense to
add the >1 lane protocols to phylink_sfp_interfaces. I think if we want
to do that, we need to include the number of lanes that the cage has at
the use sites.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

Powered by blists - more mailing lists