Message-ID: <70bccb10-cc76-4eec-b2cf-975ed422c443@lunn.ch>
Date: Thu, 11 Apr 2024 19:32:04 +0200
From: Andrew Lunn <andrew@...n.ch>
To: Alexander Duyck <alexander.duyck@...il.com>
Cc: Jakub Kicinski <kuba@...nel.org>, pabeni@...hat.com,
	John Fastabend <john.fastabend@...il.com>,
	Alexander Lobakin <aleksander.lobakin@...el.com>,
	Florian Fainelli <f.fainelli@...il.com>,
	Daniel Borkmann <daniel@...earbox.net>,
	Edward Cree <ecree.xilinx@...il.com>, netdev@...r.kernel.org,
	bhelgaas@...gle.com, linux-pci@...r.kernel.org,
	Alexander Duyck <alexanderduyck@...com>,
	Willem de Bruijn <willemdebruijn.kernel@...il.com>
Subject: Re: [net-next PATCH 00/15] eth: fbnic: Add network driver for Meta
 Platforms Host Network Interface

On Thu, Apr 11, 2024 at 09:00:17AM -0700, Alexander Duyck wrote:
> On Wed, Apr 10, 2024 at 3:37 PM Andrew Lunn <andrew@...n.ch> wrote:
> >
> > > Well I was referring more to the data path level more than the phy
> > > configuration. I suspect different people have different levels of
> > > expectations on what minimal firmware is. With this hardware we at
> > > least don't need to use firmware commands to enable or disable queues,
> > > get the device stats, or update a MAC address.
> > >
> > > When it comes to multi-host NICs I am not sure there are going to be
> > > any solutions that don't have some level of firmware due to the fact
> > > that the cable is physically shared with multiple slots.
> >
> > This is something Russell King at least considered. I don't really
> > know enough to know why its impossible for Linux to deal with multiple
> > slots.
> 
> It mostly has to do with the arbitration between them. It is a matter
> of having to pass a TON of info to the individual slice and then the
> problem is it would have to do things correctly and not manage to take
> out it's neighbor or the BMC.

How much is specific to your device? How much is just following 802.3
and the CMIS standards? I assume anything which is just following
802.3 and CMIS could actually be re-used? And you have some glue to
combine them in a way that is specific to your device?
 
> > > I am assuming we still want to do the PCS driver. So I will still see
> > > what I can do to get that setup.
> >
> > You should look at the API offered by drivers in drivers/net/pcs. It
> > is designed to be used with drivers which actually drive the hardware,
> > and use phylink. Who is responsible for configuring and looking at the
> > results of auto negotiation? Who is responsible for putting the PCS
> > into the correct mode depending on the SFP modules capabilities?
> > Because you seemed to of split the PCS into two, and hidden some of it
> > away, i don't know if it makes sense to try to shoehorn what is left
> > into a Linux driver.
> 
> We have control of the auto negotiation as that is north of the PMA
> and is configured per host. We should support clause 73 autoneg.
> Although we haven't done much with it as most of our use cases are
> just fixed speed setups to the switch over either 25G-CR1, 50G-CR2,
> 50G-CR1, or 100G-CR2. So odds are we aren't going to be doing anything
> too terribly exciting.

Maybe not, but you might of gained from the community here, if others
could of adopted this code for their devices. You might not need
clause 73, but phylink provides helpers to implement it, so it is
pretty easy to add. Maybe your initial PCS driver does not support it,
but later adopters who also licence this PCS might add it, and you get
the feature for free. The corrected/uncorrected counters i asked
about, are something you might not export in your current code via
ethtool. But again, this is something which somebody else could add a
helper for, and you would get it nearly for free.

> As far as the QSFP setup the FW is responsible for any communication
> with it. I suspect that the expectation is that we aren't going to
> need much in the way of config since we are just using direct attach
> cables.

Another place you might of got features for free. The Linux SFP driver
exports HWMON values for temperature, power, received power, etc, but
for 1G. The QSFP+ standard Versatile Diagnostics Monitoring is
different, but i could see somebody adding a generic implementation in
the Linux SFP driver, so that the HWMON support is just free. Same
goes for the error performance statics. Parts of power management
could easily be generic. It might be possible to use Linux regulators
to describe what your board is capable if, and the SFP core could then
implement the ethtool ops, checking with the regulator to see if the
power is actually available, and then talking to the SFP to tell it to
change its power class?

Florian posted some interesting statistics, that vendors tend to
maintain their own drivers, and don't get much support from the
community. However i suspect it is a different story for shared
infrastructure like PCS drivers, PHY drivers, SFP drivers. That is
where you get the most community support and the most stuff for free.
But you actually have to use it to benefit from it.

	Andrew

Powered by blists - more mailing lists