netdev - Re: [net-next PATCH v2 09/11] fbnic: Add SW shim for MDIO interface to PMA/PMD and PCS

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <9628b972-d11f-4517-97db-a4c3c288dbfa@lunn.ch>
Date: Mon, 3 Nov 2025 22:49:13 +0100
From: Andrew Lunn <andrew@...n.ch>
To: Alexander Duyck <alexander.duyck@...il.com>
Cc: netdev@...r.kernel.org, kuba@...nel.org, kernel-team@...a.com,
	andrew+netdev@...n.ch, hkallweit1@...il.com, linux@...linux.org.uk,
	pabeni@...hat.com, davem@...emloft.net
Subject: Re: [net-next PATCH v2 09/11] fbnic: Add SW shim for MDIO interface
 to PMA/PMD and PCS

On Mon, Nov 03, 2025 at 12:18:38PM -0800, Alexander Duyck wrote:
> On Mon, Nov 3, 2025 at 10:59 AM Andrew Lunn <andrew@...n.ch> wrote:
> >
> > > The interface will consist of 2 PHYs each consisting of a PMA/PMD and a PCS
> > > located at addresses 0 and 1.
> >
> > I'm missing a bit of architecture here.
> >
> > At least for speeds up to 10G, we have the MAC enumerate what it can
> > do, the PCS enumerates its capabilities, and we read the EERPOM of the
> > SFP to find out what it supports. From that, we can figure out the
> > subset of link modes which are supported, and configure the MAC and
> > PCS as required.
> 
> The hardware we have is divisible with multiple entities running it
> parallel. It can be used as a single instance, or multiple. With our
> hardware we have 2 MACs that are sharing a single QSFP connection, but
> the hardware can in theory have 4 MACs sharing a QSFP-DD connection.
> The basic limitation is that underneath each MAC we can support at
> most 2 lanes of traffic, so just the Base-R/R2 modes. Effectively what
> we would end up with is the SFP PHY having to be chained behind the
> internal PHY if there is one. In the case of the CR/KR setups though
> we are usually just running straight from point-to-point with a few
> meter direct attach cable or internal backplane connection.

We need Russell to confirm, but i would expect the SFP driver will
enumerate the capabilities of the SFP and include all the -1, -2 and
-4 link modes. phylink will then call the pcs_validate, passing this
list of link modes. The PCS knows it only supports 1 or 2 lanes, so it
will remove all the -4 modes from the list. phylink will also pass the
list to the MAC driver, and it can remove any it does not support.

It also sounds like you need to ask the firmware about
provisioning. Does this instance have access to 1 or 2 lanes? That
could be done in either the PCS or the MAC? The .validate can then
remove even more link modes.

> To
> support that we will need to have access to 2 PCS instances as the IP
> is divisible to support either 1 or 2 lanes through a single instance.

Another architecture question.... Should phylink know there are two
PCS instances? Or should it see just one? 802.3 defines registers for
lanes 0-3, sometimes 0-7, sometimes 0-9, and even 0-19. So a single
PCS should be enough for 2 lanes, or 4 lanes.

> Then underneath that is an internal PCS PMA which I plan to merge in
> with the PMA/PMD I am representing here as the RSFEC registers are
> supposed to be a part of the PMA. Again with 2 lanes supported I need
> to access two instances of it for the R2 modes. Then underneath that
> we have the PMD which is configurable on a per-lane basis.

There is already some support for pma configuration in pcs-xpcs. See
pcs-xpcs-nxp.c. 

> The issue is that the firmware is managing the PMD underneath us. As a
> result we don't have full control of the link. One issue we are
> running into is that the FW will start training when it first gets a
> signal and it doesn't block the signal from getting to the PCS. The
> PCS will see the signal and immediately report the link as "up" if the
> quality is good enough. This results in us suddenly seeing the link
> flapping for about 2-3 seconds while the training is happening. So to
> prevent that from happening we are adding the phydev representing the
> PMD to delay the link up by the needed 4 seconds to prevent the link
> flap noise.

So it seems like you need to extend dw_xpcs_compat with a .get_state
callback. You can then have your own implementation which adds this 4
second delay, before chaining into xpcs_get_state() to return the true
state.

	Andrew