Message-ID: <20200702170433.GO1551@shell.armlinux.org.uk>
Date:   Thu, 2 Jul 2020 18:04:33 +0100
From:   Russell King - ARM Linux admin <linux@...linux.org.uk>
To:     Marek Behún <marek.behun@....cz>
Cc:     netdev@...r.kernel.org
Subject: Re: 2500base-x capable sfp copper module?

On Thu, Jul 02, 2020 at 06:21:20PM +0200, Marek Behún wrote:
> We are trying to find a copper SFP module capable of 2.5G speeds for
> Turris Omnia (2500base-x is max for SERDES on Omnia).
> 
> We have tried MikroTik S+RJ10, which is a 10G capable copper SFP
> module. But this module does not export access (via I2C) to its internal
> PHY (which should be Marvell 88X3310).

That seems to follow on from the S-RJ01, which uses an Atheros 803x
PHY but also doesn't allow access to it.  It seems that MikroTik
products don't offer much flexibility, and are likely designed for
use in their own products rather than general use.

> Without access to the PHY it seems that the host side of the SFP is
> configured at 10G and we are unable to change it to 2500base-x.

The problem here is that we have no idea what the module supports.
It may be possible to use the quirk stuff I added for GPON to detect
the MikroTik module, and enable the other speeds.  We would then
have to add the ability to switch not only between 1000base-X and
2500base-X, but all the other modes as well via ethtool.  And then,
lastly, you would have to manually set the interface, guessing what
speed the module would want.

That doesn't sound like a good solution.

I can only assume that Mikrotik have implemented some kind of auto-
detection - maybe they have hardware in their switches that is
capable of analysing the serdes lines from the module.

> We have another module, Rollball RTSFP-10G, which contains the same
> PHY, but this is visible on the I2C bus at address 0x56.
> For some reason I am unable to access registers of the PHY via clause
> 45 protocol. The code in drivers/net/phy/mdio-i2c.c always returns
> 0xffff when reading via clause 45.

The 88x3310 does not support I2C, it only supports MDIO.  So, for it
to appear on the I2C bus, there must be some kind of vendor specific
protocol conversion going on, most likely a small dumb MCU from what
I read in various public SFP+ datasheets.

The modules that mdio-i2c.c has been tested with so far have done the
"sensible" thing of clock-stretching the I2C bus while they access the
PHY.  This means that using the _safe_ i2c_transfer() is possible, just
as we do for Clause 22 PHYs.  I term this safe, because the bus remains
owned during the whole cycle, so there's no chance something else (e.g.
i2c tools) will get in the way and potentially disrupt the access.

However, a number of copper SFP+ datasheets (which seem to be derived
from the same source - as can be seen from the graphics and layout)
require a 1ms delay between the I2C "mdio address write" message and
the I2C "mdio read" message.  That prevents i2c_transfer() being used
for the entire access - it has to be done as two separate accesses with
a delay inbetween - which likely will make PHY accesses rather slow
when you compare that normally, a clause 45 access may take around 25
or so microseconds, possibly faster.

As I don't have any modules that require that, I only ever noted that
such modules exist, but never implemented the split-access with a delay,
especially as it would require a fair number of code changes to mdio-i2c.

Note that mdelay(1) may not be sufficient (the *delay() functions are
_not_ guaranteed to wait at least the requested delay).

> When accessing via clause 22, the registers are visible, but we are
> unable to change to 2500base-x with these registers.

There has been talk in phylib about adding support to access a clause
45 PHY register set through clause 22 cycles via the clause 45
registers contained therein - but I don't think that went anywhere.

> Do you think this is a problem of how the SFP module is
> wired/programmed?

It's likely to be due to the missing 1ms delay.

> Do you know of 2500base-x capable copper SFP module which would work?
> Maybe one based on the same Marvell PHY, but such that the clause 45
> register access works?

The only copper module that's capable of 10G, 5G, 2.5G, 1G, 100M that I
have is a Methode DM7052 which has a Broadcom PHY on it, which behaves
as I mention above (clock stretching, without the need for a delay.)

The other 2.5G capable modules I have are Avago AFBR-57R5AEZ
fibrechannel modules designed to operate up to 4300M baud, which is
more than enough bandwidth to cover 2500base-X - but obviously no PHY
being fibre modules.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

Powered by blists - more mailing lists