Message-ID: <7833948c-d9a9-7051-8fa6-8feeb5528e8b@gmail.com>
Date:   Wed, 20 Mar 2019 15:17:20 -0700
From:   Florian Fainelli <f.fainelli@...il.com>
To:     Christian Lamparter <chunkeey@...il.com>
Cc:     netdev@...r.kernel.org, devicetree@...r.kernel.org,
        Vivien Didelot <vivien.didelot@...il.com>,
        Andrew Lunn <andrew@...n.ch>, Rob Herring <robh+dt@...nel.org>,
        Mark Rutland <mark.rutland@....com>
Subject: Re: [PATCH v3 3/3] net: dsa: qca8k: extend slave-bus implementations

On 3/20/19 3:02 PM, Christian Lamparter wrote:
> Sorry. I hit Sent by accident and then I had to run...
> This is the full response.
> 
> On Wednesday, March 20, 2019 7:27:09 PM CET Florian Fainelli wrote:
>> On 3/19/19 12:54 PM, Christian Lamparter wrote:
>>> This patch implements accessors for the QCA8337 MDIO access
>>> through the MDIO_MASTER register, which makes it possible to
>>> access the PHYs on slave-bus through the switch. In cases
>>> where the switch ports are already mapped via external
>>> "phy-phandles", the internal mdio-bus is disabled in order to
>>> prevent a duplicated discovery and enumeration of the same
>>> PHYs. Don't use mixed external and internal mdio-bus
>>> configurations, as this is not supported by the hardware.
>>>
>>> Signed-off-by: Christian Lamparter <chunkeey@...il.com>
>>> ---
>>> Changes from v2:
>>>  - Make it compatible with existing configurations
>>>  - make it clear that's sadly a either external or
>>>    internal mdio bus access.
>>>
>>> Changes from v1:
>>>  - drop DT port <-> phy mapping
>>>  - added register definitions for the MDIO control register
>>>  - implemented new slave-mdio bus accessors
>>>  - DT-binding: fix switch's PSEUDO_PHY address. It's 0x10 not 0.
>>>
>>> Old patch (+ discussion) for reference:
>>> <https://patchwork.ozlabs.org/patch/1036309/>
>>
>> LGTM, just a few comments/nits below.
>>
>>>
>>> Tested on a Compex WPQ864 (IPQ8064 + QCA8337N)
>>> internal bus:
>>> qca8k 37000000.mdio-mii:10 lan1 (uninitialized): PHY [!mdio@...00000!switch@10:01] driver [Generic PHY]
>>> qca8k 37000000.mdio-mii:10 lan2 (uninitialized): PHY [!mdio@...00000!switch@10:02] driver [Generic PHY]
>>> qca8k 37000000.mdio-mii:10 lan3 (uninitialized): PHY [!mdio@...00000!switch@10:03] driver [Generic PHY]
>>> qca8k 37000000.mdio-mii:10 lan4 (uninitialized): PHY [!mdio@...00000!switch@10:04] driver [Generic PHY]
>>> qca8k 37000000.mdio-mii:10 wan (uninitialized): PHY [!mdio@...00000!switch@10:05] driver [Generic PHY]
>>>
>>> external bus:
>>> qca8k 37000000.mdio-mii:10 lan1 (uninitialized): PHY [37000000.mdio-mii:00] driver [Generic PHY]
>>> qca8k 37000000.mdio-mii:10 lan2 (uninitialized): PHY [37000000.mdio-mii:01] driver [Generic PHY]
>>> qca8k 37000000.mdio-mii:10 lan3 (uninitialized): PHY [37000000.mdio-mii:02] driver [Generic PHY]
>>> qca8k 37000000.mdio-mii:10 lan4 (uninitialized): PHY [37000000.mdio-mii:03] driver [Generic PHY]
>>> qca8k 37000000.mdio-mii:10 wan (uninitialized): PHY [37000000.mdio-mii:04] driver [Generic PHY]
>>> ---
>>
>> [snip]
>>
>>> +static int
>>> +qca8k_mdio_write(struct qca8k_priv *priv, int port, int regnum, u16 data)
>>> +{
>>> +	u32 val;
>>> +	int phy;
>>> +
>>> +	phy = qca8k_port_to_phy(port);
>>> +	if (phy < 0 || (regnum < 0 || regnum >= QCA8K_MDIO_MASTER_MAX_REG))
>>> +		return -EINVAL;
>>
>> Is not the first condition always true? We should fix the signature of
>> phy_{read,write} in dsa.h to match what mdiobus_{read,write} takes,
>> which is an u32 regnum.
> 
> Yes in that case regnum will never be negative so this check can be
> futher simplyfied as dsa_slave_phy_{read|write} checks ds->phy_mii_mask.
> So port can't be invalid. In the next version this will be:
> 
> if (regnum >= QCA8K_MDIO_MASTER_MAX_REG)
> 	return -EINVAL;
> 
> Since regnum > 31 will spill into QCA8K_MDIO_MASTER_PHY_ADDR
> and the higher control bits.
> 
>>> +
>>> +	val = QCA8K_MDIO_MASTER_BUSY | QCA8K_MDIO_MASTER_EN |
>>> +	      QCA8K_MDIO_MASTER_WRITE | QCA8K_MDIO_MASTER_PHY_ADDR(phy) |
>>> +	      QCA8K_MDIO_MASTER_REG_ADDR(regnum) |
>>> +	      QCA8K_MDIO_MASTER_DATA(data);
>>> +
>>> +	qca8k_write(priv, QCA8K_MDIO_MASTER_CTRL, val);
>>> +
>>> +	return qca8k_busy_wait(priv, QCA8K_MDIO_MASTER_CTRL,
>>> +		QCA8K_MDIO_MASTER_BUSY);
>>> +}
>>> +
>>> +static int
>>> +qca8k_mdio_read(struct qca8k_priv *priv, int port, int regnum)
>>> +{
>>> +	u32 val;
>>> +	int phy;
>>> +
>>> +	phy = qca8k_port_to_phy(port);
>>> +	if (phy < 0 || (regnum < 0 || regnum >= QCA8K_MDIO_MASTER_MAX_REG))
>>> +		return -EINVAL;
>>
>> Likewise.
> Yes.
>  
>> [snip]
>>
>>> +static int
>>> +qca8k_phy_read(struct dsa_switch *ds, int port, int regnum)
>>> +{
>>> +	struct qca8k_priv *priv = ds->priv;
>>> +	int ret = -EIO;
>>> +
>>> +	if (ds->slave_mii_bus->phy_mask & BIT(port))
>>> +		ret = qca8k_mdio_read(priv, port, regnum);
>>
>> I suppose in theory you could also look at the external_mdio_mask and do
>> something like this:
>>
>> 	if (ds->slave_mii_bus->phy_mask & BIT(port))
>> 		ret = qca8k_mdio_read(priv, port, regnum);
>> 	else
>> 		ret = mdiobus_read_nested(priv->bus, port, regnum);
>>
>> Not strictly necessary for now.
> 
> Oh, in the external mdio bus scenario, I don't have qca8k_phy_read wired up.
> So the else branch would be dead code for now. (but see below)
> 
> But there's some other room for improvement:
> However since we are always called from dsa_slave_phy_read the 
> if (ds->slave_mii_bus->phy_mask & BIT(port))
> could be removed since dsa_slave_phy_read does that already.
>  
>>> +
>>> +	return ret;
>>> +}
>>> +
>>> +static int
>>> +qca8k_setup_mdio_bus(struct qca8k_priv *priv)
>>> +{
>>> +	struct device_node *ports, *port;
>>> +	struct mii_bus *bus;
>>> +	u32 internal_mdio_mask = 0;
>>> +	u32 external_mdio_mask = 0;
>>> +	u32 reg;
>>> +	int err;
>>> +
>>> +	ports = of_get_child_by_name(priv->dev->of_node, "ports");
>>> +	if (!ports)
>>> +		return -EINVAL;
>>> +
>>> +	for_each_available_child_of_node(ports, port) {
>>> +		err = of_property_read_u32(port, "reg", &reg);
>>> +		if (err)
>>> +			return err;
>>> +
>>> +		if (dsa_is_user_port(priv->ds, reg)) {
>>
>> You could reduce indentation a bit with:
>>
>> 	if (!dsa_is_user_port(priv->ds, reg))
>> 		continue;
> 
> Yes. This is nicer.
>>
>>> +			if (of_property_read_bool(port, "phy-handle"))
>>> +				external_mdio_mask |= BIT(reg);
>>> +			else
>>> +				internal_mdio_mask |= BIT(reg);
>>> +		}
>>> +	}
>>> +
>>> +	if (!external_mdio_mask && !internal_mdio_mask) {
>>> +		dev_err(priv->dev, "no PHYs are defined.\n");
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	/* The QCA8K_MDIO_MASTER_EN Bit, which grants access to PHYs through
>>> +	 * the MDIO_MASTER register also _disconnects_ the external MDC
>>> +	 * passthrough to the internal PHYs. It's not possible to use both
>>> +	 * configurations at the same time!
>>> +	 */
>>
>> Right, but presumably you can do this on a per-port basis and support
>> both types of configuration? bcm_sf2 is pretty much the same thing.
> per-port? Sadly not that I know of, it's a per-chip (or per-switch) setting
> from what I can tell by poking the chip.

The MDIO_MASTER enable is also global to the switch with the bcm_sf2. It
is basically a mux to indicate whether the internal roboswitch MDIO
master should be used, or the glued MDIO controller should be used. Both
are integrated into the switch, just not at the same level, which makes
things confusing.

The issue in the D0 revision of the silicon was that when you used the
glued MDIO controller, any MDIO access would be snooped and interpreted
by the internal MDIO slave of the switch and so while programming a
seemingly ressembling external Broadcom switch, you would also happen to
program the internal Broadcom switch (where they are register
compatible, which is like 95%) but with possibly incompatible settings.

> 
> I've looked at bcm_sf2 and the best hint seems to be in this commit:
> 
> |b8c6cd1d316f net: dsa: bcm_sf2: do not use indirect reads and writes for 7445E0
> |
> |7445E0 contains an ECO which disconnected the internal SF2 pseudo-PHY which was
> |known to conflict with the external pseudo-PHY of BCM53125 switches. This
> |motivated the need to utilize the internal SF2 MDIO controller via indirect
> |register reads/writes to control external Broadcom switches due to this address
> |conflict (both responded at address 30d). [...]
> 
> But this seems to involve the pseudo-PHYs (i.e. register access)? Which does not
> sound like it what happens on the QCA8337.

It's largely problematic with the pseudo-PHY of the switch itself, but
can be generalized to any internal or external PHY really.

> 
> For the QCA8337 the QCA8K_MDIO_MASTER_EN bit works as a changeover switch
> for just the MDC line (datasheet 5.2.13 MDIO Master Control). So the User-Port
> PHYs for LAN1-4 and WAN are moved (as in "mv" not "cp") between the external or
> internal bus. This causes some very weird behavior from any PHY access of the
> "disabled" bus:
> 
> [   17.036963] Generic PHY 37000000.mdio-mii:01: Master/Slave resolution failed, maybe conflicting manual settings?
> [   17.116927] Generic PHY 37000000.mdio-mii:02: Master/Slave resolution failed, maybe conflicting manual settings?
> [   17.196894] Generic PHY 37000000.mdio-mii:03: Master/Slave resolution failed, maybe conflicting manual settings?
> 
> I did investigate this in https://patchwork.ozlabs.org/comment/2086257/ but
> I didn't find a viable solution to make this work as from my POV, this
> requires synchronization between the qca8k code and the variety of external
> mdio-bus driver to pull this off.

So I guess my point really is that on a per MDIO transaction basis, you
could theoretically flip the MDIO_MASTER_EN bit such that you always
read/write to the desired PHY address, whether it sits on the internal
QCA8K bus, or on an external MDIO bus. Not that I think you should do
it, but it sounds like it ought to be possible unless I completely
misunderstand something here.

> 
> In my case: The WPQ864 (IPQ806x) can either use a virtual mdio-gpio driver
> (which is what most boards are using) or a some dedicated hardware that's
> living in GMAC0's core but has no upstream linux driver (yet).
> 
> Note3:
> The QCA8337 can be interfaced through either MDIO or UART (share the
> same pins, so only one option). Or alternatively through special/
> properitary ethernet-frames sent/received on the cpu port (I guess
> that's "in-band").
> 
> (There's also a SPI interface but it's soley for connecting a
> SPI EEPROM that would store VLAN, 802.1p QoS, DiffServ/TOS
> settings. So this is probably aimed at standalone switches)
> 
>>
>>> +	if (external_mdio_mask && internal_mdio_mask) {
>>> +		dev_err(priv->dev, "either internal or external mdio bus configuration is supported.\n");
>>> +		return -EINVAL;
>>> +	}
>>
>> Don't both conditions amount to:
>>
>> 	if (external_mdio_mask == internal_mdio_mask)
>> 		error()?
> I think it is either (!!external_mdio_mask == !!internal_mdio_mask)
> or (!external_mdio_mask == !internal_mdio_mask).
> 
> if (external_mdio_mask == internal_mdio_mask) will always be false,
> because the external_mdio_mask and internal_mdio_mask are bitmasks
> on whenever the (enumerated) port has a phy-handle or not. It's not
> possible for external_mdio_mask and internal_mdio_mask to be the
> same non-null value.
> 
> As for why I did it this way: I liked having different, somewhat
> descriptive error messages for these cases. Because I have been on
> so many -EINVAL wild-goose chases before. So please let me keep the
> two distinct messages, OK?

Certainly, no problem with that.
-- 
Florian

Powered by blists - more mailing lists