Date:   Wed, 8 Apr 2020 13:52:33 -0700
From:   Florian Fainelli <f.fainelli@...il.com>
To:     Vladimir Oltean <olteanv@...il.com>
Cc:     netdev <netdev@...r.kernel.org>, Andrew Lunn <andrew@...n.ch>,
        Vivien Didelot <vivien.didelot@...il.com>,
        Ido Schimmel <idosch@...sch.org>,
        Jiri Pirko <jiri@...nulli.us>, Jakub Kicinski <kuba@...nel.org>
Subject: Re: Changing devlink port flavor dynamically for DSA



On 4/8/2020 1:10 PM, Vladimir Oltean wrote:
> On Wed, 8 Apr 2020 at 23:05, Florian Fainelli <f.fainelli@...il.com> wrote:
>>
>>
>>
>> On 4/8/2020 12:51 PM, Vladimir Oltean wrote:
>>> Hi Florian,
>>>
>>> On Sun, 5 Apr 2020 at 23:42, Florian Fainelli <f.fainelli@...il.com> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> On a BCM7278 system, we have two ports of the switch: 5 and 8, that
>>>> connect to separate Ethernet MACs that the host/CPU can control. In
>>>> premise they are both interchangeable because the switch supports
>>>> configuring the management port to be either 5 or 8 and the Ethernet
>>>> MACs are two identical instances.
>>>>
>>>> The Ethernet MACs are scheduled differently across the memory controller
>>>> (they have different bandwidth and priority allocations) so it is
>>>> desirable to select an Ethernet MAC capable of sustaining bandwidth and
>>>> latency for host networking. Our current (in the downstream kernel) use
>>>> case is to expose port 5 solely as a control end-point to the user and
>>>> leave it to the user how they wish to use the Ethernet MAC behind port
>>>> 5. Some customers use it to bridge Wi-Fi traffic, some simply keep it
>>>> disabled. Port 5 of that switch does not make use of Broadcom tags in
>>>> that case, since ARL-based forwarding works just fine.
>>>>
>>>> The current Device Tree representation that we have for that system
>>>> makes it possible for either port to be elected as the CPU port from a
>>>> DSA perspective as they both have an "ethernet" phandle property that
>>>> points to the appropriate Ethernet MAC node, because of that the DSA
>>>> framework treats them as CPU ports.
>>>>
>>>> My current line of thinking is to permit a port to be configured as
>>>> either "cpu" or "user" flavor and do that through devlink. This can
>>>> create some challenges but hopefully this also paves the way for finally
>>>> supporting "multi-CPU port" configurations. I am thinking something like
>>>> this would be how I would like it to be configured:
>>>>
>>>> # First configure port 8 as the new CPU port
>>>> devlink port set pci/0000:01:00.0/8 type cpu
>>>> # Now unmap port 5 from being a CPU port
>>>> devlink port set pci/0000:01:00.0/1 type eth
>>>>
>>>> and this would do a simple "swap" of all user ports being now associated
>>>> with port 8, and no longer with port 5, thus permitting port 5 from
>>>> becoming a standard user port. Or maybe, we need to do this as an atomic
>>>> operation in order to avoid a switch being configured with no CPU port
>>>> anymore, so something like this instead:
>>>>
>>>> devlink port set pci/0000:01:00.0/5 type eth mgmt pci/0000:01:00.0/8
>>>>
>>>> The latter could also be used to define groups of ports within a switch
>>>> that has multiple CPU ports, e.g.:
>>>>
>>>> # Ports 1 through 4 "bound" to CPU port 5:
>>>>
>>>> for i in $(seq 0 3)
>>>> do
>>>>         devlink port set pci/0000:01:00.0/$i type eth mgmt pci/0000:01:00.0/5
>>>> done
>>>>
>>>> # Ports 7 bound to CPU port 8:
>>>>
>>>> devlink port set pci/0000:01:00.0/1 type eth mgmt pci/0000:01:00.0/8
>>>>
>>>> Let me know what you think!
>>>>
>>>> Thanks
>>>> --
>>>> Florian
>>>
>>> What is missing from your argumentation is what would the new devlink
>>> mechanism of changing the CPU port bring for your particular use case.
>>> I mean you can already remove the "ethernet" device tree property from
>>> port 5 and end up exactly with the configuration that you want, no?
>>
>> That's what I do in our downstream tree for now, should I submit this
>> upstream? I doubt it would be accepted.
>> --
>> Florian
> 
> This is exactly what we do for the NXP LS1028A (ocelot/felix driver),
> where we enable just one of the 2 CPU ports by default (and the other
> one, just as a simple user port in the very few situations that
> require it):
> https://github.com/torvalds/linux/blob/master/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi#L943
> Although to be fair, for LS1028A we can't even dream of enabling DSA
> tagging on both CPU ports at the same time, since there is a hardware
> limitation in place that only a single port may carry DSA tags at any
> given moment in time.

The firmware is provided by the boot loader in my case, and while it can
be changed, dealing with incompatibilities is a support burden, so
instead, I have two lines of code in the sf2 driver to delete the DT
property and that gets me going.

I still think there is value in being able to assign groups of user
ports to a specific CPU/management port (maybe even DSA, say if we
supported trunking/bonding at some point).
-- 
Florian

Powered by blists - more mailing lists