Message-ID: <176481d3-eeee-feae-095a-6a3023d986f5@gmail.com>
Date:   Mon, 6 Apr 2020 11:41:25 -0700
From:   Florian Fainelli <f.fainelli@...il.com>
To:     Jiri Pirko <jiri@...nulli.us>
Cc:     netdev <netdev@...r.kernel.org>, Andrew Lunn <andrew@...n.ch>,
        Vivien Didelot <vivien.didelot@...il.com>,
        Vladimir Oltean <olteanv@...il.com>,
        Ido Schimmel <idosch@...sch.org>,
        Jakub Kicinski <kuba@...nel.org>
Subject: Re: Changing devlink port flavor dynamically for DSA



On 4/6/2020 11:20 AM, Jiri Pirko wrote:
> Mon, Apr 06, 2020 at 08:11:18PM CEST, f.fainelli@...il.com wrote:
>>
>>
>> On 4/6/2020 11:04 AM, Jiri Pirko wrote:
>>> Sun, Apr 05, 2020 at 10:42:29PM CEST, f.fainelli@...il.com wrote:
>>>> Hi all,
>>>>
>>>> On a BCM7278 system, we have two ports of the switch: 5 and 8, that
>>>> connect to separate Ethernet MACs that the host/CPU can control. In
>>>> premise they are both interchangeable because the switch supports
>>>> configuring the management port to be either 5 or 8 and the Ethernet
>>>> MACs are two identical instances.
>>>>
>>>> The Ethernet MACs are scheduled differently across the memory controller
>>>> (they have different bandwidth and priority allocations) so it is
>>>> desirable to select an Ethernet MAC capable of sustaining bandwidth and
>>>> latency for host networking. Our current (in the downstream kernel) use
>>>> case is to expose port 5 solely as a control end-point to the user and
>>>> leave it to the user how they wish to use the Ethernet MAC behind port
>>>> 5. Some customers use it to bridge Wi-Fi traffic, some simply keep it
>>>> disabled. Port 5 of that switch does not make use of Broadcom tags in
>>>> that case, since ARL-based forwarding works just fine.
>>>>
>>>> The current Device Tree representation that we have for that system
>>>> makes it possible for either port to be elected as the CPU port from a
>>>> DSA perspective as they both have an "ethernet" phandle property that
>>>> points to the appropriate Ethernet MAC node, because of that the DSA
>>>> framework treats them as CPU ports.
>>>>
>>>> My current line of thinking is to permit a port to be configured as
>>>> either "cpu" or "user" flavor and do that through devlink. This can
>>>> create some challenges but hopefully this also paves the way for finally
>>>> supporting "multi-CPU port" configurations. I am thinking something like
>>>> this would be how I would like it to be configured:
>>>>
>>>> # First configure port 8 as the new CPU port
>>>> devlink port set pci/0000:01:00.0/8 type cpu
>>>> # Now unmap port 5 from being a CPU port
>>>> devlink port set pci/0000:01:00.0/1 type eth
>>>
>>> You are mixing "type" and "flavour".
>>>
>>> Flavours: cpu/physical. I guess that is what you wanted to set, correct?
>>
>> Correct, flavor is really what we want to change here.
>>
>>>
>>> I'm not sure, it would make sense. The CPU port is still CPU port, it is
>>> just not used. You can never make is really "physical", am I correct? 
>>
>> True, although with DSA as you may know if we have a DSA_PORT_TYPE_CPU
>> (or DSA_PORT_TYPE_DSA), then we do not create a corresponding net_device
>> instance because that would duplicate the Ethernet MAC net_device. This
>> is largely the reason for suggesting doing this via devlink (so that we
>> do not rely on a net_device handle). So by changing from a
>> DSA_PORT_TYPE_CPU flavor to DSA_PORT_TYPE_USER, this means you would now
>> see a corresponding net_device instance. Conversely when you migrate
>>from DSA_PORT_TYPE_USER to DSA_PORT_TYPE_CPU, the corresponding
>> net_device would be removed.
> 
> Wait, why would you ever want to have a netdevice for CPU port (changed
> to "user")? What am I missing? Is there a usecase, some multi-person
> port?
> 

I believe I explained the use case in my first email. The way the Device
Tree description is currently laid out makes it that Port 5 gets picked
up as the CPU port for the system. This is because the DSA layer stops
whenever it encounters the first "CPU" port, which it determines by
having an "ethernet" phandle (reference) to an Ethernet MAC controller node.

The Ethernet MAC controller behind Port 5 does not have the necessary
bandwidth allocation at the memory controller level to sustain Gigabit
traffic with 64B packets. Instead we want to use Port 8 and the Ethernet
MAC connected to it which has been budgeted to support that bandwidth.

The reason why we want to have a representor (DSA_PORT_TYPE_USER) for
port 5 of the switch is also because we need to set-up Compact Field
Processor (CFP) rules which need to identify specific packets ingressing
a specific port number and to be redirected towards another port (7) for
audio/video streaming processing. The interface to configure CFP is
currently ethool::rxnfc and that requires a net_device handle,
cls_flower would be the same AFAICT.
-- 
Florian

Powered by blists - more mailing lists