Date:   Wed, 18 Sep 2019 15:02:41 -0700
From:   Florian Fainelli <f.fainelli@...il.com>
To:     Vladimir Oltean <olteanv@...il.com>
Cc:     Sascha Hauer <s.hauer@...gutronix.de>,
        netdev <netdev@...r.kernel.org>, Andrew Lunn <andrew@...n.ch>,
        Vivien Didelot <vivien.didelot@...oirfairelinux.com>,
        kernel@...gutronix.de
Subject: Re: dsa traffic priorization

On 9/18/19 12:39 PM, Vladimir Oltean wrote:
> Hi Florian,
> 
> On Wed, 18 Sep 2019 at 20:42, Florian Fainelli <f.fainelli@...il.com> wrote:
>>
>> On 9/18/19 7:36 AM, Vladimir Oltean wrote:
>>> Hi Sascha,
>>>
>>> On Wed, 18 Sep 2019 at 17:03, Sascha Hauer <s.hauer@...gutronix.de> wrote:
>>>>
>>>> Hi All,
>>>>
>>>> We have a customer using a Marvell 88e6240 switch with Ethercat on one port and
>>>> regular network traffic on another port. The customer wants to configure two things
>>>> on the switch: First Ethercat traffic shall be priorized over other network traffic
>>>> (effectively prioritizing traffic based on port). Second the ethernet controller
>>>> in the CPU is not able to handle full bandwidth traffic, so the traffic to the CPU
>>>> port shall be rate limited.
>>>>
>>>
>>> You probably already know this, but egress shaping will not drop
>>> frames, just let them accumulate in the egress queue until something
>>> else happens (e.g. queue occupancy threshold triggers pause frames, or
>>> tail dropping is enabled, etc). Is this what you want? It sounds a bit
>>> strange to me to configure egress shaping on the CPU port of a DSA
>>> switch. That literally means you are buffering frames inside the
>>> system. What about ingress policing?
>>
>> Indeed, but I suppose that depending on the switch architecture and/or
>> nomenclature, configuring egress shaping amounts to determining ingress
>> for the ports where the frame is going to be forwarded to.
> 
> Egress shaping in the switches I've played with has nothing to do with
> the ingress port, unless that is used as a key for some sort for QoS
> classification (aka selector for a traffic class).
> Furthermore, shaping means queuing (which furthermore means delaying,
> but not dropping except in extreme cases which are outside the scope
> of shaping itself), while policing by definition means early dropping
> (admission control). Like Dave Taht pointed out too, dropping might be
> better for the system's overall latency.
> 
>>
>> For instance Broadcom switches rarely if at all mention ingress because
>> the frames have to originate from somewhere and be forwarded to other
>> port(s), therefore, they will egress their original port (which for all
>> practical purposes is the direct continuation of the ingress stage),
>> where shaping happens, which immediately influences the ingress shaping
>> of the destination port, which will egress the frame eventually because
>> packets have to be delivered to the final port's egress queue anyway.
>>
> 
> You lost me.
> I have never heard of any shaping done inside the guts of a switch, so
> 'egress of an ingress port' and 'ingress of an egress port' makes no
> sense to me.

What I meant to explain is that when a packet enters the switch, the
forwarding logic will determine the destination port(s) and queue of
that packet. As soon as the egress port and queue is known the capacity
of that port and queue can be looked up, and that has the immediate
effect of changing how the packets are ingress the switch. I suspect
that the Marvell switches (like Broadcom) use the ability to perform any
form of packet mangling from the perspective of the port's egress (as
opposed to ingress) because ultimately packets need to go somewhere and
the switch logic tries as much as possible to "connect" the ingress port
to the egress port logic to limit on-chip buffering.

> I was talking about ingress policing at the front panel ports, for
> their best-effort traffic. I think that is actually preferable to
> egress shaping at the CPU port, since I don't think they would want
> the EtherCAT traffic getting delayed.

I would view this as classifying and putting a higher priority on
specific types of traffic and making sure that the switch is configured
not to drop certain traffic landing in specific queues. Where that is
applied (ingress, or egress) amounts to the same thing IMHO.

> Alternatively, maybe the DSA master port supports per-stream hardware
> policing, although that is more exotic.

Even if the DSA master network device does not support it, the switch
probably does, so you could do it on traffic that ingresses or egresses
the CPU port.

> 
>>>
>>>> For reference the patch below configures the switch to their needs. Now the question
>>>> is how this can be implemented in a way suitable for mainline. It looks like the per
>>>> port priority mapping for VLAN tagged packets could be done via ip link add link ...
>>>> ingress-qos-map QOS-MAP. How the default priority would be set is unclear to me.
>>>>
>>>
>>> Technically, configuring a match-all rxnfc rule with ethtool would
>>> count as 'default priority' - I have proposed that before. Now I'm not
>>> entirely sure how intuitive it is, but I'm also interested in being
>>> able to configure this.
>>
>> That does not sound too crazy from my perspective.
>>
> 
> Ok, well at least that requires no user space modification, then.
> 
>>>
>>>> The other part of the problem seems to be that the CPU port has no network device
>>>> representation in Linux, so there's no interface to configure the egress limits via tc.
>>>> This has been discussed before, but it seems there hasn't been any consensous regarding how
>>>> we want to proceed?
>>
>> You have the DSA master network device which is on the other side of the
>> switch,
>> --
>> Florian
> 
> Thanks,
> -Vladimir
> 


-- 
Florian

Powered by blists - more mailing lists