Message-ID: <6d53f29d-21ab-21e6-ce94-0b69d3c3cf9f@gmail.com>
Date:   Wed, 13 Nov 2019 11:29:26 -0800
From:   Florian Fainelli <f.fainelli@...il.com>
To:     Andrew Lunn <andrew@...n.ch>, Vladimir Oltean <olteanv@...il.com>
Cc:     netdev <netdev@...r.kernel.org>
Subject: Re: Offloading DSA taggers to hardware

On 11/13/19 8:53 AM, Andrew Lunn wrote:
> Hi Vladimir
> 
> I've not seen any hardware that can do this.

Such hardware exists ant there was a prior attempt at supporting that:

http://linux-kernel.2935.n7.nabble.com/PATCH-net-next-0-3-net-Switch-tag-HW-extraction-insertion-td1162606.html

> There is an Atheros/Qualcom integrated SoC/Switch where the 'header' is actually
> just a field in the transmit/receive descriptor. There is an out of
> tree driver for it, and the tag driver is very minimal. But clearly
> this only works for integrated systems.

It can work between discrete components in premise, it just is unlikely
because of the flexibility of DSA to mix and match MAC and switches and
having different vendors on either end. Of course, even between the same
vendor, the right hand rarely talks to the left hand, so it only has to
be the work of someone who knows both ends.

> 
> The other 'smart' features i've seen in NICs with respect to DSA is
> being able to do hardware checksums. Freescale FEC for example cannot
> figure out where the IP header is, because of the DSA header, and so
> cannot calculate IP/TCP/UDP checksums. Marvell, and i expect some
> other vendors of both MAC and switch devices, know about these
> headers, and can do checksumming.

This is probably to be blamed to the fact that most Ethernet switch
tagging protocols did not assign themselves an EtherType, otherwise they
just could do that checksumming. In fact, this even trip up controllers
that are from the same vendor:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a40061ea2e39494104602b3048751341bda374a1

> 
> I'm not even sure there are any NICs which can do GSO or LRO when
> there is a DSA header involved.

Similarly to VLAN devices, would not this be done at the DSA virtual
device level instead?

> 
> In the direction CPU to switch, i think many of the QoS issues are
> higher up the stack. By the time the tagger is involved, all the queue
> discipline stuff has been done, and it really is time to send the
> frame. In the 'post buffer bloat world', the NICs hardware queue
> should be small, so QoS is not so relevant once you reach the TX
> queue. The real QoS issue i guess is that the slave interfaces have no
> idea they are sharing resources at the lowest level. So a high
> priority frames from slave 1 are not differentiated from best effort
> frames from slave 2. If we were serious about improving QoS, we need a
> meta scheduler across the slaves feeding the master interface in a QoS
> aware way.
> 
> In the other direction, how much is the NIC really looking at QoS
> information on the receive path? Are you thinking RPS? I'm not sure
> any of the NICs commonly used today with DSA are actually multi-queue
> and do RPS.

Same hardware as presented above can deliver frames to the desired
switch output queue:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d156576362c07e954dc36e07b0d7b0733a010f7d

> 
> Another aspect here might be, what Linux is doing with DSA is probably
> well past the silicon vendors expected use cases. None of the 'vendor
> crap' drivers i've seen for these SOHO class switches have the level
> of integration we have in Linux. We are pushing the limits of the
> host/switch interfaces much more then vendors do, and so silicon
> vendors are not so aware of the limits in these areas? 

Maybe, but vendors support many basic things we still don't like
controlling broadcast storm suppression (commonly requested feature),
offloading QoS properly etc. etc. What we have achieved so far is IMHO a
solid framework, but there are still many, many features unsupported.

> But DSA is being successful, vendors are taking more notice of it, and maybe with
> time, the host/switch interface will improve. NICs might start
> supporting GSO/LRO when there is a DSA header involved? Multi-queue
> NICs become more popular in this class of hardware and RPS knows how
> to handle DSA headers. But my guess would be, it will be for a Marvell
> NIC paired with a Marvell Switch, Broadcom NIC paired with a Broadcom
> switch, etc. I doubt there will be cross vendor support.
> 
> 	Andrew
> 


-- 
Florian

Powered by blists - more mailing lists