Message-ID: <1480531441.3563.148.camel@infinera.com>
Date:   Wed, 30 Nov 2016 18:44:04 +0000
From:   Joakim Tjernlund <Joakim.Tjernlund@...inera.com>
To:     "f.fainelli@...il.com" <f.fainelli@...il.com>,
        "andrew@...n.ch" <andrew@...n.ch>
CC:     "netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: Re: DSA vs. SWTICHDEV ?

On Wed, 2016-11-30 at 10:10 -0800, Florian Fainelli wrote:
> On 11/30/2016 09:44 AM, Joakim Tjernlund wrote:
> > On Wed, 2016-11-30 at 17:55 +0100, Andrew Lunn wrote:
> > > > This is an embedded system with several boards in a subrack.
> > > > Each board has eth I/F connected to a switch to communicate with each other.
> > > > One of the board will also house the actual switch device and manage the switch.
> > > > Then the normal app just communicates over the physical eth I/F like any other board
> > > > in the system. There is a "manage switch app" which brings the switch up and partition
> > > > phys VLANs etc. (each phys I/F would be a a separate domain so no loop)
> > > 
> > > So you are planning on throwing away the "manage switch app", and just
> > > use standard linux networking commands? That is what switchdev is all
> > > about really, throwing away the vendor SDK for the switch, making a
> > > switch just a bunch on interfaces on the host which you manage as
> > > normal interfaces.
> > 
> > Something like that. I need to run routing protocols on the switch I/Fs and egress
> > pkgs on selected switch I/Fs bypassing ARP, just like DSA does with its vendor
> > tags.
> > 
> > > 
> > > > I guess I could skip the phys I/F and have the switch app create a virtual eth0 I/F over PCIe
> > > 
> > > No need to create this interface. It will exist if you go the
> > > switchdev route.
> > > 
> > > > > > And switchdev can do all this over PCIe instead? Can you have a
> > > > > > switch tree in switchdev too?
> > > > > 
> > > > > Mellonex says so, but i don't think they have actually implemented it.
> > > > 
> > > > Not impl. any of DSAs features? What can you do with a Mellonex switch then?
> > > 
> > > They don't have a tree of switches, as far as i know. Just a single
> > > switch. But DSA does support a tree of switches, that is what the D in
> > > DSA means, distributed. And there are a couple of boards which have 2
> > > to 4 switches in a tree.
> > 
> > We might have a tree as well so now I really wonder: Given we write a
> > proper switchdev driver, can it support switchtrees without touching
> > switchdev infra structure? If not I guess we will attach a physical
> > eth I/F to the switch and use both DSA and switchdev to support both trees
> > and HW offload. 
> 
> switchdev in itself really is the glue layer between the networking
> stack and how to push specific objects down to the Ethernet switch
> driver, and that Ethernet switch driver. Switchdev does not enforce a
> specific network device driver model object, and just provides standard
> hooks for your network devices to register with switchdev in order to
> push/receive offloads. DSA on the other hand, utilizes switchdev to get
> notifications about offloads from the networking stack, but also exposes
> a clearly and well defined Ethernet switch device driver model, as
> Andrew described, it creates per-port network devices, binds the ports
> to their PHYs (built-in, or external), and also takes care of
> encapsulating/decapsulating the switch specific tagging protocol.

Lets see, switchdev is mainly for offloading L2/L3 into HW and does NOT create
virtual I/F(one for each phys sw port) so if my only goal is to offload I don't
need DSA? (How do one create routes if no virtual I/Fs I wonder ..) 

DSA then does create virt. I/Fs and manages switch trees, to actually tx pkg it needs
a phys I/F using vendor specific tags(for PCIe connected switches I can envision using
the PCIe connection instead).

Who is the master when using both DSA and switchdev w.r.t initil conf/bring up of switch?
configuring VLANs?

> 
> We should probably put that in some crystal clear sentence somewhere in
> Documentation/networking/ but switchdev and DSA are complementary and
> not competitors, they just do not tackle the problems from the same angle.

Yes :)

> 
> > 
> > > 
> > > I think this is partially down to market segments. Mellonex market is
> > > top of rack switches. High port count, very high bandwidth. DSA is
> > > more wireless access points, set top boxes, generally up to 7 ports of
> > > 1Gbps and a few custom embedded products which need more ports, so
> > > build a tree of switches.
> > 
> > We have on an existing board with a BCM ROBO switch with lots of ports(>24),
> > managed over SPI. Looking at BCM DSA tag code it looks like it only supports
> > some 8 ports or so. I still have to find out if this is a limitation in BCM tagging
> > protocol or if just not impl. in DSA yet.
> 
> Oh cool, can you share the model by chance? I suspect the tagging format

I think it is an BCM5322

> of that switch is going to be different than what net/dsa/tag_brcm.c, so
> feel free to add something NET_DSA_TAG_BRCM8B (for 8 bytes) or something
> like that.

I will have to look at that, don't have an docs handy.

> 
> Note that DSA currently hardcodes the maximum number of ports to 14
> (DSA_MAX_PORTS), but this should obviously be something dynamically
> determined based on probing the switch device.
> 
> Can you also evaluate if using drivers/net/dsa/b53/ would work for you?
> My hope would be that they preserved the register compatibility here,
> but since this has a large number of ports, it may have completely
> offset most registers.

I will have a look, I sure want to try out DSA/switchdev on existing boards if I can :)

> 
> BTW, there is #linuxswitch on Freenode if you want to chat!

Powered by blists - more mailing lists