Message-ID: <20210118095928.001b5687@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
Date:   Mon, 18 Jan 2021 09:59:28 -0800
From:   Jakub Kicinski <kuba@...nel.org>
To:     Jiri Pirko <jiri@...nulli.us>
Cc:     netdev@...r.kernel.org, davem@...emloft.net,
        jacob.e.keller@...el.com, roopa@...dia.com, mlxsw@...dia.com
Subject: Re: [patch net-next RFC 00/10] introduce line card support for
 modular switch

On Mon, 18 Jan 2021 14:00:09 +0100 Jiri Pirko wrote:
> >> >Or to put it differently IMO the netdev should be provisioned if the
> >> >system has a port into which user can plug in a cable. When there is     
> >> 
> >> Not really. For slit cables, the ports are provisioned not matter which
> >> cable is connected, slitter 1->2/1->4 or 1->1 cable.
> >> 
> >>   
> >> >a line card-sized hole in the chassis, I'd be surprised to see ports.
> >> >
> >> >That said I never worked with real world routers so maybe that's what
> >> >they do. Maybe some with a Cisco router in the basement can tell us? :)    
> >> 
> >> The need for provision/pre-configure splitter/linecard is that the
> >> ports/netdevices do not disapper/reappear when you replace
> >> splitter/linecard. Consider a faulty linecard with one port burned. You
> >> just want to replace it with new one. And in that case, you really don't
> >> want kernel to remove netdevices and possibly mess up routing for
> >> example.  
> >
> >Having a single burned port sounds like a relatively rare scenario.  
> 
> Hmm, rare in scale is common...

Sure but at a scale of million switches it doesn't matter if a couple
are re-configuring their routing.

> >Reconfiguring routing is not the end of the world.  
> 
> Well, yes, but you don't really want netdevices to come and go then you
> plug in/out cables/modules. That's why we have split implemented as we
> do. I don't understand why do you think linecards are different.

If I have an unused port it will still show up as a netdev.
If I have an unused phymod slot w/ a slot cover in it, why would there
be a netdev? Our definition of a physical port is something like "a
socket for a networking cable on the outside of the device". With your
code I can "provision" a phymod and there is no whole to plug in a
cable. If we follow the same logic, if I have a server with PCIe
hotplug, why can't I "provision" some netdevs for a NIC that I will
plug in later?

> Plus, I'm not really sure that our hw can report the type, will check.

I think that's key.

> One way or another, I think that both configuration flows have valid
> usecase. Some user may want pre-configuration, some user may want auto.
> Btw, it is possible to implement splitter cable in auto mode as well.

Auto as in iterate over possible configs until link up? That's nasty.

> >> >If the device really needs this configuration / can't detect things
> >> >automatically, then we gotta do something like what you have.
> >> >The only question is do we still want to call it a line card.
> >> >Sounds more like a front panel module. At Netronome we called 
> >> >those phymods.    
> >> 
> >> Sure, the name is up to the discussion. We call it "linecard"
> >> internally. I don't care about the name.  
> >
> >Yeah, let's call it something more appropriate to indicate its
> >breakout/retimer/gearbox nature, and we'll be good :)  
> 
> Well, it can contain much more. It can contain a smartnic/fpga/whatever
> for example. Not sure we can find something that fits to all cases.
> I was thinking about it in the past, I think that the linecard is quite
> appropriate. It connects with lines/lanes, and it does something,
> either phy/gearbox, or just interconnects the lanes using smartnic/fpga
> for example.

If it has a FPGA / NPU in it, it's definitely auto-discoverable. 
I don't understand why you think that it's okay to "provision" NICs
which aren't there but only for this particular use case.

Powered by blists - more mailing lists