netdev - Re: [RFC] current devlink extension plan for NICs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200323233200.GD21532@C02YVCJELVCG.greyhouse.net>
Date:   Mon, 23 Mar 2020 19:32:00 -0400
From:   Andy Gospodarek <andy@...yhouse.net>
To:     Jiri Pirko <jiri@...nulli.us>
Cc:     Jakub Kicinski <kuba@...nel.org>, netdev@...r.kernel.org,
        davem@...emloft.net, parav@...lanox.com, yuvalav@...lanox.com,
        jgg@...pe.ca, saeedm@...lanox.com, leon@...nel.org,
        andrew.gospodarek@...adcom.com, michael.chan@...adcom.com,
        moshe@...lanox.com, ayal@...lanox.com, eranbe@...lanox.com,
        vladbu@...lanox.com, kliteyn@...lanox.com, dchickles@...vell.com,
        sburla@...vell.com, fmanlunas@...vell.com, tariqt@...lanox.com,
        oss-drivers@...ronome.com, snelson@...sando.io,
        drivers@...sando.io, aelior@...vell.com,
        GR-everest-linux-l2@...vell.com, grygorii.strashko@...com,
        mlxsw@...lanox.com, idosch@...lanox.com, markz@...lanox.com,
        jacob.e.keller@...el.com, valex@...lanox.com,
        linyunsheng@...wei.com, lihong.yang@...el.com,
        vikas.gupta@...adcom.com, magnus.karlsson@...el.com
Subject: Re: [RFC] current devlink extension plan for NICs

On Sat, Mar 21, 2020 at 10:35:25AM +0100, Jiri Pirko wrote:
> Fri, Mar 20, 2020 at 10:25:08PM CET, kuba@...nel.org wrote:
> >On Fri, 20 Mar 2020 08:35:55 +0100 Jiri Pirko wrote:
> >> Fri, Mar 20, 2020 at 04:32:53AM CET, kuba@...nel.org wrote:
> >> >On Thu, 19 Mar 2020 20:27:19 +0100 Jiri Pirko wrote:  
[...]
> >
> >Also, once the PFs are created user may want to use them together 
> >or delegate to a VM/namespace. So when I was thinking we'd need some 
> >sort of a secure handshake between PFs and FW for the host to prove 
> >to FW that the PFs belong to the same domain of control, and their
> >resources (and eswitches) can be pooled.
> >
> >I'm digressing..
> 
> Yeah. This needs to be sorted out.
> 
> 
> >
> >> Now the PF itself can have a "nested eswitch" to manage. The "parent
> >> eswitch" where the PF was created would only see one leg to the "nested
> >> eswitch".
> >> 
> >> This "nested eswitch management" might or might not be required. Depends
> >> on a usecare. The question was, how to configure that I as a user
> >> want this or not.
> >
> >Ack. I'm extending your question. I think the question is not only who
> >controls the eswitch but also which PFs share the eswitch.
> 
> Yes.
> 

So we have implemented the notion of an 'adminstrative PF.'  This is a
gross simplification, but the idea is that the PCI domain (or CPU
complex) that contains this PF is the one that is 'in-charge' of the
eSwitch and the rest of the resources (firmware/phycode update) and
might also be the one that gets the VF representors when VFs are created
on any other PCI host/domains.

I'm not sure we need a kernel API to set it as I would leave this as
something that might be burned into the hardware in some manner.

> >
> >I think eswitch is just one capability, but SmartNIC will want to
> >control which ports see what capabilities in general. crypto offloads
> >and such.
> >
> >I presume in your model if host controls eswitch the smartNIC sees just
> 
> host may control the "nested eswitch" in the SmartNIC case.
> 

I'm not sure programming the eswitch in a nested manner is realistic.
Sure we can make hardware do it, but it's probably more trouble than
it's worth.  If a smartnic wants to give control of flows to the host
then it makes more sense to allow some communication at a higher layer
so that requests for hardware offload can be easily validated against
some sort of policy set forth by the admin of the smartnic.

> >what what comes out of Hosts single "uplink"? What if SmartNIC wants
> >the host to be able to control the forwarding but not loose the ability
> >to tap the VF to VF traffic?
> 
> You mean that the VF representors would be in both SmartNIC host and
> host? I don't know how that could work. I think it has to be either
> there or there.
> 

Agreed.  The VF reps should probably appear on whichever host/domain has
the Admin PF.