netdev - Re: [RFC] current devlink extension plan for NICs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200324132044.GI20941@ziepe.ca>
Date:   Tue, 24 Mar 2020 10:20:44 -0300
From:   Jason Gunthorpe <jgg@...pe.ca>
To:     Jakub Kicinski <kuba@...nel.org>
Cc:     Jiri Pirko <jiri@...nulli.us>, netdev@...r.kernel.org,
        davem@...emloft.net, parav@...lanox.com, yuvalav@...lanox.com,
        saeedm@...lanox.com, leon@...nel.org,
        andrew.gospodarek@...adcom.com, michael.chan@...adcom.com,
        moshe@...lanox.com, ayal@...lanox.com, eranbe@...lanox.com,
        vladbu@...lanox.com, kliteyn@...lanox.com, dchickles@...vell.com,
        sburla@...vell.com, fmanlunas@...vell.com, tariqt@...lanox.com,
        oss-drivers@...ronome.com, snelson@...sando.io,
        drivers@...sando.io, aelior@...vell.com,
        GR-everest-linux-l2@...vell.com, grygorii.strashko@...com,
        mlxsw@...lanox.com, idosch@...lanox.com, markz@...lanox.com,
        jacob.e.keller@...el.com, valex@...lanox.com,
        linyunsheng@...wei.com, lihong.yang@...el.com,
        vikas.gupta@...adcom.com, magnus.karlsson@...el.com
Subject: Re: [RFC] current devlink extension plan for NICs

On Mon, Mar 23, 2020 at 08:56:19PM -0700, Jakub Kicinski wrote:
> On Mon, 23 Mar 2020 19:06:05 -0300 Jason Gunthorpe wrote:
> > On Mon, Mar 23, 2020 at 12:21:23PM -0700, Jakub Kicinski wrote:
> > > > >I see so you want the creation to be controlled by the same entity that
> > > > >controls the eswitch..
> > > > >
> > > > >To me the creation should be on the side that actually needs/will use
> > > > >the new port. And if it's not eswitch manager then eswitch manager
> > > > >needs to ack it.    
> > > > 
> > > > Hmm. The question is, is it worth to complicate things in this way?
> > > > I don't know. I see a lot of potential misunderstandings :/  
> > > 
> > > I'd see requesting SFs over devlink/sysfs as simplification, if
> > > anything.  
> > 
> > We looked at it for a while, working the communication such that the
> > 'untrusted' side could request a port be created with certain
> > parameters and the 'secure eswitch' could know those parameters to
> > authorize and wire it up was super complicated and very hard to do
> > without races.
> > 
> > Since it is a security sensitive operation it seems like a much more
> > secure design to have the secure side do all the creation and present
> > the fully operational object to the insecure side.
> > 
> > To draw a parallel to qemu & kvm, the untrusted guest VM can't request
> > that qemu create a virtio-net for it. Those are always hot plugged in
> > by the secure side. Same flow here.
> 
> Could you tell us a little more about the races? Other than the
> communication channel what changes between issuing from cloud API
> vs devlink?

If I recall the problems came when trying to work with existing cloud
infrastructure that doesn't assume this operating model. You need to
somehow adapt an async APIs of secure/insecure communication with an
async API inside the cloud world. It was a huge mess.

> Side note - there is no communication channel between VM and hypervisor
> right now, which is the cause for weird designs e.g. the failover/auto
> bond mechanism.

Right, and considering the security concerns building one hidden
inside a driver seems like a poor idea..

> > The VF model is poor because the VF is just a dummy stub until the
> > representor/eswitch side is fully configured. There is no way for the
> > Linux driver to know if the VF is operational or not, so we get weird
> > artifacts where we sometimes bind a driver to a VF (and get a
> > non-working ethXX) and sometimes we don't. 
> 
> Sounds like an implementation issue :S

How so?

> > The only reason it is like this is because of how SRIOV requires
> > everything to be preallocated.
> 
> SF also requires pre-allocated resources, so you're not talking about
> PCI mem space etc. here I assume.

It isn't pre-allocated, the usage of the BAR space is dynamic.

> > The SFs can't even exist until they are configured, so there is no
> > state where a driver is connected to an inoperative SF.
> 
> You mean it doesn't exist in terms of sysfs device entry?

I mean literally do not exist at the HW level.

Jason