netdev - Re: [PATCH net-next 0/2] devlink: Add port function attribute to enable/disable roce

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20210203132605.7faf8ca0@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
Date:   Wed, 3 Feb 2021 13:26:05 -0800
From:   Jakub Kicinski <kuba@...nel.org>
To:     Saeed Mahameed <saeed@...nel.org>
Cc:     Yishai Hadas <yishaih@...dia.com>, netdev@...r.kernel.org,
        davem@...emloft.net, parav@...dia.com
Subject: Re: [PATCH net-next 0/2] devlink: Add port function attribute to
 enable/disable roce

On Wed, 03 Feb 2021 11:22:44 -0800 Saeed Mahameed wrote:
> On Wed, 2021-02-03 at 10:51 -0800, Jakub Kicinski wrote:
> > On Tue, 02 Feb 2021 20:13:48 -0800 Saeed Mahameed wrote:  
> > > yes, user in this case is the admin, who controls the provisioned
> > > network function SF/VFs.. by turning off this knob it allows to
> > > create
> > > more of that resource in case the user/admin is limited by memory.  
> > 
> > Ah, so in case of the SmartNIC this extra memory is allocated on the
> > control system, not where the function resides?
> 
> most of the memeory are actually allocated from where the function
> resides, some are on the management system but it is not as critical.
> SFs for now can only be probed on the management system, so the main
> issue will be on the SmartNIC side for now.

Why not leave the decision whether to allocate that memory or not to
the SF itself? If user never binds the RDMA driver to the SF they
clearly don't care for RDMA. No extra knobs needed.

> > My next question is regarding the behavior on the target system -
> > what
> > does "that user" see? Can we expect they will understand that the
> > limitation was imposed by the admin and not due to some
> > initialization
> > failure or SW incompatibility?
>
> the whole thing works with only real HW capabilities, there is no
> synthetic SW capabilities. 
> 
> when mlx5 instance driver loads, it doesn't assume anything about
> underlying HW, and it queries for the advertised FW capability
> according to the HW spec before it enables a feature.
> 
> so this patch adds the ability for admin to enforce a specific HW cap
> "off" for a VF/SF hca slice.
> 
> > > RAW eth QP, i think you already know this one, it is a very thin
> > > layer
> > > that doesn't require the whole rdma stack.  
> > 
> > Sorry for asking a leading question. You know how we'll feel about
> > that one, do we need to talk this out or can we save ourselves the
> > battle? :S  
> 
> I know, I know :/
> 
> So, there is no rdma bit/cap in HW.. to disable non-RoCE commands we
> will have to disable etherent capability. 

It's your driver, you can make it do what you need to. Why does 
the RDMA driver bind successfully to a non-RoCE Ethernet device 
in the first place?

> The user interface here has no synthetic semantics, all knobs will
> eventually be mapped to real HW/FW capabilities to get disabled.
> 
> the whole feature is about allowing admin to ship network functions
> with different capabilities that are actually enforced by FW/HW.. 
> so the user of the VF will see, RDMA/ETH only cards or both.

RDMA-only, ETH-only, RDMA+ETH makes sense to me. Having an ETH-only
device also exposed though rdma subsystem does not.