netdev - RE: [PATCH net-next 00/19] Mellanox, mlx5 sub function support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <AM0PR05MB48667670B88B06C8A2401F97D17B0@AM0PR05MB4866.eurprd05.prod.outlook.com>
Date:   Fri, 8 Nov 2019 16:06:16 +0000
From:   Parav Pandit <parav@...lanox.com>
To:     Jiri Pirko <jiri@...nulli.us>,
        Jakub Kicinski <jakub.kicinski@...ronome.com>
CC:     "alex.williamson@...hat.com" <alex.williamson@...hat.com>,
        "davem@...emloft.net" <davem@...emloft.net>,
        "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        Saeed Mahameed <saeedm@...lanox.com>,
        "kwankhede@...dia.com" <kwankhede@...dia.com>,
        "leon@...nel.org" <leon@...nel.org>,
        "cohuck@...hat.com" <cohuck@...hat.com>,
        Jiri Pirko <jiri@...lanox.com>,
        "linux-rdma@...r.kernel.org" <linux-rdma@...r.kernel.org>,
        Or Gerlitz <gerlitz.or@...il.com>
Subject: RE: [PATCH net-next 00/19] Mellanox, mlx5 sub function support



> -----Original Message-----
> From: Jiri Pirko <jiri@...nulli.us>
> Sent: Friday, November 8, 2019 6:13 AM
> To: Jakub Kicinski <jakub.kicinski@...ronome.com>
> Cc: Parav Pandit <parav@...lanox.com>; alex.williamson@...hat.com;
> davem@...emloft.net; kvm@...r.kernel.org; netdev@...r.kernel.org;
> Saeed Mahameed <saeedm@...lanox.com>; kwankhede@...dia.com;
> leon@...nel.org; cohuck@...hat.com; Jiri Pirko <jiri@...lanox.com>; linux-
> rdma@...r.kernel.org; Or Gerlitz <gerlitz.or@...il.com>
> Subject: Re: [PATCH net-next 00/19] Mellanox, mlx5 sub function support
> 
> Thu, Nov 07, 2019 at 09:32:34PM CET, jakub.kicinski@...ronome.com wrote:
> >On Thu,  7 Nov 2019 10:04:48 -0600, Parav Pandit wrote:
> >> Mellanox sub function capability allows users to create several
> >> hundreds of networking and/or rdma devices without depending on PCI
> SR-IOV support.
> >
> >You call the new port type "sub function" but the devlink port flavour
> >is mdev.
> >
> >As I'm sure you remember you nacked my patches exposing NFP's PCI sub
> >functions which are just regions of the BAR without any mdev
> >capability. Am I in the clear to repost those now? Jiri?
> 
> Well question is, if it makes sense to have SFs without having them as mdev?
> I mean, we discussed the modelling thoroughtly and eventually we realized
> that in order to model this correctly, we need SFs on "a bus".
> Originally we were thinking about custom bus, but mdev is already there to
> handle this.
> 
> Our SFs are also just regions of the BAR, same thing as you have.
> 
> Can't you do the same for nfp SFs?
> Then the "mdev" flavour is enough for all.
> 
> 
> >
> >> Overview:
> >> ---------
> >> Mellanox ConnectX sub functions are exposed to user as a mediated
> >> device (mdev) [2] as discussed in RFC [3] and further during
> >> netdevconf0x13 at [4].
> >>
> >> mlx5 mediated device (mdev) enables users to create multiple
> >> netdevices and/or RDMA devices from single PCI function.
> >>
> >> Each mdev maps to a mlx5 sub function.
> >> mlx5 sub function is similar to PCI VF. However it doesn't have its
> >> own PCI function and MSI-X vectors.
> >>
> >> mlx5 mdevs share common PCI resources such as PCI BAR region, MSI-X
> >> interrupts.
> >>
> >> Each mdev has its own window in the PCI BAR region, which is
> >> accessible only to that mdev and applications using it.
> >>
> >> Each mlx5 sub function has its own resource namespace for RDMA
> resources.
> >>
> >> mdevs are supported when eswitch mode of the devlink instance is in
> >> switchdev mode described in devlink documentation [5].
> >
> >So presumably the mdevs don't spawn their own devlink instance today,
> >but once mapped via VIRTIO to a VM they will create one?
> 
> I don't think it is needed for anything. Maybe one day if there is a need to
> create devlink instance for VF or SF, we can add it. But currently, I don't see
> the need.
> 
> 
> >
> >It could be useful to specify.
> >
> >> Network side:
> >> - By default the netdevice and the rdma device of mlx5 mdev cannot
> >> send or receive any packets over the network or to any other mlx5 mdev.
> >
> >Does this mean the frames don't fall back to the repr by default?
> 
> That would be the sane default. If I up the representor, I should see packets
> coming in from SF/VF and I should be able to send packets back.

It is similar to VF. I clarified in previous email.
Wil update the cover letter to update the description.