netdev - RE: [for-next 4/6] net/mlx5: FPGA, Add basic support for Innova

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Sun, 28 May 2017 07:22:27 +0000
From:   Ilan Tayari <ilant@...lanox.com>
To:     Jason Gunthorpe <jgunthorpe@...idianresearch.com>,
        Alexei Starovoitov <alexei.starovoitov@...il.com>
CC:     Saeed Mahameed <saeedm@....mellanox.co.il>,
        "David S. Miller" <davem@...emloft.net>,
        Doug Ledford <dledford@...hat.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "linux-rdma@...r.kernel.org" <linux-rdma@...r.kernel.org>,
        "jsorensen@...com" <jsorensen@...com>,
        Andy Shevchenko <andy.shevchenko@...il.com>,
        "linux-fpga@...r.kernel.org" <linux-fpga@...r.kernel.org>,
        Alan Tull <atull@...nsource.altera.com>,
        "yi1.li@...ux.intel.com" <yi1.li@...ux.intel.com>,
        Boris Pismenny <borisp@...lanox.com>
Subject: RE: [for-next 4/6] net/mlx5: FPGA, Add basic support for Innova

> -----Original Message-----
> From: Jason Gunthorpe [mailto:jgunthorpe@...idianresearch.com]
> 
> On Fri, May 26, 2017 at 10:56:25AM -0700, Alexei Starovoitov wrote:
> 
> > > for that feature which is the originating place, before defining
> > > APIs/infrastructures,
> > > until the feature is complete and every body is happy about it.
> >
> > There is driver/fpga to manage fpga, but mlx fpga+nic combo
> > will be managed via mlx5/core/fpga/
> >
> > Adding fpga folks for visibility.
> 
> It would be good to use the existing fpga loading infrastructure to
> get the bitstream into the NIC, and to use the same Xilinx bitstream
> format as eg Zynq does for consistency.
> 
> I'm unclear how this works - there must be more to it than just a
> 'bump on the wire', there must be some communication channel between
> the FPGA and Linux to set operational data (eg load keys) etc.
> 
> If that is register mapped into a PCI-BAR someplace then it really
> should use the FPGA layer functions to manage binding drivers to that
> register window.
> 
> If it is mailbox command based then it is not as good of a fit.
> 

Hi Jason,
Thanks for taking the time to consider this!

This is neither PCI-bar mapped, nor mailbox command.
The FPGA is indeed a bump-on-the-wire.
(It has I2C to the CX4 chip, but that is for debug purposes, and too slow
to perform real programming)

The FPGA is programmed with RoCE packets. We are going to share a lot of
details about this in the IPSec offload patchset, so I don't want to 
repeat it here.
In short, we open a RoCEv2 QP between the connect and the FPGA chips.
Over this QP, we can communicate at high speed with the FPGA.

> Is this FPGA expected to be customer programmable? In that case you
> really need the full infrastructure to bind the right driver (possibly
> a customer driver) to the current FPGA, to expose the correct
> operational interface to the kernel.

Most of the Innova family of cards are not customer-programmable.
Over time, they may require an upgrade (essentially a FW upgrade) but not
with customer logic.

One flavor of the product, the Innova Flex, allows customer logic in the
FPGA. But even then it is "wrapped" by Mellanox shell logic.
We plan to have an in-kernel API for writing client drivers for Innova
Flex.

> 
> Jason