netdev - RE: [for-next 4/6] net/mlx5: FPGA, Add basic support for Innova

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <AM4PR0501MB194008AAABEB6AAAA2ADFC82DBCB0@AM4PR0501MB1940.eurprd05.prod.outlook.com>
Date:   Tue, 6 Jun 2017 06:52:15 +0000
From:   Ilan Tayari <ilant@...lanox.com>
To:     Jason Gunthorpe <jgunthorpe@...idianresearch.com>
CC:     Alexei Starovoitov <alexei.starovoitov@...il.com>,
        Saeed Mahameed <saeedm@....mellanox.co.il>,
        "David S. Miller" <davem@...emloft.net>,
        "Doug Ledford" <dledford@...hat.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "linux-rdma@...r.kernel.org" <linux-rdma@...r.kernel.org>,
        "jsorensen@...com" <jsorensen@...com>,
        "Andy Shevchenko" <andy.shevchenko@...il.com>,
        "linux-fpga@...r.kernel.org" <linux-fpga@...r.kernel.org>,
        Alan Tull <atull@...nsource.altera.com>,
        "yi1.li@...ux.intel.com" <yi1.li@...ux.intel.com>,
        Boris Pismenny <borisp@...lanox.com>
Subject: RE: [for-next 4/6] net/mlx5: FPGA, Add basic support for Innova

> -----Original Message-----
> From: Jason Gunthorpe [mailto:jgunthorpe@...idianresearch.com]
> Subject: Re: [for-next 4/6] net/mlx5: FPGA, Add basic support for Innova
> 
> On Sun, Jun 04, 2017 at 07:51:24AM +0000, Ilan Tayari wrote:
> > > From: Jason Gunthorpe [mailto:jgunthorpe@...idianresearch.com]
> > > Subject: Re: [for-next 4/6] net/mlx5: FPGA, Add basic support for
> Innova
> > >
> > > On Mon, May 29, 2017 at 04:09:06PM +0000, Ilan Tayari wrote:
> > >
> > > > > For IPSec, this is already in the kernel.
> > > > > See this patchset:
> > > > > http://www.mail-archive.com/netdev@vger.kernel.org/msg162876.html
> > > >
> > > > Sorry, I pointed at the RFC by mistake.
> > > >
> > > > This is the relevant pull request:
> > > > https://patchwork.ozlabs.org/patch/752707/
> > >
> > > This is connecting ipsec to a netdev, while Innova seems to be a
> >
> > Jason,
> >
> > "network connected ipsec accelerator configured using IP packets."
> > No. This is incorrect.
> > Where did you get that from?
> 
> That is what you described to me - you said the only way to configure
> the FPGA was via IP packets in-band. It is not part of the NIC and the
> NIC only loads the FPGA bitstream.
> 
> Maybe you should explain more how this works?

I think this is just a misunderstanding.
The FPGA is part of this NIC, not something external.

The fact that we configure the FPGA using special inband packets isn't
changing anything. IMO, it might have been any other bus on the card,
standard or proprietary, and the arguments for how to design the driver
would stay the same.

Note that these packets are not generated as IP packets at the host.
They are RoCE packets. The command payload is generated by the driver,
then the QP and RoCE headers are generates by the ConnectX ASIC. The packet
is then consumed by the FPGA.
On the way back, the response is generated by the FPGA, and consumed by
the ConnectX ASIC, with the response payload delivered to the QP in the
driver.

So neither the host stack nor the network are aware of them.
They exist momentarily only on the internal traces on the board and not
anywhere else.

> 
> > So you configure it from userspace with regular IPSec 'ip xfrm
> > state' commands or over netlink with your favorite IKE daemon.
> 
> But you don't give an ip xfrm configuration to the NIC when you submit
> the work request? By your description it sounded liked the FPGA
> pattern matches packets from the NIC side to apply the xfrm.
> 
> Jason

I didn't want to dig into this, because it's not submitted yet.

But here's an example configuration flow for creating an IPSec SA:

* User gives 'ip xfrm state add' command in shell, with 'offload' argument
* iproute2 sends XFRM_MSG_NEWSA netlink message with XFRMA_OFFLOAD_DEV attribute
* Xfrm stack generates a new xfrm_state object and calls the netdevice's xdo_dev_state_add callback
* mlx5e driver formats HW-specific ADD_SA command packet and sends it over RoCE QP to FPGA
* FPGA adds the SA to its offloaded SADB, and sends a response packet for 'success'

That's pretty much it.
Does this help to understand what goes on?

I don't mind explaining further, but I think you will just see it in the
patchset when we submit.

Ilan.