Date:   Wed, 31 Jan 2018 19:53:24 +0100
From:   Björn Töpel <bjorn.topel@...il.com>
To:     Ilias Apalodimas <ilias.apalodimas@...aro.org>
Cc:     Björn Töpel <bjorn.topel@...el.com>,
        "Karlsson, Magnus" <magnus.karlsson@...el.com>,
        "Duyck, Alexander H" <alexander.h.duyck@...el.com>,
        John Fastabend <john.fastabend@...il.com>,
        Alexei Starovoitov <ast@...com>,
        Jesper Dangaard Brouer <brouer@...hat.com>,
        michael.lundkvist@...csson.com, ravineet.singh@...csson.com,
        Daniel Borkmann <daniel@...earbox.net>,
        Netdev <netdev@...r.kernel.org>,
        Francois Ozog <francois.ozog@...aro.org>,
        MykytaI Iziumtsev <mykyta.iziumtsev@...aro.org>,
        "Brandeburg, Jesse" <jesse.brandeburg@...el.com>,
        "Singhai, Anjali" <anjali.singhai@...el.com>,
        "Rosen, Rami" <rami.rosen@...el.com>,
        "Shaw, Jeffrey B" <jeffrey.b.shaw@...el.com>,
        "Yigit, Ferruh" <ferruh.yigit@...el.com>,
        "Zhang, Qi Z" <qi.z.zhang@...el.com>
Subject: Re: AF_PACKET V4/AF_XDP userspace API questions

2018-01-30 8:58 GMT+01:00 Ilias Apalodimas <ilias.apalodimas@...aro.org>:
> We've noticed 3 different hardware approaches in receiving payloads
>
> 1. Host driver needs to pre-load descriptor ring with addresses of RAM
> buffers to write arriving data.
> The "standard" functionality for most NICs is (in little detail) fetch the
> descriptor, write the payload to host RAM and update the descriptor
> accordingly.
> So for these NICs, buffer addresses are provided in RX descriptors (RX
> descriptors are two-way communication entity).
> This translates to "1 ring + 1 buffer array" model, or the packet array
> model in short.
>
> 2. There's a category of NICs (Chelsio and Netcope are the ones we are aware
> of) that split that into two one-way entities:
> One to communicate buffer addresses from host to NIC and one to communicate
> packets/payloads from NIC to host.
> So the driver provides a set of unstructured, contiguous memory areas to the
> NIC, the NIC decides where to place the packets in memory and updates the
> descriptors accordingly (the descriptor ring is not pre-loaded with any data
> and the NIC is free to write the packet anywhere in the provided contiguous
> memory).
> This is a "1 ring + 1 set of areas" model, or the tape model in short.
>
> 3. The last hardware approach we are aware of is NICs that you provide
> multiple array buffers (128, 256, 1500, 9000 etc).
> The NIC then decides in which array slot to place the packet depending on
> it's size.
> This is "1 ring + X buffer arrays" model or the multi packet array in short.
>
> Is memory schemes 2 and 3 supported? If not do you plan on supporting them?
>

Note that AF_PACKET V4 has evolved into AF_XDP, and will have the same
buffer constraints as XDP has/will get.

Option 2 (zero copy) with the current interface will probably be a bit
tricky. It could be possible, but would require the kernel/driver to
track when enough buffers has been gifted to the kernel to form a a
contiguous memory.

As for option 3, supporting different buffer sizes on the same socket
could in theory be supported, but that would require too much
house-holding from the driver. Supporting multiple (different) buffer
sizes to the same interface using different sockets should be doable
from a driver perspective.

Thanks for raising the concern -- the kernel side of AF_XDP/zero copy
must be usable for multiple vendors to make sense. I'm open to any
suggestions to the API that would make it easier for other vendors.


Thanks,
Björn

> Regards
> Ilias
>
>
>
>

Powered by blists - more mailing lists