Date:	Tue, 26 Jul 2016 11:58:32 -0700
From:	Tom Herbert <tom@...bertland.com>
To:	John Fastabend <john.fastabend@...il.com>
Cc:	Thomas Monjalon <thomas.monjalon@...nd.com>,
	Jesper Dangaard Brouer <brouer@...hat.com>,
	Tom Herbert via iovisor-dev <iovisor-dev@...ts.iovisor.org>,
	Jakub Kicinski <jakub.kicinski@...ronome.com>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	"Fastabend, John R" <john.r.fastabend@...el.com>,
	Edward Cree <ecree@...arflare.com>,
	Simon Horman <simon.horman@...ronome.com>,
	Rana Shahout <ranas@...lanox.com>,
	Or Gerlitz <ogerlitz@...lanox.com>, Ari Saha <as754m@....com>,
	adrien.mazarguil@...nd.com
Subject: Re: [iovisor-dev] XDP seeking input from NIC hardware vendors

On Tue, Jul 26, 2016 at 10:53 AM, John Fastabend
<john.fastabend@...il.com> wrote:
> On 16-07-26 09:08 AM, Tom Herbert wrote:
>> On Tue, Jul 26, 2016 at 6:31 AM, Thomas Monjalon
>> <thomas.monjalon@...nd.com> wrote:
>>> Hi,
>>>
>>> About RX filtering, there is an ongoing effort in DPDK to write an API
>>> which could leverage most of the hardware capabilities of any NICs:
>>>         https://rawgit.com/6WIND/rte_flow/master/rte_flow.html
>>>         http://thread.gmane.org/gmane.comp.networking.dpdk.devel/43352
>>> I understand that XDP does not target to support every hardware features,
>>> though it may be an interesting approach to check.
>>>
>> Thomas,
>>
>> A major goal of XDP is to leverage and in fact encourage innovation in
>> hardware features. But, we are asking that vendors design the APIs
>> with the community in mind. For instance, if XDP supports crypto
>> offload it should have one API that different companies, we don't want
>> every vendor coming up with their own.
>
> The work in those threads is to create a single API for users of DPDK
> to interact with their hardware. The equivalent interface in Linux
> kernel is ntuple filters from ethtool the effort here is to make a
> usable interface to manage this from an application and also expose
> all the hardware features. Ethtool does a fairly poor job on both
> fronts IMO.
>
> If we evolve the mechanism to run per rx queue xdp programs this
> interface could easily be used to forward packets to specific rx
> queues and run targeted xdp programs.
>
> Integrating this functionality into running XDP programs as ebpf code
> seems a bit challenging to me because there is no software equivalent.
> Once XDP ebpf program is running the pkt has already landed on the rx
> queue. To me the mechanism to bind XDP programs to rx queues and steer
> specific flows (e.g. match a flow label and forward to a queue) needs
> to be part of the runtime environment not part of the main ebpf program
> itself. The runtime environment could use the above linked API. I know
> we debated earlier including this in the ebpf program itself but that
> really doesn't seem feasible to me. Whether the steering is expresses
> as an ebpf program or an API like above seems like a reasonable
> discussion. Perhaps a section could be used to describe the per program
> filter for example which would be different from an API approach used
> in the proposal above or the JIT could translate it into the above
> API for devices without instruction based hardware.
>
I think your convoluting two different mechanisms. If the device has
the capability to different packets and steer them to different
queues, then XDP should be able to make use of that by running
different programs appropriate for each queue. Packet steering is the
domain of HW, for that we have ntuple filtering now but there is no
reason to believe that XDP won't be used for that also. Per queue
program in the host is just configuration, i.e. bind this program to
that queue. Even if the first is not ready, I don't see why the second
is so complex; it should just be a matter of per queue configuration
for which there is already a lot of infrastructure.

> Step 0 should be to show a set of compelling use cases that want to run
> per queue programs then we can talk about the runtime.

Imagine we are able to split VM traffic and non-VM traffic to
different RX queues. The program we will want to run would be very
different in those cases.

Tom

>
>
>>
>>> 2016-07-12 22:32, Jesper Dangaard Brouer via iovisor-dev:
>>>> On Tue, 12 Jul 2016 12:13:01 -0700
>>>> John Fastabend <john.fastabend@...il.com> wrote:
>>>>>
>>>>> Another use case I have is to make a really high performance AF_PACKET
>>>>> interface. So if there was a way to say bind a queue to an AF_PACKET
>>>>> ring and run a policy XDP program before hitting the AF_PACKET
>>>>> descriptor bit that would be really interesting because it would solve
>>>>> some of my need for poll mode drivers in userspace.
>>>
>>> Have you started this work?
>>> Do you have an idea of how RX would perform through XDP + AF_PACKET + DPDK?
>>>
>> I don't understand why the AF_PACKET with DPDK. They should be
>> mutually exclusive. XDP over DPDK does make sense.
>>
>
> Because DPDK is more than just a poll mode driver that binds to a
> device. AF_Packet could be used as a replacement for a specific poll
> mode driver but the application could still use the other libraries
> provided by DPDK to build ACLs for example or deep packet inspection.
>
>
>> Tom
>>
>

Powered by blists - more mailing lists