Message-ID: <20160713041423.GA69076@ast-mbp.thefacebook.com>
Date:	Tue, 12 Jul 2016 21:14:29 -0700
From:	Alexei Starovoitov <alexei.starovoitov@...il.com>
To:	Tom Herbert <tom@...bertland.com>
Cc:	Jesper Dangaard Brouer <brouer@...hat.com>,
	Brenden Blanco <bblanco@...mgrid.com>,
	"David S. Miller" <davem@...emloft.net>,
	Linux Kernel Network Developers <netdev@...r.kernel.org>,
	Jamal Hadi Salim <jhs@...atatu.com>,
	Saeed Mahameed <saeedm@....mellanox.co.il>,
	Martin KaFai Lau <kafai@...com>, Ari Saha <as754m@....com>,
	Or Gerlitz <gerlitz.or@...il.com>,
	john fastabend <john.fastabend@...il.com>,
	Hannes Frederic Sowa <hannes@...essinduktion.org>,
	Thomas Graf <tgraf@...g.ch>,
	Daniel Borkmann <daniel@...earbox.net>
Subject: Re: [PATCH v8 01/11] bpf: add XDP prog type for early driver filter

On Tue, Jul 12, 2016 at 07:52:53AM -0700, Tom Herbert wrote:
> On Tue, Jul 12, 2016 at 6:14 AM, Jesper Dangaard Brouer
> <brouer@...hat.com> wrote:
> >
> > On Tue, 12 Jul 2016 00:51:24 -0700 Brenden Blanco <bblanco@...mgrid.com> wrote:
> >
> >> Add a new bpf prog type that is intended to run in early stages of the
> >> packet rx path. Only minimal packet metadata will be available, hence a
> >> new context type, struct xdp_md, is exposed to userspace. So far only
> >> expose the packet start and end pointers, and only in read mode.
> >>
> >> An XDP program must return one of the well known enum values, all other
> >> return codes are reserved for future use. Unfortunately, this
> >> restriction is hard to enforce at verification time, so take the
> >> approach of warning at runtime when such programs are encountered. Out
> >> of bounds return codes should alias to XDP_ABORTED.
> >
> > This is going to be a serious usability problem, when XDP gets extended
> > with newer features.
> >
> > How can users know what XDP capabilities a given driver support?
> >
> We talked about this a the XDP summit and I have some slides on this
> (hope to have slides posted shortly) . Drivers, more generally XDP
> platforms, will provide a list APIs that they support. APIs would be
> contained in a sort common specification files that and would have
> some name like basic_xdp_api, adv_switch_api, etc. An XDP program is
> written using one of the published APIs and the API it uses is
> expressed as part of the program. At runtime (possibly compile time)
> the API used by the program is checked against the list of APIs for
> the target platform-- if the API is not in the set then the program is
> not allowed to be loaded. To whatever extent possible we should also
> try to verify that program adhere's to the API as load and compile
> time. In the event that program violates the API it claims to be
> using, such as return invalid return code for the API, that is an
> error condition.
> 
> > If the user loads a XDP program using capabilities not avail in the
> > driver, then all his traffic will be hard dropped.
> >
> >
> > My proposal is to NOT allow XDP programs to be loaded if they want to use
> > return-codes/features not supported by the driver.  Thus, eBPF programs
> > register/annotate their needed capabilities upfront (I guess this could
> > also help HW offload engines).
> >
> Exactly. We just need to define exactly how this is done ;-)

yep. As we discussed at the summit there is a basic set of commands
drop/pass/tx that all drivers are expected to perform if they claim
xdp support and there will be driver/hw specific extensions.
We should be able to exercise all hw capabilites in a vendor neutral way.
It's a bit contradiction, obviously. We want xdp to be generic
and at the same time allow hw specific extensions.

> One open issue is whether XDP needs to be binary compatible across
> platforms or source code compatible (also require recompile in latter
> case for each platform). Personally I prefer source code compatible
> since that might allow platforms to implement the API specifically for
> their needs (like the ordering of fields in a meta data structure.

I agree that for some cases we have to give up on binary compatibility.
I think it will be similar to cpu world. Where applications
are compiled for different flavors of instruction set.
Like we might have encryption facility that is available
for sw path and offloaded into the most capable NICs.
I don't think annotations will solve that, since annotations
are hints when compiler/user space can be trusted which is
not the case here. It's more likely that we'll have driver
specific metadata that will hook into the verifier framework, so
we can make sure at load time that memory accesses and helper
calls are valid. That will make programs using hw specific
extensions to be loadable only on the given hw which I think
is inevitable, since we're operating at the lowest level,
right next to hw and any sw abstraction/generalization will
only slow things down.
skb is a generic facility where physical and virtual devices
look the same to programs. xdp is hw specific place.
If we try to generalize things too much at xdp level,
it will become skb-like. So beyond drop/pass/tx I can only
think of very few 'for all drivers' features.

Powered by blists - more mailing lists