Message-ID: <CAEf4BzZFL6h7auopO+HEaarWb3RNAcAvsMn+aUcskBhamHwWyg@mail.gmail.com>
Date:   Tue, 24 Mar 2020 15:56:25 -0700
From:   Andrii Nakryiko <andrii.nakryiko@...il.com>
To:     David Ahern <dsahern@...il.com>
Cc:     Toke Høiland-Jørgensen <toke@...hat.com>,
        John Fastabend <john.fastabend@...il.com>,
        Jakub Kicinski <kuba@...nel.org>,
        Alexei Starovoitov <ast@...nel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        Martin KaFai Lau <kafai@...com>,
        Song Liu <songliubraving@...com>, Yonghong Song <yhs@...com>,
        Andrii Nakryiko <andriin@...com>,
        "David S. Miller" <davem@...emloft.net>,
        Jesper Dangaard Brouer <brouer@...hat.com>,
        Lorenz Bauer <lmb@...udflare.com>,
        Andrey Ignatov <rdna@...com>,
        Networking <netdev@...r.kernel.org>, bpf <bpf@...r.kernel.org>
Subject: Re: [PATCH bpf-next 1/4] xdp: Support specifying expected existing
 program when attaching XDP

On Tue, Mar 24, 2020 at 1:55 PM David Ahern <dsahern@...il.com> wrote:
>
> On 3/23/20 10:53 PM, Andrii Nakryiko wrote:
> > On Mon, Mar 23, 2020 at 6:01 PM David Ahern <dsahern@...il.com> wrote:
> >>
> >> On 3/23/20 1:23 PM, Toke Høiland-Jørgensen wrote:
> >>>>>> I agree here. And yes, I've been working on extending bpf_link into
> >>>>>> cgroup and then to XDP. We are still discussing some cgroup-specific
> >>>>>> details, but the patch is ready. I'm going to post it as an RFC to get
> >>>>>> the discussion started, before we do this for XDP.
> >>>>>
> >>>>> Well, my reason for being skeptic about bpf_link and proposing the
> >>>>> netlink-based API is actually exactly this, but in reverse: With
> >>>>> bpf_link we will be in the situation that everything related to a netdev
> >>>>> is configured over netlink *except* XDP.
> >>
> >> +1
> >
> > Hm... so using **libbpf**'s bpf_set_link_xdp_fd() API (notice "bpf" in
> > the name of the library and function, and notice no "netlink"), which
> > exposes absolutely nothing about netlink (it's just an internal
> > implementation detail and can easily change), is ok. But actually
> > switching to libbpf's bpf_link would be out of ordinary? Especially
> > considering that to use freplace programs (for libxdp and chaining)
> > with libbpf you will use bpf_program and bpf_link abstractions
> > anyways.
>
> It seems to me you are conflating libbpf api with the kernel uapi.

I'm not, as you can see in other email where I explicitly asked about
which ones we care in this discussion the most.

> Making libbpf user friendly certainly encourages standardization on its
> use, but there is no requirement that use of bpf means use of libbpf.

Agree, we can't force anyone to use libbpf. But it seems a pretty
popular choice in practice.

>
> >
> >>
> >>>>
> >>>> One can argue that everything related to use of BPF is going to be
> >>>> uniform and done through BPF syscall? Given variety of possible BPF
> >>>> hooks/targets, using custom ways to attach for all those many cases is
> >>>> really bad as well, so having a unifying concept and single entry to
> >>>> do this is good, no?
> >>>
> >>> Well, it depends on how you view the BPF subsystem's relation to the
> >>> rest of the kernel, I suppose. I tend to view it as a subsystem that
> >>> provides a bunch of functionality, which you can setup (using "internal"
> >>> BPF APIs), and then attach that object to a different subsystem
> >>> (networking) using that subsystem's configuration APIs.
> >>>
> >>
> >> again, +1.
> >>
> >> bpf syscall is used for program related manipulations like load and
> >
> > bpf syscall is used for way more than that, actually...
> >
> >> unload. Attaching that program to an object has a type unique solution -
> >> e.g., netlink for XDP and ioctl for perf_events.
> >
> > That's not true and hasn't been true for at least a while now. cgroup
> > programs, flow_dissector, lirc_mode2 (whatever that is, I have no
> > idea) are attached with BPF_PROG_ATTACH. raw_tracepoint and all the
> > fentry/fexit/fmod_ret/freplace attachments are done also through bpf
> > syscall. For perf_event related stuff it's done through ioctls right
> > now, but with bpf_link unification I wouldn't be surprised if it will
>
> and it always will be able to. Kernel uapi will not be revoked because a
> new way to do something comes along.

Good that we are in agreement that BPF attachment is not really a type
unique solution.

Also, I didn't say any of the existing APIs will go away. But to
support pinnable/queryable bpf_link, we'll need a new API for
perf_event. And I believe it should be done through bpf syscall, not
through more ioctls. Which is what we are discussing here w.r.t. XDP
as well. Existing way of attaching BPF program directly (with no
bpf_link created, no way to pin and query that bpf_link, etc) won't go
away anywhere, of course. But there is no need to duplicate
bpf_link-related APIs in netlink, if we are going to do it as part of
bpf syscall.

>
> > be done through the same LINK_CREATE command soon, as is done for
> > cgroup and *other* tracing bpf_links. Because consistent API and
> > semantics is good, rather than having to do it N different ways for N
> > different subsystems.
> >
>
> That's a bpf / libbpf centric perspective. What Toke and I are saying is
> the networking centric perspective matters to and networking uses
> netlink for configuration.

It's BPF-centric because BPF is much wider than networking which
allows to keep things in perspective beyond networking world. It's
about cost of maintaining UAPIs and consistency across whole range of
BPF program types. I don't see a good reason to maintain duplicate
APIs. We are going to have bpf_link API through bpf syscall (because
cgroups, tracing, etc) and it is going to be generic. So what's the
upside to duplicating it in netlink as well?

Can one work with XDP without bpf syscall? No, one cannot. So we are
not adding a new "syscall dependency" or anything like that.

On the other hand, as a developer, can I develop XDP application
without using netlink API at all? Funnily enough, I could if BPF
syscall allowed attaching to ifindex, couldn't I? If I develop some
monitoring application using XDP and not intending to configure any
network interface, just attach my BPF program and let it run for a
bit. Why would I bother with implementing entire netlink protocol just
to attach BPF program? But I also don't subscribe to a notion of
"attaching BPF program is configuration", so...

Powered by blists - more mailing lists