Message-ID: <20200324191554.46a7e0c2@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
Date:   Tue, 24 Mar 2020 19:15:54 -0700
From:   Jakub Kicinski <kuba@...nel.org>
To:     Alexei Starovoitov <alexei.starovoitov@...il.com>
Cc:     John Fastabend <john.fastabend@...il.com>,
        Toke Høiland-Jørgensen 
        <toke@...hat.com>, Andrii Nakryiko <andrii.nakryiko@...il.com>,
        Alexei Starovoitov <ast@...nel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        Martin KaFai Lau <kafai@...com>,
        Song Liu <songliubraving@...com>, Yonghong Song <yhs@...com>,
        Andrii Nakryiko <andriin@...com>,
        "David S. Miller" <davem@...emloft.net>,
        Jesper Dangaard Brouer <brouer@...hat.com>,
        Lorenz Bauer <lmb@...udflare.com>,
        Andrey Ignatov <rdna@...com>,
        Networking <netdev@...r.kernel.org>, bpf <bpf@...r.kernel.org>
Subject: Re: [PATCH bpf-next 1/4] xdp: Support specifying expected existing
 program when attaching XDP

On Tue, 24 Mar 2020 18:36:31 -0700 Alexei Starovoitov wrote:
> On Tue, Mar 24, 2020 at 12:22:47PM -0700, John Fastabend wrote:
> > > Well, I wasn't talking about any of those subsystems, I was talking
> > > about networking :)  
> > 
> > My experience has been that networking in the strict sense of XDP no
> > longer exists on its own without cgroups, flow dissector, sockops,
> > sockmap, tracing, etc. All of these pieces are built, patched, loaded,
> > pinned and otherwise managed and manipulated as BPF objects via libbpf.
> > 
> > Because I have all this infra in place for other items its a bit odd
> > imo to drop out of BPF apis to then swap a program differently in the
> > XDP case from how I would swap a program in any other place. I'm
> > assuming ability to swap links will be enabled at some point.
> > 
> > Granted it just means I have some extra functions on the side to manage
> > the swap similar to how 'qdisc' would be handled today but still not as
> > nice an experience in my case as if it was handled natively.
> > 
> > Anyways the netlink API is going to have to call into the BPF infra
> > on the kernel side for verification, etc so its already not pure
> > networking.
> >   
> > > 
> > > In particular, networking already has a consistent and fairly
> > > well-designed configuration mechanism (i.e., netlink) that we are
> > > generally trying to move more functionality *towards* not *away from*
> > > (see, e.g., converting ethtool to use netlink).  
> > 
> > True. But BPF programs are going to exist and interop with other
> > programs not exactly in the networking space. Actually library calls
> > might be used in tracing, cgroups, and XDP side. It gets a bit more
> > interesting if the "same" object file (with some patching) runs in both
> > XDP and sockops land for example.  
> 
> Thanks John for summarizing it very well.
> It looks to me that netlink proponents fail to realize that "bpf for
> networking" goes way beyond what netlink is doing and capable of doing in the
> future. BPF_*_INET_* progs do core networking without any smell of netlink
> anywhere. "But, but, but, netlink is the way to configure networking"... is
> simply not true. Even in years before BPF sockets and syscalls were the way to
> do it. netlink has plenty of awesome properties, but arguing that it's the
> only true way to do networking is not matching the reality.

It is the way to configure XDP today, so it's only natural to
scrutinize the attempts to replace it. 

Also I personally don't think you'd see this much push back trying to
add bpf_link-based stuff to cls_bpf, that's an add-on. XDP is
integrated very fundamentally with the networking stack at this point.

> Details are important and every case is different. So imo:
> converting ethtool to netlink - great stuff.
> converting netdev irq/queue management to netlink - great stuff too.
> adding more netlink api for xdp - really bad idea.

Why is it a bad idea?

There are plenty things which will only be available over netlink.
Configuring the interface so installing the XDP program is possible
(disabling features, configuring queues etc.). Chances are user gets
the ifindex of the interface to attach to over netlink in the first
place. The queue configuration (which you agree belongs in netlink)
will definitely get more complex to allow REDIRECTs to work more
smoothly. AF_XDP needs all sort of netlink stuff.

Netlink gives us the notification mechanism which is how we solve
coordination across daemons (something that BPF subsystem is only 
now trying to solve).

BPF subsystem has a proven track record of reimplementing things devs
don't like or haven't studied (bpftool net, netlink library). So it is
a real concern to allow duplicating parts of the kernel netlink API.

Powered by blists - more mailing lists