Date:   Thu, 5 Nov 2020 12:45:39 -0800
From:   Andrii Nakryiko <andrii.nakryiko@...il.com>
To:     David Ahern <dsahern@...il.com>
Cc:     Daniel Borkmann <daniel@...earbox.net>,
        Alexei Starovoitov <alexei.starovoitov@...il.com>,
        Hangbin Liu <haliu@...hat.com>,
        Stephen Hemminger <stephen@...workplumber.org>,
        Alexei Starovoitov <ast@...nel.org>,
        Martin KaFai Lau <kafai@...com>,
        Song Liu <songliubraving@...com>, Yonghong Song <yhs@...com>,
        David Miller <davem@...emloft.net>,
        Jesper Dangaard Brouer <brouer@...hat.com>,
        Networking <netdev@...r.kernel.org>, bpf <bpf@...r.kernel.org>,
        Jiri Benc <jbenc@...hat.com>,
        Andrii Nakryiko <andrii@...nel.org>,
        Toke Høiland-Jørgensen <toke@...hat.com>
Subject: Re: [PATCHv3 iproute2-next 0/5] iproute2: add libbpf support

On Wed, Nov 4, 2020 at 7:19 PM David Ahern <dsahern@...il.com> wrote:
>
> On 11/4/20 3:21 AM, Daniel Borkmann wrote:
> >
> >> Then libbpf release process can incorporate proper testing of libbpf
> >> and iproute2 combination.
> >> Or iproute2 should stay as-is with obsolete bpf support.
> >>
> >> Few years from now the situation could be different and shared libbpf
> >> would
> >> be the most appropriate choice. But that day is not today.
> >
> > Yep, for libbpf to be in same situation as libelf or libmnl basically
> > feature
> > development would have to pretty much come to a stop so that even minor
> > or exotic
> > distros get to a point where they ship same libbpf version as major
> > distros where
> > then users can start to rely on the base feature set for developing
> > programs
> > against it.
>
> User experience keeps getting brought up, but I also keep reading the
> stance that BPF users can not expect a consistent experience unless they
> are constantly chasing latest greatest versions of *ALL* S/W related to

That's not true. If you need new functionality like BTF, CO-RE,
function-by-function verification, etc., then yes, you have to update
kernel, compiler, libbpf, sometimes pahole. But if you have an BPF
application that doesn't use and need any of the newer features, it
will keep working just fine with the old kernel, old libbpf, and old
compiler.

Life is a bit more nuanced, of course. Sometimes a Clang update will
cause a shift in code generation patterns and you'd need either kernel
update (to get improved verifier logic) and/or libbpf update (to
compensate for either kernel or Clang change). Or update Clang again
to get a fixed version. That's life, bugs and problems are real.

If you care about using BTF-powered features, yes, you might need to
update pahole to get basic BTF, or get new BTF funcs needed for
fentry/fexit, or soon you'll need v1.19 if you want kernel module
BTFs. If you don't care about BTF, don't set CONFIG_DEBUG_INFO_BTF=y
and you won't even need pahole. For kernel module BTFs, you can't
request module BTF generation, unless you have a recent enough pahole.
I'm not sure how this can be handled better.

But if you have a plain old boring BPF program using
BPF_MAP_ARRAY/BPF_MAP_HASH, no global variables, you attach it to old
and stable BPF hooks like tracepoint, kprobe, etc., then it will work
with pretty much every version of libbpf, clang, and kernel. Don't
pass '-g' to Clang and BTF won't be generated at all, so you won't
even need BTF sanitization at all. And so on.

The problem is that users do want those new features, because those
allow to do new things or do existing things better/easier/faster. So
then we do ask to upgrade regularly to provide adequate support. But
it's like complaining that you need to update Java VM, compiler, Java
standard library, when you do want to use some new functionality.

> BPF. That is not a realistic expectation for users. Distributions exist
> for a reason. They solve real packaging problems.
>
> As libbpf and bpf in general reach a broader audience, the requirements
> to use, deploy and even tryout BPF features needs to be more user
> friendly and that starts with maintainers of the BPF code and how they
> approach extensions and features. Telling libbpf consumers to make
> libbpf a submodule of their project and update the reference point every
> time a new release comes out is not user friendly.

I have all the rights to ask for this, if I believe it's a better way
to go. Users have the right to refuse. But also iproute2 is not
exactly an end user in this situation, it is part of the BPF
ecosystem. So I think it's reasonable to have a healthy discussion
about the best way to facilitate BPF end-users.

>
> Similarly, it is not realistic or user friendly to *require* general
> Linux users to constantly chase latest versions of llvm, clang, dwarves,
> bcc, bpftool, libbpf, (I am sure I am missing more), and, by extension
> of what you want here, iproute2 just to upgrade their production kernel
> to say v5.10, the next LTS, or to see what relevant new ebpf features
> exists in the new kernel. As a specific example BTF extensions are added
> in a way that is all or nothing. Meaning, you want to compile kernel
> version X with CONFIG_DEBUG_INFO_BTF enabled, update your toolchain.
> Sure, you are using the latest LTS of $distro, and it worked fine with
> kernel version X-1 last week, but now compile fails completely unless
> the pahole version is updated. Horrible user experience. Again, just an
> example and one I brought up in July. I am sure there more.
>
> Linux APIs are about stability and consistency. Commands and libraries
> that work on v5.9 should work exactly the same on v5.10, 5.11, 5.12, ...
> *IF* I want a new feature (kernel, bpf or libbpf), then the requirement
> to upgrade is justified. But if I am just updating my kernel, or
> updating my compiler, or updating iproute2 because I want to try out
> some new nexthop feature, I should not be cornered into an all or
> nothing scheme.

Powered by blists - more mailing lists