Message-ID: <20210926203409.kn3gzz2eaodflels@ast-mbp.dhcp.thefacebook.com>
Date:   Sun, 26 Sep 2021 13:34:09 -0700
From:   Alexei Starovoitov <alexei.starovoitov@...il.com>
To:     bpf@...r.kernel.org
Subject: Happy birthday BPF!

Today BPF is 7 years old. On Sep 26, 2014
the commit 99c55f7d47c0 ("bpf: introduce BPF syscall and maps")
introduced the BPF system call. I’m very proud to say that BPF has grown from a
Linux curiosity to a cornerstone of the way many technologies are built, but
that hasn’t come without a fair amount of growing pain. 

We’ve been able to scale significantly by treating the first BPF
implementations of runtime and tool chain as the reference, but with Windows
and a number of new tool chains and libraries coming into the picture, we need
to reconsider this approach to ensure that we avoid fragmentation and lose
interoperability. Specific examples of potential divergence are all over the
place. Libbpf equivalent libraries have been reimplemented at least twice in
golang and rust. GCC and LLVM have BPF backends that are similar but not
equivalent. There are implementations of BPF in DPDK and other user space
frameworks. BPF can run in the Netronome NIC and was prototyped in HW for
inclusion into general purpose CPUs. There are several verifiers (in the Linux
Kernel, PREVAIL in user space, and experimental ExoBPF). There are many JITs
for different architectures inside the Linux Kernel and in user space. BPF
programs can be written in C, Rust, bpftrace, and various assembly languages.

This diversity is a sign of healthy and rapidly growing ecosystem, but it leads
to a confusing user experience. BPF implementations compete with each other.
Despite books about BPF and pretty complete documentation at
https://ebpf.io/what-is-ebpf, developers and users complain that the
documentation is spread around.

In response, we collectively created the BPF Foundation and BPF Steering
Committee (out of the most active BPF developers) to strengthen the
collaboration. Moving forward, we must all focus on maximizing growth while
maintaining interoperability. While we will maintain the Linux kernel, libbpf,
and LLVM as the reference implementations, it's not our goal to force every
user to embrace each part of the reference stack. Hopefully the committee will
add a bit of formal structure to coordinate collaboration.

More specifically, these implementations define the de-facto standard for
various parts of the BPF ecosystem:
- The Linux Kernel verifier defines BPF instruction set, BTF format,
  map and program types, helpers, hook points.
- The libbpf defines ELF file format and CO-RE features.
- The LLVM defines BPF C language.

7 year old BPF is mature enough to put the standards before the implementations.

For example:
The kernel verifier shouldn't be a gatekeeper of new instructions in BPF. GCC
added SDIV instruction. It's not implemented by LLVM and will be rejected by
the verifier, but such new instruction has all rights to be a part of BPF
instruction set standard even when some implementations don't support it. The
BPF steering committee (BSC) could make such a vote.
Can GCC continue inventing instructions without ever talking to BSC ? Sure, but
once BSC agrees to this extension of the standard (whether it's a new
instruction or new C language attribute((btf_tag))) it will give a technical
direction to the whole BPF ecosystem. The creation of BSC hopefully will
provide implementations a way to collaborate instead of competing for BPF
users.

If you are interested in joining this collaboration, there are many ways to
reach developers in the Linux BPF community:
- The bpf@...r mailing list was an excellent place for the collaboration
  and probably should continue to be such a place.
- When email is too slow the discussion can move to BPF office hours
  (zoom call every Thursday at 9am PST).
- There are Linux Plumbers and LSFMMBPF conferences to amplify the reach.
- BPF slack channel https://ebpf.io/slack.

Linux, Windows, Rust, Golang, GCC, LLVM folks,
cheers to BPF birthday !

Powered by blists - more mailing lists