| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Sat, 5 May 2018 02:25:47 +0200
From: Daniel Borkmann <daniel@...earbox.net>
To: davem@...emloft.net
Cc: daniel@...earbox.net, ast@...nel.org, netdev@...r.kernel.org
Subject: pull-request: bpf-next 2018-05-05
Hi David,
The following pull-request contains BPF updates for your *net-next* tree.
The main changes are:
1) Add initial infrastructure for AF_XDP sockets, which is optimized
for high performance packet processing. This early work only adds
copy-mode, and zero-copy semantics with driver changes will land in
subsequent patches. An AF_XDP socket has RX and/or TX queue associated
to it for receiving and sending packets. In contrast to AF_PACKET v2/3
descriptor queues are separated from packet buffers such that a RX or
TX descriptor points to a data buffer in a memory area called UMEM.
Latter can be shared so that packets don't need to be copied between
RX and TX. A XDP BPF program will steer the packets to one of the
AF_XDP sockets via a new BPF map called XSKMAP, from Björn and Magnus.
2) Add nfp BPF offload support for bpf_event_output() helper. Having
the driver reimplement and manage the perf array itself seems fragile
and unnecessary, therefore approach taken is that FW messages that
carry the events are pushed out to the RB. Additionally bpftool gets
support to connect to a perf event map and dump ring buffer contents,
useful for debugging purposes, from Jakub.
3) Add a new eBPF JIT for x86_32. Like in arm32 case, 64 bit div/mod
and xadd is still missing as well as BPF to BPF calls but other than
that it's functional and numbers show 30% to 50% improvement compared
to interpreter, from Wang.
4) Implement a new BPF helper bpf_get_stack() to overcome limitations
of stackmap and bpf_get_stackid() helper. bpf_get_stack() allows
to send stack traces directly to the BPF program which can perform
in-kernel processing and push them out via bpf_perf_event_output(),
from Yonghong.
5) Remove LD_ABS and LD_IND as native eBPF instructions and implement
them as rewrites. This significantly reduces complexity from JITs
while keeping similar performance characteristics, and allows to
better evolve JITs long term by having them all in C only, from Daniel.
6) Improve the code logic related to managing subprog information by
unifying main prog and subprogs, unifying entry points and stack
depth tracking into struct bpf_subprog_info, and adding end marker
into subprog_info array to simplify iteration logic, from Jiong.
7) Remove tracepoints from BPF core as they started to rot away,
causing panics triggered from syzkaller. Earlier ones from BPF
fs got already removed, so follow-up with rest since we also have
better introspection infrastructure these days, from Alexei.
8) Relax the bpf_current_task_under_cgroup() helper to allow usage in
interrupt which is particularly useful for BPF programs attached
to perf events, from Teng.
9) Formatting fixes in the new BPF uapi helper documentation for
bpf_perf_event_read() and bpf_get_stack() and relaxing whitespace
constraints in bpf_helpers_doc.py to ease documentation, from Quentin.
10) Dump the bpftool 'loaded at:' information in ISO 8601 format in
the plain variant and seconds since the Epoch in JSON to ease parsing,
also from Quentin.
11) Various cleanups mostly around coding and comment style, and several
capitalization, typo and grammar fixups in comments for the x64 BPF
JIT, from Ingo.
12) Fix up BPF context struct types in uapi BPF helper documentation
where some of them were mistakenly using kernel types, from Andrey.
13) Document that under CONFIG_BPF_JIT_ALWAYS_ON mode the bpf_jit_enable
mode 2 is not available, from Leo.
14) Import erspan uapi header file into tools infra so that BPF tunnel
helpers can use it and won't cause issues due to missing headers on
some systems, from William.
Please consider pulling these changes from:
git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
This has a minor merge conflict in tools/testing/selftests/bpf/test_progs.c.
Resolution is to take the hunk from bpf-next tree and change the first CHECK()
condition such that the missing '\n' is added to the end of the string, like:
if (CHECK(build_id_matches < 1, "build id match",
"Didn't find expected build ID from the map\n"))
goto disable_pmu;
Let me know if you run into any other unforeseen issue. Thanks a lot!
----------------------------------------------------------------
The following changes since commit 79741a38b4a2538a68342c45b813ecb9dd648ee8:
Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next (2018-04-26 21:19:50 -0400)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
for you to fetch changes up to e94fa1d93117e7f1eb783dc9cae6c70650944449:
bpf, xskmap: fix crash in xsk_map_alloc error path handling (2018-05-04 14:55:54 -0700)
----------------------------------------------------------------
Alexei Starovoitov (5):
Merge branch 'bpf_get_stack'
Merge branch 'fix-bpf-helpers-doc'
bpf: remove tracepoints from bpf core
Merge branch 'AF_XDP-initial-support'
Merge branch 'move-ld_abs-to-native-BPF'
Andrey Ignatov (2):
bpf: Fix helpers ctx struct types in uapi doc
bpf: Sync bpf.h to tools/
Björn Töpel (7):
net: initial AF_XDP skeleton
xsk: add user memory registration support sockopt
xsk: add Rx queue setup and mmap support
xsk: add Rx receive functions and poll support
bpf: introduce new bpf AF_XDP map type BPF_MAP_TYPE_XSKMAP
xsk: wire up XDP_DRV side of AF_XDP
xsk: wire up XDP_SKB side of AF_XDP
Daniel Borkmann (17):
Merge branch 'bpf-formatting-fixes-helpers'
bpf: prefix cbpf internal helpers with bpf_
bpf: migrate ebpf ld_abs/ld_ind tests to test_verifier
bpf: implement ld_abs/ld_ind in native bpf
bpf: add skb_load_bytes_relative helper
bpf, x64: remove ld_abs/ld_ind
bpf, arm64: remove ld_abs/ld_ind
bpf, sparc64: remove ld_abs/ld_ind
bpf, arm32: remove ld_abs/ld_ind
bpf, mips64: remove ld_abs/ld_ind
bpf, ppc64: remove ld_abs/ld_ind
bpf, s390x: remove ld_abs/ld_ind
bpf, x32: remove ld_abs/ld_ind
bpf: sync tools bpf.h uapi header
Merge branch 'bpf-subprog-mgmt-cleanup'
Merge branch 'bpf-event-output-offload'
bpf, xskmap: fix crash in xsk_map_alloc error path handling
Ingo Molnar (1):
x86/bpf: Clean up non-standard comments, to make the code more readable
Jakub Kicinski (10):
bpf: offload: allow offloaded programs to use perf event arrays
nfp: bpf: record offload neutral maps in the driver
bpf: export bpf_event_output()
bpf: replace map pointer loads before calling into offloads
nfp: bpf: perf event output helpers support
nfp: bpf: rewrite map pointers with NFP TIDs
tools: bpftool: fold hex keyword in command help
tools: bpftool: move get_possible_cpus() to common code
tools: bpftool: add simple perf event output reader
bpf: fix references to free_bpf_prog_info() in comments
Jiong Wang (3):
bpf: unify main prog and subprog
bpf: centre subprog information fields
bpf: add faked "ending" subprog
Leo Yan (1):
bpf, doc: Update bpf_jit_enable limitation for CONFIG_BPF_JIT_ALWAYS_ON
Magnus Karlsson (8):
xsk: add umem fill queue support and mmap
xsk: add support for bind for Rx
xsk: add umem completion queue support and mmap
xsk: add Tx queue setup and mmap support
dev: packet: make packet_direct_xmit a common function
xsk: support for Tx
xsk: statistics support
samples/bpf: sample application and documentation for AF_XDP sockets
Quentin Monnet (5):
bpf: fix formatting for bpf_perf_event_read() helper doc
bpf: fix formatting for bpf_get_stack() helper doc
bpf: update bpf.h uapi header for tools
tools: bpftool: change time format for program 'loaded at:' information
bpf: relax constraints on formatting for eBPF helper documentation
Teng Qin (1):
bpf: Allow bpf_current_task_under_cgroup in interrupt
Wang YanQing (1):
bpf, x86_32: add eBPF JIT compiler for ia32
William Tu (1):
tools, include: Grab a copy of linux/erspan.h
Yonghong Song (11):
bpf: change prototype for stack_map_get_build_id_offset
bpf: add bpf_get_stack helper
bpf/verifier: refine retval R0 state for bpf_get_stack helper
bpf: remove never-hit branches in verifier adjust_scalar_min_max_vals
bpf/verifier: improve register value range tracking with ARSH
tools/bpf: add bpf_get_stack helper to tools headers
samples/bpf: move common-purpose trace functions to selftests
tools/bpf: add a verifier test case for bpf_get_stack helper and ARSH
tools/bpf: add a test for bpf_get_stack with raw tracepoint prog
tools/bpf: add a test for bpf_get_stack with tracepoint prog
samples/bpf: fix kprobe attachment issue on x64
Documentation/networking/af_xdp.rst | 297 +++
Documentation/networking/filter.txt | 6 +
Documentation/networking/index.rst | 1 +
Documentation/sysctl/net.txt | 1 +
MAINTAINERS | 9 +-
arch/arm/net/bpf_jit_32.c | 77 -
arch/arm64/net/bpf_jit_comp.c | 65 -
arch/mips/net/ebpf_jit.c | 104 -
arch/powerpc/net/Makefile | 2 +-
arch/powerpc/net/bpf_jit64.h | 37 +-
arch/powerpc/net/bpf_jit_asm64.S | 180 --
arch/powerpc/net/bpf_jit_comp64.c | 109 +-
arch/s390/net/Makefile | 2 +-
arch/s390/net/bpf_jit.S | 116 -
arch/s390/net/bpf_jit.h | 20 +-
arch/s390/net/bpf_jit_comp.c | 127 +-
arch/sparc/net/Makefile | 5 +-
arch/sparc/net/bpf_jit_64.h | 29 -
arch/sparc/net/bpf_jit_asm_64.S | 162 --
arch/sparc/net/bpf_jit_comp_64.c | 79 +-
arch/x86/Kconfig | 2 +-
arch/x86/include/asm/nospec-branch.h | 30 +-
arch/x86/net/Makefile | 7 +-
arch/x86/net/bpf_jit.S | 154 --
arch/x86/net/bpf_jit_comp.c | 343 +--
arch/x86/net/bpf_jit_comp32.c | 2419 ++++++++++++++++++++
drivers/net/ethernet/netronome/nfp/bpf/cmsg.c | 16 +-
drivers/net/ethernet/netronome/nfp/bpf/fw.h | 20 +-
drivers/net/ethernet/netronome/nfp/bpf/jit.c | 76 +-
drivers/net/ethernet/netronome/nfp/bpf/main.c | 28 +-
drivers/net/ethernet/netronome/nfp/bpf/main.h | 24 +-
drivers/net/ethernet/netronome/nfp/bpf/offload.c | 172 +-
drivers/net/ethernet/netronome/nfp/bpf/verifier.c | 78 +-
drivers/net/ethernet/netronome/nfp/nfp_app.c | 2 +-
include/linux/bpf.h | 35 +-
include/linux/bpf_trace.h | 1 -
include/linux/bpf_types.h | 3 +
include/linux/bpf_verifier.h | 9 +-
include/linux/filter.h | 9 +-
include/linux/netdevice.h | 1 +
include/linux/socket.h | 5 +-
include/linux/tnum.h | 4 +-
include/net/xdp.h | 1 +
include/net/xdp_sock.h | 66 +
include/trace/events/bpf.h | 355 ---
include/uapi/linux/bpf.h | 94 +-
include/uapi/linux/if_xdp.h | 87 +
kernel/bpf/Makefile | 3 +
kernel/bpf/core.c | 108 +-
kernel/bpf/inode.c | 16 +-
kernel/bpf/offload.c | 6 +-
kernel/bpf/stackmap.c | 80 +-
kernel/bpf/syscall.c | 17 +-
kernel/bpf/tnum.c | 10 +
kernel/bpf/verifier.c | 247 +-
kernel/bpf/xskmap.c | 241 ++
kernel/trace/bpf_trace.c | 52 +-
lib/test_bpf.c | 570 +++--
net/Kconfig | 1 +
net/Makefile | 1 +
net/core/dev.c | 73 +-
net/core/filter.c | 345 ++-
net/core/sock.c | 12 +-
net/core/xdp.c | 15 +-
net/packet/af_packet.c | 42 +-
net/xdp/Kconfig | 7 +
net/xdp/Makefile | 2 +
net/xdp/xdp_umem.c | 260 +++
net/xdp/xdp_umem.h | 67 +
net/xdp/xdp_umem_props.h | 23 +
net/xdp/xsk.c | 656 ++++++
net/xdp/xsk_queue.c | 73 +
net/xdp/xsk_queue.h | 247 ++
samples/bpf/Makefile | 15 +-
samples/bpf/bpf_load.c | 97 +-
samples/bpf/bpf_load.h | 7 -
samples/bpf/offwaketime_user.c | 1 +
samples/bpf/sampleip_user.c | 1 +
samples/bpf/spintest_user.c | 1 +
samples/bpf/trace_event_user.c | 1 +
samples/bpf/trace_output_user.c | 110 +-
samples/bpf/xdpsock.h | 11 +
samples/bpf/xdpsock_kern.c | 56 +
samples/bpf/xdpsock_user.c | 948 ++++++++
scripts/bpf_helpers_doc.py | 14 +-
security/selinux/hooks.c | 4 +-
security/selinux/include/classmap.h | 4 +-
tools/bpf/bpftool/Documentation/bpftool-map.rst | 40 +-
tools/bpf/bpftool/Documentation/bpftool.rst | 2 +-
tools/bpf/bpftool/Makefile | 7 +-
tools/bpf/bpftool/bash-completion/bpftool | 36 +-
tools/bpf/bpftool/common.c | 77 +-
tools/bpf/bpftool/main.h | 7 +-
tools/bpf/bpftool/map.c | 80 +-
tools/bpf/bpftool/map_perf_ring.c | 347 +++
tools/bpf/bpftool/prog.c | 8 +-
tools/include/uapi/linux/bpf.h | 93 +-
tools/include/uapi/linux/erspan.h | 52 +
tools/testing/selftests/bpf/Makefile | 4 +-
tools/testing/selftests/bpf/bpf_helpers.h | 2 +
tools/testing/selftests/bpf/test_get_stack_rawtp.c | 102 +
tools/testing/selftests/bpf/test_progs.c | 242 +-
.../selftests/bpf/test_stacktrace_build_id.c | 20 +-
tools/testing/selftests/bpf/test_stacktrace_map.c | 19 +-
tools/testing/selftests/bpf/test_verifier.c | 311 ++-
tools/testing/selftests/bpf/trace_helpers.c | 180 ++
tools/testing/selftests/bpf/trace_helpers.h | 23 +
107 files changed, 8852 insertions(+), 2713 deletions(-)
create mode 100644 Documentation/networking/af_xdp.rst
delete mode 100644 arch/powerpc/net/bpf_jit_asm64.S
delete mode 100644 arch/s390/net/bpf_jit.S
delete mode 100644 arch/sparc/net/bpf_jit_asm_64.S
delete mode 100644 arch/x86/net/bpf_jit.S
create mode 100644 arch/x86/net/bpf_jit_comp32.c
create mode 100644 include/net/xdp_sock.h
delete mode 100644 include/trace/events/bpf.h
create mode 100644 include/uapi/linux/if_xdp.h
create mode 100644 kernel/bpf/xskmap.c
create mode 100644 net/xdp/Kconfig
create mode 100644 net/xdp/Makefile
create mode 100644 net/xdp/xdp_umem.c
create mode 100644 net/xdp/xdp_umem.h
create mode 100644 net/xdp/xdp_umem_props.h
create mode 100644 net/xdp/xsk.c
create mode 100644 net/xdp/xsk_queue.c
create mode 100644 net/xdp/xsk_queue.h
create mode 100644 samples/bpf/xdpsock.h
create mode 100644 samples/bpf/xdpsock_kern.c
create mode 100644 samples/bpf/xdpsock_user.c
create mode 100644 tools/bpf/bpftool/map_perf_ring.c
create mode 100644 tools/include/uapi/linux/erspan.h
create mode 100644 tools/testing/selftests/bpf/test_get_stack_rawtp.c
create mode 100644 tools/testing/selftests/bpf/trace_helpers.c
create mode 100644 tools/testing/selftests/bpf/trace_helpers.h
Powered by blists - more mailing lists