[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20250224140151.667679-1-jolsa@kernel.org>
Date: Mon, 24 Feb 2025 15:01:32 +0100
From: Jiri Olsa <jolsa@...nel.org>
To: Oleg Nesterov <oleg@...hat.com>,
Peter Zijlstra <peterz@...radead.org>,
Andrii Nakryiko <andrii@...nel.org>
Cc: bpf@...r.kernel.org,
linux-kernel@...r.kernel.org,
linux-trace-kernel@...r.kernel.org,
x86@...nel.org,
Song Liu <songliubraving@...com>,
Yonghong Song <yhs@...com>,
John Fastabend <john.fastabend@...il.com>,
Hao Luo <haoluo@...gle.com>,
Steven Rostedt <rostedt@...dmis.org>,
Masami Hiramatsu <mhiramat@...nel.org>,
Alan Maguire <alan.maguire@...cle.com>,
David Laight <David.Laight@...LAB.COM>,
Thomas Weißschuh <thomas@...ch.de>
Subject: [PATCH RFCv2 00/18] uprobes: Add support to optimize usdt probes on x86_64
hi,
this patchset adds support to optimize usdt probes on top of 5-byte
nop instruction.
The generic approach (optimize all uprobes) is hard due to emulating
possible multiple original instructions and its related issues. The
usdt case, which stores 5-byte nop seems much easier, so starting
with that.
The basic idea is to replace breakpoint exception with syscall which
is faster on x86_64. For more details please see changelog of patch 8.
The run_bench_uprobes.sh benchmark triggers uprobe (on top of different
original instructions) in a loop and counts how many of those happened
per second (the unit below is million loops).
There's big speed up if you consider current usdt implementation
(uprobe-nop) compared to proposed usdt (uprobe-nop5):
# ./benchs/run_bench_uprobes.sh
usermode-count : 818.386 ± 1.886M/s
syscall-count : 8.923 ± 0.003M/s
--> uprobe-nop : 3.086 ± 0.005M/s
uprobe-push : 2.751 ± 0.001M/s
uprobe-ret : 1.481 ± 0.000M/s
--> uprobe-nop5 : 4.016 ± 0.002M/s
uretprobe-nop : 1.712 ± 0.008M/s
uretprobe-push : 1.616 ± 0.001M/s
uretprobe-ret : 1.052 ± 0.000M/s
uretprobe-nop5 : 2.015 ± 0.000M/s
rfc v2 changes:
- make uretprobe work properly with optimized uprobe
- make the uprobe optimized code x86_64 specific [Peter]
- rework the verify function logic, using it now as callback
- fix find_nearest_page to include [PAGE_SIZE, ... ] area [Andrii]
- try lockless vma lookup in in_uprobe_trampoline [Peter]
- do per partes instructions update using int3 like in text_poke_bp_batch [David]
- map uprobe trampoline via single global page [Thomas]
- keep track of uprobes per mm_struct
pending todo (follow ups):
- use PROCMAP_QUERY in tests
- alloc 'struct uprobes_state' for mm_struct only when needed [Andrii]
- seccomp change for new uprobe syscall (same as for uretprobe)
thanks,
jirka
---
Jiri Olsa (18):
uprobes: Rename arch_uretprobe_trampoline function
uprobes: Make copy_from_page global
uprobes: Move ref_ctr_offset update out of uprobe_write_opcode
uprobes: Add uprobe_write function
uprobes: Add nbytes argument to uprobe_write_opcode
uprobes: Add orig argument to uprobe_write and uprobe_write_opcode
uprobes: Add swbp argument to arch_uretprobe_hijack_return_addr
uprobes/x86: Add uprobe syscall to speed up uprobe
uprobes/x86: Add mapping for optimized uprobe trampolines
uprobes/x86: Add mm_uprobe objects to track uprobes within mm
uprobes/x86: Add support to emulate nop5 instruction
uprobes/x86: Add support to optimize uprobes
selftests/bpf: Reorg the uprobe_syscall test function
selftests/bpf: Use 5-byte nop for x86 usdt probes
selftests/bpf: Add uprobe/usdt syscall tests
selftests/bpf: Add hit/attach/detach race optimized uprobe test
selftests/bpf: Add uprobe syscall sigill signal test
selftests/bpf: Add 5-byte nop uprobe trigger bench
arch/arm/probes/uprobes/core.c | 4 +-
arch/arm64/kernel/probes/uprobes.c | 2 +-
arch/csky/kernel/probes/uprobes.c | 2 +-
arch/loongarch/kernel/uprobes.c | 2 +-
arch/mips/kernel/uprobes.c | 2 +-
arch/powerpc/kernel/uprobes.c | 2 +-
arch/riscv/kernel/probes/uprobes.c | 2 +-
arch/s390/kernel/uprobes.c | 2 +-
arch/sparc/kernel/uprobes.c | 2 +-
arch/x86/entry/syscalls/syscall_64.tbl | 1 +
arch/x86/include/asm/uprobes.h | 6 ++
arch/x86/kernel/uprobes.c | 530 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
include/linux/syscalls.h | 2 +
include/linux/uprobes.h | 23 +++-
kernel/events/uprobes.c | 147 +++++++++++++++++--------
kernel/fork.c | 1 +
kernel/sys_ni.c | 1 +
tools/testing/selftests/bpf/bench.c | 12 +++
tools/testing/selftests/bpf/benchs/bench_trigger.c | 42 ++++++++
tools/testing/selftests/bpf/benchs/run_bench_uprobes.sh | 2 +-
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c | 342 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
tools/testing/selftests/bpf/progs/uprobe_syscall_executed.c | 34 +++++-
tools/testing/selftests/bpf/sdt.h | 9 +-
23 files changed, 1093 insertions(+), 79 deletions(-)
Powered by blists - more mailing lists