| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <ZFOSUab5XEJD0kxj@kernel.org>
Date: Thu, 4 May 2023 08:09:05 -0300
From: Arnaldo Carvalho de Melo <acme@...nel.org>
To: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Song Liu <song@...nel.org>, Andrii Nakryiko <andrii@...nel.org>,
Ingo Molnar <mingo@...nel.org>,
Thomas Gleixner <tglx@...utronix.de>,
Jiri Olsa <jolsa@...nel.org>,
Namhyung Kim <namhyung@...nel.org>,
Clark Williams <williams@...hat.com>,
Kate Carcia <kcarcia@...hat.com>, linux-kernel@...r.kernel.org,
linux-perf-users@...r.kernel.org,
Adrian Hunter <adrian.hunter@...el.com>,
Changbin Du <changbin.du@...wei.com>,
Hao Luo <haoluo@...gle.com>, Ian Rogers <irogers@...gle.com>,
James Clark <james.clark@....com>,
Kan Liang <kan.liang@...ux.intel.com>,
Roman Lozko <lozko.roma@...il.com>,
Stephane Eranian <eranian@...gle.com>,
Thomas Richter <tmricht@...ux.ibm.com>,
Arnaldo Carvalho de Melo <acme@...hat.com>
Subject: BPF skels in perf .Re: [GIT PULL] perf tools changes for v6.4
Em Wed, May 03, 2023 at 08:12:20PM -0700, Linus Torvalds escreveu:
> On Wed, May 3, 2023 at 8:00 PM Linus Torvalds <torvalds@...ux-foundation.org> wrote:
> > I did consider it, but the end result doesn't even build, so I unpulled again..
> > I get some libbpf error, and I'm just not interested in trying to
> > debug it. This has clearly not been tested well enough to be merged.
Its the default (opt-out) in the development branch for a while and
stayed in linux-next, but as it has been opt-in it hasn't received the
same amount of testing as the default build in the past development
cycles, even with the first feature that uses it having been introduced
back in 2020 :-\
> Side note: its' not even about testing.
> The error message makes it clear that this is garbage and should never
> be merged even if it were to compile.
> There is not a way in hell that it is correct that a 'perf' tool build
> should ever even look at the vmlinux binary to build.
> The fact that it does shows that something is seriously wrong in
> perf-tool land, and I will not be touching any pulls until that
> fundamental mistake is entirely gone.
> The vmlinux image that is present in my tree (ie
> /home/torvalds/v2.6/linux/vmlinux) is a test build with an insane
> config. And the fact that the perf tool even looks at it is seriously
> broken.
Humm,
Does building runqslower works for you in this same environment where
building perf failed?
I ask this because it uses the same libbpf technique (CO-RE) to allow
tools that access kernel data structures from BPF to work with multiple
kernels, even those where the layout of the accessed kernel structures
changed.
To build it:
$ make -C tools/bpf/runqslower/
make: Entering directory '/home/acme/git/perf-tools/tools/bpf/runqslower'
MKDIR /home/acme/git/perf-tools/tools/bpf/runqslower/.output/libbpf/
GEN /home/acme/git/perf-tools/tools/bpf/runqslower/.output/libbpf/bpf_helper_defs.h
CC /home/acme/git/perf-tools/tools/bpf/runqslower/.output/libbpf/staticobjs/libbpf.o
<SNIP>
LINK /home/acme/git/perf-tools/tools/bpf/runqslower/.output/libbpf/libbpf.a
INSTALL /home/acme/git/perf-tools/tools/bpf/runqslower/.output/libbpf//include/bpf/bpf.h
INSTALL /home/acme/git/perf-tools/tools/bpf/runqslower/.output/libbpf//include/bpf/libbpf.h
INSTALL /home/acme/git/perf-tools/tools/bpf/runqslower/.output/libbpf//include/bpf/btf.h
<SNIP>
INSTALL /home/acme/git/perf-tools/tools/bpf/runqslower/.output/libbpf//include/bpf/bpf_helper_defs.h
INSTALL libbpf_headers
MKDIR /home/acme/git/perf-tools/tools/bpf/runqslower/.output/bpftool/bootstrap/libbpf/include/bpf
INSTALL /home/acme/git/perf-tools/tools/bpf/runqslower/.output/bpftool/bootstrap/libbpf/include/bpf/hashmap.h
<SNIP>
LINK /home/acme/git/perf-tools/tools/bpf/runqslower/.output/bpftool/bootstrap/bpftool
GEN /home/acme/git/perf-tools/tools/bpf/runqslower/.output//vmlinux.h
GEN /home/acme/git/perf-tools/tools/bpf/runqslower/.output//runqslower.bpf.o
GEN /home/acme/git/perf-tools/tools/bpf/runqslower/.output//runqslower.skel.h
CC /home/acme/git/perf-tools/tools/bpf/runqslower/.output//runqslower.o
LINK /home/acme/git/perf-tools/tools/bpf/runqslower/.output//runqslower
make: Leaving directory '/home/acme/git/perf-tools/tools/bpf/runqslower'
$
# strip tools/bpf/runqslower/.output/runqslower
# file tools/bpf/runqslower/.output/runqslower
tools/bpf/runqslower/.output/runqslower: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=ab0306ee3f6cdc671d9aac7006457a3646e1c266, for GNU/Linux 3.2.0, stripped
[root@...co perf-tools]# tools/bpf/runqslower/.output/runqslower 100
Tracing run queue latency higher than 100 us
TIME COMM PID LAT(us)
08:00:17 swapper/4 30951 1001
08:00:18 kworker/7:8 2604 101
08:00:19 ksoftirqd/2 15665 105
08:00:22 swapper/2 904400 179
08:00:22 swapper/2 1247 102
08:00:23 TaskCon~ller #7 643789 102
08:00:26 ksoftirqd/2 849302 109
08:00:26 gmain 3107 238
08:00:26 rcu_tasks_trace 152972 634
08:00:27 systemd-oomd 899470 4887
08:00:28 Timer 30951 262
08:00:28 ksoftirqd/2 15665 103
08:00:29 systemd-resolve 895022 104
08:00:29 ksoftirqd/2 15665 145
08:00:30 ksoftirqd/2 15665 117
08:00:30 ksoftirqd/2 640315 149
08:00:30 gmain 3074 122
08:00:30 goa-identity-se 3107 109
^C
#
It builds and uses the tools/bpf/bpftool tool to generate the vmlinux.h
file to build the tool:
$ strace -f -e access,open,openat make -C tools/bpf/runqslower/ |& grep vmlinux
GEN /home/acme/git/perf-tools/tools/bpf/runqslower/.output//vmlinux.h
[pid 902901] openat(AT_FDCWD, "/home/acme/git/perf-tools/tools/bpf/runqslower/.output//vmlinux.h", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
[pid 902901] openat(AT_FDCWD, "/sys/kernel/btf/vmlinux", O_RDONLY) = 3
[pid 902909] openat(AT_FDCWD, "/home/acme/git/perf-tools/tools/bpf/runqslower/.output/vmlinux.h", O_RDONLY) = 4
$
But here it is using /sys/kernel/btf/vmlinux, which is way more sensible
than what you noticed.
Looking at tools/bpf/runqslower/Makefile:
# Try to detect best kernel BTF source
KERNEL_REL := $(shell uname -r)
VMLINUX_BTF_PATHS := $(if $(O),$(O)/vmlinux) \
$(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux) \
../../../vmlinux /sys/kernel/btf/vmlinux \
/boot/vmlinux-$(KERNEL_REL)
VMLINUX_BTF_PATH := $(or $(VMLINUX_BTF),$(firstword \
$(wildcard $(VMLINUX_BTF_PATHS))))
It tries to use vmlinux.
I'll check why it isn't using the same technique, possibly you don't
generate BTF?
In this fedora 37 kernel:
$ grep DEBUG_INFO_BTF /boot/config-6.2.9-200.fc37.x86_64
CONFIG_DEBUG_INFO_BTF=y
CONFIG_DEBUG_INFO_BTF_MODULES=y
$
Having said that probably we should go back to making build with BPF
skels a opt in feature, as it has been since the first feature using it
was introduced, in:
commit fbcdaa1908e8f61aa56c71a1db9a9deb72110a9d
Author: Song Liu <songliubraving@...com>
Date: Tue Dec 29 13:42:13 2020 -0800
perf build: Support build BPF skeletons with perf
BPF programs are useful in perf to profile BPF programs.
BPF skeleton is by far the easiest way to write BPF tools. Enable
building BPF skeletons in util/bpf_skel. A dummy bpf skeleton is added.
More bpf skeletons will be added for different use cases.
Signed-off-by: Song Liu <songliubraving@...com>
Tested-by: Arnaldo Carvalho de Melo <acme@...hat.com>
Acked-by: Jiri Olsa <jolsa@...hat.com>
Acked-by: Namhyung Kim <namhyung@...nel.org>
Cc: Alexander Shishkin <alexander.shishkin@...ux.intel.com>
Cc: Mark Rutland <mark.rutland@....com>
Cc: Peter Zijlstra <peterz@...radead.org>
Cc: kernel-team@...com
Link: http://lore.kernel.org/lkml/20201229214214.3413833-3-songliubraving@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@...hat.com>
Until we sort out these build robustness issues.
- Arnaldo
Powered by blists - more mailing lists