| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <Z3boDyuus0fiGcok@x1>
Date: Thu, 2 Jan 2025 16:25:03 -0300
From: Arnaldo Carvalho de Melo <acme@...nel.org>
To: Namhyung Kim <namhyung@...nel.org>
Cc: Christophe Leroy <christophe.leroy@...roup.eu>,
Adrian Hunter <adrian.hunter@...el.com>,
Ian Rogers <irogers@...gle.com>,
James Clark <james.clark@...aro.org>, Jiri Olsa <jolsa@...nel.org>,
Kan Liang <kan.liang@...ux.intel.com>,
Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
linux-perf-users@...r.kernel.org
Subject: Re: [BUG] perf top reports not being able to resolve kernel symbols
On Thu, Jan 02, 2025 at 03:41:34PM -0300, Arnaldo Carvalho de Melo wrote:
> While investigating a report by Christophe I stumbled on this:
>
> ⬢ [acme@...lbox perf-tools-next]$ git bisect bad
> 77b004f4c5c3c90b20ad61c5fa2ba7d494c1dba1 is the first bad commit
> commit 77b004f4c5c3c90b20ad61c5fa2ba7d494c1dba1 (HEAD)
> Author: Namhyung Kim <namhyung@...nel.org>
> Date: Thu Sep 12 15:42:08 2024 -0700
> perf symbol: Do not fixup end address of labels
>
> ⬢ [acme@...lbox perf-tools-next]$ git tag --contains 77b004f4c5c3c90b20ad61c5fa2ba7d494c1dba1 | grep ^v6
> v6.13-rc1
> v6.13-rc2
> v6.13-rc3
> v6.13-rc4
> v6.13-rc5
> ⬢ [acme@...lbox perf-tools-next]$
> ⬢ [acme@...lbox perf-tools-next]$ git merge perf-tools/perf-tools
> Already up to date.
> ⬢ [acme@...lbox perf-tools-next]$ git merge perf-tools/tmp.perf-tools
> Already up to date.
> ⬢ [acme@...lbox perf-tools-next]$
>
> Running just:
>
> # perf top --stdio
> PerfTop: 0 irqs/sec kernel: 0.0% exact: 0.0% lost: 0/0 drop: 0/0 [cpu_atom/cycles/P], (all, 28 CPUs)
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Warning:
> A vmlinux file was not found.
> Kernel samples will not be resolved.
> ^C^C^C^Z
> [1]+ Stopped perf top --stdio
> root@...ber:/home/acme/git/pahole#
>
> But kernel addresses are being used and there is a matching vmlinux:
>
> # pahole --running_kernel_vmlinux
> /lib/modules/6.13.0-rc2/build/vmlinux
> #
>
> root@...ber:/home/acme/git/pahole# file /lib/modules/6.13.0-rc2/build/vmlinux
> /lib/modules/6.13.0-rc2/build/vmlinux: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=d6f220b80bb50b35238aec50ed10f98df58849c0, with debug_info, not stripped
> root@...ber:/home/acme/git/pahole# perf buildid-list --kernel
> d6f220b80bb50b35238aec50ed10f98df58849c0
> root@...ber:/home/acme/git/pahole#
> root@...ber:/home/acme/git/pahole# perf buildid-list -h --kernel
>
> Usage: perf buildid-list [<options>]
>
> -k, --kernel Show current kernel build id
>
> root@...ber:/home/acme/git/pahole#
>
> I'm trying to figure this out now.
So the original logic seems borked, its a kernel sample that isn't
getting resolved, but that doesn't mean that other samples are not being
resolved, so we can't say a vmlinux wasn't found, as it _was_:
Thread 31 "perf" hit Breakpoint 2, perf_event__process_sample (tool=0x7fffffff9bd0, event=0x28ef890, evsel=0xf68860, sample=0x7fff867fb470, machine=0xf8e818) at builtin-top.c:813
813 if (symbol_conf.vmlinux_name) {
(gdb) p symbol_conf.vmlinux_name
$8 = 0x0
(gdb) p al
$9 = {thread = 0xfe8cd0, maps = 0xf8ee80, map = 0xf8f120, sym = 0x0, srcline = 0x0, addr = 23076781, level = 107 'k', filtered = 0 '\000', cpumode = 1 '\001', cpu = 0, socket = -1}
(gdb) p al.map
$10 = (struct map *) 0xf8f120
(gdb) p al.map->dso
$11 = (struct dso *) 0xf8ef30
(gdb) p al.map->dso->name
$12 = 0xf8f0bb "[kernel.kallsyms]"
(gdb) p __map__is_kernel(al.map)
(gdb) $13 = true
(gdb) p map__has_symbols(al.map)
$14 = true
So looking at that logic I think we should just do those warnings if
there are no symbols:
diff --git a/tools/perf/builtin-top.c b/tools/perf/builtin-top.c
index 724a7938632126bf..ca3e8eca6610e851 100644
--- a/tools/perf/builtin-top.c
+++ b/tools/perf/builtin-top.c
@@ -809,7 +809,7 @@ static void perf_event__process_sample(const struct perf_tool *tool,
* invalid --vmlinux ;-)
*/
if (!machine->kptr_restrict_warned && !top->vmlinux_warned &&
- __map__is_kernel(al.map) && map__has_symbols(al.map)) {
+ __map__is_kernel(al.map) && !map__has_symbols(al.map)) {
if (symbol_conf.vmlinux_name) {
char serr[256];
With this in place we get there when we specify an ELF file that is not
suitable as a vmlinux, be it a mismatching vmlinux or some other random
ELF file, when we then end up with the kernel map "loaded" (attempted to
load but found it invalid, so no symbols in that map):
PerfTop: 0 irqs/sec kernel: 0.0% exact: 0.0% lost: 0/0 drop: 0/0 [cpu_atom/cycles/P], (all, 28 CPUs)
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Warning:
The /bin/bash file can't be used: Mismatching build id
Kernel samples will not be resolved.
[2]+ Stopped perf top --stdio --vmlinux /bin/bash
root@...ber:~#
But why is that before your commig we seemingly would have entries in
the symtab that would cover addresses we now can't resolve?
One of these addresses is:
0.58% [kernel] [k] 0x00000000016001c1
Which...
root@...ber:~# pahole --running_kernel_vmlinux
/lib/modules/6.13.0-rc2/build/vmlinux
root@...ber:~#
root@...ber:~# readelf -sw /lib/modules/6.13.0-rc2/build/vmlinux | grep -B5 -A5 ' 0000000001600'
259227: ffffffff8156e290 262 FUNC GLOBAL DEFAULT 1 zs_free
259228: ffffffff8183a4d0 269 FUNC GLOBAL DEFAULT 1 security_inode_g[...]
259229: ffffffff81c8d900 191 FUNC GLOBAL DEFAULT 1 devres_find
259230: ffffffff812e11c0 16 FUNC GLOBAL DEFAULT 1 __pfx___probestu[...]
259231: ffffffff81c985a0 16 FUNC GLOBAL DEFAULT 1 __pfx_pm_qos_sys[...]
259232: 0000000001600000 0 NOTYPE GLOBAL DEFAULT ABS text_size
259233: ffffffff81487f10 117 FUNC GLOBAL DEFAULT 1 shmem_read_folio_gfp
259234: ffffffff81e08540 155 FUNC GLOBAL DEFAULT 1 __traceiter_smbu[...]
259235: ffffffff811e13a0 16 FUNC GLOBAL DEFAULT 1 __pfx_thaw_workqueues
259236: ffffffff81b04c70 599 FUNC GLOBAL DEFAULT 1 acpi_install_method
259237: ffffffff81de7d40 16 FUNC GLOBAL DEFAULT 1 __pfx_psmouse_se[...]
root@...ber:~#
There it is, that "text_size" symbol stayed with with a prev->end equal
to prev->start and thus 0x00000000016001c1 stops being resolved, which
leads us to get to that buggy warning.
I'll put all this into a patch and send it for review,
Thanks,
- Arnaldo
Powered by blists - more mailing lists