linux-kernel - Re: [PATCH v4 1/5] perf: support specify vdso path in cmdline

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <05f95eb8-9b4c-4327-a97f-a15654278c41@intel.com>
Date: Wed, 26 Jun 2024 13:32:42 +0300
From: Adrian Hunter <adrian.hunter@...el.com>
To: duchangbin <changbin.du@...wei.com>
Cc: Peter Zijlstra <peterz@...radead.org>, Ingo Molnar <mingo@...hat.com>,
 Arnaldo Carvalho de Melo <acme@...nel.org>,
 Namhyung Kim <namhyung@...nel.org>, Nathan Chancellor <nathan@...nel.org>,
 Mark Rutland <mark.rutland@....com>,
 Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
 Jiri Olsa <jolsa@...nel.org>, Ian Rogers <irogers@...gle.com>,
 "Liang, Kan" <kan.liang@...ux.intel.com>,
 Nick Desaulniers <ndesaulniers@...gle.com>, Bill Wendling
 <morbo@...gle.com>, Justin Stitt <justinstitt@...gle.com>,
 "linux-perf-users@...r.kernel.org" <linux-perf-users@...r.kernel.org>,
 "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
 "llvm@...ts.linux.dev" <llvm@...ts.linux.dev>
Subject: Re: [PATCH v4 1/5] perf: support specify vdso path in cmdline

On 26/06/24 05:26, duchangbin wrote:
> On Tue, Jun 25, 2024 at 04:20:49PM +0300, Adrian Hunter wrote:
>> On 25/06/24 06:37, Changbin Du wrote:
>>> The vdso dumped from process memory (in buildid-cache) lacks debugging
>>> info. To annotate vdso symbols with source lines we need specify a
>>> debugging version.
>>>
>>> For x86, we can find them from your local build as
>>> arch/x86/entry/vdso/vdso{32,64}.so.dbg. Or they may reside in
>>> /lib/modules/<version>/vdso/vdso{32,64}.so on Ubuntu. But notice that
>>> the buildid has to match.
>>>
>>> $ sudo perf record -a
>>> $ sudo perf report --objdump=llvm-objdump \
>>>   --vdso arch/x86/entry/vdso/vdso64.so.dbg,arch/x86/entry/vdso/vdso32.so.dbg
>>>
>>> Samples: 17K of event 'cycles:P', 4000 Hz, Event count (approx.): 1760
>>> __vdso_clock_gettime  /work/linux-host/arch/x86/entry/vdso/vdso64.so.d
>>> Percent│       movq    -48(%rbp),%rsi
>>>        │       testq   %rax,%rax
>>>        │     ;               return vread_hvclock();
>>>        │       movq    %rax,%rdx
>>>        │     ;               if (unlikely(!vdso_cycles_ok(cycles)))
>>>        │     ↑ js      eb
>>>        │     ↑ jmp     74
>>>        │     ;               ts->tv_sec = vdso_ts->sec;
>>>   0.02 │147:   leaq    2(%rbx),%rax
>>>        │       shlq    $4, %rax
>>>        │       addq    %r10,%rax
>>>        │     ;               while ((seq = READ_ONCE(vd->seq)) & 1) {
>>>   9.38 │152:   movl    (%r10),%ecx
>>>
>>> When doing cross platform analysis, we also need specify the vdso path if
>>> we are interested in its symbols.
>>
>> Would it be possible to add vdso and vdso debug to the build-id
>> cache and ensure perf can find it there?
>>
>> Typically, getting dsos from another machine is handled via
>> build-id cache e.g. what perf-archive does
>>
> Hmm. I agree this is better alternative approach for cross-machine analysis.
> When collecting vdsos to buildid cache, I think we can use the local searched
> objects (with debug symbols) instead if its build-id matches vdsos from process
> dumping (the real code ran).
> 
> Currently I just follow what perf does for vmlinux so to reuse most of existing
> code. Maybe vmlinux is too big to add to buildid-cahce?
> 
> Can we keep our current strategy for now? I'll think about above options when
> I have more time.
> 

I tried adding vdso via perf buildid-cache.  It doesn't work only
because the lookup expects the basename to be "vdso" but it is
"elf".

Adding a link from "vdso" to "elf" made it work e.g.

$ cat gettimeofday-test.c
#include <stdio.h>
#include <sys/time.h>

int main()
{
        struct timeval tv;
        int ret;

        ret = gettimeofday(&tv, NULL);
        if (ret == -1) {
                fprintf(stderr, "gettimeofday failed\n");
                return 1;
        }

        printf("%lu.%lu\n", (unsigned long)tv.tv_sec, (unsigned long)tv.tv_usec);

        return 0;
$ perf record -e intel_pt//u ./gettimeofday-test
1719397042.892837
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.026 MB perf.data ]
$ perf script --itrace=e
$ perf buildid-cache --remove /lib/modules/6.5.0-41-generic/vdso/vdso64.so
$ perf script --itrace=e
Warning:
2 instruction trace errors
 instruction trace error type 1 time 525345.386424204 cpu 4 pid 198976 tid 198976 ip 0x7ffddb0e8e00 code 5: Failed to get instruction
 instruction trace error type 1 time 525345.386424829 cpu 4 pid 198976 tid 198976 ip 0x7ffddb0e884d code 5: Failed to get instruction
$ perf buildid-cache --add /lib/modules/6.5.0-41-generic/vdso/vdso64.so
$ perf script --itrace=e
Warning:
2 instruction trace errors
 instruction trace error type 1 time 525345.386424204 cpu 4 pid 198976 tid 198976 ip 0x7ffddb0e8e00 code 5: Failed to get instruction
 instruction trace error type 1 time 525345.386424829 cpu 4 pid 198976 tid 198976 ip 0x7ffddb0e884d code 5: Failed to get instruction
$ cd ~/.debug/.build-id/c3/530aed66e71bfd10af66039f58cc7c4d2eaba8
~/.debug/.build-id/c3/530aed66e71bfd10af66039f58cc7c4d2eaba8$ ln -s elf vdso
~/.debug/.build-id/c3/530aed66e71bfd10af66039f58cc7c4d2eaba8$ ls -l
total 36
-rw-r--r-- 1 ahunter ahunter 33272 Jun 26 13:17 elf
-rw-r----- 1 ahunter ahunter     0 Jun 26 13:17 probes
lrwxrwxrwx 1 ahunter ahunter     3 Jun 26 13:18 vdso -> elf
/.debug/.build-id/c3/530aed66e71bfd10af66039f58cc7c4d2eaba8$ cd
$ perf script --itrace=e
$ 

So maybe a change could be made to build_id_cache__add() to add
the extra link if the file name matches vdso