| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Mon, 1 Jul 2024 02:30:09 +0000
From: duchangbin <changbin.du@...wei.com>
To: Adrian Hunter <adrian.hunter@...el.com>
CC: duchangbin <changbin.du@...wei.com>, Namhyung Kim <namhyung@...nel.org>,
Peter Zijlstra <peterz@...radead.org>, Ingo Molnar <mingo@...hat.com>,
Arnaldo Carvalho de Melo <acme@...nel.org>, Nathan Chancellor
<nathan@...nel.org>, Mark Rutland <mark.rutland@....com>, Alexander Shishkin
<alexander.shishkin@...ux.intel.com>, Jiri Olsa <jolsa@...nel.org>, "Ian
Rogers" <irogers@...gle.com>, "Liang, Kan" <kan.liang@...ux.intel.com>, "Nick
Desaulniers" <ndesaulniers@...gle.com>, Bill Wendling <morbo@...gle.com>,
Justin Stitt <justinstitt@...gle.com>, "linux-perf-users@...r.kernel.org"
<linux-perf-users@...r.kernel.org>, "linux-kernel@...r.kernel.org"
<linux-kernel@...r.kernel.org>, "llvm@...ts.linux.dev" <llvm@...ts.linux.dev>
Subject: Re: [PATCH v4 1/5] perf: support specify vdso path in cmdline
On Sat, Jun 29, 2024 at 09:47:10PM +0300, Adrian Hunter wrote:
> On 29/06/24 05:32, duchangbin wrote:
> > On Fri, Jun 28, 2024 at 08:27:55PM +0300, Adrian Hunter wrote:
> >> On 28/06/24 07:21, duchangbin wrote:
> >>> On Thu, Jun 27, 2024 at 04:53:18PM -0700, Namhyung Kim wrote:
> >>>> Hello guys,
> >>>>
> >>>> On Wed, Jun 26, 2024 at 01:32:42PM +0300, Adrian Hunter wrote:
> >>>>> On 26/06/24 05:26, duchangbin wrote:
> >>>>>> On Tue, Jun 25, 2024 at 04:20:49PM +0300, Adrian Hunter wrote:
> >>>>>>> On 25/06/24 06:37, Changbin Du wrote:
> >>>>>>>> The vdso dumped from process memory (in buildid-cache) lacks debugging
> >>>>>>>> info. To annotate vdso symbols with source lines we need specify a
> >>>>>>>> debugging version.
> >>>>>>>>
> >>>>>>>> For x86, we can find them from your local build as
> >>>>>>>> arch/x86/entry/vdso/vdso{32,64}.so.dbg. Or they may reside in
> >>>>>>>> /lib/modules/<version>/vdso/vdso{32,64}.so on Ubuntu. But notice that
> >>>>>>>> the buildid has to match.
> >>>>>>>>
> >>>>>>>> $ sudo perf record -a
> >>>>>>>> $ sudo perf report --objdump=llvm-objdump \
> >>>>>>>> --vdso arch/x86/entry/vdso/vdso64.so.dbg,arch/x86/entry/vdso/vdso32.so.dbg
> >>>>>>>>
> >>>>>>>> Samples: 17K of event 'cycles:P', 4000 Hz, Event count (approx.): 1760
> >>>>>>>> __vdso_clock_gettime /work/linux-host/arch/x86/entry/vdso/vdso64.so.d
> >>>>>>>> Percent│ movq -48(%rbp),%rsi
> >>>>>>>> │ testq %rax,%rax
> >>>>>>>> │ ; return vread_hvclock();
> >>>>>>>> │ movq %rax,%rdx
> >>>>>>>> │ ; if (unlikely(!vdso_cycles_ok(cycles)))
> >>>>>>>> │ ↑ js eb
> >>>>>>>> │ ↑ jmp 74
> >>>>>>>> │ ; ts->tv_sec = vdso_ts->sec;
> >>>>>>>> 0.02 │147: leaq 2(%rbx),%rax
> >>>>>>>> │ shlq $4, %rax
> >>>>>>>> │ addq %r10,%rax
> >>>>>>>> │ ; while ((seq = READ_ONCE(vd->seq)) & 1) {
> >>>>>>>> 9.38 │152: movl (%r10),%ecx
> >>>>>>>>
> >>>>>>>> When doing cross platform analysis, we also need specify the vdso path if
> >>>>>>>> we are interested in its symbols.
> >>>>>>>
> >>>>>>> Would it be possible to add vdso and vdso debug to the build-id
> >>>>>>> cache and ensure perf can find it there?
> >>>>>>>
> >>>>>>> Typically, getting dsos from another machine is handled via
> >>>>>>> build-id cache e.g. what perf-archive does
> >>>>>>>
> >>>>>> Hmm. I agree this is better alternative approach for cross-machine analysis.
> >>>>>> When collecting vdsos to buildid cache, I think we can use the local searched
> >>>>>> objects (with debug symbols) instead if its build-id matches vdsos from process
> >>>>>> dumping (the real code ran).
> >>>>>>
> >>>>>> Currently I just follow what perf does for vmlinux so to reuse most of existing
> >>>>>> code. Maybe vmlinux is too big to add to buildid-cahce?
> >>>>>>
> >>>>>> Can we keep our current strategy for now? I'll think about above options when
> >>>>>> I have more time.
> >>>>>>
> >>>>>
> >>>>> I tried adding vdso via perf buildid-cache. It doesn't work only
> >>>>> because the lookup expects the basename to be "vdso" but it is
> >>>>> "elf".
> >>>>>
> >>>>> Adding a link from "vdso" to "elf" made it work e.g.
> >>>>>
> >>>>> $ cat gettimeofday-test.c
> >>>>> #include <stdio.h>
> >>>>> #include <sys/time.h>
> >>>>>
> >>>>> int main()
> >>>>> {
> >>>>> struct timeval tv;
> >>>>> int ret;
> >>>>>
> >>>>> ret = gettimeofday(&tv, NULL);
> >>>>> if (ret == -1) {
> >>>>> fprintf(stderr, "gettimeofday failed\n");
> >>>>> return 1;
> >>>>> }
> >>>>>
> >>>>> printf("%lu.%lu\n", (unsigned long)tv.tv_sec, (unsigned long)tv.tv_usec);
> >>>>>
> >>>>> return 0;
> >>>>> $ perf record -e intel_pt//u ./gettimeofday-test
> >>>>> 1719397042.892837
> >>>>> [ perf record: Woken up 1 times to write data ]
> >>>>> [ perf record: Captured and wrote 0.026 MB perf.data ]
> >>>>> $ perf script --itrace=e
> >>>>> $ perf buildid-cache --remove /lib/modules/6.5.0-41-generic/vdso/vdso64.so
> >>>>> $ perf script --itrace=e
> >>>>> Warning:
> >>>>> 2 instruction trace errors
> >>>>> instruction trace error type 1 time 525345.386424204 cpu 4 pid 198976 tid 198976 ip 0x7ffddb0e8e00 code 5: Failed to get instruction
> >>>>> instruction trace error type 1 time 525345.386424829 cpu 4 pid 198976 tid 198976 ip 0x7ffddb0e884d code 5: Failed to get instruction
> >>>>> $ perf buildid-cache --add /lib/modules/6.5.0-41-generic/vdso/vdso64.so
> >>>>> $ perf script --itrace=e
> >>>>> Warning:
> >>>>> 2 instruction trace errors
> >>>>> instruction trace error type 1 time 525345.386424204 cpu 4 pid 198976 tid 198976 ip 0x7ffddb0e8e00 code 5: Failed to get instruction
> >>>>> instruction trace error type 1 time 525345.386424829 cpu 4 pid 198976 tid 198976 ip 0x7ffddb0e884d code 5: Failed to get instruction
> >>>>> $ cd ~/.debug/.build-id/c3/530aed66e71bfd10af66039f58cc7c4d2eaba8
> >>>>> ~/.debug/.build-id/c3/530aed66e71bfd10af66039f58cc7c4d2eaba8$ ln -s elf vdso
> >>>>> ~/.debug/.build-id/c3/530aed66e71bfd10af66039f58cc7c4d2eaba8$ ls -l
> >>>>> total 36
> >>>>> -rw-r--r-- 1 ahunter ahunter 33272 Jun 26 13:17 elf
> >>>>> -rw-r----- 1 ahunter ahunter 0 Jun 26 13:17 probes
> >>>>> lrwxrwxrwx 1 ahunter ahunter 3 Jun 26 13:18 vdso -> elf
> >>>>> /.debug/.build-id/c3/530aed66e71bfd10af66039f58cc7c4d2eaba8$ cd
> >>>>> $ perf script --itrace=e
> >>>>> $
> >>>>>
> >>>>> So maybe a change could be made to build_id_cache__add() to add
> >>>>> the extra link if the file name matches vdso
> >>>>
> >>>> Thanks for doing this! I noticed buildid_cache__basename() will handle
> >>>> the name properly once it realizes the file is a vdso. Maybe we can
> >>>> check the filepath with some patterns, or simply add an command line
> >>>> option to say it's a vdso.
> >>>>
> >>>> Thanks,
> >>>> Namhyung
> >>>>
> >>> I added some tricks for vdso in build_id_cache__add(). It replace vdso object
> >>> with debugging one if found. Is this okay?
> >>
> >> perf has support for storing debug files in the build-id
> >> cache using the base name "debug" instead of "elf", so really
> >> it would be better to make that work
> >>
> > My thoughts are:
> > 1. add debugging vdso file as "debug" in buildid cache.
> > 2. add a new perf config item 'annotate.prefer_debug_file' (default false) to
> > annotate with debugging object files.
>
> Is that needed? Isn't the debug information read from
> the "debug" file? If not, does that need fixing?
>
Is it by design as you mentioned before?
https://lore.kernel.org/r/all/395cfff7-9692-4123-96b6-353752007f46@intel.com/:
"In the past, there have been cases where the debugging version has not
worked for reading object code. I don't remember the details, but the
symbols and debugging information was OK while the object code was not.
In general, using anything other than the file that was actually executed
for reading object code seems like a bad idea."
I don't know whether this really matter mostly. I can fix it if it's a bug.
https://lore.kernel.org/r/all/395cfff7-9692-4123-96b6-353752007f46@intel.com/
>
--
Cheers,
Changbin Du
Powered by blists - more mailing lists