linux-kernel - Re: [RFC] perf record: missing buildid for callstack modules

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160108181942.GB20576@kernel.org>
Date:	Fri, 8 Jan 2016 15:19:42 -0300
From:	Arnaldo Carvalho de Melo <acme@...nel.org>
To:	Stephane Eranian <eranian@...gle.com>
Cc:	Namhyung Kim <namhyung@...il.com>,
	LKML <linux-kernel@...r.kernel.org>,
	Jiri Olsa <jolsa@...nel.org>,
	Namhyung Kim <namhyung@...nel.org>,
	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...nel.org>,
	Adrian Hunter <adrian.hunter@...el.com>,
	"ak@...ux.intel.com" <ak@...ux.intel.com>
Subject: Re: [RFC] perf record: missing buildid for callstack modules

Em Fri, Jan 08, 2016 at 10:01:24AM -0800, Stephane Eranian escreveu:
> On Thu, Jan 7, 2016 at 3:47 PM, Arnaldo Carvalho de Melo
> <acme@...nel.org> wrote:
> > Em Fri, Jan 08, 2016 at 07:47:03AM +0900, Namhyung Kim escreveu:
> >> On January 8, 2016 7:00:35 AM GMT+09:00, Stephane Eranian <eranian@...gle.com> wrote:
> >> >>> How about providing an option to perf record to force collecting
> >> >>> buildid for all IPs
> >> >>> captured in the callstack? I understand that would cost more at the
> >> >end of the
> >> >>> collection, but this would be beneficial to several monitoring
> >> >scenarios.

> >> >> I agree, would consider applying a patch that provides the option but
> >> >> does not do this by default.

> >> >I agree, not the default.

> >> Please see

> >> https://lkml.org/lkml/2015/3/22/249

> > Oops, Stephane, please try this, so that we can finally merge it :-\

> I will try it today. However, I am a bit worried about the performance
> impact. Unless I am missing something in this approach we may end up
> looking up N times the same module if it appears in N callstacks. In
> Andi's suggested approach, there would be only one pass at the beginning
> (or the end of the run). But you could miss some modules if they are gone
> by the time you run the pass.

For kernel modules, yeah, since we'll have to synthesize them at session
start, we could as well save the buildids at that point, but then this,
as well as the saving of buildids for normal DSOs is racy, since we
collect it just at the end of the session.

We already discussed how to solve it, and it involves extending once
more PERF_RECORD_MMAP, so that, when we load a DSO we stash its build-id
in a per-DSO data structure in the kernel, then, when generating
PERF_RECORD_MMAP3 we put the buildid there, this way if any of those
binaries gets replaced while we're recording samples, we would notice,
i.e. we wouldn't care that much about the pathname, looking everything
by the content based buildid instead.

And also for modules, what if a module is loaded during the session?
We'll miss, or gets replaced by a newer version? We'll miss as well.

We need to hook into module loading/unloading and generate
PERF_RECORD_MMAP3 records there as well.

Best thing we can do right now (by adding extra events to the 'perf
record' command line) is to hook into places to detect these issues and
at least warn the user that some of the DSOs with hits got updated
during the session.

Namhyungs approach is no better than what we do now, i.e. it doesn't
detects such flaps, but will be just fine if no updates take place,
guaranteeing that later, at analysis time, we pick the right
ELF/kallsyms files, keyed by their build ids.

- Arnaldo