linux-kernel - Re: [PATCH v6 0/4] perf: add support for profiling jitted code

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAE40pded-oFw0YcnaxW-BjSQcRYiSQhTPgcTPUWZ=Srop+Os_A@mail.gmail.com>
Date:	Tue, 31 Mar 2015 00:33:10 -0700
From:	Brendan Gregg <brendan.d.gregg@...il.com>
To:	Stephane Eranian <eranian@...gle.com>
Cc:	LKML <linux-kernel@...r.kernel.org>, acme@...hat.com,
	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...e.hu>, Andi Kleen <ak@...ux.intel.com>,
	Jiri Olsa <jolsa@...hat.com>,
	Namhyung Kim <namhyung@...nel.org>,
	Rose Belcher <cel@...ibm.com>, sukadev@...ux.vnet.ibm.com,
	sonnyrao@...omium.org, johnmccutchan@...gle.com,
	David Ahern <dsahern@...il.com>, adrian.hunter@...el.com,
	Pawel Moll <pawel.moll@....com>
Subject: Re: [PATCH v6 0/4] perf: add support for profiling jitted code

G'Day Stephane,

On Mon, Mar 30, 2015 at 3:19 PM, Stephane Eranian <eranian@...gle.com> wrote:
[...]
> The current support only works when the runtime is monitored from
> start to finish: perf record java --agentpath:libpfmjvmti.so my_class.
>
> Once the run is completed, the jitdump file needs to be injected into
> the perf.data file. This is accomplished by using the perf inject command.
> This will also generate an ELF image for each jitted function. The
> inject MMAP records will point to those ELF images. The reasoning
> behind using ELF images is that it makes processing for perf report
> and annotate automatic and transparent. It also makes it easier to
> package and analyze on a remote machine.
[...]

This is really impressive work. Do we have an idea of the overhead for
running the java agent?

Today, I'm using perf-map-agent, loaded dynamically, to dump a
/tmp/perf*.map file as needed. My company has tens of thousands of
Linux instances running Java, but very few need profiling, and we
don't know which beforehand. So a snapshot-on-demand approach is
ideal. An always-on approach, well, we'd have to know the overhead (I
can build the agent and test...).

During a profile, our symbols can change, but it's not been a serious
problem. I can take a before and after snapshot of symbols
(/tmp/perf.map files), then highlight which have moved and are
therefore suspicious (not perfect, but ok). Yes, having a timestamped
symbol log would be most accurate, but I wouldn't pay a lot of
overhead for it. Is there a (horror) story behind why this was so
necessary?

Minor comment: I didn't notice an fsync() in the jvmti code. Can
symbol updates be buffered and not written out by the time perf is
reading them?

Again, this is impressive work. Just wanted to share comments from a
heavy user of perf with Java. You may already have other, better,
ideas for doing on-demand symbol dumps anyway.

Brendan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/