linux-kernel - Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <545829CA.7040900@voxpopuli.im>
Date:	Tue, 04 Nov 2014 02:20:10 +0100
From:	Alexandre Montplaisir <alexmonthy@...populi.im>
To:	Sebastian Andrzej Siewior <bigeasy@...utronix.de>,
	Jiri Olsa <jolsa@...hat.com>
CC:	linux-kernel@...r.kernel.org,
	Dominique Toupin <dominique.toupin@...csson.com>,
	Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
	Tom Zanussi <tzanussi@...il.com>,
	Jeremie Galarneau <jgalar@...icios.com>,
	David Ahern <dsahern@...il.com>,
	Arnaldo Carvalho de Melo <acme@...hat.com>
Subject: Re: FW: [RFC 0/5] perf tools: Add perf data CTF conversion

Hi Sebastian,

On 11/03/2014 06:58 PM, Sebastian Andrzej Siewior wrote:
> [...]
> I did on the linux side:
>
> |perf record \
> | -e sched:sched_switch \
> | -a
>
> This gave me perf.data trace. No with the new extension I converted it
> into CTF data stream (perf data convert -i perf.data --to-ctf ctf) and
> imported it into eclipse as a new trace. By setting 'domain = "kernel"'
> I managed to get it accepted as a kernel trace.
>
> Additionally I had to:
> - rename sched:sched_switch -> sched_switch (I dropped the other events)
> - rename "perf_cpu" to "cpu_id" and move it within "packet context"
>    (had a patch for that but we wanted to merge this later)
> - I added "prev_tid" with the content of "prev_pid" and I added
>    "next_tid" with the content of "next_pid". Now exclipse does not
>    "freeze" after loading the first entry
> - I prefixed every entry with _ (so "prev_tid" becomes "_prev_tid") and
>    now it seems to be recognized by the tracing plugin.
>
> puh. Now I have something in "cpu usage", "control flow" windows.

This is really great! Initially, I had believed that we would have 
needed to add a separate parser plugin, and to consider "perf traces" as 
a completely different beast from LTTng traces. However if you can get 
this close to they way LTTng presents its data, then we can probably 
re-use most of the existing code. In which case we could rename the 
"LTTng Kernel Trace" type in the UI to simply "Linux Kernel Trace". And 
that would cover both LTTng kernel traces and CTF-perf traces.

> The cpu_id field change will be addressed soon on our side.
> Now, the remaining things:
> The "domain = kernel" thingy (or another identifier if desired) is
> something we could add.

Unless the event data is exactly the same, it would be easier to use a 
different name. Like "kernel-perf" for instance?
 From the user's point of view, both would still be Linux Kernel Traces, 
but we could use the domain internally to determine which event/field 
layout to use.

Mathieu, any thoughts on how CTF domains should be namespaced?

> What do we do on the naming convention?
>
> The converter takes basically the event names the way they come from
> perf. That is why we have a "system" prefix.
> For the member fields, we take all the specific ones as perf serves
> them. The only exception is for the generic fields which get a "perf_"
> prefix in order not to clash with the specific ones.
> And there is this _ prefix in front of every data member.
>
> Now that I identified the differences between the CTF from lttng and
> perf, any suggestions / ideas how this could be solved?

I suppose it would be better/cleaner if the event and field names would 
remain the same, or at least be similar, in the perf.data and perf-CTF 
formats.

If the trace events from both LTTng and perf represent the same thing 
(and I assume they should, since they come from the same tracepoints, 
right?), then we could just add a wrapper on the viewer side to decide 
which event/field names to use, depending on the trace type.

Right now, we only define LTTng event and field names:
http://git.eclipse.org/c/tracecompass/org.eclipse.tracecompass.git/tree/org.eclipse.tracecompass.lttng2.kernel.core/src/org/eclipse/tracecompass/internal/lttng2/kernel/core/LttngStrings.java

But if you could for example tell me the perf equivalents of all the 
strings in that file, I could hack together such wrapper. With that, in 
theory, perf traces should behave exactly the same as LTTng traces in 
the viewer!

Cheers,
Alexandre

ps.  fyi, we have recently moved the trace viewer code to its own 
project, now called "Trace Compass". We will still support the Linux 
Tools plugins for the time being, but all new developments are done in 
Trace Compass. If you want to check it out: http://eclipse.org/tracecompass

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/