| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <1437150840-31811-1-git-send-email-adrian.hunter@intel.com>
Date: Fri, 17 Jul 2015 19:33:35 +0300
From: Adrian Hunter <adrian.hunter@...el.com>
To: Arnaldo Carvalho de Melo <acme@...nel.org>
Cc: Ingo Molnar <mingo@...nel.org>, linux-kernel@...r.kernel.org,
Jiri Olsa <jolsa@...hat.com>
Subject: [PATCH V8 00/25] perf tools: Introduce an abstraction for AUX Area and Instruction Tracing
Hi
Here is V8 patches for the introduction of an abstraction for
using the AUX area and Instruction tracing. The patches for
AUX area support have been applied, just leaving patches for
Intel PT and Intel BTS. These have now been updated to reflect
new Intel PT features, plus a couple of fixes.
The patches can also be found here:
http://git.infradead.org/users/ahunter/linux-perf.git
An example (unchanged from V3) perf.data file and build id archive
can be found here:
http://git.infradead.org/~ahunter/tfr/
There is also a tar of the 3 most relevant files with debugging
symbols. These need to be placed in under the correct paths in
/usr/lib/debug to get symbols.
Changes in V8:
New patches:
perf auxtrace: Fix period type 'i' not working
perf tools: Fix perf-with-kcore handling of arguments containing spaces
perf tools: Fix Intel PT 'instructions' sample period
perf tools: Add perf_pmu__format_bits()
perf tools: Validate config term maximum value
perf tools: Extend the event parser maximum error index
perf tools: Add Intel PT support for PSB periods
perf tools: Add new Intel PT packet definitions
perf tools: Pass Intel PT information for decoding MTC and CYC
perf tools: Add Intel PT support for decoding MTC packets
perf tools: Add Intel PT support for using MTC packets
perf tools: Add Intel PT support for decoding CYC packets
perf tools: Add Intel PT support for using CYC packets
perf tools: Add Intel PT support for decoding TRACESTOP packets
perf tools: Update Intel PT documentation
Changes in V7:
Patches already applied:
perf db-export: Fix thread ref-counting
perf tools: Ensure thread-stack is flushed
perf tools: Allow auxtrace data alignment
perf tools: Add Intel PT instruction decoder
Copy the x86 instruction decoder into perf tools source
perf tools: Add Intel PT decoder
Fix Intel PT getting stuck in a loop
Check for being stuck in a loop. That can happen if a
decoder error results in the decoder erroneously setting
the ip to an address that is itself in an infinite loop
that consumes no packets. The only way to be in a loop
that consumes no packets is if it consists of unconditional
branches. So the check for being stuck is if we see
a repeating cycle of consecutive unconditional branches.
perf tools: Add Intel PT support
Add missing err check in intel-pt.
Fix missing thread__puts
Improve Intel PT sync to sideband events
To help synchronize trace data with sideband events
the timestamp when returning to userspace is estimated.
That was not always being done if switch information
was not available, but it is still useful for sync'ing
to mmap changes, so simplify by doing it always when
TSC is available. Also add log prints to help debug
synchronization to sideband.
Improve Intel PT timestamp estimation
Intel PT uses timestamps to synchronize side-band information
to trace data. However timestamps may not be frequent enough.
To improve accuracy, an estimated timestamp is calculated based
on the number of instructions executed since the last known
timestamp.
This patch improves that estimate by taking into account the CPU
frequency as represented by the Intel PT CBR (core-to-bus ratio)
packet.
perf tools: Add Intel BTS support
Fix missing thread__puts
Add a fix for an infinite loop in intel_bts_process_buffer
misplaced in a followup patch in the original patchkit
perf tools: Output sample flags and insn_len from intel_pt
Folded into: perf tools: Add Intel PT support
perf tools: Output sample flags and insn_len from intel_bts
Folded into: perf tools: Add Intel BTS support
perf tools: Intel PT to always update thread stack trace number
Folded into: perf tools: Add Intel PT support
perf tools: Intel BTS to always update thread stack trace number
Folded into: perf tools: Add Intel BTS support
Changes in V6:
Some minor expansion of commit messages.
Patches already applied:
perf tools: Disallow PMU events intel_pt and intel_bts until there is support
perf db-export: Fix thread ref-counting
New patch
perf tools: Ensure thread-stack is flushed
New patch
perf tools: Add Intel PT support
Support thread ref-counting
perf tools: Add Intel PT decoder
Fix a bug: FUP packet in PSB to update last IP
perf tools: Take Intel PT into use
Add Overview and Quickstart sections to intel_pt.txt
perf tools: Add Intel BTS support
Add Overview to intel_bts.txt
Support thread ref-counting
perf tools: Add example call-graph script
Add documentation comments to scripts
Changes in V5:
Patches already applied:
perf report: Fix placement of itrace option in documentation
perf tools: Add AUX area tracing index
perf tools: Hit all build ids when AUX area tracing
perf tools: Add build option NO_AUXTRACE to exclude AUX area tracing
perf auxtrace: Add option to synthesize events for transactions
perf tools: Add support for PERF_RECORD_AUX
perf tools: Add support for PERF_RECORD_ITRACE_START
perf tools: Add AUX area tracing Snapshot Mode
perf record: Add AUX area tracing Snapshot Mode support
perf tools: Disallow PMU events intel_pt and intel_bts until there is support
New patch
perf tools: Add Intel PT decoder
Style improvements pointed out by Acme: aligning '=', single line initializing
Make use of zalloc() not malloc / memset
Make use of zfree
Map internal error codes to fixed constants for output
Change intel_pt_error_message() to intel_pt__strerror()
perf tools: Add Intel PT support
Make use of zfree
perf tools: Take Intel PT into use
Allow "intel_pt" PMU to be selected as an event
perf tools: Add Intel BTS support
Allow "intel_bts" PMU to be selected as an event
Make use of zfree
Map internal error codes to fixed constants for output
Let "intel_bts" show up in 'perf list'
perf tools: Output sample flags and insn_len from intel_bts
Map internal error codes to fixed constants for output
Changes on V4:
perf tools: Amend mmap ref counting for the AUX area mmap
Dropped because already applied
perf script: Always allow fields 'addr' and 'cpu' for auxtrace
Dropped because already applied
perf report: Add Instruction Tracing support
Dropped because already applied
perf report: Fix placement of itrace option in documentation
New patch
perf tools: Add AUX area tracing index
Change size checks for more flexibility i.e.
- don't mind if an indexed auxtrace_event is bigger than
struct auxtrace_event
- don't mind if the auxtrace index does not fill the whole
file section
Rename 'index' variable to 'ent' to avoid build errors on
older gcc
perf tools: Add build option NO_AUXTRACE to exclude AUX area tracing
Fix whitespace alignment of NO_AUXTRACE=1
Add NO_AUXTRACE=1 to make_minimal
perf tools: Add support for PERF_RECORD_AUX
Expand commit message
perf tools: Add AUX area tracing Snapshot Mode
Whitespace fixups
perf record: Add AUX area tracing Snapshot Mode support
Whitespace fixups
Don't init static variables to 0 or NULL
perf tools: Add Intel PT packet decoder
Whitespace fixups
perf tools: Add Intel PT instruction decoder
Avoid build error on older (broken) gcc by adding -Wno-override-init
Avoid build errors due to funny collate sequences i.e. use LC_COLLATE=C etc
perf tools: Add Intel PT decoder
Avoid build errors initializing structures to 0
perf tools: Add Intel PT support
Avoid build errors initializing structures to 0
Allow for perf_pmu__config_terms() having an extra parameter now
Allow for parse_events() having an extra parameter now
Rename 'div' variable to 'd' to avoid build errors
Whitespace fixup
Remove a couple of unused enums
perf tools: Add Intel BTS support
Avoid build errors initializing structures to 0
Allow for parse_events() having an extra parameter now
perf tools: Put itrace options into an asciidoc include
New patch
Changes in V3:
New patch:
perf tools: Amend mmap ref counting for the AUX area mmap
Move some code under arch:
perf tools: Add Intel PT support
perf tools: Add Intel BTS support
Updated documentation:
perf report: Add Instruction Tracing support
perf auxtrace: Add option to synthesize events for transactions
perf tools: Take Intel PT into use
perf tools: Add Intel BTS support
Patches already applied:
perf header: Add AUX area tracing feature
perf evlist: Add support for mmapping an AUX area buffer
perf tools: Add user events for AUX area tracing
perf tools: Add support for AUX area recording
perf record: Add basic AUX area tracing support
perf record: Extend -m option for AUX area tracing mmap pages
perf tools: Add a user event for AUX area tracing errors
perf session: Add hooks to allow transparent decoding of AUX area tracing data
perf session: Add instruction tracing options
perf auxtrace: Add helpers for AUX area tracing errors
perf auxtrace: Add helpers for queuing AUX area tracing data
perf auxtrace: Add a heap for sorting AUX area tracing queues
perf auxtrace: Add processing for AUX area tracing events
perf auxtrace: Add a hashtable for caching
perf tools: Add member to struct dso for an instruction cache
perf script: Add Instruction Tracing support
perf inject: Re-pipe AUX area tracing events
perf inject: Add Instruction Tracing support
perf script: Add field option 'flags' to print sample flags
perf tools: Add aux_watermark member of struct perf_event_attr
Changes in V2:
Get rid of MIN()
perf auxtrace: Add helpers for AUX area tracing errors
perf inject: Re-pipe AUX area tracing events
perf tools: Add build option NO_AUXTRACE to exclude AUX area tracing
Intel BTS can be used on most recent Intel CPUs. Intel PT
is available on Broadwell.
Examples:
Trace 'ls' with Intel BTS userspace only
perf record --per-thread -e intel_bts//u ls
perf report
perf script
Trace 'ls' with Intel BTS kernel and userspace
~/libexec/perf-core/perf-with-kcore record bts-ls --per-thread -e intel_bts// -- ls
~/libexec/perf-core/perf-with-kcore report bts-ls
~/libexec/perf-core/perf-with-kcore script bts-ls
Trace 'ls' with Intel PT userspace only
perf record -e intel_pt//u ls
perf report
perf script
Trace 'ls' with Intel PT kernel and userspace
~/libexec/perf-core/perf-with-kcore record pt-ls -e intel_pt// -- ls
~/libexec/perf-core/perf-with-kcore report pt-ls
~/libexec/perf-core/perf-with-kcore script pt-ls
The abstraction has two separate aspects:
1. recording AUX area data
2. processing AUX area data
Recording consists of mmapping a separate buffer and copying
the data into the perf.data file. The buffer is an AUX area
buffer. The data is written preceded by a new user event
PERF_RECORD_AUXTRACE. The data is too big to fit in the event
but follows immediately afterward. Session processing has to
skip to get to the next event header in a similar fashion to
the existing PERF_RECORD_HEADER_TRACING_DATA
event. The main recording patches are:
perf evlist: Add support for mmapping an AUX area buffer
perf tools: Add user events for AUX area tracing
perf tools: Add support for AUX area recording
perf record: Add basic AUX area tracing support
Processing consists of providing hooks in session processing
to enable a decoder to see all the events and deliver synthesized
events transparently into the event stream. The main processing
patch is:
perf session: Add hooks to allow transparent decoding of AUX area tracing data
Adrian Hunter (25):
perf auxtrace: Add Intel PT as an AUX area tracing type
perf tools: Add Intel PT packet decoder
perf tools: Add Intel PT instruction decoder
perf tools: Add Intel PT log
perf tools: Add Intel PT decoder
perf tools: Add Intel PT support
perf tools: Take Intel PT into use
perf tools: Add Intel BTS support
perf tools: Put itrace options into an asciidoc include
perf tools: Add example call-graph script
perf auxtrace: Fix period type 'i' not working
perf tools: Fix perf-with-kcore handling of arguments containing spaces
perf tools: Fix Intel PT 'instructions' sample period
perf tools: Add perf_pmu__format_bits()
perf tools: Validate config term maximum value
perf tools: Extend the event parser maximum error index
perf tools: Add Intel PT support for PSB periods
perf tools: Add new Intel PT packet definitions
perf tools: Pass Intel PT information for decoding MTC and CYC
perf tools: Add Intel PT support for decoding MTC packets
perf tools: Add Intel PT support for using MTC packets
perf tools: Add Intel PT support for decoding CYC packets
perf tools: Add Intel PT support for using CYC packets
perf tools: Add Intel PT support for decoding TRACESTOP packets
perf tools: Update Intel PT documentation
tools/build/Makefile.build | 2 +
tools/perf/.gitignore | 1 +
tools/perf/Documentation/intel-bts.txt | 86 +
tools/perf/Documentation/intel-pt.txt | 766 +++++++
tools/perf/Documentation/itrace.txt | 22 +
tools/perf/Documentation/perf-inject.txt | 23 +-
tools/perf/Documentation/perf-report.txt | 23 +-
tools/perf/Documentation/perf-script.txt | 23 +-
tools/perf/Makefile.perf | 12 +-
tools/perf/arch/x86/util/Build | 5 +
tools/perf/arch/x86/util/auxtrace.c | 83 +
tools/perf/arch/x86/util/intel-bts.c | 458 ++++
tools/perf/arch/x86/util/intel-pt.c | 1007 +++++++++
tools/perf/arch/x86/util/pmu.c | 18 +
tools/perf/perf-with-kcore.sh | 28 +-
.../scripts/python/call-graph-from-postgresql.py | 327 +++
tools/perf/scripts/python/export-to-postgresql.py | 47 +
tools/perf/util/Build | 3 +
tools/perf/util/auxtrace.c | 15 +-
tools/perf/util/auxtrace.h | 2 +
tools/perf/util/intel-bts.c | 933 ++++++++
tools/perf/util/intel-bts.h | 43 +
tools/perf/util/intel-pt-decoder/Build | 11 +
.../util/intel-pt-decoder/gen-insn-attr-x86.awk | 386 ++++
tools/perf/util/intel-pt-decoder/inat.c | 96 +
tools/perf/util/intel-pt-decoder/inat.h | 221 ++
tools/perf/util/intel-pt-decoder/inat_types.h | 29 +
tools/perf/util/intel-pt-decoder/insn.c | 594 +++++
tools/perf/util/intel-pt-decoder/insn.h | 201 ++
.../perf/util/intel-pt-decoder/intel-pt-decoder.c | 2345 ++++++++++++++++++++
.../perf/util/intel-pt-decoder/intel-pt-decoder.h | 109 +
.../util/intel-pt-decoder/intel-pt-insn-decoder.c | 246 ++
.../util/intel-pt-decoder/intel-pt-insn-decoder.h | 65 +
tools/perf/util/intel-pt-decoder/intel-pt-log.c | 155 ++
tools/perf/util/intel-pt-decoder/intel-pt-log.h | 52 +
.../util/intel-pt-decoder/intel-pt-pkt-decoder.c | 518 +++++
.../util/intel-pt-decoder/intel-pt-pkt-decoder.h | 70 +
.../perf/util/intel-pt-decoder/x86-opcode-map.txt | 970 ++++++++
tools/perf/util/intel-pt.c | 1956 ++++++++++++++++
tools/perf/util/intel-pt.h | 56 +
tools/perf/util/parse-events.c | 2 +-
tools/perf/util/pmu.c | 51 +-
tools/perf/util/pmu.h | 1 +
43 files changed, 11970 insertions(+), 91 deletions(-)
create mode 100644 tools/perf/Documentation/intel-bts.txt
create mode 100644 tools/perf/Documentation/intel-pt.txt
create mode 100644 tools/perf/Documentation/itrace.txt
create mode 100644 tools/perf/arch/x86/util/auxtrace.c
create mode 100644 tools/perf/arch/x86/util/intel-bts.c
create mode 100644 tools/perf/arch/x86/util/intel-pt.c
create mode 100644 tools/perf/arch/x86/util/pmu.c
create mode 100644 tools/perf/scripts/python/call-graph-from-postgresql.py
create mode 100644 tools/perf/util/intel-bts.c
create mode 100644 tools/perf/util/intel-bts.h
create mode 100644 tools/perf/util/intel-pt-decoder/Build
create mode 100644 tools/perf/util/intel-pt-decoder/gen-insn-attr-x86.awk
create mode 100644 tools/perf/util/intel-pt-decoder/inat.c
create mode 100644 tools/perf/util/intel-pt-decoder/inat.h
create mode 100644 tools/perf/util/intel-pt-decoder/inat_types.h
create mode 100644 tools/perf/util/intel-pt-decoder/insn.c
create mode 100644 tools/perf/util/intel-pt-decoder/insn.h
create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-decoder.c
create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-decoder.h
create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c
create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.h
create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-log.c
create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-log.h
create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-pkt-decoder.c
create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-pkt-decoder.h
create mode 100644 tools/perf/util/intel-pt-decoder/x86-opcode-map.txt
create mode 100644 tools/perf/util/intel-pt.c
create mode 100644 tools/perf/util/intel-pt.h
Regards
Adrian
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists