lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-Id: <1430404667-10593-1-git-send-email-adrian.hunter@intel.com>
Date:	Thu, 30 Apr 2015 17:37:23 +0300
From:	Adrian Hunter <adrian.hunter@...el.com>
To:	Arnaldo Carvalho de Melo <acme@...nel.org>
Cc:	Peter Zijlstra <peterz@...radead.org>,
	linux-kernel@...r.kernel.org, David Ahern <dsahern@...il.com>,
	Frederic Weisbecker <fweisbec@...il.com>,
	Jiri Olsa <jolsa@...hat.com>,
	Namhyung Kim <namhyung@...il.com>,
	Stephane Eranian <eranian@...gle.com>
Subject: [PATCH V4 00/24] perf tools: Introduce an abstraction for AUX Area  and Instruction Tracing

Hi

Here is V4 patches for the introduction of an abstraction for
using the AUX area and Instruction tracing.

The patches can also be found here:

	http://git.infradead.org/users/ahunter/linux-perf.git

An example (unchanged from V3) perf.data file and build id archive
can be found here:

	http://git.infradead.org/~ahunter/tfr/

There is also a tar of the 3 most relevant files with debugging
symbols. These need to be placed in under the correct paths in
/usr/lib/debug to get symbols.

Changes on V4:

   perf tools: Amend mmap ref counting for the AUX area mmap
      Dropped because already applied

   perf script: Always allow fields 'addr' and 'cpu' for auxtrace
      Dropped because already applied

   perf report: Add Instruction Tracing support
      Dropped because already applied

   perf report: Fix placement of itrace option in documentation
      New patch

   perf tools: Add AUX area tracing index
      Change size checks for more flexibility i.e.
      - don't mind if an indexed auxtrace_event is bigger than
      struct auxtrace_event
      - don't mind if the auxtrace index does not fill the whole
      file section
      Rename 'index' variable to 'ent' to avoid build errors on
      older gcc

   perf tools: Add build option NO_AUXTRACE to exclude AUX area tracing
      Fix whitespace alignment of NO_AUXTRACE=1
      Add NO_AUXTRACE=1 to make_minimal

   perf tools: Add support for PERF_RECORD_AUX
      Expand commit message

   perf tools: Add AUX area tracing Snapshot Mode
      Whitespace fixups

   perf record: Add AUX area tracing Snapshot Mode support
      Whitespace fixups
      Don't init static variables to 0 or NULL

   perf tools: Add Intel PT packet decoder
      Whitespace fixups

   perf tools: Add Intel PT instruction decoder
      Avoid build error on older (broken) gcc by adding -Wno-override-init
      Avoid build errors due to funny collate sequences i.e. use LC_COLLATE=C etc

   perf tools: Add Intel PT decoder
      Avoid build errors initializing structures to 0

   perf tools: Add Intel PT support
      Avoid build errors initializing structures to 0
      Allow for perf_pmu__config_terms() having an extra parameter now
      Allow for parse_events() having an extra parameter now
      Rename 'div' variable to 'd' to avoid build errors
      Whitespace fixup
      Remove a couple of unused enums

   perf tools: Add Intel BTS support
      Avoid build errors initializing structures to 0
      Allow for parse_events() having an extra parameter now

   perf tools: Put itrace options into an asciidoc include
      New patch

Changes in V3:

   New patch:
      perf tools: Amend mmap ref counting for the AUX area mmap

   Move some code under arch:
      perf tools: Add Intel PT support
      perf tools: Add Intel BTS support

   Updated documentation:
      perf report: Add Instruction Tracing support
      perf auxtrace: Add option to synthesize events for transactions
      perf tools: Take Intel PT into use
      perf tools: Add Intel BTS support

   Patches already applied:
      perf header: Add AUX area tracing feature
      perf evlist: Add support for mmapping an AUX area buffer
      perf tools: Add user events for AUX area tracing
      perf tools: Add support for AUX area recording
      perf record: Add basic AUX area tracing support
      perf record: Extend -m option for AUX area tracing mmap pages
      perf tools: Add a user event for AUX area tracing errors
      perf session: Add hooks to allow transparent decoding of AUX area tracing data
      perf session: Add instruction tracing options
      perf auxtrace: Add helpers for AUX area tracing errors
      perf auxtrace: Add helpers for queuing AUX area tracing data
      perf auxtrace: Add a heap for sorting AUX area tracing queues
      perf auxtrace: Add processing for AUX area tracing events
      perf auxtrace: Add a hashtable for caching
      perf tools: Add member to struct dso for an instruction cache
      perf script: Add Instruction Tracing support
      perf inject: Re-pipe AUX area tracing events
      perf inject: Add Instruction Tracing support
      perf script: Add field option 'flags' to print sample flags
      perf tools: Add aux_watermark member of struct perf_event_attr

Changes in V2:

   Get rid of MIN()
      perf auxtrace: Add helpers for AUX area tracing errors
      perf inject: Re-pipe AUX area tracing events
      perf tools: Add build option NO_AUXTRACE to exclude AUX area tracing


Intel BTS can be used on most recent Intel CPUs. Intel PT
is available on Broadwell.

Examples:

	Trace 'ls' with Intel BTS userspace only

	perf record --per-thread -e intel_bts//u ls
	perf report
	perf script

	Trace 'ls' with Intel BTS kernel and userspace

	~/libexec/perf-core/perf-with-kcore record bts-ls --per-thread -e intel_bts// -- ls
	~/libexec/perf-core/perf-with-kcore report bts-ls
	~/libexec/perf-core/perf-with-kcore script bts-ls

	Trace 'ls' with Intel PT userspace only

	perf record -e intel_pt//u ls
	perf report
	perf script

	Trace 'ls' with Intel PT kernel and userspace

	~/libexec/perf-core/perf-with-kcore record pt-ls -e intel_pt// -- ls
	~/libexec/perf-core/perf-with-kcore report pt-ls
	~/libexec/perf-core/perf-with-kcore script pt-ls


The abstraction has two separate aspects:
	1. recording AUX area data
	2. processing AUX area data

Recording consists of mmapping a separate buffer and copying
the data into the perf.data file.  The buffer is an AUX area
buffer.  The data is written preceded by a new user event
PERF_RECORD_AUXTRACE.  The data is too big to fit in the event
but follows immediately afterward. Session processing has to
skip to get to the next event header in a similar fashion to
the existing PERF_RECORD_HEADER_TRACING_DATA
event.  The main recording patches are:

      perf evlist: Add support for mmapping an AUX area buffer
      perf tools: Add user events for AUX area tracing
      perf tools: Add support for AUX area recording
      perf record: Add basic AUX area tracing support

Processing consists of providing hooks in session processing
to enable a decoder to see all the events and deliver synthesized
events transparently into the event stream.  The main processing
patch is:

      perf session: Add hooks to allow transparent decoding of AUX area tracing data


Adrian Hunter (24):
      perf report: Fix placement of itrace option in documentation
      perf tools: Add AUX area tracing index
      perf tools: Hit all build ids when AUX area tracing
      perf tools: Add build option NO_AUXTRACE to exclude AUX area tracing
      perf auxtrace: Add option to synthesize events for transactions
      perf tools: Add support for PERF_RECORD_AUX
      perf tools: Add support for PERF_RECORD_ITRACE_START
      perf tools: Add AUX area tracing Snapshot Mode
      perf record: Add AUX area tracing Snapshot Mode support
      perf auxtrace: Add Intel PT as an AUX area tracing type
      perf tools: Add Intel PT packet decoder
      perf tools: Add Intel PT instruction decoder
      perf tools: Add Intel PT log
      perf tools: Add Intel PT decoder
      perf tools: Add Intel PT support
      perf tools: Take Intel PT into use
      perf tools: Allow auxtrace data alignment
      perf tools: Add Intel BTS support
      perf tools: Output sample flags and insn_len from intel_pt
      perf tools: Output sample flags and insn_len from intel_bts
      perf tools: Intel PT to always update thread stack trace number
      perf tools: Intel BTS to always update thread stack trace number
      perf tools: Add example call-graph script
      perf tools: Put itrace options into an asciidoc include

 tools/build/Makefile.build                         |    2 +
 tools/perf/.gitignore                              |    2 +
 tools/perf/Documentation/intel-bts.txt             |   69 +
 tools/perf/Documentation/intel-pt.txt              |  467 +++++
 tools/perf/Documentation/itrace.txt                |   22 +
 tools/perf/Documentation/perf-inject.txt           |   22 +-
 tools/perf/Documentation/perf-record.txt           |    7 +
 tools/perf/Documentation/perf-report.txt           |   26 +-
 tools/perf/Documentation/perf-script.txt           |   22 +-
 tools/perf/Makefile.perf                           |   14 +-
 tools/perf/arch/x86/util/Build                     |    5 +
 tools/perf/arch/x86/util/auxtrace.c                |   83 +
 tools/perf/arch/x86/util/intel-bts.c               |  458 +++++
 tools/perf/arch/x86/util/intel-pt.c                |  752 ++++++++
 tools/perf/arch/x86/util/pmu.c                     |   15 +
 tools/perf/builtin-buildid-list.c                  |    9 +
 tools/perf/builtin-inject.c                        |   78 +-
 tools/perf/builtin-record.c                        |  172 +-
 tools/perf/config/Makefile                         |    5 +
 tools/perf/perf.h                                  |    3 +
 .../scripts/python/call-graph-from-postgresql.py   |  285 +++
 tools/perf/tests/make                              |    4 +-
 tools/perf/util/Build                              |    5 +-
 tools/perf/util/auxtrace.c                         |  321 +++-
 tools/perf/util/auxtrace.h                         |  220 +++
 tools/perf/util/event.c                            |   39 +
 tools/perf/util/event.h                            |   24 +
 tools/perf/util/header.c                           |   31 +-
 tools/perf/util/intel-bts.c                        |  919 ++++++++++
 tools/perf/util/intel-bts.h                        |   43 +
 tools/perf/util/intel-pt-decoder/Build             |   14 +
 .../perf/util/intel-pt-decoder/intel-pt-decoder.c  | 1738 ++++++++++++++++++
 .../perf/util/intel-pt-decoder/intel-pt-decoder.h  |   89 +
 .../util/intel-pt-decoder/intel-pt-insn-decoder.c  |  246 +++
 .../util/intel-pt-decoder/intel-pt-insn-decoder.h  |   65 +
 tools/perf/util/intel-pt-decoder/intel-pt-log.c    |  155 ++
 tools/perf/util/intel-pt-decoder/intel-pt-log.h    |   52 +
 .../util/intel-pt-decoder/intel-pt-pkt-decoder.c   |  400 +++++
 .../util/intel-pt-decoder/intel-pt-pkt-decoder.h   |   64 +
 tools/perf/util/intel-pt.c                         | 1897 ++++++++++++++++++++
 tools/perf/util/intel-pt.h                         |   51 +
 tools/perf/util/machine.c                          |   21 +
 tools/perf/util/machine.h                          |    4 +
 tools/perf/util/parse-options.h                    |    4 +
 tools/perf/util/session.c                          |   32 +
 tools/perf/util/session.h                          |    1 +
 tools/perf/util/tool.h                             |    2 +
 47 files changed, 8837 insertions(+), 122 deletions(-)
 create mode 100644 tools/perf/Documentation/intel-bts.txt
 create mode 100644 tools/perf/Documentation/intel-pt.txt
 create mode 100644 tools/perf/Documentation/itrace.txt
 create mode 100644 tools/perf/arch/x86/util/auxtrace.c
 create mode 100644 tools/perf/arch/x86/util/intel-bts.c
 create mode 100644 tools/perf/arch/x86/util/intel-pt.c
 create mode 100644 tools/perf/arch/x86/util/pmu.c
 create mode 100644 tools/perf/scripts/python/call-graph-from-postgresql.py
 create mode 100644 tools/perf/util/intel-bts.c
 create mode 100644 tools/perf/util/intel-bts.h
 create mode 100644 tools/perf/util/intel-pt-decoder/Build
 create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-decoder.c
 create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-decoder.h
 create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c
 create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.h
 create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-log.c
 create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-log.h
 create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-pkt-decoder.c
 create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-pkt-decoder.h
 create mode 100644 tools/perf/util/intel-pt.c
 create mode 100644 tools/perf/util/intel-pt.h


Regards
Adrian
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ