linux-kernel - [RFC/PATCH 00/14] perf record: Add support to store data in directory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <20190203153018.9650-1-jolsa@kernel.org>
Date:   Sun,  3 Feb 2019 16:30:04 +0100
From:   Jiri Olsa <jolsa@...nel.org>
To:     Arnaldo Carvalho de Melo <acme@...nel.org>
Cc:     lkml <linux-kernel@...r.kernel.org>,
        Ingo Molnar <mingo@...nel.org>,
        Namhyung Kim <namhyung@...nel.org>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Peter Zijlstra <a.p.zijlstra@...llo.nl>,
        Adrian Hunter <adrian.hunter@...el.com>,
        Andi Kleen <ak@...ux.intel.com>,
        Stephane Eranian <eranian@...gle.com>,
        Alexey Budankov <alexey.budankov@...ux.intel.com>
Subject: [RFC/PATCH 00/14] perf record: Add support to store data in directory

hi,
this patchset adds the --dir option to record command (and all
the other record command that overload cmd_record) that allows
the data to be stored in directory with multiple data files.

It's next step for multiple threads implementation in record.
It's now possible to make directory data via --dir option, like:

  $ perf record --dir perf bench sched messaging
  $ ls -l perf.data
  total 344
  -rw-------. 1 jolsa jolsa 43864 Jan 20 22:26 data.0
  -rw-------. 1 jolsa jolsa 30464 Jan 20 22:26 data.1
  -rw-------. 1 jolsa jolsa 53816 Jan 20 22:26 data.2
  -rw-------. 1 jolsa jolsa 30368 Jan 20 22:26 data.3
  -rw-------. 1 jolsa jolsa 40088 Jan 20 22:26 data.4
  -rw-------. 1 jolsa jolsa 42592 Jan 20 22:26 data.5
  -rw-------. 1 jolsa jolsa 56136 Jan 20 22:26 data.6
  -rw-------. 1 jolsa jolsa 25992 Jan 20 22:26 data.7
  -rw-------. 1 jolsa jolsa  8832 Jan 20 22:26 header

There's a data file created for every cpu and it's storing
data for those cpu maps. The report command will read it
transparently, sort it and display as single file data.

It's possible to transform directory data into standard
perf.data file via simple inject command:
    
  $ perf inject -o perf.data.file -i perf.data

The old perf fails over the directory data with following message:
  $ perf report
  incompatible file format (rerun with -v to learn more)

I'm now testing the record threads support, so I'd like to
have some agreement on the directory data support before.

It's also available in here:
  git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
  perf/dir

thanks,
jirka


---
Jiri Olsa (14):
      perf tools: Make rm_rf to remove single file
      perf session: Add process callback to reader object
      perf data: Move size to struct perf_data_file
      perf data: Add global path holder
      perf data: Make check_backup work over directories
      perf data: Add perf_data__(create_dir|free_dir) functions
      perf data: Add perf_data__open_dir_data function
      perf data: Add directory support
      perf data: Don't store auxtrace index for directory data file
      perf data: Add perf_data__update_dir function
      perf data: Make perf_data__size to work over directory
      perf session: Add __perf_session__process_dir_events function
      perf session: Add path to reader object
      perf record: Add --dir option to store data in directory

 tools/perf/Documentation/perf-record.txt |   3 ++
 tools/perf/builtin-annotate.c            |   4 +--
 tools/perf/builtin-buildid-cache.c       |   4 +--
 tools/perf/builtin-buildid-list.c        |   8 ++---
 tools/perf/builtin-c2c.c                 |   4 +--
 tools/perf/builtin-diff.c                |  12 +++----
 tools/perf/builtin-evlist.c              |   4 +--
 tools/perf/builtin-inject.c              |  10 +++---
 tools/perf/builtin-kmem.c                |   2 +-
 tools/perf/builtin-kvm.c                 |   8 ++---
 tools/perf/builtin-lock.c                |   8 ++---
 tools/perf/builtin-mem.c                 |   8 ++---
 tools/perf/builtin-record.c              |  72 +++++++++++++++++++++++++++++++------
 tools/perf/builtin-report.c              |   6 ++--
 tools/perf/builtin-sched.c               |  16 ++++-----
 tools/perf/builtin-script.c              |  12 +++----
 tools/perf/builtin-stat.c                |   6 ++--
 tools/perf/builtin-timechart.c           |   8 ++---
 tools/perf/builtin-trace.c               |   8 ++---
 tools/perf/util/data-convert-bt.c        |   2 +-
 tools/perf/util/data.c                   | 231 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/data.h                   |  24 +++++++++----
 tools/perf/util/mmap.h                   |  23 ++++++------
 tools/perf/util/session.c                | 117 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 tools/perf/util/util.c                   |  10 ++++--
 25 files changed, 479 insertions(+), 131 deletions(-)