linux-kernel - Re: [PATCHv6 0/5] perf record: Add support to store data in directory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAM9d7chND4RwmsS3Bf24KKp320myqZ6B6ZfaCGRZ1LM0dAp+9A@mail.gmail.com>
Date:   Wed, 13 Mar 2019 23:29:51 +0900
From:   Namhyung Kim <namhyung@...nel.org>
To:     Jiri Olsa <jolsa@...nel.org>
Cc:     Arnaldo Carvalho de Melo <acme@...nel.org>,
        lkml <linux-kernel@...r.kernel.org>,
        Ingo Molnar <mingo@...nel.org>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Peter Zijlstra <a.p.zijlstra@...llo.nl>,
        Adrian Hunter <adrian.hunter@...el.com>,
        Andi Kleen <ak@...ux.intel.com>,
        Stephane Eranian <eranian@...gle.com>,
        Alexey Budankov <alexey.budankov@...ux.intel.com>
Subject: Re: [PATCHv6 0/5] perf record: Add support to store data in directory

On Mon, Mar 11, 2019 at 12:30 AM Jiri Olsa <jolsa@...nel.org> wrote:
>
> hi,
> this patchset adds the --dir option to record command (and all
> the other record command that overload cmd_record) that allows
> the data to be stored in directory with multiple data files.
>
> It's next step for multiple threads implementation in record.
> It's now possible to make directory data via --dir option, like:
>
>   $ perf record --dir perf bench sched messaging
>   $ ls -l perf.data
>   total 344
>   -rw-------. 1 jolsa jolsa 43864 Jan 20 22:26 data.0
>   -rw-------. 1 jolsa jolsa 30464 Jan 20 22:26 data.1
>   -rw-------. 1 jolsa jolsa 53816 Jan 20 22:26 data.2
>   -rw-------. 1 jolsa jolsa 30368 Jan 20 22:26 data.3
>   -rw-------. 1 jolsa jolsa 40088 Jan 20 22:26 data.4
>   -rw-------. 1 jolsa jolsa 42592 Jan 20 22:26 data.5
>   -rw-------. 1 jolsa jolsa 56136 Jan 20 22:26 data.6
>   -rw-------. 1 jolsa jolsa 25992 Jan 20 22:26 data.7
>   -rw-------. 1 jolsa jolsa  8832 Jan 20 22:26 header
>
> There's a data file created for every cpu and it's storing
> data for those cpu maps. The report command will read it
> transparently, sort it and display as single file data.
>
> There's new DIR_FORMAT feature to describe directory data
> layout/format. In future we can describe different data files
> layout according to special needs.
>
> It's possible to transform directory data into standard
> perf.data file via simple inject command:
>
>   $ perf inject -o perf.data.file -i perf.data
>
> The old perf fails over the directory data with following message:
>   $ perf report
>   incompatible file format (rerun with -v to learn more)
>
> I'm now testing the record threads support, so I'd like to
> have some agreement on the directory data support before.
>
> v6 changes:
>   - rebased to latest Arnaldo's perf/core
>   - some of the patches already taken
>   - adding comments to __perf_session__process_dir_events
>
> v5 changes:
>   - rebased to latest Arnaldo's perf/core
>
> v4 changes:
>   - some of the patches already taken
>   - removing up perf.data directory on perf_data__open
>     error path
>
> v3 changes:
>   - add rm_rf_perf_data to safely remove file/directory perf data
>   - allocation fix in perf_data__create_dir
>
> v2 changes:
>   - rm_rf changes are already accepted with requested changes
>   - updated doc/man plus adding perf.data-directory-format.txt
>     to describe directory format/layout
>   - the --switch-output options now works over directory data
>   - data rollback is not part of this patchset, updated my TODO though ;-)
>   - added --output-dir option to combine -o and --dir
>   - added DIR_FORMAT feature to describe directory data
>   - disabling directory output for aio for now
>
> It's also available in here:
>   git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
>   perf/dir
>

It seems I'm too late, but anyway for the series

Acked-by: Namhyung Kim <namhyung@...nel.org>

Thanks for the work!
Namhyung



> Jiri Olsa (5):
>       perf session: Add __perf_session__process_dir_events function
>       perf session: Add path to reader object
>       perf record: Add --dir option to store data in directory
>       perf record: Add --output-dir option to store data in directory
>       perf record: Describe perf.data directory format
>
>  tools/lib/subcmd/parse-options.h                        |   4 ++++
>  tools/perf/Documentation/perf-record.txt                |   6 ++++++
>  tools/perf/Documentation/perf.data-directory-format.txt |  54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tools/perf/builtin-record.c                             |  93 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
>  tools/perf/util/mmap.h                                  |  23 ++++++++++++-----------
>  tools/perf/util/session.c                               | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----
>  6 files changed, 272 insertions(+), 22 deletions(-)
>  create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt