linux-kernel - Re: [PATCH 14/14] perf record: Add --dir option to store data in directory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20190205135137.GI4794@krava>
Date:   Tue, 5 Feb 2019 14:51:37 +0100
From:   Jiri Olsa <jolsa@...hat.com>
To:     Alexey Budankov <alexey.budankov@...ux.intel.com>
Cc:     Jiri Olsa <jolsa@...nel.org>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        lkml <linux-kernel@...r.kernel.org>,
        Ingo Molnar <mingo@...nel.org>,
        Namhyung Kim <namhyung@...nel.org>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Peter Zijlstra <a.p.zijlstra@...llo.nl>,
        Adrian Hunter <adrian.hunter@...el.com>,
        Andi Kleen <ak@...ux.intel.com>,
        Stephane Eranian <eranian@...gle.com>
Subject: Re: [PATCH 14/14] perf record: Add --dir option to store data in
 directory

On Tue, Feb 05, 2019 at 03:36:24PM +0300, Alexey Budankov wrote:
> 
> On 03.02.2019 18:30, Jiri Olsa wrote:
> > Adding --dir option to store data in directory. It's next
> > step for multiple threads in record. It's not possible
> > to make directory data via --dir option, like:
> > 
> >   $ perf record --dir perf bench sched messaging
> >   $ ls -l perf.data
> >   total 344
> >   -rw-------. 1 jolsa jolsa 43864 Jan 20 22:26 data.0
> >   -rw-------. 1 jolsa jolsa 30464 Jan 20 22:26 data.1
> >   -rw-------. 1 jolsa jolsa 53816 Jan 20 22:26 data.2
> >   -rw-------. 1 jolsa jolsa 30368 Jan 20 22:26 data.3
> >   -rw-------. 1 jolsa jolsa 40088 Jan 20 22:26 data.4
> >   -rw-------. 1 jolsa jolsa 42592 Jan 20 22:26 data.5
> >   -rw-------. 1 jolsa jolsa 56136 Jan 20 22:26 data.6
> >   -rw-------. 1 jolsa jolsa 25992 Jan 20 22:26 data.7
> >   -rw-------. 1 jolsa jolsa  8832 Jan 20 22:26 header
> > 
> > There's a data file created for every cpu and it's storing
> > data for those cpu maps.
> > 
> > It's possible to transform directory data into standard
> > perf.data file via following inject command:
> > 
> >   $ perf inject -o perf.data.file -i perf.data
> > 
> > Link: http://lkml.kernel.org/n/tip-0kjm8wpglzu2tm18tpagfm4d@git.kernel.org
> > Signed-off-by: Jiri Olsa <jolsa@...nel.org>
> > ---
> >  tools/perf/Documentation/perf-record.txt |  3 ++
> >  tools/perf/builtin-record.c              | 59 ++++++++++++++++++++++--
> >  tools/perf/util/mmap.h                   | 23 ++++-----
> >  3 files changed, 70 insertions(+), 15 deletions(-)
> > 
> > diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt
> > index d232b13ea713..8dcdc8cabcad 100644
> > --- a/tools/perf/Documentation/perf-record.txt
> > +++ b/tools/perf/Documentation/perf-record.txt
> > @@ -505,6 +505,9 @@ config terms. For example: 'cycles/overwrite/' and 'instructions/no-overwrite/'.
> >  
> >  Implies --tail-synthesize.
> >  
> > +--dir::
> > +Store data into directory with one data file for cpu.
> > +
> 
> Makes sense to mention compatibility with -o option and per-thread buffer mapping.

right, will do

> 
> >  SEE ALSO
> >  --------
> >  linkperf:perf-stat[1], linkperf:perf-list[1]
> > diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
> > index cd02ab3ec4ff..87e39b9cc7bd 100644
> > --- a/tools/perf/builtin-record.c
> > +++ b/tools/perf/builtin-record.c
> > @@ -111,17 +111,21 @@ static bool switch_output_time(struct record *rec)
> >  	       trigger_is_ready(&switch_output_trigger);
> >  }
> >  
> > -static int record__write(struct record *rec, struct perf_mmap *map __maybe_unused,
> > +static int record__write(struct record *rec, struct perf_mmap *map,
> >  			 void *bf, size_t size)
> >  {
> > -	struct perf_data_file *file = &rec->session->data->file;
> > +	struct perf_data_file *file = &rec->data.file;
> > +
> > +	if (map && map->file)
> > +		file = map->file;
> 
> For AIO mode per-cpu streaming could be done in parallel because Posix 
> AIO API uses a separate thread for every open data.# or header fd.

yes, I was thinking of that and I plan to try that to see
if there's some benefit

> 
> >  
> >  	if (perf_data_file__write(file, bf, size) < 0) {
> >  		pr_err("failed to write perf data, error: %m\n");
> >  		return -1;
> >  	}
> >  
> > -	rec->bytes_written += size;
> > +	if (file == &rec->data.file)
> > +		rec->bytes_written += size;
> 
> switch-output logic now tracks header file size only? 
> If so, it probably needs to be corrected or simply disabled 
> for --output_dir mode.

the rec->bytes_written value is used to track header file size,
which is used to fill in the header data section in here
record__finish_output, we can't mix other file sizes into it

the switch-output option still needs some changes to work
over the directory data

jirka