linux-kernel - Re: [PATCH 14/42] perf record: Add --index option for building index table

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150201180635.GA6317@krava.brq.redhat.com>
Date:	Sun, 1 Feb 2015 19:06:35 +0100
From:	Jiri Olsa <jolsa@...hat.com>
To:	Namhyung Kim <namhyung@...nel.org>
Cc:	Arnaldo Carvalho de Melo <acme@...nel.org>,
	Ingo Molnar <mingo@...nel.org>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	LKML <linux-kernel@...r.kernel.org>,
	David Ahern <dsahern@...il.com>,
	Adrian Hunter <adrian.hunter@...el.com>,
	Andi Kleen <andi@...stfloor.org>,
	Stephane Eranian <eranian@...gle.com>,
	Frederic Weisbecker <fweisbec@...il.com>
Subject: Re: [PATCH 14/42] perf record: Add --index option for building index
 table

On Thu, Jan 29, 2015 at 05:06:55PM +0900, Namhyung Kim wrote:
> The new --index option will create indexed data file which can be
> processed by multiple threads parallelly.  It saves meta event and
> sample data in separate files and merges them with an index table.
> 
> To build an index table, it needs to know exact offsets and sizes for
> each sample data.  However the offset only can be calculated after the
> feature data is fixed, and to save feature data it needs to access to
> the sample data because it needs to mark used DSOs for build-id table.
> 
> So I ended up with reserving 1MB hole for the feature data area and then
> put sample data and calculated offsets.  Now an indexed perf data file
> will look like below:
> 
>         +---------------------+
>         |     file header     |
>         |---------------------|
>         |                     |
>         |     meta events     |
>         |                     |
>         |---------------------|
>         |     feature data    |
>         |   (contains index) -+--+
>         |---------------------|  |
>         |      ~1MB hole      |  |
>         |---------------------|  |
>         |                     |  |
>         |    sample data[1] <-+--+
>         |                     |  |
>         |---------------------|  |
>         |                     |  |
>         |    sample data[2] <-|--+
>         |                     |  |
>         |---------------------|  |
>         |         ...         | ...
>         +---------------------+

I also dont see how to store it in a nice way under current header layout,
but how about bump up the header version for this feature? ;-)

currently it's:

struct perf_file_header {
        u64                             magic;
        u64                             size;
        u64                             attr_size;
        struct perf_file_section        attrs;
        struct perf_file_section        data;
        /* event_types is ignored */
        struct perf_file_section        event_types;
        DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
};


- we already store attrs as a FEATURE so we could omit that
- your patch stores only synthesized data into 'data' section (-1 idx)
  this could be stored into separate file and get merged with the rest
- new header version would have 'features' section, so the features
  position wouldnt depend on the 'data' end as of now and we could
  easily store after all data is merged:

struct perf_file_header {
        u64                             magic;
        u64                             size;
        u64                             attr_size;
        struct perf_file_section        features;
        DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
};


thoughts?
jirka
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/