linux-kernel - Re: [PATCH 05/16] perf tools: Add facility to export data in database-friendly way

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <544A099C.50104@intel.com>
Date:	Fri, 24 Oct 2014 11:11:08 +0300
From:	Adrian Hunter <adrian.hunter@...el.com>
To:	Namhyung Kim <namhyung@...nel.org>
CC:	Arnaldo Carvalho de Melo <acme@...nel.org>,
	Peter Zijlstra <peterz@...radead.org>,
	linux-kernel@...r.kernel.org, David Ahern <dsahern@...il.com>,
	Frederic Weisbecker <fweisbec@...il.com>,
	Jiri Olsa <jolsa@...hat.com>,
	Paul Mackerras <paulus@...ba.org>,
	Stephane Eranian <eranian@...gle.com>
Subject: Re: [PATCH 05/16] perf tools: Add facility to export data in database-friendly
 way

On 24/10/14 09:02, Namhyung Kim wrote:
> On Thu, 23 Oct 2014 13:45:13 +0300, Adrian Hunter wrote:
>> This patch introduces an abstraction for exporting sample
>> data in a database-friendly way.  The abstraction does not
>> implement the actual output.  A subsequent patch takes this
>> facility into use for extending the script interface.
>>
>> The abstraction is needed because static data like symbols,
>> dsos, comms etc need to be exported only once.  That means
>> allocating them a unique identifier and recording it on each
>> structure.  The member 'db_id' is used for that.  'db_id'
>> is just a 64-bit sequence number.
> 
> Can we do it somewhere in a script not in the core code?  I don't feel
> comfortable to add those bits into the core code.  What if we export

Please explain what you mean by "comfortable".

> meta events like task, mmap or some user events to the script optionally
> and let it process the data?

Intel PT decoding can generate a lot of data.  Many millions of samples at
least.

Each sample contains a lot of duplicated information like symbol names, dso
names, comms, etc.  So it is not optimal to export that for every sample.
Even then, each piece of supporting information must be looked up - was it
the same as the last sample, no then look it up, was it found, no then add a
record.  That is very very inefficient.

Exporting each piece of information once, instead of over and over again, is
a reasonable thing to do.

> 
> Thanks,
> Namhyung
> 
> 
>>
>> Exporting centres around the db_export__sample() function
>> which exports the associated data structures if they have
>> not yet been allocated a db_id.
> 
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/