linux-kernel - Re: [PATCHv3 00/19] perf tools: Factor ordered samples queue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 21 Jul 2014 10:02:19 +0200
From:	Jiri Olsa <jolsa@...hat.com>
To:	Adrian Hunter <adrian.hunter@...el.com>
Cc:	Jiri Olsa <jolsa@...nel.org>, linux-kernel@...r.kernel.org,
	Arnaldo Carvalho de Melo <acme@...nel.org>,
	Corey Ashford <cjashfor@...ux.vnet.ibm.com>,
	David Ahern <dsahern@...il.com>,
	Frederic Weisbecker <fweisbec@...il.com>,
	Ingo Molnar <mingo@...nel.org>,
	Jean Pihet <jean.pihet@...aro.org>,
	Namhyung Kim <namhyung@...nel.org>,
	Paul Mackerras <paulus@...ba.org>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>
Subject: Re: [PATCHv3 00/19] perf tools: Factor ordered samples queue

On Mon, Jul 21, 2014 at 09:43:58AM +0300, Adrian Hunter wrote:
> On 07/21/2014 12:55 AM, Jiri Olsa wrote:
> > hi,
> > this patchset factors session's ordered samples queue,
> > and allows to limit the size of this queue.
> > 
> > v3 changes:
> >   - rebased to latest tip/perf/core
> >   - add comment for WARN in patch 8 (David)
> >   - added ordered-events debug variable (David)
> >   - renamed ordered_events_(get|put) to ordered_events_(new|delete)
> >   - renamed struct ordered_events_queue to struct ordered_events
> > 
> > v2 changes:
> >   - several small changes for review comments (Namhyung)
> > 
> > 
> > The report command queues events till any of following
> > conditions is reached:
> >   - PERF_RECORD_FINISHED_ROUND event is processed
> >   - end of the file is reached
> > 
> > Any of above conditions will force the queue to flush some
> > events while keeping all allocated memory for next events.
> > 
> > If PERF_RECORD_FINISHED_ROUND is missing the queue will
> 
> Why is it missing?

it's stored only for tracepoints now patch 17 fixies that

> 
> > allocate memory for every single event in the perf.data.
> > This could lead to enormous memory consuption and speed
> > degradation of report command for huge perf.data files.
> > 
> > With the quue allocation limit of 100 MB, I've got around
> > 15% speedup on reporting of ~10GB perf.data file.
> 
> How do you know the results are still valid?  Wouldn't it
> be better to wait that extra 15% and know that the data has
> been processed correctly?

The HALF flush could cause the out of order message
(which I get occasionaly anyway). Patch 19 allows
out of order events after HALF flush.

The main reason for me was to control the memory allocation,
which could get enormous without ROUND events being stored.
The 100MB queue limit seems to be enough not to hit out of
order event due to the HALF flush.

jirka
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/