linux-kernel - Re: [BUG] perf report: ordered events and flushing bug

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150312075739.GK943@sejong>
Date:	Thu, 12 Mar 2015 16:57:39 +0900
From:	Namhyung Kim <namhyung@...nel.org>
To:	Stephane Eranian <eranian@...gle.com>
Cc:	LKML <linux-kernel@...r.kernel.org>,
	Arnaldo Carvalho de Melo <acme@...hat.com>,
	Jiri Olsa <jolsa@...hat.com>, David Ahern <dsahern@...il.com>,
	Peter Zijlstra <peterz@...radead.org>,
	"mingo@...e.hu" <mingo@...e.hu>
Subject: Re: [BUG] perf report: ordered events and flushing bug

Hi Stephane,

On Wed, Mar 11, 2015 at 11:32:15PM -0400, Stephane Eranian wrote:
> Hi,
> 
> I am working on the JIT support to improve the flow and have
> perf record inject the MMAPs at the end of the collection. For
> that I piggyback on the buildid pass. To avoid rewriting the entire perf.data
> file, I simply append the the MMAP records at the end of the file. And that
> puts them out-of-order in time systematically. But I thought it would be okay
> because perf report would sort them by timestamps again.
> 
> Well, it does not in all cases! Why?
> 
> Because of the round flushing. Based on how far out-of-order an event is,
> it may not be processed correctly because of round flushing. I believe this
> may only impact MMAP records.  This is a serious issue because
> mmaps drive symbolization of samples. If samples are processed without the
> proper dso mapping, then samples may not be symbolized or may be wrongly
> symbolized.
> 
> So far, the workaround I found was to set the oe->next_flush = 0 for
> the ROUND mode.
> In other words, do not flush anything until FINAL. To me, this is the
> only sensible
> way of avoiding this kind of problems. I am not sure I understand the
> point of flushing
> anyway, except to minimize memory footprint, maybe. But it does not
> work with vastly
> out-of-order mmaps.
> 
> Do you have a better solution?

You may want to see my multi-thread work [1] for this.  It supports
out-of-time processing of samples by splitting metadata and samples
with index.  I think you can just inject the MMAPs to the metadata
area - yes, it requires whole file rewrite anyway but you can skip the
sample data parts.

[1] https://lkml.org/lkml/2015/3/2/988

Thanks,
Namhyung
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/