linux-kernel - Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20151214093841.GB30347@gmail.com>
Date:	Mon, 14 Dec 2015 10:38:41 +0100
From:	Ingo Molnar <mingo@...nel.org>
To:	Peter Zijlstra <peterz@...radead.org>
Cc:	David Ahern <dsahern@...il.com>, Namhyung Kim <namhyung@...il.com>,
	Namhyung Kim <namhyung@...nel.org>,
	Arnaldo Carvalho de Melo <acme@...nel.org>,
	Jiri Olsa <jolsa@...hat.com>,
	LKML <linux-kernel@...r.kernel.org>,
	Frederic Weisbecker <fweisbec@...il.com>,
	Andi Kleen <andi@...stfloor.org>,
	Stephane Eranian <eranian@...gle.com>,
	Adrian Hunter <adrian.hunter@...el.com>
Subject: Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1)


* Peter Zijlstra <peterz@...radead.org> wrote:

> On Fri, Dec 11, 2015 at 08:01:31AM -0700, David Ahern wrote:
> > On 12/11/15 1:11 AM, Ingo Molnar wrote:
> > >
> > >* Namhyung Kim <namhyung@...il.com> wrote:
> > >
> > >>IIRC David said that thread per cpu seems too much especially on a large system
> > >>(like ~1024 cpu). [...]
> > >
> > >Too much in what fashion? For recording I think it's the fastest, most natural
> > >model - anything else will create cache line bounces.
> > 
> > The intrusiveness of perf on the system under observation. I understand
> > there are a lot of factors that go into it.
> 
> So I can see some of that, if every cpu has its own thread then every
> cpu will occasionally schedule that thread. Whereas if there were less,
> you'd not have that.
> 
> Still, I think it makes sense to implement it, we need the multi-file
> option anyway. Once we have that, we can also implement a per-node
> option, which should be a fairly simple hybrid of the two approaches.
> 
> The thing is, perf-record is really struggling on big machines.
> 
> And in an unrelated note, I absolutely detest --buildid being the
> default, it makes perf-record blow chunks.

So I'd absolutely _love_ to split up the singular perf.data into a hierarchy of 
files in a .perf directory, with a structure like this (4-core system):

	.perf/cmdline
	.perf/features
	.perf/evlist
	.perf/ring_buffers/cpu0/raw.trace
	.perf/ring_buffers/cpu1/raw.trace
	.perf/ring_buffers/cpu2/raw.trace
	.perf/ring_buffers/cpu3/raw.trace
	...

I.e. the current single file format of perf.data would be split up into individual 
files. Each CPU would get its own trace file output - any sorting and ordering 
would be done afterwards. 'perf record' itself would never by default have to do 
any of that, it's a pure recording session.

'perf archive' would still create a single file to make transport between machines 
easy.

perf.data.old would be replaced by a .perf.old directory or so.

Debugging would be easier too I think, as there's no complex perf data format 
anymore, it's all in individual (typically text, or binary dump) files in the 
.perf directory.

This would solve all the scalability problems - and would make the format more 
extensible and generally more accessible as well.

What do you think?

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/