linux-kernel - Re: [tip:perf/core] perf record: Add option to disable collecting build-ids

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20101129151443.GA11102@ghostprotocols.net>
Date:	Mon, 29 Nov 2010 13:14:43 -0200
From:	Arnaldo Carvalho de Melo <acme@...radead.org>
To:	Stephane Eranian <eranian@...gle.com>
Cc:	mingo@...hat.com, hpa@...or.com, paulus@...ba.org,
	linux-kernel@...r.kernel.org, tzanussi@...il.com,
	a.p.zijlstra@...llo.nl, efault@....de, fweisbec@...il.com,
	tglx@...utronix.de
Subject: Re: [tip:perf/core] perf record: Add option to disable collecting
	build-ids

Em Mon, Nov 29, 2010 at 11:22:50AM +0100, Stephane Eranian escreveu:
> Arnaldo,
 
> Indeed, collecting buildids at the end of a perf record session is a
> very time AND memory consuming phase. I have seen system oom because
> of this when running inside cgroup with low memory.
 
> This is easy to reproduce running: perf record -a -- ./Run shell.
> With this, you see perf record reaching a RSS first plateau during the
> active collection of the samples, i.e., dumping the kernel buffer on
> disk. But then, when it calls process_buildids(), it shoots way up in
> memory consumption. For instance, I have seen a perf record running at
> 10MB RSS shooting all the way to 250MB RSS during that phase. At
> first, I thought there was a memory leak somewhere. But after
> instrumenting for a while, nothing really showed up.
 
> I think the problem for RSS is not so much reloading the entire
> buffer, but rather that you are recreating the entire addresses spaces
> of all processes captured. The reason: you only want to save the
> buildids of the DSO for which you had at least one sample. Thus, you
> have to allocate/de-allocate tons of threads and map structures.  I
> wonder if simply looking for MMAP samples and storing the buildids
> (even if they have no samples) wouldn't be more efficient in some
> cases. I believe it would be faster and less memory greedy.

Indeed, probably it is better to do that and only when doing a 'perf
archive' to try to compact the resulting perf.data.tar.bz2 file by
picking just the ones with hits.

I'll look into that, possibly today.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/