linux-kernel - Re: [BUG] perf report/annotate: consuming too many file descriptors

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <52FC12E3.9060300@lge.com>
Date:	Thu, 13 Feb 2014 09:33:39 +0900
From:	Namhyung Kim <namhyung.kim@....com>
To:	Stephane Eranian <eranian@...gle.com>
CC:	LKML <linux-kernel@...r.kernel.org>,
	Arnaldo Carvalho de Melo <acme@...hat.com>,
	Jiri Olsa <jolsa@...hat.com>, David Ahern <dsahern@...il.com>,
	Peter Zijlstra <peterz@...radead.org>,
	"mingo@...e.hu" <mingo@...e.hu>
Subject: Re: [BUG] perf report/annotate: consuming too many file descriptors

Hi Stephane,

(I'd be better if you use my korg address).

2014-02-12 PM 11:32, Stephane Eranian wrote:
> Hi,
>
>
> I was testing 3.14-rc2 (tip.git) perf and ran into a serious problem
> with report/annotate on a collection with lots of shared libraries (500).
>
> Perf ran out of file descriptors (ulimit set to 1024). It did not print
> an error message, but simply refused to proceed to objdump.
>
> I ran strace and discovered it was running out of descriptors to
> my big surprise! I narrowed it down with strace -etrace=open.
> I saw an appalling result.
>
> My .so files were opened at least 4 times each, each
> allocated an fd that was kept open, each open incur a read of the ELF
> headers. So each dso was consuming 4 fd.
>
> Why?
>
> Because of what's going on in dso__load() when perf iterates
> over the binary_type_symtab[]. It tries a bunch of types. For
> each match, symsrc_init() is invoked to read the ELF image.
> The fd opened there is kept open (for later loading).
>
> The first problem is why is dso__read_binary_type_filename()
> blindly returning success on many types. For my files, I got matches
> on DEBUGLINK, SYSTEM_DSO_PATH, GUEST_KMODULE,
> SYSTEM_PATH_KMODULE.
>
> I have a tendency to believe that the dso__read_binary_type_filename()
> meaning has been abused to stash things that do not really
> belong there.
>
> Furthermore, I believe many of the type matches do NOT need an ELF
> read and certainly not one that keeps a descriptor opened.
>
> This problem is not just about consuming too many fd, it is also about
> execution overhead. Why read the ELF headers 4 times?
>
> The current situation makes reporting on large collection impossible for
> regular users. I don't quite know how to best address this issue. One way
> I found would be for  dso__read_binary_type_filename() to return a value
> meaning success but no ELF read. Not sure would be correct, though.

It looks like a case of stripped binary or static build.  dso__load() 
looked for both of symtab and dynsym, but it failed.  So it iterated 
through the binary types and continued to fail with fd open. :(

I think we should close/destroy the duplicate symsrcs.  Does attached 
patch fix your problem?

Thanks,
Namhyung


View attachment "destory-unnecessary-symsrc.patch" of type "text/plain" (354 bytes)