linux-kernel - Re: [PATCH 13/16] perf callchain: Switch default to 'graph,0.5,caller'

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 9 Oct 2015 15:10:29 -0700
From:	Brendan Gregg <brendan.d.gregg@...il.com>
To:	Arnaldo Carvalho de Melo <arnaldo.melo@...il.com>
Cc:	Ingo Molnar <mingo@...nel.org>,
	LKML <linux-kernel@...r.kernel.org>,
	Adrian Hunter <adrian.hunter@...el.com>,
	Borislav Petkov <bp@...e.de>,
	Chandler Carruth <chandlerc@...il.com>,
	David Ahern <dsahern@...il.com>,
	Frederic Weisbecker <fweisbec@...il.com>,
	Jiri Olsa <jolsa@...hat.com>,
	Namhyung Kim <namhyung@...nel.org>,
	Stephane Eranian <eranian@...gle.com>,
	Wang Nan <wangnan0@...wei.com>
Subject: Re: [PATCH 13/16] perf callchain: Switch default to 'graph,0.5,caller'

On Fri, Oct 9, 2015 at 2:56 PM, Arnaldo Carvalho de Melo
<arnaldo.melo@...il.com> wrote:
>
> Em Fri, Oct 09, 2015 at 01:34:33PM -0700, Brendan Gregg escreveu:
> > On Mon, Oct 5, 2015 at 2:03 PM, Arnaldo Carvalho de Melo
> > <acme@...nel.org> wrote:
> > >
> > > From: Arnaldo Carvalho de Melo <acme@...hat.com>
> > >
> > > Which is the most common default found in other similar tools.
> >
> > Interactive tools, sure, like the perf report TUI.
>
> > But this also changes the ordering of the non-interactive tools which
> > dump stacks: "perf report -n --stdio" and "perf script". The most
> > common default for dumping stacks is caller. Eg:
>
> And you use that for scripting?

Yes; how I typically CPU profile:

git clone https://github.com/brendangregg/FlameGraph
cd FlameGraph
perf record -F 99 -a -g -- sleep 60
perf script | ./stackcollapse-perf.pl | /flamegraph.pl > flame.svg

Then open flame.svg in a browser and click around. Try it. :)

But it's not just scripting; We often email around "perf report -n
--stdio" output, or attach it to tickets, when working on an issue.
Easier than trying to grab the right TUI screenshot.

>
> > # perf report -n --stdio
> > [...]
> >     16.87%        334           iperf  [kernel.kallsyms]     [k] copy_user_enhanced_fast_string
> >                      |
> >                      --- 0x7f0683ba1ccd
> >                          system_call_fastpath
> >                          sys_write
> >                          vfs_write
> >                          do_sync_write
> >                          sock_aio_write
> >                          do_sock_write.isra.10
> >                          inet_sendmsg
> >                          copy_user_enhanced_fast_string
> > [...]
> >
> > That's upside down. The current default preserves ordering from the
> > informational line onwards:
> >
> > # perf report -n --stdio -g fractal,0.5,callee
> > [...]
> >     16.87%        334           iperf  [kernel.kallsyms]     [k] copy_user_enhanced_fast_string
> >                      |
> >                      --- copy_user_enhanced_fast_string
> >                         |
> >                         |--64.37%-- inet_sendmsg
> >                         |          do_sock_write.isra.10
> >                         |          sock_aio_write
> >                         |          do_sync_write
> >                         |          vfs_write
> >                         |          sys_write
> >                         |          system_call_fastpath
> >                         |          0x7f0683ba1ccd
> >
> > ... Those are just short examples. Another profile I'm working on now
> > gets really messy on "perf report -n --stdio"; eg:
> >
> > perf report -n --stdio -g graph,0.5,caller
> >     94.80%     0.10%             2  iperf     [kernel.vmlinux]    [k]
> > entry_SYSCALL_64_fastpath
> >                |
> >                |--94.70%-- entry_SYSCALL_64_fastpath
> >                |          |
>
> >
> > The current default never gets beyond 5 levels deep. The new default
> > goes to 25 levels. At least with perf report I can override the
> > default using "-g". perf script doesn't support that.
>
> Ok, so changing defaults is not nice, but in this case looked sensible,
> ends up not being for you...

I'm pretty sure this would surprise anyone looking at dumped stacks,
where the convention is caller. pstack, jstack, gdb, systemtap,
dtrace, oops message, etc. I get that we want this for the TUI, but
not dumped stacks.

>
> > Can this patch please preserve the callee ordering for non-interactive
> > output? (perf script, perf report -n --stdio). Thanks,
>
> If this is because you do scripting on it? Wouldn't it be better to not
> depend on defaults, always specify what you want and then the bug would
> be constrained to 'perf script' where we need to provide a way to change
> the default?

Actually, for my flame graphs we should really have perf report have a
--folded output to emit folded output (I emailed perf-users), callee.

For scripting we can always specify -g.

I'm thinking of others who use perf report/script at the CLI, and
expect callee output.

Brendan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/