| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CAE40pdfA+QXufOhFSw4T4zm0QPXjymR1cGa_r1tGyABXvPV+rA@mail.gmail.com>
Date: Tue, 21 Jun 2016 11:36:10 -0700
From: Brendan Gregg <brendan.d.gregg@...il.com>
To: Arnaldo Carvalho de Melo <acme@...nel.org>
Cc: Ingo Molnar <mingo@...nel.org>,
LKML <linux-kernel@...r.kernel.org>,
Paolo Bonzini <pbonzini@...hat.com>,
Brendan Gregg <bgregg@...flix.com>,
Arnaldo Carvalho de Melo <acme@...hat.com>
Subject: Re: [PATCH 10/10] perf script: Add stackcollapse.py script
On Mon, Jun 20, 2016 at 3:23 PM, Arnaldo Carvalho de Melo
<acme@...nel.org> wrote:
> From: Paolo Bonzini <pbonzini@...hat.com>
>
> Add stackcollapse.py script as an example of parsing call chains, and
> also of using optparse to access command line options.
>
> The flame graph tools include a set of scripts that parse output from
> various tools (including "perf script"), remove the offsets in the
> function and collapse each stack to a single line. The website also
> says "perf report could have a report style [...] that output folded
> stacks directly, obviating the need for stackcollapse-perf.pl", so here
> it is.
>
> This script is a Python rewrite of stackcollapse-perf.pl, using the perf
> scripting interface to access the perf data directly from Python.
>
> Signed-off-by: Paolo Bonzini <pbonzini@...hat.com>
> Acked-by: Jiri Olsa <jolsa@...nel.org>
> Cc: Brendan Gregg <bgregg@...flix.com>
> Link: http://lkml.kernel.org/r/1460467573-22989-1-git-send-email-pbonzini@redhat.com
> Signed-off-by: Arnaldo Carvalho de Melo <acme@...hat.com>
> ---
> tools/perf/scripts/python/bin/stackcollapse-record | 8 ++
> tools/perf/scripts/python/bin/stackcollapse-report | 3 +
> tools/perf/scripts/python/stackcollapse.py | 127 +++++++++++++++++++++
> 3 files changed, 138 insertions(+)
> create mode 100755 tools/perf/scripts/python/bin/stackcollapse-record
> create mode 100755 tools/perf/scripts/python/bin/stackcollapse-report
> create mode 100755 tools/perf/scripts/python/stackcollapse.py
>
> diff --git a/tools/perf/scripts/python/bin/stackcollapse-record b/tools/perf/scripts/python/bin/stackcollapse-record
> new file mode 100755
> index 000000000000..9d8f9f0f3a17
> --- /dev/null
> +++ b/tools/perf/scripts/python/bin/stackcollapse-record
> @@ -0,0 +1,8 @@
> +#!/bin/sh
> +
> +#
> +# stackcollapse.py can cover all type of perf samples including
> +# the tracepoints, so no special record requirements, just record what
> +# you want to analyze.
> +#
> +perf record "$@"
> diff --git a/tools/perf/scripts/python/bin/stackcollapse-report b/tools/perf/scripts/python/bin/stackcollapse-report
> new file mode 100755
> index 000000000000..356b9656393d
> --- /dev/null
> +++ b/tools/perf/scripts/python/bin/stackcollapse-report
> @@ -0,0 +1,3 @@
> +#!/bin/sh
> +# description: produce callgraphs in short form for scripting use
> +perf script -s "$PERF_EXEC_PATH"/scripts/python/stackcollapse.py -- "$@"
> diff --git a/tools/perf/scripts/python/stackcollapse.py b/tools/perf/scripts/python/stackcollapse.py
> new file mode 100755
> index 000000000000..a2dfcda41ae6
> --- /dev/null
> +++ b/tools/perf/scripts/python/stackcollapse.py
> @@ -0,0 +1,127 @@
> +#!/usr/bin/perl -w
Perl? I guess this line is ignored when invoked.
Rest looks good. Glad it supports tidy_java code, and kernel annotations.
Brendan
> +#
> +# stackcollapse.py - format perf samples with one line per distinct call stack
> +#
> +# This script's output has two space-separated fields. The first is a semicolon
> +# separated stack including the program name (from the "comm" field) and the
> +# function names from the call stack. The second is a count:
> +#
> +# swapper;start_kernel;rest_init;cpu_idle;default_idle;native_safe_halt 2
> +#
> +# The file is sorted according to the first field.
> +#
> +# Input may be created and processed using:
> +#
> +# perf record -a -g -F 99 sleep 60
> +# perf script report stackcollapse > out.stacks-folded
> +#
> +# (perf script record stackcollapse works too).
> +#
> +# Written by Paolo Bonzini <pbonzini@...hat.com>
> +# Based on Brendan Gregg's stackcollapse-perf.pl script.
> +
> +import os
> +import sys
> +from collections import defaultdict
> +from optparse import OptionParser, make_option
> +
> +sys.path.append(os.environ['PERF_EXEC_PATH'] + \
> + '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
> +
> +from perf_trace_context import *
> +from Core import *
> +from EventClass import *
> +
> +# command line parsing
> +
> +option_list = [
> + # formatting options for the bottom entry of the stack
> + make_option("--include-tid", dest="include_tid",
> + action="store_true", default=False,
> + help="include thread id in stack"),
> + make_option("--include-pid", dest="include_pid",
> + action="store_true", default=False,
> + help="include process id in stack"),
> + make_option("--no-comm", dest="include_comm",
> + action="store_false", default=True,
> + help="do not separate stacks according to comm"),
> + make_option("--tidy-java", dest="tidy_java",
> + action="store_true", default=False,
> + help="beautify Java signatures"),
> + make_option("--kernel", dest="annotate_kernel",
> + action="store_true", default=False,
> + help="annotate kernel functions with _[k]")
> +]
> +
> +parser = OptionParser(option_list=option_list)
> +(opts, args) = parser.parse_args()
> +
> +if len(args) != 0:
> + parser.error("unexpected command line argument")
> +if opts.include_tid and not opts.include_comm:
> + parser.error("requesting tid but not comm is invalid")
> +if opts.include_pid and not opts.include_comm:
> + parser.error("requesting pid but not comm is invalid")
> +
> +# event handlers
> +
> +lines = defaultdict(lambda: 0)
> +
> +def process_event(param_dict):
> + def tidy_function_name(sym, dso):
> + if sym is None:
> + sym = '[unknown]'
> +
> + sym = sym.replace(';', ':')
> + if opts.tidy_java:
> + # the original stackcollapse-perf.pl script gives the
> + # example of converting this:
> + # Lorg/mozilla/javascript/MemberBox;.<init>(Ljava/lang/reflect/Method;)V
> + # to this:
> + # org/mozilla/javascript/MemberBox:.init
> + sym = sym.replace('<', '')
> + sym = sym.replace('>', '')
> + if sym[0] == 'L' and sym.find('/'):
> + sym = sym[1:]
> + try:
> + sym = sym[:sym.index('(')]
> + except ValueError:
> + pass
> +
> + if opts.annotate_kernel and dso == '[kernel.kallsyms]':
> + return sym + '_[k]'
> + else:
> + return sym
> +
> + stack = list()
> + if 'callchain' in param_dict:
> + for entry in param_dict['callchain']:
> + entry.setdefault('sym', dict())
> + entry['sym'].setdefault('name', None)
> + entry.setdefault('dso', None)
> + stack.append(tidy_function_name(entry['sym']['name'],
> + entry['dso']))
> + else:
> + param_dict.setdefault('symbol', None)
> + param_dict.setdefault('dso', None)
> + stack.append(tidy_function_name(param_dict['symbol'],
> + param_dict['dso']))
> +
> + if opts.include_comm:
> + comm = param_dict["comm"].replace(' ', '_')
> + sep = "-"
> + if opts.include_pid:
> + comm = comm + sep + str(param_dict['sample']['pid'])
> + sep = "/"
> + if opts.include_tid:
> + comm = comm + sep + str(param_dict['sample']['tid'])
> + stack.append(comm)
> +
> + stack_string = ';'.join(reversed(stack))
> + lines[stack_string] = lines[stack_string] + 1
> +
> +def trace_end():
> + list = lines.keys()
> + list.sort()
> + for stack in list:
> + print "%s %d" % (stack, lines[stack])
Powered by blists - more mailing lists