linux-kernel - Re: [PATCH v9 00/22] tracing: 'hist' triggers

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAE40pdeforYyCsLJVY4FQh2sN5bfjiSd3Txw1H+9vYQZmqxnoQ@mail.gmail.com>
Date:	Wed, 22 Jul 2015 11:29:47 -0700
From:	Brendan Gregg <brendan.d.gregg@...il.com>
To:	Tom Zanussi <tom.zanussi@...ux.intel.com>
Cc:	Steven Rostedt <rostedt@...dmis.org>, daniel.wagner@...-carit.de,
	Masami Hiramatsu <masami.hiramatsu.pt@...achi.com>,
	Namhyung Kim <namhyung@...nel.org>, josh@...htriplett.org,
	Andi Kleen <andi@...stfloor.org>,
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v9 00/22] tracing: 'hist' triggers

G'Day Tom,

On Thu, Jul 16, 2015 at 10:22 AM, Tom Zanussi
<tom.zanussi@...ux.intel.com> wrote:
>
> This is v9 of the 'hist triggers' patchset.
>
[...]

I've browsed the functionality (sorry, catching up), and it looks like
this will solve a number of common problems. But it seems
tantalizingly close to solving a few more. These may already be on
your future todo list.

A) CPU stack profiling

Kernel stacktrace as a key will be hugely useful; is there a way to
enable this for a sampling profile? (eg, what perf record -F 99 does).
I take CPU profiles daily, and would prefer to aggregate stacks
in-kernel. Also, would like user stacktrace as a key (even if it's
just the hex).

B) Key buckets

Eg, imagine:

echo 'hist:keys=common_pid.execname,count.log2:val=count' >
/sys/kernel/debug/tracing/events/syscalls/sys_enter_read/trigger

to get a log2 bucketized histogram of syscall read request size. Same
for any value where using the value as a key gets too verbose, and you
just want a rough look at the distribution. (Would make it more
readable if it could also be sorted by the log2 value.)

C) Latency as a bucket key

With kprobes, we could then have a log2 histogram of any function call
latency, collected efficiently. (There's already the function timers
in ftrace, which I'm using from function_graph with filters sets to
only match the target function.)

... Those are the other common use cases, that the hist functionality
seemed suited for. Beyond that gets more custom, and we can use eBPF.

Brendan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/