linux-kernel - RE: [RFC] perf: probe_finder: continue if atleast one probe point found

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:	Tue, 1 Mar 2016 04:29:18 +0000
From:	平松雅巳 / HIRAMATU，MASAMI 
	<masami.hiramatsu.pt@...achi.com>
To:	"'ltc-kernel@...yrl.intra.hitachi.co.jp'" 
	<ltc-kernel@...yrl.intra.hitachi.co.jp>,
	"'Arnaldo Carvalho de Melo'" <acme@...nel.org>
CC:	Joel Fernandes <agnel.joel@...il.com>,
	Taeung Song <treeze.taeung@...il.com>,
	Joel Fernandes <joel.opensrc@...il.com>,
	"linux-perf-users@...r.kernel.org" <linux-perf-users@...r.kernel.org>,
	"Linux Kernel Mailing List" <linux-kernel@...r.kernel.org>,
	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...hat.com>,
	Namhyung Kim <namhyung@...nel.org>,
	Jiri Olsa <jolsa@...nel.org>,
	sysp-manager <cti.systems-productivity-manager.ts@...achi.com>
Subject: RE: [RFC] perf: probe_finder: continue if atleast one probe point
 found

BTW,

./perf probe  \
  -k ./vmlinux -s ./ -x /lib/x86_64-linux-gnu/libc.so.6 -a 'malloc $params'

might help your case. $params is expanded to function parameters automatically
and if there is no parameters, it is just ignored :)

Thank you,

>From: 平松雅巳 / HIRAMATU，MASAMI [mailto:masami.hiramatsu.pt@...achi.com]
>
>Hi,
>
>>From: Arnaldo Carvalho de Melo [mailto:acme@...nel.org]
>>
>>Em Sun, Feb 28, 2016 at 03:49:44AM -0800, Joel Fernandes escreveu:
>>> Taeung,
>>>
>>> Thanks for that. As this is an RFC, I have added them now to this email
>>> thread and when I post the final patch, I'll include them in CC.
>>>
>>> All,
>>> Please provide your comments on my patch.
>>
>>Masami, are you ok with this patch?
>
>Hmm, I think this is not enough. I concider that the case if users
>expect probing on "__malloc_get_state+11", just ignoring it may
>confuse or mislead them that perf-probe has a bug not to find it.
>
>IOW, this was by design that perf probe refuses all probe point
>if it hits any error.
>
>I think we have 2 options, one is adding an option (or reuse --force)
>to ignore error on finding probe points (but this MUST show the error
>message by pr_error.) The other is extending perf probe's argument
>format so that it can accept local-vars which may not exist, e.g.
>'malloc *bytes' or 'malloc {bytes,}' :)
>
>Thank you,
>
>
>>
>>- Arnaldo
>>
>>> Thanks,
>>> Joel
>>>
>>> On Sun, Feb 28, 2016 at 3:35 AM, Taeung Song <treeze.taeung@...il.com>
>>> wrote:
>>>
>>> > Hi, Joel
>>> >
>>> > perf subsystem's maintainers are
>>> > Arnaldo Carvalho de Melo <acme@...nel.org>
>>> > Peter Zijlstra <peterz@...radead.org>
>>> > Ingo Molnar <mingo@...hat.com>
>>> >
>>> > (or Namhyung Kim <namhyung@...nel.org>, Jiri Olsa <jolsa@...nel.org>)
>>> >
>>> > AFAIK,
>>> > When sending some patches, it would be better to add them into To: or Cc:
>>> >
>>> >
>>> > Thanks,
>>> > Taeung
>>> >
>>> >
>>> > On 02/28/2016 08:14 PM, Joel Fernandes wrote:
>>> >
>>> >> Sometimes for inline functions, perf probe can fail such as if an
>>> >> arguments are
>>> >> requested at probe points. This is probably because when the compiler
>>> >> inlines,
>>> >> for some instances it optimizes away arguments. Either way, the DWARF has
>>> >> missing arguments for certain probe points of inline functions causing
>>> >> 'perf
>>> >> probe' to fail. I noticed this when probing the C library that ships with
>>> >> my
>>> >> distribution. With the following patch I am successfully able to record
>>> >> kprobe
>>> >> events with arguments.
>>> >>
>>> >> Test
>>> >> ----
>>> >>   ./perf probe \
>>> >>     -k ./vmlinux -s ./ -x /lib/x86_64-linux-gnu/libc.so.6 -a 'malloc
>>> >> bytes' -v
>>> >>
>>> >> Without the patch
>>> >> -----------------
>>> >> Matched function: __libc_malloc
>>> >> found inline addr: 0x831a6
>>> >> Probe point found: malloc_atfork+150
>>> >> Searching 'bytes' variable in context.
>>> >> Converting variable bytes into trace event.
>>> >> bytes type is long unsigned int.
>>> >> found inline addr: 0x844a0
>>> >> Probe point found: __libc_malloc+0
>>> >> Searching 'bytes' variable in context.
>>> >> Converting variable bytes into trace event.
>>> >> bytes type is long unsigned int.
>>> >> found inline addr: 0x8463b
>>> >> Probe point found: __malloc_get_state+11
>>> >> Searching 'bytes' variable in context.
>>> >> Failed to find 'bytes' in this function.
>>> >> An error occurred in debuginfo analysis (-2).
>>> >>    Error: Failed to add events. Reason: No such file or directory (Code:
>>> >> -2)
>>> >>
>>> >> With the patch
>>> >> --------------
>>> >> Open Debuginfo file: /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.21.so
>>> >> Try to find probe point from debuginfo.
>>> >> Symbol malloc address found : 844a0
>>> >> Matched function: __libc_malloc
>>> >> found inline addr: 0x831a6
>>> >> Probe point found: malloc_atfork+150
>>> >> Searching 'bytes' variable in context.
>>> >> Converting variable bytes into trace event.
>>> >> bytes type is long unsigned int.
>>> >> found inline addr: 0x844a0
>>> >> Probe point found: __libc_malloc+0
>>> >> Searching 'bytes' variable in context.
>>> >> Converting variable bytes into trace event.
>>> >> bytes type is long unsigned int.
>>> >> found inline addr: 0x8463b
>>> >> Probe point found: __malloc_get_state+11
>>> >> Searching 'bytes' variable in context.
>>> >> Failed to find 'bytes' in this function.
>>> >> Probe point error, ignoring. Atleast one probe point found
>>> >> found inline addr: 0x84bba
>>> >> Probe point found: __libc_realloc+410
>>> >> Searching 'bytes' variable in context.
>>> >> Failed to find 'bytes' in this function.
>>> >> Probe point error, ignoring. Atleast one probe point found
>>> >> found inline addr: 0x84d08
>>> >> ...
>>> >> Probe point error, ignoring. Atleast one probe point found
>>> >> Found 3 probe_trace_events.
>>> >> Opening /sys/kernel/debug/tracing//uprobe_events write=1
>>> >> Writing event: p:probe_libc/malloc /lib/x86_64-linux-gnu/libc-2.21.so:0x831a6
>>> >> bytes=%bp:u64
>>> >> Writing event: p:probe_libc/malloc_1 /lib/x86_64-linux-gnu/libc-2.21.so:0x844a0
>>> >> bytes=%di:u64
>>> >> Writing event: p:probe_libc/malloc_2 /lib/x86_64-linux-gnu/libc-2.21.so:0x85af7
>>> >> bytes=%di:u64
>>> >> Added new events:
>>> >>    probe_libc:malloc    (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>>> >> with bytes)
>>> >>    probe_libc:malloc_1  (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>>> >> with bytes)
>>> >>    probe_libc:malloc_2  (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>>> >> with bytes)
>>> >>
>>> >> Signed-off-by: Joel Fernandes <agnel.joel@...il.com>
>>> >> ---
>>> >>   tools/perf/util/probe-finder.c | 4 ++++
>>> >>   1 file changed, 4 insertions(+)
>>> >>
>>> >> diff --git a/tools/perf/util/probe-finder.c
>>> >> b/tools/perf/util/probe-finder.c
>>> >> index 4ce5c5e..3ac9481 100644
>>> >> --- a/tools/perf/util/probe-finder.c
>>> >> +++ b/tools/perf/util/probe-finder.c
>>> >> @@ -1255,6 +1255,10 @@ end:
>>> >>         if (ret) {
>>> >>                 clear_probe_trace_event(tev);
>>> >>                 tf->ntevs--;
>>> >> +               if (tf->ntevs != 0) {
>>> >> +                       pr_debug("Ignoring error as atleast one probe
>>> >> point found.\n");
>>> >> +                       ret = 0;
>>> >> +               }
>>> >>         }
>>> >>         free(args);
>>> >>         return ret;
>>> >>
>>> >>