linux-kernel - RE: [RFC] perf: probe_finder: continue if atleast one probe point found

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:	Tue, 1 Mar 2016 03:09:57 +0000
From:	平松雅巳 / HIRAMATU，MASAMI 
	<masami.hiramatsu.pt@...achi.com>
To:	"'Arnaldo Carvalho de Melo'" <acme@...nel.org>
CC:	Joel Fernandes <agnel.joel@...il.com>,
	Taeung Song <treeze.taeung@...il.com>,
	Joel Fernandes <joel.opensrc@...il.com>,
	"linux-perf-users@...r.kernel.org" <linux-perf-users@...r.kernel.org>,
	"Linux Kernel Mailing List" <linux-kernel@...r.kernel.org>,
	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...hat.com>,
	Namhyung Kim <namhyung@...nel.org>,
	Jiri Olsa <jolsa@...nel.org>,
	sysp-manager <cti.systems-productivity-manager.ts@...achi.com>
Subject: RE: [RFC] perf: probe_finder: continue if atleast one probe point
 found

Hi,

>From: Arnaldo Carvalho de Melo [mailto:acme@...nel.org]
>
>Em Sun, Feb 28, 2016 at 03:49:44AM -0800, Joel Fernandes escreveu:
>> Taeung,
>>
>> Thanks for that. As this is an RFC, I have added them now to this email
>> thread and when I post the final patch, I'll include them in CC.
>>
>> All,
>> Please provide your comments on my patch.
>
>Masami, are you ok with this patch?

Hmm, I think this is not enough. I concider that the case if users
expect probing on "__malloc_get_state+11", just ignoring it may
confuse or mislead them that perf-probe has a bug not to find it.

IOW, this was by design that perf probe refuses all probe point
if it hits any error.

I think we have 2 options, one is adding an option (or reuse --force)
to ignore error on finding probe points (but this MUST show the error
message by pr_error.) The other is extending perf probe's argument
format so that it can accept local-vars which may not exist, e.g.
'malloc *bytes' or 'malloc {bytes,}' :)

Thank you,


>
>- Arnaldo
>
>> Thanks,
>> Joel
>>
>> On Sun, Feb 28, 2016 at 3:35 AM, Taeung Song <treeze.taeung@...il.com>
>> wrote:
>>
>> > Hi, Joel
>> >
>> > perf subsystem's maintainers are
>> > Arnaldo Carvalho de Melo <acme@...nel.org>
>> > Peter Zijlstra <peterz@...radead.org>
>> > Ingo Molnar <mingo@...hat.com>
>> >
>> > (or Namhyung Kim <namhyung@...nel.org>, Jiri Olsa <jolsa@...nel.org>)
>> >
>> > AFAIK,
>> > When sending some patches, it would be better to add them into To: or Cc:
>> >
>> >
>> > Thanks,
>> > Taeung
>> >
>> >
>> > On 02/28/2016 08:14 PM, Joel Fernandes wrote:
>> >
>> >> Sometimes for inline functions, perf probe can fail such as if an
>> >> arguments are
>> >> requested at probe points. This is probably because when the compiler
>> >> inlines,
>> >> for some instances it optimizes away arguments. Either way, the DWARF has
>> >> missing arguments for certain probe points of inline functions causing
>> >> 'perf
>> >> probe' to fail. I noticed this when probing the C library that ships with
>> >> my
>> >> distribution. With the following patch I am successfully able to record
>> >> kprobe
>> >> events with arguments.
>> >>
>> >> Test
>> >> ----
>> >>   ./perf probe \
>> >>     -k ./vmlinux -s ./ -x /lib/x86_64-linux-gnu/libc.so.6 -a 'malloc
>> >> bytes' -v
>> >>
>> >> Without the patch
>> >> -----------------
>> >> Matched function: __libc_malloc
>> >> found inline addr: 0x831a6
>> >> Probe point found: malloc_atfork+150
>> >> Searching 'bytes' variable in context.
>> >> Converting variable bytes into trace event.
>> >> bytes type is long unsigned int.
>> >> found inline addr: 0x844a0
>> >> Probe point found: __libc_malloc+0
>> >> Searching 'bytes' variable in context.
>> >> Converting variable bytes into trace event.
>> >> bytes type is long unsigned int.
>> >> found inline addr: 0x8463b
>> >> Probe point found: __malloc_get_state+11
>> >> Searching 'bytes' variable in context.
>> >> Failed to find 'bytes' in this function.
>> >> An error occurred in debuginfo analysis (-2).
>> >>    Error: Failed to add events. Reason: No such file or directory (Code:
>> >> -2)
>> >>
>> >> With the patch
>> >> --------------
>> >> Open Debuginfo file: /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.21.so
>> >> Try to find probe point from debuginfo.
>> >> Symbol malloc address found : 844a0
>> >> Matched function: __libc_malloc
>> >> found inline addr: 0x831a6
>> >> Probe point found: malloc_atfork+150
>> >> Searching 'bytes' variable in context.
>> >> Converting variable bytes into trace event.
>> >> bytes type is long unsigned int.
>> >> found inline addr: 0x844a0
>> >> Probe point found: __libc_malloc+0
>> >> Searching 'bytes' variable in context.
>> >> Converting variable bytes into trace event.
>> >> bytes type is long unsigned int.
>> >> found inline addr: 0x8463b
>> >> Probe point found: __malloc_get_state+11
>> >> Searching 'bytes' variable in context.
>> >> Failed to find 'bytes' in this function.
>> >> Probe point error, ignoring. Atleast one probe point found
>> >> found inline addr: 0x84bba
>> >> Probe point found: __libc_realloc+410
>> >> Searching 'bytes' variable in context.
>> >> Failed to find 'bytes' in this function.
>> >> Probe point error, ignoring. Atleast one probe point found
>> >> found inline addr: 0x84d08
>> >> ...
>> >> Probe point error, ignoring. Atleast one probe point found
>> >> Found 3 probe_trace_events.
>> >> Opening /sys/kernel/debug/tracing//uprobe_events write=1
>> >> Writing event: p:probe_libc/malloc /lib/x86_64-linux-gnu/libc-2.21.so:0x831a6
>> >> bytes=%bp:u64
>> >> Writing event: p:probe_libc/malloc_1 /lib/x86_64-linux-gnu/libc-2.21.so:0x844a0
>> >> bytes=%di:u64
>> >> Writing event: p:probe_libc/malloc_2 /lib/x86_64-linux-gnu/libc-2.21.so:0x85af7
>> >> bytes=%di:u64
>> >> Added new events:
>> >>    probe_libc:malloc    (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>> >> with bytes)
>> >>    probe_libc:malloc_1  (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>> >> with bytes)
>> >>    probe_libc:malloc_2  (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>> >> with bytes)
>> >>
>> >> Signed-off-by: Joel Fernandes <agnel.joel@...il.com>
>> >> ---
>> >>   tools/perf/util/probe-finder.c | 4 ++++
>> >>   1 file changed, 4 insertions(+)
>> >>
>> >> diff --git a/tools/perf/util/probe-finder.c
>> >> b/tools/perf/util/probe-finder.c
>> >> index 4ce5c5e..3ac9481 100644
>> >> --- a/tools/perf/util/probe-finder.c
>> >> +++ b/tools/perf/util/probe-finder.c
>> >> @@ -1255,6 +1255,10 @@ end:
>> >>         if (ret) {
>> >>                 clear_probe_trace_event(tev);
>> >>                 tf->ntevs--;
>> >> +               if (tf->ntevs != 0) {
>> >> +                       pr_debug("Ignoring error as atleast one probe
>> >> point found.\n");
>> >> +                       ret = 0;
>> >> +               }
>> >>         }
>> >>         free(args);
>> >>         return ret;
>> >>
>> >>