linux-kernel - Re: Re: [PATCH 14/28] ktap: add runtime/kp

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <533A63C8.70800@hitachi.com>
Date:	Tue, 01 Apr 2014 15:59:20 +0900
From:	Masami Hiramatsu <masami.hiramatsu.pt@...achi.com>
To:	Jovi Zhangwei <jovi.zhangwei@...il.com>
Cc:	Ingo Molnar <mingo@...hat.com>,
	Steven Rostedt <rostedt@...dmis.org>,
	LKML <linux-kernel@...r.kernel.org>,
	Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
	Frederic Weisbecker <fweisbec@...il.com>
Subject: Re: Re: [PATCH 14/28] ktap: add runtime/kp_events.[c|h]

(2014/03/31 19:14), Jovi Zhangwei wrote:
> On Mon, Mar 31, 2014 at 5:10 PM, Masami Hiramatsu
> <masami.hiramatsu.pt@...achi.com> wrote:
>> (2014/03/28 22:47), Jovi Zhangwei wrote:
>>> kp_events.c handle ktap events management(registry, destroy, event callback)
>>>
>>> This file is core event management interface between ktap and kernel.
>>>
>>> Exposed functions:
>>> 1). kp_events_init/kp_events_exit
>>>
>>> 2). kp_event_create_kprobe
>>>         create kprobe event, for example:
>>>                 kdebug.kprobe("SyS_futex", function () {})
>>>
>>> 3). kp_event_create_tracepoint
>>>         create tracepoint event, for example"
>>>                 kdebug.tracepoint("sys_futex_enter", function () {})
>>>
>>> 4). kp_event_create
>>>         create perf backend event, for example:
>>>                 trace sched:sched_switch { print(argstr) }
>>>
>>>         It call kernel function 'perf_event_create_kernel_counter' to
>>>         register event(tracepoint/kprobe/uprobe)
>>>
>>> 5). kp_event_getarg
>>>         get argument of event, from arg0 to arg9,
>>>         only can be called in probe context.
>>>                 trace sched:sched_switch { print(arg0, arg1) }
>>>
>>> 6). kp_event_stringify/kp_event_tostr
>>>         stringify argstr, sometimes if store argstr as key to table,
>>>         then it need to stringify firstly, like below:
>>>                 var s={} trace sched:sched_switch { s[argstr] += 1 }
>>>         (This is quite rare usage, but ktap support it)
>>>
>>> Note:
>>> Why ktap support 'kdebug.kprobe' and 'kdebug.tracepoint' when
>>> it already support perf backend event(trace xxx {})?
>>>
>>> Because benchmark shows raw kprobe and tracpoint interface is faster
>>> than perf backed tracing, nearly 10+%, it's more fair to compare
>>> with Systemtap by raw tracing syntax, not perf backend tracing.
>>>
>>
>> Do we really need it just for a +10% performance? I doubt that.
>> I think the benefit point of ktap is "dynamic & simple programmable
>> tracer in kernel", not the good performance at least at this point.
>> Thus I think we should start ktap only with perf backend.
>>
> Yeah, agreed, most people like the perf-backed tracing syntax,
> that raw trace interface is just for benchmark when I wanted to look
> overhead compare with stap, the result is very inspiring, ktap table
> operation overhead is lower than stap.
> 
> On the performance overhead of dynamic tracing tools(ktap/stap/dtrace),
> it's interesting enough that dtrace was used in production many year,
> _but_ IMO the runtime of dtrace is slow after I checked dtrace source
> code :),  system workload does big matter than tracing tool overhead.

Yeah, I see that less overhead is also required especially for enterprise
people. I just doubt that it is solved by ktap itself. Should we improve
perf(or ftrace) to export more effective interfaces for this kind of
tracers?

Thank you,

-- 
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@...achi.com


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/