linux-kernel - Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <52F84F61.7060103@hitachi.com>
Date:	Mon, 10 Feb 2014 13:02:41 +0900
From:	Masami Hiramatsu <masami.hiramatsu.pt@...achi.com>
To:	Ingo Molnar <mingo@...nel.org>
Cc:	"Frank Ch. Eigler" <fche@...hat.com>, linux-arch@...r.kernel.org,
	Ananth N Mavinakayanahalli <ananth@...ibm.com>,
	Sandeepa Prabhu <sandeepa.prabhu@...aro.org>, x86@...nel.org,
	lkml <linux-kernel@...r.kernel.org>,
	"Steven Rostedt (Red Hat)" <rostedt@...dmis.org>,
	systemtap@...rceware.org, "David S. Miller" <davem@...emloft.net>
Subject: Re: [PATCH -tip v6 00/22] kprobes: introduce NOKPROBE_SYMBOL(), cleanup
 and fixes crash bugs

(2014/02/09 23:37), Ingo Molnar wrote:
> 
> * Masami Hiramatsu <masami.hiramatsu.pt@...achi.com> wrote:
> 
>> I guess the second reason is why the stap takes so long time to set 
>> probes. stap tries to register kprobes without disabled flag, that 
>> means we enables thousands of probes (overheads).
>>
>> So the similar thing happens when we enables events as below;
>>
>>   # for i in /sys/kernel/debug/tracing/events/kprobes/* ; do date; echo 1 > $i; done
>>   Wed Jan 29 10:44:50 UTC 2014
>>   ...
>>
>> I tried it and canceled after 4 min passed. It enabled about 17k 
>> events and slowed down my system very much(I almost got hang check 
>> timer).
> 
> Ok, I guess that's the slowdown bug that Frank reported.
> 
>> I think we should have some performance statistics (hit count?) and 
>> if it goes over a threshold, we should stop enabling other events.
> 
> That really feels like a hack. How about fixing the root cause? Does 
> the enabling of all probes have to be so slow?

When I tried to use perf top, most of the time was consumed in
kprobe_ftrace_handler and optimized_callback, both of them
are the handler of kprobes. Since I just tried on a VM guest and it
didn't support NMI nor PMU, thus I have to use a bare metal machine
for deeper investigation (and I'll do).
And yes, it seems that the performance problem comes from probing
and tracing itself.

Thank you,

-- 
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@...achi.com


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/