linux-kernel - Re: 2.6.33: ftrace triggers soft lockup

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4B94A816.4090608@cn.fujitsu.com>
Date:	Mon, 08 Mar 2010 15:32:38 +0800
From:	Li Zefan <lizf@...fujitsu.com>
To:	Frederic Weisbecker <fweisbec@...il.com>
CC:	Steven Rostedt <rostedt@...dmis.org>,
	Américo Wang <xiyou.wangcong@...il.com>,
	LKML <linux-kernel@...r.kernel.org>, Ingo Molnar <mingo@...e.hu>
Subject: Re: 2.6.33: ftrace triggers soft lockup

>>>>>> I am not sure if this is ftrace's fault, but it is ftrace who triggers
>>>>>> the soft lockup. On my machine, it is pretty easy, just run:
>>>>>>
>>>>>> echo function_graph > current_tracer
>>>>>>
>>>> I can't say that because I didn't try -rc6.
>>>>
>>> Sigh, 2.6.33-rc6 doesn't work, even 2.6.32 doesn't work...
>> So basically you are saying that the function_graph tracer, when enabled
>> has a high overhead? Well, unfortunately, that's expected.
>>
>> The function_graph tracer traces the start and end of every function. It
>> uses the same mechanism as function tracer to trace the start of the
>> function (mcount), but to trace the exit of a function, in the enter of
>> the function it hijacks the return address and replaces it to call a
>> trampoline. This trampoline will do the trace and then jump back to the
>> original return address.
>>
>> Doing this breaks branch prediction in the CPU, as the CPU uses call/ret
>> as part of its branch prediction analysis. So function graph tracing is
>> not just twice as slow as function tracing, it actually has a bigger
>> impact than that.
> 
> 
> It's true it has a high overhead, but not to the point of
> making the whole system unusable. We are supposed to be even
> far from that. I'm currently able to turn on the function graph
> tracer and use firefox without problems. It's just a bit slower
> but it's far from a visible starvation.
> 
> And Li seems to see the same thing.

Yes, and more than this. I can see segmentation fault while testing,
both my testing threads and kernel building threads that are running
at the same time can get segfault.

> For now I can not test, but I will try this week-end.
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/