linux-kernel - Re: BUG: Function graph tracer hang

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.DEB.2.00.0904290818450.8066@gandalf.stny.rr.com>
Date:	Wed, 29 Apr 2009 08:24:36 -0400 (EDT)
From:	Steven Rostedt <rostedt@...dmis.org>
To:	Ingo Molnar <mingo@...e.hu>
cc:	Frederic Weisbecker <fweisbec@...il.com>,
	linux-kernel@...r.kernel.org
Subject: Re: BUG: Function graph tracer hang



On Wed, 29 Apr 2009, Ingo Molnar wrote:

> 
> * Steven Rostedt <rostedt@...dmis.org> wrote:
> 
> > 
> > On Tue, 28 Apr 2009, Frederic Weisbecker wrote:
> > 
> > > 
> > > Note that the branch profiler does that:
> > > 
> > > ______f.miss_hit[______r]++;
> > > 
> > > Which is a read + write on the cacheline.
> > > If each "if" are profiled in the timer interrupt, we can
> > > have the cachelines doing a ping-pong of dirtifying since the above
> > > variable is shared.
> > > 
> > > Then the timer interrupt becomes slower. The function graph tracer itself makes
> > > it slower.
> > > Moreover it is traced itself. So not only the "if" in code are traced, but also
> > > each "if" processed by the function graph tracer on function calls and returns.
> > > 
> > > Which means a fair amount of cacheline dirtifying.
> > > 
> > > Then if the timer interrupt is slowed, and we have a lot of them (1000 Hz),
> > > the system spends all of its time inside it.
> > > 
> > > At least we need the branch tracing to be done per cpu, I guess.
> > 
> > This can be done by basically reimplementing what percpu does. 
> > This is because the data is saved off in its own section at every 
> > if statement. We could copy that section per cpu and add code to 
> > the incrementors to add only to their own CPU buffers.
> 
> Why not just make these variables regular percpu constructs?

Because it is a special section. The variables are embedded into the "if" 
statement. We would need to add it to both the percpu section as well as 
its own section.

One section for the per cpu data, the other to print out the data. We use 
the section to gather all the if statements. Hmm, maybe we can simply make 
the linker script put the if section within the percpu section?

> 
> > This is low in my priority of things to do, but I'll at least add 
> > it to my "to do" list.
> 
> I think we should mark the branch tracer as CONFIG_BROKEN - there's 
> been too many problems with it. Thoughts?

You mean the branch profiler? I've been using it without any issues. The 
issues usually come when we have it combined with other tracers on 
large(?) SMP boxes. I don't hit the cacheline bouncing hang on my 4 cores.

-- Steve

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/