linux-kernel - Re: [PATCH 0/5] [GIT PULL] updates for tip/tracing/ftrace

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090321200919.GA23992@elte.hu>
Date:	Sat, 21 Mar 2009 21:09:19 +0100
From:	Ingo Molnar <mingo@...e.hu>
To:	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
Cc:	Steven Rostedt <rostedt@...dmis.org>,
	Frederic Weisbecker <fweisbec@...il.com>,
	LKML <linux-kernel@...r.kernel.org>,
	Thomas Gleixner <tglx@...utronix.de>,
	Peter Zijlstra <peterz@...radead.org>
Subject: Re: [PATCH 0/5] [GIT PULL] updates for tip/tracing/ftrace


* Paul E. McKenney <paulmck@...ux.vnet.ibm.com> wrote:

> On Sat, Mar 21, 2009 at 01:25:23PM -0400, Steven Rostedt wrote:
> > 
> > 
> > On Sat, 21 Mar 2009, Ingo Molnar wrote:
> > 
> > > 
> > > * Ingo Molnar <mingo@...e.hu> wrote:
> > > 
> > > > 
> > > > * Steven Rostedt <rostedt@...dmis.org> wrote:
> > > > 
> > > > > 
> > > > > On Fri, 20 Mar 2009, Frederic Weisbecker wrote:
> > > > > > > >  
> > > > > > > >  	VERBOSE_PRINTK_STRING("rcu_torture_reader task started");
> > > > > > > > -	set_user_nice(current, 19);
> > > > > > > > +	set_user_nice(current, -1);
> > > > > > > >  	if (irqreader && cur_ops->irqcapable)
> > > > > > > >  		setup_timer_on_stack(&t, rcu_torture_timer, 0);
> > > > > > > 
> > > > > > > i dont have a reproducer right now. Can you trigger it with latest 
> > > > > > > -tip, which has this commit included:
> > > > > > > 
> > > > > > > 04cb9ac: rcu: rcu_barrier VS cpu_hotplug: Ensure callbacks in dead cpu are migrated to o
> > > > > > > 
> > > > > > > ?
> > > > > > > 
> > > > > > > 	Ingo
> > > > > > 
> > > > > > 
> > > > > > I tested three times the same things but with 04cb9ac and... it didn't triggered
> > > > > > anymore :-)
> > > > > 
> > > > > So lets hope that was the culprit.
> > > > > 
> > > > > Great work Frederic!
> > > > 
> > > > No new lockups of this nature in overnight -tip testing. It's 
> > > > still a bit too early to tell for sure but it's promising ;-)
> > > 
> > > just got a lockup again :-/ It hangs here:
> > > 
> > > calling  init_mmio_trace+0x0/0x12 @ 1
> > > initcall init_mmio_trace+0x0/0x12 returned 0 after 0 usecs
> > > calling  init_graph_trace+0x0/0x12 @ 1
> > > Testing tracer function_graph: 
> > > 
> > > and this time i got good stackdumps as well - see below. Config 
> > > attached.
> > 
> > > CONFIG_CLASSIC_RCU=y
> > 
> > All the crashes you reported only happen with classic RCU.
> > 
> > Paul,
> > 
> > Did anything change recently that could cause this lockup?
> 
> Arjan van de Ven is seeing a problem where a single 
> synchronize_rcu() during bootup is taking a full second, which is 
> currently thought to be due to some drivers spinning in the kernel 
> (Arjan is working on a bootgraph that will hopefully pinpoint the 
> problem: http://lkml.org/lkml/2009/3/21/7).  If the drivers were 
> also instrumented with ftrace, they might (or might not)slow down 
> even further, depending on exactly why they are spinning.

for one of the hung boxes in the past i waited 24 hours but it never 
unwedged itself. The box that hung today is still hanging and the 
RCU stall detector is still busy printing out those backtraces.

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/