linux-kernel - Re: PROBLEM: 3.0-rc kernels unbootable since -rc3

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20110712164620.GG2326@linux.vnet.ibm.com>
Date:	Tue, 12 Jul 2011 09:46:20 -0700
From:	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
To:	Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>
Cc:	Jeremy Fitzhardinge <jeremy@...p.org>,
	xen-devel@...ts.xensource.com,
	julie Sullivan <kernelmail.jms@...il.com>,
	linux-kernel@...r.kernel.org, chengxu@...ux.vnet.ibm.com,
	kulkarni.ravi4@...il.com
Subject: Re: PROBLEM: 3.0-rc kernels unbootable since -rc3 - under Xen,
 32-bit guest only.

On Tue, Jul 12, 2011 at 12:32:10PM -0400, Konrad Rzeszutek Wilk wrote:
> > > > > http://darnok.org/xen/cpu1.log
> > > > 
> > > > OK, a fair amount of variety, then lots and lots of task_waking_fair(),
> > > > so I still feel good about asking you for the following.
> > > 
> > > But...  But...  But...
> > > 
> > > Just how accurate are these stack traces?  For example, do you have
> > > frame pointers enabled?  If not, could you please enable them?
> 
> Frame pointers are enabled.
> > > 
> > > The reason that I ask is that the wakeme_after_rcu() looks like it is
> > > being invoked from softirq, which would be grossly illegal and could
> > > cause any manner of misbehavior.  Did someone put a synchronize_rcu()
> > > into an RCU callback or something?  Or did I do something really really
> 
> This is a 3.0-rc6 based kernels with the debug patch, the initial
> RCU inhibit patch (where you disable the RCU checking during bootup) and
> that is it.
> 
> What is bizzare is that the soft_irq shows but there is no corresponding
> Xen eventchannel stack trace - there should have been also xen_evtchn_upcall
> (which is the general code that calls the main IRQ handler.. which would make
> the softirq call). This is assuming that the IRQ (timer one) is reguarly dispatching
> (which it looks to be doing). Somehow getting just the softirq by itself is bizzre.
> 
> Perhaps an IPI has been sent that does this. Let me see what a stack
> trace for an IPI looks like.

Thank you for the info!

> > > braindead inside the RCU implementation?
> > > 
> > > (I am looking into this last question, but would appreciate any and all
> > > help with the other questions!)
> > 
> > OK, I was confusing Julie's, Ravi's, and Konrad's situations.
> 
> Do you want me to create a new email thread to keep this one seperate?

Let's please keep everyone on copy.  I bet that these problems are
related.  Plus once we get something that works, it would be good if
everyone could test it.

> > The wakeme_after_rcu() is in fact OK to call from sofirq -- if and
> > only if the scheduler is actually running.  This is what happens if
> > you do a synchronize_rcu() given your CONFIG_TREE_RCU setup -- an RCU
> > callback is posted that, when invoked, awakens the task that invoked
> > synchronize_rcu().
> > 
> > And, based on http://darnok.org/xen/log-rcu-stall, Konrad's system
> > appears to be well past the point where the scheduler is initialized.
> > 
> > So I am coming back around to the loop in task_waking_fair().
> > 
> > Though the patch I sent out earlier might help, for example, if early
> > invocation of RCU callbacks is somehow messing up the scheduler's
> > initialization.
> 
> Ok, let me try it out.

Thank you again!

							Thanx, Paul
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/