linux-kernel - Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <20160630175216.GA8535@linux.vnet.ibm.com>
Date:	Thu, 30 Jun 2016 10:52:16 -0700
From:	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
To:	Ross Green <rgkernel@...il.com>
Cc:	Peter Zijlstra <peterz@...radead.org>,
	Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
	"Chatre, Reinette" <reinette.chatre@...el.com>,
	Jacob Pan <jacob.jun.pan@...ux.intel.com>,
	Josh Triplett <josh@...htriplett.org>,
	John Stultz <john.stultz@...aro.org>,
	Thomas Gleixner <tglx@...utronix.de>,
	lkml <linux-kernel@...r.kernel.org>,
	Ingo Molnar <mingo@...nel.org>,
	Lai Jiangshan <jiangshanlai@...il.com>, dipankar@...ibm.com,
	Andrew Morton <akpm@...ux-foundation.org>,
	rostedt <rostedt@...dmis.org>,
	David Howells <dhowells@...hat.com>,
	Eric Dumazet <edumazet@...gle.com>,
	Darren Hart <dvhart@...ux.intel.com>,
	Frédéric Weisbecker <fweisbec@...il.com>,
	Oleg Nesterov <oleg@...hat.com>,
	pranith kumar <bobby.prani@...il.com>
Subject: Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sat, May 07, 2016 at 08:25:01AM -0700, Paul E. McKenney wrote:
> On Fri, May 06, 2016 at 04:25:16PM +1000, Ross Green wrote:
> > On Sun, Apr 3, 2016 at 6:18 PM, Paul E. McKenney
> > <paulmck@...ux.vnet.ibm.com> wrote:

[ . . . ]

> > Thought i would update a few runs with the linux-4.6-rc kernels.
> > 
> > I have attached log outputs through dmesg showing rcu_preempt stall warnings.
> > 
> > 
> > Thought it might be interesting for someone else to look at.
> > 
> > Currently running linux-4.6-rc6 in testing.
> 
> Thank you for sending these, I will look them over!
> 
> Still working to reproduce this quickly enough to do real debug...  :-/

And Peter Zijlstra's patch looks to have hugely reduced the rate of
occurrence of this bug in my testing:

lkml.kernel.org/r/20160523091907.GD15728@...ktop.ger.corp.intel.com

Almost all of the issues I am seeing now are transient, do not trigger
RCU CPU stall warnings, and would not have been visible to me before
I upgraded my testing scripts and in-kernel code.  In all the tests I
have run with Peter's fix, I have seen only one run with RCU CPU stall
warnings, and all the stalls in that run were transient, unlike those
that I was seeing before his fix.

I will of course be tracking this stuff down, but the low reproduction
rates will make it slow going.

I am guessing that you no longer see this issue?

							Thanx, Paul