linux-kernel - Re: lots of brief rcu stalls.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20131205164912.GA1585@linux.vnet.ibm.com>
Date:	Thu, 5 Dec 2013 08:49:12 -0800
From:	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
To:	Dave Jones <davej@...hat.com>,
	Linux Kernel <linux-kernel@...r.kernel.org>
Subject: Re: lots of brief rcu stalls.

On Wed, Dec 04, 2013 at 08:22:06PM -0500, Dave Jones wrote:
> On Wed, Dec 04, 2013 at 04:16:14PM -0800, Paul E. McKenney wrote:
>  > On Wed, Dec 04, 2013 at 06:28:38PM -0500, Dave Jones wrote:
>  > > Paul,
>  > > I'm seeing this happening more and more lately...
>  > > 
>  > > [  771.786462] INFO: rcu_preempt detected stalls on CPUs/tasks:
>  > > [  771.786552] 	Tasks blocked on level-0 rcu_node (CPUs 0-3):
>  > > [  771.786574] 	Tasks blocked on level-0 rcu_node (CPUs 0-3):
>  > > [  771.786595] 	(detected by 0, t=6502 jiffies, g=20611, c=20610, q=0)
>  > > [  771.786620] INFO: Stall ended before state dump start
>  > > [  966.724546] INFO: rcu_preempt detected stalls on CPUs/tasks:
>  > > [  966.724854] 	Tasks blocked on level-0 rcu_node (CPUs 0-3):
>  > > [  966.724931] 	Tasks blocked on level-0 rcu_node (CPUs 0-3):
>  > > [  966.725005] 	(detected by 0, t=26007 jiffies, g=20611, c=20610, q=0)
>  > > [  966.725093] INFO: Stall ended before state dump start
>  > > [ 1161.661459] INFO: rcu_preempt detected stalls on CPUs/tasks:
>  > > [ 1161.661763] 	Tasks blocked on level-0 rcu_node (CPUs 0-3):
>  > > [ 1161.661840] 	Tasks blocked on level-0 rcu_node (CPUs 0-3):
>  > > [ 1161.661915] 	(detected by 0, t=45512 jiffies, g=20611, c=20610, q=0)
>  > > [ 1161.662001] INFO: Stall ended before state dump start
>  > > [ 1356.598205] INFO: rcu_preempt detected stalls on CPUs/tasks:
>  > > [ 1356.598513] 	Tasks blocked on level-0 rcu_node (CPUs 0-3):
>  > > [ 1356.598590] 	Tasks blocked on level-0 rcu_node (CPUs 0-3):
>  > > [ 1356.598664] 	(detected by 0, t=65017 jiffies, g=20611, c=20610, q=0)
>  > > [ 1356.598751] INFO: Stall ended before state dump start
>  > > [ 1551.536099] INFO: rcu_preempt detected stalls on CPUs/tasks:
>  > > [ 1551.536408] 	Tasks blocked on level-0 rcu_node (CPUs 0-3):
>  > > [ 1551.536485] 	Tasks blocked on level-0 rcu_node (CPUs 0-3):
>  > > [ 1551.536559] 	(detected by 0, t=84522 jiffies, g=20611, c=20610, q=0)
>  > > [ 1551.536645] INFO: Stall ended before state dump start
>  > > 
>  > > While it's apparently a non-problem, it's pretty noisy.
>  > > Any ideas?
>  > 
>  > Does the following help?
> 
> Seems so so far!

Nice!

But what workload are you running that causes this?  The only way I
can think of to make this happen requires that you have at least one
CPU spending more than 21 seconds running continuously in kernel mode.
Is that what you are doing, or have you come up with some other creative
way for this to happen?  In the latter case, I would really like to know
about it in case it exposes yet another flaw in my RCU design.  ;-)

							Thanx, Paul

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/