linux-kernel - Re: Mysterious CFQ crash and RCU

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20110603153344.GB2333@linux.vnet.ibm.com>
Date:	Fri, 3 Jun 2011 08:33:44 -0700
From:	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
To:	Vivek Goyal <vgoyal@...hat.com>
Cc:	Paul Bolle <pebolle@...cali.nl>, Jens Axboe <jaxboe@...ionio.com>,
	linux kernel mailing list <linux-kernel@...r.kernel.org>
Subject: Re: Mysterious CFQ crash and RCU

On Fri, Jun 03, 2011 at 09:45:14AM -0400, Vivek Goyal wrote:
> On Thu, Jun 02, 2011 at 10:07:24PM -0700, Paul E. McKenney wrote:
> 
> [..]
> > > Thu May 26 10:47:20 CEST 2011
> > > /sys/kernel/debug/rcu/rcugp:
> > > rcu_sched: completed=682249  gpnum=682250
> > 
> > 15 more seconds, a few thousand more grace periods.  About 500 grace
> > periods per second, which is quite reasonable on a single-CPU system.
> 
> PaulB mentioned that crash happened at May 26 10:47:07. I am wondering
> how are we able to sample the data after the crash. I am assuming
> that above data gives information only before crash and does not
> tell us anything about what happened just before crash. What am I missing.
> 
> PaulM, in one of the mails you had mentioned that one could print
> context switch id to make sure we did not block in rcu section. Would
> you have quick pointer where is context switch id stored. May be
> I can write a small patch for PaulB.

>From what I can see, the task_struct nvcsw and nivcsw fields should do
it, though I am not seeing where these are incremented.

So if these don't do what you need, the following (untested but trivial)
patch will provide an rcu_switch_count in the task structure.

							Thanx, Paul

------------------------------------------------------------------------

rcu: add diagnostic per-task context-switch count

Note that this is also incremented by softirqs.

Signed-off-by: Paul E. McKenney <paulmck@...ux.vnet.ibm.com>

diff --git a/include/linux/sched.h b/include/linux/sched.h
index 2a8621c..5ef22e2 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1261,6 +1261,7 @@ struct task_struct {
 #ifdef CONFIG_RCU_BOOST
 	struct rt_mutex *rcu_boost_mutex;
 #endif /* #ifdef CONFIG_RCU_BOOST */
+	unsigned long rcu_switch_count;
 
 #if defined(CONFIG_SCHEDSTATS) || defined(CONFIG_TASK_DELAY_ACCT)
 	struct sched_info sched_info;
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 89419ff..080c6eb 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -154,6 +154,7 @@ void rcu_bh_qs(int cpu)
  */
 void rcu_note_context_switch(int cpu)
 {
+	current->rcu_switch_count++;
 	rcu_sched_qs(cpu);
 	rcu_preempt_note_context_switch(cpu);
 }
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/