linux-kernel - Re: rcu_bh stalls on 3.2.28

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 3 Sep 2012 14:24:41 -0300
From:	Henrique de Moraes Holschuh <hmh@....eng.br>
To:	Michael Wang <wangyun@...ux.vnet.ibm.com>
Cc:	Ben Hutchings <ben@...adent.org.uk>, linux-kernel@...r.kernel.org,
	"paulmck@...ux.vnet.ibm.com" <paulmck@...ux.vnet.ibm.com>
Subject: Re: rcu_bh stalls on 3.2.28

On Mon, 03 Sep 2012, Michael Wang wrote:
> On 09/01/2012 07:02 AM, Henrique de Moraes Holschuh wrote:
> > Just got one of these:
> > 
> > kernel: INFO: rcu_bh detected stall on CPU 2 (t=0 jiffies)
> > kernel: Pid: 0, comm: swapper/2 Not tainted 3.2.28+ #2
> > kernel: Call Trace:
> > kernel: <IRQ>  [<ffffffff810d1609>] __rcu_pending+0x159/0x400
> > kernel: [<ffffffff810d20bb>] rcu_check_callbacks+0x9b/0x120
> > kernel: [<ffffffff81089673>] update_process_times+0x43/0x80
> > kernel: [<ffffffff810a836f>] tick_sched_timer+0x5f/0xb0
> > kernel: [<ffffffff8109c097>] __run_hrtimer.isra.30+0x57/0x100
> > kernel: [<ffffffff8109c8f5>] hrtimer_interrupt+0xe5/0x220
> > kernel: [<ffffffff8104ce14>] smp_apic_timer_interrupt+0x64/0xa0
> > kernel: [<ffffffff8159b5cb>] apic_timer_interrupt+0x6b/0x70
> > kernel: <EOI>  [<ffffffff81315645>] ? intel_idle+0xe5/0x140
> > kernel: [<ffffffff81315623>] ? intel_idle+0xc3/0x140
> > kernel: [<ffffffff814420ee>] cpuidle_idle_call+0x8e/0xf0
> > kernel: [<ffffffff81032425>] cpu_idle+0xa5/0x110
> > kernel: [<ffffffff8158a9ac>] start_secondary+0x1e5/0x1ec
> 
> Hi, Henrique
> 
> rsp->gp_start and rsp->jiffies_stall should already set before we start
> check stall for this gp, but the INFO show that we have a current
> jiffies which bigger then rsp->jiffies_stall but equal to rsp->gp_start,
> really strange...
> 
> Could you please have a try on the latest kernel and confirm whether
> this issue still exist?

It is a production box, it is difficult to run a -rc kernel there.

And the stalls are very rare, too.  That's the only one I got, so at this
point I cannot tell you whether something fixed the problem or not, just try
to give you clues if a stall does happen.

> BTW:
> 	Is this stall info comes from a virtual machine?

No, it runs on baremetal.  The box has one Xeon X5550 processor, 4 cores, 8
threads, and it is allowed to go into C1, C3 and C6 (which it does very very
often).

It might be some sort of race related to SMIs?  The worst-case SMM-induced
delay on this box is quite high (I don't recall if that means 150ms or
150us), as measured by the Intel BITS[1].

[1] http://biosbits.org/

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/