| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20090521025037.GD6839@linux.vnet.ibm.com> Date: Wed, 20 May 2009 19:50:37 -0700 From: "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com> To: Janos Haar <djani22@...center.hu> Cc: linux-kernel@...r.kernel.org Subject: Re: Fw: RCU detected CPU 1 stall (t=4295904002/751 jiffies) Pid: 902, comm: md1_raid5 On Wed, May 20, 2009 at 11:46:07AM +0200, Janos Haar wrote: > Hello list, > > Somebody know, what is this? > May 17 23:12:13 gladiator-afth1 kernel: RCU detected CPU 1 stall > (t=4295904002/751 jiffies) > May 17 23:12:13 gladiator-afth1 kernel: Pid: 902, comm: md1_raid5 Not > tainted 2.6.28.10 #1 > May 17 23:12:13 gladiator-afth1 kernel: Call Trace: > May 17 23:12:13 gladiator-afth1 kernel: <IRQ> [<ffffffff80265395>] ? > get_timestamp+0x9/0xf > May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff8026854f>] > __rcu_pending+0x64/0x1e2 > May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff80268703>] > rcu_pending+0x36/0x6f > May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff8023f546>] > update_process_times+0x37/0x5f > May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff802504a7>] > tick_periodic+0x6e/0x7a > May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff802504d4>] > tick_handle_periodic+0x21/0x65 > May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff8021ad60>] > smp_apic_timer_interrupt+0x8f/0xad > May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff8020bf1b>] > apic_timer_interrupt+0x6b/0x70 > May 17 23:12:13 gladiator-afth1 kernel: <EOI> [<ffffffff80820b8f>] ? > _spin_unlock_irqrestore+0x13/0x17 One of the following functions is looping in the kernel. If you are running with HZ=250, it has been looping for about three seconds. Interrupts are enabled, but preemption must be disabled (perhaps due to !CONFIG_PREEMPT). Thanx, Paul > May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff80728720>] ? > bitmap_daemon_work+0x142/0x3b0 > May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff80724c58>] ? > md_check_recovery+0x1b/0x45b > May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff8071976c>] ? > raid5d+0x5d/0x503 > May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff80726533>] ? > md_thread+0xd5/0xed > May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff80248e4a>] ? > autoremove_wake_function+0x0/0x38 > May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff8072645e>] ? > md_thread+0x0/0xed > May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff80248d3b>] ? > kthread+0x49/0x76 > May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff8020c169>] ? > child_rip+0xa/0x11 > May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff80248cf2>] ? > kthread+0x0/0x76 > May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff8020c15f>] ? > child_rip+0x0/0x11 > Neilbrown from the RAID list suggested to ask someone else...(The mail is > below.)Thanks,Janos Haar----- Original Message ----- From: "Janos Haar" > <janos.haar@...center.hu> > To: "Neil Brown" <neilb@...e.de> > Cc: <linux-raid@...r.kernel.org> > Sent: Tuesday, May 19, 2009 12:30 PM > Subject: Re: RCU detected CPU 1 stall (t=4295904002/751 jiffies) Pid: 902, > comm: md1_raid5 > > >> >> ----- Original Message ----- From: "Neil Brown" <neilb@...e.de> >> To: "Janos Haar" <janos.haar@...center.hu> >> Cc: <linux-raid@...r.kernel.org> >> Sent: Tuesday, May 19, 2009 3:05 AM >> Subject: Re: RCU detected CPU 1 stall (t=4295904002/751 jiffies) Pid: 902, >> comm: md1_raid5 >> >> >>> On Tuesday May 19, janos.haar@...center.hu wrote: >>>> Hello list, Neil, >>>> >>>> Somebody can say something about this issue? >>>> I am not surprised, if it is hardware related, but this is on a brand >>>> new >>>> server, so i am looking for a solution... :-) >>>> May 17 23:12:13 gladiator-afth1 kernel: RCU detected CPU 1 stall >>>> (t=4295904002/751 jiffies) >>> >>> I have no idea what this means. >>> I've occasionally seen this sort of message in early boot then the >>> system continued to work perfectly so I figured it was an early-boot >>> glitch. I suggest asking someone who understands RCU. >>> >>>> >>>> The entire log is here: >>>> http://download.netcenter.hu/bughunt/20090518/messages >>>> >>>> The system is on the md1, and working, but slowly. >>> >>> How slowly? Is the slowness due to disk throughput? >> >> No no, this is a fresh and idle server. >> I have configured the disks, raid on another PC, and when it finished, i >> have copy up the known good, pre-installed sw pack with old 2.6.18. >> This pack is good, tested on many times, and this reports too this issue >> on this machine. (first) >> I have compiled the 2.6.28.10 on it, it takes about 6 hour! 8-/ >> But the 2.6.28.10 reports this too. >> >> The slowness is not disk based, i think, on idle time if i move the >> selector line in mc, this stopps too for some seconds or i can't type in >> bash when this happens, and another one RCU message comes to the log... >> (It happens periodically, independently of i am doing something or not.) >> >> I am not sure, it is raid related or not, but the kernel reports only the >> md1_raid5 pid, not another one. >> This is why i am asking here first. >> >> Thanks anyway. :-) >> >>> Have you tested the individual drives and compared that with the >>> array? >> >> This is a brand new hw, with 4x500GB samsung drive, wich reports no >> problem at all by smart. >> >>> >>> >>>> If i left the server for 1 day, it will crash without a saved log. >>> >>> This is a concern! It usually points to some sort of hardware >>> problem, but it is very hard to trace. >>> Is the power supply rated high enough to support all devices? >> >> I am using 550W good quality new PS, and the PC uses only 55-65W, >> measured. ;-) >> (1x core2duo, 4x hdd, nothing more interesting) >> >>> I cannot think of anything else to suggest .. except start swapping >>> components until the problem goes away... >> >> In this way, i need to start with the motherboard. 8-( >> >> Thanks a lot, >> Janos Haar >> >>> >>> NeilBrown >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-raid" in >> the body of a message to majordomo@...r.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@...r.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists