| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <013501c9d9cf$161a74a0$0400a8c0@dcccs> Date: Thu, 21 May 2009 06:46:15 +0200 From: "Janos Haar" <janos.haar@...center.hu> To: <paulmck@...ux.vnet.ibm.com> Cc: <linux-kernel@...r.kernel.org> Subject: Re: Fw: RCU detected CPU 1 stall (t=4295904002/751 jiffies) Pid:902, comm: md1_raid5 Paul, Thank you for your attention. Yes, the PC makes 2-3 second "pauses" and drop this message again and again. If i remove the RCU debugging, the message disappears, but the pauses still here, and makes 2-3 load on the idle system. Can i do something? You suggest to use PREEMPT? (This is a server.) Thank you, Janos Haar ----- Original Message ----- From: "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com> To: "Janos Haar" <djani22@...center.hu> Cc: <linux-kernel@...r.kernel.org> Sent: Thursday, May 21, 2009 4:50 AM Subject: Re: Fw: RCU detected CPU 1 stall (t=4295904002/751 jiffies) Pid:902, comm: md1_raid5 > On Wed, May 20, 2009 at 11:46:07AM +0200, Janos Haar wrote: >> Hello list, >> >> Somebody know, what is this? >> May 17 23:12:13 gladiator-afth1 kernel: RCU detected CPU 1 stall >> (t=4295904002/751 jiffies) >> May 17 23:12:13 gladiator-afth1 kernel: Pid: 902, comm: md1_raid5 Not >> tainted 2.6.28.10 #1 >> May 17 23:12:13 gladiator-afth1 kernel: Call Trace: >> May 17 23:12:13 gladiator-afth1 kernel: <IRQ> [<ffffffff80265395>] ? >> get_timestamp+0x9/0xf >> May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff8026854f>] >> __rcu_pending+0x64/0x1e2 >> May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff80268703>] >> rcu_pending+0x36/0x6f >> May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff8023f546>] >> update_process_times+0x37/0x5f >> May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff802504a7>] >> tick_periodic+0x6e/0x7a >> May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff802504d4>] >> tick_handle_periodic+0x21/0x65 >> May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff8021ad60>] >> smp_apic_timer_interrupt+0x8f/0xad >> May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff8020bf1b>] >> apic_timer_interrupt+0x6b/0x70 >> May 17 23:12:13 gladiator-afth1 kernel: <EOI> [<ffffffff80820b8f>] ? >> _spin_unlock_irqrestore+0x13/0x17 > > One of the following functions is looping in the kernel. If you are > running with HZ=250, it has been looping for about three seconds. > Interrupts are enabled, but preemption must be disabled (perhaps due > to !CONFIG_PREEMPT). > > Thanx, Paul > >> May 17 23:12:13 gladiator-afth1 kernel: [<ffffffff80728720>] ? >> bitmap_daemon_work+0x142/0x3b0 >> May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff80724c58>] ? >> md_check_recovery+0x1b/0x45b >> May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff8071976c>] ? >> raid5d+0x5d/0x503 >> May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff80726533>] ? >> md_thread+0xd5/0xed >> May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff80248e4a>] ? >> autoremove_wake_function+0x0/0x38 >> May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff8072645e>] ? >> md_thread+0x0/0xed >> May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff80248d3b>] ? >> kthread+0x49/0x76 >> May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff8020c169>] ? >> child_rip+0xa/0x11 >> May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff80248cf2>] ? >> kthread+0x0/0x76 >> May 17 23:12:18 gladiator-afth1 kernel: [<ffffffff8020c15f>] ? >> child_rip+0x0/0x11 >> Neilbrown from the RAID list suggested to ask someone else...(The mail is >> below.)Thanks,Janos Haar----- Original Message ----- From: "Janos Haar" >> <janos.haar@...center.hu> >> To: "Neil Brown" <neilb@...e.de> >> Cc: <linux-raid@...r.kernel.org> >> Sent: Tuesday, May 19, 2009 12:30 PM >> Subject: Re: RCU detected CPU 1 stall (t=4295904002/751 jiffies) Pid: >> 902, >> comm: md1_raid5 >> >> >>> >>> ----- Original Message ----- From: "Neil Brown" <neilb@...e.de> >>> To: "Janos Haar" <janos.haar@...center.hu> >>> Cc: <linux-raid@...r.kernel.org> >>> Sent: Tuesday, May 19, 2009 3:05 AM >>> Subject: Re: RCU detected CPU 1 stall (t=4295904002/751 jiffies) Pid: >>> 902, >>> comm: md1_raid5 >>> >>> >>>> On Tuesday May 19, janos.haar@...center.hu wrote: >>>>> Hello list, Neil, >>>>> >>>>> Somebody can say something about this issue? >>>>> I am not surprised, if it is hardware related, but this is on a brand >>>>> new >>>>> server, so i am looking for a solution... :-) >>>>> May 17 23:12:13 gladiator-afth1 kernel: RCU detected CPU 1 stall >>>>> (t=4295904002/751 jiffies) >>>> >>>> I have no idea what this means. >>>> I've occasionally seen this sort of message in early boot then the >>>> system continued to work perfectly so I figured it was an early-boot >>>> glitch. I suggest asking someone who understands RCU. >>>> >>>>> >>>>> The entire log is here: >>>>> http://download.netcenter.hu/bughunt/20090518/messages >>>>> >>>>> The system is on the md1, and working, but slowly. >>>> >>>> How slowly? Is the slowness due to disk throughput? >>> >>> No no, this is a fresh and idle server. >>> I have configured the disks, raid on another PC, and when it finished, i >>> have copy up the known good, pre-installed sw pack with old 2.6.18. >>> This pack is good, tested on many times, and this reports too this issue >>> on this machine. (first) >>> I have compiled the 2.6.28.10 on it, it takes about 6 hour! 8-/ >>> But the 2.6.28.10 reports this too. >>> >>> The slowness is not disk based, i think, on idle time if i move the >>> selector line in mc, this stopps too for some seconds or i can't type in >>> bash when this happens, and another one RCU message comes to the log... >>> (It happens periodically, independently of i am doing something or not.) >>> >>> I am not sure, it is raid related or not, but the kernel reports only >>> the >>> md1_raid5 pid, not another one. >>> This is why i am asking here first. >>> >>> Thanks anyway. :-) >>> >>>> Have you tested the individual drives and compared that with the >>>> array? >>> >>> This is a brand new hw, with 4x500GB samsung drive, wich reports no >>> problem at all by smart. >>> >>>> >>>> >>>>> If i left the server for 1 day, it will crash without a saved log. >>>> >>>> This is a concern! It usually points to some sort of hardware >>>> problem, but it is very hard to trace. >>>> Is the power supply rated high enough to support all devices? >>> >>> I am using 550W good quality new PS, and the PC uses only 55-65W, >>> measured. ;-) >>> (1x core2duo, 4x hdd, nothing more interesting) >>> >>>> I cannot think of anything else to suggest .. except start swapping >>>> components until the problem goes away... >>> >>> In this way, i need to start with the motherboard. 8-( >>> >>> Thanks a lot, >>> Janos Haar >>> >>>> >>>> NeilBrown >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in >>> the body of a message to majordomo@...r.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-kernel" >> in >> the body of a message to majordomo@...r.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists