linux-kernel - Re: rb tree hrtimer lockup bug (found by perf

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <alpine.DEB.2.10.1403181523360.15687@vincent-weaver-1.um.maine.edu>
Date:	Tue, 18 Mar 2014 15:25:55 -0400 (EDT)
From:	Vince Weaver <vincent.weaver@...ne.edu>
To:	Thomas Gleixner <tglx@...utronix.de>
cc:	Vince Weaver <vincent.weaver@...ne.edu>,
	linux-kernel@...r.kernel.org, "H. Peter Anvin" <hpa@...or.com>,
	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...hat.com>
Subject: Re: rb tree hrtimer lockup bug (found by perf_fuzzer)

On Tue, 18 Mar 2014, Thomas Gleixner wrote:

> On Tue, 18 Mar 2014, Vince Weaver wrote:
> 
> > 
> > The perf_fuzzer can quickly cause a machine to lockup with an hrtimer 
> > related rb tree related oops.  I've had a hard time debugging this in any 
> > useful manner, but I can trigger it on both core2 and haswell test systems 
> > on 3.14-rc7.
> > 
> > This involves making a large number of perf_event events of all types and 
> > then forking a lot.
> 
> Can you enable debugobjects please? The should give us an hint what
> corrupts the rbtree.

I enabled debugobjects and then said Y to most of the questions brought up 
by make oldconfig but now the system crashes at boot:

[    3.678040] BUG: unable to handle kernel NULL pointer dereference at 0000000000000018                                                                        
[    3.686776] IP: [<ffffffff8106d7a8>] get_next_timer_interrupt+0x168/0x250    
[    3.694289] PGD 0                                                            
[    3.696642] Oops: 0000 [#1] SMP                                              
[    3.700394] Modules linked in: sg sd_mod sr_mod crc_t10dif crct10dif_common cdrom hid_generic usbhid hid ahci e1000e libahci ehci_pci ptp ehci_hcd xhci_hcd libata pps_core usbcore crc32c_intel scsi_mod usb_common fan thermal thermal_sys 
[    3.725377] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 3.14.0-rc7 #2          
[    3.732217] Hardware name: LENOVO 10AM000AUS/SHARKBAY, BIOS FBKT72AUS 01/26/2014                                                                             
[    3.740296] task: ffff880118e989a0 ti: ffff880118e9e000 task.ti: ffff880118e9e000                                                                            
[    3.748447] RIP: 0010:[<ffffffff8106d7a8>]  [<ffffffff8106d7a8>] get_next_timer_interrupt+0x168/0x250                                                        
[    3.758601] RSP: 0018:ffff880118e9fe58  EFLAGS: 00010017                     
[    3.764413] RAX: 0000000000000000 RBX: 000000013ffede62 RCX: 0000000000000000
[    3.772162] RDX: 0000000000000000 RSI: ffff880118ecd228 RDI: 0000000000fffedf
[    3.779863] RBP: ffff880118e9fea0 R08: 0000000000000001 R09: 0000000000000020
[    3.787553] R10: 000000000000001f R11: ffff880118ecd028 R12: ffff880118ecc000
[    3.795295] R13: 00000000fffede63 R14: ffff880118e9fe60 R15: ffff880118e9fe78
[    3.803003] FS:  0000000000000000(0000) GS:ffff88011ea40000(0000) knlGS:0000000000000000                                                                     
[    3.811760] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033                
[    3.818042] CR2: 0000000000000018 CR3: 000000000180e000 CR4: 00000000001407e0
[    3.825772] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    3.833506] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[    3.841257] Stack:                                                           
[    3.843536]  ffff880118ecd028 ffff880118ecd428 ffff880118ecd828 ffff880118ecdc28                                                                             
[    3.851967]  0000000000000001 00000000cc902a00 ffff88011ea4de00 0000000000000000                                                                             
[    3.860406]  ffff88011ea4eda0 00000000cc91c17c ffffffff810c5525 00000000fffede63                                                                             
[    3.868786] Call Trace:                                                      
[    3.871512]  [<ffffffff810c5525>] ? __tick_nohz_idle_enter+0x2c5/0x460       
[    3.878634]  [<ffffffff810c56f4>] ? tick_nohz_idle_enter+0x34/0x60           
[    3.885374]  [<ffffffff810b089e>] ? cpu_startup_entry+0x3e/0x230             
[    3.891895] Code: 24 18 41 89 fa 41 83 e2 3f 45 89 d1 0f 1f 80 00 00 00 00 49 63 f1 48 c1 e6 04 4c 01 de 48 8b 06 48 39 f0 74 25 66 0f 1f 44 00 00 <f6> 40 18 01 75 11 48 8b 48 10 41 b8 01 00 00 00 48 39 d1 48 0f                          
[    3.918182] RIP  [<ffffffff8106d7a8>] get_next_timer_interrupt+0x168/0x250   
[    3.925697]  RSP <ffff880118e9fe58>                                          
[    3.929514] CR2: 0000000000000018                                            
[    3.933151] ---[ end trace aff36205690b9b9e ]---                             
[    3.938191] Kernel panic - not syncing: Attempted to kill the idle task!     
[    3.945483] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)

this is a haswell system, 3.14-rc7

Vince
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/