lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date:	Sun, 9 Sep 2012 15:59:27 +0800
From:	Fengguang Wu <fengguang.wu@...el.com>
To:	John Stultz <john.stultz@...aro.org>
Cc:	Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar <mingo@...e.hu>,
	LKML <linux-kernel@...r.kernel.org>
Subject: BUG: soft lockup - CPU#1 stuck for 1024s!

Hi John,

The below lockup warning pops up very occasionally in kvm guest
kernels and it's bisected down to

        commit 2a8c0883c3cfffcc148ea606e2a4e7453cd75e73
        Author: John Stultz <john.stultz@...aro.org>
        Date:   Fri Jul 13 01:21:56 2012 -0400

            time: Move xtime_nsec adjustment underflow handling timekeeping_adjust

[    8.340327] Freeing unused kernel memory: 460k freed
[ 1107.515496] [sched_delayed] sched: RT throttling activated
[ 1107.516439] BUG: soft lockup - CPU#1 stuck for 1024s! [swapper/1:0]
[ 1107.516439] irq event stamp: 13644
[ 1107.516439] hardirqs last  enabled at (13641): [<c167d0c5>] restore_all_notrace+0x0/0x18
[ 1107.516439] hardirqs last disabled at (13642): [<c167d3cf>] apic_timer_interrupt+0x2f/0x40
[ 1107.516439] softirqs last  enabled at (13644): [<c102e6ae>] _local_bh_enable+0xd/0xf
[ 1107.516439] softirqs last disabled at (13643): [<c102eb79>] irq_enter+0x2c/0x4d
[ 1107.516439] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-00023-g8a13a4c #66  
[ 1107.516439] EIP: 0060:[<c101da64>] EFLAGS: 00000286 CPU: 1
[ 1107.516439] EIP is at native_safe_halt+0x5/0x7
[ 1107.516439] EAX: ce06a260 EBX: ffffffff ECX: ce06a260 EDX: 00000007
[ 1107.516439] ESI: cfff9e80 EDI: cfff9e70 EBP: ce071f6c ESP: ce071f6c
[ 1107.516439]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1107.516439] CR0: 8005003b CR2: b766c130 CR3: 0e83b000 CR4: 00000690
[ 1107.516439] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1107.516439] DR6: ffff0ff0 DR7: 00000400
[ 1107.516439] Process swapper/1 (pid: 0, ti=ce070000 task=ce06a260 task.ti=ce070000)
[ 1107.516439] Stack:
[ 1107.516439]  ce071f74 c1008a89 ce071f88 c10092c6 cde0a192 dc4426ba 168f476e ce071fb4
[ 1107.516439]  c166d17b 00000000 00000000 00000000 00000000 86be5003 168f476e 01000800
[ 1107.516439]  0000d248 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 1107.516439] Call Trace:
[ 1107.516439]  [<c1008a89>] default_idle+0x2f/0x55
[ 1107.516439]  [<c10092c6>] cpu_idle+0xc7/0xf9
[ 1107.516439]  [<c166d17b>] start_secondary+0x296/0x29e
[ 1107.516439] Code: 0f 22 e0 5d c3 55 89 e5 0f 09 5d c3 55 89 e5 9c 58 5d c3 55 89 e5 50 9d 5d c3 55 89 e5 fa 5d c3 55 89 e5 fb 5d c3 55 89 e5 fb f4 <5d> c3 55 89 e5 f4 5d c3 55 89 e5 57 56 53 53 89 c7 89 55 f0 89

Below are all the oops messages during the bisect.

[    4.459980] Freeing unused kernel memory: 460k freed
[ 1103.514811] BUG: soft lockup - CPU#1 stuck for 1024s! [swapper/1:0]
[ 1103.514811] irq event stamp: 7390
[ 1103.514811] hardirqs last  enabled at (7387): [<c167f085>] restore_all_notrace+0x0/0x18
[ 1103.514811] hardirqs last disabled at (7388): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1103.514811] softirqs last  enabled at (7390): [<c102e637>] _local_bh_enable+0xd/0xf
[ 1103.514811] softirqs last disabled at (7389): [<c102eac7>] irq_enter+0x2c/0x4d
[ 1103.514811] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1103.514811] EIP: 0060:[<c101da32>] EFLAGS: 00000286 CPU: 1
[ 1103.514811] EIP is at native_safe_halt+0x5/0x7
[ 1103.523837] [sched_delayed] sched: RT throttling activated
[ 1103.514811] EAX: ce06a260 EBX: ffffffff ECX: ce06a260 EDX: 00000007
[ 1103.514811] ESI: c19716f0 EDI: ffff8b9d EBP: ce071f68 ESP: ce071f68
[ 1103.514811]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.514811] CR0: 8005003b CR2: b767dfb8 CR3: 0ee9c000 CR4: 00000690
[ 1103.514811] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.514811] DR6: ffff0ff0 DR7: 00000400
[ 1103.514811] Process swapper/1 (pid: 0, ti=ce070000 task=ce06a260 task.ti=ce070000)
[ 1103.514811] Stack:
[ 1103.514811]  ce071f70 c1008a09 ce071f88 c1009250 a987df14 b7eb6434 f2365b03 cfff9fa0
[ 1103.514811]  ce071fb4 c166ed10 00000000 00000000 00000000 00000000 626563a0 f2365b03
[ 1103.514811]  01000800 0000d248 00000000 00000000 00000000 00000000 00000000 00000000
[ 1103.514811] Call Trace:
[ 1103.514811]  [<c1008a09>] default_idle+0x2f/0x55
[ 1103.514811]  [<c1009250>] cpu_idle+0xc9/0xfb
[ 1103.514811]  [<c166ed10>] start_secondary+0x292/0x29a
[ 1103.514811] Code: 0f 22 e0 5d c3 55 89 e5 0f 09 5d c3 55 89 e5 9c 58 5d c3 55 89 e5 50 9d 5d c3 55 89 e5 fa 5d c3 55 89 e5 fb 5d c3 55 89 e5 fb f4 <5d> c3 55 89 e5 f4 5d c3 55 89 e5 57 56 53 53 89 c6 89 55 f0 89
[ 1103.514811] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.514811] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1103.514811] Call Trace:
[ 1103.514811]  [<c1673678>] panic+0x97/0x1aa
[ 1103.514811]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1103.514811]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.514811]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1103.514811]  [<c103205a>] run_local_timers+0x8/0x14
[ 1103.514811]  [<c103208a>] update_process_times+0x24/0x58
[ 1103.514811]  [<c106276e>] tick_periodic+0x66/0x72
[ 1103.514811]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1103.514811]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.514811]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.514811]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1103.514811]  [<c101da32>] ? native_safe_halt+0x5/0x7
[ 1103.514811]  [<c1008a09>] default_idle+0x2f/0x55
[ 1103.514811]  [<c1009250>] cpu_idle+0xc9/0xfb
[ 1103.514811]  [<c166ed10>] start_secondary+0x292/0x29a
[ 1103.514811] Rebooting in 10 seconds..
--
[    5.274691] Freeing unused kernel memory: 460k freed
[ 1104.516417] BUG: soft lockup - CPU#1 stuck for 1024s! [umount:132]
[ 1104.516417] irq event stamp: 3118
[ 1104.516417] hardirqs last  enabled at (3117): [<c167e841>] _raw_spin_unlock_irqrestore+0x42/0x63
[ 1104.516417] hardirqs last disabled at (3118): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.516417] softirqs last  enabled at (0): [<c1026857>] copy_process+0x465/0xf8a
[ 1104.516417] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1104.516417] Pid: 132, comm: umount Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.516417] EIP: 0060:[<c10ad429>] EFLAGS: 00000202 CPU: 1
[ 1104.522300] [sched_delayed] sched: RT throttling activated
[ 1104.516417] EIP is at unmap_single_vma+0x206/0x473
[ 1104.516417] EAX: ce8afec0 EBX: ce68bde0 ECX: 0a66f025 EDX: b76d7000
[ 1104.516417] ESI: ce89eb5c EDI: 00000000 EBP: ce8afe98 ESP: ce8afe34
[ 1104.516417]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.516417] CR0: 80050033 CR2: b75bbfb8 CR3: 0e897000 CR4: 00000690
[ 1104.516417] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.516417] DR6: ffff0ff0 DR7: 00000400
[ 1104.516417] Process umount (pid: 132, ti=ce8ae000 task=cf8a9130 task.ti=ce8ae000)
[ 1104.516417] Stack:
[ 1104.516417]  00000002 00000203 006f0203 00000000 cf8a95a4 b76fbfff ce148f34 ce89eb58
[ 1104.516417]  0a66f025 ce897b74 ce897b74 ce148f00 00000000 ce8afec0 b76fc000 b76fc000
[ 1104.516417]  ce8ad1b8 b76d7000 ce8afe80 ffffffff 00000000 00000000 ce8ad1b8 ce8afec0
[ 1104.516417] Call Trace:
[ 1104.516417]  [<c10adbca>] unmap_vmas+0x2a/0x40
[ 1104.516417]  [<c10b19d9>] exit_mmap+0x76/0xe4
[ 1104.516417]  [<c1025e1a>] mmput+0x42/0xa7
[ 1104.516417]  [<c102cab2>] exit_mm+0xde/0xe6
[ 1104.516417]  [<c102ce13>] do_exit+0x1d0/0x72c
[ 1104.516417]  [<c10c81e1>] ? vfs_write+0x105/0x10f
[ 1104.516417]  [<c102d3f6>] do_group_exit+0x5e/0x81
[ 1104.516417]  [<c102d42c>] sys_exit_group+0x13/0x13
[ 1104.516417]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1104.516417] Code: 89 d0 8d b6 00 00 00 00 89 45 bc 83 3d b8 1f 91 c1 00 75 02 0f 0b 8b 45 c8 8b 55 e0 89 f1 8d b6 00 00 00 00 8b 45 d0 80 48 0c 01 <85> db 0f 84 8e 01 00 00 80 7d cc 00 74 5a 8b 0f 85 c9 74 54 8b
[ 1104.516417] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.516417] Pid: 132, comm: umount Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.516417] Call Trace:
[ 1104.516417]  [<c1673678>] panic+0x97/0x1aa
[ 1104.516417]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.516417]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.516417]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.516417]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.516417]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.516417]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.516417]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.516417]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.516417]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.516417]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.516417]  [<c10b007b>] ? munlock_vma_pages_range+0x34/0xa1
[ 1104.516417]  [<c10ad429>] ? unmap_single_vma+0x206/0x473
[ 1104.516417]  [<c10adbca>] unmap_vmas+0x2a/0x40
[ 1104.516417]  [<c10b19d9>] exit_mmap+0x76/0xe4
[ 1104.516417]  [<c1025e1a>] mmput+0x42/0xa7
[ 1104.516417]  [<c102cab2>] exit_mm+0xde/0xe6
[ 1104.516417]  [<c102ce13>] do_exit+0x1d0/0x72c
[ 1104.516417]  [<c10c81e1>] ? vfs_write+0x105/0x10f
[ 1104.516417]  [<c102d3f6>] do_group_exit+0x5e/0x81
[ 1104.516417]  [<c102d42c>] sys_exit_group+0x13/0x13
[ 1104.516417]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1104.516417] Rebooting in 10 seconds..
--
[    5.814040] Freeing unused kernel memory: 460k freed
[ 1104.524796] BUG: soft lockup - CPU#1 stuck for 1023s! [sh:130]
[ 1104.524796] irq event stamp: 1712
[ 1104.524796] hardirqs last  enabled at (1711): [<c101e5d2>] do_page_fault+0x98/0x36e
[ 1104.524796] hardirqs last disabled at (1712): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.524796] softirqs last  enabled at (0): [<c1026857>] copy_process+0x465/0xf8a
[ 1104.524796] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1104.524796] Pid: 130, comm: sh Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.524796] EIP: 0060:[<c10675e9>] EFLAGS: 00000282 CPU: 1
[ 1104.524796] EIP is at lock_acquire+0xb4/0xc0
[ 1104.524796] EAX: 00000282 EBX: cf89d5f0 ECX: 006f4012 EDX: 006f4012
[ 1104.524796] ESI: 00000000 EDI: c19136b8 EBP: ce8bdedc ESP: ce8bdeb0
[ 1104.524796]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.524796] CR0: 8005003b CR2: b75df2c0 CR3: 0e872000 CR4: 00000690
[ 1104.524796] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.524796] DR6: ffff0ff0 DR7: 00000400
[ 1104.524796] Process sh (pid: 130, ti=ce8bc000 task=cf89d5f0 task.ti=ce8bc000)
[ 1104.524796] Stack:
[ 1104.524796]  00000002 00000001 00000000 00000000 c10c2db3 00000000 00000000 00000282
[ 1104.524796]  ce85a780 0000000b ce85a780 ce8bdefc c10c2de0 00000002 00000001 00000000
[ 1104.524796]  c10c2db3 ce8909f8 b75df2c0 ce8bdf20 c10aef10 ce8bdf20 c10432c5 00000001
[ 1104.524796] Call Trace:
[ 1104.524796]  [<c10c2db3>] ? mem_cgroup_count_vm_event+0x1e/0x163
[ 1104.524796]  [<c10c2de0>] mem_cgroup_count_vm_event+0x4b/0x163
[ 1104.531682] [sched_delayed] sched: RT throttling activated
[ 1104.524796]  [<c10c2db3>] ? mem_cgroup_count_vm_event+0x1e/0x163
[ 1104.524796]  [<c10aef10>] handle_mm_fault+0x2f/0xb0
[ 1104.524796]  [<c10432c5>] ? down_read_trylock+0x51/0x59
[ 1104.524796]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.524796]  [<c101e870>] do_page_fault+0x336/0x36e
[ 1104.524796]  [<c10c939f>] ? fget_light+0xf7/0x334
[ 1104.524796]  [<c12b3908>] ? copy_to_user+0x3f/0x46
[ 1104.524796]  [<c167f085>] ? restore_all+0xf/0xf
[ 1104.524796]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.524796]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.524796]  [<c101d8ab>] do_async_page_fault+0x23/0x5f
[ 1104.524796]  [<c167f5cc>] error_code+0x6c/0x80
[ 1104.524796] Code: 45 08 89 04 24 8b 4d ec 89 f2 89 f8 e8 57 e7 ff ff c7 83 58 04 00 00 00 00 00 00 83 3d 8c 20 91 c1 00 75 02 0f 0b 8b 45 f0 50 9d <8d> 74 26 00 83 c4 20 5b 5e 5f 5d c3 55 89 e5 57 56 53 83 ec 1c
[ 1104.524796] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.524796] Pid: 130, comm: sh Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.524796] Call Trace:
[ 1104.524796]  [<c1673678>] panic+0x97/0x1aa
[ 1104.524796]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.524796]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.524796]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.524796]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.524796]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.524796]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.524796]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.524796]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.524796]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.524796]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.524796]  [<c10675e9>] ? lock_acquire+0xb4/0xc0
[ 1104.524796]  [<c10c2db3>] ? mem_cgroup_count_vm_event+0x1e/0x163
[ 1104.524796]  [<c10c2de0>] mem_cgroup_count_vm_event+0x4b/0x163
[ 1104.524796]  [<c10c2db3>] ? mem_cgroup_count_vm_event+0x1e/0x163
[ 1104.524796]  [<c10aef10>] handle_mm_fault+0x2f/0xb0
[ 1104.524796]  [<c10432c5>] ? down_read_trylock+0x51/0x59
[ 1104.524796]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.524796]  [<c101e870>] do_page_fault+0x336/0x36e
[ 1104.524796]  [<c10c939f>] ? fget_light+0xf7/0x334
[ 1104.524796]  [<c12b3908>] ? copy_to_user+0x3f/0x46
[ 1104.524796]  [<c167f085>] ? restore_all+0xf/0xf
[ 1104.524796]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.524796]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.524796]  [<c101d8ab>] do_async_page_fault+0x23/0x5f
[ 1104.524796]  [<c167f5cc>] error_code+0x6c/0x80
[ 1104.524796] Rebooting in 10 seconds..
--
[ 1103.513737] [sched_delayed] sched: RT throttling activated
[ 1103.513714] BUG: soft lockup - CPU#1 stuck for 1024s! [hwclock:126]
[ 1103.513714] irq event stamp: 6902
[ 1103.513714] hardirqs last  enabled at (6901): [<c167e884>] _raw_spin_unlock_irq+0x22/0x40
[ 1103.513714] hardirqs last disabled at (6902): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1103.513714] softirqs last  enabled at (2618): [<c1601126>] unix_sock_destructor+0xae/0xb3
[ 1103.513714] softirqs last disabled at (2616): [<c1601113>] unix_sock_destructor+0x9b/0xb3
[ 1103.513714] Pid: 126, comm: hwclock Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1103.513714] EIP: 0060:[<c167e891>] EFLAGS: 00000286 CPU: 1
[ 1103.513714] EIP is at _raw_spin_unlock_irq+0x2f/0x40
[ 1103.513714] EAX: cf8aa260 EBX: ce832084 ECX: 00000000 EDX: 00000007
[ 1103.513714] ESI: 00000001 EDI: 00000000 EBP: ce895f28 ESP: ce895f24
[ 1103.513714]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.513714] CR0: 8005003b CR2: b7642fb8 CR3: 019f0000 CR4: 00000690
[ 1103.513714] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.513714] DR6: ffff0ff0 DR7: 00000400
[ 1103.513714] Process hwclock (pid: 126, ti=ce894000 task=cf8aa260 task.ti=ce894000)
[ 1103.513714] Stack:
[ 1103.513714]  00000000 ce895f40 c1320af4 cf8aa260 cf8aa260 00000000 0000000f ce895f8c
[ 1103.513714]  c102ce62 b7642fb8 00000000 00000282 afb408d9 00001043 ce895f94 ce895f9c
[ 1103.513714]  00000000 00000001 c102dd6d c12b3946 00000007 00000006 00000000 cf883000
[ 1103.513714] Call Trace:
[ 1103.513714]  [<c1320af4>] disassociate_ctty+0xce/0x1cb
[ 1103.513714]  [<c102ce62>] do_exit+0x21f/0x72c
[ 1103.513714]  [<c102dd6d>] ? do_sys_settimeofday+0x88/0x93
[ 1103.513714]  [<c12b3946>] ? _copy_from_user+0x37/0x4b
[ 1103.513714]  [<c102d3f6>] do_group_exit+0x5e/0x81
[ 1103.513714]  [<c102d42c>] sys_exit_group+0x13/0x13
[ 1103.513714]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1103.513714] Code: 89 c3 8d 40 10 8b 4d 04 ba 01 00 00 00 e8 1e 8b 9e ff 89 d8 e8 38 40 c4 ff e8 69 6a 9e ff 83 3d 94 20 91 c1 00 75 02 0f 0b fb 90 <8d> 74 26 00 89 e0 25 00 e0 ff ff ff 48 14 5b 5d c3 55 89 e5 53
[ 1103.513714] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.513714] Pid: 126, comm: hwclock Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1103.513714] Call Trace:
[ 1103.513714]  [<c1673678>] panic+0x97/0x1aa
[ 1103.513714]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1103.513714]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.513714]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1103.513714]  [<c103205a>] run_local_timers+0x8/0x14
[ 1103.513714]  [<c103208a>] update_process_times+0x24/0x58
[ 1103.513714]  [<c106276e>] tick_periodic+0x66/0x72
[ 1103.513714]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1103.513714]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.513714]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.513714]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1103.513714]  [<c167e891>] ? _raw_spin_unlock_irq+0x2f/0x40
[ 1103.513714]  [<c1320af4>] disassociate_ctty+0xce/0x1cb
[ 1103.513714]  [<c102ce62>] do_exit+0x21f/0x72c
[ 1103.513714]  [<c102dd6d>] ? do_sys_settimeofday+0x88/0x93
[ 1103.513714]  [<c12b3946>] ? _copy_from_user+0x37/0x4b
[ 1103.513714]  [<c102d3f6>] do_group_exit+0x5e/0x81
[ 1103.513714]  [<c102d42c>] sys_exit_group+0x13/0x13
[ 1103.513714]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1103.513714] Rebooting in 10 seconds..
--
[ 1103.518186] [sched_delayed] sched: RT throttling activated
[ 1103.517941] BUG: soft lockup - CPU#1 stuck for 1023s! [chroot:133]
[ 1103.517941] irq event stamp: 1984
[ 1103.517941] hardirqs last  enabled at (1983): [<c101e5d2>] do_page_fault+0x98/0x36e
[ 1103.517941] hardirqs last disabled at (1984): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1103.517941] softirqs last  enabled at (1934): [<c1601126>] unix_sock_destructor+0xae/0xb3
[ 1103.517941] softirqs last disabled at (1932): [<c1601113>] unix_sock_destructor+0x9b/0xb3
[ 1103.517941] Pid: 133, comm: chroot Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1103.517941] EIP: 0060:[<c1065d13>] EFLAGS: 00000282 CPU: 1
[ 1103.517941] EIP is at lock_is_held+0x9f/0xb2
[ 1103.517941] EAX: 00000282 EBX: 00000001 ECX: 00000000 EDX: 00000000
[ 1103.517941] ESI: ce88e720 EDI: c19136b8 EBP: ce861ef8 ESP: ce861ee8
[ 1103.517941]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.517941] CR0: 80050033 CR2: b763c720 CR3: 0e872000 CR4: 00000690
[ 1103.517941] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.517941] DR6: ffff0ff0 DR7: 00000400
[ 1103.517941] Process chroot (pid: 133, ti=ce860000 task=ce88e720 task.ti=ce860000)
[ 1103.517941] Stack:
[ 1103.517941]  00000282 0000046c c181eefe 00000000 ce861f20 c104ba8b c101d888 ce861f20
[ 1103.517941]  c10432c5 00000001 00000002 ce861fb4 ce861fb4 c101d888 ce861f9c c101e6cd
[ 1103.517941]  00000028 ce861f38 ce88e720 ce83ca54 ce83ca00 ce88e720 00000028 00000004
[ 1103.517941] Call Trace:
[ 1103.517941]  [<c104ba8b>] __might_sleep+0x2b/0x1cd
[ 1103.517941]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.517941]  [<c10432c5>] ? down_read_trylock+0x51/0x59
[ 1103.517941]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.517941]  [<c101e6cd>] do_page_fault+0x193/0x36e
[ 1103.517941]  [<c167e884>] ? _raw_spin_unlock_irq+0x22/0x40
[ 1103.517941]  [<c167f085>] ? restore_all+0xf/0xf
[ 1103.517941]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.517941]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.517941]  [<c101d8ab>] do_async_page_fault+0x23/0x5f
[ 1103.517941]  [<c167f5cc>] error_code+0x6c/0x80
[ 1103.517941] Code: 7c e2 31 c9 eb 05 b9 01 00 00 00 64 a1 28 06 9e c1 c7 80 58 04 00 00 00 00 00 00 83 3d 8c 20 91 c1 00 75 02 0f 0b 8b 45 f0 50 9d <8d> 74 26 00 eb 05 b9 01 00 00 00 89 c8 5b 5b 5e 5f 5d c3 55 89
[ 1103.517941] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.517941] Pid: 133, comm: chroot Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1103.517941] Call Trace:
[ 1103.517941]  [<c1673678>] panic+0x97/0x1aa
[ 1103.517941]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1103.517941]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.517941]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1103.517941]  [<c103205a>] run_local_timers+0x8/0x14
[ 1103.517941]  [<c103208a>] update_process_times+0x24/0x58
[ 1103.517941]  [<c106276e>] tick_periodic+0x66/0x72
[ 1103.517941]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1103.517941]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.517941]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.517941]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1103.517941]  [<c1065d13>] ? lock_is_held+0x9f/0xb2
[ 1103.517941]  [<c104ba8b>] __might_sleep+0x2b/0x1cd
[ 1103.517941]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.517941]  [<c10432c5>] ? down_read_trylock+0x51/0x59
[ 1103.517941]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.517941]  [<c101e6cd>] do_page_fault+0x193/0x36e
[ 1103.517941]  [<c167e884>] ? _raw_spin_unlock_irq+0x22/0x40
[ 1103.517941]  [<c167f085>] ? restore_all+0xf/0xf
[ 1103.517941]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.517941]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.517941]  [<c101d8ab>] do_async_page_fault+0x23/0x5f
[ 1103.517941]  [<c167f5cc>] error_code+0x6c/0x80
[ 1103.517941] Rebooting in 10 seconds..
--
[    5.212332] Freeing unused kernel memory: 460k freed
[ 1104.518431] BUG: soft lockup - CPU#1 stuck for 1024s! [swapper/1:0]
[ 1104.518431] irq event stamp: 5734
[ 1104.518431] hardirqs last  enabled at (5731): [<c167f085>] restore_all_notrace+0x0/0x18
[ 1104.518431] hardirqs last disabled at (5732): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.518431] softirqs last  enabled at (5734): [<c102e637>] _local_bh_enable+0xd/0xf
[ 1104.518431] softirqs last disabled at (5733): [<c102eac7>] irq_enter+0x2c/0x4d
[ 1104.518431] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.518431] EIP: 0060:[<c101da32>] EFLAGS: 00000286 CPU: 1
[ 1104.518431] EIP is at native_safe_halt+0x5/0x7
[ 1104.518431] EAX: ce06a260 EBX: ffffffff ECX: ce06a260 EDX: 00000007
[ 1104.518431] ESI: c19716f0 EDI: ffff8b9f EBP: ce071f68 ESP: ce071f68
[ 1104.518431]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.518431] CR0: 8005003b CR2: b768ffb8 CR3: 0f8a4000 CR4: 00000690
[ 1104.518431] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.518431] DR6: ffff0ff0 DR7: 00000400
[ 1104.518431] Process swapper/1 (pid: 0, ti=ce070000 task=ce06a260 task.ti=ce070000)
[ 1104.518431] Stack:
[ 1104.518431]  ce071f70 c1008a09 ce071f88 c1009250 fcc4d513 0b285a35 45735797 cfff9fa0
[ 1104.518431]  ce071fb4 c166ed10 00000000 00000000 00000000 00000000 b5a26032 45735797
[ 1104.518431]  01000800 0000d248 00000000 00000000 00000000 00000000 00000000 00000000
[ 1104.518431] Call Trace:
[ 1104.518431]  [<c1008a09>] default_idle+0x2f/0x55
[ 1104.518431]  [<c1009250>] cpu_idle+0xc9/0xfb
[ 1104.518431]  [<c166ed10>] start_secondary+0x292/0x29a
[ 1104.518431] Code: 0f 22 e0 5d c3 55 89 e5 0f 09 5d c3 55 89 e5 9c 58 5d c3 55 89 e5 50 9d 5d c3 55 89 e5 fa 5d c3 55 89 e5 fb 5d c3 55 89 e5 fb f4 <5d> c3 55 89 e5 f4 5d c3 55 89 e5 57 56 53 53 89 c6 89 55 f0 89
[ 1104.518431] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.518431] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.518431] Call Trace:
[ 1104.518431]  [<c1673678>] panic+0x97/0x1aa
[ 1104.518431]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.518431]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.518431]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.518431]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.518431]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.518431]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.518431]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.518431]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.518431]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.518431]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.518431]  [<c101da32>] ? native_safe_halt+0x5/0x7
[ 1104.518431]  [<c1008a09>] default_idle+0x2f/0x55
[ 1104.518431]  [<c1009250>] cpu_idle+0xc9/0xfb
[ 1104.518431]  [<c166ed10>] start_secondary+0x292/0x29a
[ 1104.518431] Rebooting in 10 seconds..
--
[ 1104.513600] [sched_delayed] sched: RT throttling activated
[ 1104.513573] BUG: soft lockup - CPU#1 stuck for 1025s! [run-parts:123]
[ 1104.513573] irq event stamp: 1114
[ 1104.513573] hardirqs last  enabled at (1113): [<c167f085>] restore_all_notrace+0x0/0x18
[ 1104.513573] hardirqs last disabled at (1114): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.513573] softirqs last  enabled at (0): [<c1026857>] copy_process+0x465/0xf8a
[ 1104.513573] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1104.513573] Pid: 123, comm: run-parts Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.513573] EIP: 0060:[<c10a036c>] EFLAGS: 00000202 CPU: 1
--
[    5.941592] Freeing unused kernel memory: 460k freed
[ 1104.521335] BUG: soft lockup - CPU#1 stuck for 1023s! [umount:132]
[ 1104.521335] irq event stamp: 2688
[ 1104.521335] hardirqs last  enabled at (2687): [<c167f898>] sysenter_exit+0xf/0x1e
[ 1104.521335] hardirqs last disabled at (2688): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.521335] softirqs last  enabled at (0): [<c1026857>] copy_process+0x465/0xf8a
[ 1104.521335] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1104.521335] Pid: 132, comm: umount Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.521335] EIP: 0060:[<c10675e9>] EFLAGS: 00000282 CPU: 1
[ 1104.521335] EIP is at lock_acquire+0xb4/0xc0
[ 1104.521335] EAX: 00000282 EBX: cf899130 ECX: 000001e4 EDX: 000001e4
[ 1104.521335] ESI: 00000000 EDI: cb48f9bc EBP: ce8bfeec ESP: ce8bfec0
[ 1104.521335]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.521335] CR0: 80050033 CR2: b773f8d0 CR3: 0e8a1000 CR4: 00000690
[ 1104.521335] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.521335] DR6: ffff0ff0 DR7: 00000400
[ 1104.521335] Process umount (pid: 132, ti=ce8be000 task=cf899130 task.ti=ce8be000)
[ 1104.521335] Stack:
[ 1104.521335]  00000000 00000002 00000000 00000000 c10d99bd 00000000 00000000 00000282
[ 1104.521335]  cb48f9ac ffffff9c cb48f9ac ce8bff08 c167e6a2 00000000 00000002 00000000
[ 1104.521335]  c10d99bd cb48f960 ce8bff20 c10d99bd ffffff9c ce8bff38 ffffff9c 00000000
[ 1104.521335] Call Trace:
[ 1104.521335]  [<c10d99bd>] ? dput+0x37/0x143
[ 1104.521335]  [<c167e6a2>] _raw_spin_lock+0x3d/0x4a
[ 1104.521335]  [<c10d99bd>] ? dput+0x37/0x143
[ 1104.521335]  [<c10d99bd>] dput+0x37/0x143
[ 1104.521335]  [<c10d1041>] path_put+0xe/0x18
[ 1104.521335]  [<c10cc05b>] vfs_fstatat+0x56/0x60
[ 1104.521335]  [<c10cc07e>] vfs_stat+0x19/0x1b
[ 1104.521335]  [<c10cc22a>] sys_stat64+0x11/0x22
[ 1104.521335]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1104.521335] Code: 45 08 89 04 24 8b 4d ec 89 f2 89 f8 e8 57 e7 ff ff c7 83 58 04 00 00 00 00 00 00 83 3d 8c 20 91 c1 00 75 02 0f 0b 8b 45 f0 50 9d <8d> 74 26 00 83 c4 20 5b 5e 5f 5d c3 55 89 e5 57 56 53 83 ec 1c
[ 1104.521335] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.521335] Pid: 132, comm: umount Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.521335] Call Trace:
[ 1104.521335]  [<c1673678>] panic+0x97/0x1aa
[ 1104.521335]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.521335]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.521335]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.521335]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.521335]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.521335]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.521335]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.521335]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.521335]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.521335]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.521335]  [<c10d00d8>] ? pipe_fcntl+0xbc/0x140
[ 1104.521335]  [<c10675e9>] ? lock_acquire+0xb4/0xc0
[ 1104.521335]  [<c10d99bd>] ? dput+0x37/0x143
[ 1104.521335]  [<c167e6a2>] _raw_spin_lock+0x3d/0x4a
[ 1104.521335]  [<c10d99bd>] ? dput+0x37/0x143
[ 1104.521335]  [<c10d99bd>] dput+0x37/0x143
[ 1104.521335]  [<c10d1041>] path_put+0xe/0x18
[ 1104.521335]  [<c10cc05b>] vfs_fstatat+0x56/0x60
[ 1104.521335]  [<c10cc07e>] vfs_stat+0x19/0x1b
[ 1104.521335]  [<c10cc22a>] sys_stat64+0x11/0x22
[ 1104.521335]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1104.521335] Rebooting in 10 seconds..
--
[    5.028838] Freeing unused kernel memory: 460k freed
[ 1104.517771] BUG: soft lockup - CPU#1 stuck for 1025s! [mountall:135]
[ 1104.517771] irq event stamp: 9160
[ 1104.517771] hardirqs last  enabled at (9159): [<c167f085>] restore_all_notrace+0x0/0x18
[ 1104.517771] hardirqs last disabled at (9160): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.517771] softirqs last  enabled at (3012): [<c1601126>] unix_sock_destructor+0xae/0xb3
[ 1104.517771] softirqs last disabled at (3010): [<c1601113>] unix_sock_destructor+0x9b/0xb3
[ 1104.517771] Pid: 135, comm: mountall Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.517771] EIP: 0073:[<b7722c34>] EFLAGS: 00000202 CPU: 1
[ 1104.517771] EIP is at 0xb7722c34
[ 1104.517771] EAX: ff8760ae EBX: b7739ff4 ECX: 000000ae EDX: 00020415
[ 1104.517771] ESI: b76f1590 EDI: 00000005 EBP: 007fc3b0 ESP: bfc17470
[ 1104.517771]  DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 1104.517771] Process mountall (pid: 135, ti=ce8c6000 task=ce842260 task.ti=ce8c6000)
[ 1104.517771] 
[ 1104.517771] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.517771] Pid: 135, comm: mountall Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.517771] Call Trace:
[ 1104.526641] [sched_delayed] sched: RT throttling activated
[ 1104.517771]  [<c1673678>] panic+0x97/0x1aa
[ 1104.517771]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.517771]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.517771]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.517771]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.517771]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.517771]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.517771]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.517771]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.517771]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.517771]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.517771] Rebooting in 10 seconds..
--
[    5.786476] Freeing unused kernel memory: 460k freed
[ 1104.518564] BUG: soft lockup - CPU#1 stuck for 1023s! [grep:129]
[ 1104.518564] irq event stamp: 2294
[ 1104.518564] hardirqs last  enabled at (2293): [<c167e841>] _raw_spin_unlock_irqrestore+0x42/0x63
[ 1104.518564] hardirqs last disabled at (2294): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.518564] softirqs last  enabled at (240): [<c102e8ff>] __do_softirq+0x131/0x153
[ 1104.523873] [sched_delayed] sched: RT throttling activated
[ 1104.518564] softirqs last disabled at (205): [<c1003f90>] do_softirq+0x74/0xdf
[ 1104.518564] Pid: 129, comm: grep Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.518564] EIP: 0060:[<c167e850>] EFLAGS: 00000282 CPU: 1
[ 1104.518564] EIP is at _raw_spin_unlock_irqrestore+0x51/0x63
[ 1104.518564] EAX: 00000282 EBX: 00000282 ECX: 00000000 EDX: 00000007
[ 1104.518564] ESI: c196c730 EDI: 00000002 EBP: ce89fe2c ESP: ce89fe24
[ 1104.518564]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.518564] CR0: 8005003b CR2: b7627fb8 CR3: 019f0000 CR4: 00000690
[ 1104.518564] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.518564] DR6: ffff0ff0 DR7: 00000400
[ 1104.518564] Process grep (pid: 129, ti=ce89e000 task=ce849130 task.ti=ce89e000)
[ 1104.518564] Stack:
[ 1104.518564]  ce538018 c196c490 ce89fe54 c10a0736 00000000 c109fe68 c196c490 ce68f3c0
[ 1104.518564]  00000282 00000001 ce538098 00000005 ce89fe68 c10a0a6d ce89fec0 ce89fee4
[ 1104.518564]  ce89fed8 ce89fe70 c10a0a8a ce89fe8c c10b9614 bff4f000 00000008 ce89fec0
[ 1104.518564] Call Trace:
[ 1104.518564]  [<c10a0736>] pagevec_lru_move_fn+0x8d/0xa8
[ 1104.518564]  [<c109fe68>] ? pagevec_move_tail_fn+0x54/0x54
[ 1104.518564]  [<c10a0a6d>] lru_add_drain_cpu+0xe0/0xe5
[ 1104.518564]  [<c10a0a8a>] lru_add_drain+0x18/0x24
[ 1104.518564]  [<c10b9614>] free_pages_and_swap_cache+0x13/0x7b
[ 1104.518564]  [<c10ac647>] tlb_flush_mmu+0x56/0x6c
[ 1104.518564]  [<c10ac66f>] tlb_finish_mmu+0x12/0x31
[ 1104.518564]  [<c10b19f9>] exit_mmap+0x96/0xe4
[ 1104.518564]  [<c1025e1a>] mmput+0x42/0xa7
[ 1104.518564]  [<c102cab2>] exit_mm+0xde/0xe6
[ 1104.518564]  [<c102ce13>] do_exit+0x1d0/0x72c
[ 1104.518564]  [<c102d3f6>] do_group_exit+0x5e/0x81
[ 1104.518564]  [<c102d42c>] sys_exit_group+0x13/0x13
[ 1104.518564]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1104.518564] Code: 3d 8c 20 91 c1 00 74 1d 89 d8 50 9d 8d 74 26 00 e8 dd 6b 9e ff eb 18 e8 ac 6a 9e ff 83 3d 8c 20 91 c1 00 75 02 0f 0b 89 d8 50 9d <8d> 74 26 00 89 e0 25 00 e0 ff ff ff 48 14 5b 5e 5d c3 55 89 e5
[ 1104.518564] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.518564] Pid: 129, comm: grep Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.518564] Call Trace:
[ 1104.518564]  [<c1673678>] panic+0x97/0x1aa
[ 1104.518564]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.518564]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.518564]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.518564]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.518564]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.518564]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.518564]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.518564]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.518564]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.518564]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.518564]  [<c167e850>] ? _raw_spin_unlock_irqrestore+0x51/0x63
[ 1104.518564]  [<c10a0736>] pagevec_lru_move_fn+0x8d/0xa8
[ 1104.518564]  [<c109fe68>] ? pagevec_move_tail_fn+0x54/0x54
[ 1104.518564]  [<c10a0a6d>] lru_add_drain_cpu+0xe0/0xe5
[ 1104.518564]  [<c10a0a8a>] lru_add_drain+0x18/0x24
[ 1104.518564]  [<c10b9614>] free_pages_and_swap_cache+0x13/0x7b
[ 1104.518564]  [<c10ac647>] tlb_flush_mmu+0x56/0x6c
[ 1104.518564]  [<c10ac66f>] tlb_finish_mmu+0x12/0x31
[ 1104.518564]  [<c10b19f9>] exit_mmap+0x96/0xe4
[ 1104.518564]  [<c1025e1a>] mmput+0x42/0xa7
[ 1104.518564]  [<c102cab2>] exit_mm+0xde/0xe6
[ 1104.518564]  [<c102ce13>] do_exit+0x1d0/0x72c
[ 1104.518564]  [<c102d3f6>] do_group_exit+0x5e/0x81
[ 1104.518564]  [<c102d42c>] sys_exit_group+0x13/0x13
[ 1104.518564]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1104.518564] Rebooting in 10 seconds..
--
[    5.159006] Freeing unused kernel memory: 460k freed
[ 1104.513734] BUG: soft lockup - CPU#1 stuck for 1025s! [swapper/1:0]
[ 1104.522610] [sched_delayed] sched: RT throttling activated
[ 1104.513734] irq event stamp: 12942
[ 1104.513734] hardirqs last  enabled at (12939): [<c167f085>] restore_all_notrace+0x0/0x18
[ 1104.513734] hardirqs last disabled at (12940): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.513734] softirqs last  enabled at (12942): [<c102e637>] _local_bh_enable+0xd/0xf
[ 1104.513734] softirqs last disabled at (12941): [<c102eac7>] irq_enter+0x2c/0x4d
[ 1104.513734] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.513734] EIP: 0060:[<c101da32>] EFLAGS: 00000286 CPU: 1
[ 1104.513734] EIP is at native_safe_halt+0x5/0x7
[ 1104.513734] EAX: ce06a260 EBX: ffffffff ECX: ce06a260 EDX: 00000007
[ 1104.513734] ESI: c19716f0 EDI: ffff8b9f EBP: ce071f68 ESP: ce071f68
[ 1104.513734]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.513734] CR0: 8005003b CR2: b7710000 CR3: 0ee89000 CR4: 00000690
[ 1104.513734] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.513734] DR6: ffff0ff0 DR7: 00000400
[ 1104.513734] Process swapper/1 (pid: 0, ti=ce070000 task=ce06a260 task.ti=ce070000)
[ 1104.513734] Stack:
[ 1104.513734]  ce071f70 c1008a09 ce071f88 c1009250 302764a8 3e8ae9cb 78d59122 cfff9fa0
[ 1104.552695]  ce071fb4 c166ed10 00000000 00000000 00000000 00000000 e90499bd 78d59122
[ 1104.552695]  01000800 0000d248 00000000 00000000 00000000 00000000 00000000 00000000
[ 1104.552695] Call Trace:
[ 1104.552695]  [<c1008a09>] default_idle+0x2f/0x55
[ 1104.552695]  [<c1009250>] cpu_idle+0xc9/0xfb
[ 1104.552695]  [<c166ed10>] start_secondary+0x292/0x29a
[ 1104.552695] Code: 0f 22 e0 5d c3 55 89 e5 0f 09 5d c3 55 89 e5 9c 58 5d c3 55 89 e5 50 9d 5d c3 55 89 e5 fa 5d c3 55 89 e5 fb 5d c3 55 89 e5 fb f4 <5d> c3 55 89 e5 f4 5d c3 55 89 e5 57 56 53 53 89 c6 89 55 f0 89
[ 1104.552695] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.552695] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.552695] Call Trace:
[ 1104.552695]  [<c1673678>] panic+0x97/0x1aa
[ 1104.552695]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.552695]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.552695]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.552695]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.552695]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.552695]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.552695]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.552695]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.552695]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.552695]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.552695]  [<c101da32>] ? native_safe_halt+0x5/0x7
[ 1104.552695]  [<c1008a09>] default_idle+0x2f/0x55
[ 1104.552695]  [<c1009250>] cpu_idle+0xc9/0xfb
[ 1104.552695]  [<c166ed10>] start_secondary+0x292/0x29a
[ 1104.552695] Rebooting in 10 seconds..
--
[    5.258753] Freeing unused kernel memory: 460k freed
[ 1104.517703] BUG: soft lockup - CPU#1 stuck for 1025s! [plymouthd:125]
[ 1104.517703] irq event stamp: 6312
[ 1104.517703] hardirqs last  enabled at (6311): [<c167f085>] restore_all_notrace+0x0/0x18
[ 1104.517703] hardirqs last disabled at (6312): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.517703] softirqs last  enabled at (4300): [<c102e8ff>] __do_softirq+0x131/0x153
[ 1104.517703] softirqs last disabled at (4287): [<c1003f90>] do_softirq+0x74/0xdf
[ 1104.517703] Pid: 125, comm: plymouthd Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.517703] EIP: 0060:[<c10ad232>] EFLAGS: 00000282 CPU: 1
[ 1104.517703] EIP is at unmap_single_vma+0xf/0x473
[ 1104.517703] EAX: ce86fe60 EBX: ce846318 ECX: b76db000 EDX: ce846318
[ 1104.526960] [sched_delayed] sched: RT throttling activated
[ 1104.517703] ESI: ce86fe60 EDI: b76db000 EBP: ce86fe34 ESP: ce86fdd0
[ 1104.517703]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.517703] CR0: 8005003b CR2: b76daedc CR3: 0e86a000 CR4: 00000690
[ 1104.517703] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.517703] DR6: ffff0ff0 DR7: 00000400
[ 1104.517703] Process plymouthd (pid: 125, ti=ce86e000 task=cf86e720 task.ti=ce86e000)
[ 1104.517703] Stack:
[ 1104.517703]  ce86fe04 c10a06a1 00000000 00000001 ce538120 00000282 c196c730 00000001
[ 1104.517703]  ce86fdf0 ce86fdf0 ce538118 c196c490 00000001 ce86fe60 c10a0743 00000001
[ 1104.517703]  ce846318 c196c490 ce73e940 00000282 00000001 ce538118 ce846318 ce86fe60
[ 1104.517703] Call Trace:
[ 1104.517703]  [<c10a06a1>] ? release_pages+0x171/0x179
[ 1104.517703]  [<c10a0743>] ? pagevec_lru_move_fn+0x9a/0xa8
[ 1104.517703]  [<c10adbca>] unmap_vmas+0x2a/0x40
[ 1104.517703]  [<c10b0843>] unmap_region+0x5c/0x92
[ 1104.517703]  [<c10b1893>] do_munmap+0x1d3/0x247
[ 1104.517703]  [<c10b21fa>] mmap_region+0x68/0x38a
[ 1104.517703]  [<c10b270b>] do_mmap_pgoff+0x1ef/0x22d
[ 1104.517703]  [<c10a69ab>] vm_mmap_pgoff+0x47/0x59
[ 1104.517703]  [<c10b1265>] sys_mmap_pgoff+0x80/0xa2
[ 1104.517703]  [<c167f04c>] syscall_call+0x7/0xb
[ 1104.517703] Code: 00 00 00 85 db 75 04 eb 09 89 c6 89 d8 e8 34 31 ff ff 89 f0 83 c4 2c 5b 5e 5f 5d c3 55 89 e5 57 56 53 83 ec 58 89 45 d0 89 55 dc <8b> 42 04 89 ce 39 c1 73 02 89 c6 8b 4d dc 8b 51 08 39 d6 0f 83
[ 1104.517703] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.517703] Pid: 125, comm: plymouthd Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.517703] Call Trace:
[ 1104.517703]  [<c1673678>] panic+0x97/0x1aa
[ 1104.517703]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.517703]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.517703]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.517703]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.517703]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.517703]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.517703]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.517703]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.517703]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.517703]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.517703]  [<c10600e0>] ? timecounter_init+0x13/0x21
[ 1104.517703]  [<c10ad232>] ? unmap_single_vma+0xf/0x473
[ 1104.517703]  [<c10a06a1>] ? release_pages+0x171/0x179
[ 1104.517703]  [<c10a0743>] ? pagevec_lru_move_fn+0x9a/0xa8
[ 1104.517703]  [<c10adbca>] unmap_vmas+0x2a/0x40
[ 1104.517703]  [<c10b0843>] unmap_region+0x5c/0x92
[ 1104.517703]  [<c10b1893>] do_munmap+0x1d3/0x247
[ 1104.517703]  [<c10b21fa>] mmap_region+0x68/0x38a
[ 1104.517703]  [<c10b270b>] do_mmap_pgoff+0x1ef/0x22d
[ 1104.517703]  [<c10a69ab>] vm_mmap_pgoff+0x47/0x59
[ 1104.517703]  [<c10b1265>] sys_mmap_pgoff+0x80/0xa2
[ 1104.517703]  [<c167f04c>] syscall_call+0x7/0xb
[ 1104.517703] Rebooting in 10 seconds..
--
[    5.088389] Freeing unused kernel memory: 460k freed
[ 1104.515816] BUG: soft lockup - CPU#1 stuck for 1025s! [99-trinity:126]
--
[    4.371778] Freeing unused kernel memory: 460k freed
[ 1103.516949] BUG: soft lockup - CPU#1 stuck for 1024s! [99-trinity:135]
[ 1103.516949] irq event stamp: 214
[ 1103.516949] hardirqs last  enabled at (213): [<c101e5d2>] do_page_fault+0x98/0x36e
[ 1103.516949] hardirqs last disabled at (214): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1103.516949] softirqs last  enabled at (0): [<c1026857>] copy_process+0x465/0xf8a
[ 1103.516949] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1103.516949] Pid: 135, comm: 99-trinity Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1103.516949] EIP: 0060:[<c1065d13>] EFLAGS: 00000282 CPU: 1
[ 1103.516949] EIP is at lock_is_held+0x9f/0xb2
[ 1103.516949] EAX: 00000282 EBX: 00000001 ECX: 00000001 EDX: 00000003
[ 1103.516949] ESI: ce8b3390 EDI: c19136b8 EBP: ce8cdedc ESP: ce8cdecc
[ 1103.516949]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.526882] [sched_delayed] sched: RT throttling activated
[ 1103.516949] CR0: 80050033 CR2: 080aaed0 CR3: 0e898000 CR4: 00000690
[ 1103.516949] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.516949] DR6: ffff0ff0 DR7: 00000400
mountall: ply-ev[ 1103.516949] Process 99-trinity (pid: 135, ti=ce8cc000 task=ce8b3390 task.ti=ce8cc000)
ent-loop.c:497: [ 1103.516949] Stack:
ply_event_loop_n[ 1103.516949]  00000282ew: Assertion `l ce8b3390oop->epoll_fd >= 0000000b 0' failed.
 ce130c80 ce8cdefc c10c2e53 00000002 00000001
[ 1103.516949]  00000000 c10c2db3 ce89a6e0 080aaed0 ce8cdf20 c10aef10 ce8cdf20 c10432c5
[ 1103.516949]  00000001 ce89a6e0 ce89a6e0 ce8cdfb4 c101d888 ce8cdf9c c101e870 00000028
--
[    4.924030] Freeing unused kernel memory: 460k freed
[ 1103.513755] BUG: soft lockup - CPU#1 stuck for 1024s! [init:1]
[ 1103.513755] irq event stamp: 744676
[ 1103.513755] hardirqs last  enabled at (744675): [<c167e841>] _raw_spin_unlock_irqrestore+0x42/0x63
[ 1103.513755] hardirqs last disabled at (744676): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1103.513755] softirqs last  enabled at (741152): [<c102e8ff>] __do_softirq+0x131/0x153
[ 1103.513755] softirqs last disabled at (741139): [<c1003f90>] do_softirq+0x74/0xdf
[ 1103.513755] Pid: 1, comm: init Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1103.513755] EIP: 0060:[<c167e850>] EFLAGS: 00000282 CPU: 1
[ 1103.513755] EIP is at _raw_spin_unlock_irqrestore+0x51/0x63
[ 1103.513755] EAX: 00000282 EBX: 00000282 ECX: 00000000 EDX: 00000007
[ 1103.513755] ESI: cf89b778 EDI: 00000001 EBP: ce043dfc ESP: ce043df4
[ 1103.513755]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.513755] CR0: 80050033 CR2: b76aa920 CR3: 0ee86000 CR4: 00000690
[ 1103.523273] [sched_delayed] sched: RT throttling activated
[ 1103.513755] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.513755] DR6: ffff0ff0 DR7: 00000400
[ 1103.513755] Process init (pid: 1, ti=ce042000 task=ce048000 task.ti=ce042000)
[ 1103.513755] Stack:
[ 1103.513755]  00000000 cf89b390 ce043e20 c104a23a 00000282 cf89b778 0cb4c000 00000000
[ 1103.513755]  cf89b390 00000000 00000007 ce043e28 c104a280 ce043e38 c10306b6 cf89b390
[ 1103.513755]  00000000 ce043ef4 c1031596 003f8011 00000000 ce048474 00000000 ce048000
[ 1103.513755] Call Trace:
[ 1103.513755]  [<c104a23a>] try_to_wake_up+0x248/0x266
[ 1103.513755]  [<c104a280>] wake_up_state+0xa/0xc
[ 1103.513755]  [<c10306b6>] ptrace_resume+0x76/0x7c
[ 1103.513755]  [<c1031596>] ptrace_request+0x368/0x4f8
[ 1103.513755]  [<c106716a>] ? __lock_acquire+0x1444/0x1455
[ 1103.513755]  [<c106716a>] ? __lock_acquire+0x1444/0x1455
[ 1103.513755]  [<c10671f9>] ? lock_release_non_nested+0x7e/0x21b
[ 1103.513755]  [<c106716a>] ? __lock_acquire+0x1444/0x1455
[ 1103.513755]  [<c100acba>] arch_ptrace+0x474/0x48a
[ 1103.513755]  [<c167e841>] ? _raw_spin_unlock_irqrestore+0x42/0x63
[ 1103.513755]  [<c10652f8>] ? trace_hardirqs_on+0xb/0xd
[ 1103.513755]  [<c1047e25>] ? wait_task_inactive+0x6f/0xc1
[ 1103.513755]  [<c1030972>] ? ptrace_check_attach+0xb4/0xc4
[ 1103.513755]  [<c1031192>] sys_ptrace+0x34f/0x379
[ 1103.513755]  [<c1030eb1>] ? sys_ptrace+0x6e/0x379
[ 1103.513755]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1103.513755] Code: 3d 8c 20 91 c1 00 74 1d 89 d8 50 9d 8d 74 26 00 e8 dd 6b 9e ff eb 18 e8 ac 6a 9e ff 83 3d 8c 20 91 c1 00 75 02 0f 0b 89 d8 50 9d <8d> 74 26 00 89 e0 25 00 e0 ff ff ff 48 14 5b 5e 5d c3 55 89 e5
[ 1103.513755] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.513755] Pid: 1, comm: init Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1103.513755] Call Trace:
[ 1103.513755]  [<c1673678>] panic+0x97/0x1aa
[ 1103.513755]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1103.513755]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.513755]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1103.513755]  [<c103205a>] run_local_timers+0x8/0x14
[ 1103.513755]  [<c103208a>] update_process_times+0x24/0x58
[ 1103.513755]  [<c106276e>] tick_periodic+0x66/0x72
[ 1103.513755]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1103.513755]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.513755]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.513755]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1103.513755]  [<c167e850>] ? _raw_spin_unlock_irqrestore+0x51/0x63
[ 1103.513755]  [<c104a23a>] try_to_wake_up+0x248/0x266
[ 1103.513755]  [<c104a280>] wake_up_state+0xa/0xc
[ 1103.513755]  [<c10306b6>] ptrace_resume+0x76/0x7c
[ 1103.513755]  [<c1031596>] ptrace_request+0x368/0x4f8
[ 1103.513755]  [<c106716a>] ? __lock_acquire+0x1444/0x1455
[ 1103.513755]  [<c106716a>] ? __lock_acquire+0x1444/0x1455
[ 1103.513755]  [<c10671f9>] ? lock_release_non_nested+0x7e/0x21b
[ 1103.513755]  [<c106716a>] ? __lock_acquire+0x1444/0x1455
[ 1103.513755]  [<c100acba>] arch_ptrace+0x474/0x48a
[ 1103.513755]  [<c167e841>] ? _raw_spin_unlock_irqrestore+0x42/0x63
[ 1103.513755]  [<c10652f8>] ? trace_hardirqs_on+0xb/0xd
[ 1103.513755]  [<c1047e25>] ? wait_task_inactive+0x6f/0xc1
[ 1103.513755]  [<c1030972>] ? ptrace_check_attach+0xb4/0xc4
[ 1103.513755]  [<c1031192>] sys_ptrace+0x34f/0x379
[ 1103.513755]  [<c1030eb1>] ? sys_ptrace+0x6e/0x379
[ 1103.513755]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1103.513755] Rebooting in 10 seconds..
--
[ 1104.514595] [sched_delayed] sched: RT throttling activated
[ 1104.514574] BUG: soft lockup - CPU#1 stuck for 1023s! [hwclock:126]
[ 1104.514574] irq event stamp: 6886
[ 1104.514574] hardirqs last  enabled at (6885): [<c109c1fc>] get_page_from_freelist+0x2ca/0x456
[ 1104.514574] hardirqs last disabled at (6886): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.514574] softirqs last  enabled at (2696): [<c1601126>] unix_sock_destructor+0xae/0xb3
[ 1104.514574] softirqs last disabled at (2694): [<c1601113>] unix_sock_destructor+0x9b/0xb3
[ 1104.514574] Pid: 126, comm: hwclock Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.514574] EIP: 0060:[<c10ad3a3>] EFLAGS: 00000246 CPU: 1
[ 1104.514574] EIP is at unmap_single_vma+0x180/0x473
[ 1104.514574] EAX: ce73eb80 EBX: ce73eb80 ECX: 0ffdc067 EDX: bf8e4000
[ 1104.514574] ESI: ce870390 EDI: 00000000 EBP: ce87de98 ESP: ce87de34
[ 1104.514574]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.514574] CR0: 8005003b CR2: b7666fb8 CR3: 0e877000 CR4: 00000690
[ 1104.514574] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.514574] DR6: ffff0ff0 DR7: 00000400
[ 1104.514574] Process hwclock (pid: 126, ti=ce87c000 task=cf8844c0 task.ti=ce87c000)
[ 1104.514574] Stack:
[ 1104.514574]  00000002 00000203 006f8203 00000000 cf884934 bf8e4fff ce852a34 ce87030c
[ 1104.514574]  0ffc9067 ce877bf8 ce877bf8 ce852a00 00000000 ce87dec0 bf8e5000 bf8e5000
[ 1104.514574]  ce839318 bf8e4000 ce87de80 00000000 ffffffff 00000000 ce839318 ce87dec0
[ 1104.514574] Call Trace:
[ 1104.514574]  [<c10adbca>] unmap_vmas+0x2a/0x40
[ 1104.514574]  [<c10b19d9>] exit_mmap+0x76/0xe4
[ 1104.514574]  [<c1025e1a>] mmput+0x42/0xa7
[ 1104.514574]  [<c102cab2>] exit_mm+0xde/0xe6
[ 1104.514574]  [<c102ce13>] do_exit+0x1d0/0x72c
[ 1104.514574]  [<c102dd6d>] ? do_sys_settimeofday+0x88/0x93
[ 1104.514574]  [<c12b3946>] ? _copy_from_user+0x37/0x4b
[ 1104.514574]  [<c102d3f6>] do_group_exit+0x5e/0x81
[ 1104.514574]  [<c102d42c>] sys_exit_group+0x13/0x13
[ 1104.514574]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1104.514574] Code: 7d 0c 8b 0e 85 c9 0f 84 3b 02 00 00 f7 c1 01 01 00 00 0f 84 73 01 00 00 8b 55 e0 8b 45 dc e8 d2 f7 ff ff 89 c3 85 ff 0f 95 45 cc <85> c0 74 30 80 7d cc 00 74 2a 8b 47 04 85 c0 74 09 3b 43 04 0f
[ 1104.514574] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.514574] Pid: 126, comm: hwclock Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.514574] Call Trace:
[ 1104.514574]  [<c1673678>] panic+0x97/0x1aa
[ 1104.514574]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.514574]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.514574]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.514574]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.514574]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.514574]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.514574]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.514574]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.514574]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.514574]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.514574]  [<c167007b>] ? kvm_guest_cpu_init+0x10b/0x214
[ 1104.514574]  [<c10ad3a3>] ? unmap_single_vma+0x180/0x473
[ 1104.514574]  [<c10adbca>] unmap_vmas+0x2a/0x40
[ 1104.514574]  [<c10b19d9>] exit_mmap+0x76/0xe4
[ 1104.514574]  [<c1025e1a>] mmput+0x42/0xa7
[ 1104.514574]  [<c102cab2>] exit_mm+0xde/0xe6
[ 1104.514574]  [<c102ce13>] do_exit+0x1d0/0x72c
[ 1104.514574]  [<c102dd6d>] ? do_sys_settimeofday+0x88/0x93
[ 1104.514574]  [<c12b3946>] ? _copy_from_user+0x37/0x4b
[ 1104.514574]  [<c102d3f6>] do_group_exit+0x5e/0x81
[ 1104.514574]  [<c102d42c>] sys_exit_group+0x13/0x13
[ 1104.514574]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1104.514574] Rebooting in 10 seconds..
--
[    4.775380] Freeing unused kernel memory: 460k freed
[ 1103.517476] BUG: soft lockup - CPU#1 stuck for 1024s! [sleep:135]
[ 1103.517476] irq event stamp: 1030
[ 1103.517476] hardirqs last  enabled at (1029): [<c101e5d2>] do_page_fault+0x98/0x36e
[ 1103.517476] hardirqs last disabled at (1030): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1103.521211] softirqs last  enabled at (0): [<c1026857>] copy_process+0x465/0xf8a
[ 1103.521211] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1103.521211] Pid: 135, comm: sleep Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1103.521211] EIP: 0060:[<c10675e9>] EFLAGS: 00000282 CPU: 1
[ 1103.526940] [sched_delayed] sched: RT throttling activated
[ 1103.521211] EIP is at lock_acquire+0xb4/0xc0
[ 1103.521211] EAX: 00000282 EBX: ce89e720 ECX: 006fc012 EDX: 006fc012
[ 1103.521211] ESI: 00000000 EDI: c19136b8 EBP: ce8cfedc ESP: ce8cfeb0
[ 1103.521211]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.521211] CR0: 8005003b CR2: b77222c0 CR3: 0e8c4000 CR4: 00000690
[ 1103.521211] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.521211] DR6: ffff0ff0 DR7: 00000400
[ 1103.521211] Process sleep (pid: 135, ti=ce8ce000 task=ce89e720 task.ti=ce8ce000)
[ 1103.521211] Stack:
[ 1103.521211]  00000002 00000001 00000000 00000000 c10c2db3 00000000 00000000 00000282
[ 1103.521211]  ce84aa00 0000000b ce84aa00 ce8cfefc c10c2de0 00000002 00000001 00000000
[ 1103.521211]  c10c2db3 ce843948 b77222c0 ce8cff20 c10aef10 ce8cff20 c10432c5 00000001
[ 1103.521211] Call Trace:
[ 1103.521211]  [<c10c2db3>] ? mem_cgroup_count_vm_event+0x1e/0x163
[ 1103.521211]  [<c10c2de0>] mem_cgroup_count_vm_event+0x4b/0x163
[ 1103.521211]  [<c10c2db3>] ? mem_cgroup_count_vm_event+0x1e/0x163
[ 1103.521211]  [<c10aef10>] handle_mm_fault+0x2f/0xb0
[ 1103.521211]  [<c10432c5>] ? down_read_trylock+0x51/0x59
[ 1103.521211]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.521211]  [<c101e870>] do_page_fault+0x336/0x36e
[ 1103.521211]  [<c10c8022>] ? vfs_read+0x103/0x10d
[ 1103.521211]  [<c167f085>] ? restore_all+0xf/0xf
[ 1103.521211]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.521211]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.521211]  [<c101d8ab>] do_async_page_fault+0x23/0x5f
[ 1103.521211]  [<c167f5cc>] error_code+0x6c/0x80
[ 1103.521211] Code: 45 08 89 04 24 8b 4d ec 89 f2 89 f8 e8 57 e7 ff ff c7 83 58 04 00 00 00 00 00 00 83 3d 8c 20 91 c1 00 75 02 0f 0b 8b 45 f0 50 9d <8d> 74 26 00 83 c4 20 5b 5e 5f 5d c3 55 89 e5 57 56 53 83 ec 1c
[ 1103.521211] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.521211] Pid: 135, comm: sleep Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1103.521211] Call Trace:
[ 1103.521211]  [<c1673678>] panic+0x97/0x1aa
[ 1103.521211]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1103.521211]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.521211]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1103.521211]  [<c103205a>] run_local_timers+0x8/0x14
[ 1103.521211]  [<c103208a>] update_process_times+0x24/0x58
[ 1103.521211]  [<c106276e>] tick_periodic+0x66/0x72
[ 1103.521211]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1103.521211]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.521211]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.521211]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1103.521211]  [<c10675e9>] ? lock_acquire+0xb4/0xc0
[ 1103.521211]  [<c10c2db3>] ? mem_cgroup_count_vm_event+0x1e/0x163
[ 1103.521211]  [<c10c2de0>] mem_cgroup_count_vm_event+0x4b/0x163
[ 1103.521211]  [<c10c2db3>] ? mem_cgroup_count_vm_event+0x1e/0x163
[ 1103.521211]  [<c10aef10>] handle_mm_fault+0x2f/0xb0
[ 1103.521211]  [<c10432c5>] ? down_read_trylock+0x51/0x59
[ 1103.521211]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.521211]  [<c101e870>] do_page_fault+0x336/0x36e
[ 1103.521211]  [<c10c8022>] ? vfs_read+0x103/0x10d
[ 1103.521211]  [<c167f085>] ? restore_all+0xf/0xf
[ 1103.521211]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.521211]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.521211]  [<c101d8ab>] do_async_page_fault+0x23/0x5f
[ 1103.521211]  [<c167f5cc>] error_code+0x6c/0x80
[ 1103.521211] Rebooting in 10 seconds..
--
[    4.999346] Freeing unused kernel memory: 460k freed
[ 1104.515803] BUG: soft lockup - CPU#1 stuck for 1025s! [swapper/1:0]
[ 1104.515803] irq event stamp: 12116
[ 1104.515803] hardirqs last  enabled at (12113): [<c167f085>] restore_all_notrace+0x0/0x18
[ 1104.515803] hardirqs last disabled at (12114): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.515803] softirqs last  enabled at (12116): [<c102e637>] _local_bh_enable+0xd/0xf
[ 1104.515803] softirqs last disabled at (12115): [<c102eac7>] irq_enter+0x2c/0x4d
[ 1104.515803] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.515803] EIP: 0060:[<c101da32>] EFLAGS: 00000286 CPU: 1
[ 1104.515803] EIP is at native_safe_halt+0x5/0x7
[ 1104.515803] EAX: ce06a260 EBX: ffffffff ECX: ce06a260 EDX: 00000007
[ 1104.515803] ESI: c19716f0 EDI: ffff8b9d EBP: ce071f68 ESP: ce071f68
[ 1104.515803]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.515803] CR0: 8005003b CR2: b76c7fb8 CR3: 0ee89000 CR4: 00000690
[ 1104.515803] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.522791] [sched_delayed] sched: RT throttling activated
[ 1104.527860] DR6: ffff0ff0 DR7: 00000400
[ 1104.527860] Process swapper/1 (pid: 0, ti=ce070000 task=ce06a260 task.ti=ce070000)
[ 1104.527860] Stack:
[ 1104.527860]  ce071f70 c1008a09 ce071f88 c1009250 d2ce2da2 e131b2c1 1b7c62b3 cfff9fa0
[ 1104.527860]  ce071fb4 c166ed10 00000000 00000000 00000000 00000000 8bab6b51 1b7c62b3
[ 1104.527860]  01000800 0000d248 00000000 00000000 00000000 00000000 00000000 00000000
[ 1104.527860] Call Trace:
[ 1104.527860]  [<c1008a09>] default_idle+0x2f/0x55
[ 1104.527860]  [<c1009250>] cpu_idle+0xc9/0xfb
[ 1104.527860]  [<c166ed10>] start_secondary+0x292/0x29a
[ 1104.527860] Code: 0f 22 e0 5d c3 55 89 e5 0f 09 5d c3 55 89 e5 9c 58 5d c3 55 89 e5 50 9d 5d c3 55 89 e5 fa 5d c3 55 89 e5 fb 5d c3 55 89 e5 fb f4 <5d> c3 55 89 e5 f4 5d c3 55 89 e5 57 56 53 53 89 c6 89 55 f0 89
[ 1104.527860] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.527860] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.527860] Call Trace:
[ 1104.527860]  [<c1673678>] panic+0x97/0x1aa
[ 1104.527860]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.527860]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.527860]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.527860]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.527860]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.527860]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.527860]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.527860]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.527860]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.527860]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.527860]  [<c101da32>] ? native_safe_halt+0x5/0x7
[ 1104.527860]  [<c1008a09>] default_idle+0x2f/0x55
[ 1104.527860]  [<c1009250>] cpu_idle+0xc9/0xfb
[ 1104.527860]  [<c166ed10>] start_secondary+0x292/0x29a
[ 1104.527860] Rebooting in 10 seconds..
--
[    8.316068] Freeing unused kernel memory: 460k freed
[   76.723703] BUG: soft lockup - CPU#1 stuck for 64s! [sh:133]
[   76.723703] irq event stamp: 1490
[   76.723703] hardirqs last  enabled at (1489): [<c101e5d2>] do_page_fault+0x98/0x36e
[   76.723703] hardirqs last disabled at (1490): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[   76.723703] softirqs last  enabled at (0): [<c1026857>] copy_process+0x465/0xf8a
[   76.723703] softirqs last disabled at (0): [<  (null)>]   (null)
[   76.723703] Pid: 133, comm: sh Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[   76.723703] EIP: 0060:[<c10675e9>] EFLAGS: 00000282 CPU: 1
[   76.723703] EIP is at lock_acquire+0xb4/0xc0
[   76.723703] EAX: 00000282 EBX: ce8ab390 ECX: f46fc012 EDX: f46fc012
[   76.730213] [sched_delayed] sched: RT throttling activated
[   76.723703] ESI: 00000000 EDI: c19136b8 EBP: ce893e48 ESP: ce893e1c
[   76.723703]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[   76.723703] CR0: 8005003b CR2: b752d2ad CR3: 0e889000 CR4: 00000690
[   76.723703] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[   76.723703] DR6: ffff0ff0 DR7: 00000400
[   76.723703] Process sh (pid: 133, ti=ce892000 task=ce8ab390 task.ti=ce892000)
[   76.723703] Stack:
[   76.723703]  00000002 00000001 00000000 00000000 c10b402b 00000000 00000000 00000282
[   76.723703]  ce69cae0 00000000 0aed7025 ce893e6c c10b4058 00000002 00000001 00000000
[   76.723703]  c10b402b 00000000 c10ac260 ce69cae0 ce893eb8 c10ac2ee 00000000 ce870a34
[   76.723703] Call Trace:
[   76.723703]  [<c10b402b>] ? page_add_file_rmap+0xff/0x153
[   76.723703]  [<c10b4058>] page_add_file_rmap+0x12c/0x153
[   76.723703]  [<c10b402b>] ? page_add_file_rmap+0xff/0x153
[   76.723703]  [<c10ac260>] ? __do_fault+0x222/0x3c1
[   76.723703]  [<c10ac2ee>] __do_fault+0x2b0/0x3c1
[   76.723703]  [<c10aea20>] handle_pte_fault+0x2e6/0x7a7
[   76.723703]  [<c10aef82>] handle_mm_fault+0xa1/0xb0
[   76.723703]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[   76.723703]  [<c101e870>] do_page_fault+0x336/0x36e
[   76.723703]  [<c10af956>] ? might_fault+0x7b/0x99
[   76.723703]  [<c100b0ca>] ? set_tls_desc+0x14a/0x15d
[   76.723703]  [<c167f085>] ? restore_all+0xf/0xf
[   76.723703]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[   76.723703]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[   76.723703]  [<c101d8ab>] do_async_page_fault+0x23/0x5f
[   76.723703]  [<c167f5cc>] error_code+0x6c/0x80
[   76.723703] Code: 45 08 89 04 24 8b 4d ec 89 f2 89 f8 e8 57 e7 ff ff c7 83 58 04 00 00 00 00 00 00 83 3d 8c 20 91 c1 00 75 02 0f 0b 8b 45 f0 50 9d <8d> 74 26 00 83 c4 20 5b 5e 5f 5d c3 55 89 e5 57 56 53 83 ec 1c
[   76.723703] Kernel panic - not syncing: softlockup: hung tasks
[   76.723703] Pid: 133, comm: sh Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[   76.723703] Call Trace:
[   76.723703]  [<c1673678>] panic+0x97/0x1aa
[   76.723703]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[   76.723703]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[   76.723703]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[   76.723703]  [<c103205a>] run_local_timers+0x8/0x14
[   76.723703]  [<c103208a>] update_process_times+0x24/0x58
[   76.723703]  [<c106276e>] tick_periodic+0x66/0x72
[   76.723703]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[   76.723703]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[   76.723703]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[   76.723703]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[   76.723703]  [<c10675e9>] ? lock_acquire+0xb4/0xc0
[   76.723703]  [<c10b402b>] ? page_add_file_rmap+0xff/0x153
[   76.723703]  [<c10b4058>] page_add_file_rmap+0x12c/0x153
[   76.838708]  [<c10b402b>] ? page_add_file_rmap+0xff/0x153
[   76.838708]  [<c10ac260>] ? __do_fault+0x222/0x3c1
[   76.838708]  [<c10ac2ee>] __do_fault+0x2b0/0x3c1
[   76.838708]  [<c10aea20>] handle_pte_fault+0x2e6/0x7a7
[   76.838708]  [<c10aef82>] handle_mm_fault+0xa1/0xb0
[   76.838708]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[   76.838708]  [<c101e870>] do_page_fault+0x336/0x36e
[   76.838708]  [<c10af956>] ? might_fault+0x7b/0x99
[   76.838708]  [<c100b0ca>] ? set_tls_desc+0x14a/0x15d
[   76.838708]  [<c167f085>] ? restore_all+0xf/0xf
[   76.838708]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[   76.838708]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[   76.838708]  [<c101d8ab>] do_async_page_fault+0x23/0x5f
[   76.838708]  [<c167f5cc>] error_code+0x6c/0x80
[   76.838708] Rebooting in 10 seconds..
--
[    4.225859] Freeing unused kernel memory: 460k freed
[ 1103.516157] BUG: soft lockup - CPU#1 stuck for 1024s! [sh:134]
[ 1103.516157] irq event stamp: 3580
[ 1103.516157] hardirqs last  enabled at (3579): [<c167caf0>] __mutex_unlock_slowpath+0x114/0x12c
[ 1103.516157] hardirqs last disabled at (3580): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1103.516157] softirqs last  enabled at (2952): [<c1601126>] unix_sock_destructor+0xae/0xb3
[ 1103.516157] softirqs last disabled at (2950): [<c1601113>] unix_sock_destructor+0x9b/0xb3
[ 1103.516157] Pid: 134, comm: sh Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1103.516157] EIP: 0060:[<c10675e9>] EFLAGS: 00000282 CPU: 1
[ 1103.516157] EIP is at lock_acquire+0xb4/0xc0
[ 1103.516157] EAX: 00000282 EBX: cf891130 ECX: 006f437b EDX: 006f437b
[ 1103.524872] [sched_delayed] sched: RT throttling activated
--
[    5.072722] Freeing unused kernel memory: 460k freed
[ 1104.521453] BUG: soft lockup - CPU#1 stuck for 1024s! [trinity:133]
[ 1104.521453] irq event stamp: 5428
[ 1104.521453] hardirqs last  enabled at (5427): [<c109c1fc>] get_page_from_freelist+0x2ca/0x456
[ 1104.521453] hardirqs last disabled at (5428): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.521453] softirqs last  enabled at (3010): [<c1601126>] unix_sock_destructor+0xae/0xb3
[ 1104.521453] softirqs last disabled at (3008): [<c1601113>] unix_sock_destructor+0x9b/0xb3
[ 1104.521453] Pid: 133, comm: trinity Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.521453] EIP: 0060:[<c10675e9>] EFLAGS: 00000282 CPU: 1
[ 1104.530421] [sched_delayed] sched: RT throttling activated
[ 1104.521453] EIP is at lock_acquire+0xb4/0xc0
[ 1104.521453] EAX: 00000282 EBX: cf869130 ECX: 006f8012 EDX: 006f8012
[ 1104.521453] ESI: 00000000 EDI: c19136b8 EBP: ce8cfe04 ESP: ce8cfdd8
[ 1104.521453]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.521453] CR0: 8005003b CR2: 086e4e68 CR3: 0e883000 CR4: 00000690
[ 1104.521453] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.521453] DR6: ffff0ff0 DR7: 00000400
[ 1104.521453] Process trinity (pid: 133, ti=ce8ce000 task=cf869130 task.ti=ce8ce000)
[ 1104.521453] Stack:
[ 1104.521453]  00000002 00000001 00000000 00000000 c1096b46 00000000 00000000 00000282
[ 1104.521453]  c70b97c8 00000a5c 0000069c ce8cfe2c c1096b73 00000002 00000001 00000000
[ 1104.521453]  c1096b46 00000000 c6d5a500 00000a5c 00000000 ce8cfe6c c1096e6d 00000000
[ 1104.521453] Call Trace:
[ 1104.521453]  [<c1096b46>] ? find_get_page+0x17/0x1a6
[ 1104.521453]  [<c1096b73>] find_get_page+0x44/0x1a6
[ 1104.521453]  [<c1096b46>] ? find_get_page+0x17/0x1a6
[ 1104.521453]  [<c1096e6d>] filemap_fault+0x76/0x369
[ 1104.521453]  [<c10ac0e4>] __do_fault+0xa6/0x3c1
[ 1104.521453]  [<c10aea20>] handle_pte_fault+0x2e6/0x7a7
[ 1104.521453]  [<c10aef82>] handle_mm_fault+0xa1/0xb0
[ 1104.521453]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.521453]  [<c101e870>] do_page_fault+0x336/0x36e
[ 1104.521453]  [<c1043362>] ? up_write+0x16/0x2b
[ 1104.521453]  [<c10a69b4>] ? vm_mmap_pgoff+0x50/0x59
[ 1104.521453]  [<c167f085>] ? restore_all+0xf/0xf
[ 1104.521453]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.521453]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.521453]  [<c101d8ab>] do_async_page_fault+0x23/0x5f
[ 1104.521453]  [<c167f5cc>] error_code+0x6c/0x80
[ 1104.521453] Code: 45 08 89 04 24 8b 4d ec 89 f2 89 f8 e8 57 e7 ff ff c7 83 58 04 00 00 00 00 00 00 83 3d 8c 20 91 c1 00 75 02 0f 0b 8b 45 f0 50 9d <8d> 74 26 00 83 c4 20 5b 5e 5f 5d c3 55 89 e5 57 56 53 83 ec 1c
[ 1104.521453] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.521453] Pid: 133, comm: trinity Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.521453] Call Trace:
[ 1104.521453]  [<c1673678>] panic+0x97/0x1aa
[ 1104.521453]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.521453]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.521453]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.521453]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.521453]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.521453]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.521453]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.521453]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.521453]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.521453]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.521453]  [<c10675e9>] ? lock_acquire+0xb4/0xc0
[ 1104.521453]  [<c1096b46>] ? find_get_page+0x17/0x1a6
[ 1104.521453]  [<c1096b73>] find_get_page+0x44/0x1a6
[ 1104.521453]  [<c1096b46>] ? find_get_page+0x17/0x1a6
[ 1104.521453]  [<c1096e6d>] filemap_fault+0x76/0x369
[ 1104.521453]  [<c10ac0e4>] __do_fault+0xa6/0x3c1
[ 1104.521453]  [<c10aea20>] handle_pte_fault+0x2e6/0x7a7
[ 1104.521453]  [<c10aef82>] handle_mm_fault+0xa1/0xb0
[ 1104.521453]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.521453]  [<c101e870>] do_page_fault+0x336/0x36e
[ 1104.521453]  [<c1043362>] ? up_write+0x16/0x2b
[ 1104.521453]  [<c10a69b4>] ? vm_mmap_pgoff+0x50/0x59
[ 1104.521453]  [<c167f085>] ? restore_all+0xf/0xf
[ 1104.521453]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.521453]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.521453]  [<c101d8ab>] do_async_page_fault+0x23/0x5f
[ 1104.521453]  [<c167f5cc>] error_code+0x6c/0x80
[ 1104.521453] Rebooting in 10 seconds..
--
[    6.167052] Freeing unused kernel memory: 460k freed
[ 1105.525217] BUG: soft lockup - CPU#1 stuck for 1024s! [umount:132]
[ 1105.525217] irq event stamp: 1012
[ 1105.525217] hardirqs last  enabled at (1011): [<c101e5d2>] do_page_fault+0x98/0x36e
[ 1105.525217] hardirqs last disabled at (1012): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1105.525217] softirqs last  enabled at (604): [ 1105.528841] [sched_delayed] sched: RT throttling activated
[<c102e8ff>] __do_softirq+0x131/0x153
[ 1105.525217] softirqs last disabled at (583): [<c1003f90>] do_softirq+0x74/0xdf
[ 1105.525217] Pid: 132, comm: umount Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1105.525217] EIP: 0060:[<c10675e9>] EFLAGS: 00000282 CPU: 1
[ 1105.525217] EIP is at lock_acquire+0xb4/0xc0
[ 1105.525217] EAX: 00000282 EBX: cf893390 ECX: 006f637c EDX: 006f637c
[ 1105.525217] ESI: 00000000 EDI: ce8aa544 EBP: ce8afe50 ESP: ce8afe24
[ 1105.525217]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1105.525217] CR0: 8005003b CR2: b7749073 CR3: 0e89a000 CR4: 00000690
[ 1105.525217] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1105.525217] DR6: ffff0ff0 DR7: 00000400
[ 1105.525217] Process umount (pid: 132, ti=ce8ae000 task=cf893390 task.ti=ce8ae000)
[ 1105.525217] Stack:
[ 1105.525217]  00000000 00000002 00000000 00000000 c10ac260 00000000 00000000 00000282
[ 1105.525217]  ce8aa534 00000000 00000009 ce8afe6c c167e6a2 00000000 00000002 00000000
[ 1105.525217]  c10ac260 ce689ca0 ce8afeb8 c10ac260 00000000 ce8aa534 ce89dd24 00000000
[ 1105.525217] Call Trace:
[ 1105.525217]  [<c10ac260>] ? __do_fault+0x222/0x3c1
[ 1105.525217]  [<c167e6a2>] _raw_spin_lock+0x3d/0x4a
[ 1105.525217]  [<c10ac260>] ? __do_fault+0x222/0x3c1
[ 1105.525217]  [<c10ac260>] __do_fault+0x222/0x3c1
[ 1105.525217]  [<c10aea20>] handle_pte_fault+0x2e6/0x7a7
[ 1105.525217]  [<c10aef82>] handle_mm_fault+0xa1/0xb0
[ 1105.525217]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1105.525217]  [<c101e870>] do_page_fault+0x336/0x36e
[ 1105.525217]  [<c1088b28>] ? call_rcu_sched+0x15/0x17
[ 1105.525217]  [<c10449ee>] ? __put_cred+0x35/0x37
[ 1105.525217]  [<c167f085>] ? restore_all+0xf/0xf
[ 1105.525217]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1105.525217]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1105.525217]  [<c101d8ab>] do_async_page_fault+0x23/0x5f
[ 1105.525217]  [<c167f5cc>] error_code+0x6c/0x80
[ 1105.525217] Code: 45 08 89 04 24 8b 4d ec 89 f2 89 f8 e8 57 e7 ff ff c7 83 58 04 00 00 00 00 00 00 83 3d 8c 20 91 c1 00 75 02 0f 0b 8b 45 f0 50 9d <8d> 74 26 00 83 c4 20 5b 5e 5f 5d c3 55 89 e5 57 56 53 83 ec 1c
[ 1105.525217] Kernel panic - not syncing: softlockup: hung tasks
[ 1105.525217] Pid: 132, comm: umount Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1105.525217] Call Trace:
[ 1105.525217]  [<c1673678>] panic+0x97/0x1aa
[ 1105.525217]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1105.525217]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1105.525217]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1105.525217]  [<c103205a>] run_local_timers+0x8/0x14
[ 1105.525217]  [<c103208a>] update_process_times+0x24/0x58
[ 1105.525217]  [<c106276e>] tick_periodic+0x66/0x72
[ 1105.525217]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1105.525217]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1105.525217]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1105.525217]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1105.525217]  [<c10675e9>] ? lock_acquire+0xb4/0xc0
[ 1105.525217]  [<c10ac260>] ? __do_fault+0x222/0x3c1
[ 1105.525217]  [<c167e6a2>] _raw_spin_lock+0x3d/0x4a
[ 1105.525217]  [<c10ac260>] ? __do_fault+0x222/0x3c1
[ 1105.525217]  [<c10ac260>] __do_fault+0x222/0x3c1
[ 1105.525217]  [<c10aea20>] handle_pte_fault+0x2e6/0x7a7
[ 1105.525217]  [<c10aef82>] handle_mm_fault+0xa1/0xb0
[ 1105.525217]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1105.525217]  [<c101e870>] do_page_fault+0x336/0x36e
[ 1105.525217]  [<c1088b28>] ? call_rcu_sched+0x15/0x17
[ 1105.525217]  [<c10449ee>] ? __put_cred+0x35/0x37
[ 1105.525217]  [<c167f085>] ? restore_all+0xf/0xf
[ 1105.525217]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1105.525217]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1105.525217]  [<c101d8ab>] do_async_page_fault+0x23/0x5f
[ 1105.525217]  [<c167f5cc>] error_code+0x6c/0x80
[ 1105.525217] Rebooting in 10 seconds..
--
[    4.943616] Freeing unused kernel memory: 460k freed
[ 1103.516727] BUG: soft lockup - CPU#1 stuck for 1023s! [99-trinity:129]
[ 1103.516727] irq event stamp: 632
[ 1103.516727] hardirqs last  enabled at (631): [<c167caf0>] __mutex_unlock_slowpath+0x114/0x12c
[ 1103.516727] hardirqs last disabled at (632): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1103.516727] softirqs last  enabled at (0): [<c1026857>] copy_process+0x465/0xf8a
[ 1103.516727] softirqs last disabled at (0): [ 1103.522351] [sched_delayed] sched: RT throttling activated
[<  (null)>]   (null)
[ 1103.516727] Pid: 129, comm: 99-trinity Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1103.516727] EIP: 0060:[<c1065d13>] EFLAGS: 00000282 CPU: 1
[ 1103.516727] EIP is at lock_is_held+0x9f/0xb2
[ 1103.516727] EAX: 00000282 EBX: 00000002 ECX: 00000000 EDX: 00000000
[ 1103.516727] ESI: cf899130 EDI: c19136a8 EBP: ce8bbddc ESP: ce8bbdcc
[ 1103.516727]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.516727] CR0: 8005003b CR2: 0807cbf0 CR3: 0e8b0000 CR4: 00000690
[ 1103.516727] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.516727] DR6: ffff0ff0 DR7: 00000400
[ 1103.516727] Process 99-trinity (pid: 129, ti=ce8ba000 task=cf899130 task.ti=ce8ba000)
[ 1103.516727] Stack:
[ 1103.516727]  00000282 000003c8 c182c540 00000000 ce8bbe04 c104bac6 00000001 00000282
[ 1103.516727]  ce828a34 00000000 ce828a34 023d8045 00000000 ce8b254c ce8bbe64 c10ada87
[ 1103.516727]  00000040 00000006 c1bcc504 ce829210 ce828a34 ce8291b4 ce8b2548 ce8acb74
[ 1103.516727] Call Trace:
[ 1103.516727]  [<c104bac6>] __might_sleep+0x66/0x1cd
[ 1103.516727]  [<c10ada87>] copy_pte_range+0x382/0x3bf
[ 1103.516727]  [<c10adb8c>] copy_page_range+0xc8/0xdc
[ 1103.516727]  [<c10262b7>] dup_mm+0x285/0x3a5
[ 1103.516727]  [<c1026c73>] copy_process+0x881/0xf8a
[ 1103.516727]  [<c1027493>] do_fork+0xef/0x257
[ 1103.516727]  [<c10af96b>] ? might_fault+0x90/0x99
[ 1103.516727]  [<c12b3908>] ? copy_to_user+0x3f/0x46
[ 1103.516727]  [<c100909b>] sys_clone+0x2b/0x32
[ 1103.516727]  [<c167f9f5>] ptregs_clone+0x15/0x20
[ 1103.516727]  [<c167f863>] ? sysenter_do_call+0x12/0x38
[ 1103.516727] Code: 7c e2 31 c9 eb 05 b9 01 00 00 00 64 a1 28 06 9e c1 c7 80 58 04 00 00 00 00 00 00 83 3d 8c 20 91 c1 00 75 02 0f 0b 8b 45 f0 50 9d <8d> 74 26 00 eb 05 b9 01 00 00 00 89 c8 5b 5b 5e 5f 5d c3 55 89
[ 1103.516727] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.516727] Pid: 129, comm: 99-trinity Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1103.516727] Call Trace:
[ 1103.516727]  [<c1673678>] panic+0x97/0x1aa
[ 1103.516727]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1103.516727]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.516727]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1103.516727]  [<c103205a>] run_local_timers+0x8/0x14
[ 1103.516727]  [<c103208a>] update_process_times+0x24/0x58
[ 1103.516727]  [<c106276e>] tick_periodic+0x66/0x72
[ 1103.516727]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1103.516727]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.516727]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.516727]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1103.516727]  [<c1065d13>] ? lock_is_held+0x9f/0xb2
[ 1103.516727]  [<c104bac6>] __might_sleep+0x66/0x1cd
[ 1103.516727]  [<c10ada87>] copy_pte_range+0x382/0x3bf
[ 1103.516727]  [<c10adb8c>] copy_page_range+0xc8/0xdc
[ 1103.516727]  [<c10262b7>] dup_mm+0x285/0x3a5
[ 1103.516727]  [<c1026c73>] copy_process+0x881/0xf8a
[ 1103.516727]  [<c1027493>] do_fork+0xef/0x257
[ 1103.516727]  [<c10af96b>] ? might_fault+0x90/0x99
[ 1103.516727]  [<c12b3908>] ? copy_to_user+0x3f/0x46
[ 1103.516727]  [<c100909b>] sys_clone+0x2b/0x32
[ 1103.516727]  [<c167f9f5>] ptregs_clone+0x15/0x20
[ 1103.516727]  [<c167f863>] ? sysenter_do_call+0x12/0x38
[ 1103.516727] Rebooting in 10 seconds..
--
[ 1103.522466] [sched_delayed] sched: RT throttling activated
[ 1103.514897] BUG: soft lockup - CPU#1 stuck for 1023s! [init:1]
[ 1103.514897] irq event stamp: 746164
[ 1103.514897] hardirqs last  enabled at (746163): [<c167e884>] _raw_spin_unlock_irq+0x22/0x40
[ 1103.514897] hardirqs last disabled at (746164): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1103.514897] softirqs last  enabled at (745922): [<c102e8ff>] __do_softirq+0x131/0x153
[ 1103.514897] softirqs last disabled at (745919): [<c1003f90>] do_softirq+0x74/0xdf
[ 1103.514897] Pid: 1, comm: init Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1103.514897] EIP: 0060:[<c167e891>] EFLAGS: 00000286 CPU: 1
[ 1103.514897] EIP is at _raw_spin_unlock_irq+0x2f/0x40
[ 1103.514897] EAX: ce048000 EBX: ce848504 ECX: 00000000 EDX: 00000007
[ 1103.514897] ESI: ce848158 EDI: ce043f94 EBP: ce043e8c ESP: ce043e88
[ 1103.514897]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.514897] CR0: 8005003b CR2: b75f1fb8 CR3: 0eecd000 CR4: 00000690
[ 1103.514897] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.514897] DR6: ffff0ff0 DR7: 00000400
[ 1103.514897] Process init (pid: 1, ti=ce042000 task=ce048000 task.ti=ce042000)
[ 1103.514897] Stack:
[ 1103.514897]  00000011 ce043ecc c1035a94 0000037b 0000037b 00000000 ce04845c ce048374
[ 1103.514897]  ce043f80 ce848000 ce044000 ce043fb4 ce848504 00000001 ce043fb4 bfbc73a0
[ 1103.514897]  00000000 ce043fa0 c1001cf3 00000000 00000000 ce844080 ce04845c ce048000
[ 1103.514897] Call Trace:
[ 1103.514897]  [<c1035a94>] get_signal_to_deliver+0x670/0x67a
[ 1103.514897]  [<c1001cf3>] do_signal+0x25/0x6a5
[ 1103.514897]  [<c10671f9>] ? lock_release_non_nested+0x7e/0x21b
[ 1103.514897]  [<c167e841>] ? _raw_spin_unlock_irqrestore+0x42/0x63
[ 1103.514897]  [<c10af956>] ? might_fault+0x7b/0x99
[ 1103.514897]  [<c10024f2>] do_notify_resume+0x22/0x56
[ 1103.514897]  [<c167f160>] work_notifysig+0x2b/0x3b
[ 1103.514897] Code: 89 c3 8d 40 10 8b 4d 04 ba 01 00 00 00 e8 1e 8b 9e ff 89 d8 e8 38 40 c4 ff e8 69 6a 9e ff 83 3d 94 20 91 c1 00 75 02 0f 0b fb 90 <8d> 74 26 00 89 e0 25 00 e0 ff ff ff 48 14 5b 5d c3 55 89 e5 53
[ 1103.514897] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.514897] Pid: 1, comm: init Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1103.514897] Call Trace:
[ 1103.514897]  [<c1673678>] panic+0x97/0x1aa
[ 1103.514897]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1103.514897]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.514897]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1103.514897]  [<c103205a>] run_local_timers+0x8/0x14
[ 1103.514897]  [<c103208a>] update_process_times+0x24/0x58
[ 1103.514897]  [<c106276e>] tick_periodic+0x66/0x72
[ 1103.514897]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1103.514897]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.514897]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.514897]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1103.514897]  [<c167e891>] ? _raw_spin_unlock_irq+0x2f/0x40
[ 1103.514897]  [<c1035a94>] get_signal_to_deliver+0x670/0x67a
[ 1103.514897]  [<c1001cf3>] do_signal+0x25/0x6a5
[ 1103.514897]  [<c10671f9>] ? lock_release_non_nested+0x7e/0x21b
[ 1103.514897]  [<c167e841>] ? _raw_spin_unlock_irqrestore+0x42/0x63
[ 1103.514897]  [<c10af956>] ? might_fault+0x7b/0x99
[ 1103.514897]  [<c10024f2>] do_notify_resume+0x22/0x56
[ 1103.514897]  [<c167f160>] work_notifysig+0x2b/0x3b
[ 1103.514897] Rebooting in 10 seconds..
--
[    5.094780] Freeing unused kernel memory: 460k freed
[ 1104.514155] BUG: soft lockup - CPU#1 stuck for 1024s! [sh:133]
[ 1104.514155] irq event stamp: 3594
[ 1104.514155] hardirqs last  enabled at (3593): [<c106831d>] debug_check_no_locks_freed+0x14b/0x167
[ 1104.514155] hardirqs last disabled at (3594): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.514155] softirqs last  enabled at (3024): [<c1601126>] unix_sock_destructor+0xae/0xb3
[ 1104.514155] softirqs last disabled at (3022): [<c1601113>] unix_sock_destructor+0x9b/0xb3
[ 1104.514155] Pid: 133, comm: sh Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.522513] [sched_delayed] sched: RT throttling activated
[ 1104.514155] EIP: 0060:[<c10d0574>] EFLAGS: 00000246 CPU: 1
[ 1104.514155] EIP is at generic_permission+0x2d/0x22b
[ 1104.514155] EAX: ce881f80 EBX: 00000000 ECX: 00000000 EDX: 00000001
[ 1104.514155] ESI: cf891130 EDI: cb4b78c0 EBP: ce8bbe24 ESP: ce8bbe10
[ 1104.514155]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.514155] CR0: 80050033 CR2: b74f3267 CR3: 0e88e000 CR4: 00000690
[ 1104.514155] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.514155] DR6: ffff0ff0 DR7: 00000400
[ 1104.514155] Process sh (pid: 133, ti=ce8ba000 task=cf891130 task.ti=ce8ba000)
[ 1104.514155] Stack:
[ 1104.514155]  000041ed 00000001 cb4b78c0 cf891130 ce8bbef8 ce8bbe38 c10d07c3 00000000
[ 1104.514155]  ce028800 cf891130 ce8bbe44 c10d0808 ce8bbef8 ce8bbe98 c10d18cb cf891130
[ 1104.514155]  ce8bbe64 c1098035 cf891130 cf891130 ce8bbe90 cf891130 cae37000 c1021b4a
[ 1104.514155] Call Trace:
[ 1104.514155]  [<c10d07c3>] __inode_permission+0x51/0x56
[ 1104.514155]  [<c10d0808>] inode_permission+0x40/0x42
[ 1104.514155]  [<c10d18cb>] link_path_walk+0x68/0x64b
[ 1104.514155]  [<c1098035>] ? read_cache_page_async+0x22/0x27
[ 1104.514155]  [<c1021b4a>] ? kmap+0x5a/0x5c
[ 1104.514155]  [<c10d04a9>] ? page_getlink.isra.27+0x34/0x4f
[ 1104.514155]  [<c10d3631>] path_openat+0x256/0x352
[ 1104.514155]  [<c10d3753>] do_filp_open+0x26/0x67
[ 1104.514155]  [<c10deb0e>] ? alloc_fd+0x175/0x180
[ 1104.514155]  [<c10c7541>] do_sys_open+0xe6/0x171
[ 1104.514155]  [<c1088b28>] ? call_rcu_sched+0x15/0x17
[ 1104.514155]  [<c10449ee>] ? __put_cred+0x35/0x37
[ 1104.514155]  [<c10c75f0>] sys_open+0x24/0x2a
[ 1104.514155]  [<c167f04c>] syscall_call+0x7/0xb
[ 1104.514155] Code: e5 57 56 53 83 ec 08 89 c7 89 55 f0 0f b7 00 89 45 ec 8b 5f 04 e8 81 d6 f6 ff 64 a1 28 06 9e c1 8b 80 c0 02 00 00 39 58 1c 75 09 <c1> 6d ec 06 e9 66 01 00 00 8b 47 1c f6 40 32 01 0f 84 49 01 00
[ 1104.514155] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.514155] Pid: 133, comm: sh Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.514155] Call Trace:
[ 1104.514155]  [<c1673678>] panic+0x97/0x1aa
[ 1104.514155]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.514155]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.552525]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.552525]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.552525]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.552525]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.552525]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.552525]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.552525]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.552525]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.552525]  [<c10d0574>] ? generic_permission+0x2d/0x22b
[ 1104.552525]  [<c10d07c3>] __inode_permission+0x51/0x56
[ 1104.552525]  [<c10d0808>] inode_permission+0x40/0x42
[ 1104.552525]  [<c10d18cb>] link_path_walk+0x68/0x64b
[ 1104.552525]  [<c1098035>] ? read_cache_page_async+0x22/0x27
[ 1104.552525]  [<c1021b4a>] ? kmap+0x5a/0x5c
[ 1104.552525]  [<c10d04a9>] ? page_getlink.isra.27+0x34/0x4f
[ 1104.552525]  [<c10d3631>] path_openat+0x256/0x352
[ 1104.552525]  [<c10d3753>] do_filp_open+0x26/0x67
[ 1104.552525]  [<c10deb0e>] ? alloc_fd+0x175/0x180
[ 1104.552525]  [<c10c7541>] do_sys_open+0xe6/0x171
[ 1104.552525]  [<c1088b28>] ? call_rcu_sched+0x15/0x17
[ 1104.552525]  [<c10449ee>] ? __put_cred+0x35/0x37
[ 1104.552525]  [<c10c75f0>] sys_open+0x24/0x2a
[ 1104.552525]  [<c167f04c>] syscall_call+0x7/0xb
[ 1104.552525] Rebooting in 10 seconds..
--
[    4.485480] Freeing unused kernel memory: 460k freed
[ 1103.513144] BUG: soft lockup - CPU#1 stuck for 1024s! [init:132]
[ 1103.513144] irq event stamp: 544
[ 1103.513144] hardirqs last  enabled at (543): [<c106831d>] debug_check_no_locks_freed+0x14b/0x167
[ 1103.513144] hardirqs last disabled at (544): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1103.513144] softirqs last  enabled at (122): [<c102e8ff>] __do_softirq+0x131/0x153
[ 1103.513144] softirqs last disabled at (95): [<c1003f90>] do_softirq+0x74/0xdf
[ 1103.513144] Pid: 132, comm: init Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1103.513144] EIP: 0060:[<c1065d13>] EFLAGS: 00000282 CPU: 1
[ 1103.513144] EIP is at lock_is_held+0x9f/0xb2
[ 1103.513144] EAX: 00000282 EBX: 00000001 ECX: 00000000 EDX: 00000000
[ 1103.513144] ESI: cf8c9130 EDI: c1913698 EBP: ce8dbcf4 ESP: ce8dbce4
[ 1103.513144]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.522293] [sched_delayed] sched: RT throttling activated
[ 1103.513144] CR0: 8005003b CR2: b7776000 CR3: 0e892000 CR4: 00000690
[ 1103.513144] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.513144] DR6: ffff0ff0 DR7: 00000400
[ 1103.513144] Process init (pid: 132, ti=ce8da000 task=cf8c9130 task.ti=ce8da000)
[ 1103.513144] Stack:
[ 1103.513144]  00000282 00000452 c182a2ca 00000000 ce8dbd1c c104bb01 ce8dbe34 00000001
[ 1103.513144]  ce8da000 c784a100 00000080 00000000 ffffffff 00000000 ce8dbdac c1097c2a
[ 1103.513144]  00000001 c1918cd4 ce8dbd60 00000282 00000000 00000000 00000001 ce891c84
[ 1103.513144] Call Trace:
[ 1103.513144]  [<c104bb01>] __might_sleep+0xa1/0x1cd
[ 1103.513144]  [<c1097c2a>] generic_file_aio_read+0x268/0x5f1
[ 1103.513144]  [<c1045e49>] ? lg_local_unlock+0x16/0x2f
[ 1103.513144]  [<c10df58b>] ? mntput_no_expire+0x2b/0xfb
[ 1103.513144]  [<c10c78d1>] do_sync_read+0x75/0xb0
[ 1103.513144]  [<c10c7faa>] vfs_read+0x8b/0x10d
[ 1103.513144]  [<c10c785c>] ? wait_on_retry_sync_kiocb+0x3f/0x3f
[ 1103.513144]  [<c10cd050>] kernel_read+0x30/0x41
[ 1103.513144]  [<c10f985c>] load_elf_binary+0x1ea/0xfbd
[ 1103.513144]  [<c10cdaa5>] search_binary_handler+0x14d/0x22b
[ 1103.513144]  [<c10cd992>] ? search_binary_handler+0x3a/0x22b
[ 1103.513144]  [<c10f9672>] ? load_elf_library+0x1bb/0x1bb
[ 1103.513144]  [<c10cdf0e>] do_execve+0x38b/0x456
[ 1103.513144]  [<c10cdc6e>] ? do_execve+0xeb/0x456
[ 1103.513144]  [<c1009165>] sys_execve+0x31/0x53
[ 1103.513144]  [<c167f962>] ptregs_execve+0x12/0x20
[ 1103.513144]  [<c167f863>] ? sysenter_do_call+0x12/0x38
[ 1103.513144] Code: 7c e2 31 c9 eb 05 b9 01 00 00 00 64 a1 28 06 9e c1 c7 80 58 04 00 00 00 00 00 00 83 3d 8c 20 91 c1 00 75 02 0f 0b 8b 45 f0 50 9d <8d> 74 26 00 eb 05 b9 01 00 00 00 89 c8 5b 5b 5e 5f 5d c3 55 89
[ 1103.513144] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.513144] Pid: 132, comm: init Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1103.513144] Call Trace:
[ 1103.513144]  [<c1673678>] panic+0x97/0x1aa
[ 1103.513144]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1103.513144]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.513144]  [<c106716a>] ? __lock_acquire+0x1444/0x1455
[ 1103.513144]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1103.513144]  [<c103205a>] run_local_timers+0x8/0x14
[ 1103.513144]  [<c103208a>] update_process_times+0x24/0x58
[ 1103.513144]  [<c106276e>] tick_periodic+0x66/0x72
[ 1103.513144]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1103.513144]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.513144]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.513144]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1103.513144]  [<c1065d13>] ? lock_is_held+0x9f/0xb2
[ 1103.513144]  [<c104bb01>] __might_sleep+0xa1/0x1cd
[ 1103.513144]  [<c1097c2a>] generic_file_aio_read+0x268/0x5f1
[ 1103.513144]  [<c1045e49>] ? lg_local_unlock+0x16/0x2f
[ 1103.513144]  [<c10df58b>] ? mntput_no_expire+0x2b/0xfb
[ 1103.513144]  [<c10c78d1>] do_sync_read+0x75/0xb0
[ 1103.513144]  [<c10c7faa>] vfs_read+0x8b/0x10d
[ 1103.513144]  [<c10c785c>] ? wait_on_retry_sync_kiocb+0x3f/0x3f
[ 1103.513144]  [<c10cd050>] kernel_read+0x30/0x41
[ 1103.513144]  [<c10f985c>] load_elf_binary+0x1ea/0xfbd
[ 1103.513144]  [<c10cdaa5>] search_binary_handler+0x14d/0x22b
[ 1103.513144]  [<c10cd992>] ? search_binary_handler+0x3a/0x22b
[ 1103.513144]  [<c10f9672>] ? load_elf_library+0x1bb/0x1bb
[ 1103.513144]  [<c10cdf0e>] do_execve+0x38b/0x456
[ 1103.513144]  [<c10cdc6e>] ? do_execve+0xeb/0x456
[ 1103.513144]  [<c1009165>] sys_execve+0x31/0x53
[ 1103.513144]  [<c167f962>] ptregs_execve+0x12/0x20
[ 1103.513144]  [<c167f863>] ? sysenter_do_call+0x12/0x38
[ 1103.513144] Rebooting in 10 seconds..
--
[    5.718013] Freeing unused kernel memory: 460k freed
[ 1104.517767] BUG: soft lockup - CPU#1 stuck for 1023s! [swapper/1:0]
[ 1104.517767] irq event stamp: 8968
[ 1104.517767] hardirqs last  enabled at (8965): [<c167f085>] restore_all_notrace+0x0/0x18
[ 1104.517767] hardirqs last disabled at (8966): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.517767] softirqs last  enabled at (8968): [<c102e637>] _local_bh_enable+0xd/0xf
[ 1104.517767] softirqs last disabled at (8967): [<c102eac7>] irq_enter+0x2c/0x4d
[ 1104.517767] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.517767] EIP: 0060:[<c101da32>] EFLAGS: 00000286 CPU: 1
[ 1104.517767] EIP is at native_safe_halt+0x5/0x7
[ 1104.517767] EAX: ce06a260 EBX: ffffffff ECX: ce06a260 EDX: 00000007
[ 1104.517767] ESI: c19716f0 EDI: ffff8b9f EBP: ce071f68 ESP: ce071f68
[ 1104.517767]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.517767] CR0: 8005003b CR2: b765ebd0 CR3: 0e89b000 CR4: 00000690
[ 1104.517767] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.517767] DR6: ffff0ff0 DR7: 00000400
[ 1104.517767] Process swapper/1 (pid: 0, ti=ce070000 task=ce06a260 task.ti=ce070000)
[ 1104.517767] Stack:
[ 1104.517767]  ce071f70 c1008a09 ce071f88 c1009250 43ed9c2d 52512150 8c9c3063 cfff9fa0
[ 1104.517767]  ce071fb4 c166ed10 00000000 00000000 00000000 00000000 fccb38fe 8c9c3063
[ 1104.517767]  01000800 0000d248 00000000 00000000 00000000 00000000 00000000 00000000
[ 1104.517767] Call Trace:
[ 1104.517767]  [<c1008a09>] default_idle+0x2f/0x55
[ 1104.517767]  [<c1009250>] cpu_idle+0xc9/0xfb
[ 1104.517767]  [<c166ed10>] start_secondary+0x292/0x29a
[ 1104.517767] Code: 0f 22 e0 5d c3 55 89 e5 0f 09 5d c3 55 89 e5 9c 58 5d c3 55 89 e5 50 9d 5d c3 55 89 e5 fa 5d c3 55 89 e5 fb 5d c3 55 89 e5 fb f4 <5d> c3 55 89 e5 f4 5d c3 55 89 e5 57 56 53 53 89 c6 89 55 f0 89
[ 1104.517767] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.517767] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.517767] Call Trace:
[ 1104.517767]  [<c1673678>] panic+0x97/0x1aa
[ 1104.517767]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.517767]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.517767]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.517767]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.517767]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.517767]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.517767]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.517767]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.517767]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.517767]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.517767]  [<c101da32>] ? native_safe_halt+0x5/0x7
[ 1104.517767]  [<c1008a09>] default_idle+0x2f/0x55
[ 1104.517767]  [<c1009250>] cpu_idle+0xc9/0xfb
[ 1104.517767]  [<c166ed10>] start_secondary+0x292/0x29a
mountall: ply-event-loop.c:497: ply_event_loop_new: Assertion `loop->epoll_fd >= 0' failed.
[ 1104.517767] Rebooting in 10 seconds..
--
[ 1104.513744] [sched_delayed] sched: RT throttling activated
[ 1104.514576] BUG: soft lockup - CPU#1 stuck for 1024s! [init:1]
[ 1104.514576] irq event stamp: 747944
[ 1104.514576] hardirqs last  enabled at (747943): [<c167e841>] _raw_spin_unlock_irqrestore+0x42/0x63
[ 1104.514576] hardirqs last disabled at (747944): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.514576] softirqs last  enabled at (744332): [<c102e8ff>] __do_softirq+0x131/0x153
[ 1104.514576] softirqs last disabled at (744329): [<c1003f90>] do_softirq+0x74/0xdf
[ 1104.514576] Pid: 1, comm: init Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.514576] EIP: 0060:[<c167e850>] EFLAGS: 00000282 CPU: 1
[ 1104.514576] EIP is at _raw_spin_unlock_irqrestore+0x51/0x63
[ 1104.514576] EAX: 00000282 EBX: 00000282 ECX: 00000000 EDX: 00000007
[ 1104.514576] ESI: cf8859d8 EDI: 00000001 EBP: ce043dfc ESP: ce043df4
[ 1104.514576]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.514576] CR0: 80050033 CR2: b76a9e60 CR3: 0eebd000 CR4: 00000690
[ 1104.514576] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.514576] DR6: ffff0ff0 DR7: 00000400
[ 1104.514576] Process init (pid: 1, ti=ce042000 task=ce048000 task.ti=ce042000)
[ 1104.514576] Stack:
[ 1104.514576]  00000000 cf8855f0 ce043e20 c104a23a 00000282 cf8859d8 0cb4c000 00000000
[ 1104.514576]  cf8855f0 00000000 00000007 ce043e28 c104a280 ce043e38 c10306b6 cf8855f0
[ 1104.514576]  00000000 ce043ef4 c1031596 003f8011 00000000 ce048474 00000000 ce048000
[ 1104.514576] Call Trace:
[ 1104.514576]  [<c104a23a>] try_to_wake_up+0x248/0x266
[ 1104.514576]  [<c104a280>] wake_up_state+0xa/0xc
[ 1104.514576]  [<c10306b6>] ptrace_resume+0x76/0x7c
[ 1104.514576]  [<c1031596>] ptrace_request+0x368/0x4f8
[ 1104.514576]  [<c106716a>] ? __lock_acquire+0x1444/0x1455
[ 1104.514576]  [<c106716a>] ? __lock_acquire+0x1444/0x1455
[ 1104.514576]  [<c10671f9>] ? lock_release_non_nested+0x7e/0x21b
[ 1104.514576]  [<c106716a>] ? __lock_acquire+0x1444/0x1455
[ 1104.514576]  [<c100acba>] arch_ptrace+0x474/0x48a
[ 1104.514576]  [<c167e841>] ? _raw_spin_unlock_irqrestore+0x42/0x63
[ 1104.514576]  [<c10652f8>] ? trace_hardirqs_on+0xb/0xd
[ 1104.514576]  [<c1047e25>] ? wait_task_inactive+0x6f/0xc1
[ 1104.514576]  [<c1030972>] ? ptrace_check_attach+0xb4/0xc4
[ 1104.514576]  [<c1031192>] sys_ptrace+0x34f/0x379
[ 1104.514576]  [<c1030eb1>] ? sys_ptrace+0x6e/0x379
[ 1104.514576]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1104.514576] Code: 3d 8c 20 91 c1 00 74 1d 89 d8 50 9d 8d 74 26 00 e8 dd 6b 9e ff eb 18 e8 ac 6a 9e ff 83 3d 8c 20 91 c1 00 75 02 0f 0b 89 d8 50 9d <8d> 74 26 00 89 e0 25 00 e0 ff ff ff 48 14 5b 5e 5d c3 55 89 e5
[ 1104.514576] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.514576] Pid: 1, comm: init Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.514576] Call Trace:
[ 1104.514576]  [<c1673678>] panic+0x97/0x1aa
[ 1104.514576]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.514576]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.514576]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.514576]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.514576]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.514576]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.514576]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.514576]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.514576]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.514576]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.514576]  [<c167e850>] ? _raw_spin_unlock_irqrestore+0x51/0x63
[ 1104.514576]  [<c104a23a>] try_to_wake_up+0x248/0x266
[ 1104.514576]  [<c104a280>] wake_up_state+0xa/0xc
[ 1104.514576]  [<c10306b6>] ptrace_resume+0x76/0x7c
[ 1104.514576]  [<c1031596>] ptrace_request+0x368/0x4f8
[ 1104.514576]  [<c106716a>] ? __lock_acquire+0x1444/0x1455
[ 1104.514576]  [<c106716a>] ? __lock_acquire+0x1444/0x1455
[ 1104.514576]  [<c10671f9>] ? lock_release_non_nested+0x7e/0x21b
[ 1104.514576]  [<c106716a>] ? __lock_acquire+0x1444/0x1455
[ 1104.514576]  [<c100acba>] arch_ptrace+0x474/0x48a
[ 1104.514576]  [<c167e841>] ? _raw_spin_unlock_irqrestore+0x42/0x63
[ 1104.514576]  [<c10652f8>] ? trace_hardirqs_on+0xb/0xd
[ 1104.514576]  [<c1047e25>] ? wait_task_inactive+0x6f/0xc1
[ 1104.514576]  [<c1030972>] ? ptrace_check_attach+0xb4/0xc4
[ 1104.514576]  [<c1031192>] sys_ptrace+0x34f/0x379
[ 1104.514576]  [<c1030eb1>] ? sys_ptrace+0x6e/0x379
[ 1104.514576]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1104.514576] Rebooting in 10 seconds..
--
[    5.876251] Freeing unused kernel memory: 460k freed
[ 1104.522650] BUG: soft lockup - CPU#1 stuck for 1023s! [swapper/1:0]
[ 1104.522650] irq event stamp: 14128
[ 1104.522650] hardirqs last  enabled at (14125): [<c167f085>] restore_all_notrace+0x0/0x18
[ 1104.522650] hardirqs last disabled at (14126): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.528994] [sched_delayed] sched: RT throttling activated
[ 1104.532753] softirqs last  enabled at (14128): [<c102e637>] _local_bh_enable+0xd/0xf
[ 1104.532753] softirqs last disabled at (14127): [<c102eac7>] irq_enter+0x2c/0x4d
[ 1104.532753] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.540165] EIP: 0060:[<c101da32>] EFLAGS: 00000286 CPU: 1
[ 1104.540165] EIP is at native_safe_halt+0x5/0x7
[ 1104.540165] EAX: ce06a260 EBX: ffffffff ECX: ce06a260 EDX: 00000007
[ 1104.540165] ESI: c19716f0 EDI: ffff8ba4 EBP: ce071f68 ESP: ce071f68
[ 1104.540165]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.540165] CR0: 8005003b CR2: b77b152c CR3: 019f0000 CR4: 00000690
[ 1104.540165] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
--
[    6.225744] Freeing unused kernel memory: 460k freed
[ 1105.529503] BUG: soft lockup - CPU#1 stuck for 1024s! [swapper/1:0]
[ 1105.529503] irq event stamp: 14064
[ 1105.529503] hardirqs last  enabled at (14061): [<c167f085>] restore_all_notrace+0x0/0x18
[ 1105.529503] hardirqs last disabled at (14062): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1105.529503] softirqs last  enabled at (14064): [<c102e637>] _local_bh_enable+0xd/0xf
[ 1105.529503] softirqs last disabled at (14063): [<c102eac7>] irq_enter+0x2c/0x4d
[ 1105.529503] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1105.529503] EIP: 0060:[<c101da32>] EFLAGS: 00000286 CPU: 1
[ 1105.529503] EIP is at native_safe_halt+0x5/0x7
[ 1105.529503] EAX: ce06a260 EBX: ffffffff ECX: ce06a260 EDX: 00000007
[ 1105.529503] ESI: c19716f0 EDI: ffff8b9e EBP: ce071f68 ESP: ce071f68
[ 1105.529503]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1105.529503] CR0: 8005003b CR2: b7667fb8 CR3: 019f0000 CR4: 00000690
[ 1105.529503] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1105.529503] DR6: ffff0ff0 DR7: 00000400
[ 1105.529503] Process swapper/1 (pid: 0, ti=ce070000 task=ce06a260 task.ti=ce070000)
[ 1105.529503] Stack:
[ 1105.529503]  ce071f70 c1008a09 ce071f88 c1009250 b6ce783b c531fd5c ff7d2fe6 cfff9fa0
[ 1105.529503]  ce071fb4 c166ed10 00000000 00000000 00000000 00000000 6fac3882 ff7d2fe6
[ 1105.529503]  01000800 0000d248 00000000 00000000 00000000 00000000 00000000 00000000
[ 1105.529503] Call Trace:
[ 1105.529503]  [<c1008a09>] default_idle+0x2f/0x55
[ 1105.529503]  [<c1009250>] cpu_idle+0xc9/0xfb
[ 1105.529503]  [<c166ed10>] start_secondary+0x292/0x29a
[ 1105.529503] Code: 0f 22 e0 5d c3 55 89 e5 0f 09 5d c3 55 89 e5 9c 58 5d c3 55 89 e5 50 9d 5d c3 55 89 e5 fa 5d c3 55 89 e5 fb 5d c3 55 89 e5 fb f4 <5d> c3 55 89 e5 f4 5d c3 55 89 e5 57 56 53 53 89 c6 89 55 f0 89
[ 1105.529503] Kernel panic - not syncing: softlockup: hung tasks
[ 1105.529503] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1105.529503] Call Trace:
[ 1105.529503]  [<c1673678>] panic+0x97/0x1aa
[ 1105.529503]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1105.529503]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1105.529503]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1105.529503]  [<c103205a>] run_local_timers+0x8/0x14
[ 1105.529503]  [<c103208a>] update_process_times+0x24/0x58
[ 1105.529503]  [<c106276e>] tick_periodic+0x66/0x72
[ 1105.529503]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1105.529503]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1105.529503]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1105.529503]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1105.529503]  [<c101da32>] ? native_safe_halt+0x5/0x7
[ 1105.529503]  [<c1008a09>] default_idle+0x2f/0x55
[ 1105.529503]  [<c1009250>] cpu_idle+0xc9/0xfb
[ 1105.529503]  [<c166ed10>] start_secondary+0x292/0x29a
mountall: ply-event-loop.c:497: ply_event_loop_new: Assertion `loop->epoll_fd >= 0' failed.
[ 1105.529503] Rebooting in 10 seconds..
--
[    5.084082] Freeing unused kernel memory: 460k freed
[ 1104.521392] BUG: soft lockup - CPU#1 stuck for 1025s! [swapper/1:0]
[ 1104.521414] irq event stamp: 14272
[ 1104.521414] hardirqs last  enabled at (14269): [<c10089f9>] default_idle+0x1f/0x55
[ 1104.523434] [sched_delayed] sched: RT throttling activated
[ 1104.521414] hardirqs last disabled at (14270): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.521414] softirqs last  enabled at (14272): [<c102e637>] _local_bh_enable+0xd/0xf
[ 1104.521414] softirqs last disabled at (14271): [<c102eac7>] irq_enter+0x2c/0x4d
[ 1104.521414] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.521414] EIP: 0060:[<c101da32>] EFLAGS: 00000286 CPU: 1
[ 1104.521414] EIP is at native_safe_halt+0x5/0x7
[ 1104.521414] EAX: ce06a260 EBX: ffffffff ECX: ce06a260 EDX: 00000007
[ 1104.521414] ESI: c19716f0 EDI: ffff8b9f EBP: ce071f68 ESP: ce071f68
[ 1104.521414]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.521414] CR0: 8005003b CR2: b7611bd0 CR3: 0e866000 CR4: 00000690
[ 1104.521414] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
mountall: ply-ev[ 1104.521414] DR6: ffff0ff0 DR7: 00000400
ent-loop.c:497: [ 1104.521414] Process swapper/1 (pid: 0, ti=ce070000 task=ce06a260 task.ti=ce070000)
ply_event_loop_n[ 1104.521414] Stack:
ew: Assertion `l[ 1104.521414]  ce071f70oop->epoll_fd >= c1008a09 0' failed.
 ce071f88 c1009250 22af12a3 311297c7 6b5d3c03 cfff9fa0
[ 1104.521414]  ce071fb4 c166ed10 00000000 00000000 00000000 00000000 db8c449d 6b5d3c03
--
[    5.014137] Freeing unused kernel memory: 460k freed
[ 1104.514173] BUG: soft lockup - CPU#1 stuck for 1024s! [init:1]
[ 1104.514173] irq event stamp: 745706
[ 1104.514173] hardirqs last  enabled at (745705): [<c167f085>] restore_all_notrace+0x0/0x18
[ 1104.514173] hardirqs last disabled at (745706): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1104.514173] softirqs last  enabled at (741514): [<c102e8ff>] __do_softirq+0x131/0x153
[ 1104.514173] softirqs last disabled at (741501): [<c1003f90>] do_softirq+0x74/0xdf
[ 1104.514173] Pid: 1, comm: init Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1104.514173] EIP: 0060:[<c105e677>] EFLAGS: 00000202 CPU: 1
[ 1104.522897] [sched_delayed] sched: RT throttling activated
[ 1104.514173] EIP is at current_kernel_time+0x10/0x48
[ 1104.514173] EAX: 000003f3 EBX: ce02d800 ECX: 00020001 EDX: 0000001d
[ 1104.514173] ESI: c797d300 EDI: 00000000 EBP: ce043e50 ESP: ce043e48
[ 1104.514173]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.514173] CR0: 8005003b CR2: b7762000 CR3: 0eec0000 CR4: 00000690
[ 1104.514173] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.514173] DR6: ffff0ff0 DR7: 00000400
[ 1104.514173] Process init (pid: 1, ti=ce042000 task=ce048000 task.ti=ce042000)
[ 1104.514173] Stack:
[ 1104.514173]  c797d300 00000000 ce043e5c c102de8e c70ca210 ce043e7c c10dd8d0 00000001
[ 1104.514173]  00000041 0000001d cf83a3c0 00000001 00000001 ce043ecc c10cf835 00000041
[ 1104.514173]  cf83a40c ce048000 cf869c00 00000008 00000000 c70ca298 c797d300 00000010
[ 1104.514173] Call Trace:
[ 1104.514173]  [<c102de8e>] current_fs_time+0xb/0x19
[ 1104.514173]  [<c10dd8d0>] file_update_time+0x21/0x91
[ 1104.514173]  [<c10cf835>] pipe_write+0x3bf/0x3cf
[ 1104.514173]  [<c10c80a1>] do_sync_write+0x75/0xb0
[ 1104.514173]  [<c10c8169>] vfs_write+0x8d/0x10f
[ 1104.514173]  [<c10c802c>] ? vfs_read+0x10d/0x10d
[ 1104.514173]  [<c10c8289>] sys_write+0x3d/0x61
[ 1104.514173]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1104.514173] Code: a1 c1 0f ad fe d3 ef f6 c1 20 74 04 89 fe 31 ff a1 70 d6 a1 c1 89 f2 5e 5f 5d c3 55 89 e5 57 56 a1 b4 d6 a1 c1 a8 01 74 04 f3 90 <eb> f3 8b 15 70 d6 a1 c1 8b 35 78 d6 a1 c1 8b 3d 7c d6 a1 c1 8b
[ 1104.514173] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.514173] Pid: 1, comm: init Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1104.514173] Call Trace:
[ 1104.514173]  [<c1673678>] panic+0x97/0x1aa
[ 1104.514173]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1104.514173]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.514173]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1104.514173]  [<c103205a>] run_local_timers+0x8/0x14
[ 1104.514173]  [<c103208a>] update_process_times+0x24/0x58
[ 1104.514173]  [<c106276e>] tick_periodic+0x66/0x72
[ 1104.514173]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1104.514173]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.514173]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.514173]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1104.514173]  [<c167007b>] ? kvm_guest_cpu_init+0x10b/0x214
[ 1104.514173]  [<c105e677>] ? current_kernel_time+0x10/0x48
[ 1104.514173]  [<c102de8e>] current_fs_time+0xb/0x19
[ 1104.514173]  [<c10dd8d0>] file_update_time+0x21/0x91
[ 1104.514173]  [<c10cf835>] pipe_write+0x3bf/0x3cf
[ 1104.514173]  [<c10c80a1>] do_sync_write+0x75/0xb0
[ 1104.514173]  [<c10c8169>] vfs_write+0x8d/0x10f
[ 1104.514173]  [<c10c802c>] ? vfs_read+0x10d/0x10d
[ 1104.514173]  [<c10c8289>] sys_write+0x3d/0x61
[ 1104.514173]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1104.514173] Rebooting in 10 seconds..
--
[    4.537105] Freeing unused kernel memory: 460k freed
[ 1103.514239] BUG: soft lockup - CPU#1 stuck for 1024s! [swapper/1:0]
[ 1103.514239] irq event stamp: 7850
[ 1103.514239] hardirqs last  enabled at (7847): [<c167f085>] restore_all_notrace+0x0/0x18
[ 1103.514239] hardirqs last disabled at (7848): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1103.514239] softirqs last  enabled at (7850): [<c102e637>] _local_bh_enable+0xd/0xf
[ 1103.514239] softirqs last disabled at (7849): [<c102eac7>] irq_enter+0x2c/0x4d
[ 1103.514239] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1103.514239] EIP: 0060:[<c101da32>] EFLAGS: 00000286 CPU: 1
[ 1103.514239] EIP is at native_safe_halt+0x5/0x7
[ 1103.514239] EAX: ce06a260 EBX: ffffffff ECX: ce06a260 EDX: 00000007
[ 1103.514239] ESI: c19716f0 EDI: ffff8b9d EBP: ce071f68 ESP: ce071f68
[ 1103.523641] [sched_delayed] sched: RT throttling activated
[ 1103.514239]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.514239] CR0: 8005003b CR2: b7677fb8 CR3: 0ee96000 CR4: 00000690
[ 1103.514239] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.514239] DR6: ffff0ff0 DR7: 00000400
[ 1103.514239] Process swapper/1 (pid: 0, ti=ce070000 task=ce06a260 task.ti=ce070000)
[ 1103.514239] Stack:
[ 1103.514239]  ce071f70 c1008a09 ce071f88 c1009250 d5b8c18f e41c46af 1e665633 cfff9fa0
[ 1103.514239]  ce071fb4 c166ed10 00000000 00000000 00000000 00000000 8e955ed0 1e665633
[ 1103.514239]  01000800 0000d248 00000000 00000000 00000000 00000000 00000000 00000000
[ 1103.514239] Call Trace:
[ 1103.514239]  [<c1008a09>] default_idle+0x2f/0x55
[ 1103.514239]  [<c1009250>] cpu_idle+0xc9/0xfb
[ 1103.514239]  [<c166ed10>] start_secondary+0x292/0x29a
[ 1103.514239] Code: 0f 22 e0 5d c3 55 89 e5 0f 09 5d c3 55 89 e5 9c 58 5d c3 55 89 e5 50 9d 5d c3 55 89 e5 fa 5d c3 55 89 e5 fb 5d c3 55 89 e5 fb f4 <5d> c3 55 89 e5 f4 5d c3 55 89 e5 57 56 53 53 89 c6 89 55 f0 89
[ 1103.514239] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.514239] Pid: 0, comm: swapper/1 Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1103.514239] Call Trace:
[ 1103.514239]  [<c1673678>] panic+0x97/0x1aa
[ 1103.514239]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1103.514239]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.514239]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1103.514239]  [<c103205a>] run_local_timers+0x8/0x14
[ 1103.514239]  [<c103208a>] update_process_times+0x24/0x58
[ 1103.514239]  [<c106276e>] tick_periodic+0x66/0x72
[ 1103.514239]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1103.514239]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.514239]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.514239]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1103.514239]  [<c101da32>] ? native_safe_halt+0x5/0x7
[ 1103.514239]  [<c1008a09>] default_idle+0x2f/0x55
[ 1103.514239]  [<c1009250>] cpu_idle+0xc9/0xfb
[ 1103.514239]  [<c166ed10>] start_secondary+0x292/0x29a
[ 1103.514239] Rebooting in 10 seconds..
--
[    6.525425] Freeing unused kernel memory: 464k freed
[ 1105.523006] BUG: soft lockup - CPU#1 stuck for 1023s! [99-trinity:133]
[ 1105.523006] irq event stamp: 632
[ 1105.523006] hardirqs last  enabled at (631): [<c166e268>] __mutex_unlock_slowpath+0x114/0x12c
[ 1105.523006] hardirqs last disabled at (632): [<c16712ff>] apic_timer_interrupt+0x2f/0x40
[ 1105.523006] softirqs last  enabled at (0): [<c10252e4>] copy_process+0x471/0xf82
[ 1105.523006] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1105.523006] Pid: 133, comm: 99-trinity Not tainted 3.5.0-bisect2-05018-g614a6d43 #82  
[ 1105.523006] EIP: 0060:[<c1064141>] EFLAGS: 00000202 CPU: 1
[ 1105.523006] EIP is at lockdep_trace_alloc+0x3/0xe0
[ 1105.523006] EAX: 000280da EBX: 000280da ECX: c1958810 EDX: 00000000
[ 1105.523006] ESI: 0000000a EDI: ce807680 EBP: ce8a3d5c ESP: ce8a3d5c
[ 1105.523006]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1105.523006] CR0: 80050033 CR2: b75e5baf CR3: 0e868000 CR4: 00000690
[ 1105.523006] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1105.523006] DR6: ffff0ff0 DR7: 00000400
[ 1105.523006] Process 99-trinity (pid: 133, ti=ce8a2000 task=ce8a8000 task.ti=ce8a2000)
[ 1105.523006] Stack:
[ 1105.523006]  ce8a3dfc c109a6d2 ce8a8000 00000040 00000006 c1bc8eb4 ce8a8458 ce8a8000
[ 1105.523006]  ce8a3da0 c1063a52 00000378 00000007 00000000 00000000 ce8a8000 c1900ad8
[ 1105.523006]  ce8a3db4 00000286 00000286 000000aa c181b224 00000000 ce8a3ddc 00000286
[ 1105.523006] Call Trace:
[ 1105.523006]  [<c109a6d2>] __alloc_pages_nodemask+0x78/0x5db
[ 1105.523006]  [<c1063a52>] ? mark_held_locks+0xa7/0xce
[ 1105.523006]  [<c10b19e1>] ? anon_vma_prepare+0x24/0x131
[ 1105.523006]  [<c10abbdd>] handle_pte_fault+0x16b/0x74c
[ 1105.523006]  [<c1020637>] ? kmap_atomic_prot+0x11f/0x133
[ 1105.523006]  [<c10ac253>] handle_mm_fault+0x95/0xa4
[ 1105.523006]  [<c10ac545>] __get_user_pages+0x264/0x39e
[ 1105.523006]  [<c10ac71d>] get_user_pages+0x4b/0x53
[ 1105.523006]  [<c10c3f2f>] get_arg_page+0x4a/0xa9
[ 1105.523006]  [<c10c408c>] copy_strings+0xfe/0x1c6
[ 1105.523006]  [<c10c44e9>] copy_strings_kernel+0x1c/0x2b
[ 1105.523006]  [<c10c56ff>] do_execve+0x343/0x456
[ 1105.523006]  [<c10c54a7>] ? do_execve+0xeb/0x456
[ 1105.523006]  [<c1009135>] sys_execve+0x31/0x53
[ 1105.523006]  [<c16718e2>] ptregs_execve+0x12/0x20
[ 1105.523006]  [<c16717e3>] ? sysenter_do_call+0x12/0x38
[ 1105.523006] Code: 3d 1c b7 95 c1 01 74 16 ba 9a 0a 00 00 b8 8b 35 81 c1 e8 7e 26 fc ff c6 05 1c b7 95 c1 01 64 ff 05 bc dc 9c c1 5b 5d c3 55 89 e5 <57> 56 53 89 c6 64 8b 1d 28 c6 9c c1 83 bb 54 04 00 00 00 0f 85
[ 1105.523006] Kernel panic - not syncing: softlockup: hung tasks
[ 1105.523006] Pid: 133, comm: 99-trinity Not tainted 3.5.0-bisect2-05018-g614a6d43 #82
[ 1105.523006] Call Trace:
[ 1105.523006]  [<c1664e92>] panic+0x81/0x194
[ 1105.523006]  [<c107e316>] watchdog_timer_fn+0x115/0x134
[ 1105.523006]  [<c10416fe>] hrtimer_run_queues+0x14c/0x1c6
[ 1105.523006]  [<c107e201>] ? __touch_watchdog+0x1d/0x1d
[ 1105.523006]  [<c10306a2>] run_local_timers+0x8/0x14
[ 1105.523006]  [<c10306d2>] update_process_times+0x24/0x58
[ 1105.523006]  [<c1061092>] tick_periodic+0x66/0x72
[ 1105.523006]  [<c10610ec>] tick_handle_periodic+0x19/0x61
[ 1105.523006]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1105.523006]  [<c12aa9a0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1105.523006]  [<c1671306>] apic_timer_interrupt+0x36/0x40
[ 1105.523006]  [<c1064141>] ? lockdep_trace_alloc+0x3/0xe0
[ 1105.523006]  [<c109a6d2>] __alloc_pages_nodemask+0x78/0x5db
[ 1105.523006]  [<c1063a52>] ? mark_held_locks+0xa7/0xce
[ 1105.523006]  [<c10b19e1>] ? anon_vma_prepare+0x24/0x131
[ 1105.581916]  [<c10abbdd>] handle_pte_fault+0x16b/0x74c
[ 1105.581916]  [<c1020637>] ? kmap_atomic_prot+0x11f/0x133
[ 1105.581916]  [<c10ac253>] handle_mm_fault+0x95/0xa4
[ 1105.581916]  [<c10ac545>] __get_user_pages+0x264/0x39e
[ 1105.581916]  [<c10ac71d>] get_user_pages+0x4b/0x53
[ 1105.581916]  [<c10c3f2f>] get_arg_page+0x4a/0xa9
[ 1105.581916]  [<c10c408c>] copy_strings+0xfe/0x1c6
[ 1105.581916]  [<c10c44e9>] copy_strings_kernel+0x1c/0x2b
[ 1105.581916]  [<c10c56ff>] do_execve+0x343/0x456
[ 1105.581916]  [<c10c54a7>] ? do_execve+0xeb/0x456
[ 1105.581916]  [<c1009135>] sys_execve+0x31/0x53
[ 1105.581916]  [<c16718e2>] ptregs_execve+0x12/0x20
[ 1105.581916]  [<c16717e3>] ? sysenter_do_call+0x12/0x38
[ 1105.581916] Rebooting in 10 seconds..
--
[    4.563976] Freeing unused kernel memory: 464k freed
[ 1103.520656] BUG: soft lockup - CPU#1 stuck for 1023s! [shutdown:135]
[ 1103.520656] irq event stamp: 2580
[ 1103.520656] hardirqs last  enabled at (2579): [<c166b883>] mutex_lock_nested+0x2da/0x308
[ 1103.520656] hardirqs last disabled at (2580): [<c166f4bf>] apic_timer_interrupt+0x2f/0x40
[ 1103.520656] softirqs last  enabled at (0): [<c1025204>] copy_process+0x471/0xf82
mountall: ply-ev[ 1103.520656] softirqs last disabled at (0): ent-loop.c:497: [<  (null)>]   (null)
ply_event_loop_n[ 1103.520656] Pid: 135, comm: shutdown Not tainted 3.5.0-bisect2-00313-g7cd58b0 #86  
ew: Assertion `l[ 1103.520656] EIP: 0060:[<c166b893>] EFLAGS: 00000286 CPU: 1
[ 1103.520656] EIP is at mutex_lock_nested+0x2ea/0x308
[ 1103.520656] EAX: 00000286 EBX: cb4e699c ECX: 00000006 EDX: cf8826c8
[ 1103.520656] ESI: ce82dbb0 EDI: ce873d7c EBP: ce873d98 ESP: ce873d5c
[ 1103.520656]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.520656] CR0: 8005003b CR2: b77911b8 CR3: 0e88b000 CR4: 00000690
[ 1103.520656] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
oop->epoll_fd >=[ 1103.520656] DR6: ffff0ff0 DR7: 00000400
[ 1103.520656] Process shutdown (pid: 135, ti=ce872000 task=cf882240 task.ti=ce872000)
 0' failed.
[ 1103.520656] Stack:
[ 1103.520656]  00000000 00000002 00000000 c10addf7 cb4e69c0 cf882240 cb4e69a0 00000286
[ 1103.520656]  ce873d7c ce873d7c 00000000 ce873d7c ce82d790 ce82dbb0 00000000 ce873dec
[ 1103.520656]  c10addf7 00000000 00000000 00000000 00000000 b775e000 ce0f6400 00000000
[ 1103.520656] Call Trace:
[ 1103.520656]  [<c10addf7>] ? vma_adjust+0x134/0x30e
[ 1103.520656]  [<c10addf7>] vma_adjust+0x134/0x30e
[ 1103.520656]  [<c10ae0bb>] __split_vma+0xea/0x13e
[ 1103.520656]  [<c10ae8fd>] do_munmap+0xab/0x24b
[ 1103.520656]  [<c10aeacd>] vm_munmap+0x30/0x41
[ 1103.520656]  [<c166505c>] elf_map+0x80/0xa6
[ 1103.520656]  [<c10f0e65>] load_elf_binary+0x7b7/0xfbd
[ 1103.520656]  [<c10c51f6>] search_binary_handler+0x14d/0x22b
[ 1103.520656]  [<c10c50e3>] ? search_binary_handler+0x3a/0x22b
[ 1103.520656]  [<c10f06ae>] ? load_elf_library+0x1bb/0x1bb
[ 1103.520656]  [<c10c565f>] do_execve+0x38b/0x456
[ 1103.520656]  [<c10c53bf>] ? do_execve+0xeb/0x456
[ 1103.520656]  [<c1009135>] sys_execve+0x31/0x53
[ 1103.520656]  [<c166faa2>] ptregs_execve+0x12/0x20
[ 1103.520656]  [<c166f9a3>] ? sysenter_do_call+0x12/0x38
[ 1103.520656] Code: d4 8f c1 00 74 1e 8b 45 e0 50 9d 8d 74 26 00 e8 5b 81 9f ff eb 19 e8 2a 80 9f ff 83 3d 6c d4 8f c1 00 75 02 0f 0b 8b 45 e0 50 9d <8d> 74 26 00 8d 45 e4 e8 c0 5f 9f ff 89 e0 25 00 e0 ff ff ff 48
[ 1103.520656] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.520656] Pid: 135, comm: shutdown Not tainted 3.5.0-bisect2-00313-g7cd58b0 #86
[ 1103.520656] Call Trace:
[ 1103.520656]  [<c1662a5f>] panic+0x81/0x194
[ 1103.520656]  [<c107e1de>] watchdog_timer_fn+0x115/0x134
[ 1103.520656]  [<c10413be>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.520656]  [<c107e0c9>] ? __touch_watchdog+0x1d/0x1d
[ 1103.520656]  [<c10305c6>] run_local_timers+0x8/0x14
[ 1103.520656]  [<c10305f6>] update_process_times+0x24/0x58
[ 1103.520656]  [<c1060d2e>] tick_periodic+0x66/0x72
[ 1103.520656]  [<c1060d88>] tick_handle_periodic+0x19/0x61
[ 1103.520656]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.520656]  [<c12aa260>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.520656]  [<c166f4c6>] apic_timer_interrupt+0x36/0x40
[ 1103.520656]  [<c166b893>] ? mutex_lock_nested+0x2ea/0x308
[ 1103.520656]  [<c10addf7>] ? vma_adjust+0x134/0x30e
[ 1103.520656]  [<c10addf7>] vma_adjust+0x134/0x30e
[ 1103.520656]  [<c10ae0bb>] __split_vma+0xea/0x13e
[ 1103.520656]  [<c10ae8fd>] do_munmap+0xab/0x24b
[ 1103.520656]  [<c10aeacd>] vm_munmap+0x30/0x41
[ 1103.520656]  [<c166505c>] elf_map+0x80/0xa6
[ 1103.520656]  [<c10f0e65>] load_elf_binary+0x7b7/0xfbd
[ 1103.520656]  [<c10c51f6>] search_binary_handler+0x14d/0x22b
[ 1103.520656]  [<c10c50e3>] ? search_binary_handler+0x3a/0x22b
[ 1103.520656]  [<c10f06ae>] ? load_elf_library+0x1bb/0x1bb
[ 1103.520656]  [<c10c565f>] do_execve+0x38b/0x456
[ 1103.520656]  [<c10c53bf>] ? do_execve+0xeb/0x456
[ 1103.520656]  [<c1009135>] sys_execve+0x31/0x53
[ 1103.520656]  [<c166faa2>] ptregs_execve+0x12/0x20
[ 1103.520656]  [<c166f9a3>] ? sysenter_do_call+0x12/0x38
[ 1103.520656] Rebooting in 10 seconds..
--
[ 1103.521634] [sched_delayed] sched: RT throttling activated
[ 1103.523066] BUG: soft lockup - CPU#1 stuck for 1023s! [hwclock:126]
[ 1103.523066] irq event stamp: 7436
[ 1103.523066] hardirqs last  enabled at (7435): [<c167e841>] _raw_spin_unlock_irqrestore+0x42/0x63
[ 1103.523066] hardirqs last disabled at (7436): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1103.523066] softirqs last  enabled at (3224): [<c1601126>] unix_sock_destructor+0xae/0xb3
[ 1103.523066] softirqs last disabled at (3222): [<c1601113>] unix_sock_destructor+0x9b/0xb3
[ 1103.523066] Pid: 126, comm: hwclock Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1103.523066] EIP: 0060:[<c167e850>] EFLAGS: 00000282 CPU: 1
[ 1103.523066] EIP is at _raw_spin_unlock_irqrestore+0x51/0x63
[ 1103.523066] EAX: 00000282 EBX: 00000282 ECX: 00000000 EDX: 00000007
[ 1103.523066] ESI: c1fa259c EDI: 00000000 EBP: ce891da4 ESP: ce891d9c
[ 1103.523066]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.523066] CR0: 8005003b CR2: b76b7fb8 CR3: 019f0000 CR4: 00000690
[ 1103.523066] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.523066] DR6: ffff0ff0 DR7: 00000400
[ 1103.523066] Process hwclock (pid: 126, ti=ce890000 task=cf89a260 task.ti=ce890000)
[ 1103.523066] Stack:
[ 1103.523066]  00000001 00000001 ce891dd8 c12c3bde ce891de0 00000282 c1fa2598 ce44d000
[ 1103.523066]  c1fa259c ce44c000 ce44c000 00000000 ce707980 00001000 00000000 ce891de0
[ 1103.523066]  c12c4324 ce891e04 c109bf2e 00000000 00000001 00000000 00000001 ce707a00
[ 1103.523066] Call Trace:
[ 1103.523066]  [<c12c3bde>] __debug_check_no_obj_freed+0xe4/0x137
[ 1103.523066]  [<c12c4324>] debug_check_no_obj_freed+0x11/0x15
[ 1103.523066]  [<c109bf2e>] free_pages_prepare+0x10a/0x10e
[ 1103.523066]  [<c109d0fa>] free_hot_cold_page+0x20/0x155
[ 1103.523066]  [<c167e841>] ? _raw_spin_unlock_irqrestore+0x42/0x63
[ 1103.523066]  [<c109d24c>] free_hot_cold_page_list+0x1d/0x31
[ 1103.523066]  [<c10a06a1>] release_pages+0x171/0x179
[ 1103.523066]  [<c10b9669>] free_pages_and_swap_cache+0x68/0x7b
[ 1103.523066]  [<c10ac647>] tlb_flush_mmu+0x56/0x6c
[ 1103.523066]  [<c10ac66f>] tlb_finish_mmu+0x12/0x31
[ 1103.523066]  [<c10b19f9>] exit_mmap+0x96/0xe4
[ 1103.523066]  [<c1025e1a>] mmput+0x42/0xa7
[ 1103.523066]  [<c102cab2>] exit_mm+0xde/0xe6
[ 1103.523066]  [<c102ce13>] do_exit+0x1d0/0x72c
[ 1103.523066]  [<c102dd6d>] ? do_sys_settimeofday+0x88/0x93
[ 1103.523066]  [<c12b3946>] ? _copy_from_user+0x37/0x4b
[ 1103.523066]  [<c102d3f6>] do_group_exit+0x5e/0x81
[ 1103.523066]  [<c102d42c>] sys_exit_group+0x13/0x13
[ 1103.523066]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1103.523066] Code: 3d 8c 20 91 c1 00 74 1d 89 d8 50 9d 8d 74 26 00 e8 dd 6b 9e ff eb 18 e8 ac 6a 9e ff 83 3d 8c 20 91 c1 00 75 02 0f 0b 89 d8 50 9d <8d> 74 26 00 89 e0 25 00 e0 ff ff ff 48 14 5b 5e 5d c3 55 89 e5
[ 1103.523066] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.523066] Pid: 126, comm: hwclock Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1103.523066] Call Trace:
[ 1103.523066]  [<c1673678>] panic+0x97/0x1aa
[ 1103.523066]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1103.523066]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.523066]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1103.523066]  [<c103205a>] run_local_timers+0x8/0x14
[ 1103.523066]  [<c103208a>] update_process_times+0x24/0x58
[ 1103.523066]  [<c106276e>] tick_periodic+0x66/0x72
[ 1103.523066]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1103.523066]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.523066]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.523066]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1103.523066]  [<c167e850>] ? _raw_spin_unlock_irqrestore+0x51/0x63
[ 1103.523066]  [<c12c3bde>] __debug_check_no_obj_freed+0xe4/0x137
[ 1103.523066]  [<c12c4324>] debug_check_no_obj_freed+0x11/0x15
[ 1103.523066]  [<c109bf2e>] free_pages_prepare+0x10a/0x10e
[ 1103.523066]  [<c109d0fa>] free_hot_cold_page+0x20/0x155
[ 1103.523066]  [<c167e841>] ? _raw_spin_unlock_irqrestore+0x42/0x63
[ 1103.523066]  [<c109d24c>] free_hot_cold_page_list+0x1d/0x31
[ 1103.523066]  [<c10a06a1>] release_pages+0x171/0x179
[ 1103.523066]  [<c10b9669>] free_pages_and_swap_cache+0x68/0x7b
[ 1103.523066]  [<c10ac647>] tlb_flush_mmu+0x56/0x6c
[ 1103.523066]  [<c10ac66f>] tlb_finish_mmu+0x12/0x31
[ 1103.523066]  [<c10b19f9>] exit_mmap+0x96/0xe4
[ 1103.523066]  [<c1025e1a>] mmput+0x42/0xa7
[ 1103.523066]  [<c102cab2>] exit_mm+0xde/0xe6
[ 1103.523066]  [<c102ce13>] do_exit+0x1d0/0x72c
[ 1103.523066]  [<c102dd6d>] ? do_sys_settimeofday+0x88/0x93
[ 1103.523066]  [<c12b3946>] ? _copy_from_user+0x37/0x4b
[ 1103.523066]  [<c102d3f6>] do_group_exit+0x5e/0x81
[ 1103.523066]  [<c102d42c>] sys_exit_group+0x13/0x13
[ 1103.523066]  [<c167f863>] sysenter_do_call+0x12/0x38
[ 1103.523066] Rebooting in 10 seconds..
--
[    4.801716] Freeing unused kernel memory: 460k freed
[ 1103.512941] BUG: soft lockup - CPU#1 stuck for 1023s! [sleep:134]
[ 1103.512941] irq event stamp: 780
[ 1103.512941] hardirqs last  enabled at (779): [<c101e5d2>] do_page_fault+0x98/0x36e
[ 1103.512941] hardirqs last disabled at (780): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1103.517007] softirqs last  enabled at (22): [<c102e8ff>] __do_softirq+0x131/0x153
--
[ 1105.513232] [sched_delayed] sched: RT throttling activated
[ 1105.513206] BUG: soft lockup - CPU#1 stuck for 1023s! [rc.local:123]
[ 1105.513206] irq event stamp: 326
[ 1105.513206] hardirqs last  enabled at (325): [<c106831d>] debug_check_no_locks_freed+0x14b/0x167
[ 1105.513206] hardirqs last disabled at (326): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1105.513206] softirqs last  enabled at (0): [<c1026857>] copy_process+0x465/0xf8a
[ 1105.513206] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1105.513206] Pid: 123, comm: rc.local Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1105.513206] EIP: 0060:[<c10df7c4>] EFLAGS: 00000246 CPU: 1
[ 1105.513206] EIP is at __mnt_drop_write+0x1/0x1a
[ 1105.513206] EAX: ce014110 EBX: cb438000 ECX: 00000000 EDX: 00000001
[ 1105.513206] ESI: ce014110 EDI: 00000000 EBP: ce879dbc ESP: ce879da4
[ 1105.513206]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1105.513206] CR0: 80050033 CR2: b7553baf CR3: 0e85a000 CR4: 00000690
[ 1105.513206] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1105.513206] DR6: ffff0ff0 DR7: 00000400
[ 1105.513206] Process rc.local (pid: 123, ti=ce878000 task=cf8ad5f0 task.ti=ce878000)
[ 1105.513206] Stack:
[ 1105.513206]  ce879dbc c10dd7bf 504bf223 ffffff88 ce889800 ce6a2f80 ce879e4c c1097f77
[ 1105.513206]  00001000 006f41e4 00000000 00000080 00000000 00000000 00000001 ce889884
[ 1105.513206]  ce879e90 00000000 00001000 00000080 00000000 cb438000 ce879ed4 00000000
[ 1105.513206] Call Trace:
[ 1105.513206]  [<c10dd7bf>] ? touch_atime+0xdc/0x104
[ 1105.513206]  [<c1097f77>] generic_file_aio_read+0x5b5/0x5f1
[ 1105.513206]  [<c10c78d1>] do_sync_read+0x75/0xb0
[ 1105.513206]  [<c10c7faa>] vfs_read+0x8b/0x10d
[ 1105.513206]  [<c10af901>] ? might_fault+0x26/0x99
[ 1105.513206]  [<c10c785c>] ? wait_on_retry_sync_kiocb+0x3f/0x3f
[ 1105.513206]  [<c10cd050>] kernel_read+0x30/0x41
[ 1105.513206]  [<c10cd8d6>] prepare_binprm+0xfb/0x103
[ 1105.513206]  [<c10cdeaa>] do_execve+0x327/0x456
[ 1105.513206]  [<c10cdc6e>] ? do_execve+0xeb/0x456
[ 1105.546433]  [<c1009165>] sys_execve+0x31/0x53
[ 1105.546433]  [<c167f962>] ptregs_execve+0x12/0x20
[ 1105.546433]  [<c167f863>] ? sysenter_do_call+0x12/0x38
[ 1105.546433] Code: 00 00 e8 6c ba fe ff 89 d8 e8 91 ff ff ff 89 c6 85 c0 74 10 8b 43 08 8b 40 04 ba 01 00 00 00 e8 dd b9 fe ff 89 f0 5b 5e 5d c3 55 <89> e5 89 e2 81 e2 00 e0 ff ff ff 42 14 8b 40 0c 64 ff 48 04 ff
[ 1105.546433] Kernel panic - not syncing: softlockup: hung tasks
[ 1105.546433] Pid: 123, comm: rc.local Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1105.546433] Call Trace:
[ 1105.546433]  [<c1673678>] panic+0x97/0x1aa
[ 1105.546433]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1105.546433]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1105.546433]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1105.546433]  [<c103205a>] run_local_timers+0x8/0x14
[ 1105.546433]  [<c103208a>] update_process_times+0x24/0x58
[ 1105.546433]  [<c106276e>] tick_periodic+0x66/0x72
[ 1105.546433]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1105.546433]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1105.546433]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1105.546433]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1105.546433]  [<c10d00d8>] ? pipe_fcntl+0xbc/0x140
[ 1105.546433]  [<c10df7c4>] ? __mnt_drop_write+0x1/0x1a
[ 1105.546433]  [<c10dd7bf>] ? touch_atime+0xdc/0x104
[ 1105.546433]  [<c1097f77>] generic_file_aio_read+0x5b5/0x5f1
[ 1105.546433]  [<c10c78d1>] do_sync_read+0x75/0xb0
[ 1105.546433]  [<c10c7faa>] vfs_read+0x8b/0x10d
[ 1105.546433]  [<c10af901>] ? might_fault+0x26/0x99
[ 1105.546433]  [<c10c785c>] ? wait_on_retry_sync_kiocb+0x3f/0x3f
[ 1105.546433]  [<c10cd050>] kernel_read+0x30/0x41
[ 1105.546433]  [<c10cd8d6>] prepare_binprm+0xfb/0x103
[ 1105.546433]  [<c10cdeaa>] do_execve+0x327/0x456
[ 1105.546433]  [<c10cdc6e>] ? do_execve+0xeb/0x456
[ 1105.546433]  [<c1009165>] sys_execve+0x31/0x53
[ 1105.546433]  [<c167f962>] ptregs_execve+0x12/0x20
[ 1105.546433]  [<c167f863>] ? sysenter_do_call+0x12/0x38
[ 1105.546433] Rebooting in 10 seconds..
--
[    4.563365] Freeing unused kernel memory: 464k freed
[ 1103.514494] BUG: soft lockup - CPU#1 stuck for 1024s! [rc.local:122]
[ 1103.514494] irq event stamp: 8
[ 1103.514494] hardirqs last  enabled at (7): [<c101cfda>] do_page_fault+0x98/0x36e
[ 1103.514494] hardirqs last disabled at (8): [<c166f4bf>] apic_timer_interrupt+0x2f/0x40
[ 1103.514494] softirqs last  enabled at (0): [<c1025204>] copy_process+0x471/0xf82
[ 1103.514494] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1103.514494] Pid: 122, comm: rc.local Not tainted 3.5.0-bisect2-00313-g7cd58b0 #86  
[ 1103.520829] [sched_delayed] sched: RT throttling activated
[ 1103.514494] EIP: 0060:[<c10ae5c3>] EFLAGS: 00000212 CPU: 1
[ 1103.514494] EIP is at find_vma+0x48/0x68
[ 1103.514494] EAX: ce816780 EBX: ceea0d68 ECX: ceea0ddc EDX: b7654d0c
[ 1103.514494] ESI: ce849fb4 EDI: c101c2e6 EBP: ce849f20 ESP: ce849f1c
[ 1103.514494]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.514494] CR0: 8005003b CR2: b7654d0c CR3: 0e842000 CR4: 00000690
[ 1103.514494] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.514494] DR6: ffff0ff0 DR7: 00000400
[ 1103.514494] Process rc.local (pid: 122, ti=ce848000 task=cf1fd5a0 task.ti=ce848000)
[ 1103.514494] Stack:
[ 1103.514494]  ce849fb4 ce849f9c c101d0e5 00000000 00000000 cf1fd5a0 ce8167d4 ce816780
[ 1103.514494]  cf1fd5a0 00000028 00000004 b7654d0c c12aa250 00000001 00000003 00000000
[ 1103.514494]  c166e9b5 bffffffd b75a0728 00000000 00000001 ce816780 00000007 00000006
[ 1103.514494] Call Trace:
[ 1103.514494]  [<c101d0e5>] do_page_fault+0x1a3/0x36e
[ 1103.514494]  [<c12aa250>] ? trace_hardirqs_on_thunk+0xc/0x10
[ 1103.514494]  [<c166e9b5>] ? restore_all+0xf/0xf
[ 1103.514494]  [<c166e9b5>] ? restore_all+0xf/0xf
[ 1103.514494]  [<c101c2e6>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.514494]  [<c101c309>] do_async_page_fault+0x23/0x5f
[ 1103.514494]  [<c166f6fc>] error_code+0x6c/0x80
[ 1103.514494] Code: e8 2e 81 f7 ff c6 05 05 94 95 c1 01 eb 38 8b 58 08 85 db 74 0a 39 53 08 76 05 39 53 04 76 27 8b 48 04 31 db eb 15 39 51 ec 76 0d <8d> 59 e4 39 51 e8 76 0c 8b 49 08 eb 03 8b 49 04 85 c9 75 e7 85
[ 1103.514494] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.514494] Pid: 122, comm: rc.local Not tainted 3.5.0-bisect2-00313-g7cd58b0 #86
[ 1103.514494] Call Trace:
[ 1103.514494]  [<c1662a5f>] panic+0x81/0x194
[ 1103.514494]  [<c107e1de>] watchdog_timer_fn+0x115/0x134
[ 1103.514494]  [<c10413be>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.514494]  [<c107e0c9>] ? __touch_watchdog+0x1d/0x1d
[ 1103.514494]  [<c10305c6>] run_local_timers+0x8/0x14
[ 1103.514494]  [<c10305f6>] update_process_times+0x24/0x58
[ 1103.514494]  [<c101c2e6>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.514494]  [<c1060d2e>] tick_periodic+0x66/0x72
[ 1103.514494]  [<c1060d88>] tick_handle_periodic+0x19/0x61
[ 1103.514494]  [<c101c2e6>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.514494]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.514494]  [<c12aa260>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.514494]  [<c166f4c6>] apic_timer_interrupt+0x36/0x40
[ 1103.514494]  [<c101c2e6>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.514494]  [<c101007b>] ? intel_pmu_lbr_init_core+0x34/0x3a
[ 1103.514494]  [<c10400e0>] ? posix_cpu_timer_set+0x29e/0x361
[ 1103.514494]  [<c10ae5c3>] ? find_vma+0x48/0x68
[ 1103.514494]  [<c101d0e5>] do_page_fault+0x1a3/0x36e
[ 1103.514494]  [<c12aa250>] ? trace_hardirqs_on_thunk+0xc/0x10
[ 1103.514494]  [<c166e9b5>] ? restore_all+0xf/0xf
[ 1103.514494]  [<c166e9b5>] ? restore_all+0xf/0xf
[ 1103.514494]  [<c101c2e6>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.514494]  [<c101c309>] do_async_page_fault+0x23/0x5f
[ 1103.514494]  [<c166f6fc>] error_code+0x6c/0x80
[ 1103.514494] Rebooting in 10 seconds..
--
[    4.379731] Freeing unused kernel memory: 464k freed
[ 1103.518739] BUG: soft lockup - CPU#1 stuck for 1024s! [init:1]
[ 1103.518739] irq event stamp: 761948
[ 1103.518739] hardirqs last  enabled at (761947): [<c101cfda>] do_page_fault+0x98/0x36e
[ 1103.518739] hardirqs last disabled at (761948): [<c166f4bf>] apic_timer_interrupt+0x2f/0x40
[ 1103.518739] softirqs last  enabled at (761912): [<c15f048e>] unix_accept+0xd2/0xe8
[ 1103.518739] softirqs last disabled at (761910): mountall: ply-ev[<c166e66a>] _raw_write_lock_bh+0xe/0x4f
ent-loop.c:497: [ 1103.518739] Pid: 1, comm: init Not tainted 3.5.0-bisect2-00313-g7cd58b0 #86  
ply_event_loop_n[ 1103.518739] EIP: 0060:[<c1065aea>] EFLAGS: 00000286 CPU: 1
ew: Assertion `l[ 1103.518739] EIP is at lock_release+0x194/0x19f
oop->epoll_fd >=[ 1103.518739] EAX: 00000286 EBX: ce040000 ECX: 0000036f EDX: 0000036f
 0' failed.
[ 1103.518739] ESI: 0000036f EDI: 00000000 EBP: ce03de60 ESP: ce03de4c
[ 1103.528082] [sched_delayed] sched: RT throttling activated
[ 1103.518739]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.518739] CR0: 80050033 CR2: b7665b90 CR3: 0f1ea000 CR4: 00000690
[ 1103.518739] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.518739] DR6: ffff0ff0 DR7: 00000400
[ 1103.518739] Process init (pid: 1, ti=ce03c000 task=ce040000 task.ti=ce03c000)
[ 1103.518739] Stack:
[ 1103.518739]  00000001 00000286 ce116534 00000000 00000000 ce03de6c c166e11b ce68afe0
[ 1103.518739]  ce03deb8 c10a95e0 c10acb05 ce116534 cf1eb994 00000000 b7665b90 ce116500
[ 1103.518739]  00000000 00000200 cf81cf20 0a5ff025 00000028 0000000a b7665000 ce68afe0
[ 1103.518739] Call Trace:
[ 1103.518739]  [<c166e11b>] _raw_spin_unlock+0x16/0x2a
[ 1103.518739]  [<c10a95e0>] __do_fault+0x2fa/0x38a
[ 1103.518739]  [<c10acb05>] ? might_fault+0x90/0x97
[ 1103.518739]  [<c10abbf5>] handle_pte_fault+0x2c3/0x74c
[ 1103.518739]  [<c102055f>] ? kmap_atomic_prot+0x11f/0x133
[ 1103.518739]  [<c10ac113>] handle_mm_fault+0x95/0xa4
[ 1103.518739]  [<c101c2e6>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.518739]  [<c101d278>] do_page_fault+0x336/0x36e
[ 1103.518739]  [<c12aa4e6>] ? _copy_from_user+0x37/0x4b
[ 1103.518739]  [<c159f160>] ? sys_socketcall+0xf1/0x2b2
[ 1103.518739]  [<c166e9b5>] ? restore_all+0xf/0xf
[ 1103.518739]  [<c101c2e6>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.518739]  [<c101c2e6>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.518739]  [<c101c309>] do_async_page_fault+0x23/0x5f
[ 1103.518739]  [<c166f6fc>] error_code+0x6c/0x80
[ 1103.518739] Code: 85 c0 74 07 89 d8 e8 0a c5 ff ff 64 a1 28 a6 9c c1 c7 80 54 04 00 00 00 00 00 00 83 3d 6c d4 8f c1 00 75 02 0f 0b 8b 45 f0 50 9d <8d> 74 26 00 5f 58 5b 5e 5f 5d c3 55 89 e5 57 56 53 83 ec 20 89
[ 1103.518739] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.518739] Pid: 1, comm: init Not tainted 3.5.0-bisect2-00313-g7cd58b0 #86
[ 1103.518739] Call Trace:
[ 1103.518739]  [<c1662a5f>] panic+0x81/0x194
[ 1103.518739]  [<c107e1de>] watchdog_timer_fn+0x115/0x134
[ 1103.518739]  [<c10413be>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.518739]  [<c107e0c9>] ? __touch_watchdog+0x1d/0x1d
[ 1103.518739]  [<c10305c6>] run_local_timers+0x8/0x14
[ 1103.518739]  [<c10305f6>] update_process_times+0x24/0x58
[ 1103.518739]  [<c1060d2e>] tick_periodic+0x66/0x72
[ 1103.518739]  [<c1060d88>] tick_handle_periodic+0x19/0x61
[ 1103.518739]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.518739]  [<c12aa260>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.518739]  [<c166f4c6>] apic_timer_interrupt+0x36/0x40
[ 1103.518739]  [<c1065aea>] ? lock_release+0x194/0x19f
[ 1103.518739]  [<c166e11b>] _raw_spin_unlock+0x16/0x2a
[ 1103.518739]  [<c10a95e0>] __do_fault+0x2fa/0x38a
[ 1103.518739]  [<c10acb05>] ? might_fault+0x90/0x97
[ 1103.518739]  [<c10abbf5>] handle_pte_fault+0x2c3/0x74c
[ 1103.518739]  [<c102055f>] ? kmap_atomic_prot+0x11f/0x133
[ 1103.518739]  [<c10ac113>] handle_mm_fault+0x95/0xa4
[ 1103.518739]  [<c101c2e6>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.518739]  [<c101d278>] do_page_fault+0x336/0x36e
[ 1103.518739]  [<c12aa4e6>] ? _copy_from_user+0x37/0x4b
[ 1103.518739]  [<c159f160>] ? sys_socketcall+0xf1/0x2b2
[ 1103.518739]  [<c166e9b5>] ? restore_all+0xf/0xf
[ 1103.518739]  [<c101c2e6>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.518739]  [<c101c2e6>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.518739]  [<c101c309>] do_async_page_fault+0x23/0x5f
[ 1103.518739]  [<c166f6fc>] error_code+0x6c/0x80
[ 1103.518739] Rebooting in 10 seconds..
--
[    5.795207] Freeing unused kernel memory: 464k freed
[ 1104.515634] BUG: soft lockup - CPU#1 stuck for 1023s! [99-trinity:126]
[ 1104.515634] irq event stamp: 732
[ 1104.515634] hardirqs last  enabled at (731): [<c101cfce>] do_page_fault+0x98/0x36e
[ 1104.515634] hardirqs last disabled at (732): [<c166ecff>] apic_timer_interrupt+0x2f/0x40
[ 1104.515634] softirqs last  enabled at (0): [<c1025204>] copy_process+0x471/0xf82
[ 1104.515634] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1104.515634] Pid: 126, comm: 99-trinity Not tainted 3.5.0-bisect2-01297-ga17f29a #84  
[ 1104.515634] EIP: 0060:[<c1064337>] EFLAGS: 00000286 CPU: 1
[ 1104.515634] EIP is at lock_is_held+0x9f/0xb2
[ 1104.515634] EAX: 00000286 EBX: 00000001 ECX: 00000000 EDX: 00000000
[ 1104.515634] ESI: ce859120 EDI: c18feae8 EBP: ce869ef8 ESP: ce869ee8
[ 1104.515634]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.515634] CR0: 8005003b CR2: b7779ee0 CR3: 0e849000 CR4: 00000690
[ 1104.515634] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.515634] DR6: ffff0ff0 DR7: 00000400
[ 1104.515634] Process 99-trinity (pid: 126, ti=ce868000 task=ce859120 task.ti=ce868000)
[ 1104.515634] Stack:
[ 1104.515634]  00000286 0000046c c180ae2c 00000000 ce869f20 c1049cb6 c101c2da ce869f20
[ 1104.515634]  c10415f1 00000001 00000002 ce869fb4 ce869fb4 c101c2da ce869f9c c101d0c9
[ 1104.515634]  00000028 00000000 00000000 ce834f54 ce834f00 ce859120 00000029 00000007
[ 1104.515634] Call Trace:
[ 1104.515634]  [<c1049cb6>] __might_sleep+0x66/0x1cd
[ 1104.515634]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.515634]  [<c10415f1>] ? down_read_trylock+0x51/0x59
[ 1104.515634]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.515634]  [<c101d0c9>] do_page_fault+0x193/0x36e
[ 1104.515634]  [<c104168e>] ? up_write+0x16/0x2b
[ 1104.515634]  [<c10a3f6b>] ? vm_mmap_pgoff+0x50/0x59
[ 1104.515634]  [<c166e1f5>] ? restore_all+0xf/0xf
[ 1104.515634]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.515634]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.515634]  [<c101c2fd>] do_async_page_fault+0x23/0x5f
[ 1104.515634]  [<c166ef3c>] error_code+0x6c/0x80
[ 1104.515634] Code: 7c e2 31 c9 eb 05 b9 01 00 00 00 64 a1 28 a6 9c c1 c7 80 54 04 00 00 00 00 00 00 83 3d 6c d4 8f c1 00 75 02 0f 0b 8b 45 f0 50 9d <8d> 74 26 00 eb 05 b9 01 00 00 00 89 c8 5b 5b 5e 5f 5d c3 55 89
[ 1104.515634] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.515634] Pid: 126, comm: 99-trinity Not tainted 3.5.0-bisect2-01297-ga17f29a #84
[ 1104.515634] Call Trace:
[ 1104.515634]  [<c16622f2>] panic+0x81/0x194
[ 1104.515634]  [<c107e1d2>] watchdog_timer_fn+0x115/0x134
[ 1104.515634]  [<c10413fe>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.515634]  [<c107e0bd>] ? __touch_watchdog+0x1d/0x1d
[ 1104.515634]  [<c10305c6>] run_local_timers+0x8/0x14
[ 1104.515634]  [<c10305f6>] update_process_times+0x24/0x58
[ 1104.515634]  [<c1060d92>] tick_periodic+0x66/0x72
[ 1104.515634]  [<c1060dec>] tick_handle_periodic+0x19/0x61
[ 1104.515634]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.515634]  [<c12aa840>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.515634]  [<c166ed06>] apic_timer_interrupt+0x36/0x40
[ 1104.515634]  [<c1064337>] ? lock_is_held+0x9f/0xb2
[ 1104.515634]  [<c1049cb6>] __might_sleep+0x66/0x1cd
[ 1104.515634]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.515634]  [<c10415f1>] ? down_read_trylock+0x51/0x59
[ 1104.515634]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.515634]  [<c101d0c9>] do_page_fault+0x193/0x36e
[ 1104.515634]  [<c104168e>] ? up_write+0x16/0x2b
[ 1104.515634]  [<c10a3f6b>] ? vm_mmap_pgoff+0x50/0x59
[ 1104.515634]  [<c166e1f5>] ? restore_all+0xf/0xf
[ 1104.515634]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.515634]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.515634]  [<c101c2fd>] do_async_page_fault+0x23/0x5f
[ 1104.515634]  [<c166ef3c>] error_code+0x6c/0x80
[ 1104.515634] Rebooting in 10 seconds..
--
[    4.912862] Freeing unused kernel memory: 460k freed
[ 1103.516433] BUG: soft lockup - CPU#1 stuck for 1023s! [99-trinity:128]
[ 1103.516433] irq event stamp: 32
[ 1103.516433] hardirqs last  enabled at (31): [<c101e5d2>] do_page_fault+0x98/0x36e
[ 1103.516433] hardirqs last disabled at (32): [<c167f36f>] apic_timer_interrupt+0x2f/0x40
[ 1103.516433] softirqs last  enabled at (0): [<c1026857>] copy_process+0x465/0xf8a
[ 1103.516433] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1103.516433] Pid: 128, comm: 99-trinity Not tainted 3.6.0-rc1-bisect2 #80  
[ 1103.516433] EIP: 0060:[<c109bf33>] EFLAGS: 00000202 CPU: 1
[ 1103.516433] EIP is at get_page_from_freelist+0x1/0x456
[ 1103.516433] EAX: 000200da EBX: 000200da ECX: 00000000 EDX: 00000000
[ 1103.516433] ESI: 00000003 EDI: c196cfa0 EBP: ce857e78 ESP: ce857de0
[ 1103.516433]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.516433] CR0: 8005003b CR2: b7779460 CR3: 0e878000 CR4: 00000690
[ 1103.516433] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.516433] DR6: ffff0ff0 DR7: 00000400
[ 1103.516433] Process 99-trinity (pid: 128, ti=ce856000 task=cf8b2260 task.ti=ce856000)
[ 1103.516433] Stack:
[ 1103.516433]  ce857e78 c109c83a c196cfa0 00000003 00000041 c196c490 00000002 c1823e91
[ 1103.516433]  00000000 ce857e2c 00000000 00000000 cf8b2260 c1913698 ce857e30 00000282
[ 1103.516433]  00000282 000000aa c182c5ad 000200da 00000010 00000282 0000037b 0000037b
[ 1103.516433] Call Trace:
[ 1103.516433]  [<c109c83a>] ? __alloc_pages_nodemask+0x11e/0x5f3
[ 1103.516433]  [<c10b47b1>] ? anon_vma_prepare+0x24/0x131
[ 1103.516433]  [<c10ace14>] do_wp_page+0x40d/0x63c
[ 1103.516433]  [<c167e689>] ? _raw_spin_lock+0x44/0x4a
[ 1103.516433]  [<c10aecb2>] handle_pte_fault+0x758/0x7a7
[ 1103.516433]  [<c10aeda2>] handle_mm_fault+0xa1/0xb0
[ 1103.516433]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.516433]  [<c101e870>] do_page_fault+0x336/0x36e
[ 1103.516433]  [<c12b3690>] ? trace_hardirqs_on_thunk+0xc/0x10
[ 1103.516433]  [<c167f065>] ? restore_all+0xf/0xf
[ 1103.516433]  [<c167f065>] ? restore_all+0xf/0xf
[ 1103.516433]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.516433]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.516433]  [<c101d8ab>] do_async_page_fault+0x23/0x5f
[ 1103.516433]  [<c167f5ac>] error_code+0x6c/0x80
[ 1103.516433] Code: be 00 10 00 00 8a 4d e4 d3 e6 89 d8 e8 e6 f4 00 00 89 f2 e8 b2 c2 fc ff 89 d8 e8 d8 f4 00 00 89 f2 e8 c5 83 22 00 b0 01 eb ce 55 <89> e5 57 56 53 83 ec 5c 89 45 dc 89 55 c0 89 4d e4 8b 45 14 2b
[ 1103.516433] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.516433] Pid: 128, comm: 99-trinity Not tainted 3.6.0-rc1-bisect2 #80
[ 1103.516433] Call Trace:
[ 1103.516433]  [<c1673658>] panic+0x97/0x1aa
[ 1103.516433]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1103.516433]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.516433]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1103.516433]  [<c103205a>] run_local_timers+0x8/0x14
[ 1103.516433]  [<c103208a>] update_process_times+0x24/0x58
[ 1103.516433]  [<c106276e>] tick_periodic+0x66/0x72
[ 1103.516433]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1103.516433]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.516433]  [<c12b36a0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.516433]  [<c167f376>] apic_timer_interrupt+0x36/0x40
[ 1103.516433]  [<c109bf33>] ? get_page_from_freelist+0x1/0x456
[ 1103.516433]  [<c109c83a>] ? __alloc_pages_nodemask+0x11e/0x5f3
[ 1103.516433]  [<c10b47b1>] ? anon_vma_prepare+0x24/0x131
[ 1103.516433]  [<c10ace14>] do_wp_page+0x40d/0x63c
[ 1103.516433]  [<c167e689>] ? _raw_spin_lock+0x44/0x4a
[ 1103.516433]  [<c10aecb2>] handle_pte_fault+0x758/0x7a7
[ 1103.516433]  [<c10aeda2>] handle_mm_fault+0xa1/0xb0
[ 1103.516433]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.516433]  [<c101e870>] do_page_fault+0x336/0x36e
[ 1103.516433]  [<c12b3690>] ? trace_hardirqs_on_thunk+0xc/0x10
[ 1103.516433]  [<c167f065>] ? restore_all+0xf/0xf
[ 1103.516433]  [<c167f065>] ? restore_all+0xf/0xf
[ 1103.516433]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.516433]  [<c101d888>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.516433]  [<c101d8ab>] do_async_page_fault+0x23/0x5f
[ 1103.516433]  [<c167f5ac>] error_code+0x6c/0x80
[ 1103.516433] Rebooting in 10 seconds..
--
[    5.018929] Freeing unused kernel memory: 464k freed
[ 1104.515153] BUG: soft lockup - CPU#1 stuck for 1025s! [sh:135]
[ 1104.515153] irq event stamp: 3314
[ 1104.515153] hardirqs last  enabled at (3313): [<c16707f5>] restore_all_notrace+0x0/0x18
[ 1104.515153] hardirqs last disabled at (3314): [<c16712ff>] apic_timer_interrupt+0x2f/0x40
[ 1104.515153] softirqs last  enabled at (3026): [<c15f31f2>] unix_sock_destructor+0xae/0xb3
[ 1104.515153] softirqs last disabled at (3024): [<c15f31df>] unix_sock_destructor+0x9b/0xb3
[ 1104.515153] Pid: 135, comm: sh Not tainted 3.5.0-bisect2-05018-g614a6d43 #82  
[ 1104.515153] EIP: 0060:[<c1095f63>] EFLAGS: 00000296 CPU: 1
[ 1104.515153] EIP is at generic_file_aio_read+0x27f/0x5f1
[ 1104.524187] [sched_delayed] sched: RT throttling activated
[ 1104.515153] EAX: ce688fe0 EBX: 00000001 ECX: 00000000 EDX: 00000000
[ 1104.515153] ESI: ffffffff EDI: 00000000 EBP: ce893ecc ESP: ce893e44
[ 1104.515153]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.515153] CR0: 80050033 CR2: b75d4198 CR3: 0e8a5000 CR4: 00000690
[ 1104.515153] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.515153] DR6: ffff0ff0 DR7: 00000400
[ 1104.515153] Process sh (pid: 135, ti=ce892000 task=cf8b3360 task.ti=ce892000)
[ 1104.515153] Stack:
[ 1104.515153]  00000001 00000000 ce893e68 00000000 00000000 00000000 00000001 ce8a4c84
[ 1104.515153]  ce893f10 00000000 00000001 00000fff ffffffff cb4d6210 ce893f54 00000000
[ 1104.515153]  ce8a4c00 00000000 cb4d6328 00000001 c10c8890 00000001 00000000 c1bb0d04
[ 1104.515153] Call Trace:
[ 1104.515153]  [<c10c8890>] ? path_put+0x15/0x18
[ 1104.515153]  [<c10bf446>] do_sync_read+0x9e/0xd6
[ 1104.515153]  [<c10bfb1c>] vfs_read+0x8b/0x10d
[ 1104.515153]  [<c10bf3a8>] ? wait_on_retry_sync_kiocb+0x3f/0x3f
[ 1104.515153]  [<c10bfdc0>] sys_read+0x3d/0x61
[ 1104.515153]  [<c16707bc>] syscall_call+0x7/0xb
[ 1104.515153] Code: a8 89 df 31 c9 ba 52 04 00 00 b8 3a 8f 81 c1 e8 04 40 fb ff e8 ed 92 5d 00 8b 5d 90 29 fb 89 5d a0 89 fa 8b 45 c0 e8 fc ee ff ff <89> c6 85 c0 75 2c 8b 5d a0 89 5c 24 04 89 3c 24 8b 4d b8 8b 55
[ 1104.515153] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.515153] Pid: 135, comm: sh Not tainted 3.5.0-bisect2-05018-g614a6d43 #82
[ 1104.515153] Call Trace:
[ 1104.515153]  [<c1664e92>] panic+0x81/0x194
[ 1104.515153]  [<c107e316>] watchdog_timer_fn+0x115/0x134
[ 1104.515153]  [<c10416fe>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.515153]  [<c1065a8e>] ? __lock_acquire+0x1444/0x1455
[ 1104.515153]  [<c107e201>] ? __touch_watchdog+0x1d/0x1d
[ 1104.515153]  [<c10306a2>] run_local_timers+0x8/0x14
[ 1104.515153]  [<c10306d2>] update_process_times+0x24/0x58
[ 1104.515153]  [<c1061092>] tick_periodic+0x66/0x72
[ 1104.515153]  [<c10610ec>] tick_handle_periodic+0x19/0x61
[ 1104.515153]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.515153]  [<c12aa9a0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.515153]  [<c1671306>] apic_timer_interrupt+0x36/0x40
[ 1104.515153]  [<c109007b>] ? perf_event_comm_output+0x31/0xe9
[ 1104.515153]  [<c1095f63>] ? generic_file_aio_read+0x27f/0x5f1
[ 1104.515153]  [<c10c8890>] ? path_put+0x15/0x18
[ 1104.515153]  [<c10bf446>] do_sync_read+0x9e/0xd6
[ 1104.515153]  [<c10bfb1c>] vfs_read+0x8b/0x10d
[ 1104.515153]  [<c10bf3a8>] ? wait_on_retry_sync_kiocb+0x3f/0x3f
[ 1104.515153]  [<c10bfdc0>] sys_read+0x3d/0x61
[ 1104.515153]  [<c16707bc>] syscall_call+0x7/0xb
[ 1104.515153] Rebooting in 10 seconds..
--
[    5.793227] Freeing unused kernel memory: 464k freed
[ 1104.521267] BUG: soft lockup - CPU#1 stuck for 1023s! [swapper/1:0]
[ 1104.521267] irq event stamp: 13836
[ 1104.521267] hardirqs last  enabled at (13833): [<c166e1f5>] restore_all_notrace+0x0/0x18
[ 1104.521267] hardirqs last disabled at (13834): [<c166ecff>] apic_timer_interrupt+0x2f/0x40
[ 1104.521267] softirqs last  enabled at (13836): [ 1104.527910] [sched_delayed] sched: RT throttling activated
[<c102cbbf>] _local_bh_enable+0xd/0xf
[ 1104.528470] softirqs last disabled at (13835): [<c102d02c>] irq_enter+0x2c/0x4d
[ 1104.528470] Pid: 0, comm: swapper/1 Not tainted 3.5.0-bisect2-01297-ga17f29a #84  
[ 1104.528470] EIP: 0060:[<c101c42e>] EFLAGS: 00000282 CPU: 1
[ 1104.528470] EIP is at native_safe_halt+0x5/0x7
[ 1104.528470] EAX: ce060000 EBX: ffffffff ECX: ce060000 EDX: 00000007
[ 1104.528470] ESI: c195a818 EDI: ffff8b9e EBP: ce05ff68 ESP: ce05ff68
[ 1104.528470]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.528470] CR0: 8005003b CR2: b75f5fb8 CR3: 0ee64000 CR4: 00000690
[ 1104.528470] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.528470] DR6: ffff0ff0 DR7: 00000400
[ 1104.528470] Process swapper/1 (pid: 0, ti=ce05e000 task=ce060000 task.ti=ce05e000)
[ 1104.528470] Stack:
[ 1104.528470]  ce05ff70 c10089d9 ce05ff88 c1009220 3c8ef9e1 4af27f04 853dc1b9 cfff9fa0
[ 1104.528470]  ce05ffb4 c165d2ba 00000000 00000000 00000000 00000000 f56cca54 853dc1b9
[ 1104.528470]  01000800 0000d248 00000000 00000000 00000000 00000000 00000000 00000000
[ 1104.528470] Call Trace:
[ 1104.528470]  [<c10089d9>] default_idle+0x2f/0x55
[ 1104.528470]  [<c1009220>] cpu_idle+0xc9/0xfb
[ 1104.528470]  [<c165d2ba>] start_secondary+0x292/0x29a
[ 1104.528470] Code: 0f 22 e0 5d c3 55 89 e5 0f 09 5d c3 55 89 e5 9c 58 5d c3 55 89 e5 50 9d 5d c3 55 89 e5 fa 5d c3 55 89 e5 fb 5d c3 55 89 e5 fb f4 <5d> c3 55 89 e5 f4 5d c3 55 89 e5 57 56 53 53 89 c6 89 55 f0 89
[ 1104.528470] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.528470] Pid: 0, comm: swapper/1 Not tainted 3.5.0-bisect2-01297-ga17f29a #84
[ 1104.528470] Call Trace:
[ 1104.528470]  [<c16622f2>] panic+0x81/0x194
[ 1104.528470]  [<c107e1d2>] watchdog_timer_fn+0x115/0x134
[ 1104.528470]  [<c10413fe>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.528470]  [<c107e0bd>] ? __touch_watchdog+0x1d/0x1d
[ 1104.528470]  [<c10305c6>] run_local_timers+0x8/0x14
[ 1104.528470]  [<c10305f6>] update_process_times+0x24/0x58
[ 1104.528470]  [<c1060d92>] tick_periodic+0x66/0x72
[ 1104.528470]  [<c1060dec>] tick_handle_periodic+0x19/0x61
[ 1104.528470]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.528470]  [<c12aa840>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.528470]  [<c166ed06>] apic_timer_interrupt+0x36/0x40
[ 1104.528470]  [<c101c42e>] ? native_safe_halt+0x5/0x7
[ 1104.528470]  [<c10089d9>] default_idle+0x2f/0x55
[ 1104.528470]  [<c1009220>] cpu_idle+0xc9/0xfb
[ 1104.528470]  [<c165d2ba>] start_secondary+0x292/0x29a
[ 1104.528470] Rebooting in 10 seconds..
--
[ 1104.519836] [sched_delayed] sched: RT throttling activated
[ 1104.520679] BUG: soft lockup - CPU#1 stuck for 1023s! [swapper/1:0]
[ 1104.520679] irq event stamp: 6818
[ 1104.520679] hardirqs last  enabled at (6815): [<c10089c9>] default_idle+0x1f/0x55
[ 1104.520679] hardirqs last disabled at (6816): [<c16712ff>] apic_timer_interrupt+0x2f/0x40
[ 1104.520679] softirqs last  enabled at (6818): [<c102cc9f>] _local_bh_enable+0xd/0xf
[ 1104.520679] softirqs last disabled at (6817): [<c102d10c>] irq_enter+0x2c/0x4d
[ 1104.520679] Pid: 0, comm: swapper/1 Not tainted 3.5.0-bisect2-05018-g614a6d43 #82  
[ 1104.520679] EIP: 0060:[<c101c512>] EFLAGS: 00000282 CPU: 1
[ 1104.520679] EIP is at native_safe_halt+0x5/0x7
[ 1104.520679] EAX: ce062240 EBX: ffffffff ECX: ce062240 EDX: 00000007
[ 1104.520679] ESI: c195cbf0 EDI: ffff8ba3 EBP: ce06bf68 ESP: ce06bf68
[ 1104.533562]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.533562] CR0: 8005003b CR2: b754c460 CR3: 0e88f000 CR4: 00000690
[ 1104.533562] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.533562] DR6: ffff0ff0 DR7: 00000400
[ 1104.533562] Process swapper/1 (pid: 0, ti=ce06a000 task=ce062240 task.ti=ce06a000)
[ 1104.533562] Stack:
[ 1104.533562]  ce06bf70 c10089d9 ce06bf88 c1009220 a752cb21 b5b65043 f001826a cfff9fa0
[ 1104.533562]  ce06bfb4 c166027a 00000000 00000000 00000000 00000000 60308b05 f001826a
[ 1104.533562]  01000800 0000d248 00000000 00000000 00000000 00000000 00000000 00000000
[ 1104.533562] Call Trace:
[ 1104.533562]  [<c10089d9>] default_idle+0x2f/0x55
[ 1104.533562]  [<c1009220>] cpu_idle+0xc9/0xfb
[ 1104.533562]  [<c166027a>] start_secondary+0x292/0x29a
[ 1104.533562] Code: 0f 22 e0 5d c3 55 89 e5 0f 09 5d c3 55 89 e5 9c 58 5d c3 55 89 e5 50 9d 5d c3 55 89 e5 fa 5d c3 55 89 e5 fb 5d c3 55 89 e5 fb f4 <5d> c3 55 89 e5 f4 5d c3 55 89 e5 57 56 53 53 89 c6 89 55 f0 89
[ 1104.533562] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.533562] Pid: 0, comm: swapper/1 Not tainted 3.5.0-bisect2-05018-g614a6d43 #82
[ 1104.533562] Call Trace:
[ 1104.533562]  [<c1664e92>] panic+0x81/0x194
[ 1104.533562]  [<c107e316>] watchdog_timer_fn+0x115/0x134
[ 1104.533562]  [<c10416fe>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.533562]  [<c107e201>] ? __touch_watchdog+0x1d/0x1d
[ 1104.533562]  [<c10306a2>] run_local_timers+0x8/0x14
[ 1104.533562]  [<c10306d2>] update_process_times+0x24/0x58
[ 1104.533562]  [<c1061092>] tick_periodic+0x66/0x72
[ 1104.533562]  [<c10610ec>] tick_handle_periodic+0x19/0x61
[ 1104.533562]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.533562]  [<c12aa9a0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.533562]  [<c1671306>] apic_timer_interrupt+0x36/0x40
[ 1104.533562]  [<c101c512>] ? native_safe_halt+0x5/0x7
[ 1104.533562]  [<c10089d9>] default_idle+0x2f/0x55
[ 1104.533562]  [<c1009220>] cpu_idle+0xc9/0xfb
[ 1104.533562]  [<c166027a>] start_secondary+0x292/0x29a
[ 1104.533562] Rebooting in 10 seconds..
--
[ 1104.522067] [sched_delayed] sched: RT throttling activated
[ 1104.522043] BUG: soft lockup - CPU#1 stuck for 1024s! [99-trinity:129]
[ 1104.522043] irq event stamp: 2948
[ 1104.522043] hardirqs last  enabled at (2947): [<c16707f5>] restore_all_notrace+0x0/0x18
[ 1104.522043] hardirqs last disabled at (2948): [<c16712ff>] apic_timer_interrupt+0x2f/0x40
[ 1104.522043] softirqs last  enabled at (2348): [<c15f31f2>] unix_sock_destructor+0xae/0xb3
[ 1104.522043] softirqs last disabled at (2346): [<c15f31df>] unix_sock_destructor+0x9b/0xb3
[ 1104.522043] Pid: 129, comm: 99-trinity Not tainted 3.5.0-bisect2-05018-g614a6d43 #82  
[ 1104.522043] EIP: 0060:[<c1670770>] EFLAGS: 00000246 CPU: 1
[ 1104.522043] EIP is at system_call+0x0/0x45
[ 1104.522043] EAX: 000000c0 EBX: 00000000 ECX: 0000c404 EDX: 00000005
[ 1104.522043] ESI: 00000802 EDI: 00000003 EBP: 00000000 ESP: ce881fe4
[ 1104.522043]  DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068
[ 1104.522043] CR0: 80050033 CR2: b752b267 CR3: 0e867000 CR4: 00000690
[ 1104.522043] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.522043] DR6: ffff0ff0 DR7: 00000400
[ 1104.522043] Process 99-trinity (pid: 129, ti=ce880000 task=cf8b3360 task.ti=ce880000)
[ 1104.522043] Stack:
[ 1104.522043]  b773fd63 00000073 00000246 bfc9319c 0000007b cccccccc cccccccc
[ 1104.522043] Call Trace:
[ 1104.522043] Code: 67 9f ff 5a 59 58 fa 8d b6 00 00 00 00 e8 3d a2 c3 ff 8b 4d 08 81 e1 2e fe 00 00 0f 85 0a 01 00 00 eb 7e 90 8d b4 26 00 00 00 00 <50> fc 0f a8 0f a0 06 1e 50 55 57 56 52 51 53 ba 7b 00 00 00 8e
[ 1104.522043] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.522043] Pid: 129, comm: 99-trinity Not tainted 3.5.0-bisect2-05018-g614a6d43 #82
[ 1104.522043] Call Trace:
[ 1104.522043]  [<c1664e92>] panic+0x81/0x194
[ 1104.522043]  [<c107e316>] watchdog_timer_fn+0x115/0x134
[ 1104.522043]  [<c10416fe>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.522043]  [<c107e201>] ? __touch_watchdog+0x1d/0x1d
[ 1104.522043]  [<c10306a2>] run_local_timers+0x8/0x14
[ 1104.522043]  [<c10306d2>] update_process_times+0x24/0x58
[ 1104.522043]  [<c1061092>] tick_periodic+0x66/0x72
[ 1104.522043]  [<c10610ec>] tick_handle_periodic+0x19/0x61
[ 1104.522043]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.522043]  [<c12aa9a0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.522043]  [<c1671306>] apic_timer_interrupt+0x36/0x40
[ 1104.522043]  [<c1670770>] ? resume_userspace+0x30/0x30
[ 1104.522043] Rebooting in 10 seconds..
--
[    5.773899] Freeing unused kernel memory: 464k freed
[ 1104.518996] BUG: soft lockup - CPU#1 stuck for 1023s! [sh:134]
[ 1104.518996] irq event stamp: 1020
[ 1104.518996] hardirqs last  enabled at (1019): [<c166b6e8>] __mutex_unlock_slowpath+0x114/0x12c
[ 1104.518996] hardirqs last disabled at (1020): [<c166e77f>] apic_timer_interrupt+0x2f/0x40
[ 1104.518996] softirqs last  enabled at (0): [<c1025204>] copy_process+0x471/0xf82
[ 1104.518996] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1104.518996] Pid: 134, comm: sh Not tainted 3.5.0-bisect2-00646-g5b160bd #85  
[ 1104.518996] EIP: 0060:[<c1065bcd>] EFLAGS: 00000286 CPU: 1
[ 1104.526690] [sched_delayed] sched: RT throttling activated
[ 1104.518996] EIP is at lock_acquire+0xb4/0xc0
[ 1104.518996] EAX: 00000286 EBX: cf8966c0 ECX: 000001e3 EDX: 000001e3
[ 1104.518996] ESI: 00000000 EDI: cb732cdc EBP: ce869e3c ESP: ce869e10
[ 1104.518996]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.518996] CR0: 8005003b CR2: 08124f6c CR3: 0e86c000 CR4: 00000690
[ 1104.518996] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.518996] DR6: ffff0ff0 DR7: 00000400
[ 1104.518996] Process sh (pid: 134, ti=ce868000 task=cf8966c0 task.ti=ce868000)
[ 1104.518996] Stack:
[ 1104.518996]  00000000 00000002 00000000 00000000 c10d0b7e 00000000 00000000 00000286
[ 1104.518996]  cb732ccc fffffffe cb732ccc ce869e58 c166d29a 00000000 00000002 00000000
[ 1104.518996]  c10d0b7e cb732c80 ce869e70 c10d0b7e fffffffe ce869ef4 fffffffe 00000041
[ 1104.518996] Call Trace:
[ 1104.518996]  [<c10d0b7e>] ? dput+0x37/0x143
[ 1104.518996]  [<c166d29a>] _raw_spin_lock+0x3d/0x4a
[ 1104.518996]  [<c10d0b7e>] ? dput+0x37/0x143
[ 1104.518996]  [<c10d0b7e>] dput+0x37/0x143
[ 1104.518996]  [<c10c86ac>] path_put+0xe/0x18
[ 1104.518996]  [<c10c8ad7>] terminate_walk+0x10/0x86
[ 1104.518996]  [<c10c9655>] path_lookupat+0x1cd/0x535
[ 1104.518996]  [<c10c99dc>] do_path_lookup+0x1f/0x8e
[ 1104.518996]  [<c10c9d2d>] user_path_at_empty+0x41/0x66
[ 1104.518996]  [<c101d248>] ? do_page_fault+0x312/0x36e
[ 1104.518996]  [<c10bbdaf>] ? kmem_cache_alloc+0xab/0xcb
[ 1104.518996]  [<c10c9d6c>] user_path_at+0x1a/0x1f
[ 1104.518996]  [<c10be6d0>] sys_faccessat+0x90/0x150
[ 1104.518996]  [<c166dc75>] ? restore_all+0xf/0xf
[ 1104.518996]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.518996]  [<c10be7b0>] sys_access+0x20/0x22
[ 1104.518996]  [<c166dc3c>] syscall_call+0x7/0xb
[ 1104.518996] Code: 45 08 89 04 24 8b 4d ec 89 f2 89 f8 e8 57 e7 ff ff c7 83 54 04 00 00 00 00 00 00 83 3d 6c b4 8f c1 00 75 02 0f 0b 8b 45 f0 50 9d <8d> 74 26 00 83 c4 20 5b 5e 5f 5d c3 55 89 e5 57 56 53 83 ec 1c
[ 1104.518996] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.518996] Pid: 134, comm: sh Not tainted 3.5.0-bisect2-00646-g5b160bd #85
[ 1104.518996] Call Trace:
[ 1104.518996]  [<c1661ced>] panic+0x81/0x194
[ 1104.518996]  [<c107e202>] watchdog_timer_fn+0x115/0x134
[ 1104.518996]  [<c10413be>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.518996]  [<c107e0ed>] ? __touch_watchdog+0x1d/0x1d
[ 1104.518996]  [<c10305c6>] run_local_timers+0x8/0x14
[ 1104.518996]  [<c10305f6>] update_process_times+0x24/0x58
[ 1104.518996]  [<c1060d52>] tick_periodic+0x66/0x72
[ 1104.518996]  [<c1060dac>] tick_handle_periodic+0x19/0x61
[ 1104.518996]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.518996]  [<c12aa280>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.518996]  [<c166e786>] apic_timer_interrupt+0x36/0x40
[ 1104.518996]  [<c1065bcd>] ? lock_acquire+0xb4/0xc0
[ 1104.518996]  [<c10d0b7e>] ? dput+0x37/0x143
[ 1104.518996]  [<c166d29a>] _raw_spin_lock+0x3d/0x4a
[ 1104.518996]  [<c10d0b7e>] ? dput+0x37/0x143
[ 1104.518996]  [<c10d0b7e>] dput+0x37/0x143
[ 1104.518996]  [<c10c86ac>] path_put+0xe/0x18
[ 1104.518996]  [<c10c8ad7>] terminate_walk+0x10/0x86
[ 1104.518996]  [<c10c9655>] path_lookupat+0x1cd/0x535
[ 1104.518996]  [<c10c99dc>] do_path_lookup+0x1f/0x8e
[ 1104.518996]  [<c10c9d2d>] user_path_at_empty+0x41/0x66
[ 1104.518996]  [<c101d248>] ? do_page_fault+0x312/0x36e
[ 1104.518996]  [<c10bbdaf>] ? kmem_cache_alloc+0xab/0xcb
[ 1104.518996]  [<c10c9d6c>] user_path_at+0x1a/0x1f
[ 1104.518996]  [<c10be6d0>] sys_faccessat+0x90/0x150
[ 1104.518996]  [<c166dc75>] ? restore_all+0xf/0xf
[ 1104.518996]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.518996]  [<c10be7b0>] sys_access+0x20/0x22
[ 1104.518996]  [<c166dc3c>] syscall_call+0x7/0xb
[ 1104.518996] Rebooting in 10 seconds..
--
[ 1104.515042] [sched_delayed] sched: RT throttling activated
[ 1104.515017] BUG: soft lockup - CPU#1 stuck for 1024s! [99-trinity:129]
[ 1104.515017] irq event stamp: 1404
[ 1104.515017] hardirqs last  enabled at (1403): [<c16707f5>] restore_all_notrace+0x0/0x18
[ 1104.515017] hardirqs last disabled at (1404): [<c16712ff>] apic_timer_interrupt+0x2f/0x40
[ 1104.515017] softirqs last  enabled at (0): [<c10252e4>] copy_process+0x471/0xf82
[ 1104.515017] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1104.515017] Pid: 129, comm: 99-trinity Not tainted 3.5.0-bisect2-05018-g614a6d43 #82  
[ 1104.515017] EIP: 0073:[<b77231e7>] EFLAGS: 00000202 CPU: 1
[ 1104.515017] EIP is at 0xb77231e7
[ 1104.515017] EAX: 08124f88 EBX: b7735ff4 ECX: b75466c0 EDX: 0805db8c
[ 1104.515017] ESI: b7736918 EDI: 00000000 EBP: 08124f50 ESP: bfba3268
[ 1104.515017]  DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 1104.515017] Process 99-trinity (pid: 129, ti=ce890000 task=cf8ad5a0 task.ti=ce890000)
[ 1104.515017] 
[ 1104.515017] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.515017] Pid: 129, comm: 99-trinity Not tainted 3.5.0-bisect2-05018-g614a6d43 #82
[ 1104.515017] Call Trace:
[ 1104.515017]  [<c1664e92>] panic+0x81/0x194
[ 1104.515017]  [<c107e316>] watchdog_timer_fn+0x115/0x134
[ 1104.515017]  [<c10416fe>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.515017]  [<c107e201>] ? __touch_watchdog+0x1d/0x1d
[ 1104.515017]  [<c10306a2>] run_local_timers+0x8/0x14
[ 1104.515017]  [<c10306d2>] update_process_times+0x24/0x58
[ 1104.515017]  [<c1061092>] tick_periodic+0x66/0x72
[ 1104.515017]  [<c10610ec>] tick_handle_periodic+0x19/0x61
[ 1104.515017]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.515017]  [<c12aa9a0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.515017]  [<c1671306>] apic_timer_interrupt+0x36/0x40
[ 1104.515017] Rebooting in 10 seconds..
--
[    4.455784] Freeing unused kernel memory: 464k freed
[ 1103.514772] BUG: soft lockup - CPU#1 stuck for 1024s! [init:1]
[ 1103.524384] [sched_delayed] sched: RT throttling activated
[ 1103.514772] irq event stamp: 759084
[ 1103.514772] hardirqs last  enabled at (759083): [<c166dc81>] _raw_spin_unlock_irqrestore+0x42/0x63
[ 1103.514772] hardirqs last disabled at (759084): [<c166efcf>] apic_timer_interrupt+0x2f/0x40
[ 1103.514772] softirqs last  enabled at (754272): [<c15f0a02>] unix_sock_destructor+0xae/0xb3
[ 1103.514772] softirqs last disabled at (754270): [<c15f09ef>] unix_sock_destructor+0x9b/0xb3
[ 1103.514772] 
[ 1103.514772] Pid: 1, comm: init Not tainted 3.5.0-bisect2-00242-g3992c03 #88  
[ 1103.514772] EIP: 0060:[<c166dc90>] EFLAGS: 00000282 CPU: 1
[ 1103.514772] EIP is at _raw_spin_unlock_irqrestore+0x51/0x63
[ 1103.514772] EAX: 00000282 EBX: 00000282 ECX: 00000000 EDX: 00000007
[ 1103.514772] ESI: ce8583e4 EDI: 00000001 EBP: ce03ddfc ESP: ce03ddf4
[ 1103.514772]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.514772] CR0: 80050033 CR2: 080b1bc0 CR3: 0e80b000 CR4: 00000690
[ 1103.514772] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.514772] DR6: ffff0ff0 DR7: 00000400
[ 1103.514772] Process init (pid: 1, ti=ce03c000 task=ce040000 task.ti=ce03c000)
[ 1103.514772] Stack:
[ 1103.514772]  00000000 ce858000 ce03de20 c104842a 00000282 ce8583e4 0cb64000 00000000
[ 1103.514772]  ce858000 00000000 00000007 ce03de28 c1048470 ce03de38 c102ec42 ce858000
[ 1103.514772]  00000000 ce03def4 c102fb22 003f0011 00000000 ce040470 00000000 ce040000
[ 1103.514772] Call Trace:
[ 1103.514772]  [<c104842a>] try_to_wake_up+0x248/0x266
[ 1103.514772]  [<c1048470>] wake_up_state+0xa/0xc
[ 1103.514772]  [<c102ec42>] ptrace_resume+0x76/0x7c
[ 1103.514772]  [<c102fb22>] ptrace_request+0x368/0x4f8
[ 1103.514772]  [<c106574a>] ? __lock_acquire+0x1444/0x1455
[ 1103.514772]  [<c106574a>] ? __lock_acquire+0x1444/0x1455
[ 1103.514772]  [<c10657d9>] ? lock_release_non_nested+0x7e/0x21b
[ 1103.514772]  [<c106574a>] ? __lock_acquire+0x1444/0x1455
[ 1103.514772]  [<c100ac76>] arch_ptrace+0x474/0x48a
[ 1103.514772]  [<c166dc81>] ? _raw_spin_unlock_irqrestore+0x42/0x63
[ 1103.514772]  [<c10638d8>] ? trace_hardirqs_on+0xb/0xd
[ 1103.514772]  [<c1046015>] ? wait_task_inactive+0x6f/0xc1
[ 1103.514772]  [<c102eefe>] ? ptrace_check_attach+0xb4/0xc4
[ 1103.514772]  [<c102f71e>] sys_ptrace+0x34f/0x379
[ 1103.514772]  [<c102f43d>] ? sys_ptrace+0x6e/0x379
[ 1103.514772]  [<c166f4a3>] sysenter_do_call+0x12/0x38
[ 1103.514772] Code: 3d 6c b4 8f c1 00 74 1d 89 d8 50 9d 8d 74 26 00 e8 7d 5d 9f ff eb 18 e8 4c 5c 9f ff 83 3d 6c b4 8f c1 00 75 02 0f 0b 89 d8 50 9d <8d> 74 26 00 89 e0 25 00 e0 ff ff ff 48 14 5b 5e 5d c3 55 89 e5 
[ 1103.514772] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.514772] Pid: 1, comm: init Not tainted 3.5.0-bisect2-00242-g3992c03 #88
[ 1103.514772] Call Trace:
[ 1103.514772]  [<c166256f>] panic+0x81/0x194
[ 1103.514772]  [<c107e1fe>] watchdog_timer_fn+0x115/0x134
[ 1103.514772]  [<c10413de>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.514772]  [<c107e0e9>] ? __touch_watchdog+0x1d/0x1d
[ 1103.514772]  [<c10305e6>] run_local_timers+0x8/0x14
[ 1103.514772]  [<c1030616>] update_process_times+0x24/0x58
[ 1103.514772]  [<c1060d4e>] tick_periodic+0x66/0x72
[ 1103.514772]  [<c1060da8>] tick_handle_periodic+0x19/0x61
[ 1103.514772]  [<c10176c5>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.514772]  [<c12aa280>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.514772]  [<c166efd6>] apic_timer_interrupt+0x36/0x40
[ 1103.514772]  [<c166dc90>] ? _raw_spin_unlock_irqrestore+0x51/0x63
[ 1103.514772]  [<c104842a>] try_to_wake_up+0x248/0x266
[ 1103.514772]  [<c1048470>] wake_up_state+0xa/0xc
[ 1103.514772]  [<c102ec42>] ptrace_resume+0x76/0x7c
[ 1103.514772]  [<c102fb22>] ptrace_request+0x368/0x4f8
[ 1103.514772]  [<c106574a>] ? __lock_acquire+0x1444/0x1455
[ 1103.514772]  [<c106574a>] ? __lock_acquire+0x1444/0x1455
[ 1103.514772]  [<c10657d9>] ? lock_release_non_nested+0x7e/0x21b
[ 1103.514772]  [<c106574a>] ? __lock_acquire+0x1444/0x1455
[ 1103.514772]  [<c100ac76>] arch_ptrace+0x474/0x48a
[ 1103.514772]  [<c166dc81>] ? _raw_spin_unlock_irqrestore+0x42/0x63
[ 1103.514772]  [<c10638d8>] ? trace_hardirqs_on+0xb/0xd
[ 1103.514772]  [<c1046015>] ? wait_task_inactive+0x6f/0xc1
[ 1103.514772]  [<c102eefe>] ? ptrace_check_attach+0xb4/0xc4
[ 1103.514772]  [<c102f71e>] sys_ptrace+0x34f/0x379
[ 1103.514772]  [<c102f43d>] ? sys_ptrace+0x6e/0x379
[ 1103.514772]  [<c166f4a3>] sysenter_do_call+0x12/0x38
[ 1103.514772] Rebooting in 10 seconds..
--
[    5.165476] Freeing unused kernel memory: 464k freed
[ 1104.519906] BUG: soft lockup - CPU#1 stuck for 1024s! [swapper/1:0]
[ 1104.519906] irq event stamp: 6640
[ 1104.519906] hardirqs last  enabled at (6637): [<c16707f5>] restore_all_notrace+0x0/0x18
[ 1104.519906] hardirqs last disabled at (6638): [<c16712ff>] apic_timer_interrupt+0x2f/0x40
[ 1104.519906] softirqs last  enabled at (6640): [<c102cc9f>] _local_bh_enable+0xd/0xf
[ 1104.519906] softirqs last disabled at (6639): [<c102d10c>] irq_enter+0x2c/0x4d
[ 1104.519906] Pid: 0, comm: swapper/1 Not tainted 3.5.0-bisect2-05018-g614a6d43 #82  
[ 1104.519906] EIP: 0060:[<c101c512>] EFLAGS: 00000282 CPU: 1
[ 1104.519906] EIP is at native_safe_halt+0x5/0x7
[ 1104.519906] EAX: ce062240 EBX: ffffffff ECX: ce062240 EDX: 00000007
[ 1104.519906] ESI: c195cbf0 EDI: ffff8b9e EBP: ce06bf68 ESP: ce06bf68
[ 1104.519906]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.519906] CR0: 8005003b CR2: b76acc50 CR3: 019dc000 CR4: 00000690
[ 1104.519906] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.519906] DR6: ffff0ff0 DR7: 00000400
[ 1104.519906] Process swapper/1 (pid: 0, ti=ce06a000 task=ce062240 task.ti=ce06a000)
[ 1104.519906] Stack:
[ 1104.519906]  ce06bf70 c10089d9 ce06bf88 c1009220 7b380d04 899b9224 c3e68548 cfff9fa0
[ 1104.519906]  ce06bfb4 c166027a 00000000 00000000 00000000 00000000 34158de5 c3e68548
[ 1104.519906]  01000800 0000d248 00000000 00000000 00000000 00000000 00000000 00000000
[ 1104.519906] Call Trace:
[ 1104.519906]  [<c10089d9>] default_idle+0x2f/0x55
[ 1104.519906]  [<c1009220>] cpu_idle+0xc9/0xfb
[ 1104.519906]  [<c166027a>] start_secondary+0x292/0x29a
[ 1104.519906] Code: 0f 22 e0 5d c3 55 89 e5 0f 09 5d c3 55 89 e5 9c 58 5d c3 55 89 e5 50 9d 5d c3 55 89 e5 fa 5d c3 55 89 e5 fb 5d c3 55 89 e5 fb f4 <5d> c3 55 89 e5 f4 5d c3 55 89 e5 57 56 53 53 89 c6 89 55 f0 89
[ 1104.519906] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.519906] Pid: 0, comm: swapper/1 Not tainted 3.5.0-bisect2-05018-g614a6d43 #82
[ 1104.519906] Call Trace:
[ 1104.519906]  [<c1664e92>] panic+0x81/0x194
[ 1104.519906]  [<c107e316>] watchdog_timer_fn+0x115/0x134
[ 1104.519906]  [<c10416fe>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.519906]  [<c107e201>] ? __touch_watchdog+0x1d/0x1d
[ 1104.519906]  [<c10306a2>] run_local_timers+0x8/0x14
[ 1104.519906]  [<c10306d2>] update_process_times+0x24/0x58
[ 1104.519906]  [<c1061092>] tick_periodic+0x66/0x72
[ 1104.519906]  [<c10610ec>] tick_handle_periodic+0x19/0x61
[ 1104.519906]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.519906]  [<c12aa9a0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.519906]  [<c1671306>] apic_timer_interrupt+0x36/0x40
[ 1104.519906]  [<c101c512>] ? native_safe_halt+0x5/0x7
[ 1104.519906]  [<c10089d9>] default_idle+0x2f/0x55
[ 1104.519906]  [<c1009220>] cpu_idle+0xc9/0xfb
[ 1104.519906]  [<c166027a>] start_secondary+0x292/0x29a
[ 1104.519906] Rebooting in 10 seconds..
--
[    4.683031] Freeing unused kernel memory: 460k freed
[ 1103.515937] BUG: soft lockup - CPU#1 stuck for 1024s! [chroot:134]
[ 1103.515937] irq event stamp: 3200
[ 1103.515937] hardirqs last  enabled at (3199): [<c106831d>] debug_check_no_locks_freed+0x14b/0x167
[ 1103.515937] hardirqs last disabled at (3200): [<c167f36f>] apic_timer_interrupt+0x2f/0x40
[ 1103.515937] softirqs last  enabled at (3056): [<c1601106>] unix_sock_destructor+0xae/0xb3
[ 1103.515937] softirqs last disabled at (3054): [<c16010f3>] unix_sock_destructor+0x9b/0xb3
[ 1103.515937] Pid: 134, comm: chroot Not tainted 3.6.0-rc1-bisect2 #80  
[ 1103.515937] EIP: 0060:[<c1065d13>] EFLAGS: 00000282 CPU: 1
[ 1103.515937] EIP is at lock_is_held+0x9f/0xb2
[ 1103.515937] EAX: 00000282 EBX: 00000001 ECX: 00000000 EDX: 00000000
[ 1103.515937] ESI: ce8b2260 EDI: c1913698 EBP: ce8c7f0c ESP: ce8c7efc
[ 1103.515937]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.515937] CR0: 8005003b CR2: 080b6730 CR3: 0e8be000 CR4: 00000690
[ 1103.525323] [sched_delayed] sched: RT throttling activated
[ 1103.515937] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.515937] DR6: ffff0ff0 DR7: 00000400
[ 1103.515937] Process chroot (pid: 134, ti=ce8c6000 task=ce8b2260 task.ti=ce8c6000)
[ 1103.515937] Stack:
[ 1103.515937]  00000282 000001c5 c182dba3 00000000 ce8c7f34 c104bb01 bfc5c5fc ce8c7f34
[ 1103.515937]  c10af78b 00000001 00000002 00000008 ce8b2260 bfc5c5fc ce8c7f48 c10cc958
[ 1103.515937]  c7b3a300 00000006 00000001 ce8c7f88 c10cde5f 00000002 00000001 00000000
[ 1103.515937] Call Trace:
[ 1103.515937]  [<c104bb01>] __might_sleep+0xa1/0x1cd
[ 1103.515937]  [<c10af78b>] ? might_fault+0x90/0x99
[ 1103.515937]  [<c10cc958>] count.constprop.35+0x78/0x82
[ 1103.515937]  [<c10cde5f>] do_execve+0x310/0x456
[ 1103.515937]  [<c10cdc3a>] ? do_execve+0xeb/0x456
[ 1103.515937]  [<c1009165>] sys_execve+0x31/0x53
[ 1103.515937]  [<c167f942>] ptregs_execve+0x12/0x20
[ 1103.515937]  [<c167f843>] ? sysenter_do_call+0x12/0x38
[ 1103.515937] Code: 7c e2 31 c9 eb 05 b9 01 00 00 00 64 a1 28 06 9e c1 c7 80 58 04 00 00 00 00 00 00 83 3d 8c 20 91 c1 00 75 02 0f 0b 8b 45 f0 50 9d <8d> 74 26 00 eb 05 b9 01 00 00 00 89 c8 5b 5b 5e 5f 5d c3 55 89
[ 1103.515937] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.515937] Pid: 134, comm: chroot Not tainted 3.6.0-rc1-bisect2 #80
[ 1103.515937] Call Trace:
[ 1103.515937]  [<c1673658>] panic+0x97/0x1aa
[ 1103.515937]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1103.515937]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.515937]  [<c106716a>] ? __lock_acquire+0x1444/0x1455
[ 1103.515937]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1103.515937]  [<c103205a>] run_local_timers+0x8/0x14
[ 1103.515937]  [<c103208a>] update_process_times+0x24/0x58
[ 1103.515937]  [<c106276e>] tick_periodic+0x66/0x72
[ 1103.515937]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1103.515937]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.515937]  [<c12b36a0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.515937]  [<c167f376>] apic_timer_interrupt+0x36/0x40
[ 1103.515937]  [<c1065d13>] ? lock_is_held+0x9f/0xb2
[ 1103.515937]  [<c104bb01>] __might_sleep+0xa1/0x1cd
[ 1103.515937]  [<c10af78b>] ? might_fault+0x90/0x99
[ 1103.515937]  [<c10cc958>] count.constprop.35+0x78/0x82
[ 1103.515937]  [<c10cde5f>] do_execve+0x310/0x456
[ 1103.515937]  [<c10cdc3a>] ? do_execve+0xeb/0x456
[ 1103.515937]  [<c1009165>] sys_execve+0x31/0x53
[ 1103.515937]  [<c167f942>] ptregs_execve+0x12/0x20
[ 1103.515937]  [<c167f843>] ? sysenter_do_call+0x12/0x38
[ 1103.515937] Rebooting in 10 seconds..
--
[    5.214556] Freeing unused kernel memory: 464k freed
[ 1104.514802] BUG: soft lockup - CPU#1 stuck for 1025s! [init:1]
[ 1104.514802] irq event stamp: 756026
[ 1104.514802] hardirqs last  enabled at (756025): [<c166e171>] _raw_spin_unlock_irqrestore+0x42/0x63
[ 1104.514802] hardirqs last disabled at (756026): [<c166f4bf>] apic_timer_interrupt+0x2f/0x40
[ 1104.514802] softirqs last  enabled at (751184): [<c15f0ef2>] unix_sock_destructor+0xae/0xb3
[ 1104.514802] softirqs last disabled at (751182): [<c15f0edf>] unix_sock_destructor+0x9b/0xb3
[ 1104.514802] Pid: 1, comm: init Not tainted 3.5.0-bisect2-00313-g7cd58b0 #86  
[ 1104.514802] EIP: 0060:[<c166e180>] EFLAGS: 00000286 CPU: 1
[ 1104.514802] EIP is at _raw_spin_unlock_irqrestore+0x51/0x63
[ 1104.514802] EAX: 00000286 EBX: 00000286 ECX: 00000000 EDX: 00000007
[ 1104.514802] ESI: ce861504 EDI: 00000001 EBP: ce03ddfc ESP: ce03ddf4
[ 1104.514802]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.524377] [sched_delayed] sched: RT throttling activated
[ 1104.514802] CR0: 80050033 CR2: b760b600 CR3: 0ee6a000 CR4: 00000690
[ 1104.514802] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.514802] DR6: ffff0ff0 DR7: 00000400
[ 1104.514802] Process init (pid: 1, ti=ce03c000 task=ce040000 task.ti=ce03c000)
[ 1104.514802] Stack:
[ 1104.514802]  00000000 ce861120 ce03de20 c104840a 00000286 ce861504 0cb62000 00000000
[ 1104.514802]  ce861120 00000000 00000007 ce03de28 c1048450 ce03de38 c102ec22 ce861120
[ 1104.514802]  00000000 ce03def4 c102fb02 003f0011 00000000 ce040470 00000000 ce040000
[ 1104.514802] Call Trace:
[ 1104.514802]  [<c104840a>] try_to_wake_up+0x248/0x266
[ 1104.514802]  [<c1048450>] wake_up_state+0xa/0xc
[ 1104.514802]  [<c102ec22>] ptrace_resume+0x76/0x7c
[ 1104.514802]  [<c102fb02>] ptrace_request+0x368/0x4f8
[ 1104.514802]  [<c106572a>] ? __lock_acquire+0x1444/0x1455
[ 1104.514802]  [<c106572a>] ? __lock_acquire+0x1444/0x1455
[ 1104.514802]  [<c10657b9>] ? lock_release_non_nested+0x7e/0x21b
[ 1104.514802]  [<c106572a>] ? __lock_acquire+0x1444/0x1455
[ 1104.514802]  [<c100ac8a>] arch_ptrace+0x474/0x48a
[ 1104.514802]  [<c166e171>] ? _raw_spin_unlock_irqrestore+0x42/0x63
[ 1104.514802]  [<c10638b8>] ? trace_hardirqs_on+0xb/0xd
[ 1104.514802]  [<c1045ff5>] ? wait_task_inactive+0x6f/0xc1
[ 1104.514802]  [<c102eede>] ? ptrace_check_attach+0xb4/0xc4
[ 1104.514802]  [<c102f6fe>] sys_ptrace+0x34f/0x379
[ 1104.514802]  [<c102f41d>] ? sys_ptrace+0x6e/0x379
[ 1104.514802]  [<c166f9a3>] sysenter_do_call+0x12/0x38
[ 1104.514802] Code: 3d 6c d4 8f c1 00 74 1d 89 d8 50 9d 8d 74 26 00 e8 6d 58 9f ff eb 18 e8 3c 57 9f ff 83 3d 6c d4 8f c1 00 75 02 0f 0b 89 d8 50 9d <8d> 74 26 00 89 e0 25 00 e0 ff ff ff 48 14 5b 5e 5d c3 55 89 e5
[ 1104.514802] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.514802] Pid: 1, comm: init Not tainted 3.5.0-bisect2-00313-g7cd58b0 #86
[ 1104.514802] Call Trace:
[ 1104.514802]  [<c1662a5f>] panic+0x81/0x194
[ 1104.514802]  [<c107e1de>] watchdog_timer_fn+0x115/0x134
[ 1104.514802]  [<c10413be>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.514802]  [<c107e0c9>] ? __touch_watchdog+0x1d/0x1d
[ 1104.514802]  [<c10305c6>] run_local_timers+0x8/0x14
[ 1104.514802]  [<c10305f6>] update_process_times+0x24/0x58
[ 1104.514802]  [<c1060d2e>] tick_periodic+0x66/0x72
[ 1104.514802]  [<c1060d88>] tick_handle_periodic+0x19/0x61
[ 1104.514802]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.514802]  [<c12aa260>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.514802]  [<c166f4c6>] apic_timer_interrupt+0x36/0x40
[ 1104.514802]  [<c166e180>] ? _raw_spin_unlock_irqrestore+0x51/0x63
[ 1104.514802]  [<c104840a>] try_to_wake_up+0x248/0x266
[ 1104.514802]  [<c1048450>] wake_up_state+0xa/0xc
[ 1104.514802]  [<c102ec22>] ptrace_resume+0x76/0x7c
[ 1104.514802]  [<c102fb02>] ptrace_request+0x368/0x4f8
[ 1104.514802]  [<c106572a>] ? __lock_acquire+0x1444/0x1455
[ 1104.514802]  [<c106572a>] ? __lock_acquire+0x1444/0x1455
[ 1104.514802]  [<c10657b9>] ? lock_release_non_nested+0x7e/0x21b
[ 1104.514802]  [<c106572a>] ? __lock_acquire+0x1444/0x1455
[ 1104.514802]  [<c100ac8a>] arch_ptrace+0x474/0x48a
[ 1104.514802]  [<c166e171>] ? _raw_spin_unlock_irqrestore+0x42/0x63
[ 1104.514802]  [<c10638b8>] ? trace_hardirqs_on+0xb/0xd
[ 1104.514802]  [<c1045ff5>] ? wait_task_inactive+0x6f/0xc1
[ 1104.514802]  [<c102eede>] ? ptrace_check_attach+0xb4/0xc4
[ 1104.514802]  [<c102f6fe>] sys_ptrace+0x34f/0x379
[ 1104.514802]  [<c102f41d>] ? sys_ptrace+0x6e/0x379
[ 1104.514802]  [<c166f9a3>] sysenter_do_call+0x12/0x38
[ 1104.514802] Rebooting in 10 seconds..
--
[    5.704992] Freeing unused kernel memory: 464k freed
[ 1104.527176] BUG: soft lockup - CPU#1 stuck for 1023s! [swapper/1:0]
[ 1104.527176] irq event stamp: 14280
[ 1104.527176] hardirqs last  enabled at (14277): [<c166e1f5>] restore_all_notrace+0x0/0x18
[ 1104.527176] hardirqs last disabled at (14278): [<c166ecff>] apic_timer_interrupt+0x2f/0x40
[ 1104.527176] softirqs last  enabled at (14280): [<c102cbbf>] _local_bh_enable+0xd/0xf
[ 1104.527176] softirqs last disabled at (14279): [<c102d02c>] irq_enter+0x2c/0x4d
[ 1104.527176] Pid: 0, comm: swapper/1 Not tainted 3.5.0-bisect2-01297-ga17f29a #84  
[ 1104.527176] EIP: 0060:[<c101c42e>] EFLAGS: 00000282 CPU: 1
[ 1104.527176] EIP is at native_safe_halt+0x5/0x7
[ 1104.527176] EAX: ce060000 EBX: ffffffff ECX: ce060000 EDX: 00000007
[ 1104.527176] ESI: c195a818 EDI: ffff8b9e EBP: ce05ff68 ESP: ce05ff68
[ 1104.527176]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.527176] CR0: 8005003b CR2: b76c8000 CR3: 0ee58000 CR4: 00000690
[ 1104.527176] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.527176] DR6: ffff0ff0 DR7: 00000400
[ 1104.527176] Process swapper/1 (pid: 0, ti=ce05e000 task=ce060000 task.ti=ce05e000)
[ 1104.527176] Stack:
[ 1104.527176]  ce05ff70 c10089d9 ce05ff88 c1009220 a6cdc943 b5314e64 ef7c5be0 cfff9fa0
[ 1104.527176]  ce05ffb4 c165d2ba 00000000 00000000 00000000 00000000 5fab647c ef7c5be0
[ 1104.527176]  01000800 0000d248 00000000 00000000 00000000 00000000 00000000 00000000
[ 1104.527176] Call Trace:
[ 1104.527176]  [<c10089d9>] default_idle+0x2f/0x55
[ 1104.527176]  [<c1009220>] cpu_idle+0xc9/0xfb
[ 1104.527176]  [<c165d2ba>] start_secondary+0x292/0x29a
[ 1104.527176] Code: 0f 22 e0 5d c3 55 89 e5 0f 09 5d c3 55 89 e5 9c 58 5d c3 55 89 e5 50 9d 5d c3 55 89 e5 fa 5d c3 55 89 e5 fb 5d c3 55 89 e5 fb f4 <5d> c3 55 89 e5 f4 5d c3 55 89 e5 57 56 53 53 89 c6 89 55 f0 89
[ 1104.527176] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.527176] Pid: 0, comm: swapper/1 Not tainted 3.5.0-bisect2-01297-ga17f29a #84
[ 1104.527176] Call Trace:
[ 1104.527176]  [<c16622f2>] panic+0x81/0x194
[ 1104.527176]  [<c107e1d2>] watchdog_timer_fn+0x115/0x134
[ 1104.527176]  [<c10413fe>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.527176]  [<c107e0bd>] ? __touch_watchdog+0x1d/0x1d
[ 1104.527176]  [<c10305c6>] run_local_timers+0x8/0x14
[ 1104.527176]  [<c10305f6>] update_process_times+0x24/0x58
[ 1104.527176]  [<c1060d92>] tick_periodic+0x66/0x72
[ 1104.527176]  [<c1060dec>] tick_handle_periodic+0x19/0x61
[ 1104.527176]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.527176]  [<c12aa840>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.527176]  [<c166ed06>] apic_timer_interrupt+0x36/0x40
[ 1104.527176]  [<c101c42e>] ? native_safe_halt+0x5/0x7
[ 1104.527176]  [<c10089d9>] default_idle+0x2f/0x55
[ 1104.527176]  [<c1009220>] cpu_idle+0xc9/0xfb
[ 1104.527176]  [<c165d2ba>] start_secondary+0x292/0x29a
[ 1104.527176] Rebooting in 10 seconds..
--
[    6.525425] Freeing unused kernel memory: 464k freed
[ 1105.523006] BUG: soft lockup - CPU#1 stuck for 1023s! [99-trinity:133]
[ 1105.523006] irq event stamp: 632
[ 1105.523006] hardirqs last  enabled at (631): [<c166e268>] __mutex_unlock_slowpath+0x114/0x12c
[ 1105.523006] hardirqs last disabled at (632): [<c16712ff>] apic_timer_interrupt+0x2f/0x40
[ 1105.523006] softirqs last  enabled at (0): [<c10252e4>] copy_process+0x471/0xf82
[ 1105.523006] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1105.523006] Pid: 133, comm: 99-trinity Not tainted 3.5.0-bisect2-05018-g614a6d43 #82  
[ 1105.523006] EIP: 0060:[<c1064141>] EFLAGS: 00000202 CPU: 1
[ 1105.523006] EIP is at lockdep_trace_alloc+0x3/0xe0
[ 1105.523006] EAX: 000280da EBX: 000280da ECX: c1958810 EDX: 00000000
[ 1105.523006] ESI: 0000000a EDI: ce807680 EBP: ce8a3d5c ESP: ce8a3d5c
[ 1105.523006]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1105.523006] CR0: 80050033 CR2: b75e5baf CR3: 0e868000 CR4: 00000690
[ 1105.523006] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1105.523006] DR6: ffff0ff0 DR7: 00000400
[ 1105.523006] Process 99-trinity (pid: 133, ti=ce8a2000 task=ce8a8000 task.ti=ce8a2000)
[ 1105.523006] Stack:
[ 1105.523006]  ce8a3dfc c109a6d2 ce8a8000 00000040 00000006 c1bc8eb4 ce8a8458 ce8a8000
[ 1105.523006]  ce8a3da0 c1063a52 00000378 00000007 00000000 00000000 ce8a8000 c1900ad8
[ 1105.523006]  ce8a3db4 00000286 00000286 000000aa c181b224 00000000 ce8a3ddc 00000286
[ 1105.523006] Call Trace:
[ 1105.523006]  [<c109a6d2>] __alloc_pages_nodemask+0x78/0x5db
[ 1105.523006]  [<c1063a52>] ? mark_held_locks+0xa7/0xce
[ 1105.523006]  [<c10b19e1>] ? anon_vma_prepare+0x24/0x131
[ 1105.523006]  [<c10abbdd>] handle_pte_fault+0x16b/0x74c
[ 1105.523006]  [<c1020637>] ? kmap_atomic_prot+0x11f/0x133
[ 1105.523006]  [<c10ac253>] handle_mm_fault+0x95/0xa4
[ 1105.523006]  [<c10ac545>] __get_user_pages+0x264/0x39e
[ 1105.523006]  [<c10ac71d>] get_user_pages+0x4b/0x53
[ 1105.523006]  [<c10c3f2f>] get_arg_page+0x4a/0xa9
[ 1105.523006]  [<c10c408c>] copy_strings+0xfe/0x1c6
[ 1105.523006]  [<c10c44e9>] copy_strings_kernel+0x1c/0x2b
[ 1105.523006]  [<c10c56ff>] do_execve+0x343/0x456
[ 1105.523006]  [<c10c54a7>] ? do_execve+0xeb/0x456
[ 1105.523006]  [<c1009135>] sys_execve+0x31/0x53
[ 1105.523006]  [<c16718e2>] ptregs_execve+0x12/0x20
[ 1105.523006]  [<c16717e3>] ? sysenter_do_call+0x12/0x38
[ 1105.523006] Code: 3d 1c b7 95 c1 01 74 16 ba 9a 0a 00 00 b8 8b 35 81 c1 e8 7e 26 fc ff c6 05 1c b7 95 c1 01 64 ff 05 bc dc 9c c1 5b 5d c3 55 89 e5 <57> 56 53 89 c6 64 8b 1d 28 c6 9c c1 83 bb 54 04 00 00 00 0f 85
[ 1105.523006] Kernel panic - not syncing: softlockup: hung tasks
[ 1105.523006] Pid: 133, comm: 99-trinity Not tainted 3.5.0-bisect2-05018-g614a6d43 #82
[ 1105.523006] Call Trace:
[ 1105.523006]  [<c1664e92>] panic+0x81/0x194
[ 1105.523006]  [<c107e316>] watchdog_timer_fn+0x115/0x134
[ 1105.523006]  [<c10416fe>] hrtimer_run_queues+0x14c/0x1c6
[ 1105.523006]  [<c107e201>] ? __touch_watchdog+0x1d/0x1d
[ 1105.523006]  [<c10306a2>] run_local_timers+0x8/0x14
[ 1105.523006]  [<c10306d2>] update_process_times+0x24/0x58
[ 1105.523006]  [<c1061092>] tick_periodic+0x66/0x72
[ 1105.523006]  [<c10610ec>] tick_handle_periodic+0x19/0x61
[ 1105.523006]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1105.523006]  [<c12aa9a0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1105.523006]  [<c1671306>] apic_timer_interrupt+0x36/0x40
[ 1105.523006]  [<c1064141>] ? lockdep_trace_alloc+0x3/0xe0
[ 1105.523006]  [<c109a6d2>] __alloc_pages_nodemask+0x78/0x5db
[ 1105.523006]  [<c1063a52>] ? mark_held_locks+0xa7/0xce
[ 1105.523006]  [<c10b19e1>] ? anon_vma_prepare+0x24/0x131
[ 1105.581916]  [<c10abbdd>] handle_pte_fault+0x16b/0x74c
[ 1105.581916]  [<c1020637>] ? kmap_atomic_prot+0x11f/0x133
[ 1105.581916]  [<c10ac253>] handle_mm_fault+0x95/0xa4
[ 1105.581916]  [<c10ac545>] __get_user_pages+0x264/0x39e
[ 1105.581916]  [<c10ac71d>] get_user_pages+0x4b/0x53
[ 1105.581916]  [<c10c3f2f>] get_arg_page+0x4a/0xa9
[ 1105.581916]  [<c10c408c>] copy_strings+0xfe/0x1c6
[ 1105.581916]  [<c10c44e9>] copy_strings_kernel+0x1c/0x2b
[ 1105.581916]  [<c10c56ff>] do_execve+0x343/0x456
[ 1105.581916]  [<c10c54a7>] ? do_execve+0xeb/0x456
[ 1105.581916]  [<c1009135>] sys_execve+0x31/0x53
[ 1105.581916]  [<c16718e2>] ptregs_execve+0x12/0x20
[ 1105.581916]  [<c16717e3>] ? sysenter_do_call+0x12/0x38
[ 1105.581916] Rebooting in 10 seconds..
--
[    4.563976] Freeing unused kernel memory: 464k freed
[ 1103.520656] BUG: soft lockup - CPU#1 stuck for 1023s! [shutdown:135]
[ 1103.520656] irq event stamp: 2580
[ 1103.520656] hardirqs last  enabled at (2579): [<c166b883>] mutex_lock_nested+0x2da/0x308
[ 1103.520656] hardirqs last disabled at (2580): [<c166f4bf>] apic_timer_interrupt+0x2f/0x40
[ 1103.520656] softirqs last  enabled at (0): [<c1025204>] copy_process+0x471/0xf82
mountall: ply-ev[ 1103.520656] softirqs last disabled at (0): ent-loop.c:497: [<  (null)>]   (null)
ply_event_loop_n[ 1103.520656] Pid: 135, comm: shutdown Not tainted 3.5.0-bisect2-00313-g7cd58b0 #86  
ew: Assertion `l[ 1103.520656] EIP: 0060:[<c166b893>] EFLAGS: 00000286 CPU: 1
[ 1103.520656] EIP is at mutex_lock_nested+0x2ea/0x308
[ 1103.520656] EAX: 00000286 EBX: cb4e699c ECX: 00000006 EDX: cf8826c8
[ 1103.520656] ESI: ce82dbb0 EDI: ce873d7c EBP: ce873d98 ESP: ce873d5c
[ 1103.520656]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.520656] CR0: 8005003b CR2: b77911b8 CR3: 0e88b000 CR4: 00000690
[ 1103.520656] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
oop->epoll_fd >=[ 1103.520656] DR6: ffff0ff0 DR7: 00000400
[ 1103.520656] Process shutdown (pid: 135, ti=ce872000 task=cf882240 task.ti=ce872000)
 0' failed.
[ 1103.520656] Stack:
[ 1103.520656]  00000000 00000002 00000000 c10addf7 cb4e69c0 cf882240 cb4e69a0 00000286
[ 1103.520656]  ce873d7c ce873d7c 00000000 ce873d7c ce82d790 ce82dbb0 00000000 ce873dec
[ 1103.520656]  c10addf7 00000000 00000000 00000000 00000000 b775e000 ce0f6400 00000000
[ 1103.520656] Call Trace:
[ 1103.520656]  [<c10addf7>] ? vma_adjust+0x134/0x30e
[ 1103.520656]  [<c10addf7>] vma_adjust+0x134/0x30e
[ 1103.520656]  [<c10ae0bb>] __split_vma+0xea/0x13e
[ 1103.520656]  [<c10ae8fd>] do_munmap+0xab/0x24b
[ 1103.520656]  [<c10aeacd>] vm_munmap+0x30/0x41
[ 1103.520656]  [<c166505c>] elf_map+0x80/0xa6
[ 1103.520656]  [<c10f0e65>] load_elf_binary+0x7b7/0xfbd
[ 1103.520656]  [<c10c51f6>] search_binary_handler+0x14d/0x22b
[ 1103.520656]  [<c10c50e3>] ? search_binary_handler+0x3a/0x22b
[ 1103.520656]  [<c10f06ae>] ? load_elf_library+0x1bb/0x1bb
[ 1103.520656]  [<c10c565f>] do_execve+0x38b/0x456
[ 1103.520656]  [<c10c53bf>] ? do_execve+0xeb/0x456
[ 1103.520656]  [<c1009135>] sys_execve+0x31/0x53
[ 1103.520656]  [<c166faa2>] ptregs_execve+0x12/0x20
[ 1103.520656]  [<c166f9a3>] ? sysenter_do_call+0x12/0x38
[ 1103.520656] Code: d4 8f c1 00 74 1e 8b 45 e0 50 9d 8d 74 26 00 e8 5b 81 9f ff eb 19 e8 2a 80 9f ff 83 3d 6c d4 8f c1 00 75 02 0f 0b 8b 45 e0 50 9d <8d> 74 26 00 8d 45 e4 e8 c0 5f 9f ff 89 e0 25 00 e0 ff ff ff 48
[ 1103.520656] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.520656] Pid: 135, comm: shutdown Not tainted 3.5.0-bisect2-00313-g7cd58b0 #86
[ 1103.520656] Call Trace:
[ 1103.520656]  [<c1662a5f>] panic+0x81/0x194
[ 1103.520656]  [<c107e1de>] watchdog_timer_fn+0x115/0x134
[ 1103.520656]  [<c10413be>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.520656]  [<c107e0c9>] ? __touch_watchdog+0x1d/0x1d
[ 1103.520656]  [<c10305c6>] run_local_timers+0x8/0x14
[ 1103.520656]  [<c10305f6>] update_process_times+0x24/0x58
[ 1103.520656]  [<c1060d2e>] tick_periodic+0x66/0x72
[ 1103.520656]  [<c1060d88>] tick_handle_periodic+0x19/0x61
[ 1103.520656]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.520656]  [<c12aa260>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.520656]  [<c166f4c6>] apic_timer_interrupt+0x36/0x40
[ 1103.520656]  [<c166b893>] ? mutex_lock_nested+0x2ea/0x308
[ 1103.520656]  [<c10addf7>] ? vma_adjust+0x134/0x30e
[ 1103.520656]  [<c10addf7>] vma_adjust+0x134/0x30e
[ 1103.520656]  [<c10ae0bb>] __split_vma+0xea/0x13e
[ 1103.520656]  [<c10ae8fd>] do_munmap+0xab/0x24b
[ 1103.520656]  [<c10aeacd>] vm_munmap+0x30/0x41
[ 1103.520656]  [<c166505c>] elf_map+0x80/0xa6
[ 1103.520656]  [<c10f0e65>] load_elf_binary+0x7b7/0xfbd
[ 1103.520656]  [<c10c51f6>] search_binary_handler+0x14d/0x22b
[ 1103.520656]  [<c10c50e3>] ? search_binary_handler+0x3a/0x22b
[ 1103.520656]  [<c10f06ae>] ? load_elf_library+0x1bb/0x1bb
[ 1103.520656]  [<c10c565f>] do_execve+0x38b/0x456
[ 1103.520656]  [<c10c53bf>] ? do_execve+0xeb/0x456
[ 1103.520656]  [<c1009135>] sys_execve+0x31/0x53
[ 1103.520656]  [<c166faa2>] ptregs_execve+0x12/0x20
[ 1103.520656]  [<c166f9a3>] ? sysenter_do_call+0x12/0x38
[ 1103.520656] Rebooting in 10 seconds..
--
[    4.801716] Freeing unused kernel memory: 460k freed
[ 1103.512941] BUG: soft lockup - CPU#1 stuck for 1023s! [sleep:134]
[ 1103.512941] irq event stamp: 780
[ 1103.512941] hardirqs last  enabled at (779): [<c101e5d2>] do_page_fault+0x98/0x36e
[ 1103.512941] hardirqs last disabled at (780): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1103.517007] softirqs last  enabled at (22): [<c102e8ff>] __do_softirq+0x131/0x153
--
[    5.230581] Freeing unused kernel memory: 464k freed
[ 1104.515302] BUG: soft lockup - CPU#1 stuck for 1024s! [run-parts:122]
[ 1104.515302] irq event stamp: 1238
[ 1104.515302] hardirqs last  enabled at (1237): [<c101cfce>] do_page_fault+0x98/0x36e
[ 1104.515302] hardirqs last disabled at (1238): [<c166e77f>] apic_timer_interrupt+0x2f/0x40
[ 1104.515302] softirqs last  enabled at (0): [<c1025204>] copy_process+0x471/0xf82
[ 1104.515302] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1104.515302] Pid: 122, comm: run-parts Not tainted 3.5.0-bisect2-00646-g5b160bd #85  
[ 1104.515302] EIP: 0060:[<c1086e16>] EFLAGS: 00000246 CPU: 1
[ 1104.515302] EIP is at rcu_lockdep_current_cpu_online+0x0/0x51
[ 1104.515302] EAX: 00000000 EBX: cb4e695c ECX: 00000001 EDX: ce83c000
[ 1104.522494] [sched_delayed] sched: RT throttling activated
[ 1104.515302] ESI: 0000000c EDI: cb4bf851 EBP: ce83ddfc ESP: ce83dde0
[ 1104.515302]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.515302] CR0: 8005003b CR2: b774b696 CR3: 0e863000 CR4: 00000690
[ 1104.515302] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.515302] DR6: ffff0ff0 DR7: 00000400
[ 1104.515302] Process run-parts (pid: 122, ti=ce83c000 task=cf8a0000 task.ti=ce83c000)
[ 1104.515302] Stack:
[ 1104.515302]  c12a4768 00000000 00000001 00000000 cb4e695c 00000021 0000000c ce83de04
[ 1104.515302]  c12a4a88 ce83de2c c1094dcd 00000002 00000001 00000000 c1094d5a cb4e695c
[ 1104.515302]  ce861200 00000021 00000000 ce83de6c c1095082 00000000 00000001 0000036f
[ 1104.515302] Call Trace:
[ 1104.515302]  [<c12a4768>] ? radix_tree_lookup_element+0x3c/0x107
[ 1104.515302]  [<c12a4a88>] radix_tree_lookup_slot+0xd/0xf
[ 1104.515302]  [<c1094dcd>] find_get_page+0x8a/0x1a6
[ 1104.515302]  [<c1094d5a>] ? find_get_page+0x17/0x1a6
[ 1104.515302]  [<c1095082>] filemap_fault+0x77/0x35b
[ 1104.515302]  [<c10a9392>] __do_fault+0x88/0x38a
[ 1104.515302]  [<c10abc19>] handle_pte_fault+0x2c3/0x74c
[ 1104.515302]  [<c1020553>] ? kmap_atomic_prot+0x11f/0x133
[ 1104.515302]  [<c10ac137>] handle_mm_fault+0x95/0xa4
[ 1104.515302]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.515302]  [<c101d26c>] do_page_fault+0x336/0x36e
[ 1104.515302]  [<c10acb14>] ? might_fault+0x7b/0x97
[ 1104.515302]  [<c100b09a>] ? set_tls_desc+0x14a/0x15d
[ 1104.515302]  [<c166dc75>] ? restore_all+0xf/0xf
[ 1104.515302]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.515302]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.515302]  [<c101c2fd>] do_async_page_fault+0x23/0x5f
[ 1104.515302]  [<c166e9bc>] error_code+0x6c/0x80
[ 1104.515302] Code: 51 c7 04 24 00 00 00 00 b9 00 f4 8f c1 e8 b9 fd ff ff c9 c3 55 89 e5 51 c7 04 24 00 00 00 00 b9 90 f6 8f c1 e8 a2 fd ff ff c9 c3 <55> 89 e5 89 e2 81 e2 00 e0 ff ff 8b 4a 14 b0 01 f7 c1 00 00 00
[ 1104.515302] Kernel panic - not syncing: softlockup: hung tasks
[ 1104.515302] Pid: 122, comm: run-parts Not tainted 3.5.0-bisect2-00646-g5b160bd #85
[ 1104.515302] Call Trace:
[ 1104.515302]  [<c1661ced>] panic+0x81/0x194
[ 1104.515302]  [<c107e202>] watchdog_timer_fn+0x115/0x134
[ 1104.515302]  [<c10413be>] hrtimer_run_queues+0x14c/0x1c6
[ 1104.515302]  [<c107e0ed>] ? __touch_watchdog+0x1d/0x1d
[ 1104.515302]  [<c10305c6>] run_local_timers+0x8/0x14
[ 1104.515302]  [<c10305f6>] update_process_times+0x24/0x58
[ 1104.515302]  [<c1060d52>] tick_periodic+0x66/0x72
[ 1104.515302]  [<c1060dac>] tick_handle_periodic+0x19/0x61
[ 1104.515302]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1104.515302]  [<c12aa280>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1104.515302]  [<c166e786>] apic_timer_interrupt+0x36/0x40
[ 1104.515302]  [<c1086e16>] ? call_rcu_sched+0x17/0x17
[ 1104.515302]  [<c12a4768>] ? radix_tree_lookup_element+0x3c/0x107
[ 1104.515302]  [<c12a4a88>] radix_tree_lookup_slot+0xd/0xf
[ 1104.515302]  [<c1094dcd>] find_get_page+0x8a/0x1a6
[ 1104.515302]  [<c1094d5a>] ? find_get_page+0x17/0x1a6
[ 1104.515302]  [<c1095082>] filemap_fault+0x77/0x35b
[ 1104.515302]  [<c10a9392>] __do_fault+0x88/0x38a
[ 1104.515302]  [<c10abc19>] handle_pte_fault+0x2c3/0x74c
[ 1104.515302]  [<c1020553>] ? kmap_atomic_prot+0x11f/0x133
[ 1104.515302]  [<c10ac137>] handle_mm_fault+0x95/0xa4
[ 1104.515302]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.515302]  [<c101d26c>] do_page_fault+0x336/0x36e
[ 1104.515302]  [<c10acb14>] ? might_fault+0x7b/0x97
[ 1104.515302]  [<c100b09a>] ? set_tls_desc+0x14a/0x15d
[ 1104.515302]  [<c166dc75>] ? restore_all+0xf/0xf
[ 1104.515302]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.515302]  [<c101c2da>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1104.515302]  [<c101c2fd>] do_async_page_fault+0x23/0x5f
[ 1104.515302]  [<c166e9bc>] error_code+0x6c/0x80
[ 1104.515302] Rebooting in 10 seconds..
--
[ 1105.513232] [sched_delayed] sched: RT throttling activated
[ 1105.513206] BUG: soft lockup - CPU#1 stuck for 1023s! [rc.local:123]
[ 1105.513206] irq event stamp: 326
[ 1105.513206] hardirqs last  enabled at (325): [<c106831d>] debug_check_no_locks_freed+0x14b/0x167
[ 1105.513206] hardirqs last disabled at (326): [<c167f38f>] apic_timer_interrupt+0x2f/0x40
[ 1105.513206] softirqs last  enabled at (0): [<c1026857>] copy_process+0x465/0xf8a
[ 1105.513206] softirqs last disabled at (0): [<  (null)>]   (null)
[ 1105.513206] Pid: 123, comm: rc.local Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79  
[ 1105.513206] EIP: 0060:[<c10df7c4>] EFLAGS: 00000246 CPU: 1
[ 1105.513206] EIP is at __mnt_drop_write+0x1/0x1a
[ 1105.513206] EAX: ce014110 EBX: cb438000 ECX: 00000000 EDX: 00000001
[ 1105.513206] ESI: ce014110 EDI: 00000000 EBP: ce879dbc ESP: ce879da4
[ 1105.513206]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1105.513206] CR0: 80050033 CR2: b7553baf CR3: 0e85a000 CR4: 00000690
[ 1105.513206] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1105.513206] DR6: ffff0ff0 DR7: 00000400
[ 1105.513206] Process rc.local (pid: 123, ti=ce878000 task=cf8ad5f0 task.ti=ce878000)
[ 1105.513206] Stack:
[ 1105.513206]  ce879dbc c10dd7bf 504bf223 ffffff88 ce889800 ce6a2f80 ce879e4c c1097f77
[ 1105.513206]  00001000 006f41e4 00000000 00000080 00000000 00000000 00000001 ce889884
[ 1105.513206]  ce879e90 00000000 00001000 00000080 00000000 cb438000 ce879ed4 00000000
[ 1105.513206] Call Trace:
[ 1105.513206]  [<c10dd7bf>] ? touch_atime+0xdc/0x104
[ 1105.513206]  [<c1097f77>] generic_file_aio_read+0x5b5/0x5f1
[ 1105.513206]  [<c10c78d1>] do_sync_read+0x75/0xb0
[ 1105.513206]  [<c10c7faa>] vfs_read+0x8b/0x10d
[ 1105.513206]  [<c10af901>] ? might_fault+0x26/0x99
[ 1105.513206]  [<c10c785c>] ? wait_on_retry_sync_kiocb+0x3f/0x3f
[ 1105.513206]  [<c10cd050>] kernel_read+0x30/0x41
[ 1105.513206]  [<c10cd8d6>] prepare_binprm+0xfb/0x103
[ 1105.513206]  [<c10cdeaa>] do_execve+0x327/0x456
[ 1105.513206]  [<c10cdc6e>] ? do_execve+0xeb/0x456
[ 1105.546433]  [<c1009165>] sys_execve+0x31/0x53
[ 1105.546433]  [<c167f962>] ptregs_execve+0x12/0x20
[ 1105.546433]  [<c167f863>] ? sysenter_do_call+0x12/0x38
[ 1105.546433] Code: 00 00 e8 6c ba fe ff 89 d8 e8 91 ff ff ff 89 c6 85 c0 74 10 8b 43 08 8b 40 04 ba 01 00 00 00 e8 dd b9 fe ff 89 f0 5b 5e 5d c3 55 <89> e5 89 e2 81 e2 00 e0 ff ff ff 42 14 8b 40 0c 64 ff 48 04 ff
[ 1105.546433] Kernel panic - not syncing: softlockup: hung tasks
[ 1105.546433] Pid: 123, comm: rc.local Not tainted 3.6.0-rc1-bisect2-00023-g8a13a4c #79
[ 1105.546433] Call Trace:
[ 1105.546433]  [<c1673678>] panic+0x97/0x1aa
[ 1105.546433]  [<c107fada>] watchdog_timer_fn+0x115/0x134
[ 1105.546433]  [<c10430d2>] hrtimer_run_queues+0x14c/0x1c6
[ 1105.546433]  [<c107f9c5>] ? __touch_watchdog+0x1d/0x1d
[ 1105.546433]  [<c103205a>] run_local_timers+0x8/0x14
[ 1105.546433]  [<c103208a>] update_process_times+0x24/0x58
[ 1105.546433]  [<c106276e>] tick_periodic+0x66/0x72
[ 1105.546433]  [<c10627c8>] tick_handle_periodic+0x19/0x61
[ 1105.546433]  [<c1018bcd>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1105.546433]  [<c12b36c0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1105.546433]  [<c167f396>] apic_timer_interrupt+0x36/0x40
[ 1105.546433]  [<c10d00d8>] ? pipe_fcntl+0xbc/0x140
[ 1105.546433]  [<c10df7c4>] ? __mnt_drop_write+0x1/0x1a
[ 1105.546433]  [<c10dd7bf>] ? touch_atime+0xdc/0x104
[ 1105.546433]  [<c1097f77>] generic_file_aio_read+0x5b5/0x5f1
[ 1105.546433]  [<c10c78d1>] do_sync_read+0x75/0xb0
[ 1105.546433]  [<c10c7faa>] vfs_read+0x8b/0x10d
[ 1105.546433]  [<c10af901>] ? might_fault+0x26/0x99
[ 1105.546433]  [<c10c785c>] ? wait_on_retry_sync_kiocb+0x3f/0x3f
[ 1105.546433]  [<c10cd050>] kernel_read+0x30/0x41
[ 1105.546433]  [<c10cd8d6>] prepare_binprm+0xfb/0x103
[ 1105.546433]  [<c10cdeaa>] do_execve+0x327/0x456
[ 1105.546433]  [<c10cdc6e>] ? do_execve+0xeb/0x456
[ 1105.546433]  [<c1009165>] sys_execve+0x31/0x53
[ 1105.546433]  [<c167f962>] ptregs_execve+0x12/0x20
[ 1105.546433]  [<c167f863>] ? sysenter_do_call+0x12/0x38
[ 1105.546433] Rebooting in 10 seconds..
--
[    6.292044] Freeing unused kernel memory: 464k freed
[ 1105.523875] BUG: soft lockup - CPU#1 stuck for 1024s! [swapper/1:0]
[ 1105.523875] irq event stamp: 13590
[ 1105.523875] hardirqs last  enabled at (13587): [<c16707f5>] restore_all_notrace+0x0/0x18
[ 1105.523875] hardirqs last disabled at (13588): [<c16712ff>] apic_timer_interrupt+0x2f/0x40
[ 1105.523875] softirqs last  enabled at (13590): [<c102cc9f>] _local_bh_enable+0xd/0xf
[ 1105.523875] softirqs last disabled at (13589): [<c102d10c>] irq_enter+0x2c/0x4d
[ 1105.523875] Pid: 0, comm: swapper/1 Not tainted 3.5.0-bisect2-05018-g614a6d43 #82  
[ 1105.523875] EIP: 0060:[<c101c512>] EFLAGS: 00000282 CPU: 1
[ 1105.523875] EIP is at native_safe_halt+0x5/0x7
[ 1105.523875] EAX: ce062240 EBX: ffffffff ECX: ce062240 EDX: 00000007
[ 1105.523875] ESI: c195cbf0 EDI: ffff8b9f EBP: ce06bf68 ESP: ce06bf68
[ 1105.523875]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1105.523875] CR0: 8005003b CR2: b76e4fb8 CR3: 0e888000 CR4: 00000690
[ 1105.523875] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1105.523875] DR6: ffff0ff0 DR7: 00000400
[ 1105.523875] Process swapper/1 (pid: 0, ti=ce06a000 task=ce062240 task.ti=ce06a000)
[ 1105.523875] Stack:
[ 1105.523875]  ce06bf70 c10089d9 ce06bf88 c1009220 d1bf6876 e022ed99 1a6ddff1 cfff9fa0
[ 1105.523875]  ce06bfb4 c166027a 00000000 00000000 00000000 00000000 8a9ce88b 1a6ddff1
[ 1105.523875]  01000800 0000d248 00000000 00000000 00000000 00000000 00000000 00000000
[ 1105.523875] Call Trace:
[ 1105.523875]  [<c10089d9>] default_idle+0x2f/0x55
[ 1105.523875]  [<c1009220>] cpu_idle+0xc9/0xfb
[ 1105.523875]  [<c166027a>] start_secondary+0x292/0x29a
[ 1105.523875] Code: 0f 22 e0 5d c3 55 89 e5 0f 09 5d c3 55 89 e5 9c 58 5d c3 55 89 e5 50 9d 5d c3 55 89 e5 fa 5d c3 55 89 e5 fb 5d c3 55 89 e5 fb f4 <5d> c3 55 89 e5 f4 5d c3 55 89 e5 57 56 53 53 89 c6 89 55 f0 89
[ 1105.523875] Kernel panic - not syncing: softlockup: hung tasks
[ 1105.523875] Pid: 0, comm: swapper/1 Not tainted 3.5.0-bisect2-05018-g614a6d43 #82
[ 1105.523875] Call Trace:
[ 1105.523875]  [<c1664e92>] panic+0x81/0x194
[ 1105.523875]  [<c107e316>] watchdog_timer_fn+0x115/0x134
[ 1105.523875]  [<c10416fe>] hrtimer_run_queues+0x14c/0x1c6
[ 1105.523875]  [<c107e201>] ? __touch_watchdog+0x1d/0x1d
[ 1105.523875]  [<c10306a2>] run_local_timers+0x8/0x14
[ 1105.523875]  [<c10306d2>] update_process_times+0x24/0x58
[ 1105.523875]  [<c1061092>] tick_periodic+0x66/0x72
[ 1105.523875]  [<c10610ec>] tick_handle_periodic+0x19/0x61
[ 1105.523875]  [<c10176ad>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1105.523875]  [<c12aa9a0>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1105.523875]  [<c1671306>] apic_timer_interrupt+0x36/0x40
[ 1105.523875]  [<c101c512>] ? native_safe_halt+0x5/0x7
[ 1105.523875]  [<c10089d9>] default_idle+0x2f/0x55
[ 1105.523875]  [<c1009220>] cpu_idle+0xc9/0xfb
[ 1105.523875]  [<c166027a>] start_secondary+0x292/0x29a
mountall: ply-event-loop.c:497: ply_event_loop_new: Assertion `loop->epoll_fd >= 0' failed.
[ 1105.523875] Rebooting in 10 seconds..
--
[ 1103.519588] [sched_delayed] sched: RT throttling activated
[ 1103.517937] BUG: soft lockup - CPU#1 stuck for 1024s! [trinity:131]
[ 1103.517937] irq event stamp: 5400
[ 1103.517937] hardirqs last disabled at (5400): [<c166ce9f>] apic_timer_interrupt+0x2f/0x40
[ 1103.517937] softirqs last  enabled at (2968): [<c15eefbe>] unix_sock_destructor+0xae/0xb3
[ 1103.517937] softirqs last disabled at (2966): [<c15eefab>] unix_sock_destructor+0x9b/0xb3
[ 1103.517937] 
[ 1103.517937] Pid: 131, comm: trinity Not tainted 3.5.0-rc7-bisect2-00018-g2a8c088 #93  
[ 1103.517937] EIP: 0060:[<c10a7a06>] EFLAGS: 00010206 CPU: 1
[ 1103.517937] EIP is at __do_fault+0x100/0x38a
[ 1103.517937] EAX: c23b8000 EBX: c63b9000 ECX: 00000400 EDX: 00000163
[ 1103.517937] ESI: c63b9000 EDI: c23b8000 EBP: ce8b1eb8 ESP: ce8b1e74
[ 1103.517937]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.517937] CR0: 8005003b CR2: 0824cc48 CR3: 0e86d000 CR4: 00000690
[ 1103.517937] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.517937] DR6: ffff0ff0 DR7: 00000400
[ 1103.517937] Process trinity (pid: 131, ti=ce8b0000 task=ce8a9120 task.ti=ce8b0000)
[ 1103.517937] Stack:
[ 1103.517937]  00000000 ce826a34 00000001 00000001 0824cc48 ce826a00 00000370 00000200
[ 1103.517937]  ce850370 ce586700 00000029 00000204 0824c000 ce606720 ce850370 ce878930
[ 1103.517937]  00000204 ce8b1efc c10aa215 ce86d080 00000204 00000029 00000000 ce70ff00
[ 1103.517937] Call Trace:
[ 1103.517937]  [<c10aa215>] handle_pte_fault+0x2c3/0x74c
[ 1103.517937]  [<c101f00f>] ? kmap_atomic_prot+0x11f/0x133
[ 1103.517937]  [<c10aa733>] handle_mm_fault+0x95/0xa4
[ 1103.517937]  [<c101ad7a>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.517937]  [<c101bd28>] do_page_fault+0x336/0x36e
[ 1103.517937]  [<c10400da>] ? up_write+0x16/0x2b
[ 1103.517937]  [<c10a259b>] ? vm_mmap_pgoff+0x50/0x59
[ 1103.517937]  [<c166c395>] ? restore_all+0xf/0xf
[ 1103.517937]  [<c101ad7a>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.517937]  [<c101ad7a>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.517937]  [<c101ad9d>] do_async_page_fault+0x23/0x5f
[ 1103.517937]  [<c166d0dc>] error_code+0x6c/0x80
[ 1103.517937] Code: c8 00 0f 84 cd 00 00 00 8b 55 dc f6 42 18 08 75 42 89 d8 e8 30 76 f7 ff 89 c3 8b 45 e0 e8 26 76 f7 ff b9 00 04 00 00 89 c7 89 de <f3> a5 e8 36 76 f7 ff 89 d8 e8 2f 76 f7 ff 8b 45 e0 0f ba 28 03 
[ 1103.517937] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.517937] Pid: 131, comm: trinity Not tainted 3.5.0-rc7-bisect2-00018-g2a8c088 #93
[ 1103.517937] Call Trace:
[ 1103.517937]  [<c1660345>] panic+0x81/0x194
[ 1103.517937]  [<c107cc9a>] watchdog_timer_fn+0x115/0x134
[ 1103.517937]  [<c103fe4a>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.517937]  [<c107cb85>] ? __touch_watchdog+0x1d/0x1d
[ 1103.517937]  [<c102f052>] run_local_timers+0x8/0x14
[ 1103.517937]  [<c102f082>] update_process_times+0x24/0x58
[ 1103.517937]  [<c105f7ae>] tick_periodic+0x66/0x72
[ 1103.517937]  [<c105f808>] tick_handle_periodic+0x19/0x61
[ 1103.517937]  [<c1016169>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.517937]  [<c12a8880>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.517937]  [<c166cea6>] apic_timer_interrupt+0x36/0x40
[ 1103.517937]  [<c10a7a06>] ? __do_fault+0x100/0x38a
[ 1103.517937]  [<c10aa215>] handle_pte_fault+0x2c3/0x74c
[ 1103.517937]  [<c101f00f>] ? kmap_atomic_prot+0x11f/0x133
[ 1103.517937]  [<c10aa733>] handle_mm_fault+0x95/0xa4
[ 1103.517937]  [<c101ad7a>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.517937]  [<c101bd28>] do_page_fault+0x336/0x36e
[ 1103.517937]  [<c10400da>] ? up_write+0x16/0x2b
[ 1103.517937]  [<c10a259b>] ? vm_mmap_pgoff+0x50/0x59
[ 1103.517937]  [<c166c395>] ? restore_all+0xf/0xf
[ 1103.517937]  [<c101ad7a>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.517937]  [<c101ad7a>] ? kvm_read_and_reset_pf_reason+0x33/0x33
[ 1103.517937]  [<c101ad9d>] do_async_page_fault+0x23/0x5f
[ 1103.517937]  [<c166d0dc>] error_code+0x6c/0x80
[ 1103.517937] hardirqs last  enabled at (5399): [<c109865c>] get_page_from_freelist+0x2ca/0x446
[ 1103.517937] Rebooting in 10 seconds..
--
[ 1104.518348] [sched_delayed] sched: RT throttling activated
[ 1104.519792] BUG: soft lockup - CPU#1 stuck for 1025s! [swapper/1:0]
[ 1104.519792] irq event stamp: 12060
[ 1104.519792] hardirqs last  enabled at (12057): [<c166c395>] restore_all_notrace+0x0/0x18
[ 1104.519792] hardirqs last disabled at (12058): [<c166ce9f>] apic_timer_interrupt+0x2f/0x40
[ 1104.519792] softirqs last  enabled at (12060): [<c102b64b>] _local_bh_enable+0xd/0xf
[ 1104.519792] softirqs last disabled at (12059): [<c102bab8>] irq_enter+0x2c/0x4d
[ 1104.519792] 
[ 1104.519792] Pid: 0, comm: swapper/1 Not tainted 3.5.0-rc7-bisect2-00018-g2a8c088 #93  
[ 1104.519792] EIP: 0060:[<c101aece>] EFLAGS: 00000286 CPU: 1
[ 1104.519792] EIP is at native_safe_halt+0x5/0x7
[ 1104.519792] EAX: ce060000 EBX: ffffffff ECX: ce060000 EDX: 00000007
[ 1104.519792] ESI: c1954c80 EDI: ffff8b9e EBP: ce05ff68 ESP: ce05ff68
[ 1104.519792]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1104.519792] CR0: 8005003b CR2: b77e052c CR3: 0f878000 CR4: 00000690
[ 1104.519792] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1104.519792] DR6: ffff0ff0 DR7: 00000400
[ 1104.519792] Process swapper/1 (pid: 0, ti=ce05e000 task=ce060000 task.ti=ce05e000)
[ 1104.519792] Stack:
[ 1104.519792]  ce05ff70 c10089b1 ce05ff88 c100920c 157956ca 23dcdbea 5e27d951 cfff9fa0
[ 1104.519792]  ce05ffb4 c165b4be 00000000 00000000 00000000 00000000 ce56e1ef 5e27d951
[ 1104.519792]  01000800 0000d248 00000000 00000000 00000000 00000000 00000000 00000000
[ 1104.519792] Call Trace:
--
[    4.512630] Freeing unused kernel memory: 456k freed
[ 1103.517164] BUG: soft lockup - CPU#1 stuck for 1023s! [swapper/1:0]
[ 1103.517164] irq event stamp: 5018
[ 1103.517164] hardirqs last  enabled at (5015): [<c10089a1>] default_idle+0x1f/0x55
mountall: ply-ev[ 1103.517164] hardirqs last disabled at (5016): ent-loop.c:497: [<c166ce9f>] apic_timer_interrupt+0x2f/0x40
ply_event_loop_n[ 1103.517164] softirqs last  enabled at (5018): ew: Assertion `l[<c102b64b>] _local_bh_enable+0xd/0xf
oop->epoll_fd >=[ 1103.517164] softirqs last disabled at (5017):  0' failed.[<c102bab8>] irq_enter+0x2c/0x4d
[ 1103.526001] [sched_delayed] sched: RT throttling activated
[ 1103.517164] 
[ 1103.517164] Pid: 0, comm: swapper/1 Not tainted 3.5.0-rc7-bisect2-00018-g2a8c088 #93  
[ 1103.517164] EIP: 0060:[<c101aece>] EFLAGS: 00000286 CPU: 1
[ 1103.517164] EIP is at native_safe_halt+0x5/0x7

[ 1103.517164] EAX: ce060000 EBX: ffffffff ECX: ce060000 EDX: 00000007
[ 1103.517164] ESI: c1954c80 EDI: ffff8b9e EBP: ce05ff68 ESP: ce05ff68
[ 1103.517164]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.517164] CR0: 8005003b CR2: b7665bd0 CR3: 0e852000 CR4: 00000690
[ 1103.517164] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.517164] DR6: ffff0ff0 DR7: 00000400
[ 1103.517164] Process swapper/1 (pid: 0, ti=ce05e000 task=ce060000 task.ti=ce05e000)
[ 1103.517164] Stack:
[ 1103.517164]  ce05ff70 c10089b1 ce05ff88 c100920c
--
[    4.696250] Freeing unused kernel memory: 456k freed
[ 1103.520327] BUG: soft lockup - CPU#1 stuck for 1023s! [swapper/1:0]
[ 1103.520327] irq event stamp: 13124
[ 1103.520327] hardirqs last  enabled at (13121): [<c166c395>] restore_all_notrace+0x0/0x18
[ 1103.520327] hardirqs last disabled at (13122): [<c166ce9f>] apic_timer_interrupt+0x2f/0x40
[ 1103.520327] softirqs last  enabled at (13124): [<c102b64b>] _local_bh_enable+0xd/0xf
[ 1103.520327] softirqs last disabled at (13123): [<c102bab8>] irq_enter+0x2c/0x4d
[ 1103.520327] 
[ 1103.520327] Pid: 0, comm: swapper/1 Not tainted 3.5.0-rc7-bisect2-00018-g2a8c088 #93  
[ 1103.520327] EIP: 0060:[<c101aece>] EFLAGS: 00000286 CPU: 1
[ 1103.520327] EIP is at native_safe_halt+0x5/0x7
[ 1103.520327] EAX: ce060000 EBX: ffffffff ECX: ce060000 EDX: 00000007
[ 1103.520327] ESI: c1954c80 EDI: ffff8b9f EBP: ce05ff68 ESP: ce05ff68
[ 1103.520327]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1103.520327] CR0: 8005003b CR2: 08051f68 CR3: 0e876000 CR4: 00000690
[ 1103.520327] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1103.520327] DR6: ffff0ff0 DR7: 00000400
[ 1103.520327] Process swapper/1 (pid: 0, ti=ce05e000 task=ce060000 task.ti=ce05e000)
[ 1103.520327] Stack:
[ 1103.520327]  ce05ff70 c10089b1 ce05ff88 c100920c 95c987b1 a42d0cd0 de77b93b cfff9fa0
[ 1103.520327]  ce05ffb4 c165b4be 00000000 00000000 00000000 00000000 4ea6c1d9 de77b93b
[ 1103.520327]  01000800 0000d248 00000000 00000000 00000000 00000000 00000000 00000000
[ 1103.520327] Call Trace:
[ 1103.520327]  [<c10089b1>] default_idle+0x2f/0x55
[ 1103.520327]  [<c100920c>] cpu_idle+0xc9/0xfb
[ 1103.520327]  [<c165b4be>] start_secondary+0x29c/0x2a4
[ 1103.520327] Code: 0f 22 e0 5d c3 55 89 e5 0f 09 5d c3 55 89 e5 9c 58 5d c3 55 89 e5 50 9d 5d c3 55 89 e5 fa 5d c3 55 89 e5 fb 5d c3 55 89 e5 fb f4 <5d> c3 55 89 e5 f4 5d c3 55 89 e5 57 56 53 53 89 c6 89 55 f0 89 
[ 1103.520327] Kernel panic - not syncing: softlockup: hung tasks
[ 1103.520327] Pid: 0, comm: swapper/1 Not tainted 3.5.0-rc7-bisect2-00018-g2a8c088 #93
[ 1103.520327] Call Trace:
[ 1103.520327]  [<c1660345>] panic+0x81/0x194
[ 1103.520327]  [<c107cc9a>] watchdog_timer_fn+0x115/0x134
[ 1103.520327]  [<c103fe4a>] hrtimer_run_queues+0x14c/0x1c6
[ 1103.520327]  [<c107cb85>] ? __touch_watchdog+0x1d/0x1d
[ 1103.520327]  [<c102f052>] run_local_timers+0x8/0x14
[ 1103.520327]  [<c102f082>] update_process_times+0x24/0x58
[ 1103.520327]  [<c105f7ae>] tick_periodic+0x66/0x72
[ 1103.520327]  [<c105f808>] tick_handle_periodic+0x19/0x61
[ 1103.520327]  [<c1016169>] smp_apic_timer_interrupt+0x6d/0x7f
[ 1103.520327]  [<c12a8880>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 1103.520327]  [<c166cea6>] apic_timer_interrupt+0x36/0x40
[ 1103.520327]  [<c101aece>] ? native_safe_halt+0x5/0x7
[ 1103.520327]  [<c10089b1>] default_idle+0x2f/0x55
[ 1103.520327]  [<c100920c>] cpu_idle+0xc9/0xfb
[ 1103.520327]  [<c165b4be>] start_secondary+0x29c/0x2a4
[ 1103.520327] Rebooting in 10 seconds..

Thanks,
Fengguang

View attachment "dmesg-kvm-waimea-5039-2012-09-05-17-32-55-3.6.0-rc1-00023-g8a13a4c-66" of type "text/plain" (158276 bytes)

View attachment "config-3.6.0-rc1-00023-g8a13a4c" of type "text/plain" (82049 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ