[<prev] [next>] [day] [month] [year] [list]
Message-ID: <556FA0C3.4050606@intel.com>
Date: Thu, 04 Jun 2015 08:50:11 +0800
From: Huang Ying <ying.huang@...el.com>
To: Trond Myklebust <trond.myklebust@...marydata.com>
CC: LKML <linux-kernel@...r.kernel.org>, LKP ML <lkp@...org>
Subject: [lkp] [NFSv4] 5cae02f4279: INFO: task dmesg:3593 blocked for more
than 120 seconds.
FYI, we noticed the below changes on
git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master
commit 5cae02f42793130e1387f4ec09c4d07056ce9fa5 ("NFSv4: Always drain the slot table before re-establishing the lease")
+--------------------------------------------------+------------+------------+
| | e8d975e73e | 5cae02f427 |
+--------------------------------------------------+------------+------------+
| boot_successes | 2 | 1 |
| boot_failures | 20 | 20 |
| BUG:kernel_test_crashed | 20 | |
| INFO:task_blocked_for_more_than#seconds | 0 | 16 |
| RIP:native_safe_halt | 0 | 16 |
| RIP:flat_send_IPI_mask | 0 | 16 |
| RIP:mark_lock | 0 | 1 |
| Kernel_panic-not_syncing:hung_task:blocked_tasks | 0 | 16 |
| backtrace:__close_fd | 0 | 15 |
| backtrace:SyS_close | 0 | 15 |
| backtrace:cpu_startup_entry | 0 | 13 |
| backtrace:watchdog | 0 | 16 |
| backtrace:ep_poll | 0 | 7 |
| backtrace:SyS_epoll_wait | 0 | 8 |
| BUG:kernel_test_hang | 0 | 4 |
| RIP:lock_is_held | 0 | 1 |
| RIP:__switch_to | 0 | 1 |
| RIP:ep_send_events_proc | 0 | 1 |
| RIP:ep_poll | 0 | 1 |
| RIP:__schedule | 0 | 1 |
| RIP:find_busiest_group | 0 | 1 |
| RIP:__lock_is_held | 0 | 1 |
| RIP:pvclock_clocksource_read | 0 | 2 |
| RIP:lock_acquired | 0 | 1 |
+--------------------------------------------------+------------+------------+
[ 185.679164] ??? Writer stall state 8 g1925 c1925 f0x0
[ 185.680231] rcu_sched: wait state: 1 ->state: 0x1
[ 185.681258] rcu_bh: wait state: 1 ->state: 0x1
[ 240.431123] INFO: task dmesg:3593 blocked for more than 120 seconds.
[ 240.432384] Not tainted 4.1.0-rc6-wl-ath-03998-g482548f #15
[ 240.433582] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 240.435373] dmesg D ffff88007da07b08 13320 3593 3554 0x00000000
[ 240.436804] ffff88007da07b08 0000000000000000 ffff8800763d83c0 ffff88007db10000
[ 240.438839] ffff8800744d0000 ffff88007da07ae8 ffff88007da08000 ffff8800763d7840
[ 240.440869] 7fffffffffffffff ffffffff81c6812e 0000000000000002 ffff88007da07b28
[ 240.442897] Call Trace:
[ 240.443677] [<ffffffff81c6812e>] ? bit_wait+0x4b/0x4b
[ 240.444755] [<ffffffff81c67a8c>] schedule+0x74/0x83
[ 240.445810] [<ffffffff81c6b3c8>] schedule_timeout+0x30/0x336
[ 240.446953] [<ffffffff810c08fb>] ? local_clock+0x19/0x22
[ 240.448056] [<ffffffff810d59b7>] ? lock_release_holdtime+0xd1/0xdd
[ 240.449328] [<ffffffff81109bd2>] ? ktime_get+0x90/0x119
[ 240.450428] [<ffffffff81c6812e>] ? bit_wait+0x4b/0x4b
[ 240.451513] [<ffffffff8107edd6>] ? kvm_clock_read+0x25/0x2e
[ 240.452639] [<ffffffff8107ede8>] ? kvm_clock_get_cycles+0x9/0xb
[ 240.453804] [<ffffffff81109c01>] ? ktime_get+0xbf/0x119
[ 240.454898] [<ffffffff81c6812e>] ? bit_wait+0x4b/0x4b
[ 240.455970] [<ffffffff81c66a72>] io_schedule_timeout+0xba/0x131
[ 240.457136] [<ffffffff810ceaef>] ? prepare_to_wait+0x7e/0x89
[ 240.458283] [<ffffffff81c6817f>] bit_wait_io+0x51/0x55
[ 240.459371] [<ffffffff81c67e21>] __wait_on_bit+0x4d/0x82
[ 240.460482] [<ffffffff81178bbb>] ? find_get_pages_tag+0x1fd/0x25b
[ 240.461673] [<ffffffff81178206>] wait_on_page_bit+0x76/0x78
[ 240.462806] [<ffffffff810ceffd>] ? autoremove_wake_function+0x3d/0x3d
[ 240.464034] [<ffffffff81178315>] filemap_fdatawait_range+0xc9/0x13a
[ 240.465244] [<ffffffff81179e8e>] filemap_write_and_wait_range+0x42/0x5b
[ 240.466500] [<ffffffff81315fec>] nfs4_file_fsync+0x10d/0x3bb
[ 240.467645] [<ffffffff811fcb73>] vfs_fsync_range+0x8f/0x9e
[ 240.468771] [<ffffffff811fcb9e>] vfs_fsync+0x1c/0x1e
[ 240.469838] [<ffffffff812ebcf9>] nfs_file_flush+0x73/0x77
[ 240.470952] [<ffffffff811cc8f0>] filp_close+0x40/0x75
[ 240.472026] [<ffffffff811ea9e6>] __close_fd+0xc2/0xdd
[ 240.473097] [<ffffffff811cc948>] SyS_close+0x23/0x4c
[ 240.474165] [<ffffffff81c6cf2e>] system_call_fastpath+0x12/0x76
[ 240.475332] no locks held by dmesg/3593.
[ 240.476270] sending NMI to all CPUs:
[ 240.478218] NMI backtrace for cpu 0
[ 240.479100] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.1.0-rc6-wl-ath-03998-g482548f #15
[ 240.480926] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
[ 240.482964] task: ffffffff82264580 ti: ffffffff82254000 task.ti: ffffffff82254000
[ 240.484709] RIP: 0010:[<ffffffff8107efd1>] [<ffffffff8107efd1>] native_safe_halt+0x6/0x8
[ 240.486530] RSP: 0018:ffffffff82257e88 EFLAGS: 00000202
[ 240.487622] RAX: ffffffff82264580 RBX: 0000000000000000 RCX: 000000000000000f
[ 240.488965] RDX: 0000000000000001 RSI: 0000000000000004 RDI: ffffffff82264580
[ 240.490290] RBP: ffffffff82257e88 R08: 0000000000000006 R09: 0000000000000001
[ 240.491619] R10: 0000000000000040 R11: ffffffff83530ff8 R12: 0000000000000000
[ 240.492958] R13: ffffffff82254000 R14: ffffffff82254000 R15: ffffffff82254000
[ 240.494250] FS: 0000000000000000(0000) GS:ffff880075c00000(0000) knlGS:0000000000000000
[ 240.496065] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 240.497202] CR2: 00007fe0cf6fd000 CR3: 000000000225f000 CR4: 00000000000006f0
[ 240.498505] Stack:
[ 240.499227] ffffffff82257ea8 ffffffff8104f427 0000000000000000 0000000000000000
[ 240.500962] ffffffff82257eb8 ffffffff8104fdd3 ffffffff82257f18 ffffffff810cf58e
[ 240.502700] ffffffff82257ed8 ffffffff82258000 ffff880078c88c80 ffffffff82258000
[ 240.504429] Call Trace:
[ 240.505203] [<ffffffff8104f427>] default_idle+0x62/0x1db
[ 240.506304] [<ffffffff8104fdd3>] arch_cpu_idle+0xf/0x11
[ 240.507392] [<ffffffff810cf58e>] cpu_idle_loop+0x394/0x50f
[ 240.508513] [<ffffffff810cf71c>] cpu_startup_entry+0x13/0x13
[ 240.509649] [<ffffffff81c4d8b6>] rest_init+0x13d/0x143
[ 240.510726] [<ffffffff825f7f0a>] start_kernel+0x433/0x440
[ 240.511838] [<ffffffff825f7120>] ? early_idt_handler_array+0x120/0x120
[ 240.513070] [<ffffffff825f74a2>] x86_64_start_reservations+0x2a/0x2c
[ 240.514284] [<ffffffff825f75dc>] x86_64_start_kernel+0x138/0x147
[ 240.515467] Code: 48 89 e5 0f 09 5d c3 55 48 89 e5 9c 58 5d c3 55 48 89 e5 57 9d 5d c3 55 48 89 e5 fa 5d c3 55 48 89 e5 fb 5d c3 55 48 89 e5 fb f4 <5d> c3 55 48 89 e5 f4 5d c3 55 49 89 ca 49 89 d1 8b 07 48 89 e5
[ 240.518874] NMI backtrace for cpu 1
[ 240.519766] CPU: 1 PID: 620 Comm: khungtaskd Not tainted 4.1.0-rc6-wl-ath-03998-g482548f #15
[ 240.521624] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
[ 240.523661] task: ffff8800744d25b0 ti: ffff88007527c000 task.ti: ffff88007527c000
[ 240.525397] RIP: 0010:[<ffffffff810785b4>] [<ffffffff810785b4>] flat_send_IPI_mask+0x77/0xa9
[ 240.527255] RSP: 0018:ffff88007527fd58 EFLAGS: 00000046
[ 240.528343] RAX: 000000000f000000 RBX: 0000000000000c00 RCX: 0000000000000000
[ 240.529639] RDX: 0000000000000c00 RSI: ffffffff81078569 RDI: 0000000000000300
[ 240.530930] RBP: ffff88007527fd78 R08: 0000000000000001 R09: 0000000000000000
[ 240.532229] R10: ffff880078a023a0 R11: 00000000000fffa0 R12: 0000000000000246
[ 240.533525] R13: 000000000000000f R14: 0000000000000002 R15: ffff8800744d0330
[ 240.534813] FS: 0000000000000000(0000) GS:ffff880075e00000(0000) knlGS:0000000000000000
[ 240.536621] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 240.537760] CR2: 00007f84d08c2220 CR3: 000000000225f000 CR4: 00000000000006e0
[ 240.539051] Stack:
[ 240.539756] 0000000000000040 000000000000c700 0000000000000001 0000000000000078
[ 240.541483] ffff88007527fdc8 ffffffff81075bf6 ffff88007527fdb8 ffffffff810d5e8d
[ 240.543204] ffff88007527fc8c ffff8800744d0000 ffff8800744d0000 0000000000000003
[ 240.544936] Call Trace:
[ 240.545709] [<ffffffff81075bf6>] arch_trigger_all_cpu_backtrace+0x162/0x294
[ 240.546993] [<ffffffff810d5e8d>] ? lockdep_print_held_locks+0x3b/0x9d
[ 240.548209] [<ffffffff8113d035>] watchdog+0x47b/0x5bc
[ 240.549283] [<ffffffff8113cbba>] ? reset_hung_task_detector+0x15/0x15
[ 240.550510] [<ffffffff810aefa3>] kthread+0x116/0x11e
[ 240.551574] [<ffffffff810d7f0a>] ? trace_hardirqs_on+0xd/0xf
[ 240.552712] [<ffffffff810aee8d>] ? kthread_create_on_node+0x1b6/0x1b6
[ 240.553935] [<ffffffff81c6d362>] ret_from_fork+0x42/0x70
[ 240.555039] [<ffffffff810aee8d>] ? kthread_create_on_node+0x1b6/0x1b6
[ 240.556262] Code: c4 10 74 04 f3 90 eb f0 44 89 e8 c1 e0 18 89 04 25 10 c3 5f ff 44 89 f2 09 da 80 cf 04 41 83 fe 02 0f 44 d3 89 14 25 00 c3 5f ff <41> f7 c4 00 02 00 00 75 11 4c 89 e7 57 9d 66 66 90 66 90 e8 fa
[ 240.559680] NMI backtrace for cpu 2
[ 240.560568] CPU: 2 PID: 2085 Comm: systemd-journal Not tainted 4.1.0-rc6-wl-ath-03998-g482548f #15
[ 240.562477] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
[ 240.564516] task: ffff88007ddc8000 ti: ffff88007d8dc000 task.ti: ffff88007d8dc000
[ 240.566256] RIP: 0010:[<ffffffff810c066f>] [<ffffffff810c066f>] sched_clock_stable+0x6/0x14
[ 240.568107] RSP: 0018:ffff88007d8dfd20 EFLAGS: 00000046
[ 240.569196] RAX: 0000000000000000 RBX: ffff88007ddc8000 RCX: ffff88007ddc8898
[ 240.570492] RDX: 0000000000000038 RSI: 0000000000000001 RDI: 0000000000000002
[ 240.571788] RBP: ffff88007d8dfd48 R08: 0000000000000004 R09: 0000000000000001
[ 240.573078] R10: ffff88007ddc8000 R11: 000000000000bc45 R12: ffff88007ddc8898
[ 240.574370] R13: 0000000000000002 R14: 0000000000000001 R15: 0000000000000046
[ 240.575666] FS: 00007f86bba7d840(0000) GS:ffff880076000000(0000) knlGS:0000000000000000
[ 240.577471] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 240.578608] CR2: 00007f86b9b76018 CR3: 000000007f35f000 CR4: 00000000000006e0
[ 240.579901] Stack:
[ 240.580630] ffffffff810c080e ffff88007ddc8898 ffff88007ddc8000 ffff88007ddc8898
[ 240.582357] ffffffff8120d086 ffff88007d8dfd58 ffffffff810c08fb ffff88007d8dfd78
[ 240.584090] ffffffff810d58f5 ffff88007ddc8000 ffff88007d861a58 ffff88007d8dfdb8
[ 240.585824] Call Trace:
[ 240.586600] [<ffffffff810c080e>] ? sched_clock_cpu+0x12/0xaf
[ 240.587734] [<ffffffff8120d086>] ? ep_scan_ready_list+0x98/0x221
[ 240.588913] [<ffffffff810c08fb>] local_clock+0x19/0x22
[ 240.589996] [<ffffffff810d58f5>] lock_release_holdtime+0xf/0xdd
[ 240.591232] [<ffffffff810da3f2>] lock_release+0x23a/0x33f
[ 240.592344] [<ffffffff81c6c582>] _raw_spin_unlock_irqrestore+0x24/0x5d
[ 240.593587] [<ffffffff8120c5be>] ? ep_read_events_proc+0xf2/0xf2
[ 240.594758] [<ffffffff8120d086>] ep_scan_ready_list+0x98/0x221
[ 240.595913] [<ffffffff8120d3aa>] ep_poll+0x177/0x2f9
[ 240.596973] [<ffffffff810bca6d>] ? wake_up_process+0x39/0x39
[ 240.598108] [<ffffffff8120e51e>] SyS_epoll_wait+0x96/0xbb
[ 240.599214] [<ffffffff81c6cf2e>] system_call_fastpath+0x12/0x76
[ 240.600379] Code: 48 85 c9 48 0f 4e d7 48 39 c2 48 0f 49 d0 48 89 f0 f0 49 0f b1 54 24 10 48 39 f0 75 9c 5b 48 89 d0 41 5c 5d c3 8b 05 f5 33 71 01 <55> 48 89 e5 5d 85 c0 0f 9f c0 0f b6 c0 c3 55 48 89 e5 e8 e3 ff
[ 240.603794] NMI backtrace for cpu 3
[ 240.604684] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.1.0-rc6-wl-ath-03998-g482548f #15
[ 240.617413] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
[ 240.619454] task: ffff8800757825b0 ti: ffff880075794000 task.ti: ffff880075794000
[ 240.621189] RIP: 0010:[<ffffffff8107efd1>] [<ffffffff8107efd1>] native_safe_halt+0x6/0x8
[ 240.623012] RSP: 0018:ffff880075797e88 EFLAGS: 00000206
[ 240.624099] RAX: ffff8800757825b0 RBX: 0000000000000000 RCX: 000000000000000f
[ 240.625386] RDX: 0000000000000001 RSI: 0000000000000004 RDI: ffff8800757825b0
[ 240.626679] RBP: ffff880075797e88 R08: 0000000000000004 R09: 0000000000000001
[ 240.627968] R10: 0000000000000016 R11: 0000000000000005 R12: 0000000000000003
[ 240.629264] R13: ffff880075794000 R14: ffff880075794000 R15: ffff880075794000
[ 240.630560] FS: 0000000000000000(0000) GS:ffff880076200000(0000) knlGS:0000000000000000
[ 240.632369] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 240.633507] CR2: 00007f9ba94e9000 CR3: 000000000225f000 CR4: 00000000000006e0
[ 240.634795] Stack:
[ 240.635519] ffff880075797ea8 ffffffff8104f427 0000000000000000 0000000000000000
[ 240.637242] ffff880075797eb8 ffffffff8104fdd3 ffff880075797f18 ffffffff810cf58e
[ 240.638980] ffffffff822b6750 0000000000000286 ffff880075797f08 ffff880075798000
[ 240.640719] Call Trace:
[ 240.641491] [<ffffffff8104f427>] default_idle+0x62/0x1db
[ 240.642590] [<ffffffff8104fdd3>] arch_cpu_idle+0xf/0x11
[ 240.643676] [<ffffffff810cf58e>] cpu_idle_loop+0x394/0x50f
[ 240.644796] [<ffffffff810cf71c>] cpu_startup_entry+0x13/0x13
[ 240.645933] [<ffffffff81071f57>] start_secondary+0x14d/0x151
[ 240.647068] Code: 48 89 e5 0f 09 5d c3 55 48 89 e5 9c 58 5d c3 55 48 89 e5 57 9d 5d c3 55 48 89 e5 fa 5d c3 55 48 89 e5 fb 5d c3 55 48 89 e5 fb f4 <5d> c3 55 48 89 e5 f4 5d c3 55 49 89 ca 49 89 d1 8b 07 48 89 e5
[ 240.650548] Kernel panic - not syncing: hung_task: blocked tasks
[ 240.651483] CPU: 1 PID: 620 Comm: khungtaskd Not tainted 4.1.0-rc6-wl-ath-03998-g482548f #15
[ 240.651483] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
[ 240.651483] ffff8800744d0000 ffff88007527fd48 ffffffff81c5d7a5 0000000000000006
[ 240.651483] ffffffff820c4053 ffff88007527fdc8 ffffffff81c57323 ffff88007620cc98
[ 240.651483] 0000000000000008 ffff88007527fdd8 ffff88007527fd78 ffff88007527fdc8
[ 240.651483] Call Trace:
[ 240.651483] [<ffffffff81c5d7a5>] dump_stack+0x4c/0x65
[ 240.651483] [<ffffffff81c57323>] panic+0xc9/0x1f7
[ 240.651483] [<ffffffff8113d043>] watchdog+0x489/0x5bc
[ 240.651483] [<ffffffff8113cbba>] ? reset_hung_task_detector+0x15/0x15
[ 240.651483] [<ffffffff810aefa3>] kthread+0x116/0x11e
[ 240.651483] [<ffffffff810d7f0a>] ? trace_hardirqs_on+0xd/0xf
[ 240.651483] [<ffffffff810aee8d>] ? kthread_create_on_node+0x1b6/0x1b6
[ 240.651483] [<ffffffff81c6d362>] ret_from_fork+0x42/0x70
[ 240.651483] [<ffffffff810aee8d>] ? kthread_create_on_node+0x1b6/0x1b6
[ 240.651483] Kernel Offset: disabled
Elapsed time: 250
Thanks,
Ying Huang
-------------------------------------
lkp@...ists.intel.com
https://eclists.intel.com/sympa/info/lkp
Unsubscribe by sending email to sympa@...ists.intel.com with subject "Unsubscribe lkp"
View attachment "config-4.1.0-rc6-00202-g5cae02f" of type "text/plain" (103213 bytes)
View attachment ".dmesg" of type "text/plain" (46792 bytes)
Powered by blists - more mailing lists