Message-ID: <20170313014222.efwlhhj2xo5b6gkn@wfg-t540p.sh.intel.com>
Date:   Mon, 13 Mar 2017 09:42:22 +0800
From:   Fengguang Wu <fengguang.wu@...el.com>
To:     ACPI Devel Maling List <linux-acpi@...r.kernel.org>
Cc:     linux-kernel@...r.kernel.org, LKP <lkp@...org>
Subject: [acpi] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s!
 [swapper/0:1]

Greetings,

This is an old bug and cannot be bisected to its root cause.
Perhaps it requires some manual investigation. The attached
reproduce-* may help the debugging.

efdb4167e6  scripts/faddr2line: Fix "size mismatch" error
bb5e5ce545  x86/dumpstack: Remove kernel text addresses from stack dump
106e4da602  Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
5be4921c99  Add linux-next specific files for 20170310
+------------------------------------------------+------------+------------+------------+---------------+
|                                                | efdb4167e6 | bb5e5ce545 | 106e4da602 | next-20170310 |
+------------------------------------------------+------------+------------+------------+---------------+
| boot_successes                                 | 0          | 0          | 18         | 2             |
| boot_failures                                  | 74         | 18         | 8          | 18            |
| BUG:soft_lockup-CPU##stuck_for#s               | 53         | 17         | 8          | 18            |
| EIP_is_at_raw_spin_unlock_irqrestore           | 52         |            |            |               |
| calltrace:virtio_pci_driver_init               | 53         |            |            |               |
| Kernel_panic-not_syncing:softlockup:hung_tasks | 53         | 17         | 8          | 18            |
| BUG:unable_to_handle_kernel                    | 21         | 1          |            |               |
| Oops:#[##]                                     | 21         | 1          |            |               |
| EIP_is_at_i2c_do_del_adapter                   | 21         |            |            |               |
| calltrace:of_unittest                          | 21         |            |            |               |
| Kernel_panic-not_syncing:Fatal_exception       | 21         | 1          |            |               |
| EIP_is_at_kmem_cache_alloc                     | 1          |            |            |               |
| EIP:_raw_spin_unlock_irqrestore                | 0          | 17         | 8          | 16            |
| EIP:i2c_do_del_adapter                         | 0          | 1          |            |               |
| EIP:acpi_ut_update_ref_count                   | 0          | 0          | 0          | 1             |
| EIP:kmem_cache_alloc                           | 0          | 0          | 0          | 1             |
+------------------------------------------------+------------+------------+------------+---------------+

[   12.055229] isapnp: No Plug & Play device found
[   14.198035] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
[   22.614543] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
[   31.724849] ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10
[   40.140941] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11
[   65.545954] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [swapper/0:1]
[   65.547202] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.9.0-rc1-00121-gbb5e5ce5 #1
[   65.548373] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.9.3-20161025_171302-gandalf 04/01/2014
[   65.549994] task: b02c9000 task.stack: b02d4000
[   65.550724] EIP: _raw_spin_unlock_irqrestore+0x76/0xc0
[   65.551525] EFLAGS: 00200246 CPU: 0
[   65.552082] EAX: 00200246 EBX: 00200246 ECX: 00000000 EDX: 00000000
[   65.553054] ESI: b39c5ac0 EDI: 00000016 EBP: b02d5b60 ESP: b02d5b58
[   65.554075]  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
[   65.554919] CR0: 80050033 CR2: 00000000 CR3: 0424c000 CR4: 000406d0
[   65.555897] Stack:
[   65.556223]  b39c5adc b39c5af0 b02d5b8c b13f8385 00200246 024080c0 00000008 c25ee59c
[   65.557654]  0000002c 00000000 00000001 002887d5 b02a42d0 b02d5bc4 b13fbaa9 b02d5bb0
[   65.559038]  b11c5203 00000000 b02a4518 00000000 00000000 b02d5bc0 024080c0 00200246
[   65.560465] Call Trace:
[   65.560871]  slob_alloc+0x3c5/0x6f0
[   65.561481]  kmem_cache_alloc+0x5a9/0xb60
[   65.562080]  ? do_raw_spin_lock+0x1a3/0x420
[   65.562703]  acpi_ut_create_generic_state+0xa3/0x10b
[   65.563484]  acpi_ut_create_update_state+0x35/0x90
[   65.564193]  acpi_ut_create_update_state_and_push+0x76/0xe7
[   65.565012]  acpi_ut_update_object_reference+0x274/0x534
[   65.565793]  acpi_ut_add_reference+0x123/0x136
[   65.566487]  acpi_ds_method_data_get_value+0x38f/0x3cf
[   65.567245]  acpi_ex_resolve_to_value+0x1e3/0x970
[   65.567938]  acpi_ex_resolve_operands+0x8cd/0x14f9
[   65.568643]  acpi_ds_exec_end_op+0x2ec/0x1417
[   65.569288]  acpi_ps_parse_loop+0x1502/0x1667
[   65.569980]  ? acpi_ut_trace+0x32/0xff
[   65.570538]  acpi_ps_parse_aml+0x467/0xd50
[   65.571161]  acpi_ps_execute_method+0x54f/0x733
[   65.571836]  acpi_ns_evaluate+0x677/0x9c5
[   65.572429]  acpi_ut_evaluate_object+0x175/0x596
[   65.573158]  acpi_rs_get_prt_method_data+0x59/0x12f
[   65.573878]  acpi_get_irq_routing_table+0xcc/0x10f
[   65.574578]  acpi_pci_irq_find_prt_entry+0x20a/0x7c7
[   65.575315]  acpi_pci_irq_lookup+0x30/0x705
[   65.575937]  acpi_pci_irq_disable+0x110/0x21e
[   65.576625]  pcibios_disable_device+0x4d/0x80
[   65.577270]  do_pci_disable_device+0xb2/0xf0
[   65.577904]  pci_disable_device+0x265/0x290
[   65.578523]  ? virtio_pci_modern_remove+0xd4/0x130
[   65.579235]  virtio_pci_remove+0xa2/0x100
[   65.579877]  pci_device_remove+0x12a/0x140
[   65.580481]  driver_probe_device+0x397/0xb10
[   65.581113]  __driver_attach+0x29f/0x2c0
[   65.581699]  ? driver_probe_device+0xb10/0xb10
[   65.582348]  bus_for_each_dev+0xe4/0x170
[   65.582931]  driver_attach+0x2c/0x40
[   65.583514]  ? driver_probe_device+0xb10/0xb10
[   65.584171]  bus_add_driver+0x2fd/0x620
[   65.584742]  driver_register+0x131/0x380
[   65.585324]  ? __raw_spin_lock_init+0x52/0x90
[   65.585968]  ? virtio_mmio_init+0x2d/0x2d
[   65.586604]  __pci_register_driver+0x66/0x80
[   65.587249]  virtio_pci_driver_init+0x22/0x32
[   65.587897]  do_one_initcall+0x12c/0x2de
[   65.588481]  ? kernel_init_freeable+0x23c/0x42b
[   65.589152]  kernel_init_freeable+0x26a/0x42b
[   65.589846]  ? rest_init+0x180/0x180
[   65.590372]  kernel_init+0x1f/0x360
[   65.590895]  ? schedule_tail_wrapper+0x9/0xc
[   65.591520]  ? rest_init+0x180/0x180
[   65.592059]  ret_from_fork+0x19/0x24
[   65.592584] Code: 21 da 32 fe 83 05 b0 ba 85 b4 01 89 f0 83 15 b4 ba 85 b4 00 e8 6c 53 33 fe 83 05 b8 ba 85 b4 01 89 d8 83 15 bc ba 85 b4 00 50 9d <8d> 74 26 00 83 05 c0 ba 85 b4 01 83 15 c4 ba 85 b4 00 83 05 c8
[   65.596837] Kernel panic - not syncing: softlockup: hung tasks
[   65.597685] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G             L  4.9.0-rc1-00121-gbb5e5ce5 #1
[   65.598959] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.9.3-20161025_171302-gandalf 04/01/2014
[   65.600473]  b02d5a24 b1f8dca4 00200046 00000000 00000001 00000000 b02d5a3c b1322da2
[   65.601774]  00000001 00000000 00000001 b02c9000 b02d5a70 b1293edb b379a907 00000000
[   65.603072]  00000018 b02c938c 00000001 b02d5b1c 00000000 00000000 00000001 c2597640
[   65.604427] Call Trace:
[   65.604806]  dump_stack+0x153/0x1cf
[   65.605329]  panic+0x1a9/0x598
[   65.605800]  watchdog_timer_fn+0x5db/0x670
[   65.606402]  hrtimer_run_queues+0x9b5/0x1340
[   65.607092]  ? watchdog+0xa0/0xa0
[   65.607584]  run_local_timers+0x36/0x110
[   65.608171]  update_process_times+0x3d/0xe0
[   65.608799]  tick_nohz_handler+0x173/0x300
[   65.609403]  local_apic_timer_interrupt+0x6a/0xd0
[   65.610145]  smp_apic_timer_interrupt+0x79/0xb0
[   65.610815]  apic_timer_interrupt+0x32/0x38
[   65.611435] EIP: _raw_spin_unlock_irqrestore+0x76/0xc0
[   65.612188] EFLAGS: 00200246 CPU: 0
[   65.612710] EAX: 00200246 EBX: 00200246 ECX: 00000000 EDX: 00000000
[   65.613657] ESI: b39c5ac0 EDI: 00000016 EBP: b02d5b60 ESP: b02d5b58
[   65.614562]  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
[   65.615358]  ? l_show+0x228/0x230
[   65.615858]  slob_alloc+0x3c5/0x6f0
[   65.616508]  kmem_cache_alloc+0x5a9/0xb60
[   65.617104]  ? do_raw_spin_lock+0x1a3/0x420
[   65.617727]  acpi_ut_create_generic_state+0xa3/0x10b
[   65.618454]  acpi_ut_create_update_state+0x35/0x90
[   65.619173]  acpi_ut_create_update_state_and_push+0x76/0xe7
[   65.620051]  acpi_ut_update_object_reference+0x274/0x534
[   65.620830]  acpi_ut_add_reference+0x123/0x136
[   65.621479]  acpi_ds_method_data_get_value+0x38f/0x3cf
[   65.622240]  acpi_ex_resolve_to_value+0x1e3/0x970
[   65.622936]  acpi_ex_resolve_operands+0x8cd/0x14f9
[   65.623682]  acpi_ds_exec_end_op+0x2ec/0x1417
[   65.624329]  acpi_ps_parse_loop+0x1502/0x1667
[   65.624979]  ? acpi_ut_trace+0x32/0xff
[   65.625532]  acpi_ps_parse_aml+0x467/0xd50
[   65.626141]  acpi_ps_execute_method+0x54f/0x733
[   65.626857]  acpi_ns_evaluate+0x677/0x9c5
[   65.627454]  acpi_ut_evaluate_object+0x175/0x596
[   65.628135]  acpi_rs_get_prt_method_data+0x59/0x12f
[   65.628856]  acpi_get_irq_routing_table+0xcc/0x10f
[   65.629557]  acpi_pci_irq_find_prt_entry+0x20a/0x7c7
[   65.630332]  acpi_pci_irq_lookup+0x30/0x705
[   65.630956]  acpi_pci_irq_disable+0x110/0x21e
[   65.631604]  pcibios_disable_device+0x4d/0x80
[   65.632259]  do_pci_disable_device+0xb2/0xf0
[   65.632893]  pci_disable_device+0x265/0x290
[   65.635170]  ? virtio_pci_modern_remove+0xd4/0x130
[   65.635936]  virtio_pci_remove+0xa2/0x100
[   65.636577]  pci_device_remove+0x12a/0x140
[   65.637177]  driver_probe_device+0x397/0xb10
[   65.637829]  __driver_attach+0x29f/0x2c0
[   65.638395]  ? driver_probe_device+0xb10/0xb10
[   65.639034]  bus_for_each_dev+0xe4/0x170
[   65.639631]  driver_attach+0x2c/0x40
[   65.641178]  ? driver_probe_device+0xb10/0xb10
[   65.641715]  bus_add_driver+0x2fd/0x620
[   65.642175]  driver_register+0x131/0x380
[   65.642640]  ? __raw_spin_lock_init+0x52/0x90
[   65.643353]  ? virtio_mmio_init+0x2d/0x2d
[   65.643842]  __pci_register_driver+0x66/0x80
[   65.644356]  virtio_pci_driver_init+0x22/0x32
[   65.644888]  do_one_initcall+0x12c/0x2de
[   65.645354]  ? kernel_init_freeable+0x23c/0x42b
[   65.645903]  kernel_init_freeable+0x26a/0x42b
[   65.646419]  ? rest_init+0x180/0x180
[   65.646897]  kernel_init+0x1f/0x360
[   65.647431]  ? schedule_tail_wrapper+0x9/0xc
[   65.648050]  ? rest_init+0x180/0x180
[   65.648587]  ret_from_fork+0x19/0x24
[   65.649232] Kernel Offset: disabled


                                                          # HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start v4.10 v4.9 --
git bisect  bad 9e3596b0c6539e28546ff7c72a06576627068353  # 14:25  B      3     7    0   0  kbuild: initramfs cleanup, set target from Kconfig
git bisect  bad 5e176d6973bdac04d9f298ca384c39f08eb084cb  # 14:54  B      1     2    0   0  Merge tag 'for-linus-4.10-ofs1' of git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux
git bisect  bad 098c30557a9a19827240aaadc137e4668157dc6b  # 15:10  B      1     1    1   1  Merge tag 'driver-core-4.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core
git bisect  bad e71c3978d6f97659f6c3ee942c3e581299e4adf2  # 15:21  B      0     1   11   0  Merge branch 'smp-hotplug-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 92c020d08d83673ecd15a9069d4457378668da31  # 15:50  G     17     0   17  17  Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect  bad ef486c599a1f20c465ea196cd2019474708fbe52  # 16:04  B      1     4    1   1  Merge branch 'x86-cleanups-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 4ade5b2268b9ff05e48a9cb99689c4fd15fbe9c3  # 16:28  G     17     0   11  11  Merge branch 'x86-apic-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect  bad 06cc6b969c6375c65d0abbe4113af7949da29ab8  # 16:51  B      0     4   14   0  Merge branch 'x86-boot-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect  bad 5645688f9d0d5a32f030f9c5429e1a58bedca23b  # 17:02  B      1     9    1   1  Merge branch 'x86-asm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect  bad 0ee1dd9f5e7eae4e55f95935b72d4beecb03de9c  # 17:15  B      2    16    2   2  x86/dumpstack: Remove raw stack dump
git bisect good e728f61ce05404a26447e6bbc1885ca5956d6a44  # 17:40  G     18     0   18  18  x86/boot: Move the _stext marker to before the boot code
git bisect good 3b3fa11bc7000bb86c9fd30703da3689a9a9758d  # 18:11  G     18     0   18  18  x86/dumpstack: Print any pt_regs found on the stack
git bisect good 5e25d5bdf6d407224ad185a3fb8b870ad7d6c627  # 18:28  G     17     0   17  17  x86/entry64: Remove unused audit related macros
git bisect  bad bb5e5ce545f2031c96f7901cd8d1698ea3ca4c9c  # 18:40  B      0     7   17   0  x86/dumpstack: Remove kernel text addresses from stack dump
git bisect good efdb4167e676aaba7505bec739785b76e206cb45  # 18:57  G     17     0   17  17  scripts/faddr2line: Fix "size mismatch" error
# first bad commit: [bb5e5ce545f2031c96f7901cd8d1698ea3ca4c9c] x86/dumpstack: Remove kernel text addresses from stack dump
git bisect good efdb4167e676aaba7505bec739785b76e206cb45  # 19:20  G     52     0   52  69  scripts/faddr2line: Fix "size mismatch" error
# extra tests with CONFIG_DEBUG_INFO_REDUCED
git bisect good bb5e5ce545f2031c96f7901cd8d1698ea3ca4c9c  # 19:35  G     18     0   18  18  x86/dumpstack: Remove kernel text addresses from stack dump
# extra tests on HEAD of linux-devel/devel-spot-201703121227
git bisect  bad 82e79ed967147fd85df0025b2364a920c2b32ef8  # 19:35  B      2     9    0   3  0day head guard for 'devel-spot-201703121227'
# extra tests on tree/branch linus/master
git bisect  bad 106e4da60209b508894956b6adf4688f84c1766d  # 20:13  B      4     3    0   0  Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
# extra tests with first bad commit reverted
git bisect good ff74cea317355ada8f85574b09e78b4aa8c8fa22  # 20:29  G     18     0   10  10  Revert "x86/dumpstack: Remove kernel text addresses from stack dump"
# extra tests on tree/branch linux-next/master
git bisect  bad 5be4921c9958ec02a67506bd6f7a52fce663c201  # 20:46  B      2    11    2   2  Add linux-next specific files for 20170310

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/lkp                          Intel Corporation

Download attachment "dmesg-yocto-kbuild-58:20170312183949:i386-randconfig-c0-03121244:4.9.0-rc1-00121-gbb5e5ce5:1.gz" of type "application/gzip" (11591 bytes)

Download attachment "dmesg-yocto-kbuild-11:20170312185919:i386-randconfig-c0-03121244:4.9.0-rc1-00120-gefdb416:1.gz" of type "application/gzip" (11764 bytes)

View attachment "reproduce-yocto-kbuild-58:20170312183949:i386-randconfig-c0-03121244:4.9.0-rc1-00121-gbb5e5ce5:1" of type "text/plain" (901 bytes)

View attachment "config-4.9.0-rc1-00121-gbb5e5ce5" of type "text/plain" (98237 bytes)

Powered by blists - more mailing lists