Date:	Sun, 09 Feb 2014 17:38:55 +0100
From:	Toralf Förster <toralf.foerster@....de>
To:	Dave Hansen <dave.hansen@...el.com>,
	Linux Kernel <linux-kernel@...r.kernel.org>
CC:	kernel@...too.org, kvm@...r.kernel.org,
	Gleb Natapov <gleb@...nel.org>,
	Paolo Bonzini <pbonzini@...hat.com>,
	Peter Zijlstra <peterz@...radead.org>
Subject: Re: 3.13 hangs when I tried to start a KVM at a 32 bit stable Gentoo

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 01/23/2014 08:33 PM, Dave Hansen wrote:
> On 01/23/2014 10:55 AM, Dave Hansen wrote:
>> On 01/21/2014 08:38 AM, Toralf Förster wrote:
>>> Jan 21 17:18:57 n22 kernel: INFO: rcu_sched self-detected stall on CPU { 2}  (t=60001 jiffies g=18494 c=18493 q=183951)
>>> Jan 21 17:18:57 n22 kernel: sending NMI to all CPUs:
>>> Jan 21 17:18:57 n22 kernel: NMI backtrace for cpu 2
>>> Jan 21 17:18:57 n22 kernel: CPU: 2 PID: 6779 Comm: qemu-system-x86 Not tainted 3.13.0 #3
>>> Jan 21 17:18:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013
>>> Jan 21 17:18:57 n22 kernel: task: e921c370 ti: e5f36000 task.ti: e5f36000
>>
>> I'm seeing a very similar hang with an ubuntu guest and a custom kernel.
>> I'm on commit 0dc3fd0249a, and it's 100% reproducible every time I run KVM.
> 
> Did a little more LKML digging and found this:
> 
> 	http://marc.info/?l=linux-kernel&m=139038631607917&q=raw
> 
> Peter's fix works for me.  I'm also running a CONFIG_PREEMPT_VOLUNTARY=y
> config.
> 
> 
Hhhm, that fix seems not to be applicable for 3.13.x
With 3.13.2 I removed multimedia, sound and wlan and few file systems from the kernel's config, now at least the kernel doesn't hang, but I do still get while trying to start a virtual machine (and please note, I run into a similar issue while just umounting a btrfs file system) :


Feb  9 17:14:59 n22 polkitd[4129]: Operator of unix-session:/org/freedesktop/ConsoleKit/Session1 successfully authenticated as unix-user:root to gain TEMPORARY authorization for action org.libvirt.unix.manage for unix-process:6544:23328 [/usr/bin/python2.7 /usr/share/virt-manager/virt-manager] (owned by unix-user:tfoerste)
Feb  9 17:15:01 n22 kernel: type=1006 audit(1391962501.815:5): pid=6601 uid=0 old auid=4294967295 new auid=0 old ses=4294967295 new ses=4 res=1
Feb  9 17:15:01 n22 crond[6601]: pam_unix(crond:session): session opened for user root by (uid=0)
Feb  9 17:15:01 n22 CROND[6602]: (root) CMD (/usr/lib/sa/sa1 60 15 )
Feb  9 17:15:05 n22 kernel: NET: Registered protocol family 17
Feb  9 17:15:05 n22 kernel: device vnet0 entered promiscuous mode
Feb  9 17:15:05 n22 kernel: br0: port 4(vnet0) entered forwarding state
Feb  9 17:15:05 n22 kernel: br0: port 4(vnet0) entered forwarding state
Feb  9 17:15:05 n22 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): br0: link becomes ready
Feb  9 17:15:05 n22 kernel: cgroup: libvirtd (5677) created nested cgroup for controller "memory" which has incomplete hierarchy support. Nested cgroups may change behavior in the future.
Feb  9 17:15:05 n22 kernel: cgroup: "memory" requires setting use_hierarchy to 1 on the root.
Feb  9 17:15:08 n22 ntpd[5582]: Listen normally on 5 br0 fe80::acb9:fbff:fea0:57f9 UDP 123
Feb  9 17:15:08 n22 ntpd[5582]: Listen normally on 6 vnet0 fe80::fc54:ff:fe46:1fc4 UDP 123
Feb  9 17:15:08 n22 ntpd[5582]: peers refreshed
Feb  9 17:16:57 n22 kernel: INFO: rcu_sched self-detected stall on CPU { 0}  (t=60000 jiffies g=16324 c=16323 q=11304)
Feb  9 17:16:57 n22 kernel: sending NMI to all CPUs:
Feb  9 17:16:57 n22 kernel: NMI backtrace for cpu 0
Feb  9 17:16:57 n22 kernel: CPU: 0 PID: 6617 Comm: qemu-system-x86 Not tainted 3.13.2 #17
Feb  9 17:16:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013
Feb  9 17:16:57 n22 kernel: task: e70e5b40 ti: e736e000 task.ti: e736e000
Feb  9 17:16:57 n22 kernel: EIP: 0060:[<c12ac9cd>] EFLAGS: 00000006 CPU: 0
Feb  9 17:16:57 n22 kernel: EIP is at __const_udelay+0xd/0x20
Feb  9 17:16:57 n22 kernel: EAX: 01062560 EBX: 00002710 ECX: c1599460 EDX: 00278bf9
Feb  9 17:16:57 n22 kernel: ESI: c159c580 EDI: f3650c40 EBP: e736fca8 ESP: e736fca8
Feb  9 17:16:57 n22 kernel: DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
Feb  9 17:16:57 n22 kernel: CR0: 80050033 CR2: 091441f4 CR3: 00110000 CR4: 000427f0
Feb  9 17:16:57 n22 kernel: Stack:
Feb  9 17:16:57 n22 kernel: e736fcb8 c102ba30 c1500d39 c159c580 e736fcfc c107febc c150cd0c 0000ea60
Feb  9 17:16:57 n22 kernel: 00003fc4 00003fc3 00002c28 c10606d8 00000001 c159c580 c15c4a84 f3650c40
Feb  9 17:16:57 n22 kernel: 00002c28 00000000 e70e5b40 00000000 00000000 e736fd10 c1044126 f3650e60
Feb  9 17:16:57 n22 kernel: Call Trace:
Feb  9 17:16:57 n22 kernel: [<c102ba30>] arch_trigger_all_cpu_backtrace+0x50/0x70
Feb  9 17:16:57 n22 kernel: [<c107febc>] rcu_check_callbacks+0x31c/0x520
Feb  9 17:16:57 n22 kernel: [<c10606d8>] ? account_system_time+0xb8/0x170
Feb  9 17:16:57 n22 kernel: [<c1044126>] update_process_times+0x36/0x70
Feb  9 17:16:57 n22 kernel: [<c108985e>] tick_sched_handle.isra.13+0x2e/0x30
Feb  9 17:16:57 n22 kernel: [<c10899fb>] tick_sched_timer+0x3b/0x70
Feb  9 17:16:57 n22 kernel: [<c1055f52>] ? __remove_hrtimer+0x42/0xa0
Feb  9 17:16:57 n22 kernel: [<c105603c>] __run_hrtimer.isra.29+0x3c/0xb0
Feb  9 17:16:57 n22 kernel: [<c1056c75>] hrtimer_interrupt+0x1e5/0x270
Feb  9 17:16:57 n22 kernel: [<c10298ca>] local_apic_timer_interrupt+0x2a/0x50
Feb  9 17:16:57 n22 kernel: [<c103e420>] ? irq_enter+0x10/0x60
Feb  9 17:16:57 n22 kernel: [<c1029e0e>] smp_apic_timer_interrupt+0x2e/0x50
Feb  9 17:16:57 n22 kernel: [<c144ece4>] apic_timer_interrupt+0x34/0x3c
Feb  9 17:16:57 n22 kernel: [<c107d22b>] ? __srcu_read_unlock+0xb/0x20
Feb  9 17:16:57 n22 kernel: [<f88cae70>] kvm_arch_vcpu_ioctl_run+0x8c0/0xf30 [kvm]
Feb  9 17:16:57 n22 kernel: [<f88ba84b>] kvm_vcpu_ioctl+0x40b/0x4a0 [kvm]
Feb  9 17:16:57 n22 kernel: [<c10653ac>] ? put_prev_task_fair+0x5c/0x3e0
Feb  9 17:16:57 n22 kernel: [<c105bd63>] ? ttwu_do_activate.constprop.82+0x53/0x60
Feb  9 17:16:57 n22 kernel: [<c1061100>] ? __dequeue_entity+0x20/0x40
Feb  9 17:16:57 n22 kernel: [<f88ba440>] ? vcpu_put+0x20/0x20 [kvm]
Feb  9 17:16:57 n22 kernel: [<c110d34a>] do_vfs_ioctl+0x6a/0x540
Feb  9 17:16:57 n22 kernel: [<c144bb1e>] ? __schedule+0x20e/0x660
Feb  9 17:16:57 n22 kernel: [<c110d860>] SyS_ioctl+0x40/0x70
Feb  9 17:16:57 n22 kernel: [<c144f201>] sysenter_do_call+0x12/0x22
Feb  9 17:16:57 n22 kernel: [<c1440000>] ? dns_resolver_instantiate+0x400/0x480
Feb  9 17:16:57 n22 kernel: Code: fd 48 5d c3 8d 76 00 8d bc 27 00 00 00 00 55 89 e5 ff 15 6c ea 5a c1 5d c3 90 8d 74 26 00 55 c1 e0 02 89 e5 64 8b 15 5c 7d 62 c1 <69> d2 fa 00 00 00 f7 e2 8d 42 01 ff 15 6c ea 5a c1 5d c3 55 69
Feb  9 17:16:57 n22 kernel: NMI backtrace for cpu 1
Feb  9 17:16:57 n22 kernel: CPU: 1 PID: 5818 Comm: period_search_1 Not tainted 3.13.2 #17
Feb  9 17:16:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013
Feb  9 17:16:57 n22 kernel: task: f00716d0 ti: ef3be000 task.ti: ef3be000
Feb  9 17:16:57 n22 kernel: EIP: 0073:[<08050915>] EFLAGS: 00000246 CPU: 1
Feb  9 17:16:57 n22 kernel: EIP is at 0x8050915
Feb  9 17:16:57 n22 kernel: EAX: 000000ac EBX: 0000004d ECX: 00000120 EDX: 0000000e
Feb  9 17:16:57 n22 kernel: ESI: 00000980 EDI: 0a846fc8 EBP: bfcd34b8 ESP: bfcc9dc0
Feb  9 17:16:57 n22 kernel: DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
Feb  9 17:16:57 n22 kernel:
Feb  9 17:16:57 n22 kernel: NMI backtrace for cpu 2
Feb  9 17:16:57 n22 kernel: CPU: 2 PID: 5817 Comm: period_search_1 Not tainted 3.13.2 #17
Feb  9 17:16:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013
Feb  9 17:16:57 n22 kernel: task: f0075fd0 ti: f12ca000 task.ti: f12ca000
Feb  9 17:16:57 n22 kernel: EIP: 0073:[<08050d94>] EFLAGS: 00000287 CPU: 2
Feb  9 17:16:57 n22 kernel: EIP is at 0x8050d94
Feb  9 17:16:57 n22 kernel: EAX: 00000128 EBX: 0858a720 ECX: bfcb42e0 EDX: 00000180
Feb  9 17:16:57 n22 kernel: ESI: bfcb3320 EDI: 000001f8 EBP: bfcbc058 ESP: bfcb2960
Feb  9 17:16:57 n22 kernel: DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
Feb  9 17:16:57 n22 kernel:
Feb  9 17:16:57 n22 kernel: NMI backtrace for cpu 3
Feb  9 17:16:57 n22 kernel: CPU: 3 PID: 5814 Comm: einstein_S6CasA Not tainted 3.13.2 #17
Feb  9 17:16:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013
Feb  9 17:16:57 n22 kernel: task: f0073b50 ti: ef722000 task.ti: ef722000
Feb  9 17:16:57 n22 kernel: EIP: 0073:[<080558e1>] EFLAGS: 00000202 CPU: 3
Feb  9 17:16:57 n22 kernel: EIP is at 0x80558e1
Feb  9 17:16:57 n22 kernel: EAX: b3214120 EBX: bfc8d160 ECX: b4589b98 EDX: 00010450
Feb  9 17:16:57 n22 kernel: ESI: 00010450 EDI: 00016cf1 EBP: bfc8d2f8 ESP: bfc8caa0
Feb  9 17:16:57 n22 kernel: DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
Feb  9 17:16:57 n22 kernel:
Feb  9 17:17:53 n22 kernel: br0: port 4(vnet0) entered disabled state
Feb  9 17:17:53 n22 kernel: device vnet0 left promiscuous mode
Feb  9 17:17:53 n22 kernel: br0: port 4(vnet0) entered disabled state
Feb  9 17:17:54 n22 ntpd[5582]: Deleting interface #6 vnet0, fe80::fc54:ff:fe46:1fc4#123, interface stats: received=0, sent=0, dropped=0, active_time=166 secs
Feb  9 17:17:54 n22 ntpd[5582]: peers refreshed
Feb  9 17:18:25 n22 polkitd[4129]: Unregistered Authentication Agent for unix-session:/org/freedesktop/ConsoleKit/Session1 (system bus name :1.18, object path /org/kde/PolicyKit1/AuthenticationAgent, locale en_US.utf8)
Feb  9 17:18:26 n22 kdm: :0[4888]: pam_unix(kde:session): session closed for user tfoerste
Feb  9 17:18:26 n22 shutdown[7138]: shutting down for system reboot
Feb  9 17:18:27 n22 init: Switching to runlevel: 6
Feb  9 17:18:28 n22 thinkfan:
Caught deadly signal.
F



- -- 
MfG/Sincerely
Toralf Förster
pgp finger print:1A37 6F99 4A9D 026F 13E2 4DCF C4EA CDDE 0076 E94E
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlL3rx4ACgkQxOrN3gB26U4uDwD+MofngLtD/et921CrLyVrAFr+
1QkIVk0cyUFIAp4986wA/2d5zZ1jRj61uXnGidDcxCq2AyIl85Iqy0KkNkDGQ/FM
=W0Ci
-----END PGP SIGNATURE-----

View attachment ".config" of type "text/plain" (71263 bytes)

Download attachment ".config.sig" of type "application/pgp-signature" (96 bytes)

Powered by blists - more mailing lists