Message-ID: <20070923015217.GA5956@mail.ustc.edu.cn>
Date:	Sun, 23 Sep 2007 09:52:17 +0800
From:	Fengguang Wu <wfg@...l.ustc.edu.cn>
To:	Andrew Morton <akpm@...ux-foundation.org>
Cc:	linux-kernel@...r.kernel.org, Miklos Szeredi <miklos@...redi.hu>
Subject: Re: [BUG 2.6.23-rc6-mm1] NMI Watchdog detected LOCKUP on CPU 0

On Sun, Sep 23, 2007 at 09:42:14AM +0800, Fengguang Wu wrote:
> On Tue, Sep 18, 2007 at 01:18:41AM -0700, Andrew Morton wrote:
> > 
> > ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.23-rc6/2.6.23-rc6-mm1/
> > 
> > 2.6.23-rc6-mm1 is a 29MB diff against 2.6.23-rc6.
> 
> 
> This bug appears in 2.6.23-rc3-mm1, too.
> 
> The message:
> 
> [ 3267.844826] NMI Watchdog detected LOCKUP on CPU 0
> [ 3267.849515] CPU 0 
> [ 3267.851525] Modules linked in: binfmt_misc ipt_MASQUERADE iptable_mangle iptable_nat nf_conntrack_ipv4 iptable_filter ip_tables x_tables nf_nat_tftp nf_nat_ftp nf_nat nf_conntrack_tftp nf_conntrack_ftp nf_conntrack nfnetlink fan ac battery ipv6 eeprom lm85 hwmon_vid i2c_core tun fuse kvm snd_hda_intel snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd sg soundcore snd_page_alloc thermal sr_mod pcspkr evdev button processor cdrom
> [ 3267.889547] Pid: 13507, comm: gcc Not tainted 2.6.23-rc6-mm1 #4
> [ 3267.895442] RIP: 0033:[<00002ab84e34cd44>]  [<00002ab84e34cd44>]
> [ 3267.901438] RSP: 002b:00007fff5c9e03f8  EFLAGS: 00000287
> [ 3267.906726] RAX: 0000000000000000 RBX: 00007fff5c9e0580 RCX: 0000000000000000
> [ 3267.913833] RDX: 0000000000000013 RSI: 00007fff5c9e0680 RDI: 00000000012a7010
> [ 3267.920939] RBP: 00007fff5c9e0550 R08: 0000000000000050 R09: 0000000000000000
> [ 3267.928045] R10: 0000000000000000 R11: 00000000012a7410 R12: 0000000000000002
> [ 3267.935151] R13: 0000000000000003 R14: 0000000000000005 R15: 000000000000001f
> [ 3267.942258] FS:  00002ab84f144170(0000) GS:ffffffff814f3000(0000) knlGS:0000000000000000
> [ 3267.950317] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 3267.956038] CR2: 00002ab84e3a7430 CR3: 000000000d618000 CR4: 00000000000006e0
> [ 3267.963144] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 3267.970250] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 3267.977357] Process gcc (pid: 13507, threadinfo ffff81000ebe6000, task ffff810008b849d0)
> [ 3267.985416] 
> [ 3267.997480] Unable to handle kernel paging request at 00000000fffffffe RIP: 
> [ 3268.002082]  [<00000000fffffffe>]
> [ 3268.007827] PGD ea85067 PUD 0 
> [ 3268.010887] Oops: 0010 [1] SMP 
> [ 3268.014035] last sysfs file: /devices/pci0000:00/0000:00:1e.0/0000:05:04.0/resource
> [ 3268.021662] CPU 0 
> [ 3268.023674] Modules linked in: binfmt_misc ipt_MASQUERADE iptable_mangle iptable_nat nf_conntrack_ipv4 iptable_filter ip_tables x_tables nf_nat_tftp nf_nat_ftp nf_nat nf_conntrack_tftp nf_conntrack_ftp nf_conntrack nfnetlink fan ac battery ipv6 eeprom lm85 hwmon_vid i2c_core tun fuse kvm snd_hda_intel snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd sg soundcore snd_page_alloc thermal sr_mod pcspkr evdev button processor cdrom
> [ 3268.061688] Pid: 13507, comm: gcc Not tainted 2.6.23-rc6-mm1 #4
> [ 3268.067584] RIP: 0010:[<00000000fffffffe>]  [<00000000fffffffe>]
> [ 3268.073578] RSP: 0000:ffffffff8157ce38  EFLAGS: 00010296
> [ 3268.078867] RAX: 0000000000002710 RBX: ffff810009787050 RCX: ffff8100036788e0
> [ 3268.085973] RDX: 000000000000018d RSI: ffffffff810ba000 RDI: ffff810009787080
> [ 3268.093080] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
> [ 3268.100185] R10: 0000000000000000 R11: 0000000000000001 R12: ffff810008b849d0
> [ 3268.107293] R13: ffff810008b850d0 R14: 0000000000000001 R15: ffffffff8157cf58
> [ 3268.114399] FS:  00002ab84f144170(0000) GS:ffffffff814f3000(0000) knlGS:0000000000000000
> [ 3268.122455] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 3268.128178] CR2: 00000000fffffffe CR3: 0000000006bfd000 CR4: 00000000000006e0
> [ 3268.135283] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 3268.142388] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 3268.149495] Process gcc (pid: 13507, threadinfo ffff81000ebe6000, task ffff810008b849d0)
> [ 3268.157552] last branch before last exception/interrupt
> [ 3268.162753]  from  [<ffffffff811b1843>] serial_in+0x23/0x80
> [ 3268.168316]  to  [<ffffffff811b1832>] serial_in+0x12/0x80
> [ 3268.173701] Stack:  ffffffff8157ce78 ffffffff812e214f ffff81000ebe7fd8 0000000000000000
> [ 3268.181728]  0000000000000000 000000000ebe7f2d 000000000000003d ffffffff8157cf58
> [ 3268.189133]  ffffffff8157ce88 ffffffff812e218d ffffffff8157ce98 ffffffff812e21a1
> [ 3268.196358] Call Trace:
> [ 3268.198974] Inexact backtrace:
> [ 3268.202014]  <NMI>  [<ffffffff812e214f>] notifier_call_chain+0x3f/0x70
> [ 3268.208531]  [<ffffffff812e218d>] __atomic_notifier_call_chain+0xd/0x10
> [ 3268.215118]  [<ffffffff812e21a1>] atomic_notifier_call_chain+0x11/0x20
> [ 3268.221619]  [<ffffffff81056dde>] notify_die+0x2e/0x30
> [ 3268.226736]  [<ffffffff812e0b1c>] nmi_watchdog_tick+0x4c/0x1e0
> [ 3268.232545]  [<ffffffff812e0257>] default_do_nmi+0x67/0x1e0
> [ 3268.238093]  [<ffffffff812e0cdf>] do_nmi+0x2f/0x50
> [ 3268.242863]  [<ffffffff812dffef>] nmi+0x7f/0x90
> [ 3268.247377]  [<ffffffff81151c3e>] __delay+0xe/0x20
> [ 3268.252147]  <<EOE>> 
> [ 3268.254416] 
> [ 3268.254416] Code:  Bad RIP value.
> [ 3268.259216] RIP  [<00000000fffffffe>]
> [ 3268.262871]  RSP <ffffffff8157ce38>
> [ 3268.266342] CR2: 00000000fffffffe
> [ 3268.269821] Fixing recursive fault but reboot is needed!

Here are more messages on rebooting(I didn't mount any fuse fs though):

[ 3869.554219] slab error in verify_redzone_free(): cache `ext3_inode_cache': memory outside object was overwritten
[ 3869.564417] 
[ 3869.564418] Call Trace:
[ 3869.568422]  [<ffffffff8100dbc9>] dump_trace+0x3f9/0x4a0
[ 3869.573756]  [<ffffffff8100dcb3>] show_trace+0x43/0x60
[ 3869.578926]  [<ffffffff8100dce5>] dump_stack+0x15/0x20
[ 3869.584085]  [<ffffffff8109afa4>] __slab_error+0x24/0x30
[ 3869.589431]  [<ffffffff8109ba23>] cache_free_debugcheck+0x143/0x1e0
[ 3869.595716]  [<ffffffff8109dd60>] kfree+0xa0/0x240
[ 3869.600541]  [<ffffffff8114bb30>] kobject_cleanup+0x90/0xa0
[ 3869.606136]  [<ffffffff8114bb4d>] kobject_release+0xd/0x10
[ 3869.611654]  [<ffffffff8114cca3>] kref_put+0x43/0x80
[ 3869.616638]  [<ffffffff8114ba99>] kobject_put+0x19/0x20
[ 3869.621895]  [<ffffffff8114bcab>] kobject_unregister+0x2b/0x40
[ 3869.627745]  [<ffffffff8114bcd2>] kset_unregister+0x12/0x20
[ 3869.633349]  [<ffffffff8114bce9>] subsystem_unregister+0x9/0x10
[ 3869.639291]  [<ffffffff88122550>] :fuse:fuse_sysfs_cleanup+0x10/0x20
[ 3869.645669]  [<ffffffff88123b4c>] :fuse:fuse_exit+0x1c/0x2c
[ 3869.651265]  [<ffffffff81063bf1>] sys_delete_module+0x1b1/0x290
[ 3869.657202]  [<ffffffff8100c40e>] system_call+0x7e/0x83
[ 3869.662449]  [<00007f5c807c6ef7>]
[ 3869.665797] 
[ 3869.667317] ffffffff881245f3: redzone 1:0x786574756d5f74, redzone 2:0x0.
[ 3869.674050] ------------[ cut here ]------------
[ 3869.678688] kernel BUG at mm/slab.c:2901!
[ 3869.682717] invalid opcode: 0000 [2] SMP 
[ 3869.686812] last sysfs file: /devices/pci0000:00/0000:00:1e.0/0000:05:04.0/class
[ 3869.694231] CPU 0 
[ 3869.696302] Modules linked in: binfmt_misc ipt_MASQUERADE iptable_mangle iptable_nat nf_conntrack_ipv4 iptable_filter ip_tables x_tables nf_nat_tftp nf_nat_ftp nf_nat nf_conntrack_tftp nf_conntrack_ftp nf_conntrack nfnetlink fan ac battery ipv6 eeprom lm85 hwmon_vid i2c_core tun fuse kvm snd_hda_intel snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd sg soundcore snd_page_alloc thermal sr_mod pcspkr evdev button processor cdrom
[ 3869.735440] Pid: 5754, comm: rmmod Tainted: G      D 2.6.23-rc6-mm1 #4
[ 3869.741982] RIP: 0010:[<ffffffff8109ba9a>]  [<ffffffff8109ba9a>] cache_free_debugcheck+0x1ba/0x1e0
[ 3869.750994] RSP: 0018:ffff8100096cbd98  EFLAGS: 00010002
[ 3869.756324] RAX: 001f446745213ca4 RBX: ffff8100059ca440 RCX: ffff8100081240c0
[ 3869.763469] RDX: 00000000001f4467 RSI: ffffffff881245f3 RDI: ffff8100059ca440
[ 3869.770616] RBP: ffff8100096cbdc8 R08: 0000000000000000 R09: 0000000000000001
[ 3869.777762] R10: 0000000000000000 R11: ffffffff8130b040 R12: ffffffff881245f3
[ 3869.784909] R13: 00786574756d5f74 R14: 0000000000000000 R15: ffff810008124080
[ 3869.792057] FS:  00007f5c80c496e0(0000) GS:ffffffff814f3000(0000) knlGS:0000000000000000
[ 3869.800171] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3869.805932] CR2: 00007f5c807684e0 CR3: 000000000663b000 CR4: 00000000000006e0
[ 3869.813079] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 3869.820226] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 3869.827372] Process rmmod (pid: 5754, threadinfo ffff8100096ca000, task ffff810006478080)
[ 3869.835572] last branch before last exception/interrupt
[ 3869.840812]  from  [<ffffffff811b1843>] serial_in+0x23/0x80
[ 3869.846436]  to  [<ffffffff811b1832>] serial_in+0x12/0x80
[ 3869.851884] Stack:  ffffffff8114bb30 ffff8100059ca440 ffff8100059cbea8 ffffffff881245fb
[ 3869.860079]  ffffffff8812b310 0000000000000286 ffff8100096cbe18 ffffffff8109dd60
[ 3869.867634]  ffff8100096cbe18 ffffffff810f2516 ffff810007ce92d8 ffffffff8812b270
[ 3869.874968] Call Trace:
[ 3869.877660]  [<ffffffff8109dd60>] kfree+0xa0/0x240
[ 3869.882470]  [<ffffffff8114bb30>] kobject_cleanup+0x90/0xa0
[ 3869.888059]  [<ffffffff8114bb4d>] kobject_release+0xd/0x10
[ 3869.893561]  [<ffffffff8114cca3>] kref_put+0x43/0x80
[ 3869.898545]  [<ffffffff8114ba99>] kobject_put+0x19/0x20
[ 3869.903789]  [<ffffffff8114bcab>] kobject_unregister+0x2b/0x40
[ 3869.909638]  [<ffffffff8114bcd2>] kset_unregister+0x12/0x20
[ 3869.915226]  [<ffffffff8114bce9>] subsystem_unregister+0x9/0x10
[ 3869.921165]  [<ffffffff88122550>] :fuse:fuse_sysfs_cleanup+0x10/0x20
[ 3869.927539]  [<ffffffff88123b4c>] :fuse:fuse_exit+0x1c/0x2c
[ 3869.933133]  [<ffffffff81063bf1>] sys_delete_module+0x1b1/0x290
[ 3869.939069]  [<ffffffff8100c40e>] system_call+0x7e/0x83
[ 3869.944313]  [<00007f5c807c6ef7>]
[ 3869.947650] 
[ 3869.949170] 
[ 3869.949170] Code: 0f 0b eb fe 66 90 0f 0b eb fe 48 8b 52 10 e9 8b fe ff ff 0f 
[ 3869.958656] RIP  [<ffffffff8109ba9a>] cache_free_debugcheck+0x1ba/0x1e0
[ 3869.965317]  RSP <ffff8100096cbd98>
[ 3869.968831] BUG: sleeping function called from invalid context at kernel/rwsem.c:20
[ 3869.976526] in_atomic():0, irqs_disabled():1
[ 3869.980816] 
[ 3869.980817] Call Trace:
[ 3869.984808]  [<ffffffff8100dbc9>] dump_trace+0x3f9/0x4a0
[ 3869.990139]  [<ffffffff8100dcb3>] show_trace+0x43/0x60
[ 3869.995296]  [<ffffffff8100dce5>] dump_stack+0x15/0x20
[ 3870.000454]  [<ffffffff81031f51>] __might_sleep+0xc1/0xe0
[ 3870.005873]  [<ffffffff810561fd>] down_read+0x1d/0x30
[ 3870.010945]  [<ffffffff81067d25>] acct_collect+0x45/0x1d0
[ 3870.016361]  [<ffffffff8103f9f0>] do_exit+0x190/0xad0
[ 3870.021431]  [<ffffffff812e0161>] oops_end+0x91/0xa0
[ 3870.026414]  [<ffffffff8100e0fe>] die+0x5e/0x90
[ 3870.030965]  [<ffffffff812e0612>] do_trap+0x132/0x140
[ 3870.036035]  [<ffffffff8100e620>] do_invalid_op+0x90/0xb0
[ 3870.041452]  [<ffffffff812dfd4d>] error_exit+0x0/0x84
[ 3870.046523]  [<ffffffff8109ba9a>] cache_free_debugcheck+0x1ba/0x1e0
[ 3870.052804]  [<ffffffff8109dd60>] kfree+0xa0/0x240
[ 3870.057616]  [<ffffffff8114bb30>] kobject_cleanup+0x90/0xa0
[ 3870.063206]  [<ffffffff8114bb4d>] kobject_release+0xd/0x10
[ 3870.068708]  [<ffffffff8114cca3>] kref_put+0x43/0x80
[ 3870.073693]  [<ffffffff8114ba99>] kobject_put+0x19/0x20
[ 3870.078936]  [<ffffffff8114bcab>] kobject_unregister+0x2b/0x40
[ 3870.084786]  [<ffffffff8114bcd2>] kset_unregister+0x12/0x20
[ 3870.090375]  [<ffffffff8114bce9>] subsystem_unregister+0x9/0x10
[ 3870.096312]  [<ffffffff88122550>] :fuse:fuse_sysfs_cleanup+0x10/0x20
[ 3870.102687]  [<ffffffff88123b4c>] :fuse:fuse_exit+0x1c/0x2c
[ 3870.108279]  [<ffffffff81063bf1>] sys_delete_module+0x1b1/0x290
[ 3870.114216]  [<ffffffff8100c40e>] system_call+0x7e/0x83
[ 3870.119459]  [<00007f5c807c6ef7>]
[ 3870.122796] 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists