| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <1303878240.2699.41.camel@edumazet-laptop> Date: Wed, 27 Apr 2011 06:24:00 +0200 From: Eric Dumazet <eric.dumazet@...il.com> To: Maximilian Engelhardt <maxi@...monizer.de> Cc: linux-kernel@...r.kernel.org, netdev@...r.kernel.org, StuStaNet Vorstand <vorstand@...sta.mhn.de> Subject: Re: Kernel crash after using new Intel NIC (igb) Le lundi 25 avril 2011 à 00:32 +0200, Maximilian Engelhardt a écrit : > Hello, > > some time ago we switched some of our servers to a new networking card that > uses the Intel igb driver. Since that time we see regular kernel crashes. > The crashes happen at very irregular intervals, sometimes after a week uptime, > sometimes after a month or even more. They seem to be independent of the > server load as they also happen in the night when there is low traffic. > > The affected server is used as a NAT device with some iptables rules and serves > about 2000 people. > > Attached are two logs of the crashes as well as the output of dmesg, lspci, > and /proc/interrupts as well as the used kernel config. > > I have no idea what might be wrong but I think it is a kernel bug. Perhaps > someone with more knowledge has a clue. > > If needed I can provide additional information or build different kernels. > > Greetings, > Maxi Hello Maximilian We had similar reports in the past that disappeared when adding "slab_nomerge" to boot parameters. We suspect a memory corruption from another part of kernel on 64bytes kmemcache objects. In 2.6.37, inetpeer code uses 64bytes objects. Using slab_nomerge and SLUB allocator (as you already do), makes sure inetpeer kmemcache wont be shared by other 64bytes objects in kernel. In 2.6.38 and up, inetpeer objects are now larger, so you also could try latest linux-2.6 tree, just to make sure inetpeer code is not faulty. Thanks BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 IP: [<ffffffff8145ea9f>] cleanup_once+0x3f/0xa0 PGD 12d82a067 PUD 12ea49067 PMD 0 Oops: 0002 [#1] PREEMPT SMP last sysfs file: /sys/devices/virtual/vc/vcsa5/uevent CPU 0 Pid: 0, comm: swapper Not tainted 2.6.37.1 #1 Supermicro X7SB4/E/X7SB4/E RIP: 0010:[<ffffffff8145ea9f>] [<ffffffff8145ea9f>] cleanup_once+0x3f/0xa0 RSP: 0018:ffff8800cfc03e40 EFLAGS: 00010202 RAX: ffff880128167798 RBX: ffff880128167780 RCX: 0000000000000000 RDX: c398112e00026cf7 RSI: 00000000000001a2 RDI: ffffffff8166ce10 RBP: 0000000000024702 R08: 00000000003d0900 R09: 00040ea8ea5b7700 R10: ffffffff814f312d R11: 0000000000000010 R12: ffffffff8161ffd8 R13: 0000000000000102 R14: ffffffff8174b4e0 R15: ffffffff8161ffd8 FS: 0000000000000000(0000) GS:ffff8800cfc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000008 CR3: 000000012fe67000 CR4: 00000000000406f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper (pid: 0, threadinfo ffffffff8161e000, task ffffffff81638020) Stack: ffff8800cfc11f00 0000000111034f87 0000000000024702 ffffffff8145ed68 ffffffff8174a4c0 ffffffff8174a4c0 ffff8800cfc03eb0 ffffffff81044cb8 ffffffff81034079 ffffffff8145ed30 0000000000000000 ffffffff8174b8e0 Call Trace: <IRQ> [<ffffffff8145ed68>] ? peer_check_expire+0x38/0x110 [<ffffffff81044cb8>] ? run_timer_softirq+0x138/0x250 [<ffffffff81034079>] ? scheduler_tick+0xd9/0x2e0 [<ffffffff8145ed30>] ? peer_check_expire+0x0/0x110 [<ffffffff8103eb0d>] ? __do_softirq+0x9d/0x130 [<ffffffff8100320c>] ? call_softirq+0x1c/0x30 [<ffffffff8100531d>] ? do_softirq+0x4d/0x80 [<ffffffff8103e9cd>] ? irq_exit+0x8d/0x90 [<ffffffff8101d5ea>] ? smp_apic_timer_interrupt+0x6a/0xa0 [<ffffffff81002cd3>] ? apic_timer_interrupt+0x13/0x20 <EOI> [<ffffffff8100a93a>] ? mwait_idle+0x6a/0x80 [<ffffffff81001528>] ? cpu_idle+0x58/0xb0 [<ffffffff81698dd3>] ? start_kernel+0x334/0x33f [<ffffffff8169840d>] ? x86_64_start_kernel+0xf3/0xf7 Code: 00 48 8b 05 84 e3 20 00 48 3d 00 ce 66 81 74 5c 48 8d 58 e8 48 8b 15 31 5e 22 00 2b 53 28 48 39 ea 72 49 48 8b 4b 18 48 8b 53 20 <48> 89 51 08 48 89 0a 48 89 43 18 48 89 43 20 f0 ff 40 14 48 c7 RIP [<ffffffff8145ea9f>] cleanup_once+0x3f/0xa0 RSP <ffff8800cfc03e40> CR2: 0000000000000008 ---[ end trace 904f16191de0663c ]--- Kernel panic - not syncing: Fatal exception in interrupt Pid: 0, comm: swapper Tainted: G D 2.6.37.1 #1 Call Trace: <IRQ> [<ffffffff814e4152>] ? panic+0xa1/0x19e [<ffffffff810068eb>] ? oops_end+0x9b/0xa0 [<ffffffff81024523>] ? no_context+0x103/0x270 [<ffffffff81024d10>] ? do_page_fault+0x290/0x430 [<ffffffff813eabd2>] ? __alloc_skb+0x72/0x160 [<ffffffff81262f40>] ? swiotlb_dma_mapping_error+0x10/0x20 [<ffffffff8133e168>] ? igb_alloc_rx_buffers_adv+0x208/0x3a0 [<ffffffff814e780f>] ? page_fault+0x1f/0x30 [<ffffffff8145ea9f>] ? cleanup_once+0x3f/0xa0 [<ffffffff8145ed68>] ? peer_check_expire+0x38/0x110 [<ffffffff81044cb8>] ? run_timer_softirq+0x138/0x250 [<ffffffff81034079>] ? scheduler_tick+0xd9/0x2e0 [<ffffffff8145ed30>] ? peer_check_expire+0x0/0x110 [<ffffffff8103eb0d>] ? __do_softirq+0x9d/0x130 [<ffffffff8100320c>] ? call_softirq+0x1c/0x30 [<ffffffff8100531d>] ? do_softirq+0x4d/0x80 [<ffffffff8103e9cd>] ? irq_exit+0x8d/0x90 [<ffffffff8101d5ea>] ? smp_apic_timer_interrupt+0x6a/0xa0 [<ffffffff81002cd3>] ? apic_timer_interrupt+0x13/0x20 <EOI> [<ffffffff8100a93a>] ? mwait_idle+0x6a/0x80 [<ffffffff81001528>] ? cpu_idle+0x58/0xb0 [<ffffffff81698dd3>] ? start_kernel+0x334/0x33f [<ffffffff8169840d>] ? x86_64_start_kernel+0xf3/0xf7 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists