Message-ID: <352443968.11059.1479836663353.JavaMail.zimbra@efficios.com>
Date:   Tue, 22 Nov 2016 17:44:23 +0000 (UTC)
From:   Mathieu Desnoyers <mathieu.desnoyers@...icios.com>
To:     Eric Dumazet <edumazet@...gle.com>,
        Alexander Duyck <alexander.h.duyck@...el.com>,
        Willem de Bruijn <willemb@...gle.com>,
        Alexei Starovoitov <ast@...nel.org>
Cc:     "David S. Miller" <davem@...emloft.net>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Francis Deslauriers <francis.deslauriers@...icios.com>,
        linux-kernel <linux-kernel@...r.kernel.org>
Subject: Fwd: Kernel BUG_ON in stable 4.8

----- On Nov 22, 2016, at 12:01 PM, Francis Deslauriers francis.deslauriers@...icios.com wrote:

> Hi Mathieu,
> 
> Here is a description of the kernel BUG_ON I have encountered. This bug was
> triggered by our continuous integration system tracking the stable branches. I
> was only able to reproduce it on our Lava baremetal x86_64 worker. Running the
> same kernel image on our KVM worker did not trigger the bug.
> 
> The bug occurs at boot time and I believe it's during the configuration of one
> of the network cards.
> 
> See attached the .config used.
> See attached the dmesg output of the crash containing the kernel panic
> output(around line 885 of dmesg.txt)

Hi guys,

Upstream commit 34fad54c2 "net: __skb_flow_dissect() must cap its return value"
triggers this BUG_ON at boot up on our testing machines. We have observed the
BUG_ON on v4.9-rc6, as well as stable kernels v4.8.10 and v4.4.34.

Relevant bits:

[   16.841793] kernel BUG at ./include/linux/skbuff.h:1927!
[   16.847101] invalid opcode: 0000 [#1] SMP
[   16.851101] Modules linked in:
[   16.854172] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.8.10 #1
[   16.860081] Hardware name: Supermicro SYS-6018R-TDW/X10DDW-i, BIOS 1.0a 01/14/2015
[   16.867637] task: ffffffff8220d500 task.stack: ffffffff82200000
[   16.873550] RIP: 0010:[<ffffffff819a7239>]  [<ffffffff819a7239>] eth_type_trans+0xc9/0x110
[   16.881826] RSP: 0000:ffff88103fc03df0  EFLAGS: 00010297
[   16.887130] RAX: 0000000000000148 RBX: ffff881037b2b5c0 RCX: 0000000000001073
[   16.894254] RDX: ffff88103221fdc0 RSI: ffff881038358000 RDI: ffff881037c5bb00
[   16.901377] RBP: ffff88103fc03df8 R08: 0000000000000001 R09: 0000000000000800
[   16.908503] R10: ffff88103221fec0 R11: ffffea0040de7780 R12: ffff881037c5bb00
[   16.915625] R13: ffff881037458000 R14: 0000000000000156 R15: ffff881038358000
[   16.922751] FS:  0000000000000000(0000) GS:ffff88103fc00000(0000) knlGS:0000000000000000
[   16.930829] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   16.936565] CR2: ffff88207ffff000 CR3: 0000000002206000 CR4: 00000000001406f0
[   16.943689] Stack:
[   16.945702]  ffffffff815c098f ffff88103fc03e90 ffffffff8181c651 ffff88103fc0dc20
[   16.953153]  0000000000000000 0000000000000020 0000000000000002 ffff88103fc03e38
[   16.960607]  ffffea0040de7780 ffffea0000000008 ffffffff00000001 ffffea0040de7780
[   16.968061] Call Trace:
[   16.970507]  <IRQ> 
[   16.972432]  [<ffffffff815c098f>] ? swiotlb_sync_single_for_device+0xf/0x20
[   16.979575]  [<ffffffff8181c651>] igb_poll+0x691/0xe60
[   16.984714]  [<ffffffff8198383b>] net_rx_action+0x1bb/0x2f0
[   16.990291]  [<ffffffff81afcad6>] __do_softirq+0xf6/0x280
[   16.995689]  [<ffffffff810f82dc>] irq_exit+0xdc/0xf0
[   17.000648]  [<ffffffff81afc834>] do_IRQ+0x54/0xd0
[   17.005433]  [<ffffffff81afa80c>] common_interrupt+0x8c/0x8c
[   17.011083]  <EOI> 
[   17.013014]  [<ffffffff8108c866>] ? mwait_idle+0x76/0x170
[   17.018596]  [<ffffffff8108cdef>] arch_cpu_idle+0xf/0x20
[   17.023907]  [<ffffffff8113524a>] default_idle_call+0x2a/0x40
[   17.029648]  [<ffffffff811354fa>] cpu_startup_entry+0x29a/0x300
[   17.035561]  [<ffffffff81aed057>] rest_init+0x77/0x80
[   17.040608]  [<ffffffff823ecf46>] start_kernel+0x40b/0x418
[   17.046091]  [<ffffffff823ec120>] ? early_idt_handler_array+0x120/0x120
[   17.052702]  [<ffffffff823ec2d6>] x86_64_start_reservations+0x2a/0x2c
[   17.059132]  [<ffffffff823ec413>] x86_64_start_kernel+0x13b/0x14a
[   17.065215] Code: 00 04 00 00 c9 c3 48 33 86 58 03 00 00 48 c1 e0 10 48 85 c0 0f b6 87 80 00 00 00 75 25 83 e0 f8 83 c8 01 88 87 80 00 00 00 eb 9c <0f> 0b 0f b6 87 80 00 00 00 83 e0 f8 83 c8 03 88 87 80 00 00 00 
[   17.085165] RIP  [<ffffffff819a7239>] eth_type_trans+0xc9/0x110
[   17.091094]  RSP <ffff88103fc03df0>
[   17.094586] ---[ end trace e233c88f3b369632 ]---
[   17.101432] Kernel panic - not syncing: Fatal exception in interrupt
[   17.107790] Kernel Offset: disabled
[   17.113484] ---[ end Kernel panic - not syncing: Fatal exception in interrupt

Do you have clues on what is going on ?

Thanks,

Mathieu


> 
> I was able to bisect the issue to this commit in the stable branch linux-4.8.y
> [1] . This commit was also backported to the 4.4 stable branch.
> 
> Thank you,
> 
> --
> Francis Deslauriers
> Software developer
> EfficiOS inc.
> 
> 
> [1]
> https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/patch/?id=bccb4093d464ca8b33a096999c4bb317f84ad652

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

View attachment "dmesg.txt" of type "text/plain" (69129 bytes)

View attachment ".config" of type "text/x-mpsub" (126920 bytes)

Powered by blists - more mailing lists