[<prev] [next>] [day] [month] [year] [list]
Message-ID: <02b2254f122c4790b5e3e333a9855fcf@AMSPEX02CL03.citrite.net>
Date: Fri, 23 Nov 2018 15:46:30 +0000
From: Nathanael Davison <nathanael.davison@...rix.com>
To: "netdev@...r.kernel.org" <netdev@...r.kernel.org>
CC: Andrew Cooper <Andrew.Cooper3@...rix.com>,
Paul Durrant <Paul.Durrant@...rix.com>
Subject: ixgbe driver hits null pointer in net/core/dev.c
Hi all,
Running Kernel 4.19.x on Xen in machines using intel ixgbe driver the driver crashes on startup with the kernel trace below.
It appears that the loop on line 2432 of net/core/dev.c iterates beyond the end of the dev_maps->attr_map array, resulting in dereferencing garbage. A workaround is to add the condition tci<nr_ids at lines 2434, 2436, and 2439; but this does not appear to deal with the root case of the problem.
Nov 23 12:49:47 localhost kernel: [ 63.471784] BUG: unable to handle kernel paging request at 0000001400000001
Nov 23 12:49:47 localhost kernel: [ 63.471804] PGD 0 P4D 0
Nov 23 12:49:47 localhost kernel: [ 63.471811] Oops: 0000 [#1] SMP NOPTI
Nov 23 12:49:47 localhost kernel: [ 63.471818] CPU: 1 PID: 1665 Comm: lldpad Not tainted 4.19.0+0.kernel #37
Nov 23 12:49:47 localhost kernel: [ 63.471827] Hardware name: Dell Inc. PowerEdge R310/0P229K, BIOS 1.0.0 02/11/2010
Nov 23 12:49:47 localhost kernel: [ 63.471842] RIP: e030:remove_xps_queue+0x17/0x80
Nov 23 12:49:47 localhost kernel: [ 63.471849] Code: ff 0f 0b e9 31 ff ff ff 90 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 85 ff 74 49 48 63 f6 48 8b 4c f7 10 48 85 c9 74 3c <44> 8b 11 45 8d 4a ff 44 89 c8 83 f8 ff 4c 63 c0 74 2d 49 83 c0 08
Nov 23 12:49:47 localhost kernel: [ 63.471870] RSP: e02b:ffffc90040ccf9e0 EFLAGS: 00010202
Nov 23 12:49:47 localhost kernel: [ 63.471878] RAX: 0000000000000000 RBX: ffff88000456d280 RCX: 0000001400000001
Nov 23 12:49:47 localhost kernel: [ 63.471921] RDX: 0000000000000000 RSI: 0000000000000006 RDI: ffff88000456d280
Nov 23 12:49:47 localhost kernel: [ 63.471930] RBP: 0000000000000001 R08: ffffffffffffffff R09: 0000000000000000
Nov 23 12:49:47 localhost kernel: [ 63.471940] R10: 0000000000000001 R11: 000000000000cd40 R12: 0000000000000000
Nov 23 12:49:47 localhost kernel: [ 63.471949] R13: 0000000000000006 R14: 0000000100000000 R15: 0000000000000002
Nov 23 12:49:47 localhost kernel: [ 63.471966] FS: 00007f77bf9a3740(0000) GS:ffff88002d840000(0000) knlGS:0000000000000000
Nov 23 12:49:47 localhost kernel: [ 63.471976] CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 23 12:49:47 localhost kernel: [ 63.471985] CR2: 0000001400000001 CR3: 000000002738c000 CR4: 0000000000002660
Nov 23 12:49:47 localhost kernel: [ 63.471999] Call Trace:
Nov 23 12:49:47 localhost kernel: [ 63.472008] __netif_set_xps_queue+0x735/0x910
Nov 23 12:49:47 localhost kernel: [ 63.472018] netif_set_xps_queue+0x28/0x40
Nov 23 12:49:47 localhost kernel: [ 63.472028] ixgbe_configure_tx_ring+0x16d/0x270 [ixgbe]
Nov 23 12:49:47 localhost kernel: [ 63.472039] ixgbe_configure_rx_ring+0x693/0x11f0 [ixgbe]
Nov 23 12:49:47 localhost kernel: [ 63.472049] ixgbe_open+0x21b/0x7b0 [ixgbe]
Nov 23 12:49:47 localhost kernel: [ 63.472059] ixgbe_dcb_hw_config_82599+0xab2/0x1360 [ixgbe]
Nov 23 12:49:47 localhost kernel: [ 63.472069] dcbnl_setstate+0x3f/0x90
Nov 23 12:49:47 localhost kernel: [ 63.472076] dcb_doit+0x124/0x1d0
Nov 23 12:49:47 localhost kernel: [ 63.472085] rtnetlink_rcv_msg+0x2a2/0x320
Nov 23 12:49:47 localhost kernel: [ 63.472093] ? _raw_spin_unlock_irqrestore+0x14/0x20
Nov 23 12:49:47 localhost kernel: [ 63.472101] ? __skb_try_recv_datagram+0xd3/0x180
Nov 23 12:49:47 localhost kernel: [ 63.472109] ? rtnl_calcit.isra.31+0x110/0x110
Nov 23 12:49:47 localhost kernel: [ 63.472118] netlink_rcv_skb+0xd4/0x110
Nov 23 12:49:47 localhost kernel: [ 63.472126] netlink_unicast+0x182/0x230
Nov 23 12:49:47 localhost kernel: [ 63.472133] netlink_sendmsg+0x2ed/0x3e0
Nov 23 12:49:47 localhost kernel: [ 63.472142] sock_sendmsg+0x36/0x50
Nov 23 12:49:47 localhost kernel: [ 63.472149] __sys_sendto+0xdc/0x160
Nov 23 12:49:47 localhost kernel: [ 63.472158] ? __call_rcu.constprop.74+0xc8/0x1d0
Nov 23 12:49:47 localhost kernel: [ 63.472167] __x64_sys_sendto+0x24/0x30
Nov 23 12:49:47 localhost kernel: [ 63.472175] do_syscall_64+0x4e/0x100
Nov 23 12:49:47 localhost kernel: [ 63.472183] entry_SYSCALL_64_after_hwframe+0x44/0xa9
Nov 23 12:49:47 localhost kernel: [ 63.472191] RIP: 0033:0x7f77bee96353
Nov 23 12:49:47 localhost kernel: [ 63.472198] Code: 48 8b 0d 38 2b 2c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 0f 1f 44 00 00 83 3d ad 8c 2c 00 00 75 13 49 89 ca b8 2c 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 34 c3 48 83 ec 08 e8 1b c9 00 00 48 89 04 24
Nov 23 12:49:47 localhost kernel: [ 63.472219] RSP: 002b:00007fff3d4a1568 EFLAGS: 00000246 ORIG_RAX: 000000000000002c
Nov 23 12:49:47 localhost kernel: [ 63.472230] RAX: ffffffffffffffda RBX: 0000562e22ccc970 RCX: 00007f77bee96353
Nov 23 12:49:47 localhost kernel: [ 63.472239] RDX: 0000000000000028 RSI: 0000562e22ccc970 RDI: 0000000000000005
Nov 23 12:49:47 localhost kernel: [ 63.472249] RBP: 0000562e22ccc970 R08: 00007fff3d4a1570 R09: 000000000000000c
Nov 23 12:49:47 localhost kernel: [ 63.472259] R10: 0000000000000000 R11: 0000000000000246 R12: 00007fff3d4a1570
Nov 23 12:49:47 localhost kernel: [ 63.472269] R13: 0000000000000028 R14: 0000000000000000 R15: 0000000000000009
Nov 23 12:49:47 localhost kernel: [ 63.472279] Modules linked in: 8021q garp mrp stp llc openvswitch nsh nf_nat_ipv6 nf_nat_ipv4 nf_conncount nf_nat ipt_REJECT nf_reject_ipv4 xt_tcpudp xt_multiport xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_filter dm_multipath dm_mod dcdbas intel_powerclamp coretemp sg ipmi_si bnx2 ipmi_devintf ixgbe ipmi_msghandler i7core_edac acpi_power_meter lpc_ich hed mdio nls_utf8 isofs loop nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc ip_tables x_tables sha1_ssse3 sha1_generic ipv6 sd_mod sr_mod cdrom hid_generic ata_generic pata_acpi usbhid hid ata_piix libata mptsas scsi_transport_sas mptscsih mptbase ehci_pci ehci_hcd scsi_dh_rdac scsi_dh_hp_sw scsi_dh_emc scsi_dh_alua scsi_mod xen_wdt crc_ccitt
Nov 23 12:49:47 localhost kernel: [ 63.472381] CR2: 0000001400000001
Nov 23 12:49:47 localhost kernel: [ 63.472388] ---[ end trace 44555ed63c956ad6 ]---
Nov 23 12:49:47 localhost kernel: [ 63.475033] RIP: e030:remove_xps_queue+0x17/0x80
Nov 23 12:49:47 localhost kernel: [ 63.475045] Code: ff 0f 0b e9 31 ff ff ff 90 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 85 ff 74 49 48 63 f6 48 8b 4c f7 10 48 85 c9 74 3c <44> 8b 11 45 8d 4a ff 44 89 c8 83 f8 ff 4c 63 c0 74 2d 49 83 c0 08
Nov 23 12:49:47 localhost kernel: [ 63.475071] RSP: e02b:ffffc90040ccf9e0 EFLAGS: 00010202
Nov 23 12:49:47 localhost kernel: [ 63.475084] RAX: 0000000000000000 RBX: ffff88000456d280 RCX: 0000001400000001
Nov 23 12:49:47 localhost kernel: [ 63.475094] RDX: 0000000000000000 RSI: 0000000000000006 RDI: ffff88000456d280
Nov 23 12:49:47 localhost kernel: [ 63.475105] RBP: 0000000000000001 R08: ffffffffffffffff R09: 0000000000000000
Nov 23 12:49:47 localhost kernel: [ 63.475114] R10: 0000000000000001 R11: 000000000000cd40 R12: 0000000000000000
Nov 23 12:49:47 localhost kernel: [ 63.475124] R13: 0000000000000006 R14: 0000000100000000 R15: 0000000000000002
Nov 23 12:49:47 localhost kernel: [ 63.475141] FS: 00007f77bf9a3740(0000) GS:ffff88002d840000(0000) knlGS:0000000000000000
Nov 23 12:49:47 localhost kernel: [ 63.475152] CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 23 12:49:47 localhost kernel: [ 63.475162] CR2: 0000001400000001 CR3: 000000002738c000 CR4: 0000000000002660
0000000000000000 <remove_xps_queue>:
0: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1)
// if (dev_maps)
5: 48 85 ff test %rdi,%rdi
8: 74 49 je 0x53
// map = xmap_dereference(dev_maps->attr_map[tci]);
a: 48 63 f6 movslq %esi,%rsi
d: 48 8b 4c f7 10 mov 0x10(%rdi,%rsi,8),%rcx // Results in a bogus, non-NULL pointer
// if (!map)
12: 48 85 c9 test %rcx,%rcx
15: 74 3c je 0x53
// for (pos = map->len; ...
17: 44 8b 11 mov (%rcx),%r10d // <- Fault on dereferencing map
1a: 45 8d 4a ff lea -0x1(%r10),%r9d
1e: 44 89 c8 mov %r9d,%eax
21: 83 f8 ff cmp $0xffffffff,%eax
24: 4c 63 c0 movslq %eax,%r8
27: 74 2d je 0x56
29: 49 83 c0 08 add $0x8,%r8
Regards,
Nathanael
Powered by blists - more mailing lists