| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Thu, 22 Nov 2018 15:24:07 +0100
From: Petr Oros <poros@...hat.com>
To: netdev@...r.kernel.org
Cc: ivecera@...hat.com, davem@...emloft.net
Subject: [PATCH net] be2net: Fix NULL pointer dereference in be_tx_timeout()
The driver enumerates Tx queues in ndo_tx_timeout() handler, here is
possible race with be_update_queues. For this case we set carrier_off.
It prevents netdev watchdog to be fired after be_clear_queues().
The watchdog timeout doesn't make any sense here as we re-creating queues.
Reproducer:
We can reproduce bug with ethtool when changing queue count
ethtool -L $netif combined 1
ethtool -L $netif combined 32
If oops is not triggered imediately, just run it again or in loop.
Oops:
[ 865.768648] NETDEV WATCHDOG: enp4s0f0 (be2net): transmit queue 0 timed out
[ 865.775539] WARNING: CPU: 3 PID: 0 at net/sched/sch_generic.c:461 dev_watchdog+0x20d/0x220
[ 865.783796] Modules linked in: be2net intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul iTCO_wdt iTCO_vendor_support ghash_clmulni_intel mei_me intel_cstate intel_uncore ipmi_ssif mei ipmi_si pcspkr sg i2c_i801 joydev lpc_ich intel_rapl_perf ipmi_devintf ioatdma ipmi_msghandler xfs libcrc32c sd_mod mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci libahci crc32c_intel drm serio_raw libata igb dca i2c_algo_bit wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: be2net]
[ 865.834289] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.20.0-rc3+ #2
[ 865.840640] Hardware name: Supermicro X9DBU/X9DBU, BIOS 3.2 01/15/2015
[ 865.847168] RIP: 0010:dev_watchdog+0x20d/0x220
[ 865.851612] Code: 00 49 63 4e e0 eb 92 4c 89 e7 c6 05 a5 de c9 00 01 e8 f7 b2 fc ff 89 d9 4c 89 e6 48 c7 c7 a0 d1 b2 99 48 89 c2 e8 7d b0 98 ff <0f> 0b eb c0 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66
[ 865.870358] RSP: 0018:ffff9bee73ac3e88 EFLAGS: 00010282
[ 865.875583] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000000083f
[ 865.882707] RDX: 0000000000000000 RSI: 00000000000000f6 RDI: 000000000000003f
[ 865.889832] RBP: ffff9bee5fa0045c R08: 0000000000000824 R09: 0000000000000007
[ 865.896956] R10: 0000000000000000 R11: ffffffff9a3f162d R12: ffff9bee5fa00000
[ 865.904088] R13: 0000000000000003 R14: ffff9bee5fa00480 R15: 0000000000000020
[ 865.911214] FS: 0000000000000000(0000) GS:ffff9bee73ac0000(0000) knlGS:0000000000000000
[ 865.919298] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 865.925037] CR2: 00005580497ce040 CR3: 00000002cf60a004 CR4: 00000000000606e0
[ 865.932170] Call Trace:
[ 865.934626] <IRQ>
[ 865.936645] ? pfifo_fast_dequeue+0x160/0x160
[ 865.941005] call_timer_fn+0x2b/0x130
[ 865.944670] run_timer_softirq+0x3b9/0x3f0
[ 865.948768] ? tick_sched_timer+0x37/0x70
[ 865.952779] ? __hrtimer_run_queues+0x110/0x280
[ 865.957314] __do_softirq+0xdd/0x2fe
[ 865.960896] irq_exit+0xfa/0x100
[ 865.964125] smp_apic_timer_interrupt+0x74/0x140
[ 865.968745] apic_timer_interrupt+0xf/0x20
[ 865.972844] </IRQ>
[ 865.974953] RIP: 0010:cpuidle_enter_state+0xb0/0x320
[ 865.979915] Code: 89 c3 66 66 66 66 90 31 ff e8 0c 07 a6 ff 80 7c 24 0b 00 74 12 9c 58 f6 c4 02 0f 85 46 02 00 00 31 ff e8 33 e0 ab ff fb 85 ed <0f> 88 1a 02 00 00 48 b8 ff ff ff ff f3 01 00 00 48 2b 1c 24 48 39
[ 865.998661] RSP: 0018:ffffbc9ac19e7ea0 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13
[ 866.006225] RAX: ffff9bee73ae1dc0 RBX: 000000c9938e11ae RCX: 000000000000001f
[ 866.013350] RDX: 000000c9938e11ae RSI: 00000000435e532a RDI: 0000000000000000
[ 866.020474] RBP: 0000000000000005 R08: 0000000000000002 R09: 0000000000021640
[ 866.027598] R10: 00009c434b946fde R11: ffff9bee73ae0e44 R12: ffffffff99d27538
[ 866.034723] R13: ffff9bee73aec628 R14: 0000000000000005 R15: 0000000000000000
[ 866.041860] do_idle+0x1f1/0x230
[ 866.045091] cpu_startup_entry+0x19/0x20
[ 866.049016] start_secondary+0x195/0x1e0
[ 866.052943] secondary_startup_64+0xb6/0xc0
[ 866.057129] ---[ end trace dead88c26bcd8261 ]---
[ 866.061750] be2net 0000:04:00.0: TXQ Dump: 0 H: 0 T: 0 used: 0, qid: 0x2
[ 866.068452] BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
[ 866.076273] PGD 0 P4D 0
[ 866.078810] Oops: 0000 [#1] SMP PTI
[ 866.082305] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G W 4.20.0-rc3+ #2
[ 866.090041] Hardware name: Supermicro X9DBU/X9DBU, BIOS 3.2 01/15/2015
[ 866.096566] RIP: 0010:be_tx_timeout+0x7c/0x300 [be2net]
[ 866.101786] Code: 8b 45 1c 41 8b 4d 14 48 89 df 31 ed 45 8b 4d 18 48 c7 c6 80 51 2c c0 50 45 8b 45 10 8b 54 24 14 e8 09 a7 cb d8 4d 8b 7d 20 59 <41> 8b 0c af 45 8b 44 af 04 41 8b 74 af 0c 45 8b 4c af 08 89 ca 44
[ 866.120532] RSP: 0018:ffff9bee73ac3e38 EFLAGS: 00010246
[ 866.125758] RAX: 0000000000000000 RBX: ffff9bee72d6b0b0 RCX: 0000000000000002
[ 866.132882] RDX: 0000000000000000 RSI: 00000000000000f6 RDI: 000000000000003f
[ 866.140014] RBP: 0000000000000000 R08: 000000000000084d R09: 0000000000000007
[ 866.147138] R10: 0000000000000000 R11: ffffffff9a3f162d R12: ffffffffc02c60ab
[ 866.154263] R13: ffff9bee5fa04b40 R14: ffffffffc02c613a R15: 0000000000000000
[ 866.161388] FS: 0000000000000000(0000) GS:ffff9bee73ac0000(0000) knlGS:0000000000000000
[ 866.169472] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 866.175210] CR2: 0000000000000000 CR3: 00000002cf60a004 CR4: 00000000000606e0
[ 866.182334] Call Trace:
[ 866.184781] <IRQ>
[ 866.186799] dev_watchdog+0x1e4/0x220
[ 866.190466] ? pfifo_fast_dequeue+0x160/0x160
[ 866.194825] call_timer_fn+0x2b/0x130
[ 866.198491] run_timer_softirq+0x3b9/0x3f0
[ 866.202590] ? tick_sched_timer+0x37/0x70
[ 866.206604] ? __hrtimer_run_queues+0x110/0x280
[ 866.211134] __do_softirq+0xdd/0x2fe
[ 866.214715] irq_exit+0xfa/0x100
[ 866.217948] smp_apic_timer_interrupt+0x74/0x140
[ 866.222566] apic_timer_interrupt+0xf/0x20
[ 866.226664] </IRQ>
[ 866.228762] RIP: 0010:cpuidle_enter_state+0xb0/0x320
[ 866.233727] Code: 89 c3 66 66 66 66 90 31 ff e8 0c 07 a6 ff 80 7c 24 0b 00 74 12 9c 58 f6 c4 02 0f 85 46 02 00 00 31 ff e8 33 e0 ab ff fb 85 ed <0f> 88 1a 02 00 00 48 b8 ff ff ff ff f3 01 00 00 48 2b 1c 24 48 39
[ 866.252465] RSP: 0018:ffffbc9ac19e7ea0 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13
[ 866.260031] RAX: ffff9bee73ae1dc0 RBX: 000000c9938e11ae RCX: 000000000000001f
[ 866.267164] RDX: 000000c9938e11ae RSI: 00000000435e532a RDI: 0000000000000000
[ 866.274288] RBP: 0000000000000005 R08: 0000000000000002 R09: 0000000000021640
[ 866.281421] R10: 00009c434b946fde R11: ffff9bee73ae0e44 R12: ffffffff99d27538
[ 866.288545] R13: ffff9bee73aec628 R14: 0000000000000005 R15: 0000000000000000
[ 866.295680] do_idle+0x1f1/0x230
[ 866.298913] cpu_startup_entry+0x19/0x20
[ 866.302839] start_secondary+0x195/0x1e0
[ 866.306764] secondary_startup_64+0xb6/0xc0
[ 866.310948] Modules linked in: be2net intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul iTCO_wdt iTCO_vendor_support ghash_clmulni_intel mei_me intel_cstate intel_uncore ipmi_ssif mei ipmi_si pcspkr sg i2c_i801 joydev lpc_ich intel_rapl_perf ipmi_devintf ioatdma ipmi_msghandler xfs libcrc32c sd_mod mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci libahci crc32c_intel drm serio_raw libata igb dca i2c_algo_bit wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: be2net]
[ 866.361432] CR2: 0000000000000000
[ 866.364748] ---[ end trace dead88c26bcd8262 ]---
[ 866.507013] RIP: 0010:be_tx_timeout+0x7c/0x300 [be2net]
[ 866.512234] Code: 8b 45 1c 41 8b 4d 14 48 89 df 31 ed 45 8b 4d 18 48 c7 c6 80 51 2c c0 50 45 8b 45 10 8b 54 24 14 e8 09 a7 cb d8 4d 8b 7d 20 59 <41> 8b 0c af 45 8b 44 af 04 41 8b 74 af 0c 45 8b 4c af 08 89 ca 44
[ 866.530980] RSP: 0018:ffff9bee73ac3e38 EFLAGS: 00010246
[ 866.536206] RAX: 0000000000000000 RBX: ffff9bee72d6b0b0 RCX: 0000000000000002
[ 866.543330] RDX: 0000000000000000 RSI: 00000000000000f6 RDI: 000000000000003f
[ 866.550454] RBP: 0000000000000000 R08: 000000000000084d R09: 0000000000000007
[ 866.557578] R10: 0000000000000000 R11: ffffffff9a3f162d R12: ffffffffc02c60ab
[ 866.564710] R13: ffff9bee5fa04b40 R14: ffffffffc02c613a R15: 0000000000000000
[ 866.571835] FS: 0000000000000000(0000) GS:ffff9bee73ac0000(0000) knlGS:0000000000000000
[ 866.579920] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 866.585658] CR2: 0000000000000000 CR3: 00000002cf60a004 CR4: 00000000000606e0
[ 866.592784] Kernel panic - not syncing: Fatal exception in interrupt
[ 866.599179] Kernel Offset: 0x17a00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
Fixes: c1b3bdb2ffa9 ("be2net: gather debug info and reset adapter (only for Lancer) on a tx-timeout")
Signed-off-by: Petr Oros <poros@...hat.com>
---
drivers/net/ethernet/emulex/benet/be_main.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/emulex/benet/be_main.c b/drivers/net/ethernet/emulex/benet/be_main.c
index c5ad7a4f4d83..02202c5e6794 100644
--- a/drivers/net/ethernet/emulex/benet/be_main.c
+++ b/drivers/net/ethernet/emulex/benet/be_main.c
@@ -4700,8 +4700,11 @@ int be_update_queues(struct be_adapter *adapter)
struct net_device *netdev = adapter->netdev;
int status;
- if (netif_running(netdev))
+ if (netif_running(netdev)) {
+ /* prevent netdev watchdog during tx queue destroy */
+ netif_carrier_off(netdev);
be_close(netdev);
+ }
be_cancel_worker(adapter);
--
2.18.1
Powered by blists - more mailing lists