netdev - Re: [Intel-wired-lan] i40e XDP program stops transmitting after link down/up

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <47eea378-6b76-46a7-b70e-52bc61f5e9f0@molgen.mpg.de>
Date: Tue, 30 Jan 2024 17:14:23 +0100
From: Paul Menzel <pmenzel@...gen.mpg.de>
To: Seth Forshee <sforshee@...nel.org>
Cc: Jesse Brandeburg <jesse.brandeburg@...el.com>,
 Tony Nguyen <anthony.l.nguyen@...el.com>, netdev@...r.kernel.org,
 intel-wired-lan@...ts.osuosl.org
Subject: Re: [Intel-wired-lan] i40e XDP program stops transmitting after link
 down/up

Dear Seth,


Thank you for bring this up.

Am 30.01.24 um 15:17 schrieb Seth Forshee:
> I got a inquiry from a colleague about a behavior he's seeing with i40e
> but not with other NICs. The interfaces are bonded with a XDP
> load-balancer program attached to them. After 'ip link set ethX down; ip
> link set ethX up' on one of the interfaces the XDP program on that
> interface is no longer transmitting packets. He found that tx starts
> again after running 'sudo ethtool -t ethX'.
> 
> There's a 'i40e 0000:d8:00.1: VSI seid 391 XDP Tx ring 0 disable
> timeout' message in dmesg when disabling the interface. I've included
> the relevant portions from dmesg below.
> 
> This was first observed with a 6.1 kernel, but we've confirmed that the
> behavior is the same in 6.7. I realize the firmware is pretty old, so
> far our attempts to update the NVM have failed.

Does that mean, the problem didn’t happen before Linux 6.1? If so, if 
you have the reproducer and the time, bisecting the issue is normally 
the fastest way to solve the issue.


Kind regards,

Paul


> [    0.000000] Linux version 6.7.0 (root@...a530b3729) (gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #2 SMP Thu Jan 25 10:37:21 EST 2024
> ...
> [    9.038171] i40e: Intel(R) Ethernet Connection XL710 Network Driver
> [    9.044447] i40e: Copyright (c) 2013 - 2019 Intel Corporation.
> ...
> [    9.064833] i40e 0000:d8:00.0: fw 7.0.50775 api 1.8 nvm 7.00 0x80004c97 1.2154.0 [8086:1583] [8086:0002]
> ...
> [    9.320886] i40e 0000:d8:00.0: MAC address: xx:xx:xx:xx:xx:xx
> [    9.327030] i40e 0000:d8:00.0: FW LLDP is enabled
> [    9.344331] i40e 0000:d8:00.0 eth0: NIC Link is Up, 40 Gbps Full Duplex, Flow Control: None
> [    9.355552] i40e 0000:d8:00.0: PCI-Express: Speed 8.0GT/s Width x8
> [    9.374074] i40e 0000:d8:00.0: Features: PF-id[0] VFs: 64 VSIs: 66 QP: 32 RSS FD_ATR FD_SB NTUPLE DCB VxLAN Geneve PTP VEPA
> ...
> [    9.401522] i40e 0000:d8:00.1: fw 7.0.50775 api 1.8 nvm 7.00 0x80004c97 1.2154.0 [8086:1583] [8086:0002]
> ...
> [    9.652066] i40e 0000:d8:00.1: MAC address: xx:xx:xx:xx:xx:xx
> [    9.658040] i40e 0000:d8:00.1: FW LLDP is enabled
> [    9.688622] i40e 0000:d8:00.1 eth1: NIC Link is Up, 40 Gbps Full Duplex, Flow Control: None
> [    9.699822] i40e 0000:d8:00.1: PCI-Express: Speed 8.0GT/s Width x8
> [    9.719259] i40e 0000:d8:00.1: Features: PF-id[1] VFs: 64 VSIs: 66 QP: 32 RSS FD_ATR FD_SB NTUPLE DCB VxLAN Geneve PTP VEPA
> [    9.401522] i40e 0000:d8:00.1: fw 7.0.50775 api 1.8 nvm 7.00 0x80004c97 1.2154.0 [8086:1583] [8086:0002]
> ...
> [   13.948968] i40e 0000:d8:00.0: FW LLDP is disabled, attempting SW DCB
> [   13.956753] i40e 0000:d8:00.0: SW DCB initialization succeeded.
> [   13.970005] i40e 0000:d8:00.0: FW LLDP is disabled
> ...
> [   14.113004] i40e 0000:d8:00.1: FW LLDP is disabled, attempting SW DCB
> [   14.120800] i40e 0000:d8:00.1: SW DCB initialization succeeded.
> [   14.133940] i40e 0000:d8:00.1: FW LLDP is disabled
> ...
> [   14.267874] bonding: bond0 is being created...
> [   14.979070] i40e 0000:d8:00.1 eth1: set new mac address xx:xx:xx:xx:xx:xx
> [   14.994445] bond0: (slave eth1): Enslaving as a backup interface with an up link
> [   14.994681] i40e 0000:d8:00.0 eth0: set new mac address xx:xx:xx:xx:xx:xx
> [   15.010359] bond0: (slave eth0): Enslaving as a backup interface with an up link
> [   15.912874] i40e 0000:d8:00.0: Stop LLDP AQ command failed =0x1
> [   15.944842] i40e 0000:d8:00.1: Stop LLDP AQ command failed =0x1
> [   41.262871] 8021q: 802.1Q VLAN Support v1.8
> [   41.262890] 8021q: adding VLAN 0 to HW filter on device eth0
> [   41.262902] 8021q: adding VLAN 0 to HW filter on device eth1
> [   41.262914] 8021q: adding VLAN 0 to HW filter on device bond0
> [   48.272456] i40e 0000:d8:00.0: FW LLDP is disabled, attempting SW DCB
> [   48.280233] i40e 0000:d8:00.0: SW DCB initialization succeeded.
> [   48.307415] i40e 0000:d8:00.0: User requested queue count/HW max RSS count:  12/32
> [   48.440266] i40e 0000:d8:00.1: FW LLDP is disabled, attempting SW DCB
> [   48.448025] i40e 0000:d8:00.1: SW DCB initialization succeeded.
> [   48.475051] i40e 0000:d8:00.1: User requested queue count/HW max RSS count:  12/32
> [   58.935900] i40e 0000:d8:00.0: FW LLDP is disabled, attempting SW DCB
> [   58.945123] i40e 0000:d8:00.0: SW DCB initialization succeeded.
> [   59.131772] i40e 0000:d8:00.1: FW LLDP is disabled, attempting SW DCB
> [   59.139560] i40e 0000:d8:00.1: SW DCB initialization succeeded.
> [  336.363825] i40e 0000:d8:00.1: VSI seid 391 XDP Tx ring 0 disable timeout
> [  336.603619] bond0: (slave eth1): link status definitely down, disabling slave
> [  345.464976] 8021q: adding VLAN 0 to HW filter on device eth1
> [  345.547358] bond0: (slave eth1): link status definitely up, 40000 Mbps full duplex
> [  345.547379] bond0: active interface up!