netdev - i40e XDP program stops transmitting after link down/up

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date: Tue, 30 Jan 2024 08:17:16 -0600
From: Seth Forshee <sforshee@...nel.org>
To: Jesse Brandeburg <jesse.brandeburg@...el.com>,
	Tony Nguyen <anthony.l.nguyen@...el.com>
Cc: intel-wired-lan@...ts.osuosl.org, netdev@...r.kernel.org
Subject: i40e XDP program stops transmitting after link down/up

I got a inquiry from a colleague about a behavior he's seeing with i40e
but not with other NICs. The interfaces are bonded with a XDP
load-balancer program attached to them. After 'ip link set ethX down; ip
link set ethX up' on one of the interfaces the XDP program on that
interface is no longer transmitting packets. He found that tx starts
again after running 'sudo ethtool -t ethX'.

There's a 'i40e 0000:d8:00.1: VSI seid 391 XDP Tx ring 0 disable
timeout' message in dmesg when disabling the interface. I've included
the relevant portions from dmesg below.

This was first observed with a 6.1 kernel, but we've confirmed that the
behavior is the same in 6.7. I realize the firmware is pretty old, so
far our attempts to update the NVM have failed.

Thanks,
Seth

[    0.000000] Linux version 6.7.0 (root@...a530b3729) (gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #2 SMP Thu Jan 25 10:37:21 EST 2024
...
[    9.038171] i40e: Intel(R) Ethernet Connection XL710 Network Driver
[    9.044447] i40e: Copyright (c) 2013 - 2019 Intel Corporation.
...
[    9.064833] i40e 0000:d8:00.0: fw 7.0.50775 api 1.8 nvm 7.00 0x80004c97 1.2154.0 [8086:1583] [8086:0002]
...
[    9.320886] i40e 0000:d8:00.0: MAC address: xx:xx:xx:xx:xx:xx
[    9.327030] i40e 0000:d8:00.0: FW LLDP is enabled
[    9.344331] i40e 0000:d8:00.0 eth0: NIC Link is Up, 40 Gbps Full Duplex, Flow Control: None
[    9.355552] i40e 0000:d8:00.0: PCI-Express: Speed 8.0GT/s Width x8
[    9.374074] i40e 0000:d8:00.0: Features: PF-id[0] VFs: 64 VSIs: 66 QP: 32 RSS FD_ATR FD_SB NTUPLE DCB VxLAN Geneve PTP VEPA
...
[    9.401522] i40e 0000:d8:00.1: fw 7.0.50775 api 1.8 nvm 7.00 0x80004c97 1.2154.0 [8086:1583] [8086:0002]
...
[    9.652066] i40e 0000:d8:00.1: MAC address: xx:xx:xx:xx:xx:xx
[    9.658040] i40e 0000:d8:00.1: FW LLDP is enabled
[    9.688622] i40e 0000:d8:00.1 eth1: NIC Link is Up, 40 Gbps Full Duplex, Flow Control: None
[    9.699822] i40e 0000:d8:00.1: PCI-Express: Speed 8.0GT/s Width x8
[    9.719259] i40e 0000:d8:00.1: Features: PF-id[1] VFs: 64 VSIs: 66 QP: 32 RSS FD_ATR FD_SB NTUPLE DCB VxLAN Geneve PTP VEPA
[    9.401522] i40e 0000:d8:00.1: fw 7.0.50775 api 1.8 nvm 7.00 0x80004c97 1.2154.0 [8086:1583] [8086:0002]
...
[   13.948968] i40e 0000:d8:00.0: FW LLDP is disabled, attempting SW DCB
[   13.956753] i40e 0000:d8:00.0: SW DCB initialization succeeded.
[   13.970005] i40e 0000:d8:00.0: FW LLDP is disabled
...
[   14.113004] i40e 0000:d8:00.1: FW LLDP is disabled, attempting SW DCB
[   14.120800] i40e 0000:d8:00.1: SW DCB initialization succeeded.
[   14.133940] i40e 0000:d8:00.1: FW LLDP is disabled
...
[   14.267874] bonding: bond0 is being created...
[   14.979070] i40e 0000:d8:00.1 eth1: set new mac address xx:xx:xx:xx:xx:xx
[   14.994445] bond0: (slave eth1): Enslaving as a backup interface with an up link
[   14.994681] i40e 0000:d8:00.0 eth0: set new mac address xx:xx:xx:xx:xx:xx
[   15.010359] bond0: (slave eth0): Enslaving as a backup interface with an up link
[   15.912874] i40e 0000:d8:00.0: Stop LLDP AQ command failed =0x1
[   15.944842] i40e 0000:d8:00.1: Stop LLDP AQ command failed =0x1
[   41.262871] 8021q: 802.1Q VLAN Support v1.8
[   41.262890] 8021q: adding VLAN 0 to HW filter on device eth0
[   41.262902] 8021q: adding VLAN 0 to HW filter on device eth1
[   41.262914] 8021q: adding VLAN 0 to HW filter on device bond0
[   48.272456] i40e 0000:d8:00.0: FW LLDP is disabled, attempting SW DCB
[   48.280233] i40e 0000:d8:00.0: SW DCB initialization succeeded.
[   48.307415] i40e 0000:d8:00.0: User requested queue count/HW max RSS count:  12/32
[   48.440266] i40e 0000:d8:00.1: FW LLDP is disabled, attempting SW DCB
[   48.448025] i40e 0000:d8:00.1: SW DCB initialization succeeded.
[   48.475051] i40e 0000:d8:00.1: User requested queue count/HW max RSS count:  12/32
[   58.935900] i40e 0000:d8:00.0: FW LLDP is disabled, attempting SW DCB
[   58.945123] i40e 0000:d8:00.0: SW DCB initialization succeeded.
[   59.131772] i40e 0000:d8:00.1: FW LLDP is disabled, attempting SW DCB
[   59.139560] i40e 0000:d8:00.1: SW DCB initialization succeeded.
[  336.363825] i40e 0000:d8:00.1: VSI seid 391 XDP Tx ring 0 disable timeout
[  336.603619] bond0: (slave eth1): link status definitely down, disabling slave
[  345.464976] 8021q: adding VLAN 0 to HW filter on device eth1
[  345.547358] bond0: (slave eth1): link status definitely up, 40000 Mbps full duplex
[  345.547379] bond0: active interface up!