netdev - iproute2 does not enumerate vf's properly (ixgbe)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <89EBE686-6625-428C-8CCB-6A8E8B8162A8@gmail.com>
Date:	Sat, 24 Mar 2012 09:44:02 -0400
From:	Scott <ph3lpz@...il.com>
To:	netdev@...r.kernel.org
Subject: iproute2 does not enumerate vf's properly (ixgbe)

When loading the module ixgbe with sr-iov enabled (max_vfs), enumeration of the interfaces' VFs is inconsistent using the iproute2 utility.
 
Server Information:HP Product Name: ProLiant DL165 G7
Ethernet Card Information: Intel 82599EB 10-Gigabit SFI/SFP+
uname: Linux 3.0.0-16-server x86_64 GNU/Linux
OS Release: Ubuntu 11.10
 

Using a freshly compiled git pull:
# ./ip -V
ip utility, iproute2-ss120319

Steps to reproduce:

1. Determine the ethernet interface IDs:
# dmesg |grep ixgbe\ 0000:05:00.[0-9].*detected|head -n 2
[    8.448031] ixgbe 0000:05:00.0: eth4: detected SFP+: 5
[    8.816035] ixgbe 0000:05:00.1: eth5: detected SFP+: 6

2. Load the module w/40 VFs:
# modprobe ixgbe max_vfs=40

3. Sanity checks:
# lsmod |grep ixgbe
ixgbe                 206156  0 
dca                    15179  2 ixgbe,igb

# modinfo ixgbe|grep version
version:        3.6.7-NAPI
srcversion:     119ADBD2B16B0E40668FB9A
vermagic:       3.0.0-16-server SMP mod_unload modversions 
(this is the latest too)

# lspci |grep 05:1[0-9]|perl -lane 'delete @F[1,2,3,4,10,11];print "@F"'
05:10.0     82599 Ethernet Controller Virtual Function
05:10.2     82599 Ethernet Controller Virtual Function
05:10.4     82599 Ethernet Controller Virtual Function
05:10.6     82599 Ethernet Controller Virtual Function
05:11.0     82599 Ethernet Controller Virtual Function
(suppressing output of all 40 VFs)

# lspci |grep 05:1[0-9]|wc -l
40
(showing that there are actually 40)

# ls -l /sys/class/net/eth4/device/|grep virt|awk '{print $10" "$8}'|sort|head -5
../0000:05:10.0 virtfn0
../0000:05:10.2 virtfn1
../0000:05:10.4 virtfn2
../0000:05:10.6 virtfn3
../0000:05:11.0 virtfn4
(suppressing output of all 40 VFs)

5. Run ip link show:
# ./ip link show|grep -e eth[45] -e vf
(null output!)

6. Load the module with fewer VFs
# rmmod ixgbe
# modprobe ixgbe max_vfs=5
# lspci |grep 05:1[0-9]|perl -lane 'delete @F[1,2,3,4,10,11];print "@F"'
05:10.0     82599 Ethernet Controller Virtual Function
05:10.2     82599 Ethernet Controller Virtual Function
05:10.4     82599 Ethernet Controller Virtual Function
05:10.6     82599 Ethernet Controller Virtual Function
05:11.0     82599 Ethernet Controller Virtual Function

# ls -l /sys/class/net/eth4/device/|grep virt|awk '{print $10" "$8}'|sort
../0000:05:10.0 virtfn0
../0000:05:10.2 virtfn1
../0000:05:10.4 virtfn2
../0000:05:10.6 virtfn3
../0000:05:11.0 virtfn4

7. Run ip link show
# ./ip link show|grep -e eth[45] -e vf
57: eth4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    vf 0 MAC 52:a0:59:03:20:a8
    vf 1 MAC 82:5f:55:a5:1f:be
    vf 2 MAC 0a:ab:86:0c:02:b3
    vf 3 MAC 22:a7:28:ce:9c:23
    vf 4 MAC 26:ea:6c:1b:ca:57
58: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
(no vfs displayed for eth5!)

Patrick Kutch over at Intel suggested that there may be a patch to fix this problem, but I was not able to locate it.
http://communities.intel.com/thread/28411?tstart=0

There is also Launchpad bug #960497 for this issue.

Thanks in advance for any suggestions.

-SP--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html