Message-ID: <4E8D7B85.7060702@fox-it.com>
Date:	Thu, 6 Oct 2011 11:57:25 +0200
From:	Jasper Spaans <spaans@...-it.com>
To:	"netdev@...r.kernel.org" <netdev@...r.kernel.org>
CC:	<mchan@...adcom.com>
Subject: bnx2 rxhash

Hi list,

I'm doing some experiments with rxhash and packet capturing, and I'm
seeing some unexpected results when receiving data (which happens in
promiscuous mode). This behaviour seems to be similar to that seen in
the question asked previously by Christophe Ngo Van Duc:
http://kerneltrap.com/mailarchive/linux-netdev/2010/7/2/6280474

The data I'm processing consists of (almost 100% tcp) traffic between an
ssl-offloader and a cluster of webservers, so the range of mac-addresses
is rather limited. This ssl-offloader does preserve the IP-address and
tcp ports of the clients, so if the rxhash is based on that data, it
should be distributed evenly.

Is there anything I can do about this?


Thanks,
Jasper



The raw data:

spaans@...p-074:~$ uname -a
Linux dhcp-074 3.0.4 #7 SMP Tue Sep 13 12:52:01 CEST 2011 x86_64 GNU/Linux

spaans@...p-074:~$ grep eth6 /proc/interrupts  | cut -b -30,270-
 142:      57070          0      PCI-MSI-edge      eth6-0
 143:          0          0      PCI-MSI-edge      eth6-1
 144:          0          0      PCI-MSI-edge      eth6-2
 145:          0          0      PCI-MSI-edge      eth6-3
 146:          0          0      PCI-MSI-edge      eth6-4
 147:      17127          0      PCI-MSI-edge      eth6-5
 148:         24          0      PCI-MSI-edge      eth6-6
 149:          0          0      PCI-MSI-edge      eth6-7

This is rather different from the behaviour I get with an intel card:

spaans@...p-074:~$ grep eth2 /proc/interrupts  | cut -b -30,270-
  98:          3          0      PCI-MSI-edge      eth2
  99:       4492          0      PCI-MSI-edge      eth2-TxRx-0
 100:       4647          0      PCI-MSI-edge      eth2-TxRx-1
 101:       4369          0      PCI-MSI-edge      eth2-TxRx-2
 102:       4579          0      PCI-MSI-edge      eth2-TxRx-3
 103:       4184          0      PCI-MSI-edge      eth2-TxRx-4
 104:      51268          0      PCI-MSI-edge      eth2-TxRx-5
 105:       4610          0      PCI-MSI-edge      eth2-TxRx-6
 106:       4528          0      PCI-MSI-edge      eth2-TxRx-7

In both cases, only Tx-5 was used for sending data.


Information about the nics:

from dmesg:
[    5.857778] igb 0000:08:00.0: Intel(R) Gigabit Ethernet Network
Connection
[    5.857780] igb 0000:08:00.0: eth6: (PCIe:2.5Gb/s:Width x4)
00:1b:21:b9:b8:dc
[    5.858101] igb 0000:08:00.0: eth6: PBA No: G18771-002
[    5.858102] igb 0000:08:00.0: Using MSI-X interrupts. 8 rx queue(s),
8 tx queue(s)
note that this interface is renamed to eth2 after boot

spaans@...p-074:~$ ip link sh eth2
8: eth2: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq
state UP qlen 1000
    link/ether 00:1b:21:b9:b8:dc brd ff:ff:ff:ff:ff:ff


spaans@...p-074:~$ sudo ethtool -k eth2
Offload parameters for eth2:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: off
large-receive-offload: off
ntuple-filters: off
receive-hashing: off
spaans@...p-074:~$ sudo ethtool -i eth2
driver: igb
version: 3.0.6-k2
firmware-version: 1.5-1
bus-info: 0000:08:00.0

vs

from dmesg:
[    5.090860] bnx2 0000:02:00.0: eth2: Broadcom NetXtreme II BCM5709
1000Base-T (C0) PCI Express found at mem d8000000, IRQ 32, node addr
18:03:73:fa:0f:0e
note that this interface is renamed to eth6 after boot

spaans@...p-074:~$ ip link sh eth6
4: eth6: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq
state UP qlen 1000
    link/ether 18:03:73:fa:0f:0e brd ff:ff:ff:ff:ff:ff

spaans@...p-074:~$ sudo ethtool -k eth6
Offload parameters for eth6:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
ntuple-filters: off
receive-hashing: on
spaans@...p-074:~$ sudo ethtool -i eth6
driver: bnx2
version: 2.1.6
firmware-version: 6.2.12 bc 5.2.3 NCSI 2.0.11
bus-info: 0000:02:00.0


-- 
 /\____/\   Ir. Jasper Spaans      
 \   (_)/   Fox-IT Experts in IT Security!
  \    X    T: +31-15-2847999
   \  / \   M: +31-6-41588725   
    \/      KvK Haaglanden 27301624



Download attachment "smime.p7s" of type "application/pkcs7-signature" (2679 bytes)

Powered by blists - more mailing lists