netdev - RE: Low performance Intel 10GE NIC (3.2.10) on 2.6.38 Kernel

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1302324057.15899.20.camel@ahduyck-mobl2.amr.corp.intel.com>
Date:	Fri, 08 Apr 2011 21:40:57 -0700
From:	Alexander H Duyck <alexander.h.duyck@...el.com>
To:	Wei Gu <wei.gu@...csson.com>
Cc:	Eric Dumazet <eric.dumazet@...il.com>,
	netdev <netdev@...r.kernel.org>,
	"Kirsher, Jeffrey T" <jeffrey.t.kirsher@...el.com>
Subject: RE: Low performance Intel 10GE NIC (3.2.10) on 2.6.38 Kernel

On Fri, 2011-04-08 at 20:36 -0700, Wei Gu wrote:
> Hi Alexander, I do agree with you that if only the rx_missing_error
> (rx_no_buffer_count: 0) indicates the memory bandwidth issue. But the
> strange thing is I using the same test configuration on Linux 2.6.32,
> which looks no this problem at all. SO it not a HW setup problem at
> all, only difference in on the Kernel version, that's why I go back to
> you guys for this new Linux 2.6.38, if it will affact this memory
> bandwidth Or BIOS etc things?

What were the numbers you were getting with 2.6.32?  I would be
interested in seeing those number just to get an idea of how they
compare against the 2.6.38 kernel.

> The follow dump is done, while I was try to receive 290Kpps 400Byte
> pakets from IXIA, and drop them in the prerouting hook. I bind the
> eth10 8 RX queue to CPU sock ID 3 ( core 24-31) on NUMA NODE3

Just to confirm this is with DMAR off?  I saw an earlier email that said
you were getting a variable amount that was over 1Mpps and just want to
confirm this is with the same config.

> ethtool -i eth10
> driver: ixgbe
> version: 3.2.10-NAPI
> firmware-version: 0.9-3
> bus-info: 0000:8d:00.0
> 
> ethtool -S eth10
> NIC statistics:
>      rx_packets: 14222510
>      tx_packets: 109
>      rx_bytes: 5575223920
>      tx_bytes: 17790
>      rx_missed_errors: 15150244
>      rx_no_buffer_count: 0

I trimmed down your stats here pretty significantly.  This isn't an
issue with the driver not keeping up.  The problem here is memory and/or
bus bandwidth.  Based on the info you provided I am assuming you have a
quad socket system.  I'm curious how the memory is laid out.  What is
the total memory size, memory per node, and do you have all of the
memory channels on each node populated?  One common thing I've seen
cause these type of issues is an incorrect memory configuration.

Also if you could send me an lspci -vvv for 8d:00.0 specifically I would
appreciate it as I would like to look over the PCIe config just to make
sure the slot is a x8 PCIe gen 2.

Thanks,

Alex

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html