linux-kernel - linux-next network throughput performance regression

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <CY1PR03MB13760C9F1AABC4EBC455B85CC4280@CY1PR03MB1376.namprd03.prod.outlook.com>
Date:	Fri, 6 Nov 2015 20:31:13 +0000
From:	Simon Xiao <sixiao@...rosoft.com>
To:	"devel@...uxdriverproject.org" <devel@...uxdriverproject.org>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
CC:	David Miller <davem@...emloft.net>,
	KY Srinivasan <kys@...rosoft.com>,
	Haiyang Zhang <haiyangz@...rosoft.com>,
	Simon Xiao <sixiao@...rosoft.com>
Subject: linux-next network throughput performance regression

I compared the network throughput performance on SLES12 bare metal servers, between SLES12 default kernel and latest linux-next (2015-11-05) kernel, based on the test results, I suspect there is a network regression exists on Linux-Next over the 40G Ethernet network:
a) iperf3 reports 50% performance drop with single TCP stream on latest linux-next;
b) iperf3 reports 10% ~ 30% performance drop with 2 to 128 TCP streams on latest linux-next;
Another throughput benchmarking tool (ntttcp-for-linux) test result is also listed at the end of the email for reference.


Server configuration:
------------------------------
Two servers (one client and one server, cross linked by 40G Ethernet), which have:
a) CPU: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 2 sockets, 16 CPUs, cache size : 20480 KB
b) Memory: 64 GB
c) Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro], 40G Ethernet, default driver


Test with iperf3:
------------------------------
iperf3: https://github.com/esnet/iperf

a) SLES12 default kernel, network throughput tested by iperf3:
Test Connections	1	2	4	8	16	32	64	128	
Throughput (G bps)	36.7	37.3	37.6	37.7	37.7	37.7	37.7	25.7		

b) SLES12 + Linux-Next 20151105, network throughput tested by iperf3:
Test Connections	1	2	4	8	16	32	64	128
Throughput (G bps)	18.2	32.2	34.6	32.8	27.6	32.0	27.0	21.3
Percentage dropped	-50%	-14%	-8%	-13%	-27%	-15%	-28%	-17%


Test with ntttcp-for-linux:
------------------------------
ntttcp-for-linux: https://github.com/Microsoft/ntttcp-for-linux 

a) SLES12 default kernel, network throughput tested by ntttcp-for-linux:
Test Connections	1	2	4	8	16	32	64	128	256	512
Throughput (Gbps)	36.19	37.29	37.67	37.68	37.7	37.72	37.74	37.76	37.81	37.9	

b) SLES12 + Linux-Next 20151105, network throughput tested by ntttcp-for-linux:
Test Connections	1	2	4	8	16	32	64	128	256	512	
Throughput (Gbps)	28.12	34.01	37.6	36.53	32.94	33.07	33.63	33.44	33.83	34.42	
Percentage dropped	-22%	-9%	0%	-3%	-13%	-12%	-11%	-11%	-11%	-9%
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/