| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Mon, 18 Apr 2016 16:44:58 +0000
From: "Butler, Peter" <pbutler@...usnet.com>
To: Eric Dumazet <eric.dumazet@...il.com>
CC: "netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: RE: Poorer networking performance in later kernels?
P.S. Cancel my comment about some fields existing in one kernel but not the other- that was probably just an artefact of the fact that for one kernel the value was zero but not in the other kernel (and that I did not run nstat with the -z option to output zero counts). So the data I provided is still good, but the 'empty' fields that exist in one kernel or the other can safely be assumed to be zero counts where left out.
-----Original Message-----
From: Butler, Peter
Sent: April-18-16 12:27 PM
To: 'Eric Dumazet' <eric.dumazet@...il.com>
Cc: netdev@...r.kernel.org
Subject: RE: Poorer networking performance in later kernels?
Hi Eric
Thanks for your response. My apologies for being late in getting back to you - I wasn't able to have access to the lab hardware on the weekend.
I performed your test as suggested - I've provided a side-by-side diff of the nstat output below for the SCTP test only (not the TCP test). Note that the fields that are output are somewhat different for the two kernels - i.e. some fields exist in one but not the other (presumably this comes from the kernel internals?).
Other than seeing 'larger' throughput numbers in this output I'm not sure what to take from it - I'm certainly not a networking expert :-( Let me know if there's anything that speaks to you.
Note that this test was again done on a clean, freshly rebooted and idle system. Let me know if there's any issues with the output format of this data in the email.
Thanks,
Peter
3.4.2 4.4.0
-------------------------------------------------------------------------------------------------------
IpInReceives 3457295 0.0 | IpInReceives 1151189 0.0
IpInDelivers 3457295 0.0 | IpInDelivers 1151189 0.0
IpOutRequests 6864955 0.0 | IpOutRequests 2249622 0.0
IcmpInErrors 158 0.0 | IcmpInErrors 159 0.0
IcmpInTimeExcds 152 0.0 | IcmpInTimeExcds 151 0.0
IcmpInEchoReps 6 0.0 | IcmpInEchoReps 8 0.0
IcmpOutErrors 158 0.0 | IcmpOutErrors 159 0.0
IcmpOutTimeExcds 152 0.0 | IcmpOutTimeExcds 151 0.0
IcmpOutTimestamps 6 0.0 | IcmpOutTimestamps 8 0.0
IcmpMsgInType3 152 0.0 | IcmpMsgInType3 151 0.0
IcmpMsgInType8 6 0.0 | IcmpMsgInType8 8 0.0
IcmpMsgOutType0 6 0.0 | IcmpMsgOutType0 8 0.0
IcmpMsgOutType3 152 0.0 | IcmpMsgOutType3 151 0.0
TcpActiveOpens 1 0.0 TcpActiveOpens 1 0.0
TcpPassiveOpens 3 0.0 | TcpPassiveOpens 4 0.0
TcpInSegs 70 0.0 | TcpInSegs 117 0.0
TcpOutSegs 66 0.0 | TcpOutSegs 110 0.0
| TcpOutRsts 24 0.0
UdpInDatagrams 608 0.0 | UdpInDatagrams 604 0.0
UdpNoPorts 152 0.0 | UdpNoPorts 151 0.0
UdpOutDatagrams 760 0.0 | UdpOutDatagrams 755 0.0
| UdpIgnoredMulti 144 0.0
TcpExtTW 2 0.0
TcpExtDelayedACKs 3 0.0 | TcpExtDelayedACKs 4 0.0
TcpExtTCPHPHits 25 0.0 | TcpExtTCPHPHits 41 0.0
TcpExtTCPPureAcks 12 0.0 | TcpExtTCPPureAcks 14 0.0
TcpExtTCPHPAcks 18 0.0 | TcpExtTCPHPAcks 26 0.0
| TcpExtTCPRcvCoalesce 12 0.0
| TcpExtTCPOrigDataSent 57 0.0
IpExtInBcastPkts 152 0.0 | IpExtInBcastPkts 144 0.0
IpExtInOctets 166191161 0.0 | IpExtInOctets 55395212 0.0
IpExtOutOctets 9107586685 0.0 | IpExtOutOctets 2983660504 0.0
IpExtInBcastOctets 37356 0.0 | IpExtInBcastOctets 35328 0.0
| IpExtInNoECTPkts 1175 0.0
| IpExtInECT0Pkts 1150014 0.0
-----Original Message-----
From: Eric Dumazet [mailto:eric.dumazet@...il.com]
Sent: April-18-16 8:17 AM
To: Butler, Peter <pbutler@...usnet.com>
Cc: netdev@...r.kernel.org
Subject: Re: Poorer networking performance in later kernels?
On Fri, 2016-04-15 at 15:33 -0700, Eric Dumazet wrote:
> On Fri, 2016-04-15 at 21:02 +0000, Butler, Peter wrote:
> > (Please keep me CC'd to all comments/responses)
> >
> > I've tried a kernel upgrade from 3.4.2 to 4.4.0 and see a marked drop in networking performance. Nothing was changed on the test systems, other than the kernel itself (and kernel modules). The identical .config used to build the 3.4.2 kernel was brought over into the 4.4.0 kernel source tree, and any configuration differences (e.g. new parameters, etc.) were taken as default values.
> >
> > The testing was performed on the same actual hardware for both kernel versions (i.e. take the existing 3.4.2 physical setup, simply boot into the (new) kernel and run the same test). The netperf utility was used for benchmarking and the testing was always performed on idle systems.
> >
> > TCP testing yielded the following results, where the 4.4.0 kernel only got about 1/2 of the throughput:
> >
> > Recv Send Send Utilization Service Demand
> > Socket Socket Message Elapsed Send Recv Send Recv
> > Size Size Size Time Throughput local remote local remote
> > bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB
> >
> > 3.4.2 13631488 13631488 8952 30.01 9370.29 10.14 6.50 0.709 0.454
> > 4.4.0 13631488 13631488 8952 30.02 5314.03 9.14 14.31 1.127 1.765
> >
> > SCTP testing yielded the following results, where the 4.4.0 kernel only got about 1/3 of the throughput:
> >
> > Recv Send Send Utilization Service Demand
> > Socket Socket Message Elapsed Send Recv Send Recv
> > Size Size Size Time Throughput local remote local remote
> > bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB
> >
> > 3.4.2 13631488 13631488 8952 30.00 2306.22 13.87 13.19 3.941 3.747
> > 4.4.0 13631488 13631488 8952 30.01 882.74 16.86 19.14 12.516 14.210
> >
> > The same tests were performed a multitude of time, and are always consistent (within a few percent). I've also tried playing with various run-time kernel parameters (/proc/sys/kernel/net/...) on the 4.4.0 kernel to alleviate the issue but have had no success at all.
> >
> > I'm at a loss as to what could possibly account for such a discrepancy...
>
> Maybe new kernel is faster and you have drops somewhere ?
>
> nstat >/dev/null
> netperf -H ...
> nstat
>
> Would help
>
Are you receiving my mails, or simply ignoring them ?
Thanks.
Powered by blists - more mailing lists