lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening PHC | |
Open Source and information security mailing list archives
| ||
|
Date: Thu, 23 Aug 2007 23:18:20 -0400 From: Bill Fink <billfink@...dspring.com> To: Rick Jones <rick.jones2@...com> Cc: hadi@...erus.ca, David Miller <davem@...emloft.net>, krkumar2@...ibm.com, gaagaan@...il.com, general@...ts.openfabrics.org, herbert@...dor.apana.org.au, jagana@...ibm.com, jeff@...zik.org, johnpol@....mipt.ru, kaber@...sh.net, mcarlson@...adcom.com, mchan@...adcom.com, netdev@...r.kernel.org, peter.p.waskiewicz.jr@...el.com, rdreier@...co.com, Robert.Olsson@...a.slu.se, shemminger@...ux-foundation.org, sri@...ibm.com, tgraf@...g.ch, xma@...ibm.com Subject: Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB On Thu, 23 Aug 2007, Rick Jones wrote: > jamal wrote: > > [TSO already passed - iirc, it has been > > demostranted to really not add much to throughput (cant improve much > > over closeness to wire speed) but improve CPU utilization]. > > In the one gig space sure, but in the 10 Gig space, TSO on/off does make a > difference for throughput. Not too much. TSO enabled: [root@...g2 ~]# ethtool -k eth2 Offload parameters for eth2: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp segmentation offload: on [root@...g2 ~]# nuttcp -w10m 192.168.88.16 11813.4375 MB / 10.00 sec = 9906.1644 Mbps 99 %TX 80 %RX TSO disabled: [root@...g2 ~]# ethtool -K eth2 tso off [root@...g2 ~]# ethtool -k eth2 Offload parameters for eth2: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp segmentation offload: off [root@...g2 ~]# nuttcp -w10m 192.168.88.16 11818.2500 MB / 10.00 sec = 9910.0176 Mbps 100 %TX 78 %RX Pretty negligible difference it seems. This is with a 2.6.20.7 kernel, Myricom 10-GigE NICs, and 9000 byte jumbo frames, in a LAN environment. For grins, I also did a couple of tests with an MSS of 1460 to emulate a standard 1500 byte Ethernet MTU. TSO enabled: [root@...g2 ~]# ethtool -k eth2 Offload parameters for eth2: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp segmentation offload: on [root@...g2 ~]# nuttcp -M1460 -w10m 192.168.88.16 5102.8503 MB / 10.06 sec = 4253.9124 Mbps 39 %TX 99 %RX TSO disabled: [root@...g2 ~]# ethtool -K eth2 tso off [root@...g2 ~]# ethtool -k eth2 Offload parameters for eth2: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp segmentation offload: off [root@...g2 ~]# nuttcp -M1460 -w10m 192.168.88.16 5399.5625 MB / 10.00 sec = 4527.9070 Mbps 99 %TX 76 %RX Here you can see there is a major difference in the TX CPU utilization (99 % with TSO disabled versus only 39 % with TSO enabled), although the TSO disabled case was able to squeeze out a little extra performance from its extra CPU utilization. Interestingly, with TSO enabled, the receiver actually consumed more CPU than with TSO disabled, so I guess the receiver CPU saturation in that case (99 %) was what restricted its performance somewhat (this was consistent across a few test runs). -Bill - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists