netdev - Re: e1000 full-duplex TCP performance well below wire speed

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 31 Jan 2008 10:08:30 -0800
From:	"Kok, Auke" <auke-jan.h.kok@...el.com>
To:	Bruce Allen <ballen@...vity.phys.uwm.edu>
CC:	"Brandeburg, Jesse" <jesse.brandeburg@...el.com>,
	netdev@...r.kernel.org,
	Carsten Aulbert <carsten.aulbert@....mpg.de>,
	Henning Fehrmann <henning.fehrmann@....mpg.de>,
	Bruce Allen <bruce.allen@....mpg.de>
Subject: Re: e1000 full-duplex TCP performance well below wire speed

Bruce Allen wrote:
> Hi Jesse,
> 
>>> It's good to be talking directly to one of the e1000 developers and
>>> maintainers.  Although at this point I am starting to think that the
>>> issue may be TCP stack related and nothing to do with the NIC.  Am I
>>> correct that these are quite distinct parts of the kernel?
>>
>> Yes, quite.
> 
> OK.  I hope that there is also someone knowledgable about the TCP stack
> who is following this thread. (Perhaps you also know this part of the
> kernel, but I am assuming that your expertise is on the e1000/NIC bits.)
> 
>>> Important note: we ARE able to get full duplex wire speed (over 900
>>> Mb/s simulaneously in both directions) using UDP.  The problems occur
>>> only with TCP connections.
>>
>> That eliminates bus bandwidth issues, probably, but small packets take
>> up a lot of extra descriptors, bus bandwidth, CPU, and cache resources.
> 
> I see.  Your concern is the extra ACK packets associated with TCP.  Even
> those these represent a small volume of data (around 5% with MTU=1500,
> and less at larger MTU) they double the number of packets that must be
> handled by the system compared to UDP transmission at the same data
> rate. Is that correct?


A lot of people tend to forget that the pci-express bus has enough bandwidth on
first glance - 2.5gbit/sec for 1gbit of traffix, but apart from data going over it
there is significant overhead going on: each packet requires transmit, cleanup and
buffer transactions, and there are many irq register clears per second (slow
ioread/writes). The transactions double for TCP ack processing, and this all
accumulates and starts to introduce latency, higher cpu utilization etc...

Auke
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html