netdev - Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20070826.190420.41652839.davem@davemloft.net>
Date:	Sun, 26 Aug 2007 19:04:20 -0700 (PDT)
From:	David Miller <davem@...emloft.net>
To:	jheffner@....edu
Cc:	billfink@...dspring.com, rick.jones2@...com, hadi@...erus.ca,
	krkumar2@...ibm.com, gaagaan@...il.com,
	general@...ts.openfabrics.org, herbert@...dor.apana.org.au,
	jagana@...ibm.com, jeff@...zik.org, johnpol@....mipt.ru,
	kaber@...sh.net, mcarlson@...adcom.com, mchan@...adcom.com,
	netdev@...r.kernel.org, peter.p.waskiewicz.jr@...el.com,
	rdreier@...co.com, Robert.Olsson@...a.slu.se,
	shemminger@...ux-foundation.org, sri@...ibm.com, tgraf@...g.ch,
	xma@...ibm.com
Subject: Re: [PATCH 0/9 Rev3] Implement batching skb API and support in
 IPoIB

From: John Heffner <jheffner@....edu>
Date: Sun, 26 Aug 2007 21:32:26 -0400

> There are a few interesting things here.  For one, the bursts caused by 
> TSO seem to be causing the receiver to do stretch acks.  This may have a 
> negative impact on flow performance, but it's hard to say for sure how 
> much.  Interestingly, it will even further reduce the CPU load on the 
> sender, since it has to process fewer acks.
> 
> As I suspected, in the non-TSO case the receiver gets lots of packets 
> directly queued to user.  This should result in somewhat lower CPU 
> utilization on the receiver.  I don't know if it can account for all the 
> difference you see.

I had completely forgotten these stretch ACK and ucopy issues.

When the receiver gets inundated with a backlog of receive
queue packets, it just spins there copying into userspace
_every_ _single_ packet in that queue, then spits out one ACK.

Meanwhile the sender has to pause long enough for the pipe to empty
slightly.

The transfer is much better behaved if we ACK every two full sized
frames we copy into the receiver, and therefore don't stretch ACK, but
at the cost of cpu utilization.

These effects are particularly pronounced on systems where the
bus bandwidth is also one of the limiting factors.
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html