netdev - Re: [PATCH] - vxlan: gro not effective for intel 82599

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <64eee574bcb29f8ec0a6c6536a910d01@imap.linux.ibm.com>
Date:	Mon, 29 Jun 2015 12:56:47 -0700
From:	Ramu Ramamurthy <sramamur@...ux.vnet.ibm.com>
To:	Tom Herbert <tom@...bertland.com>
Cc:	Or Gerlitz <gerlitz.or@...il.com>,
	"David S. Miller" <davem@...emloft.net>,
	Jiri Benc <jbenc@...hat.com>, James Morris <jmorris@...ei.org>,
	Linux Kernel Network Developers <netdev@...r.kernel.org>,
	pradeeps@...ux.vnet.ibm.com, J Kidambi <jkidambi@...ibm.com>
Subject: Re: [PATCH] - vxlan: gro not effective for intel 82599

On 2015-06-28 14:17, Tom Herbert wrote:
> On Sun, Jun 28, 2015 at 1:19 PM, Or Gerlitz <gerlitz.or@...il.com> 
> wrote:
>> On Fri, Jun 26, 2015 at 10:59 PM, Tom Herbert <tom@...bertland.com> 
>> wrote:
>> [...]
>>> Looks like GRO was never implemented for vxlan tunnels. The driver is
>>> simply calling netif_rx instead of using the GRO cells 
>>> infrastructure.
>>> geneve is doing the same thing. For other tunnels which are used in
>>> foo-over-udp (GRE, IPIP, SIT) ip_tunnel_rcv is called which in turn
>>> calls gro_cells_receive.
>> 
>> Tom,
>> 
>> Since v3.14, when a tunneled (say VXLAN/GRE) packets are received on
>> the physical interface, they go through GRO aggregation before being
>> delivered up to the tunnel "device" (e.g either vxlan/gre netdevice or
>> OVS vxlan/gre vport) -- so in that respect, can you elaborate a little
>> further why we want to GRO them again?
>> 
> 
> If we don't have a verifiable checksum from the device GRO is not
> applied to UDP encapsulated packets at the physical interface, but can
> be done at the tunnel. Ramu is seeing poor performance because there
> is no GRO at all is happening, so doing it at the tunnel is an
> improvement. As I described before, avoiding checksum calculation in
> the device NAPI still seems to be a good thing (in my testing I do see
> a slight regression if we were to do the checksum in device NAPI).
> 
> btw, the real "fix" for this is for NICs to provide CHECKSUM_COMPLETE! 
> :-)
> 
> Tom
> 
>> Or.

When I force the sender to set a non-zero UDP checksum for vxlan 
encapsulated tcp-stream,
then, I can see the gro activated at the receiver (82599ES nic),
and the throughput is ~8.5Gbps !

So, to get gro to be effective for the 82599ES receiver, the sender 
needs to set the UDP
checksum.  If the sender does NOT set the UDP checksum (udp-checksum == 
0), then the gro-cells patch suggested by Tom
will perform gro at the tunnel device level.


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html