netdev - Re: GSO/GRO and UDP performance

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <522A2C76.10203@openvpn.net>
Date:	Fri, 06 Sep 2013 13:26:46 -0600
From:	James Yonan <james@...nvpn.net>
To:	Rick Jones <rick.jones2@...com>
CC:	Eric Dumazet <eric.dumazet@...il.com>,
	netdev <netdev@...r.kernel.org>
Subject: Re: GSO/GRO and UDP performance

On 06/09/2013 10:42, Rick Jones wrote:
> On 09/06/2013 06:07 AM, Eric Dumazet wrote:
>> On Fri, 2013-09-06 at 03:22 -0600, James Yonan wrote:
>>
>>> So I think that playing well with GSO/GRO is essential to get speedup in
>>> UDP apps because of this 43x multiplier.
>>>
>>
>> Thats not true. GRO cannot aggregate more than 16+1 packets.

Where does the 16+1 come from?  I'm getting my 43x from the ratio of max 
legal IP packet size (64KB) / internet MTU (1500).  Are you saying that 
GRO cannot aggregate up to 64 KB?

>> I think we cannot aggregate UDP packets, because UDP lacks sequence
>> numbers, so reorders would be a problem.

>> You really need something that is not UDP generic.

Right -- that's why I'm proposing a hook for UDP GSO/GRO providers that 
know about specific app-layer protocols and can provide segmentation and 
aggregation methods for them.  Such a provider would be implemented in a 
kernel module and would know about the specific app-layer protocol, so 
it would be able to losslessly segment and aggregate it (i.e. it could 
use a sequence number from the app-layer protocol).

> It may  not be as sexy, and it cannot get the 43x multiplier (just what
> *is* the service demand change on a netperf TCP_STREAM test these days
> between GSO/GRO on and off anyway?)

That's something I haven't really looked too closely at yet.  With 
MAX_GRO_SKBS set to only 8, how well would this really scale?

> but looking for basic path-length reductions would be goodness.

Path is fairly optimized as-is.

Direction 1: udp_encap_recv -> tunnel decapsulation -> netif_rx

Direction 2: ndo_start_xmit -> tunnel encapsulation -> ip_local_out

I've also looked into getting closer to driver TX by using 
dev_queue_xmit instead of ip_local_out.

Even though this is a virtual driver without interrupts, I'm also 
looking at NAPI as a way of getting packet flows into GRO on the RX side.

Bottom line is that I want to saturate 10 GigE with UDP packets without 
breaking a sweat.  ixgbe or other drivers in that class can handle it if 
the per-packet overhead in the network stack can be reduced enough.

James
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html