netdev - Re: UDP ordering when using multiple rx queue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 11 Jul 2012 10:50:29 -0700
From:	Rick Jones <rick.jones2@...com>
To:	Jean-Michel Hautbois <jhautbois@...il.com>
CC:	Merav Sicron <meravs@...adcom.com>, netdev <netdev@...r.kernel.org>
Subject: Re: UDP ordering when using multiple rx queue

On 07/11/2012 06:41 AM, Jean-Michel Hautbois wrote:
> I confirm that using ethtool -L eth1 combined 1 solves my issue.

My being pedantic or not, you have kludged around your issue, which is a 
broken application.

Can you actually ass-u-me that this application is deployed with just a 
single back-to-back link between two systems?  I'm guessing that isn't 
the way it is deployed in production or there would be zero call for 
multicast.   There is *zero* guarantee of ordering with UDP, multicast 
or otherwise - certainly not between sends involving different port 
numbers, nor for that matter even between sends involving the same port 
numbers.  Once you leave the NIC (and perhaps even before) all bets are off.

Have you tested using bonded links?  Or through switches which 
themselves are joined by bonded links? Various bonding modes can even 
re-order traffic of a single flow (eg mode-rr).  As I understand it, the 
moves to "break the bottlenecks" imposed by spanning tree will mean that 
meshes of switches, even without bonded links, will send traffic of 
different flows through different paths through the switch fabric.  In 
those cases they might send traffic to the same multicast address along 
the same path each time, but you probably cannot count on that, nor them 
sending traffic to different multicast addresses along the same path. 
Some clever meshed-switch folks may go ahead and look up at the 
transport-layer port numbers when deciding on their splits - just like 
some bonding modes can.

Until you get the application re-written to handle out-of-order traffic, 
it "works" only by chance.

> Unicast traffic seems ok (I used netperf in order to check this assumption).

Netperf does nothing to check the order of datagrams.  It is perfectly 
content receiving datagrams in any order.  So you can use it to see that 
a single flow of UDP unicast is not split-up by the NIC (by looking at 
the per-queue stats) you can assume nothing about the final ordering of 
those UDP datagrams from a "successful" netperf UDP_STREAM test.

rick jones
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html