netdev - Re: [ofa-general] Re: IPoIB forwarding

Open Source and information security mailing list archives

Message-ID: <46325DF3.2050203@hp.com>
Date:	Fri, 27 Apr 2007 13:32:51 -0700
From:	Rick Jones <rick.jones2@...com>
To:	Bryan Lawver <lawver1@...l.gov>
Cc:	"Michael S. Tsirkin" <mst@....mellanox.co.il>,
	general@...ts.openfabrics.org,
	Linux Network Development list <netdev@...r.kernel.org>
Subject: Re: [ofa-general] Re: IPoIB forwarding

Bryan Lawver wrote:
> Your right about the ipoib module not combining packets (I believed you 
> without checking) but I did never the less.  The ipoib_start_xmit 
> routine is definitely handed a "double packet"  which means that the IP 
> NIC driver or the kernel is combining two packets into a single super 
> jumbo packet.  This issue is irrespective of the IP MTU setting because 
> I have set all interfaces to 9000k yet  ipoib accepts and forwards this 
> 17964 packet to the next IB node and onto the TCP stack where it is 
> never acknowledged.  This may not have come up in prior testing because 
> I am using some of the fastest IP NICs which have no trouble keeping up 
> with or exceeding the bandwidth of the IB side.  This issue arises 
> exactly every 8 packets...(ring buffer overrun??)
> 
> I will be at Sonoma for the next few days as many on this list will be.

Some NICs (esp 10G) support large receive offload - they coalesce TCP segments 
from the wire/fiber into larger ones they pass up the stack.  Perhaps that is 
happening here?

I'm going to go out a bit on a limb, cross the streams, and include netdev, 
because I suspect that if a system is acting as an IP router, one doesn't want 
large receive offload enabled.  That may need some discussion in netdev - it may 
then require some changes to default settings or some documentation 
enhancements.  That or I'll learn that the stack is already dealing with the 
issue...

rick jones

> bryan
> 
> 
> 
> At 11:06 AM 4/26/2007, Michael S. Tsirkin wrote:
> 
>> > Quoting Bryan Lawver <lawver1@...l.gov>:
>> > Subject: Re: IPoIB forwarding
>> >
>> > Here's a tcpdump of the same sequence.  The TCP MSS is 8960 and it 
>> appears
>> > that two payloads are queued at ipoib which combines them into a single
>> > 17920 payload with assumingly correct IP header (40) and IB header
>> > (4).  The application or TCP stack does not acknowledge this double 
>> packet
>> > ie. it does not ACK until each of the 8960 packets are resent
>> > individually.  Being an IB newbie, I am guessing this combining is
>> > allowable but may violate TCP protocol.
>>
>> IPoIB does nothing like this - it's just a network device so
>> it sends all packets out as is.
>>
>> -- 
>> MST
> 
> 
> _______________________________________________
> general mailing list
> general@...ts.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit 
> http://openib.org/mailman/listinfo/openib-general

-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives