lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening PHC | |
Open Source and information security mailing list archives
| ||
|
Date: Fri, 27 Apr 2007 15:26:23 -0700 From: Bryan Lawver <lawver1@...l.gov> To: Rick Jones <rick.jones2@...com> Cc: "Michael S. Tsirkin" <mst@....mellanox.co.il>, general@...ts.openfabrics.org, Linux Network Development list <netdev@...r.kernel.org> Subject: Re: [ofa-general] Re: IPoIB forwarding I hit the IP NIC over the head with a hammer and turned off all offload features and I no longer get the super jumbo packet and I have symmetric performance. This NIC supported "ethtool -K ethx tso/tx/rx/sg on/off" and I am not sure at this time which one I needed to whack but all off solved the problem. Thanks for listening and re enforcing my search process. bryan At 01:32 PM 4/27/2007, Rick Jones wrote: >Bryan Lawver wrote: >>Your right about the ipoib module not combining packets (I believed you >>without checking) but I did never the less. The ipoib_start_xmit routine >>is definitely handed a "double packet" which means that the IP NIC >>driver or the kernel is combining two packets into a single super jumbo >>packet. This issue is irrespective of the IP MTU setting because I have >>set all interfaces to 9000k yet ipoib accepts and forwards this 17964 >>packet to the next IB node and onto the TCP stack where it is never >>acknowledged. This may not have come up in prior testing because I am >>using some of the fastest IP NICs which have no trouble keeping up with >>or exceeding the bandwidth of the IB side. This issue arises exactly >>every 8 packets...(ring buffer overrun??) >>I will be at Sonoma for the next few days as many on this list will be. > > >Some NICs (esp 10G) support large receive offload - they coalesce TCP >segments from the wire/fiber into larger ones they pass up the >stack. Perhaps that is happening here? > >I'm going to go out a bit on a limb, cross the streams, and include >netdev, because I suspect that if a system is acting as an IP router, one >doesn't want large receive offload enabled. That may need some discussion >in netdev - it may then require some changes to default settings or some >documentation enhancements. That or I'll learn that the stack is already >dealing with the issue... > >rick jones > >>bryan >> >>At 11:06 AM 4/26/2007, Michael S. Tsirkin wrote: >> >>> > Quoting Bryan Lawver <lawver1@...l.gov>: >>> > Subject: Re: IPoIB forwarding >>> > >>> > Here's a tcpdump of the same sequence. The TCP MSS is 8960 and it >>> appears >>> > that two payloads are queued at ipoib which combines them into a single >>> > 17920 payload with assumingly correct IP header (40) and IB header >>> > (4). The application or TCP stack does not acknowledge this double >>> packet >>> > ie. it does not ACK until each of the 8960 packets are resent >>> > individually. Being an IB newbie, I am guessing this combining is >>> > allowable but may violate TCP protocol. >>> >>>IPoIB does nothing like this - it's just a network device so >>>it sends all packets out as is. >>> >>>-- >>>MST >> >>_______________________________________________ >>general mailing list >>general@...ts.openfabrics.org >>http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >>To unsubscribe, please visit >>http://openib.org/mailman/listinfo/openib-general - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists