netdev - Re: Bonding gigabit and fast?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Pine.LNX.4.64.0812161331270.28806@t2.domain.actdsltmp>
Date:	Tue, 16 Dec 2008 18:53:10 -0800 (PST)
From:	Trent Piepho <tpiepho@...escale.com>
To:	"Tvrtko A. Ursulin" <tvrtko@...ulin.net>
cc:	netdev@...r.kernel.org
Subject: Re: Bonding gigabit and fast?

On Tue, 16 Dec 2008, Tvrtko A. Ursulin wrote:
> Does it make any sense from bandwith point of view to bond gigabit and fast
> ethernet?
>
> I wanted to use adaptive-alb mode to load balance both transmit and receive
> direction of traffic but it seems 8139too does not support it so I use
> balance-rr.

My experience from channel bonding in Beowulf clusters, which is now
somewhat dated wrt the current kernel, is that channel bonding with gigabit
didn't work well.

The problem is that at gigabit speeds, one IRQ per packet is very costly. 
So gigabit drivers have to use interrupt mitigation to transfer multiple
packets per interrupt (and/or use jumbo frames to move more data per
packet).

If the transmitter is sending out packets round-robin on two links, it's
sending the packets like this:

Link A:  1   3   5   7
Link B:    2   4   6   8

If the cards return two packets per interrupt, the receiver gets them like
this:

Link A:  1 3     5 7
Link B:      2 4     6 8

Well, the Linux kernel does not like getting the packets in the order (1 3
2 4 5 7 6 8).  It likes to get the packets in the correct order.  And so
performance suffers.  At the time, it suffered greatly.  Maybe it's better
or worse now?
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html