Date:	Sat, 22 Jan 2011 23:57:22 +0100
From:	Nicolas de Pesloüan 
	<nicolas.2p.debian@...il.com>
To:	Jay Vosburgh <fubar@...ibm.com>
CC:	Jiri Bohac <jbohac@...e.cz>,
	"bonding-devel@...ts.sourceforge.net" 
	<bonding-devel@...ts.sourceforge.net>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: Re: Bonding on bond

Le 20/01/2011 20:53, Jay Vosburgh a écrit :
> 	I'm in agreement that, by and large, nesting of bonds is
> pointless.  However, I suspect that there are users out in the world who
> are happily doing so, and this patch may shut them down.

Hi Jay,

I tested the following nested bonding configuration:

bond1 : eth1 + eth3, in balance-rr mode.
bond2 : eth0 + eth2, in balance-rr mode.
bond0 : bond1 + bond2, in active-backup mode.

The egress path apparently works not so bad, even if I didn't take time yet to check proper load 
balancing nor fail over.

However, the ingress path doesn't work at all. bond0 is unable to receive any packets (ARP or IP).

It doesn't sound surprising to me, having a look at the current code in __netif_receive_skb() :

>         /*
>          * bonding note: skbs received on inactive slaves should only
>          * be delivered to pkt handlers that are exact matches.  Also
>          * the deliver_no_wcard flag will be set.  If packet handlers
>          * are sensitive to duplicate packets these skbs will need to
>          * be dropped at the handler.
>          */
>         null_or_orig = NULL;
>         orig_dev = skb->dev;
>         master = ACCESS_ONCE(orig_dev->master);
>         if (skb->deliver_no_wcard)
>                 null_or_orig = orig_dev;
>         else if (master) {
>                 if (skb_bond_should_drop(skb, master)) {
>                         skb->deliver_no_wcard = 1;
>                         null_or_orig = orig_dev; /* deliver only exact match */
>                 } else
>                         skb->dev = master;
>         }

The skb_bond_should_drop() and skb->dev = master logic is only applied at a single level.

After this code, skb->dev is the master dev of the receiving dev, but skb->dev->master can be != 
NULL, if another level of bonding exists. Nothing obvious would cause the packet to be delivered to 
this possible higher level bonding interface (skb->dev->master).

Is something else expected to call __netif_receive_skb() again, with the current skb, to cause 
another level of bonding to be reachable? For as far as I understand, nothing will, but I might have 
missed something.

> 	I've not tested with nesting in a while; I know it used to work
> (at least for limited cases, typically an active-backup bond with a pair
> of balance-xor or balance-rr or sometimes 802.3ad enslaved to it), but
> has never really been a deliberate feature.  Is nesting now utterly
> broken, as suggested by the list of problems above?

I don't know whether someone really use nested bonding, but I can hardly imagine how one can have it 
works with current kernel, except for a pure egress application, without any feedback from the 
network. And such very specific application wouldn't even be able to receive an ARP reply...

> 	If nesting really doesn't work and is going to be disabled, then
> at a minimum it should also have an update to the documentation
> explaining this.

At least, we should explain that nesting bonding interfaces is known to be mostly broken and 
unsupported.

That being said, we still miss a way to achieve a simple configuration with several links doing load 
balancing to a switch and one or several links doing fail over to another switch, both switches 
*not* being 802.3ad capable.

Should we arrange for bonding to be allowed to nest, for this purpose, or should we find a way to 
setup this configuration with a single level of bonding ? I would prefer the second, but...

	Nicolas.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists