[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <4EF45E79.6020803@bootc.net>
Date: Fri, 23 Dec 2011 10:56:57 +0000
From: Chris Boot <bootc@...tc.net>
To: Nicolas de Pesloüan
<nicolas.2p.debian@...il.com>
CC: netdev <netdev@...r.kernel.org>
Subject: Re: igb + balance-rr + bridge + IPv6 = no go without promiscuous
mode
On 23/12/2011 10:48, Nicolas de Pesloüan wrote:
> [ Forwarded to netdev, because two previous e-mail erroneously sent in
> HTML ]
>
> Le 23/12/2011 11:15, Chris Boot a écrit :
>> On 23/12/2011 09:52, Nicolas de Pesloüan wrote:
>>>
>>>
>>> Le 23 déc. 2011 10:42, "Chris Boot" <bootc@...tc.net
>>> <mailto:bootc@...tc.net>> a écrit :
>>> >
>>> > Hi folks,
>>> >
>>> > As per Eric Dumazet and Dave Miller, I'm opening up a separate
>>> thread on this issue.
>>> >
>>> > I have two identical servers in a cluster for running KVM virtual
>>> machines. They each have a
>>> single connection to the Internet (irrelevant for this) and two
>>> gigabit connections between each
>>> other for cluster replication, etc... These two connections are in a
>>> balance-rr bonded connection,
>>> which is itself member of a bridge that the VMs attach to. I'm
>>> running v3.2-rc6-140-gb9e26df on
>>> Debian Wheezy.
>>> >
>>> > When the bridge is brought up, IPv4 works fine but IPv6 does not.
>>> I can use neither the
>>> automatic link-local on the brid ge nor the static global address I
>>> assign. Neither machine can
>>> perform neighbour discovery over the link until I put the bond
>>> members (eth0 and eth1) into
>>> promiscuous mode. I can do this either with tcpdump or 'ip link set
>>> dev ethX promisc on' and this
>>> is enough to make the link spring to life.
>>>
>>> For as far as I remember, setting bond0 to promisc should set the
>>> bonding member to promisc too.
>>> And inserting bond0 into br0 should set bond0 to promisc... So
>>> everything should be in promisc
>>> mode anyway... but you shoudn't have to do it by hand.
>>>
>>
>> Sorry, I should have added that I tried this. Setting bond0 or br0 to
>> promisc has no effect. I
>> discovered this by running tcpdump on br0 first, then bond0, then
>> eventually each bond member in
>> turn. Only at the last stage did things jump to life.
>>
>>> >
>>> > This cluster is not currently live so I can easily test patches
>>> and various configurations.
>>>
>>> Can you try to remove the bonding part, connecting eth0 and eth1
>>> directly to br0 and see if it
>>> works better? (This is a test ony. I perfectly understand that you
>>> would loose balance-rr in this
>>> setup.)
>>>
>>
>> Good call. Let's see.
>>
>> I took br0 and bond0 apart, took eth0 and eth1 out of enforced
>> promisc mode, then manually built a
>> br0 with eth0 in only so I didn't cause a network loop. Adding eth0
>> to br0 did not make it go into
>> promisc mode, but IPv6 does work over this setup. I also made sure ip
>> -6 neigh was empty on both
>> machines before I started.
>>
>> I then decided to try the test with just the bond0 in balance-rr
>> mode. Once again I took everything
>> down and ensured no promisc mode and no ip -6 neigh. I noticed bond0
>> wasn't getting a link-local and
>> I found out for some reason
>> /proc/sys/net/ipv6/conf/bond0/disable_ipv6 was set on both servers so I
>> set it to 0. That brought things to life.
>>
>> So then I put it all back together again and it didn't work. I once
>> again noticed disable_ipv6 was
>> set on the bond0 interfaces, now part of the bridge. Toggling this on
>> the _bond_ interface made
>> things work again.
>>
>> What's setting disable_ipv6? Should this be having an impact if the
>> port is part of a bridge?
Hmm, as a further update... I brought up my VMs on the bridge with
disable_ipv6 turned off. The VMs on one host couldn't see what was on
the other side of the bridge (on the other server) until I turned
promisc back on manually. So it's not entirely disable_ipv6's fault.
Chris
--
Chris Boot
bootc@...tc.net
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists