[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <d56867c5-9410-1256-eff0-48dfe0f25c5d@cumulusnetworks.com>
Date: Wed, 21 Nov 2018 12:26:11 -0700
From: David Ahern <dsa@...ulusnetworks.com>
To: Alexis Bauvin <abauvin@...leway.com>, roopa@...ulusnetworks.com
Cc: netdev@...r.kernel.org, akherbouche@...leway.com
Subject: Re: [RFC v3 0/3] Add VRF support for VXLAN underlay
On 11/21/18 6:30 AM, Alexis Bauvin wrote:
> Le 20 nov. 2018 à 22:45, David Ahern <dsa@...ulusnetworks.com> a écrit :
>>
>> On 11/20/18 7:23 AM, Alexis Bauvin wrote:
>>> We are trying to isolate the VXLAN traffic from different VMs with VRF as shown
>>> in the schemas below:
>>>
>>> +-------------------------+ +----------------------------+
>>> | +----------+ | | +------------+ |
>>> | | | | | | | |
>>> | | tap-red | | | | tap-blue | |
>>> | | | | | | | |
>>> | +----+-----+ | | +-----+------+ |
>>> | | | | | |
>>> | | | | | |
>>> | +----+---+ | | +----+----+ |
>>> | | | | | | | |
>>> | | br-red | | | | br-blue | |
>>> | | | | | | | |
>>> | +----+---+ | | +----+----+ |
>>> | | | | | |
>>> | | | | | |
>>> | | | | | |
>>> | +----+--------+ | | +--------------+ |
>>> | | | | | | | |
>>> | | vxlan-red | | | | vxlan-blue | |
>>> | | | | | | | |
>>> | +------+------+ | | +-------+------+ |
>>> | | | | | |
>>> | | VRF | | | VRF |
>>> | | red | | | blue |
>>> +-------------------------+ +----------------------------+
>>
>> Roopa and I were discussing this setup and are puzzled by the VRF
>> association here. Does br-red and br-blue have an address? The commands
>> below do not show it and from our perspective seems odd for this
>> scenario. If it does not have an address, then there is no reason for
>> the VRF labeling.
>
> Yes, br-red and br-blue have an address. Addresses (both MAC and IP)
> are anycast addresses, to make an anycast gateway for the VMs behind
> the taps. This serves for a classical evpn symmetric distributed
> routing, both between vxlans and towards external networks. I am using
> FRR as a control plane to exchange bgp-evpn information between
> hypervisors.
>
> The schematic could have been more complete, including more bridges
> and more taps in the vrf, with an added bridge+vxlan for the l3vni.
> I did not include them as I wanted to focus on the underlay itself
> and not on what vxlan was used for.
>
> Here is a more complete example:
> +---------+
> | |
> | vrf1000 |
> | |
> +--+-+-+--+
> | | |
> +---------------------+ | +---------------+
> | | |
> +------+------+ +------+------+ +----+---+
> | | | | | |
> | br0 | | br1 | | br1000 |
> | 10.0.0.1/24 | | 10.0.1.1/24 | | |
> | | | | +----+---+
> +----+-+-+----+ +-----+-+-----+ |
> | | | | | |
> +-------+ | +-------+ +--+ +---+ |
> | | | | | |
> +--+---+ +---+--+ +----+---+ +---+--+ +---+----+ +-----+-----+
> | | | | | | | | | | | |
> | tap0 | | tap1 | | vxlan0 | | tap2 | | vxlan1 | | vxlan1000 |
> | | | | | | | | | | | |
> +------+ +------+ +--------+ +------+ +--------+ +-----------+
> . . .
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> .
> . +---------+ +--------------+
> . | | | |
> . | vrf1001 | | vrf-underlay |
> . | | | |
> . +---+-+---+ +-------+------+
> . | | |
> . +---------+ +----------+ |
> . | | |
> . +------+------+ +----+---+ |
> . | | | | |
> . | br2 | | br1001 | |
> . | 10.0.2.1/24 | | | |
> . | | +----+---+ |
> . +----+-+-+----+ | |
> . | | | | |
> . +-----+ | +-------+ | |
> . | | | | +-------+-------+
> . +---+--+ +--+---+ +---+----+ +-----+-----+ | |
> . | | | | | | | | | eth0.2030 |
> . | tap3 | | tap4 | | vxlan2 | | vxlan1001 | | 172.16.0.2/24 |
> . | | | | | | | | | |
> . +------+ +------+ +--------+ +-----------+ +---------------+
> . . . ^ .
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> (vxlan lower device is eth0.2030) v
> +----------------+
> | |
> | eth0 |
> | 192.168.1.2/24 |
> | |
> +----------------+
>
> In this example, the underlay is rather simple. A more complex case
> could be with several uplinks in the underlay vrf, and a dummy/loopback
> with a /32 ip, used as a vxlan lower device and to bind routing daemons
> (e.g. bgpd).
>
> +--------------+
> | |
> | vrf-underlay |
> | |
> +-----+-+-+----+
> | | |
> +---------------+ | +---------------+
> | | |
> +-------+-------+ +-------+-------+ +-------+--------+
> | | | | | |
> | eth1 | | eth2 | | dummy0 |
> | 172.16.0.2/31 | | 172.16.0.4/31 | | 192.168.0.1/32 |
> | | | | | |
> +---------------+ +---------------+ +----------------+
> .
> . . . . . . . . . . . . . . . . . . . . . . .
> (vxlan lower device is dummy0)
Thanks for sending a more complete example. I forwarded it to some FRR
folks as well.
>
> As for default vrf, it would contain for example management traffic. We
> don’t want to mix the underlay routing tables to other traffics.
Management VRF? :-)
>
>> Also, it would be good to have a unit test this case. Can you create a
>> shell script that creates the setup and runs a few tests verifying
>> connectivity? You can use network namespaces and veth pairs in place of
>> the VM with a tap device. From there the functionality is the same.
>> Tests can be initial VRF association for the vxlan lower device,
>> changing the VRF to another device, and then changing again back to
>> default VRF - checking proper connectivity for each.
>
> Sure! I’ve just finished writing it, but I am not sure of the best way
> so send it. I guess I will have to add it to a v4 of the patches, in
> tools/testing/selftests/net?
yes, that would be good. Thank you for writing it.
Powered by blists - more mailing lists