Message-ID: <0a920d2d-4e97-284b-9aad-54cf75bcb755@spamtrap.tnetconsulting.net>
Date:   Wed, 20 Jun 2018 23:13:16 -0600
From:   Grant Taylor <gtaylor@...tconsulting.net>
To:     Julian Anastasov <ja@....bg>, Akshat Kakkar <akshat.1984@...il.com>
Cc:     netdev <netdev@...r.kernel.org>,
        cronolog+lartc <cronolog+lartc@...glemail.com>,
        lartc <lartc@...r.kernel.org>,
        Erik Auerswald <auerswal@...x-ag.uni-kl.de>
Subject: Re: Route fallback issue

On 06/20/2018 01:00 PM, Julian Anastasov wrote:
> You can also try alternative routes.

"Alternative routes"?  I can't say as I've heard that description as a 
specific technique / feature / capability before.

Is that it's official name?

Where can I find out more about it?

> But as the kernel supports only default alternative routes, you can put 
> them in their own table:

I don't know that that is the case any more.

I was able to issue the following commands without a problem:

# ip route append 192.0.2.128/26 via 192.0.2.62
# ip route append 192.0.2.128/26 via 192.0.2.126

I crated two network namespaces and had a pair of vEths between them 
(192.0.2.0/26 and 192.0.2.64/26).  I added a dummy network to each NetNS 
(192.0.2.128/26 and 192.0.2.192/26).

I ran the following commands while a persistent ping was running from 
one NetNS to the IP on the other's dummy0 interface:

# ip link set ns2b up && ip route append 192.0.2.192/26 via 192.0.2.126 
&& ip link set ns2a down
(pause and watch things)
# ip link set ns2a up && ip route append 192.0.2.192/26 via 192.0.2.62 
&& ip link set ns2b down
(pause and watch things)

I could iterate between the two above commands and pings continued to work.

So, I think that it's now possible to use "alternate routes" (new to me) 
on specific prefixes in addition to the default.  Thus there is no 
longer any need for a separate table and the associated IP rule.

I'm running kernel version 4.9.76.

I did go ahead and set net.ipv4.conf.ns2b.ignore_routes_with_linkdown to 1.

for i in /proc/sys/net/ipv4/conf/*/ignore_routes_with_linkdown; do echo 
1 > $i; done

Doing that dropped the number of dropped pings from 60 ~ 90 (1 / second) 
to 0 ~ 5 (1 / second).  (Rarely, maybe 1 out of 20 flips, would it take 
upwards of 10 pings / seconds.)

> # Alternative routes use same metric!!!
> ip route append default via 192.168.1.254 dev eno1 table 100
> ip route append default via 192.168.2.254 dev eno2 table 100
> ip rule add prio 100 to 172.16.0.0/12 table 100

I did have to "append" the route.  I couldn't just "add" the route. 
When I tried to "add" the second route, I got an error about the route 
already existing.  Using "append" instead of "add" with everything else 
the same worked just fine.

Note:  I did go ahead and remove the single route that was added via 
"add" and used "append" for both.

> Of course, you will get better results if an user space tool puts only 
> alive routes in service after doing health checks of all near gateways.

I've got to say, with as well as this is working, I don't feel any need 
for a user space monitoring daemon.  I agree that I've felt the need for 
such in the past before I learned about "alternative routes".

I still want to learn more about "alternative routes".

Here's a diagram of the test network if someone wants to try to 
reproduce my findings:

+-------------+                +-------------+
| NS1         |                |         NS2 |
|        ns2a +-----vEth-A-----+ ns1a        |
|             |                |             |
+ dummy0      |                |      dummy0 +
|             |                |             |
|        ns2b +-----vEth-B-----+ ns1b        |
|             |                |             |
+-------------+                +-------------+

(vEths get the name of the NS that they face.)

NS1:ns2a     192.0.2.1     /26
NS1:ns2b     192.0.2.65    /26
NS1:dummy0   192.0.2.129   /26
NS2:ns1a     192.0.2.62    /26
NS2:ns1b     192.0.2.126   /26
NS2:dummy0   192.0.2.254   /26



-- 
Grant. . . .
unix || die

Powered by blists - more mailing lists