netdev - Re: v6/sit tunnels and VRFs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 12 Apr 2018 20:25:26 -0600
From:   David Ahern <dsahern@...il.com>
To:     Jeff Barnhill <0xeffeff@...il.com>
Cc:     netdev@...r.kernel.org
Subject: Re: v6/sit tunnels and VRFs

On 4/12/18 10:54 AM, Jeff Barnhill wrote:
> Hi David,
> 
> In the slides referenced, you recommend adding an "unreachable
> default" route to the end of each VRF route table.  In my testing (for
> v4) this results in a change to fib lookup failures such that instead
> of ENETUNREACH being returned, EHOSTUNREACH is returned since the fib
> finds the unreachable route, versus failing to find a route
> altogether.
> 
> Have the implications of this been considered?  I don't see a
> clean/easy way to achieve the old behavior without affecting non-VRF
> routing (eg. remove the unreachable route and delete the non-VRF
> rules).  I'm guessing that programmatically, it may not make much
> difference, ie. lookup fails, but for debugging or to a user looking
> at it, the difference matters.  Do you (or anyone else) have any
> thoughts on this?

We have recommended moving the local table down in the FIB rules:

# ip ru ls
1000:	from all lookup [l3mdev-table]
32765:	from all lookup local
32766:	from all lookup main
32767:	from all lookup default

and adding a default route to VRF tables:

# ip ro ls vrf red
unreachable default  metric 4278198272
172.16.2.0/24  proto bgp  metric 20
	nexthop via 169.254.0.1  dev swp3 weight 1 onlink
	nexthop via 169.254.0.1  dev swp4 weight 1 onlink

# ip -6 ro ls vrf red
2001:db8:2::/64  proto bgp  metric 20
	nexthop via fe80::202:ff:fe00:e  dev swp3 weight 1
	nexthop via fe80::202:ff:fe00:f  dev swp4 weight 1
anycast fe80:: dev lo  proto kernel  metric 0  pref medium
anycast fe80:: dev lo  proto kernel  metric 0  pref medium
fe80::/64 dev swp3  proto kernel  metric 256  pref medium
fe80::/64 dev swp4  proto kernel  metric 256  pref medium
ff00::/8 dev swp3  metric 256  pref medium
ff00::/8 dev swp4  metric 256  pref medium
unreachable default dev lo  metric 4278198272  error -101 pref medium

Over the last 2 years we have not seen any negative side effects to this
and is what you want to have proper VRF separation.

Without a default route lookups will proceed to the next fib rule which
means a lookup in the next table and barring other PBR rules will be the
main table. It will lead to wrong lookups.

Here is an example:
  ip netns add foo
  ip netns exec foo bash
  ip li set lo up
  ip li add red type vrf table 123
  ip li set red up
  ip li add dummy1 type dummy
  ip addr add 10.100.1.1/24 dev dummy1
  ip li set dummy1 master red
  ip li set dummy1 up
  ip li add dummy2 type dummy
  ip addr add 10.100.1.1/24 dev dummy2
  ip li set dummy2 up
  ip ro get 10.100.2.2
  ip ro get 10.100.2.2 vrf red

# ip ru ls
0:	from all lookup local
1000:	from all lookup [l3mdev-table]
32766:	from all lookup main
32767:	from all lookup default

# ip ro ls
10.100.1.0/24 dev dummy2 proto kernel scope link src 10.100.1.1
10.100.2.0/24 via 10.100.1.2 dev dummy2

# ip ro ls vrf red
10.100.1.0/24 dev dummy1 proto kernel scope link src 10.100.1.1

That's the setup. What happens on route lookups?
# ip ro get vrf red 10.100.2.1
10.100.2.1 via 10.100.1.2 dev dummy2 src 10.100.1.1 uid 0
    cache

which is clearly wrong. Let's look at the lookup sequence

# perf record -e fib:* ip ro get vrf red 10.100.2.1
10.100.2.1 via 10.100.1.2 dev dummy2 src 10.100.1.1 uid 0
    cache
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.003 MB perf.data (4 samples) ]

#  perf script --fields trace:trace
table 255 oif 2 iif 1 src 0.0.0.0 dst 10.100.2.1 tos 0 scope 0 flags 4
table 123 oif 2 iif 1 src 0.0.0.0 dst 10.100.2.1 tos 0 scope 0 flags 4
table 254 oif 2 iif 1 src 0.0.0.0 dst 10.100.2.1 tos 0 scope 0 flags 4
nexthop dev dummy2 oif 4 src 10.100.1.1

The first one is because I did not move the local table down.
The second one is the correct vrf lookup
The third one is the continuation to the next table - the main table.

Adding a default route:
# ip ro add vrf red unreachable default

And the lookup is proper:
# ip ro get vrf red 10.100.2.1
RTNETLINK answers: No route to host