netdev - Re: vrf and ipsec xfrm routing problem

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <9893ae01-18a5-2afd-b485-459423b8adc0@candelatech.com>
Date:   Fri, 17 Jan 2020 13:52:31 -0800
From:   Ben Greear <greearb@...delatech.com>
To:     netdev <netdev@...r.kernel.org>
Cc:     David Ahern <dsahern@...il.com>
Subject: Re: vrf and ipsec xfrm routing problem

On 1/17/20 9:49 AM, Ben Greear wrote:
> Hello,
> 
> I'm back to mucking with xfrm and vrfs.  I am currently able to get the
> xfrm interface to connect to the ipsec peer and get an IP address.
> 
> But, when I bind a UDP socket to the x_eth4 xfrm device, the packets
> go out of eth4 instead.
> 
> Based on the problems I was having with multicast, I am thinking this might just be some routing problem.
> 
> # ip route show vrf _vrf4
> default via 192.168.5.1 dev eth4
> 192.168.5.0/24 dev eth4 scope link src 192.168.5.4
> 
> # ip addr show dev eth4
> 7: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master _vrf4 state UP group default qlen 1000
>      link/ether 00:30:18:01:63:eb brd ff:ff:ff:ff:ff:ff
>      inet 192.168.5.4/24 brd 192.168.5.255 scope global eth4
>         valid_lft forever preferred_lft forever
> 
> # ip addr show dev x_eth4
> 30: x_eth4@...4: <NOARP,UP,LOWER_UP> mtu 1440 qdisc noqueue master _vrf4 state UNKNOWN group default qlen 1000
>      link/none 00:30:18:01:63:eb brd ff:ff:ff:ff:ff:ff
>      inet 192.168.10.101/32 scope global x_eth4
>         valid_lft forever preferred_lft forever
>      inet6 fe80::f6ec:3e67:9b7b:60c9/64 scope link stable-privacy
>         valid_lft forever preferred_lft forever
> 
> 
> I tried adding a route to specify the x_frm as source, but that does not appear to work:
> 
> [root@...313-63e7 lanforge]# ip route add 192.168.10.0/24 via 192.168.5.1 dev x_eth4 table 4
> [root@...313-63e7 lanforge]# ip route show vrf _vrf4
> default via 192.168.5.1 dev eth4
> 192.168.5.0/24 dev eth4 scope link src 192.168.5.4
> 192.168.10.0/24 via 192.168.5.1 dev eth4
> 
> I also tried this, but no luck:
> 
> [root@...313-63e7 lanforge]# ip route add 192.168.10.0/24 via 192.168.10.1 dev x_eth4 table 4
> Error: Nexthop has invalid gateway.

I went looking for why this was failing.  The reason is that this code is hitting the error case
in the code snippet below (from 5.2.21+ kernel).

The oif is that of _vrf4, not the x_eth4 device.

David:  Is this expected behaviour?  Do you know how to tell vrf to use the x_eth4
xfrm device as oif when routing output to certain destinations?

	rcu_read_lock();
	{
		struct fib_table *tbl = NULL;
		struct flowi4 fl4 = {
			.daddr = nh->fib_nh_gw4,
			.flowi4_scope = scope + 1,
			.flowi4_oif = nh->fib_nh_oif,
			.flowi4_iif = LOOPBACK_IFINDEX,
		};

		/* It is not necessary, but requires a bit of thinking */
		if (fl4.flowi4_scope < RT_SCOPE_LINK)
			fl4.flowi4_scope = RT_SCOPE_LINK;

		if (table)
			tbl = fib_get_table(net, table);

		if (tbl)
			err = fib_table_lookup(tbl, &fl4, &res,
					       FIB_LOOKUP_IGNORE_LINKSTATE |
					       FIB_LOOKUP_NOREF);

		/* on error or if no table given do full lookup. This
		 * is needed for example when nexthops are in the local
		 * table rather than the given table
		 */
		if (!tbl || err) {
			err = fib_lookup(net, &fl4, &res,
					 FIB_LOOKUP_IGNORE_LINKSTATE);
		}

		if (err) {
			pr_err("daddr: 0x%x scope: %d  oif: %d iif: %d table: %d tbl: %p\n",
			       fl4.daddr, fl4.flowi4_scope, fl4.flowi4_oif, fl4.flowi4_iif, table, tbl);
			NL_SET_ERR_MSG(extack, "Nexthop has invalid gateway, table lookup");
			goto out;
		}
	}

Thanks,
Ben