netdev - Re: vrf and ipsec xfrm routing problem

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 17 Jan 2020 16:34:08 -0800
From:   Ben Greear <greearb@...delatech.com>
To:     netdev <netdev@...r.kernel.org>
Cc:     David Ahern <dsahern@...il.com>
Subject: Re: vrf and ipsec xfrm routing problem

On 1/17/20 1:52 PM, Ben Greear wrote:
> On 1/17/20 9:49 AM, Ben Greear wrote:
>> Hello,
>>
>> I'm back to mucking with xfrm and vrfs.  I am currently able to get the
>> xfrm interface to connect to the ipsec peer and get an IP address.
>>
>> But, when I bind a UDP socket to the x_eth4 xfrm device, the packets
>> go out of eth4 instead.
>>
>> Based on the problems I was having with multicast, I am thinking this might just be some routing problem.
>>
>> # ip route show vrf _vrf4
>> default via 192.168.5.1 dev eth4
>> 192.168.5.0/24 dev eth4 scope link src 192.168.5.4
>>
>> # ip addr show dev eth4
>> 7: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master _vrf4 state UP group default qlen 1000
>>      link/ether 00:30:18:01:63:eb brd ff:ff:ff:ff:ff:ff
>>      inet 192.168.5.4/24 brd 192.168.5.255 scope global eth4
>>         valid_lft forever preferred_lft forever
>>
>> # ip addr show dev x_eth4
>> 30: x_eth4@...4: <NOARP,UP,LOWER_UP> mtu 1440 qdisc noqueue master _vrf4 state UNKNOWN group default qlen 1000
>>      link/none 00:30:18:01:63:eb brd ff:ff:ff:ff:ff:ff
>>      inet 192.168.10.101/32 scope global x_eth4
>>         valid_lft forever preferred_lft forever
>>      inet6 fe80::f6ec:3e67:9b7b:60c9/64 scope link stable-privacy
>>         valid_lft forever preferred_lft forever
>>
>>
>> I tried adding a route to specify the x_frm as source, but that does not appear to work:
>>
>> [root@...313-63e7 lanforge]# ip route add 192.168.10.0/24 via 192.168.5.1 dev x_eth4 table 4
>> [root@...313-63e7 lanforge]# ip route show vrf _vrf4
>> default via 192.168.5.1 dev eth4
>> 192.168.5.0/24 dev eth4 scope link src 192.168.5.4
>> 192.168.10.0/24 via 192.168.5.1 dev eth4
>>
>> I also tried this, but no luck:
>>
>> [root@...313-63e7 lanforge]# ip route add 192.168.10.0/24 via 192.168.10.1 dev x_eth4 table 4
>> Error: Nexthop has invalid gateway.

So, looks like all I need to do is to pull the xfrm device out of the vrf, and now
traffic is working.  Possibly I need to put the xfrm in its own vrf, I'll need to
test a more complex case to determine that.

I will clean up my test bed and scripts and make sure this is reproducible.

Thanks,
Ben

> 
> I went looking for why this was failing.  The reason is that this code is hitting the error case
> in the code snippet below (from 5.2.21+ kernel).
> 
> The oif is that of _vrf4, not the x_eth4 device.
> 
> David:  Is this expected behaviour?  Do you know how to tell vrf to use the x_eth4
> xfrm device as oif when routing output to certain destinations?
> 
>      rcu_read_lock();
>      {
>          struct fib_table *tbl = NULL;
>          struct flowi4 fl4 = {
>              .daddr = nh->fib_nh_gw4,
>              .flowi4_scope = scope + 1,
>              .flowi4_oif = nh->fib_nh_oif,
>              .flowi4_iif = LOOPBACK_IFINDEX,
>          };
> 
>          /* It is not necessary, but requires a bit of thinking */
>          if (fl4.flowi4_scope < RT_SCOPE_LINK)
>              fl4.flowi4_scope = RT_SCOPE_LINK;
> 
>          if (table)
>              tbl = fib_get_table(net, table);
> 
>          if (tbl)
>              err = fib_table_lookup(tbl, &fl4, &res,
>                             FIB_LOOKUP_IGNORE_LINKSTATE |
>                             FIB_LOOKUP_NOREF);
> 
>          /* on error or if no table given do full lookup. This
>           * is needed for example when nexthops are in the local
>           * table rather than the given table
>           */
>          if (!tbl || err) {
>              err = fib_lookup(net, &fl4, &res,
>                       FIB_LOOKUP_IGNORE_LINKSTATE);
>          }
> 
>          if (err) {
>              pr_err("daddr: 0x%x scope: %d  oif: %d iif: %d table: %d tbl: %p\n",
>                     fl4.daddr, fl4.flowi4_scope, fl4.flowi4_oif, fl4.flowi4_iif, table, tbl);
>              NL_SET_ERR_MSG(extack, "Nexthop has invalid gateway, table lookup");
>              goto out;
>          }
>      }
> 
> Thanks,
> Ben
>