netdev - VRF + ip xfrm, egress ESP packet looping when qdisc configured

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAHgT=KfpKenfzn3+uiVdF-B3mGv30Ngu70y6Zn+wH0GcGcDFYQ@mail.gmail.com>
Date:   Thu, 2 Jan 2020 18:11:54 -0500
From:   Trev Larock <trev@...ock.ca>
To:     netdev@...r.kernel.org
Subject: VRF + ip xfrm, egress ESP packet looping when qdisc configured

With a vrf configured and an xfrm policy I see some ESP packet looping,
only with qdisc.  Tried on:
fedora31 kernel 5.3.7-301.fc31.x86_64
fedora26 kernel 4.16.11

1. VRF case, host-host tunnel mode xfrm, no qdisc
          host1                                |  host2
         +---------------+                     |
         |     vrf0      |                     |
         +---------------+                     |
            |                                  |
            |                                  |
         +--------+                            |
         | enp0s8 | 192.168.56.14 --------------- 192.168.56.16
         +--------+                            |
                                               |
vrf config:
 sysctl net.ipv4.tcp_l3mdev_accept=1
 ip link add dev vrf0 type vrf table 300
 ip link set dev vrf0 up
 ip link set dev enp0s8  master vrf0

xfrm config:
 ip xfrm policy add src 192.168.56.114/32 dst 192.168.56.116/32 \
 dir out priority 367231 ptype main tmpl src 192.168.56.114 dst \
 192.168.56.116 proto esp spi 0x1234567 reqid 1 mode tunnel

 ip xfrm state add src 192.168.56.114 dst 192.168.56.116 proto esp \
 spi 0x1234567 reqid 1 mode tunnel aead rfc4106\(gcm\(aes\)\) \
 0x68db8eabd7f61557247f28f95e668f19855e086d02b21488fde4f5fcc9d42fcfbc9a2e35 \
 128 sel src 192.168.56.114/32 dst 192.168.56.116/32

(No namespace or virtual xfrm interface config involved).

ping -c 1 -w 1 -I vrf0 192.168.56.116
tcpdump -n -i enp0s8
05:01:27.085768 IP 192.168.56.114 > 192.168.56.116:
ESP(spi=0x01234567,seq=0x1), length 120
(ESP packet goes out ok)

2.  VRF + qdisc
If activating qdisc, there is increasing sized 'looping' ESP packet:
tc qdisc add dev vrf0 root netem delay 0ms

tcpdump -n -i enp0s8
(shows nothing)

tcpdump -n -i vrf0
05:08:22.583088 IP 192.168.56.114 > 192.168.56.116: ICMP echo request,
id 8873, seq 1, length 64
05:08:22.583155 IP 192.168.56.114 > 192.168.56.116:
ESP(spi=0x01234567,seq=0xe), length 120
05:08:22.583163 IP 192.168.56.114 > 192.168.56.116:
ESP(spi=0x01234567,seq=0xf), length 176
05:08:22.583168 IP 192.168.56.114 > 192.168.56.116:
ESP(spi=0x01234567,seq=0x10), length 232
05:08:22.583172 IP 192.168.56.114 > 192.168.56.116:
ESP(spi=0x01234567,seq=0x11), length 288
05:08:22.583177 IP 192.168.56.114 > 192.168.56.116:
ESP(spi=0x01234567,seq=0x12), length 344
05:08:22.583182 IP 192.168.56.114 > 192.168.56.116:
ESP(spi=0x01234567,seq=0x13), length 400

Transport mode is same behavior.  Anyone have reference config for vrf + xfrm?
Adding "dev vrf0" to the xfrm policy/state yields cleartext pings as the
oif for xfrm_lookup is enp0s8.

Thanks,
Trev