netdev - packet seems disappeared after vxlan_rcv/gro_cells_receive/napi

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAK3+h2x2pxbU0BS=mxCZPJxy702BXFjJrQfvt4q9Ls=sijCo=w@mail.gmail.com>
Date:   Wed, 9 Jun 2021 16:50:18 -0700
From:   Vincent Li <vincent.mc.li@...il.com>
To:     netdev@...r.kernel.org
Subject: packet seems disappeared after vxlan_rcv/gro_cells_receive/napi_schedule(&cell->napi)

Hi Experts,

I am doing a tunnel communication  test between Cilium eBPF tunnel and
Linux VXLAN vni key based tunnel device (to simulate BIG-IP VXLAN vni
key based device), https://github.com/cilium/cilium/issues/16462

I came across a problem when packet is handled by
vxlan_rcv->gro_cells_receive->napi_schedule(&cell->napi),  the packet
seems getting dropped somewhere after that. I suspect I might have
done something wrong to setup the VXLAN device.

Here is how I setup the vxlan device  test on my Centos 8 with most
recent mainline git kernel build (5.13.0-rc4+ )

====start of the script====
#!/bin/bash


ip link add vxlan666 type vxlan id 666 dstport 8472 local
10.169.72.236 dev ens192 nolearning l2miss l3miss proxy

ip link set vxlan666 up

ip a add 10.0.4.236/24 dev vxlan666

ip route add 10.0.1.0/24 dev vxlan666  proto kernel  scope link  src 10.0.4.236

arp -i vxlan666 -s 10.0.1.17 6a:29:d2:78:63:7d

bridge fdb append 6a:29:d2:78:63:7d dst 10.169.72.238 dev vxlan666

====end of the script====

then I run tcpdump in the background

#tcpdump -nn -e -i vxlan666 &

and start to ping the IP 10.0.1.17 which is a POD IP in Cilium managed
K8S cluster

#ping -c 1 10.0.1.17


PING 10.0.1.17 (10.0.1.17) 56(84) bytes of data.

19:06:44.452994 d6:04:7c:b2:93:54 > 6a:29:d2:78:63:7d, ethertype IPv4
(0x0800), length 98: 10.0.4.236 > 10.0.1.17: ICMP echo request, id
1522, seq 1, length 64

19:06:44.454515 56:3d:9c:3a:09:78 > b2:1c:3c:57:9e:97, ethertype IPv4
(0x0800), length 98: 10.0.1.17 > 10.0.4.236: ICMP echo reply, id 1522,
seq 1, length 64


^C

--- 10.0.1.17 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms

You can see the tcpdump shows ICMP echo reply, but ping did not get
the ICMP echo reply  and shows 100% packet loss.

I added netdev_info log below and I can see the kernel log:

@@ -35,13 +39,17 @@ int gro_cells_receive(struct gro_cells *gcells,
struct sk_buff *skb)

        }


        __skb_queue_tail(&cell->napi_skbs, skb);
-       if (skb_queue_len(&cell->napi_skbs) == 1)
+       if (skb_queue_len(&cell->napi_skbs) == 1) {
+               netdev_info(skb->dev, "gro_cells_receive: napi_schedule\n");
                napi_schedule(&cell->napi);
+       }
+       netdev_info(skb->dev, "gro_cells_receive: NET_RX_SUCCESS\n");
        res = NET_RX_SUCCESS;

Jun  9 19:06:44 kernel-dev kernel: vxlan666: gro_cells_receive: napi_schedule

Jun  9 19:06:44 kernel-dev kernel: vxlan666: gro_cells_receive: NET_RX_SUCCESS


 I don't know where I have done wrong, any help is appreciated ! :)