netdev - Re: Veth pair swallow packets for XDP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <68645457-3A77-4AC2-A033-F09DB5AEE6F8@vmware.com>
Date:   Thu, 16 Jan 2020 22:54:29 +0000
From:   Hanlin Shi <hanlins@...are.com>
To:     Toshiaki Makita <toshiaki.makita1@...il.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>
CC:     Cheng-Chun William Tu <tuc@...are.com>
Subject: Re: Veth pair swallow packets for XDP_TX operation

Hi Toshiaki,

Thanks for your advice, and now it's working as expected in my environment. However I still have concerns on this issue. Is this dummy interface approach is a short-term work around? The behavior for native XDP is different from generic XDP, which could cause confusions for developers. Also, I'm planning to load the XDP program in container (specifically, Kubernetes pod), and I'm not sure is it's feasible for me to access the veth peer that is connected to the bridge (Linux bridge or ovs).

I wonder is that ok to have a fix, that if the XDP program on the peer of veth is not found, then fallback to a dummy XDP_PASS behavior, just like what you demonstrated? If needed I can help on the fix.

Thanks,
Hanlin

On 1/16/20, 1:01 AM, "Toshiaki Makita" <toshiaki.makita1@...il.com> wrote:

    Hi Hanlin,
    
    On 2020/01/16 7:35, Hanlin Shi wrote:
    > Hi community,
    > 
    > I’m prototyping an XDP program, and the hit issues with XDP_TX operation on veth device. The following code snippet is working as expected on 4.15.0-54-generic, but is NOT working on 4.20.17-042017-lowlatency (I got the kernel here: https://nam04.safelinks.protection.outlook.com/?url=https:%2F%2Fkernel.ubuntu.com%2F~kernel-ppa%2Fmainline%2Fv4.20.17%2F&amp;data=02%7C01%7Chanlins%40vmware.com%7Ccfd69717ebcd4251394908d79a62b253%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C1%7C637147621011879646&amp;sdata=47EGezqyQTyLkfFAoBlfYDrpvhPkTouMJi0ICbmcmfw%3D&amp;reserved=0).
    > 
    > Here’s my setup: I created a veth pair (namely veth1 and veth2), and put them in two namespaces (namely ns1 and ns2). I assigned address 60.0.0.1 on veth1 and 60.0.0.2 on veth2, set the device as the default interface in its namespace respectively (e.g. in ns1, do “ip r set default dev veth1”). Then in ns1, I ping 60.0.0.2, and tcpdump on veth1’s RX for ICMP.
    > 
    > Before loading any XDP program on veth2, I can see ICMP replies on veth1 interface. I load a program which do “XDP_TX” for all packets on veth2. I expect to see the same ICMP packet being returned, but I saw nothing.
    > 
    > I added some debugging message in the XDP program so I’m sure that the packet is processed on veth2, but on veth1, even with promisc mode on, I cannot see any ICMP packets or even ARP packets. In my understanding, 4.15 is using generic XDP mode where 4.20 is using native XDP mode for veth, so I guess there’s something wrong with veth native XDP and need some helps on fixing the issue.
    
    You need to load a dummy program to receive packets from peer XDP_TX when using native veth XDP.
    
    The dummy program is something like this:
    int xdp_pass(struct xdp_md *ctx) {
    	return XDP_PASS;
    }
    And load this program on "veth1".
    
    For more information please refer to this slides.
    https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnetdevconf.info%2F0x13%2Fsession.html%3Ftalk-veth-xdp&amp;data=02%7C01%7Chanlins%40vmware.com%7Ccfd69717ebcd4251394908d79a62b253%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C1%7C637147621011879646&amp;sdata=KIIkloefjX35ZzHk6bjx7pmO%2FRJRAB1KYw6PoQSuLmk%3D&amp;reserved=0
    
    Also there is a working example here.
    https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Ftools%2Ftesting%2Fselftests%2Fbpf%2Ftest_xdp_veth.sh&amp;data=02%7C01%7Chanlins%40vmware.com%7Ccfd69717ebcd4251394908d79a62b253%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C1%7C637147621011879646&amp;sdata=9BSgwjY%2Fn4MQ0YAFzIp6HAVLw86EmNQHb5BwSpuKS2k%3D&amp;reserved=0
    
    Toshiaki Makita
    
    > 
    > Please let me know if you need help on reproducing the issue.
    > 
    > Thanks,
    > Hanlin
    > 
    > PS: here’s the src code for the XDP program:
    > #include <stddef.h>
    > #include <string.h>
    > #include <linux/if_vlan.h>
    > #include <stdbool.h>
    > #include <bpf/bpf_endian.h>
    > #include <linux/if_ether.h>
    > #include <linux/ip.h>
    > #include <linux/tcp.h>
    > #include <linux/udp.h>
    > #include <linux/in.h>#define DEBUG
    > #include "bpf_helpers.h"
    > 
    > SEC("xdp")
    > int loadbal(struct xdp_md *ctx) {
    >    bpf_printk("got packet, direct return\n");
    >    return XDP_TX;
    > }char _license[] SEC("license") = "GPL";
    > 
    > "bpf_helpers.h" can be found here: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdropbox%2Fgoebpf%2Fraw%2Fmaster%2Fbpf_helpers.h&amp;data=02%7C01%7Chanlins%40vmware.com%7Ccfd69717ebcd4251394908d79a62b253%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C1%7C637147621011879646&amp;sdata=43ODisubHh%2BGLRDU2IcZTA4hujtLlVSzbcs5MhZpxCs%3D&amp;reserved=0
    >