Message-ID: <26548921.1r3eYUQgxm@benoit.monin>
Date: Thu, 05 Sep 2024 17:22:55 +0200
From: Benoît Monin <benoit.monin@....fr>
To: "David S. Miller" <davem@...emloft.net>, David Ahern <dsahern@...nel.org>,
 Eric Dumazet <edumazet@...gle.com>, Jakub Kicinski <kuba@...nel.org>,
 Paolo Abeni <pabeni@...hat.com>
Cc: netdev@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: PROBLEM: invalid udp checksum with ip6gre-in-udp

Hi all,

I am having issue with GRE-in-UDP (GRE with fou encapsulation) over 
IPv6: the outer UDP checksum is only valid if an inner checksum is 
present and valid. This problem is only present over IPv6, not IPv4, 
and reproducible with different kernel versions and cpu architecture.

Here is the test setup I used:

    +----------+                                    +------------+
    |          | fd00::2/64              fd00::1/64 |            |
    |  tester  | 10.0.0.33/24          10.0.0.11/24 |    DUT     |
    |          |------------------------------------|            |
    +----------+                                    +------------+

Two machines are connected with an ethernet cable, and each side is 
setup with an ipv4 and an ipv6 address.

On the device under test, two GRE-in-UDP tunnels are setup, one over 
ipv6 and one over ipv4, aimed at the tester. Each tunnel is configured 
with an ipv4 address:

    modprobe -a fou fou6

    ip link add gre6 type ip6gre local fd00::1 remote fd00::2 encap fou encap-sport 1234 encap-dport 4567
    ip link set up gre6
    ip address add 172.20.6.1/24 dev gre6

    ip link add gre4 type gre local 10.0.0.11 remote 10.0.0.33 encap fou encap-sport 3456 encap-dport 6789 encap-csum
    ip link set up gre4
    ip address add 172.20.4.1/24 dev gre4

On the tester, no setup is done, it is only running tcpdump to capture 
the traffic emitted by the DUT.

The following commands are used on the device under test to send 
packets via the tunnels:

    ping -c1 -W0.1 172.20.6.2
    ./send_udp 172.20.6.2 5555 "ip6gre-in-udp with inner udp csum"
    SO_NO_CHECK=1 ./send_udp 172.20.6.2 5555 "ip6gre-in-udp without inner udp csum"
    ping -c1 -W0.1 172.20.4.2
    ./send_udp 172.20.4.2 5555 "gre-in-udp with inner udp csum"
    SO_NO_CHECK=1 ./send_udp 172.20.4.2 5555 "gre-in-udp without inner udp csum"

Three packets are sent in each GRE tunnels :
* One ICMP echo request
* One UDP packet with a valid checksum
* One UDP packet with the checksum set to 0

Here is a link to the pcap containing the six packets generated by the 
previous commands:
https://onaip.mooo.com/pub/tmp/bug_ip6gre-in-udp.pcap

Some details about the captured packets:
IP6 fd00::1 > fd00::2: 1234 > 4567: [bad udp cksum 0xfa75 -> 0xe680!]
IP6 fd00::1 > fd00::2: 1234 > 4567: [udp sum ok]
IP6 fd00::1 > fd00::2: 1234 > 4567: [bad udp cksum 0xfa62 -> 0xb57c!]
IP 10.0.0.11.3456 > 10.0.0.33.6789: [udp sum ok]
IP 10.0.0.11.3456 > 10.0.0.33.6789: [udp sum ok]
IP 10.0.0.11.3456 > 10.0.0.33.6789: [udp sum ok]

For the tunnel over ipv6, only the UDP packet sent with a valid 
checksum get encapsulated with a valid checksum. For the ping and the 
UDP packet with a zero checksum, the outer UDP checksum matches the 
partial checksum of the pseudo-header.

For ipv4, the UDP checksum of the encapsulation are all valid.

The device under test used for the capture is an x86-64 machine with a 
realtek ethernet adapter (r8169 driver) running a 6.10.7 kernel.

The problem was also seen on an arm64 board (freescale ls1046) with 
dpaa ethernet driver running a 4.14 kernel. on this hardware, the 
packets that would have an invalid checksum are not emitted and the tx 
error counter of the ethernet interface increases.

In all cases, disabling hardware checksumming for ipv6 with ethtool can 
be used as a work-around:

    ethtool -K eth0 tx-checksum-ipv6 off

This and the partial checksum value seems to point to an error in the 
handling of hardware checksumming in the particular case of fou6 
encapsulation, but I have not been able to figure out what could be 
causing it.

Did I miss a configuration parameter for ip6gre-in-udp? Any advise on 
how to debug that would be appreciated.

-- 
Benoît

Powered by blists - more mailing lists