Message-ID: <54D1EDCE.3030602@skyatlas.com>
Date:	Wed, 04 Feb 2015 12:00:46 +0200
From:	Eren Türkay <erent@...atlas.com>
To:	Eric Dumazet <eric.dumazet@...il.com>
CC:	Network Development <netdev@...r.kernel.org>
Subject: Re: PROBLEM: Bnx2x Checksum/Length Error Over GRE Tunnel

On 03-02-2015 20:46, Eric Dumazet wrote:
> Can you please send : ifconfig -a, as I suspect an MTU issue

You are right. Previously I set gre interface MTU to 1500 and NIC interface had 9000 MTU.
I was able to get 3Gbit in a single flow, and 5-6Gbit in parallel. When I recreated the interfaces,
the MTU of gre was set 8976 and I can get 9.5Gbit in a single flow. The reason why I set 1500 on
gre interface was to mimic openvswitch setup as the main problem I have been investigating is VM
network performance problem (they have 3Gbit throughput). I have OpenStack installation using OVS with GRE
and those interfaces are set 1500 MTU by default. I tried to test iproute2 implementation with 1500.

With correct MTU setting in gre interface, it seems to work without checksumming and offloading
but I haven't tested my original OVS setting. I will tweak OVS bridge MTU settings and try to get
it work. If baremetal-to-baremetal works, I suspect there is an issue with OVS setup.
However, when I enable tx on, netperf cannot even connect.

"ifconfig -a" and "ovs-vsctl show" result is attached.

I am not sure if tx offloading will affect the VM performance. Regardless of OVS problem, there seems to be a
bug with segmentation offloading as it's only possible to use the interface with "tx off". I have compiled the
kernel with debugging on and I can enable debugging on related parts in the kernel for further information.

> Here I easily reach line rate with a single flow.
> 
> ip tunnel add gre1 mode gre remote 10.246.11.52 local 10.246.11.51
> ip link set gre1 up
> ip addr add 7.7.8.51/24 dev gre1

ip tunnel add gre1 mode gre remote 10.20.0.162 local 10.20.0.161
ip l set gre1 up
ip addr add 7.7.8.161/24 dev gre1

root@...pute1vx:~# ip r get 7.7.8.162
7.7.8.162 dev gre1  src 7.7.8.161 
    cache  expires 539sec mtu 8976


root@...pute1vx:~# ethtool -K em1 tx off
Actual changes:
tx-checksumming: off
        tx-checksum-ipv4: off
        tx-checksum-ipv6: off
tcp-segmentation-offload: off
        tx-tcp-segmentation: off [requested on]
        tx-tcp-ecn-segmentation: off [requested on]
        tx-tcp6-segmentation: off [requested on]

root@...pute1vx:~# netperf -H 7.7.8.162 -Cc                                                                                                                                                                                                   
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 7.7.8.162 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

 87380  16384  16384    10.00      9534.00   4.05     4.39     0.835   0.906  


root@...pute1vx:~# ethtool -K em1 tx on
Actual changes:
tx-checksumming: on
        tx-checksum-ipv4: on
        tx-checksum-ipv6: on
tcp-segmentation-offload: on
        tx-tcp-segmentation: on
        tx-tcp-ecn-segmentation: on
        tx-tcp6-segmentation: on

root@...pute1vx:~# netperf -H 7.7.8.162 -Cc
establish control: are you sure there is a netserver listening on 7.7.8.162 at port 12865?
establish_control could not establish the control connection from 0.0.0.0 port 0 address family AF_UNSPEC to 7.7.8.162 port 12865 address family AF_INET

Regards,
Eren

-- 
System Administrator
https://skyatlas.com/

View attachment "ifconfig-and-ovs-result.txt" of type "text/plain" (6306 bytes)

Powered by blists - more mailing lists