netdev - sit tunnel slow since added GSO/TSO support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 13 Dec 2013 17:13:31 +0100
From:	Michal Schmidt <mschmidt@...hat.com>
To:	"netdev@...r.kernel.org" <netdev@...r.kernel.org>
CC:	Eric Dumazet <edumazet@...gle.com>
Subject: sit tunnel slow since added GSO/TSO support

Hello,

I've been experimenting with IPIP and SIT tunnels. Both of them
had the support for GSO/TSO added recently, by commits:
  cb32f511a7 "ipip: add GSO/TSO support"
  61c1db7fae "ipv6: sit: add GSO/TSO support"

IPIP works fine for me.

I am seeing very low TCP performance with SIT if TSO is enabled on the
SIT interface. It does not matter if the underlying hardware supports
NETIF_F_GSO_SIT or not (I have tested with bnx2x and tg3).

This is for example with TSO on a 1Gb interface:

  $ netperf -H 2001:0DB8:1234::000e
  MIGRATED TCP STREAM TEST from ::0 (::) port 0 AF_INET6 to 2001:0DB8:1234::000e () port 0 AF_INET6
  Recv   Send    Send                          
  Socket Socket  Message  Elapsed              
  Size   Size    Size     Time     Throughput  
  bytes  bytes   bytes    secs.    10^6bits/sec  

   87380  16384  16384    10.00      35.92

After "ethtool -K sittun tso off":

  $ netperf -H 2001:0DB8:1234::000e
  MIGRATED TCP STREAM TEST from ::0 (::) port 0 AF_INET6 to 2001:0DB8:1234::000e () port 0 AF_INET6
  Recv   Send    Send                          
  Socket Socket  Message  Elapsed              
  Size   Size    Size     Time     Throughput  
  bytes  bytes   bytes    secs.    10^6bits/sec  

   87380  16384  16384    10.01     706.86

After looking into what was happening I discovered that in the
TSO-enabled case big skbs enters into ipip6_tunnel_xmit() where they are
dropped by this branch:

        if (df) {
                /* ... */
                if (skb->len > mtu) {
                        icmpv6_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu);
                        ip_rt_put(rt);
                        goto tx_error;
                }
        }

After having looked at the meaning of "df" I found that I can use
the "nopmtudisc" option when creating the sit tunnel to work around
the problem:
  ip tun add sittun mode sit remote $REMOTE local $LOCAL dev $IFACE nopmtudisc

With "nopmtudisc" and TSO enabled:

  $ netperf -H 2001:0DB8:1234::000e
  MIGRATED TCP STREAM TEST from ::0 (::) port 0 AF_INET6 to 2001:0DB8:1234::000e () port 0 AF_INET6
  Recv   Send    Send                          
  Socket Socket  Message  Elapsed              
  Size   Size    Size     Time     Throughput  
  bytes  bytes   bytes    secs.    10^6bits/sec  

   87380  16384  16384    10.01     716.85 

Regards,
Michal
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html