| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20250617144017.82931-17-maxim@isovalent.com>
Date: Tue, 17 Jun 2025 16:40:15 +0200
From: Maxim Mikityanskiy <maxtram95@...il.com>
To: Daniel Borkmann <daniel@...earbox.net>,
"David S. Miller" <davem@...emloft.net>,
Eric Dumazet <edumazet@...gle.com>,
Jakub Kicinski <kuba@...nel.org>,
Paolo Abeni <pabeni@...hat.com>,
Willem de Bruijn <willemdebruijn.kernel@...il.com>,
David Ahern <dsahern@...nel.org>,
Nikolay Aleksandrov <razor@...ckwall.org>
Cc: netdev@...r.kernel.org,
Maxim Mikityanskiy <maxim@...valent.com>
Subject: [PATCH RFC net-next 16/17] vxlan: Enable BIG TCP packets
From: Maxim Mikityanskiy <maxim@...valent.com>
In Cilium we do support BIG TCP, but so far the latter has only been
enabled for direct routing use-cases. A lot of users rely on Cilium
with vxlan/geneve tunneling though. The underlying kernel infra for
tunneling has not been supporting BIG TCP up to this point.
Given we do now, bump tso_max_size for vxlan netdevs up to GSO_MAX_SIZE
to allow the admin to use BIG TCP with vxlan tunnels.
BIG TCP on vxlan disabled:
Standard MTU:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
131072 16384 16384 30.00 34440.00
8k MTU:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
262144 32768 32768 30.00 55684.26
BIG TCP on vxlan enabled:
Standard MTU:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
131072 16384 16384 30.00 39564.78
8k MTU:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
262144 32768 32768 30.00 61466.47
When tunnel offloads are not enabled/exposed and we fully need to rely on
SW-based segmentation on transmit (e.g. in case of Azure) then the more
aggressive batching also has a visible effect. Below example was on the
same setup as with above benchmarks but with HW support disabled:
# ethtool -k enp10s0f0np0 | grep udp
tx-udp_tnl-segmentation: off
tx-udp_tnl-csum-segmentation: off
tx-udp-segmentation: off
rx-udp_tunnel-port-offload: off
rx-udp-gro-forwarding: off
Before:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
131072 16384 16384 60.00 21820.82
After:
# netperf -H 10.1.0.2 -t TCP_STREAM -l60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.0.2 () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
131072 16384 16384 60.00 29390.78
Example receive side:
swapper 0 [002] 4712.645070: net:netif_receive_skb: dev=enp10s0f0np0 skbaddr=0xffff8f3b086e0200 len=129542
ffffffff8cfe3aaa __netif_receive_skb_core.constprop.0+0x6ca ([kernel.kallsyms])
ffffffff8cfe3aaa __netif_receive_skb_core.constprop.0+0x6ca ([kernel.kallsyms])
ffffffff8cfe47dd __netif_receive_skb_list_core+0xed ([kernel.kallsyms])
ffffffff8cfe4e52 netif_receive_skb_list_internal+0x1d2 ([kernel.kallsyms])
ffffffff8d0210d8 gro_complete.constprop.0+0x108 ([kernel.kallsyms])
ffffffff8d021724 dev_gro_receive+0x4e4 ([kernel.kallsyms])
ffffffff8d021a99 gro_receive_skb+0x89 ([kernel.kallsyms])
ffffffffc06edb71 mlx5e_handle_rx_cqe_mpwrq+0x131 ([kernel.kallsyms])
ffffffffc06ee38a mlx5e_poll_rx_cq+0x9a ([kernel.kallsyms])
ffffffffc06ef2c7 mlx5e_napi_poll+0x107 ([kernel.kallsyms])
ffffffff8cfe586d __napi_poll+0x2d ([kernel.kallsyms])
ffffffff8cfe5f8d net_rx_action+0x20d ([kernel.kallsyms])
ffffffff8c35d252 handle_softirqs+0xe2 ([kernel.kallsyms])
ffffffff8c35d556 __irq_exit_rcu+0xd6 ([kernel.kallsyms])
ffffffff8c35d81e irq_exit_rcu+0xe ([kernel.kallsyms])
ffffffff8d2602b8 common_interrupt+0x98 ([kernel.kallsyms])
ffffffff8c000da7 asm_common_interrupt+0x27 ([kernel.kallsyms])
ffffffff8d2645c5 cpuidle_enter_state+0xd5 ([kernel.kallsyms])
ffffffff8cf6358e cpuidle_enter+0x2e ([kernel.kallsyms])
ffffffff8c3ba932 call_cpuidle+0x22 ([kernel.kallsyms])
ffffffff8c3bfb5e do_idle+0x1ce ([kernel.kallsyms])
ffffffff8c3bfd79 cpu_startup_entry+0x29 ([kernel.kallsyms])
ffffffff8c30a6c2 start_secondary+0x112 ([kernel.kallsyms])
ffffffff8c2c142d common_startup_64+0x13e ([kernel.kallsyms])
Example transmit side:
swapper 0 [005] 4768.021375: net:net_dev_xmit: dev=enp10s0f0np0 skbaddr=0xffff8af32ebe1200 len=129556 rc=0
ffffffffa75e19c3 dev_hard_start_xmit+0x173 ([kernel.kallsyms])
ffffffffa75e19c3 dev_hard_start_xmit+0x173 ([kernel.kallsyms])
ffffffffa7653823 sch_direct_xmit+0x143 ([kernel.kallsyms])
ffffffffa75e2780 __dev_queue_xmit+0xc70 ([kernel.kallsyms])
ffffffffa76a1205 ip_finish_output2+0x265 ([kernel.kallsyms])
ffffffffa76a1577 __ip_finish_output+0x87 ([kernel.kallsyms])
ffffffffa76a165b ip_finish_output+0x2b ([kernel.kallsyms])
ffffffffa76a179e ip_output+0x5e ([kernel.kallsyms])
ffffffffa76a19d5 ip_local_out+0x35 ([kernel.kallsyms])
ffffffffa770d0e5 iptunnel_xmit+0x185 ([kernel.kallsyms])
ffffffffc179634e nf_nat_used_tuple_new.cold+0x1129 ([kernel.kallsyms])
ffffffffc17a7301 vxlan_xmit_one+0xc21 ([kernel.kallsyms])
ffffffffc17a80a2 vxlan_xmit+0x4a2 ([kernel.kallsyms])
ffffffffa75e18af dev_hard_start_xmit+0x5f ([kernel.kallsyms])
ffffffffa75e1d3f __dev_queue_xmit+0x22f ([kernel.kallsyms])
ffffffffa76a1205 ip_finish_output2+0x265 ([kernel.kallsyms])
ffffffffa76a1577 __ip_finish_output+0x87 ([kernel.kallsyms])
ffffffffa76a165b ip_finish_output+0x2b ([kernel.kallsyms])
ffffffffa76a179e ip_output+0x5e ([kernel.kallsyms])
ffffffffa76a1de2 __ip_queue_xmit+0x1b2 ([kernel.kallsyms])
ffffffffa76a2135 ip_queue_xmit+0x15 ([kernel.kallsyms])
ffffffffa76c70a2 __tcp_transmit_skb+0x522 ([kernel.kallsyms])
ffffffffa76c931a tcp_write_xmit+0x65a ([kernel.kallsyms])
ffffffffa76cb42e tcp_tsq_write+0x5e ([kernel.kallsyms])
ffffffffa76cb7ef tcp_tasklet_func+0x10f ([kernel.kallsyms])
ffffffffa695d9f7 tasklet_action_common+0x107 ([kernel.kallsyms])
ffffffffa695db99 tasklet_action+0x29 ([kernel.kallsyms])
ffffffffa695d252 handle_softirqs+0xe2 ([kernel.kallsyms])
ffffffffa695d556 __irq_exit_rcu+0xd6 ([kernel.kallsyms])
ffffffffa695d81e irq_exit_rcu+0xe ([kernel.kallsyms])
ffffffffa78602b8 common_interrupt+0x98 ([kernel.kallsyms])
ffffffffa6600da7 asm_common_interrupt+0x27 ([kernel.kallsyms])
ffffffffa78645c5 cpuidle_enter_state+0xd5 ([kernel.kallsyms])
ffffffffa756358e cpuidle_enter+0x2e ([kernel.kallsyms])
ffffffffa69ba932 call_cpuidle+0x22 ([kernel.kallsyms])
ffffffffa69bfb5e do_idle+0x1ce ([kernel.kallsyms])
ffffffffa69bfd79 cpu_startup_entry+0x29 ([kernel.kallsyms])
ffffffffa690a6c2 start_secondary+0x112 ([kernel.kallsyms])
ffffffffa68c142d common_startup_64+0x13e ([kernel.kallsyms])
Signed-off-by: Maxim Mikityanskiy <maxim@...valent.com>
Co-developed-by: Daniel Borkmann <daniel@...earbox.net>
Signed-off-by: Daniel Borkmann <daniel@...earbox.net>
Cc: Nikolay Aleksandrov <razor@...ckwall.org>
---
drivers/net/vxlan/vxlan_core.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c
index a56d7239b127..84b6cbf207fc 100644
--- a/drivers/net/vxlan/vxlan_core.c
+++ b/drivers/net/vxlan/vxlan_core.c
@@ -3330,6 +3330,8 @@ static void vxlan_setup(struct net_device *dev)
dev->hw_features |= NETIF_F_RXCSUM;
dev->hw_features |= NETIF_F_GSO_SOFTWARE;
netif_keep_dst(dev);
+ netif_set_tso_max_size(dev, GSO_MAX_SIZE);
+
dev->priv_flags |= IFF_NO_QUEUE;
dev->change_proto_down = true;
dev->lltx = true;
--
2.49.0
Powered by blists - more mailing lists