netdev - NAT performance issue 944mbit -> ~40mbit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAA85sZvKNXCo5bB5a6kKmsOUAiw+_daAVaSYqNW6QbSBJ0TcyQ@mail.gmail.com>
Date:   Sat, 11 Jul 2020 17:53:25 +0200
From:   Ian Kumlien <ian.kumlien@...il.com>
To:     Linux Kernel Network Developers <netdev@...r.kernel.org>
Subject: NAT performance issue 944mbit -> ~40mbit

Hi,

I first detected this with 5.7.6 but it seems to apply as far back as 5.6.1...
(so, 5.7.8 client -> nat (5.6.1 -> 5.8-rc4 -> server 5.7.7)

It seems to me that the window size doesn't advance, so i did revert
the tcp: grow window for OOO packets only for SACK flows [1]
but it did no difference...

I have a 384 MB tcpdump of a iperf3 session that starts low and then
actually starts to get the bandwidth...
I do use BBR - I have tried with cubic... it didn't help  - the NAT
machine does use fq but changing it doesn't seem to yield any other
results.

Doing -P10 gives you the bandwith and can sometimes break the
stalemate but you'll end up back with the lower transfer speed again.
(it only seems to apply to NAT - the machine is a: A2SDi-12C-HLN4F and
has handled this without problems in the past...)


[1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v5.7.8&id=bf780119617797b5690e999e59a64ad79a572374

First iperf3 as a reference:
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   113 MBytes   945 Mbits/sec    0    814 KBytes
[  5]   1.00-2.00   sec   109 MBytes   912 Mbits/sec    0    806 KBytes
[  5]   2.00-3.00   sec   112 MBytes   944 Mbits/sec   31    792 KBytes
[  5]   3.00-4.00   sec   101 MBytes   849 Mbits/sec   31   1.18 MBytes
[  5]   4.00-5.00   sec   108 MBytes   902 Mbits/sec    0    783 KBytes
[  5]   5.00-6.00   sec   111 MBytes   933 Mbits/sec   31    778 KBytes
[  5]   6.00-7.00   sec   111 MBytes   933 Mbits/sec   93    772 KBytes
[  5]   7.00-8.00   sec   112 MBytes   944 Mbits/sec    0    778 KBytes
[  5]   8.00-9.00   sec   111 MBytes   933 Mbits/sec   60    778 KBytes
[  5]   9.00-10.00  sec   111 MBytes   933 Mbits/sec   92    814 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.07 GBytes   923 Mbits/sec  338             sender
[  5]   0.00-10.01  sec  1.07 GBytes   919 Mbits/sec                  receiver

After that:
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  4.77 MBytes  40.0 Mbits/sec    0   42.4 KBytes
[  5]   1.00-2.00   sec  4.10 MBytes  34.4 Mbits/sec    0   84.8 KBytes
[  5]   2.00-3.00   sec  4.60 MBytes  38.6 Mbits/sec    0   87.7 KBytes
[  5]   3.00-4.00   sec  4.23 MBytes  35.4 Mbits/sec    0   42.4 KBytes
[  5]   4.00-5.00   sec  4.23 MBytes  35.4 Mbits/sec    0   42.4 KBytes
[  5]   5.00-6.00   sec  4.47 MBytes  37.5 Mbits/sec    0   76.4 KBytes
[  5]   6.00-7.00   sec  5.47 MBytes  45.9 Mbits/sec    0   67.9 KBytes
[  5]   7.00-8.00   sec  4.66 MBytes  39.1 Mbits/sec    0   67.9 KBytes
[  5]   8.00-9.00   sec  4.35 MBytes  36.5 Mbits/sec    0   82.0 KBytes
[  5]   9.00-10.00  sec  4.66 MBytes  39.1 Mbits/sec    0    139 KBytes
- - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  45.5 MBytes  38.2 Mbits/sec    0             sender
[  5]   0.00-10.00  sec  45.0 MBytes  37.8 Mbits/sec                  receiver

You even get some:
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  5.38 MBytes  45.2 Mbits/sec    0   42.4 KBytes
[  5]   1.00-2.00   sec  7.08 MBytes  59.4 Mbits/sec    0    535 KBytes
[  5]   2.00-3.00   sec   108 MBytes   907 Mbits/sec    0    778 KBytes
[  5]   3.00-4.00   sec   111 MBytes   933 Mbits/sec    0    814 KBytes
[  5]   4.00-5.00   sec  91.2 MBytes   765 Mbits/sec    0    829 KBytes
[  5]   5.00-6.00   sec   111 MBytes   933 Mbits/sec    0    783 KBytes
[  5]   6.00-7.00   sec   111 MBytes   933 Mbits/sec    0    769 KBytes
[  5]   7.00-8.00   sec   111 MBytes   933 Mbits/sec    0    778 KBytes
[  5]   8.00-9.00   sec   112 MBytes   944 Mbits/sec    0    809 KBytes
[  5]   9.00-10.00  sec   110 MBytes   923 Mbits/sec    0    823 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   879 MBytes   738 Mbits/sec    0             sender
[  5]   0.00-10.00  sec   875 MBytes   734 Mbits/sec                  receiver