netdev - Re: TCP and BBR: reproducibly low cwnd and bandwidth

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4edcb644-4530-e771-eea7-947ce0ef12ed@applied-asynchrony.com>
Date:   Fri, 16 Feb 2018 18:56:12 +0100
From:   Holger Hoffstätte <holger@...lied-asynchrony.com>
To:     Oleksandr Natalenko <oleksandr@...alenko.name>,
        Neal Cardwell <ncardwell@...gle.com>
Cc:     Eric Dumazet <eric.dumazet@...il.com>,
        "David S. Miller" <davem@...emloft.net>,
        Netdev <netdev@...r.kernel.org>,
        Yuchung Cheng <ycheng@...gle.com>,
        Soheil Hassas Yeganeh <soheil@...gle.com>,
        Jerry Chu <hkchu@...gle.com>,
        Eric Dumazet <edumazet@...gle.com>,
        Dave Taht <dave.taht@...il.com>
Subject: Re: TCP and BBR: reproducibly low cwnd and bandwidth

On 02/16/18 18:25, Oleksandr Natalenko wrote:
> So, going on with two real HW hosts. They are both running latest stock Arch 
> Linux kernel (4.15.3-1-ARCH, CONFIG_PREEMPT=y, CONFIG_HZ=1000) and are 
> interconnected with 1 Gbps link (via switch if that matters). Using iperf3, 
> running each test for 20 seconds.
> 
> Having BBR+fq_codel (or pfifo_fast, same result) on both hosts:
> 
> Client to server: 112 Mbits/sec
> Server to client: 96.1 Mbits/sec
> 
> Having BBR+fq on both hosts:
> 
> Client to server: 347 Mbits/sec
> Server to client: 397 Mbits/sec
> 
> Having YeAH+fq on both hosts:
> 
> Client to server: 928 Mbits/sec
> Server to client: 711 Mbits/sec
> 
> (when the server generates traffic, the throughput is a little bit lower, as 
> you can see, but I assume that's because I have there low-power Silvermont 
> CPU, when the client has Ivy Bridge beast)

There is simply no reason why you shouldn't get approx. line rate (~920+-ish)
Mbit over wired 1GBit Ethernet; even my broken 10-year old Core2Duo laptop can
do that. Can you boot with spectre_v2=off and try "the simplest case" with the
defaults cubic/pfifo_fast? spectre_v2 has terrible performance impact esp.
on small/older processors.

When I last benchmarked full PREEMPT with 4.9.x it was similarly bad and also
had a noticeable network throughput impact even on my i7.

Also congratulations for being the only other person I know who ever tried
YeAH. :-)

cheers
Holger