netdev - Re: debugging TCP stalls on high-speed wifi

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <99748db5-7898-534b-d407-ed819f07f939@gmail.com>
Date:   Thu, 12 Dec 2019 10:11:14 -0800
From:   Eric Dumazet <eric.dumazet@...il.com>
To:     Johannes Berg <johannes@...solutions.net>,
        Neal Cardwell <ncardwell@...gle.com>
Cc:     Eric Dumazet <eric.dumazet@...il.com>,
        Toke Høiland-Jørgensen <toke@...hat.com>,
        linux-wireless@...r.kernel.org, Netdev <netdev@...r.kernel.org>
Subject: Re: debugging TCP stalls on high-speed wifi



On 12/12/19 7:47 AM, Johannes Berg wrote:
> Hi Neal,
> 
> On Thu, 2019-12-12 at 10:11 -0500, Neal Cardwell wrote:
>> On Thu, Dec 12, 2019 at 9:50 AM Johannes Berg <johannes@...solutions.net> wrote:
>>> If you have any thoughts on this, I'd appreciate it.
>>
>> Thanks for the detailed report!
> 
> Well, it wasn't. For example, I neglected to mention that I have to
> actually use at least 2 TCP streams (but have tried up to 20) in order
> to not run into the gbit link limit on the AP :-)
> 
>> I was curious:
>>
>> o What's the sender's qdisc configuration?
> 
> There's none, mac80211 bypasses qdiscs in favour of its internal TXQs
> with FQ/codel.
> 
>> o Would you be able to log periodic dumps (e.g. every 50ms or 100ms)
>> of the test connection using a recent "ss" binary and "ss -tinm", to
>> hopefully get a sense of buffer parameters, and whether the flow in
>> these cases is being cwnd-limited, pacing-limited,
>> send-buffer-limited, or receive-window-limited?
> 
> Sure, though I'm not sure my ss is recent enough for what you had in
> mind - if not I'll have to rebuild it (it was iproute2-ss190708).
> 
> https://p.sipsolutions.net/3e515625bf13fa69.txt
> 
> Note there are 4 connections (iperf is being used) but two are control
> and two are data. Easy to see the difference really :-)
> 
>> o Would you be able to share a headers-only tcpdump pcap trace?
> 
> I'm not sure how to do headers-only, but I guess -s100 will work.
> 
> https://johannes.sipsolutions.net/files/he-tcp.pcap.xz
> 

Lack of GRO on receiver is probably what is killing performance,
both for receiver (generating gazillions of acks) and sender (to process all these acks)

I had a plan about enabling compressing ACK as I did for SACK
in commit 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5d9f4262b7ea41ca9981cc790e37cca6e37c789e

But I have not done it yet.
It is a pity because this would tremendously help wifi I am sure.