netdev - Re: TCP stall issue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 24 Feb 2021 09:55:27 -0500
From:   Neal Cardwell <ncardwell@...gle.com>
To:     Gil Pedersen <kanongil@...il.com>
Cc:     David Miller <davem@...emloft.net>,
        Hideaki YOSHIFUJI <yoshfuji@...ux-ipv6.org>,
        dsahern@...nel.org, Netdev <netdev@...r.kernel.org>,
        Yuchung Cheng <ycheng@...gle.com>,
        Eric Dumazet <edumazet@...gle.com>
Subject: Re: TCP stall issue

On Wed, Feb 24, 2021 at 5:03 AM Gil Pedersen <kanongil@...il.com> wrote:
> Sure, I attached a trace from the server that should illustrate the issue.
>
> The trace is cut from a longer flow with the server at 188.120.85.11 and a client window scaling factor of 256.
>
> Packet 78 is a TLP, followed by a delayed DUPACK with a SACK from the client.
> The SACK triggers a single segment fast re-transmit with an ignored?? D-SACK in packet 81.
> The first RTO happens at packet 82.

Thanks for the trace! That is very helpful. I have attached a plot and
my notes on the trace, for discussion.

AFAICT the client appears to be badly misbehaving, and misrepresenting
what has happened.  At each point where the client sends a DSACK,
there is an apparent contradiction. Either the client has received
that data before, or it hasn't. If the client *has* already received
that data, then it should have already cumulatively ACKed it. If the
client has *not* already received that data, then it shouldn't send a
DSACK for it.

Given that, from the server's perspective, the client is
misbehaving/lying, it's not clear what inferences the server can
safely make. Though I agree it's probably possible to do much better
than the current server behavior.

A few questions.

(a) is there a middlebox (firewall, NAT, etc) in the path?

(b) is it possible to capture a client-side trace, to help
disambiguate whether there is a client-side Linux bug or a middlebox
bug?

thanks,
neal

View attachment "slow-recovery.txt" of type "text/plain" (3821 bytes)

Download attachment "slow-recovery-time-seq-plot.png" of type "image/png" (38360 bytes)