netdev - Re: TCP connection issues against Amazon S3

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <1420560253.32621.20.camel@edumazet-glaptop2.roam.corp.google.com>
Date:	Tue, 06 Jan 2015 08:04:13 -0800
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	Erik Grinaker <erik@...gler.no>
Cc:	linux-kernel@...r.kernel.org, Yuchung Cheng <ycheng@...gle.com>,
	netdev <netdev@...r.kernel.org>
Subject: Re: TCP connection issues against Amazon S3

On Tue, 2015-01-06 at 15:14 +0000, Erik Grinaker wrote:
> (CCing Yuchung, as his name comes up in the relevant commits)
> 
> After upgrading from Ubuntu 12.04.5 to 14.04.1 we have begun seeing
> intermittent TCP connection hangs for HTTP image requests against
> Amazon S3. 3-5% of requests will suddenly stall in the middle of the
> transfer before timing out. We see this problem across a range of
> servers, in several data centres and networks, all located in Norway.
> 
> A packet dump [1] shows repeated ACK retransmits for some of the
> requests. Using Ubuntu mainline kernels, we found the problem to have
> been introduced between 3.11.10 and 3.12.0, possibly in
> 0f7cc9a3c2bd89b15720dbf358e9b9e62af27126. The problem is also present
> in 3.18.1. Disabling tcp_window_scaling seems to solve it, but has
> obvious drawbacks for transfer speeds. Other sysctls do not seem to
> affect it.
> 
> I am not sure if this is fundamentally a kernel bug or a network
> issue, but we did not see this problem with older kernels.
> 
> [1] http://abstrakt.bengler.no/tcp-issues-s3.pcap.bz2--


CC netdev

This looks like the bug we fixed here :

http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=39bb5e62867de82b269b07df900165029b928359

Could you post output of 'nstat' command ?


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html