netdev - Re: After many hours all outbound connections get stuck in SYN

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <83a51e120712190943m3bf0e2e4v2ea6b660142e9a5a@mail.gmail.com>
Date:	Wed, 19 Dec 2007 12:43:13 -0500
From:	"James Nichols" <jamesnichols3@...il.com>
To:	"Eric Dumazet" <dada1@...mosbay.com>
Cc:	"Jan Engelhardt" <jengelh@...putergmbh.de>,
	linux-kernel@...r.kernel.org,
	"Linux Netdev List" <netdev@...r.kernel.org>
Subject: Re: After many hours all outbound connections get stuck in SYN_SENT

On 12/19/07, Eric Dumazet <dada1@...mosbay.com> wrote:
> James Nichols a écrit :
> >> So you see outgoing SYN packets, but no SYN replies coming from the remote
> >> peer ?  (you mention ACKS, but the first packet received from the remote
> >>  peer should be a SYN+ACK),
> >
> > Right, I meant to say SYN+ACK.  I don't see them coming back.
>
> So... Really unlikely a linux problem, but ...
>

I don't know how you can be so sure.  Turning tcp_sack off instantly
resovles the problem and all connections are succesful.  I can't
imagine even the most far-fetched scenario where a router or every
single remote endpoints would suddenly stop causing the problem just
by removing a single TCP option.

> > I can take these captures and take a look at the results.
> > Unfortunately, I don't think I'll be able to make the captures
> > available to the general public.
>
> I dont understand, why dont you change IPs to mask them with 192.168.X.Y, or
> just ME, and peer1, peer2, peer...

I will see if I can do that, but it's major pain with 2000 hosts.
Plus, there is application data in the packets that I can't allow into
the public domain.  I really don't think I can pull it off... I
literally would have to go through our legal department.

>
> Random ideas :
>
> 1) Is your server behind a NET router or something ?

What's a NET router?  I am behind a Cisco router and a firewall, but
these network components have completely been replaced/rebuilt several
times in the 4+ years that we've had this problem.  I've looked at the
logs there and neither are doing anything other than passing the
traffic along.

> 2) Are you sure you are not using connection tracking, and hit a limit on it ?

I'm using ip_conntrack, but the limit I have for max entries is 65K.
The most I've seen in there are a couple thousand- that was one of the
first things I monitored very closely.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html