netdev - connect returns EADDRNOTAVAIL on ~600k+ sockets host

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 12 Sep 2014 21:18:45 +0300
From:	Denys Fedoryshchenko <nuclearcat@...learcat.com>
To:	netdev@...r.kernel.org
Subject: connect returns EADDRNOTAVAIL on ~600k+ sockets host

Hi

I noticed strange behaviour on loaded server (haproxy), while it is 
handling load for https traffic well, tcp monitoring started to give 
false alerts. And here what i found:

HTTPS-BALANCER ~ # telnet 127.0.0.1 23
telnet: can't connect to remote host (127.0.0.1): Cannot assign 
requested address
HTTPS-BALANCER ~ # nc -v 127.0.0.1 23
127.0.0.1 (127.0.0.1:23) open
��.��.��.��.
HTTPS-BALANCER login: ^Cpunt!

The difference is (relevant lines from strace):

socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3
setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
bind(3, {sa_family=AF_INET, sin_port=htons(0), 
sin_addr=inet_addr("0.0.0.0")}, 16) = 0
rt_sigaction(SIGALRM, {0x41ecfb, [ALRM], SA_RESTORER|SA_RESTART, 
0x7f80725bf5b0}, {SIG_DFL, [], 0}, 8) = 0
alarm(0)                                = 0
connect(3, {sa_family=AF_INET, sin_port=htons(23), 
sin_addr=inet_addr("127.0.0.1")}, 16) = 0
rt_sigaction(SIGALRM, {SIG_IGN, [ALRM], SA_RESTORER|SA_RESTART, 
0x7f80725bf5b0}, {0x41ecfb, [ALRM], SA_RESTORER|SA_RESTART, 0x7


socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3
setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
connect(3, {sa_family=AF_INET, sin_port=htons(23), 
sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EADDRNOTAVAIL (Cannot assign 
requested address)
write(2, "telnet: can't connect to remote host (127.0.0.1): Cannot 
assign requested address\n", 82telnet: can't connect to remote host 
(127.0.0.1): Cannot assign requested address

So,
bind(3, {sa_family=AF_INET, sin_port=htons(0), 
sin_addr=inet_addr("0.0.0.0")}, 16) = 0
is fixing issue, but is it really supposed to be like this?

Just for information host load and some network stats:
from /proc/net/protocols, i removed irrelevant zero values entries

protocol  size sockets  memory press maxhdr  slab module     cl co di ac 
io in de sh ss gs se re sp bi br ha uh gp em
UDP        816      1       0   NI       0   yes  kernel      y  y  y  n 
  y  n  y  n  y  y  y  y  y  n  y  y  y  y  n
TCP       1640 728255  751777   no     208   yes  kernel      y  y  y  y 
  y  y  y  y  y  y  y  y  y  n  y  y  y  y  y

sockets: used 674375
TCP: inuse 727092 orphan 55119 tw 138533 alloc 729419 mem 745187
UDP: inuse 1 mem 0
UDPLITE: inuse 0
RAW: inuse 0
FRAG: inuse 0 memory 0

0.0.0.0 sockets:
tcp        0      0 0.0.0.0:65530           0.0.0.0:*               
LISTEN
tcp        0      0 0.0.0.0:443             0.0.0.0:*               
LISTEN
tcp        0      0 0.0.0.0:8880            0.0.0.0:*               
LISTEN
tcp        0      0 0.0.0.0:22              0.0.0.0:*               
LISTEN
tcp        0      0 0.0.0.0:23              0.0.0.0:*               
LISTEN
udp        0      0 0.0.0.0:53073           0.0.0.0:*

127.0.0.1 sockets (this is watchdog, but if i disable, it wont change 
anything)
tcp        0      0 127.0.0.1:65530         127.0.0.1:55649         
TIME_WAIT
tcp        0      0 127.0.0.1:65530         127.0.0.1:41762         
FIN_WAIT2
tcp        0      0 127.0.0.1:65530         127.0.0.1:36863         
TIME_WAIT
tcp        0      0 127.0.0.1:65530         127.0.0.1:34533         
TIME_WAIT
tcp        0      0 127.0.0.1:65530         127.0.0.1:35262         
TIME_WAIT
tcp        0      0 127.0.0.1:65530         127.0.0.1:45872         
TIME_WAIT





--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html