| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <57ADD8DE020000A1000223EB@gwsmtp1.uni-regensburg.de>
Date: Fri, 12 Aug 2016 14:10:38 +0200
From: "Ulrich Windl" <Ulrich.Windl@...uni-regensburg.de>
To: <netdev@...r.kernel.org>
Subject: Q: (linux 3.0.101) SO_LINGER2 & FIN_WAIT2
Hello!
Please excuse me for asking this question, but I'm fighting a problem I don't understand:
I have written an application that forwards data from on TCPv4 stream to another, that is, you start the utility on hostA listening at the IP of hostA, portA, redirecting to hostB, portB. Now when hostC connects to hostA:portA vir portC, the traffic is forwarded to hostB:portB, and hostB's replies are forwarded back to hostC:portC.
That works well enough (I tested with Firefox on hostC and a webserver on hostB). I only have a problem: From time to time a shutdown(sock, 1) of the socket connected to hostB:portB seems to hand forever after hostC has closed or shutdown it socket.
The effect is that connections stay in CLOSE_WAIT like this:
# netstat -tn |grep 48123
tcp 1 0 172.20.17.252:64236 172.20.17.78:48123 CLOSE_WAIT
tcp 1 0 172.20.17.252:54347 172.20.17.78:48123 CLOSE_WAIT
tcp 1 0 172.20.17.252:51009 172.20.17.78:48123 CLOSE_WAIT
tcp 1 0 172.20.17.252:49899 172.20.17.78:48123 CLOSE_WAIT
Even though I set SO_LINGER option, the shutdown never completes, and I'll have to kill the process handling the connection.
I've read about SO_LINGER2, but being confused by the description I tried to read the source code. I don't understand (/usr/src/linux/net/ipv4/tcp.c, tcp_close()) the case when tp->linger2 >= 0: The value of tp->linger2 is not used anywhere; instead tcp_fin_time(sk) is used.
The doc (man 7 tcp) says:
TCP_LINGER2 (since Linux 2.4)
The lifetime of orphaned FIN_WAIT2 state sockets. This option
can be used to override the system-wide setting in the file
/proc/sys/net/ipv4/tcp_fin_timeout for this socket. This is not(...)
Can somebody clarify?
Back to my application: I handle each direction (HostC->HostA->HostB (read), HostB->HostA->HostC (write)) in a separate thread, and I try to shutdown() the corresponding socket direction if I detect an EOF condition in any direction. typically read first. Occasionally I see an error "Transport
endpoint is not connected" when shutting down the write connection on the socket connected to HostA. Is there a mistake on my side?
(D)[16049](172.20.16.36/13939): handler uses input socket 4
(D)[16049](172.20.16.36/13939): set SO_LINGER on socket 4 to 1, 0
(D)[16049](172.20.16.36/13939): connection_handler: use output socket 0
(D)[16049](172.20.16.36/13939): set SO_LINGER on socket 0 to 1, 0
(D)[16049](172.20.16.36/13939): connection_handler: connected to 172.20.17.78/48123 in 99 microseconds
(D)[16049](172.20.16.36/13939): create input thread
(D)[16049](172.20.16.36/13939): create output thread
(D)[16049](input): received 147 bytes from 4
(D)[16049](input): sent 147 bytes to 0
(D)[16049](output): received 671 bytes from 0
(D)[16049](output): sent 671 bytes to 4
(D)[16049](input): copy 4 (172.20.16.36/13939) -> 0 (172.20.17.78/48123) finished with 0
(D)[16049](input): shutdown read direction (4 -> 0)
(D)[16049](input): shutdown socket 4 with 0
(D)[16049](input): shutdown socket 0 with 1
(D)[16049](172.20.16.36/13939): input thread 140526054442752 returned 0
(D)[16049](172.20.16.36/13939): wait for output thread 140526062835456
(D)[16049](output): copy 0 (172.20.17.78/48123) -> 4 (172.20.16.36/13939) finished with 0
(D)[16049](output): shutdown write direction (0 -> 4)
(D)[16049](output): shutdown socket 0 with 0
(N)[16049](output): failed to shutdown socket 0 with 0: Transport endpoint is not connected
(D)[16049](output): shutdown socket 4 with 1
(W)[16049](172.20.16.36/13939): output thread 140526062835456 returned -1
(D)[16049](172.20.16.36/13939): close inbound socket 4
(D)[16049](172.20.16.36/13939): close outbound socket 0
(D)[16049](172.20.16.36/13939): connection_handler finished
(D)[16049](172.20.16.36/13939): handler exiting after 17.038
Here are some code sketches (the whole thing is quite long and verbose) for both threads:
/* handle input data */
static void *input_thread(void *arg)
{
handler_t *handler = arg;
char ID[ID_MAX];
tsap_t *const i_conn = &handler->i_conn;
tsap_t *const o_conn = &handler->o_conn;
thread_result_t result;
int i;
/*...*/
result.i = socket_copy(i_conn->sock, o_conn->sock, &handler->i_bytes,
&handler->last_rd, ID);
vlogf(LOG_DEBUG,
"%s: copy %d (%s) -> %d (%s) finished with %d",
ID, i_conn->sock, i_conn->r_addr.desc_real, o_conn->sock,
o_conn->r_addr.desc_real, result.i);
vlogf(LOG_DEBUG, "%s: shutdown read direction (%d -> %d)", ID,
i_conn->sock, o_conn->sock);
if ( do_shutdown(ID, i_conn->sock, SHUT_RD) != 0 )
result.i = -1;
if ( do_shutdown(ID, o_conn->sock, SHUT_WR) != 0 )
result.i = -1;
return(result.ptr);
}
/* handle output data */
static void *output_thread(void *arg)
{
handler_t *handler = arg;
char ID[ID_MAX];
tsap_t *const i_conn = &handler->i_conn;
tsap_t *const o_conn = &handler->o_conn;
thread_result_t result;
int i;
/*...*/
result.i = socket_copy(o_conn->sock, i_conn->sock, &handler->o_bytes,
&handler->last_wr, ID);
vlogf(LOG_DEBUG,
"%s: copy %d (%s) -> %d (%s) finished with %d",
ID, o_conn->sock, o_conn->r_addr.desc_real, i_conn->sock,
i_conn->r_addr.desc_real, result.i);
vlogf(LOG_DEBUG, "%s: shutdown write direction (%d -> %d)", ID,
o_conn->sock, i_conn->sock);
if ( do_shutdown(ID, o_conn->sock, SHUT_RD) != 0 )
result.i = -1;
if ( do_shutdown(ID, i_conn->sock, SHUT_WR) != 0 )
result.i = -1;
return(result.ptr);
}
The connection_handler process does this roughly (after having set up the sockets):
...
if ( (err = pthread_create(&handler->input, NULL, input_thread,
handler)) != 0 )
{
result = -1;
}
vlogf(LOG_DEBUG, "%s: create output thread", handler->ID);
if ( (err = pthread_create(&handler->output, NULL,
output_thread, handler)) != 0 )
{
result = -1;
}
...
if ( (err = pthread_join(handler->input, &ret.ptr)) == 0 )
{
...
}
if ( (err = pthread_join(handler->output, &ret.ptr)) == 0 \
)
{
...
}
...
if ( do_close(i_conn->sock) != 0 )
result = -1;
if ( do_close(o_conn->sock) != 0 )
result = -1;
Regards,
Ulrich
Powered by blists - more mailing lists