netdev - Re: [RFC PATCH] tcp: accept RST for rcv_nxt

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Tue, 10 Jan 2017 21:17:08 -0800
From:   Christoph Paasch <christoph.paasch@...il.com>
To:     Jason Baron <jbaron@...mai.com>
Cc:     netdev@...r.kernel.org
Subject: Re: [RFC PATCH] tcp: accept RST for rcv_nxt - 1 after receiving a FIN

Hello Jason,

(resending as Gmail sent out with HTML)

On 05/01/17 - 16:33:28, Jason Baron wrote:
> Using a Mac OSX box as a client connecting to a Linux server, we have found
> that when certain applications (such as 'ab'), are abruptly terminated
> (via ^C), a FIN is sent followed by a RST packet on tcp connections. The
> FIN is accepted by the Linux stack but the RST is sent with the same
> sequence number as the FIN, and Linux responds with a challenge ACK per
> RFC 5961. The OSX client then does not reply with any RST as would be
> expected on a closed socket.

do you see this behavior consistently, even in a controlled environment?

The problem seems rather to be that after the first RST, the NAT on the path
has dropped its mapping and is thus dropping all other traffic. So, Linux's
challenge-ack does not go through to the OSX-host to "re-synchronize" the
state (which would allow OSX to send a RST with the updated sequence numbers).

This is also documented in RFC5961:

9.3.  Middleboxes That Drop the Challenge ACK

   It also needs to be noted that, some middleboxes (Firewalls/NATs)
   that don't have the fix recommended in the document, may drop the
   challenge ACK.  This can happen because, the original RST segment
   that was in window had already cleared the flow state pertaining to
   the TCP connection in the middlebox.  In such cases, the end hosts
   that have implemented the RST mitigation described in this document,
   will have the TCP connection left open.  This is a corner case and
   can go away if the middlebox is conformant with the changes proposed
   in this document.



Cheers,
Christoph

> 
> This results in sockets accumulating on the Linux server left mostly in
> the CLOSE_WAIT state, although LAST_ACK and CLOSING are also possible.
> This sequence of events can tie up a lot of resources on the Linux server
> since there may be a lot of data in write buffers at the time of the RST.
> Accepting a RST equal to rcv_nxt - 1, after we have already successfully
> processed a FIN, has made a significant difference for us in practice, by
> freeing up unneeded resources in a more expedient fashion.
> 
> I also found a posting that the iOS client behaves in a similar manner here
> (it will send a FIN followed by a RST for rcv_nxt - 1):
> https://www.snellman.net/blog/archive/2016-02-01-tcp-rst/
> 
> A packetdrill test demonstrating the behavior.
> 
> // testing mac osx rst behavior
> 
> // Establish a connection
> 0.000 socket(..., SOCK_STREAM, IPPROTO_TCP) = 3
> 0.000 setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
> 0.000 bind(3, ..., ...) = 0
> 0.000 listen(3, 1) = 0
> 
> 0.100 < S 0:0(0) win 32768 <mss 1460,nop,wscale 10>
> 0.100 > S. 0:0(0) ack 1 <mss 1460,nop,wscale 5>
> 0.200 < . 1:1(0) ack 1 win 32768
> 0.200 accept(3, ..., ...) = 4
> 
> // Client closes the connection
> 0.300 < F. 1:1(0) ack 1 win 32768
> 
> // now send rst with same sequence
> 0.300 < R. 1:1(0) ack 1 win 32768
> 
> // make sure we are in TCP_CLOSE
> 0.400 %{
> assert tcpi_state == 7
> }%
> 
> Signed-off-by: Jason Baron <jbaron@...mai.com>
> ---
>  net/ipv4/tcp_input.c | 23 ++++++++++++++++++++++-
>  1 file changed, 22 insertions(+), 1 deletion(-)
> 
> diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
> index ec6d84363024..373bea05c93b 100644
> --- a/net/ipv4/tcp_input.c
> +++ b/net/ipv4/tcp_input.c
> @@ -5249,6 +5249,24 @@ static int tcp_copy_to_iovec(struct sock *sk, struct sk_buff *skb, int hlen)
>  	return err;
>  }
>  
> +/* Accept RST for rcv_nxt - 1 after a FIN.
> + * When tcp connections are abruptly terminated from Mac OSX (via ^C), a
> + * FIN is sent followed by a RST packet. The RST is sent with the same
> + * sequence number as the FIN, and thus according to RFC 5961 a challenge
> + * ACK should be sent. However, Mac OSX does not reply to the challenge ACK
> + * with a RST on the closed socket, hence accept this class of RSTs.
> + */
> +static bool tcp_reset_check(struct sock *sk, struct sk_buff *skb)
> +{
> +	struct tcp_sock *tp = tcp_sk(sk);
> +
> +	return unlikely((TCP_SKB_CB(skb)->seq == (tp->rcv_nxt - 1)) &&
> +			(TCP_SKB_CB(skb)->end_seq == (tp->rcv_nxt - 1))	&&
> +			(sk->sk_state == TCP_CLOSE_WAIT ||
> +			 sk->sk_state == TCP_LAST_ACK ||
> +			 sk->sk_state == TCP_CLOSING));
> +}
> +
>  /* Does PAWS and seqno based validation of an incoming segment, flags will
>   * play significant role here.
>   */
> @@ -5287,6 +5305,8 @@ static bool tcp_validate_incoming(struct sock *sk, struct sk_buff *skb,
>  						  LINUX_MIB_TCPACKSKIPPEDSEQ,
>  						  &tp->last_oow_ack_time))
>  				tcp_send_dupack(sk, skb);
> +		} else if (tcp_reset_check(sk, skb)) {
> +			tcp_reset(sk);
>  		}
>  		goto discard;
>  	}
> @@ -5300,7 +5320,8 @@ static bool tcp_validate_incoming(struct sock *sk, struct sk_buff *skb,
>  		 * else
>  		 *     Send a challenge ACK
>  		 */
> -		if (TCP_SKB_CB(skb)->seq == tp->rcv_nxt) {
> +		if (TCP_SKB_CB(skb)->seq == tp->rcv_nxt ||
> +		    tcp_reset_check(sk, skb)) {
>  			rst_seq_match = true;
>  		} else if (tcp_is_sack(tp) && tp->rx_opt.num_sacks > 0) {
>  			struct tcp_sack_block *sp = &tp->selective_acks[0];
> -- 
> 2.6.1
>