netdev - Re: use-after-free warnings in tcp_v4_connect() due to inet_twsk_hashdance() inserting the object into ehash table without initializing its reference counter

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <8922b6e4-1804-4d6a-b7e5-cd4fe1623ff1@allelesecurity.com>
Date: Wed, 1 May 2024 08:30:15 -0300
From: Anderson Nascimento <anderson@...elesecurity.com>
To: Eric Dumazet <edumazet@...gle.com>, Kuniyuki Iwashima <kuniyu@...zon.com>
Cc: netdev@...r.kernel.org
Subject: Re: use-after-free warnings in tcp_v4_connect() due to
 inet_twsk_hashdance() inserting the object into ehash table without
 initializing its reference counter



On 5/1/24 03:56, Eric Dumazet wrote:
> On Wed, May 1, 2024 at 2:22 AM Kuniyuki Iwashima <kuniyu@...zon.com> wrote:
>>
>> +cc Eric
>>
>> From: Anderson Nascimento <anderson@...elesecurity.com>
>> Date: Tue, 30 Apr 2024 19:00:34 -0300
>>> Hello,
>>
>> Hi,
>>
>> Thanks for the detailed report.
>>
>>>
>>> There is a bug in inet_twsk_hashdance(). This function inserts a
>>> time-wait socket in the established hash table without initializing the
>>> object's reference counter, as seen below. The reference counter
>>> initialization is done after the object is added to the established hash
>>> table and the lock is released. Because of this, a sock_hold() in
>>> tcp_twsk_unique() and other operations on the object trigger warnings
>>> from the reference counter saturation mechanism. The warnings can also
>>> be seen below. They were triggered on Fedora 39 Linux kernel v6.8.
>>>
>>> The bug is triggered via a connect() system call on a TCP socket,
>>> reaching __inet_check_established() and then passing the time-wait
>>> socket to tcp_twsk_unique(). Other operations are also performed on the
>>> time-wait socket in __inet_check_established() before its reference
>>> counter is initialized correctly by inet_twsk_hashdance(). The fix seems
>>> to be to move the reference counter initialization inside the lock,
>>
>> or use refcount_inc_not_zero() and give up on reusing the port
>> under the race ?
>>
>> ---8<---
>> diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
>> index 0427deca3e0e..637f4965326d 100644
>> --- a/net/ipv4/tcp_ipv4.c
>> +++ b/net/ipv4/tcp_ipv4.c
>> @@ -175,8 +175,13 @@ int tcp_twsk_unique(struct sock *sk, struct sock *sktw, void *twp)
>>                          tp->rx_opt.ts_recent       = tcptw->tw_ts_recent;
>>                          tp->rx_opt.ts_recent_stamp = tcptw->tw_ts_recent_stamp;
>>                  }
>> -               sock_hold(sktw);
>> -               return 1;
>> +
>> +               /* Here, sk_refcnt could be 0 because inet_twsk_hashdance() puts
>> +                * twsk into ehash and releases the bucket lock *before* setting
>> +                * sk_refcnt.  Then, give up on reusing the port.
>> +                */
>> +               if (likely(refcount_inc_not_zero(&sktw->sk_refcnt)))
>> +                       return 1;
>>          }
>>
> 
> Thanks for CC me.
> 
> Nice analysis from Anderson ! Have you found this with a fuzzer ?

I ran the reproducer of a bug found by syzkaller on an older kernel, and 
this issue was triggered. Analyzing it, I discovered it had nothing to 
do with the problem the reproducer aimed to trigger, and it was present 
upstream. I rewrote the reproducer and triggered it on v6.8 to confirm.

https://syzkaller.appspot.com/bug?extid=278279efdd2730dd14bf

> 
> This patch would avoid the refcount splat, but would leave side
> effects on tp, I am too lazy to double check them.
> 
> Incidentally, I think we have to annotate data-races on
> tcptw->tw_ts_recent and  tcptw->tw_ts_recent_stamp
> 
> Perhaps something like this instead ?
> 
> diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
> index 0427deca3e0eb9239558aa124a41a1525df62a04..f1e3707d0b33180a270e6d3662d4cf17a4f72bb8
> 100644
> --- a/net/ipv4/tcp_ipv4.c
> +++ b/net/ipv4/tcp_ipv4.c
> @@ -155,6 +155,10 @@ int tcp_twsk_unique(struct sock *sk, struct sock
> *sktw, void *twp)
>          if (tcptw->tw_ts_recent_stamp &&
>              (!twp || (reuse && time_after32(ktime_get_seconds(),
>                                              tcptw->tw_ts_recent_stamp)))) {
> +
> +               if (!refcount_inc_not_zero(&sktw->sk_refcnt))
> +                       return 0;
> +
>                  /* In case of repair and re-using TIME-WAIT sockets we still
>                   * want to be sure that it is safe as above but honor the
>                   * sequence numbers and time stamps set as part of the repair
> @@ -175,7 +179,6 @@ int tcp_twsk_unique(struct sock *sk, struct sock
> *sktw, void *twp)
>                          tp->rx_opt.ts_recent       = tcptw->tw_ts_recent;
>                          tp->rx_opt.ts_recent_stamp = tcptw->tw_ts_recent_stamp;
>                  }
> -               sock_hold(sktw);
>                  return 1;
>          }

Download attachment "OpenPGP_signature.asc" of type "application/pgp-signature" (841 bytes)