| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20240226200050.71690-1-kuniyu@amazon.com>
Date: Mon, 26 Feb 2024 12:00:50 -0800
From: Kuniyuki Iwashima <kuniyu@...zon.com>
To: <edumazet@...gle.com>
CC: <allison.henderson@...cle.com>, <davem@...emloft.net>, <kuba@...nel.org>,
<kuni1840@...il.com>, <kuniyu@...zon.com>, <linux-rdma@...r.kernel.org>,
<netdev@...r.kernel.org>, <pabeni@...hat.com>, <rds-devel@....oracle.com>,
<sowmini.varadhan@...cle.com>, <syzkaller@...glegroups.com>
Subject: Re: [PATCH v1 net 2/2] rds: tcp: Fix use-after-free of net in reqsk_timer_handler().
From: Eric Dumazet <edumazet@...gle.com>
Date: Mon, 26 Feb 2024 20:52:35 +0100
> On Mon, Feb 26, 2024 at 8:39 PM Kuniyuki Iwashima <kuniyu@...zon.com> wrote:
> >
> > From: Allison Henderson <allison.henderson@...cle.com>
> > Date: Mon, 26 Feb 2024 19:22:01 +0000
> > > On Mon, 2024-02-26 at 11:14 -0800, Kuniyuki Iwashima wrote:
> > > > From: Kuniyuki Iwashima <kuniyu@...zon.com>
> > > > Date: Fri, 23 Feb 2024 10:28:32 -0800
> > > > > From: Eric Dumazet <edumazet@...gle.com>
> > > > > Date: Fri, 23 Feb 2024 19:09:27 +0100
> > > > > > On Fri, Feb 23, 2024 at 6:26 PM Kuniyuki Iwashima
> > > > > > <kuniyu@...zon.com> wrote:
> > > > > > >
> > > > > > > syzkaller reported a warning of netns tracker [0] followed by
> > > > > > > KASAN
> > > > > > > splat [1] and another ref tracker warning [1].
> > > > > > >
> > > > > > > syzkaller could not find a repro, but in the log, the only
> > > > > > > suspicious
> > > > > > > sequence was as follows:
> > > > > > >
> > > > > > > 18:26:22 executing program 1:
> > > > > > > r0 = socket$inet6_mptcp(0xa, 0x1, 0x106)
> > > > > > > ...
> > > > > > > connect$inet6(r0, &(0x7f0000000080)={0xa, 0x4001, 0x0,
> > > > > > > @loopback}, 0x1c) (async)
> > > > > > >
> > > > > > > The notable thing here is 0x4001 in connect(), which is
> > > > > > > RDS_TCP_PORT.
> > > > > > >
> > > > > > > So, the scenario would be:
> > > > > > >
> > > > > > > 1. unshare(CLONE_NEWNET) creates a per netns tcp listener in
> > > > > > > rds_tcp_listen_init().
> > > > > > > 2. syz-executor connect()s to it and creates a reqsk.
> > > > > > > 3. syz-executor exit()s immediately.
> > > > > > > 4. netns is dismantled. [0]
> > > > > > > 5. reqsk timer is fired, and UAF happens while freeing
> > > > > > > reqsk. [1]
> > > > > > > 6. listener is freed after RCU grace period. [2]
> > > > > > >
> > > > > > > Basically, reqsk assumes that the listener guarantees netns
> > > > > > > safety
> > > > > > > until all reqsk timers are expired by holding the listener's
> > > > > > > refcount.
> > > > > > > However, this was not the case for kernel sockets.
> > > > > > >
> > > > > > > Commit 740ea3c4a0b2 ("tcp: Clean up kernel listener's reqsk in
> > > > > > > inet_twsk_purge()") fixed this issue only for per-netns ehash,
> > > > > > > but
> > > > > > > the issue still exists for the global ehash.
> > > > > > >
> > > > > > > We can apply the same fix, but this issue is specific to RDS.
> > > > > > >
> > > > > > > Instead of iterating potentially large ehash and purging reqsk
> > > > > > > during
> > > > > > > netns dismantle, let's hold netns refcount for the kernel TCP
> > > > > > > listener.
> > > > > > >
> > > > > > >
> > > > > > > Reported-by: syzkaller <syzkaller@...glegroups.com>
> > > > > > > Fixes: 467fa15356ac ("RDS-TCP: Support multiple RDS-TCP listen
> > > > > > > endpoints, one per netns.")
> > > > > > > Signed-off-by: Kuniyuki Iwashima <kuniyu@...zon.com>
> > > > > > > ---
> > > > > > > net/rds/tcp_listen.c | 5 +++++
> > > > > > > 1 file changed, 5 insertions(+)
> > > > > > >
> > > > > > > diff --git a/net/rds/tcp_listen.c b/net/rds/tcp_listen.c
> > > > > > > index 05008ce5c421..4f7863932df7 100644
> > > > > > > --- a/net/rds/tcp_listen.c
> > > > > > > +++ b/net/rds/tcp_listen.c
> > > > > > > @@ -282,6 +282,11 @@ struct socket *rds_tcp_listen_init(struct
> > > > > > > net *net, bool isv6)
> > > > > > > goto out;
> > > > > > > }
> > > > > > >
> > > > > > > + __netns_tracker_free(net, &sock->sk->ns_tracker,
> > > > > > > false);
> > > > > > > + sock->sk->sk_net_refcnt = 1;
> > > > > > > + get_net_track(net, &sock->sk->ns_tracker, GFP_KERNEL);
> > > > > > > + sock_inuse_add(net, 1);
> > > > > > > +
> > > > > >
> > > > > > Why using sock_create_kern() then later 'convert' this kernel
> > > > > > socket
> > > > > > to a user one ?
> > > > > >
> > > > > > Would using __sock_create() avoid this ?
> > > > >
> > > > > I think yes, but LSM would see kern=0 in pre/post socket() hooks.
> > > > >
> > > > > Probably we can use __sock_create() in net-next and see if someone
> > > > > complains.
> > > >
> > > > I noticed the patchwork status is Changes Requested.
> > > > https://urldefense.com/v3/__https://patchwork.kernel.org/project/netdevbpf/list/?series=829213&state=*__;Kg!!ACWV5N9M2RV99hQ!KHKUQKUDnNCdiEcb4ZK1VBiYSitarEb-CAWeSJvaeK04fgW4cuWePg3Ac2HmIAPUHuqeCwgt466fHEKAAdfa$
> > > >
> > > >
> > > > Should we use __sock_create() for RDS or add another parameter
> > > > to __sock_create(..., kern=true/false, netref=true/false) and
> > > > fix other similar uses (MPTCP, SMC, Netlink) altogether ?
> > > >
> > > > Thanks!
> > >
> > > Hi all,
> > >
> > > Thank you for looking at this. I've been doing a little investigation
> > > in the area to better understand the issue and this fix. While I
> > > understand what this patch is trying to do here, I'd like to do a
> > > little more digging as to why 740ea3c4a0b2 didnt work for rds,
> >
> > 740ea3c4a0b2 works only for netns with its dedicated ehash, which
> > is unshare(CLONE_NEWNET)d with net.ipv4.tcp_child_ehash_entries != 0.
> >
> > With the diff below, we can fix the issue, but as noted in the
> > description, this slows down netns dismantle where no reqsk, this
> > is true if the netns did not have kernel TCP sockets.
>
> BTW I note that inet_twsk_purge() probably would need this "goto
> restart;" as well....
>
> sk_nulls_for_each_rcu(sk, node, &head->chain) is not a _safe variant...
Ah exactly! Will you post it or I can include it as the first patch.
The following patch will remove it and the end result will be the
same though.
>
> diff --git a/net/ipv4/inet_timewait_sock.c b/net/ipv4/inet_timewait_sock.c
> index 5befa4de5b2416281ad2795713a70d0fd847b0b2..a62031d85bcce00261193136dd72d9f57ffd34fc
> 100644
> --- a/net/ipv4/inet_timewait_sock.c
> +++ b/net/ipv4/inet_timewait_sock.c
> @@ -283,10 +283,11 @@ void inet_twsk_purge(struct inet_hashinfo
> *hashinfo, int family)
> * freed. Userspace listener and
> reqsk never exist here.
> */
> if (unlikely(sk->sk_state == TCP_NEW_SYN_RECV &&
> - hashinfo->pernet)) {
> +
> !refcount_read(&sock_net(sk)->ns.count))) {
> struct request_sock *req =
> inet_reqsk(sk);
>
>
> inet_csk_reqsk_queue_drop_and_put(req->rsk_listener, req);
> + goto restart;
> }
>
> continue;
Powered by blists - more mailing lists