netdev - RE: [bug report] one possible out-of-order issue in sockmap

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <633492fb8ddc2_2944220881@john.notmuch>
Date:   Wed, 28 Sep 2022 11:31:23 -0700
From:   John Fastabend <john.fastabend@...il.com>
To:     "liujian (CE)" <liujian56@...wei.com>,
        John Fastabend <john.fastabend@...il.com>,
        Cong Wang <xiyou.wangcong@...il.com>
Cc:     Jakub Sitnicki <jakub@...udflare.com>,
        Eric Dumazet <edumazet@...gle.com>,
        davem <davem@...emloft.net>,
        "yoshfuji@...ux-ipv6.org" <yoshfuji@...ux-ipv6.org>,
        "dsahern@...nel.org" <dsahern@...nel.org>,
        Jakub Kicinski <kuba@...nel.org>,
        Paolo Abeni <pabeni@...hat.com>,
        netdev <netdev@...r.kernel.org>,
        "bpf@...r.kernel.org" <bpf@...r.kernel.org>
Subject: RE: [bug report] one possible out-of-order issue in sockmap

liujian (CE) wrote:
> 
> 
> > -----Original Message-----
> > From: John Fastabend [mailto:john.fastabend@...il.com]
> > Sent: Tuesday, September 27, 2022 5:16 AM
> > To: liujian (CE) <liujian56@...wei.com>; Cong Wang
> > <xiyou.wangcong@...il.com>
> > Cc: John Fastabend <john.fastabend@...il.com>; Jakub Sitnicki
> > <jakub@...udflare.com>; Eric Dumazet <edumazet@...gle.com>; davem
> > <davem@...emloft.net>; yoshfuji@...ux-ipv6.org; dsahern@...nel.org;
> > Jakub Kicinski <kuba@...nel.org>; Paolo Abeni <pabeni@...hat.com>;
> > netdev <netdev@...r.kernel.org>; bpf@...r.kernel.org
> > Subject: RE: [bug report] one possible out-of-order issue in sockmap
> > 
> > liujian (CE) wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Cong Wang [mailto:xiyou.wangcong@...il.com]
> > > > Sent: Monday, September 26, 2022 2:26 AM
> > > > To: liujian (CE) <liujian56@...wei.com>
> > > > Cc: John Fastabend <john.fastabend@...il.com>; Jakub Sitnicki
> > > > <jakub@...udflare.com>; Eric Dumazet <edumazet@...gle.com>;
> > davem
> > > > <davem@...emloft.net>; yoshfuji@...ux-ipv6.org; dsahern@...nel.org;
> > > > Jakub Kicinski <kuba@...nel.org>; Paolo Abeni <pabeni@...hat.com>;
> > > > netdev <netdev@...r.kernel.org>; bpf@...r.kernel.org
> > > > Subject: Re: [bug report] one possible out-of-order issue in sockmap
> > > >
> > > > On Sat, Sep 24, 2022 at 07:59:15AM +0000, liujian (CE) wrote:
> > > > > Hello,
> > > > >
> > > > > I had a scp failure problem here. I analyze the code, and the
> > > > > reasons may
> > > > be as follows:
> > > > >
> > > > > From commit e7a5f1f1cd00 ("bpf/sockmap: Read psock ingress_msg
> > > > before
> > > > > sk_receive_queue", if we use sockops
> > > > > (BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB
> > > > > and BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB) to enable socket's
> > > > sockmap
> > > > > function, and don't enable strparse and verdict function, the
> > > > > out-of-order problem may occur in the following process.
> > > > >
> > > > > client SK                                   server SK
> > > > > ------------------------------------------------------------------
> > > > > ----
> > > > > ----
> > > > > tcp_rcv_synsent_state_process
> > > > >   tcp_finish_connect
> > > > >     tcp_init_transfer
> > > > >       tcp_set_state(sk, TCP_ESTABLISHED);
> > > > >       // insert SK to sockmap
> > > > >     wake up waitter
> > > > >     tcp_send_ack
> > > > >
> > > > > tcp_bpf_sendmsg(msgA)
> > > > > // msgA will go tcp stack
> > > > >                                             tcp_rcv_state_process
> > > > >                                               tcp_init_transfer
> > > > >                                                 //insert SK to sockmap
> > > > >                                               tcp_set_state(sk,
> > > > >                                                      TCP_ESTABLISHED)
> > > > >                                               wake up waitter
> > > >
> > > > Here after the socket is inserted to a sockmap, its
> > > > ->sk_data_ready() is already replaced with
> > > > sk_psock_verdict_data_ready(), so msgA should go to sockmap, not TCP
> > stack?
> > > >
> > > It is TCP stack.  Here I only enable BPF_SK_MSG_VERDICT type.
> > > bpftool prog load bpf_redir.o /sys/fs/bpf/bpf_redir map name
> > > sock_ops_map pinned /sys/fs/bpf/sock_ops_map bpftool prog attach
> > > pinned /sys/fs/bpf/bpf_redir msg_verdict pinned
> > > /sys/fs/bpf/sock_ops_map
> > 
> > Is the sender using FAST_OPEN by any chance? We know this bug exists in
> > this case. Fix tbd.
> 
> FAST_OPEN is not used.

OK thanks for the reproducer I'll take a look this afternoon.