netdev - [bpf, Bug report] get an EPOLLRDHUP event before read all data from kernel buffer when using sockmap

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <66cf4c3f.4883.1801d397ca8.Coremail.wujianguo106@163.com>
Date:   Tue, 12 Apr 2022 18:02:28 +0800 (CST)
From:   "Jianguo Wu" <wujianguo106@....com>
To:     netdev@...r.kernel.org, bpf@...r.kernel.org
Cc:     daniel@...earbox.net, john.fastabend@...il.com
Subject: [bpf, Bug report] get an EPOLLRDHUP event before read all data from
 kernel buffer when using sockmap

Hi all,
    I did some ebpf sockmap test using demo from https://github.com/ArthurChiao/socket-acceleration-with-ebpf,
that using a BPF_PROG_TYPE_SOCK_OPS prog to store local passive/active established socket in a BPF_MAP_TYPE_SOCKHASH map,
and then use a BPF_PROG_TYPE_SK_MSG prog to redirect messages between two local sockets.
    And if I use nginx as a proxy, webfsd as a upstream server, like this: curl <--> nginx(192.168.174.128:80)<-->(webfsd)127.0.0.1:8090,
it highly download failed, curl only recv partial data and was stuck.
    From nginx debug log(see the attached file), it seems nginx get an EPOLLRDHUP event before read all data from kernel buffer.
I think it happened like this:
    1)kernel send FIN after webfsd send out all data, and for nginx view, when kernel recv the FIN, will shutdown recv:
      tcp_fin(): sk->sk_shutdown |= RCV_SHUTDOWN;
    2)and nginx will get an EPOLLRDHUP event:
      tcp_poll():
      if (sk->sk_shutdown & RCV_SHUTDOWN)
          mask |= EPOLLIN | EPOLLRDNORM | EPOLLRDHUP;
    3)so nginx will not read all data that already in kernel recv buffer?

Is this a kernel ebpf/tcp BUG?
View attachment "ngx-access-debug-log.txt" of type "text/plain" (11367 bytes)