Message-ID: <de41de4c-0345-34d7-7c36-4345258b7ba8@sberdevices.ru>
Date:   Fri, 19 Aug 2022 05:21:58 +0000
From:   Arseniy Krasnov <AVKrasnov@...rdevices.ru>
To:     Stefano Garzarella <sgarzare@...hat.com>,
        "David S. Miller" <davem@...emloft.net>,
        "edumazet@...gle.com" <edumazet@...gle.com>,
        "Jakub Kicinski" <kuba@...nel.org>,
        Paolo Abeni <pabeni@...hat.com>,
        "kys@...rosoft.com" <kys@...rosoft.com>,
        "haiyangz@...rosoft.com" <haiyangz@...rosoft.com>,
        "sthemmin@...rosoft.com" <sthemmin@...rosoft.com>,
        "wei.liu@...nel.org" <wei.liu@...nel.org>,
        Dexuan Cui <decui@...rosoft.com>,
        Stefan Hajnoczi <stefanha@...hat.com>,
        Bryan Tan <bryantan@...are.com>,
        Vishnu Dasa <vdasa@...are.com>,
        Krasnov Arseniy <oxffffaa@...il.com>,
        "Michael S. Tsirkin" <mst@...hat.com>
CC:     "virtualization@...ts.linux-foundation.org" 
        <virtualization@...ts.linux-foundation.org>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-hyperv@...r.kernel.org" <linux-hyperv@...r.kernel.org>,
        "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
        kernel <kernel@...rdevices.ru>,
        VMware PV-Drivers Reviewers <pv-drivers@...are.com>
Subject: [PATCH net-next v4 0/9] vsock: updates for SO_RCVLOWAT handling

Hello,

This patchset includes some updates for SO_RCVLOWAT:

1) af_vsock:
   During my experiments with zerocopy receive, i found, that in some
   cases, poll() implementation violates POSIX: when socket has non-
   default SO_RCVLOWAT(e.g. not 1), poll() will always set POLLIN and
   POLLRDNORM bits in 'revents' even number of bytes available to read
   on socket is smaller than SO_RCVLOWAT value. In this case,user sees
   POLLIN flag and then tries to read data(for example using  'read()'
   call), but read call will be blocked, because  SO_RCVLOWAT logic is
   supported in dequeue loop in af_vsock.c. But the same time,  POSIX
   requires that:

   "POLLIN     Data other than high-priority data may be read without
               blocking.
    POLLRDNORM Normal data may be read without blocking."

   See https://www.open-std.org/jtc1/sc22/open/n4217.pdf, page 293.

   So, we have, that poll() syscall returns POLLIN, but read call will
   be blocked.

   Also in man page socket(7) i found that:

   "Since Linux 2.6.28, select(2), poll(2), and epoll(7) indicate a
   socket as readable only if at least SO_RCVLOWAT bytes are available."

   I checked TCP callback for poll()(net/ipv4/tcp.c, tcp_poll()), it
   uses SO_RCVLOWAT value to set POLLIN bit, also i've tested TCP with
   this case for TCP socket, it works as POSIX required.

   I've added some fixes to af_vsock.c and virtio_transport_common.c,
   test is also implemented.

2) virtio/vsock:
   It adds some optimization to wake ups, when new data arrived. Now,
   SO_RCVLOWAT is considered before wake up sleepers who wait new data.
   There is no sense, to kick waiter, when number of available bytes
   in socket's queue < SO_RCVLOWAT, because if we wake up reader in
   this case, it will wait for SO_RCVLOWAT data anyway during dequeue,
   or in poll() case, POLLIN/POLLRDNORM bits won't be set, so such
   exit from poll() will be "spurious". This logic is also used in TCP
   sockets.

3) vmci/vsock:
   Same as 2), but i'm not sure about this changes. Will be very good,
   to get comments from someone who knows this code.

4) Hyper-V:
   As Dexuan Cui mentioned, for Hyper-V transport it is difficult to
   support SO_RCVLOWAT, so he suggested to disable this feature for
   Hyper-V.

Thank You

Arseniy Krasnov(9):
 vsock: SO_RCVLOWAT transport set callback
 hv_sock: disable SO_RCVLOWAT support
 virtio/vsock: use 'target' in notify_poll_in callback
 vmci/vsock: use 'target' in notify_poll_in callback
 vsock: pass sock_rcvlowat to notify_poll_in as target
 vsock: add API call for data ready
 virtio/vsock: check SO_RCVLOWAT before wake up reader
 vmci/vsock: check SO_RCVLOWAT before wake up reader
 vsock_test: POLLIN + SO_RCVLOWAT test

 include/net/af_vsock.h                       |   2 +
 net/vmw_vsock/af_vsock.c                     |  33 +++++++-
 net/vmw_vsock/hyperv_transport.c             |   7 ++
 net/vmw_vsock/virtio_transport_common.c      |   7 +-
 net/vmw_vsock/vmci_transport_notify.c        |  10 +--
 net/vmw_vsock/vmci_transport_notify_qstate.c |  12 +--
 tools/testing/vsock/vsock_test.c             | 108 +++++++++++++++++++++++++++
 7 files changed, 162 insertions(+), 17 deletions(-)

 Changelog:

 v1 -> v2:
 1) Patches for VMCI transport(same as for virtio-vsock).
 2) Patches for Hyper-V transport(disabling SO_RCVLOWAT setting).
 3) Waiting logic in test was updated(sleep() -> poll()).

 v2 -> v3:
 1) Patches were reordered.
 2) Commit message updated in 0005.
 3) Check 'transport' pointer in 0001 for NULL.

 v3 -> v4:
 1) vsock_set_rcvlowat() logic changed. Previous version required
    assigned transport and always called its 'set_rcvlowat' callback
    (if present). Now, assignment is not needed.
 2) 0003,0004,0005,0006,0007,0008 - commit messages updated.
 3) 0009 - small refactoring and style fixes.
 4) RFC tag was removed.

-- 
2.25.1

Powered by blists - more mailing lists