netdev - [PATCH net-next 0/3] net: socket sendmsg MSG_ZEROCOPY

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <20240409205300.1346681-1-zijianzhang@bytedance.com>
Date: Tue,  9 Apr 2024 20:52:57 +0000
From: zijianzhang@...edance.com
To: netdev@...r.kernel.org
Cc: edumazet@...gle.com,
	willemdebruijn.kernel@...il.com,
	davem@...emloft.net,
	kuba@...nel.org,
	cong.wang@...edance.com,
	xiaochun.lu@...edance.com,
	Zijian Zhang <zijianzhang@...edance.com>
Subject: [PATCH net-next 0/3] net: socket sendmsg MSG_ZEROCOPY_UARG

From: Zijian Zhang <zijianzhang@...edance.com>

Original notification mechanism needs poll + recvmmsg which is not
easy for applcations to accommodate. And, it also incurs unignorable
overhead including extra system calls and usage of optmem.

While making maximum reuse of the existing MSG_ZEROCOPY related code,
this patch set introduces zerocopy socket send flag MSG_ZEROCOPY_UARG.
It provides a new notification method. Users of sendmsg pass a control
message as a placeholder for the incoming notifications. Upon returning,
kernel embeds notifications directly into user arguments passed in. By
doing so, we can significantly reduce the complexity and overhead for
managing notifications. In an ideal pattern, the user will keep calling
sendmsg with MSG_ZEROCOPY_UARG flag, and the notification will be
delivered as soon as possible.

MSG_ZEROCOPY_UARG does not need to queue skb into errqueue. Thus,
skbuffs allocated from optmem are not a must. In theory, a new struct
carrying the zcopy information should be defined along with its memory
management code. However, existing zcopy generic code assumes the
information is skbuff. Given the very limited performance gain or maybe
no gain of this method, and the need to change a lot of existing code,
we still use skbuffs allocated from optmem to carry zcopy information.

* Performance

I extend the selftests/msg_zerocopy.c to accommodate the new flag, test
result is as follows, the new flag performs 7% better in TCP and 4%
better in UDP.

cfg_notification_limit = 8
+---------------------+---------+---------+---------+---------+
| Test Type / Protocol| TCP v4  | TCP v6  | UDP v4  | UDP v6  |
+---------------------+---------+---------+---------+---------+
| Copy                | 5328    | 5159    | 8581    | 8457    |
+---------------------+---------+---------+---------+---------+
| ZCopy               | 5877    | 5568    | 10314   | 10091   |
+---------------------+---------+---------+---------+---------+
| New ZCopy           | 6254    | 5901    | 10674   | 10293   |
+---------------------+---------+---------+---------+---------+
| ZCopy / Copy        | 110.30% | 107.93% | 120.20% | 119.32% |
+---------------------+---------+---------+---------+---------+
| New ZCopy / Copy    | 117.38% | 114.38% | 124.39% | 121.71% |
+---------------------+---------+---------+---------+---------+

Zijian Zhang (3):
  sock: add MSG_ZEROCOPY_UARG
  selftests: fix OOM problem in msg_zerocopy selftest
  selftests: add msg_zerocopy_uarg test

 include/linux/skbuff.h                      |   7 +-
 include/linux/socket.h                      |   1 +
 include/linux/tcp.h                         |   3 +
 include/linux/udp.h                         |   3 +
 include/net/sock.h                          |  17 +++
 include/net/udp.h                           |   1 +
 include/uapi/asm-generic/socket.h           |   2 +
 include/uapi/linux/socket.h                 |  17 +++
 net/core/skbuff.c                           | 137 ++++++++++++++++--
 net/core/sock.c                             |  50 +++++++
 net/ipv4/ip_output.c                        |   6 +-
 net/ipv4/tcp.c                              |   7 +-
 net/ipv4/udp.c                              |   9 ++
 net/ipv6/ip6_output.c                       |   5 +-
 net/ipv6/udp.c                              |   9 ++
 net/vmw_vsock/virtio_transport_common.c     |   2 +-
 tools/testing/selftests/net/msg_zerocopy.c  | 150 ++++++++++++++++++--
 tools/testing/selftests/net/msg_zerocopy.sh |   1 +
 18 files changed, 397 insertions(+), 30 deletions(-)

-- 
2.20.1