| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <20251104-scratch-bobbyeshleman-devmem-tcp-token-upstream-v6-0-ea98cf4d40b3@meta.com>
Date: Tue, 04 Nov 2025 17:23:19 -0800
From: Bobby Eshleman <bobbyeshleman@...il.com>
To: "David S. Miller" <davem@...emloft.net>,
Eric Dumazet <edumazet@...gle.com>, Jakub Kicinski <kuba@...nel.org>,
Paolo Abeni <pabeni@...hat.com>, Simon Horman <horms@...nel.org>,
Kuniyuki Iwashima <kuniyu@...gle.com>,
Willem de Bruijn <willemb@...gle.com>, Neal Cardwell <ncardwell@...gle.com>,
David Ahern <dsahern@...nel.org>, Arnd Bergmann <arnd@...db.de>,
Jonathan Corbet <corbet@....net>, Andrew Lunn <andrew+netdev@...n.ch>,
Shuah Khan <shuah@...nel.org>, Mina Almasry <almasrymina@...gle.com>
Cc: netdev@...r.kernel.org, linux-kernel@...r.kernel.org,
linux-arch@...r.kernel.org, linux-doc@...r.kernel.org,
linux-kselftest@...r.kernel.org, Stanislav Fomichev <sdf@...ichev.me>,
Bobby Eshleman <bobbyeshleman@...a.com>
Subject: [PATCH net-next v6 0/6] net: devmem: improve cpu cost of RX token
management
This series improves the CPU cost of RX token management by adding a
socket option that configures the socket to avoid the xarray allocator
and instead use an niov array and a uref field in niov.
Improvement is ~13% cpu util per RX user thread.
Using kperf, the following results were observed:
Before:
Average RX worker idle %: 13.13, flows 4, test runs 11
After:
Average RX worker idle %: 26.32, flows 4, test runs 11
Two other approaches were tested, but with no improvement. Namely, 1)
using a hashmap for tokens and 2) keeping an xarray of atomic counters
but using RCU so that the hotpath could be mostly lockless. Neither of
these approaches proved better than the simple array in terms of CPU.
The sockopt SO_DEVMEM_AUTORELEASE is added to toggle the optimization.
It defaults to 0 (i.e., optimization on).
Note that prior revs reported only a 5% gain. This lower gain was
measured with cpu frequency boosting (unknowingly) disabled. A
consistent ~13% is measured for both kperf and nccl workloads with cpu
frequency boosting on.
To: David S. Miller <davem@...emloft.net>
To: Eric Dumazet <edumazet@...gle.com>
To: Jakub Kicinski <kuba@...nel.org>
To: Paolo Abeni <pabeni@...hat.com>
To: Simon Horman <horms@...nel.org>
To: Kuniyuki Iwashima <kuniyu@...gle.com>
To: Willem de Bruijn <willemb@...gle.com>
To: Neal Cardwell <ncardwell@...gle.com>
To: David Ahern <dsahern@...nel.org>
To: Mina Almasry <almasrymina@...gle.com>
To: Arnd Bergmann <arnd@...db.de>
To: Jonathan Corbet <corbet@....net>
To: Andrew Lunn <andrew+netdev@...n.ch>
To: Shuah Khan <shuah@...nel.org>
Cc: Stanislav Fomichev <sdf@...ichev.me>
Cc: netdev@...r.kernel.org
Cc: linux-kernel@...r.kernel.org
Cc: linux-arch@...r.kernel.org
Cc: linux-doc@...r.kernel.org
Cc: linux-kselftest@...r.kernel.org
Signed-off-by: Bobby Eshleman <bobbyeshleman@...a.com>
Changes in v6:
- renamed 'net: devmem: use niov array for token management' to refer to
optionality of new config
- added documentation and tests
- make autorelease flag per-socket sockopt instead of binding
field / sysctl
- many per-patch changes (see Changes sections per-patch)
- Link to v5: https://lore.kernel.org/r/20251023-scratch-bobbyeshleman-devmem-tcp-token-upstream-v5-0-47cb85f5259e@meta.com
Changes in v5:
- add sysctl to opt-out of performance benefit, back to old token release
- Link to v4: https://lore.kernel.org/all/20250926-scratch-bobbyeshleman-devmem-tcp-token-upstream-v4-0-39156563c3ea@meta.com
Changes in v4:
- rebase to net-next
- Link to v3: https://lore.kernel.org/r/20250926-scratch-bobbyeshleman-devmem-tcp-token-upstream-v3-0-084b46bda88f@meta.com
Changes in v3:
- make urefs per-binding instead of per-socket, reducing memory
footprint
- fallback to cleaning up references in dmabuf unbind if socket
leaked tokens
- drop ethtool patch
- Link to v2: https://lore.kernel.org/r/20250911-scratch-bobbyeshleman-devmem-tcp-token-upstream-v2-0-c80d735bd453@meta.com
Changes in v2:
- net: ethtool: prevent user from breaking devmem single-binding rule
(Mina)
- pre-assign niovs in binding->vec for RX case (Mina)
- remove WARNs on invalid user input (Mina)
- remove extraneous binding ref get (Mina)
- remove WARN for changed binding (Mina)
- always use GFP_ZERO for binding->vec (Mina)
- fix length of alloc for urefs
- use atomic_set(, 0) to initialize sk_user_frags.urefs
- Link to v1: https://lore.kernel.org/r/20250902-scratch-bobbyeshleman-devmem-tcp-token-upstream-v1-0-d946169b5550@meta.com
---
Bobby Eshleman (6):
net: devmem: rename tx_vec to vec in dmabuf binding
net: devmem: refactor sock_devmem_dontneed for autorelease split
net: devmem: prepare for autorelease rx token management
net: devmem: add SO_DEVMEM_AUTORELEASE for autorelease control
net: devmem: document SO_DEVMEM_AUTORELEASE socket option
net: devmem: add tests for SO_DEVMEM_AUTORELEASE socket option
Documentation/networking/devmem.rst | 70 +++++++++-
include/net/netmem.h | 1 +
include/net/sock.h | 13 +-
include/uapi/asm-generic/socket.h | 2 +
net/core/devmem.c | 54 +++++---
net/core/devmem.h | 4 +-
net/core/sock.c | 152 ++++++++++++++++++----
net/ipv4/tcp.c | 69 ++++++++--
net/ipv4/tcp_ipv4.c | 11 +-
net/ipv4/tcp_minisocks.c | 5 +-
tools/include/uapi/asm-generic/socket.h | 2 +
tools/testing/selftests/drivers/net/hw/devmem.py | 115 +++++++++++++++-
tools/testing/selftests/drivers/net/hw/ncdevmem.c | 20 ++-
13 files changed, 453 insertions(+), 65 deletions(-)
---
base-commit: 255d75ef029f33f75fcf5015052b7302486f7ad2
change-id: 20250829-scratch-bobbyeshleman-devmem-tcp-token-upstream-292be174d503
Best regards,
--
Bobby Eshleman <bobbyeshleman@...a.com>
Powered by blists - more mailing lists