netdev - [PATCH net-next 2/3] net: move sk_drops out of sock_write

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250825195947.4073595-3-edumazet@google.com>
Date: Mon, 25 Aug 2025 19:59:46 +0000
From: Eric Dumazet <edumazet@...gle.com>
To: "David S . Miller" <davem@...emloft.net>, Jakub Kicinski <kuba@...nel.org>, 
	Paolo Abeni <pabeni@...hat.com>
Cc: Simon Horman <horms@...nel.org>, netdev@...r.kernel.org, eric.dumazet@...il.com, 
	Willem de Bruijn <willemb@...gle.com>, Kuniyuki Iwashima <kuniyu@...gle.com>, 
	Eric Dumazet <edumazet@...gle.com>
Subject: [PATCH net-next 2/3] net: move sk_drops out of sock_write_rx group

Move sk_drops into a dedicated cache line.

When a packet flood hits one or more sockets, many cpus
have to update sk->sk_drops.

This slows down consumers, because currently
sk_drops is in sock_write_rx group.

Moving sk->sk_drops into a dedicated cache line
makes sure that consumers no longer suffer from
false sharing if/when producers only change sk->sk_drops.

Tested with the following stress test, sending about 11 Mpps:

super_netperf 20 -t UDP_STREAM -H DUT -l10 -- -n -P,1000 -m 120
Note: due to socket lookup, only one UDP socket will receive
packets on DUT.

Then measure receiver (DUT) behavior. We can see both
consumer and BH handlers can process more packets per second.

Before:

nstat -n ; sleep 1 ; nstat | grep Udp
Udp6InDatagrams                 615091             0.0
Udp6InErrors                    3904277            0.0
Udp6RcvbufErrors                3904277            0.0

After:
nstat -n ; sleep 1 ; nstat | grep Udp
Udp6InDatagrams                 855592             0.0
Udp6InErrors                    5621467            0.0
Udp6RcvbufErrors                5621467            0.0

Signed-off-by: Eric Dumazet <edumazet@...gle.com>
---
 include/net/sock.h | 6 +++---
 net/core/sock.c    | 1 -
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/include/net/sock.h b/include/net/sock.h
index 34d7029eb622773e40e7c4ebd422d33b1c0a7836..f40e3c4883be32c8282694ab215bcf79eb87cbd7 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -390,7 +390,6 @@ struct sock {
 
 	__cacheline_group_begin(sock_write_rx);
 
-	atomic_t		sk_drops;
 	__s32			sk_peek_off;
 	struct sk_buff_head	sk_error_queue;
 	struct sk_buff_head	sk_receive_queue;
@@ -564,13 +563,14 @@ struct sock {
 #ifdef CONFIG_BPF_SYSCALL
 	struct bpf_local_storage __rcu	*sk_bpf_storage;
 #endif
-	struct rcu_head		sk_rcu;
-	netns_tracker		ns_tracker;
 	struct xarray		sk_user_frags;
 
 #if IS_ENABLED(CONFIG_PROVE_LOCKING) && IS_ENABLED(CONFIG_MODULES)
 	struct module		*sk_owner;
 #endif
+	atomic_t		sk_drops ____cacheline_aligned_in_smp;
+	struct rcu_head		sk_rcu;
+	netns_tracker		ns_tracker;
 };
 
 struct sock_bh_locked {
diff --git a/net/core/sock.c b/net/core/sock.c
index 75368823969a7992a55a6f40d87ffb8886de2f39..cd7c7ed7ff51070d20658684ff796c58c8c09995 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -4436,7 +4436,6 @@ EXPORT_SYMBOL(sk_ioctl);
 
 static int __init sock_struct_check(void)
 {
-	CACHELINE_ASSERT_GROUP_MEMBER(struct sock, sock_write_rx, sk_drops);
 	CACHELINE_ASSERT_GROUP_MEMBER(struct sock, sock_write_rx, sk_peek_off);
 	CACHELINE_ASSERT_GROUP_MEMBER(struct sock, sock_write_rx, sk_error_queue);
 	CACHELINE_ASSERT_GROUP_MEMBER(struct sock, sock_write_rx, sk_receive_queue);
-- 
2.51.0.261.g7ce5a0a67e-goog