netdev - [PATCH net-next] udp: introduce and use indirect call wrapper for data ready()

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date: Mon, 17 Jul 2023 11:52:29 +0200
From: Paolo Abeni <pabeni@...hat.com>
To: netdev@...r.kernel.org
Cc: "David S. Miller" <davem@...emloft.net>,
	Eric Dumazet <edumazet@...gle.com>,
	Jakub Kicinski <kuba@...nel.org>,
	Willem de Bruijn <willemdebruijn.kernel@...il.com>,
	David Ahern <dsahern@...nel.org>
Subject: [PATCH net-next] udp: introduce and use indirect call wrapper for data ready()

In most cases UDP sockets use the default data ready callback.
This patch Introduces and uses a specific indirect call wrapper for
such callback to avoid an indirect call in fastpath.

The above gives small but measurable performance gain under UDP flood.

Signed-off-by: Paolo Abeni <pabeni@...hat.com>
---
Note that this helper could be used for TCP, too. I did not send such
patch right away because in my tests the perf delta there is below the
noise level even in RR scenarios and the patch would be a little more
invasive - there are more sk_data_ready() invocation places.
---
 include/net/sock.h | 4 ++++
 net/ipv4/udp.c     | 2 +-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/include/net/sock.h b/include/net/sock.h
index 2eb916d1ff64..1b26dbecdcca 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -2947,6 +2947,10 @@ static inline bool sk_dev_equal_l3scope(struct sock *sk, int dif)
 }
 
 void sock_def_readable(struct sock *sk);
+static inline void sk_data_ready(struct sock *sk)
+{
+	INDIRECT_CALL_1(sk->sk_data_ready, sock_def_readable, sk);
+}
 
 int sock_bindtoindex(struct sock *sk, int ifindex, bool lock_sk);
 void sock_set_timestamp(struct sock *sk, int optname, bool valbool);
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 42a96b3547c9..5aec1854b711 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -1553,7 +1553,7 @@ int __udp_enqueue_schedule_skb(struct sock *sk, struct sk_buff *skb)
 	spin_unlock(&list->lock);
 
 	if (!sock_flag(sk, SOCK_DEAD))
-		sk->sk_data_ready(sk);
+		sk_data_ready(sk);
 
 	busylock_release(busy);
 	return 0;
-- 
2.41.0