netdev - [PATCH net-next 02/27] sock: optimise sock_def_write

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Sun,  3 Apr 2022 14:06:14 +0100
From:   Pavel Begunkov <asml.silence@...il.com>
To:     netdev@...r.kernel.org, "David S . Miller" <davem@...emloft.net>,
        Jakub Kicinski <kuba@...nel.org>
Cc:     Eric Dumazet <edumazet@...gle.com>, Wei Liu <wei.liu@...nel.org>,
        Paul Durrant <paul@....org>,
        Pavel Begunkov <asml.silence@...il.com>
Subject: [PATCH net-next 02/27] sock: optimise sock_def_write_space send refcounting

sock_def_write_space() is extensively used by UDP and there is some
room for optimisation. When sock_wfree() needs to do ->sk_write_space(),
it modifies ->sk_wmem_alloc in two steps. First, it puts all but one
refs and calls ->sk_write_space(), and then puts down remaining 1.
That's needed because the callback relies on ->sk_wmem_alloc being
subbed but something should hold the socket alive.

The idea behind this patch is to take advantage of SOCK_RCU_FREE and
ensure the socket is not freed by wrapping ->sk_write_space() in an RCU
section. Then we can remove one extra refcount atomic.

Note: not all callbacks might be RCU prepared, so we carve out a
sock_def_write_space() specific path.

Signed-off-by: Pavel Begunkov <asml.silence@...il.com>
---
 net/core/sock.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/net/core/sock.c b/net/core/sock.c
index f5766d6e27cb..9389bb602c64 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -144,6 +144,8 @@
 static DEFINE_MUTEX(proto_list_mutex);
 static LIST_HEAD(proto_list);

+static void sock_def_write_space(struct sock *sk);
+
 /**
  * sk_ns_capable - General socket capability test
  * @sk: Socket to use a capability on or through
@@ -2300,8 +2302,20 @@ void sock_wfree(struct sk_buff *skb)
 {
 	struct sock *sk = skb->sk;
 	unsigned int len = skb->truesize;
+	bool free;

 	if (!sock_flag(sk, SOCK_USE_WRITE_QUEUE)) {
+		if (sock_flag(sk, SOCK_RCU_FREE) &&
+		    sk->sk_write_space == sock_def_write_space) {
+			rcu_read_lock();
+			free = refcount_sub_and_test(len, &sk->sk_wmem_alloc);
+			sock_def_write_space(sk);
+			rcu_read_unlock();
+			if (unlikely(free))
+				__sk_free(sk);
+			return;
+		}
+
 		/*
 		 * Keep a reference on sk_wmem_alloc, this will be released
 		 * after sk_write_space() call
-- 
2.35.1