linux-kernel - [PATCH] percpu_rwsem: let percpu

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250328030519.897032-1-CruzZhao@linux.alibaba.com>
Date: Fri, 28 Mar 2025 11:05:17 +0800
From: Cruz Zhao <CruzZhao@...ux.alibaba.com>
To: peterz@...radead.org,
	mingo@...hat.com,
	boqun.feng@...il.com,
	will@...nel.org,
	longman@...hat.com
Cc: linux-kernel@...r.kernel.org
Subject: [PATCH] percpu_rwsem: let percpu_rwsem writer get rwsem faster

In the scenario where a large number of containers are created
at the same time, there will be a lot of tasks created in a
short time, and they will be written into cgroup.procs.

copy_process() will require the cgroup_threadgroup_rwsem read
lock, cgroup_procs_write will require the cgroup_threadgroup_rwsem
write lock. As the readers will pre-increase the read_count and
then check whether there is any writers, resulting that the
writer may be starving, especially when there is a steady stream
of readers.

To alleviate this problem, we add one more check whether there
are writers waiting before increasing the read_count, to make
writers getting lock faster.

Signed-off-by: Cruz Zhao <CruzZhao@...ux.alibaba.com>
---
 kernel/locking/percpu-rwsem.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/kernel/locking/percpu-rwsem.c b/kernel/locking/percpu-rwsem.c
index 6083883c4fe0..66bf18c28b43 100644
--- a/kernel/locking/percpu-rwsem.c
+++ b/kernel/locking/percpu-rwsem.c
@@ -47,6 +47,11 @@ EXPORT_SYMBOL_GPL(percpu_free_rwsem);

 static bool __percpu_down_read_trylock(struct percpu_rw_semaphore *sem)
 {
+	if (unlikely(atomic_read_acquire(&sem->block))) {
+		rcuwait_wake_up(&sem->writer);
+		return false;
+	}
+
 	this_cpu_inc(*sem->read_count);

 	/*
-- 
2.43.5