linux-kernel - [tip: sched/core] sched/fair: Refactor can_migrate

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <173961695638.10177.15446908649592495018.tip-bot2@tip-bot2>
Date: Sat, 15 Feb 2025 10:55:56 -0000
From: "tip-bot2 for I Hsin Cheng" <tip-bot2@...utronix.de>
To: linux-tip-commits@...r.kernel.org
Cc: I Hsin Cheng <richard120310@...il.com>,
 "Peter Zijlstra (Intel)" <peterz@...radead.org>, x86@...nel.org,
 linux-kernel@...r.kernel.org
Subject:
 [tip: sched/core] sched/fair: Refactor can_migrate_task() to elimate looping

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     d34e798094ca7be935b629a42f8b237d4d5b7f1d
Gitweb:        https://git.kernel.org/tip/d34e798094ca7be935b629a42f8b237d4d5b7f1d
Author:        I Hsin Cheng <richard120310@...il.com>
AuthorDate:    Mon, 10 Feb 2025 18:30:18 +08:00
Committer:     Peter Zijlstra <peterz@...radead.org>
CommitterDate: Fri, 14 Feb 2025 10:32:01 +01:00

sched/fair: Refactor can_migrate_task() to elimate looping

The function "can_migrate_task()" utilize "for_each_cpu_and" with a
"if" statement inside to find the destination cpu. It's the same logic
to find the first set bit of the result of the bitwise-AND of
"env->dst_grpmask", "env->cpus" and "p->cpus_ptr".

Refactor it by using "cpumask_first_and_and()" to perform bitwise-AND
for "env->dst_grpmask", "env->cpus" and "p->cpus_ptr" and pick the
first cpu within the intersection as the destination cpu, so we can
elimate the need of looping and multiple times of branch.

After the refactoring this part of the code can speed up from ~115ns
to ~54ns, according to the test below.

Ran the test for 5 times and the result is showned in the following
table, and the test script is paste in next section.

  -------------------------------------------------------
  |Old method|  130|  118|  115|  109|  106|  avg ~115ns|
  -------------------------------------------------------
  |New method|   58|   55|   54|   48|   55|  avg  ~54ns|
  -------------------------------------------------------

Signed-off-by: I Hsin Cheng <richard120310@...il.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@...radead.org>
Link: https://lkml.kernel.org/r/20250210103019.283824-1-richard120310@gmail.com
---
 kernel/sched/fair.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 9279bfb..857808d 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -9439,12 +9439,11 @@ int can_migrate_task(struct task_struct *p, struct lb_env *env)
 			return 0;
 
 		/* Prevent to re-select dst_cpu via env's CPUs: */
-		for_each_cpu_and(cpu, env->dst_grpmask, env->cpus) {
-			if (cpumask_test_cpu(cpu, p->cpus_ptr)) {
-				env->flags |= LBF_DST_PINNED;
-				env->new_dst_cpu = cpu;
-				break;
-			}
+		cpu = cpumask_first_and_and(env->dst_grpmask, env->cpus, p->cpus_ptr);
+
+		if (cpu < nr_cpu_ids) {
+			env->flags |= LBF_DST_PINNED;
+			env->new_dst_cpu = cpu;
 		}
 
 		return 0;