linux-kernel - [PATCH v2] sched: Skip double execution of pick_next_task

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date:	Thu, 24 Apr 2014 10:14:34 -0700
From:	Tim Chen <tim.c.chen@...ux.intel.com>
To:	Ingo Molnar <mingo@...e.hu>, Peter Zijlstra <peterz@...radead.org>
Cc:	linux-kernel@...r.kernel.org, Andi Kleen <ak@...ux.intel.com>,
	Len Brown <len.brown@...el.com>
Subject: [PATCH v2] sched: Skip double execution of pick_next_task_fair

The current code will call pick_next_task_fair a second time in the
slow path if we did not pull any task in our first try.  This is really
unnecessary as we already know no task can be pulled and it doubles the
delay for the cpu to enter idle.

We instrumented some network workloads and saw that pick_next_task_fair
was called frequently before a cpu entered idle.  The call to
pick_next_task_fair can add non trivial latency as it calls load_balance
which runs find_busiest_group on an hierachy of sched domains spanning
the cpus.  For a large 4 socket system, we saw almost 0.25 msec spent
per call of pick_next_task_fair before a cpu can be idled.

This patch skips pick_next_task_fair in the slow path if it has already
been invoked.

Signed-off-by: Peter Zijlstra <peterz@...radead.org>
Signed-off-by: Tim Chen <tim.c.chen@...ux.intel.com>
---
 kernel/sched/core.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 1d1b87b..547ccff 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2591,8 +2591,14 @@ pick_next_task(struct rq *rq, struct task_struct *prev)
 	if (likely(prev->sched_class == class &&
 		   rq->nr_running == rq->cfs.h_nr_running)) {
 		p = fair_sched_class.pick_next_task(rq, prev);
-		if (likely(p && p != RETRY_TASK))
-			return p;
+		if (unlikely(p == RETRY_TASK))
+			goto again;
+
+		/* assumes fair_sched_class->next == idle_sched_class */
+		if (unlikely(!p))
+			p = idle_sched_class.pick_next_task(rq, prev);
+
+		return p;
 	}

 again:
-- 
1.7.11.7

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/