linux-kernel - [PATCH 1/3] sched: Allow hrticks to work with core scheduling

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20200827004250.4853-2-graf@amazon.com>
Date:   Thu, 27 Aug 2020 02:42:48 +0200
From:   Alexander Graf <graf@...zon.com>
To:     <linux-kernel@...r.kernel.org>
CC:     Vineeth Remanan Pillai <vpillai@...italocean.com>,
        Nishanth Aravamudan <naravamudan@...italocean.com>,
        Julien Desfossez <jdesfossez@...italocean.com>,
        Peter Zijlstra <peterz@...radead.org>,
        "Tim Chen" <tim.c.chen@...ux.intel.com>, <mingo@...nel.org>,
        <tglx@...utronix.de>, <pjt@...gle.com>,
        <torvalds@...ux-foundation.org>, <subhra.mazumdar@...cle.com>,
        <fweisbec@...il.com>, <keescook@...omium.org>,
        <kerrnel@...gle.com>, Phil Auld <pauld@...hat.com>,
        Aaron Lu <aaron.lwe@...il.com>,
        Aubrey Li <aubrey.intel@...il.com>,
        Valentin Schneider <valentin.schneider@....com>,
        Mel Gorman <mgorman@...hsingularity.net>,
        "Pawan Gupta" <pawan.kumar.gupta@...ux.intel.com>,
        Paolo Bonzini <pbonzini@...hat.com>,
        Joel Fernandes <joelaf@...gle.com>, <joel@...lfernandes.org>,
        <vineethrp@...il.com>, Chen Yu <yu.c.chen@...el.com>,
        Christian Brauner <christian.brauner@...ntu.com>
Subject: [PATCH 1/3] sched: Allow hrticks to work with core scheduling

The core scheduling logic bypasses the scheduling class's
pick_next_task() which starts the hrtick logic usually. Instead,
it explicitly calls set_next_task() or leaves the current task
running without any callback into the CFS scheduler.

To ensure that we still configure the hrtick timer properly when we
know which task we want to run, let's add an explicit callback to
the scheduler class which can then be triggered from the core's
pick_next_task().

With this patch, core scheduling with HRTICK enabled does see
improved responsiveness on scheduling decisions.

Signed-off-by: Alexander Graf <graf@...zon.com>
---
 kernel/sched/core.c  | 13 +++++++++++++
 kernel/sched/fair.c  |  9 +++++++++
 kernel/sched/sched.h |  4 ++++
 3 files changed, 26 insertions(+)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 0362102fa3d2..72bf837422bf 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4486,6 +4486,12 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
 			set_next_task(rq, next);
 		}
 
+#ifdef CONFIG_SCHED_HRTICK
+		/* Trigger next hrtick after task selection */
+		if (next->sched_class->hrtick_update)
+			next->sched_class->hrtick_update(rq);
+#endif
+
 		trace_printk("pick pre selected (%u %u %u): %s/%d %lx\n",
 			     rq->core->core_task_seq,
 			     rq->core->core_pick_seq,
@@ -4667,6 +4673,13 @@ next_class:;
 
 done:
 	set_next_task(rq, next);
+
+#ifdef CONFIG_SCHED_HRTICK
+	/* Trigger next hrtick after task selection */
+	if (next->sched_class->hrtick_update)
+		next->sched_class->hrtick_update(rq);
+#endif
+
 	return next;
 }
 
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 435b460d3c3f..0d4ff3ab2572 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -5512,6 +5512,11 @@ static inline void hrtick_update(struct rq *rq)
 }
 #endif
 
+static void hrtick_update_fair(struct rq *rq)
+{
+	hrtick_update(rq);
+}
+
 #ifdef CONFIG_SMP
 static inline unsigned long cpu_util(int cpu);
 
@@ -11391,6 +11396,10 @@ const struct sched_class fair_sched_class = {
 #ifdef CONFIG_UCLAMP_TASK
 	.uclamp_enabled		= 1,
 #endif
+
+#ifdef CONFIG_SCHED_HRTICK
+	.hrtick_update		= hrtick_update_fair,
+#endif
 };
 
 #ifdef CONFIG_SCHED_DEBUG
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 6445943d3215..b382e0ee0c87 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1942,6 +1942,10 @@ struct sched_class {
 #ifdef CONFIG_FAIR_GROUP_SCHED
 	void (*task_change_group)(struct task_struct *p, int type);
 #endif
+
+#ifdef CONFIG_SCHED_HRTICK
+	void (*hrtick_update)(struct rq *rq);
+#endif
 };
 
 static inline void put_prev_task(struct rq *rq, struct task_struct *prev)
-- 
2.26.2




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879