linux-kernel - [RFC][PATCH v2 4/5] cpuidle/teo: Avoid tick_nohz_next

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 02 Aug 2023 15:24:35 +0200
From:   Peter Zijlstra <peterz@...radead.org>
To:     anna-maria@...utronix.de, rafael@...nel.org, tglx@...utronix.de,
        frederic@...nel.org, gautham.shenoy@....com
Cc:     linux-kernel@...r.kernel.org, peterz@...radead.org,
        daniel.lezcano@...aro.org, linux-pm@...r.kernel.org,
        mingo@...hat.com, juri.lelli@...hat.com,
        vincent.guittot@...aro.org, dietmar.eggemann@....com,
        rostedt@...dmis.org, bsegall@...gle.com, mgorman@...e.de,
        bristot@...hat.com, vschneid@...hat.com, kajetan.puchalski@....com
Subject: [RFC][PATCH v2 4/5] cpuidle/teo: Avoid tick_nohz_next_event()

Use the new tick_get_sleep_length() call in conjunction with the new
TICK state to elide tick_nohz_next_event() when possible.

Specifically, start the state selection using the existing next timer
(tick or earlier). And only when state selection lands on the TICK
state ask for the NOHZ next timer.

Signed-off-by: Peter Zijlstra (Intel) <peterz@...radead.org>
---
 drivers/cpuidle/governors/teo.c |   21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)

--- a/drivers/cpuidle/governors/teo.c
+++ b/drivers/cpuidle/governors/teo.c
@@ -139,6 +139,7 @@
 #include <linux/sched/clock.h>
 #include <linux/sched/topology.h>
 #include <linux/tick.h>
+#include "../cpuidle.h"
 
 /*
  * The number of bits to shift the CPU's capacity by in order to determine
@@ -363,8 +364,7 @@ static int teo_select(struct cpuidle_dri
 	int constraint_idx = 0;
 	int idx0 = 0, idx = -1;
 	bool alt_intercepts, alt_recent;
-	ktime_t delta_tick;
-	s64 duration_ns;
+	s64 duration_ns, tick_ns;
 	int i;
 
 	if (dev->last_state_idx >= 0) {
@@ -374,8 +374,7 @@ static int teo_select(struct cpuidle_dri
 
 	cpu_data->time_span_ns = local_clock();
 
-	duration_ns = tick_nohz_get_sleep_length(&delta_tick);
-	cpu_data->sleep_length_ns = duration_ns;
+	duration_ns = tick_ns = tick_get_sleep_length(false);
 
 	/*
 	 * Find the deepest idle state whose target residency does not exceed
@@ -407,6 +406,14 @@ static int teo_select(struct cpuidle_dri
 		if (s->target_residency_ns > duration_ns)
 			break;
 
+		if (s->target_residency_ns == SHORT_TICK_NSEC) {
+			/*
+			 * We hit the tick state, see if it makes sense to
+			 * disable the tick and go deeper still.
+			 */
+			duration_ns = tick_get_sleep_length(true);
+		}
+
 		idx = i;
 
 		if (s->exit_latency_ns <= latency_req)
@@ -417,6 +424,8 @@ static int teo_select(struct cpuidle_dri
 		idx_recent_sum = recent_sum;
 	}
 
+	cpu_data->sleep_length_ns = duration_ns;
+
 	/* Avoid unnecessary overhead. */
 	if (idx < 0) {
 		idx = 0; /* No states enabled, must use 0. */
@@ -531,8 +540,8 @@ static int teo_select(struct cpuidle_dri
 		 * that.
 		 */
 		if (idx > idx0 &&
-		    drv->states[idx].target_residency_ns > delta_tick)
-			idx = teo_find_shallower_state(drv, dev, idx, delta_tick, false);
+		    drv->states[idx].target_residency_ns > tick_ns)
+			idx = teo_find_shallower_state(drv, dev, idx, tick_ns, false);
 	}
 
 	return idx;