linux-kernel - [RFC PATCH 06/19] sched/fair: Fixup sd->nohz

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250904041516.3046-7-kprateek.nayak@amd.com>
Date: Thu, 4 Sep 2025 04:15:02 +0000
From: K Prateek Nayak <kprateek.nayak@....com>
To: Ingo Molnar <mingo@...hat.com>, Peter Zijlstra <peterz@...radead.org>,
	Juri Lelli <juri.lelli@...hat.com>, Vincent Guittot
	<vincent.guittot@...aro.org>, Anna-Maria Behnsen <anna-maria@...utronix.de>,
	Frederic Weisbecker <frederic@...nel.org>, Thomas Gleixner
	<tglx@...utronix.de>, <linux-kernel@...r.kernel.org>
CC: Dietmar Eggemann <dietmar.eggemann@....com>, Steven Rostedt
	<rostedt@...dmis.org>, Ben Segall <bsegall@...gle.com>, Mel Gorman
	<mgorman@...e.de>, Valentin Schneider <vschneid@...hat.com>, K Prateek Nayak
	<kprateek.nayak@....com>, "Gautham R. Shenoy" <gautham.shenoy@....com>,
	Swapnil Sapkal <swapnil.sapkal@....com>
Subject: [RFC PATCH 06/19] sched/fair: Fixup sd->nohz_idle state during hotplug / cpuset

When CPUs are transitioned offline, set_cpu_sd_state_busy() is called
to exclude the CPU from nohz idle balancing. Currently, it is done
independently via sched_cpu_deactivate() but subsequent transition of
the nohz idle tracking into the sched domain hierarchy would require
each CPU's status to be updated in the new hierarchy generated during
hotplug / cpuset.

Call set_cpu_sd_state_{idle,busy}() from the fair_sched_class' offline()
and online() callback respectively to ensure "sd->nohz_idle" of the
updated hierarchy reflects the same state as "rq->nohz_tick_stopped".

The changes from the prior commit to always update the rq->sd and the
top cache domains from rq_attach_root() ensures that the offline()
callback works on the older hierarchy and the online() callback works on
the updated hierarchy.

Signed-off-by: K Prateek Nayak <kprateek.nayak@....com>
---
 kernel/sched/fair.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 0e68371365a5..61e1b4deb3e8 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -12491,6 +12491,18 @@ static void set_cpu_sd_state_idle(int cpu)
 	atomic_dec(&sd->shared->nr_busy_cpus);
 }
 
+static void cpu_sd_exit_nohz_balance(struct rq *rq)
+{
+	if (READ_ONCE(rq->nohz_tick_stopped))
+		set_cpu_sd_state_busy(cpu_of(rq));
+}
+
+static void cpu_sd_reenter_nohz_balance(struct rq *rq)
+{
+	if (READ_ONCE(rq->nohz_tick_stopped))
+		set_cpu_sd_state_idle(cpu_of(rq));
+}
+
 /*
  * This routine will record that the CPU is going idle with tick stopped.
  * This info will be used in performing idle load balancing in the future.
@@ -12741,6 +12753,8 @@ static void nohz_newidle_balance(struct rq *this_rq)
 }
 
 #else /* !CONFIG_NO_HZ_COMMON: */
+static inline void cpu_sd_exit_nohz_balance(struct rq *rq) { }
+static inline void cpu_sd_reenter_nohz_balance(struct rq *rq) { }
 static inline void nohz_balancer_kick(struct rq *rq) { }
 
 static inline bool nohz_idle_balance(struct rq *this_rq, enum cpu_idle_type idle)
@@ -12940,6 +12954,9 @@ static void rq_online_fair(struct rq *rq)
 	update_sysctl();
 
 	update_runtime_enabled(rq);
+
+	/* Fixup nr_busy_cpus and nohz stats. */
+	cpu_sd_reenter_nohz_balance(rq);
 }
 
 static void rq_offline_fair(struct rq *rq)
@@ -12951,6 +12968,9 @@ static void rq_offline_fair(struct rq *rq)
 
 	/* Ensure that we remove rq contribution to group share: */
 	clear_tg_offline_cfs_rqs(rq);
+
+	/* Fixup nr_busy_cpus and nohz stats. */
+	cpu_sd_exit_nohz_balance(rq);
 }
 
 #ifdef CONFIG_SCHED_CORE
-- 
2.34.1