linux-kernel - [PATCH] sched: add ttwu

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Message-Id: <20230425012234.15388-1-shijie@os.amperecomputing.com>
Date:   Tue, 25 Apr 2023 09:22:34 +0800
From:   Huang Shijie <shijie@...amperecomputing.com>
To:     mingo@...hat.com
Cc:     peterz@...radead.org, juri.lelli@...hat.com,
        vincent.guittot@...aro.org, dietmar.eggemann@....com,
        rostedt@...dmis.org, bsegall@...gle.com, mgorman@...e.de,
        bristot@...hat.com, vschneid@...hat.com, corbet@....net,
        linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org,
        patches@...erecomputing.com,
        Huang Shijie <shijie@...amperecomputing.com>
Subject: [PATCH] sched: add ttwu_migration counter

It maybe involve three CPUs in the try_to_wake_up(), for example:
	Current CPU:	#0
	task_cpu(p):    #1
        Target CPU :    #?  <-- get by select_task_rq()

ttwu_local: record times try_to_wake_up() was called to wake up
            the local cpu.
	1.) Note ttwu_local also includes some migrations, such as a task is
       	    migrated from CPU #1 to Target CPU #0.

ttwu_wake_remote: record times in this domain try_to_wake_up() awoke
            a task that last ran on a different cpu in this domain.
	1.) Part of the migrations are recorded in ttwu_wake_remote, such as
            a task is migrated from CPU #1 to Target CPU #2.
	2.) If a task is put in CPU #1 wake_list, the ttwu_wake_remote will
            record it too.

Current try_to_wake_up() schedstat does not have a migration counter.
And we cannot get the migration number from the ttwu_local, ttwu_count
and ttwu_wake_remote correctly.

But we really want to know the try_to_wake_up migration number for
analyzing some benchmark tests.

This patch adds the ttwu_migration counter to record the migrations.
Put it at the end, do not break some tools.

Signed-off-by: Huang Shijie <shijie@...amperecomputing.com>
---
 Documentation/scheduler/sched-stats.rst | 4 +++-
 kernel/sched/core.c                     | 4 ++++
 kernel/sched/debug.c                    | 1 +
 kernel/sched/sched.h                    | 1 +
 kernel/sched/stats.c                    | 5 +++--
 5 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/Documentation/scheduler/sched-stats.rst b/Documentation/scheduler/sched-stats.rst
index 03c062915998..3220dbe220d5 100644
--- a/Documentation/scheduler/sched-stats.rst
+++ b/Documentation/scheduler/sched-stats.rst
@@ -36,7 +36,7 @@ to write their own scripts, the fields are described here.
 
 CPU statistics
 --------------
-cpu<N> 1 2 3 4 5 6 7 8 9
+cpu<N> 1 2 3 4 5 6 7 8 9 10
 
 First field is a sched_yield() statistic:
 
@@ -61,6 +61,8 @@ Next three are statistics describing scheduling latency:
         nanoseconds)
      9) # of timeslices run on this cpu
 
+Next new item for try_to_wake_up() statistics:
+     10) # of times try_to_wake_up() was called to do the task migration.
 
 Domain statistics
 -----------------
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index e0fc8f7c7487..f0038c103843 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4288,11 +4288,15 @@ try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags)
 
 	cpu = select_task_rq(p, p->wake_cpu, wake_flags | WF_TTWU);
 	if (task_cpu(p) != cpu) {
+		struct rq *rq = this_rq();
+
 		if (p->in_iowait) {
 			delayacct_blkio_end(p);
 			atomic_dec(&task_rq(p)->nr_iowait);
 		}
 
+		schedstat_inc(rq->ttwu_migration);
+
 		wake_flags |= WF_MIGRATED;
 		psi_ttwu_dequeue(p);
 		set_task_cpu(p, cpu);
diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index 0b2340a79b65..f0fd4e2795c9 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -809,6 +809,7 @@ do {									\
 		P(sched_goidle);
 		P(ttwu_count);
 		P(ttwu_local);
+		P(ttwu_migration);
 	}
 #undef P
 
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 060616944d7a..48b552f6b87f 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1123,6 +1123,7 @@ struct rq {
 	/* try_to_wake_up() stats */
 	unsigned int		ttwu_count;
 	unsigned int		ttwu_local;
+	unsigned int		ttwu_migration;
 #endif
 
 #ifdef CONFIG_CPU_IDLE
diff --git a/kernel/sched/stats.c b/kernel/sched/stats.c
index 857f837f52cb..5651d4c0a63f 100644
--- a/kernel/sched/stats.c
+++ b/kernel/sched/stats.c
@@ -133,12 +133,13 @@ static int show_schedstat(struct seq_file *seq, void *v)
 
 		/* runqueue-specific stats */
 		seq_printf(seq,
-		    "cpu%d %u 0 %u %u %u %u %llu %llu %lu",
+		    "cpu%d %u 0 %u %u %u %u %llu %llu %lu %u",
 		    cpu, rq->yld_count,
 		    rq->sched_count, rq->sched_goidle,
 		    rq->ttwu_count, rq->ttwu_local,
 		    rq->rq_cpu_time,
-		    rq->rq_sched_info.run_delay, rq->rq_sched_info.pcount);
+		    rq->rq_sched_info.run_delay, rq->rq_sched_info.pcount,
+		    rq->ttwu_migration);
 
 		seq_printf(seq, "\n");
 
-- 
2.39.2