linux-kernel - Re: [patch 7/7] sched: only use TTWU_QUEUE when waker/wakee CPUs do not share top level cache

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1322050096.14799.49.camel@twins>
Date:	Wed, 23 Nov 2011 13:08:16 +0100
From:	Peter Zijlstra <peterz@...radead.org>
To:	Mike Galbraith <mgalbraith@...e.de>
Cc:	Suresh Siddha <suresh.b.siddha@...el.com>,
	linux-kernel <linux-kernel@...r.kernel.org>,
	Ingo Molnar <mingo@...e.hu>, Paul Turner <pjt@...gle.com>,
	Chris Mason <chris.mason@...cle.com>,
	Dave Kleikamp <dave.kleikamp@...cle.com>
Subject: Re: [patch 7/7] sched: only use TTWU_QUEUE when waker/wakee CPUs do
 not share top level cache


I changed this a little, no point in iterating the domains when
SD_SHARE_PKG_RESOURCES fails.

---
Subject: sched: Only use TTWU_QUEUE when waker/wakee CPUs do not share top level cache
From: Mike Galbraith <mgalbraith@...e.de>
Date: Tue, 22 Nov 2011 15:26:25 +0100

TTWU_QUEUE IPI overhead was measured to be as much as 13% of netperf TCP_RR
overhead when waking to a shared cache. Don't IPI unless we're waking cross
cache, where it can be a winner.

Cc: Chris Mason <chris.mason@...cle.com>
Cc: Dave Kleikamp <dave.kleikamp@...cle.com>
Suggested-by: Peter Zijlstra <a.p.zijlstra@...llo.nl>
Signed-off-by: Mike Galbraith <efault@....de>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@...llo.nl>
Link: http://lkml.kernel.org/r/1321971985.6855.26.camel@marge.simson.net
---
 kernel/sched/core.c |   26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

Index: linux-2.6/kernel/sched/core.c
===================================================================
--- linux-2.6.orig/kernel/sched/core.c
+++ linux-2.6/kernel/sched/core.c
@@ -1480,12 +1480,36 @@ static int ttwu_activate_remote(struct t
 #endif /* __ARCH_WANT_INTERRUPTS_ON_CTXSW */
 #endif /* CONFIG_SMP */
 
+static int ttwu_share_cache(int this_cpu, int cpu)
+{
+#ifndef CONFIG_X86
+	struct sched_domain *sd;
+	int ret = 0;
+
+	rcu_read_lock();
+	for_each_domain(this_cpu, sd) {
+		if (!(sd->flags & SD_SHARE_PKG_RESOURCES))
+			break;
+
+		if (cpumask_test_cpu(cpu, sched_domain_span(sd))) {
+			ret = 1;
+			break;
+		}
+	}
+	rcu_read_unlock();
+
+	return ret;
+#else
+	return per_cpu(cpu_llc_id, this_cpu) == per_cpu(cpu_llc_id, cpu);
+#endif
+}
+
 static void ttwu_queue(struct task_struct *p, int cpu)
 {
 	struct rq *rq = cpu_rq(cpu);
 
 #if defined(CONFIG_SMP)
-	if (sched_feat(TTWU_QUEUE) && cpu != smp_processor_id()) {
+	if (sched_feat(TTWU_QUEUE) && !ttwu_share_cache(smp_processor_id(), cpu)) {
 		sched_clock_cpu(cpu); /* sync clocks x-cpu */
 		ttwu_queue_remote(p, cpu);
 		return;

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/