linux-kernel - [PATCH 15/16] sched/topology: optimize topology_span

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20220718192844.1805158-16-yury.norov@gmail.com>
Date:   Mon, 18 Jul 2022 12:28:43 -0700
From:   Yury Norov <yury.norov@...il.com>
To:     linux-kernel@...r.kernel.org,
        Alexander Lobakin <alexandr.lobakin@...el.com>,
        Alexei Starovoitov <ast@...nel.org>,
        Alexey Klimov <aklimov@...hat.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Andrii Nakryiko <andrii@...nel.org>,
        Andy Shevchenko <andriy.shevchenko@...ux.intel.com>,
        Ben Segall <bsegall@...gle.com>,
        Christoph Lameter <cl@...ux.com>,
        Dan Williams <dan.j.williams@...el.com>,
        Daniel Borkmann <daniel@...earbox.net>,
        Daniel Bristot de Oliveira <bristot@...hat.com>,
        Dennis Zhou <dennis@...nel.org>,
        Dietmar Eggemann <dietmar.eggemann@....com>,
        Eric Dumazet <edumazet@...gle.com>,
        Frederic Weisbecker <fweisbec@...il.com>,
        Guenter Roeck <linux@...ck-us.net>,
        Ingo Molnar <mingo@...hat.com>,
        Isabella Basso <isabbasso@...eup.net>,
        John Fastabend <john.fastabend@...il.com>,
        Josh Poimboeuf <jpoimboe@...nel.org>,
        Juergen Gross <jgross@...e.com>,
        Juri Lelli <juri.lelli@...hat.com>,
        KP Singh <kpsingh@...nel.org>,
        Kees Cook <keescook@...omium.org>,
        Martin KaFai Lau <kafai@...com>,
        Mel Gorman <mgorman@...e.de>, Miroslav Benes <mbenes@...e.cz>,
        Nathan Chancellor <nathan@...nel.org>,
        "Paul E . McKenney" <paulmck@...nel.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Randy Dunlap <rdunlap@...radead.org>,
        Rasmus Villemoes <linux@...musvillemoes.dk>,
        Sebastian Andrzej Siewior <bigeasy@...utronix.de>,
        Song Liu <songliubraving@...com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Tejun Heo <tj@...nel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Valentin Schneider <vschneid@...hat.com>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        Vlastimil Babka <vbabka@...e.cz>, Yonghong Song <yhs@...com>,
        Yury Norov <yury.norov@...il.com>, linux-mm@...ck.org,
        netdev@...r.kernel.org, bpf@...r.kernel.org
Subject: [PATCH 15/16] sched/topology: optimize topology_span_sane()

topology_span_sane() checks if cpu == i before calling
	cpumask_equal(tl->mask(cpu), tl->mask(i)).

However, tl->mask(cpu) and tl->mask(i) may point to the same cpumask
even if i != cpu. Fix the check accordingly.

While here, move tl->mask(cpu) out of the loop, and make the in-loop
code calculating tl->mask(i) only once.

Catched with CONFIG_DEBUG_BITMAP:
[    0.867917] Call Trace:
[    0.868209]  <TASK>
[    0.868471]  build_sched_domains+0x36f/0x1a40
[    0.868576]  sched_init_smp+0x44/0xba
[    0.869012]  ? mtrr_aps_init+0x84/0xa0
[    0.869465]  kernel_init_freeable+0x12e/0x26e
[    0.869982]  ? rest_init+0xd0/0xd0
[    0.870406]  kernel_init+0x16/0x120
[    0.870821]  ret_from_fork+0x22/0x30
[    0.871244]  </TASK>
[    0.871502] ---[ end trace 0000000000000000 ]---
[    0.872040] b1:              ffffffffb1fd3480
[    0.872041] b2:              ffffffffb1fd3480
[    0.872041] b3:              0
[    0.872042] nbits:   256
[    0.872042] start:   0
[    0.872042] off:     0
[    0.872043] Bitmap: parameters check failed
[    0.872043] include/linux/bitmap.h [427]: bitmap_equal

Signed-off-by: Yury Norov <yury.norov@...il.com>
---
 kernel/sched/topology.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 05b6c2ad90b9..ad32d0a43424 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -2211,6 +2211,8 @@ static struct sched_domain *build_sched_domain(struct sched_domain_topology_leve
 static bool topology_span_sane(struct sched_domain_topology_level *tl,
 			      const struct cpumask *cpu_map, int cpu)
 {
+	const struct cpumask *mc = tl->mask(cpu);
+	const struct cpumask *mi;
 	int i;
 
 	/* NUMA levels are allowed to overlap */
@@ -2226,14 +2228,18 @@ static bool topology_span_sane(struct sched_domain_topology_level *tl,
 	for_each_cpu(i, cpu_map) {
 		if (i == cpu)
 			continue;
+
+		mi = tl->mask(i);
+		if (mi == mc)
+			continue;
+
 		/*
 		 * We should 'and' all those masks with 'cpu_map' to exactly
 		 * match the topology we're about to build, but that can only
 		 * remove CPUs, which only lessens our ability to detect
 		 * overlaps
 		 */
-		if (!cpumask_equal(tl->mask(cpu), tl->mask(i)) &&
-		    cpumask_intersects(tl->mask(cpu), tl->mask(i)))
+		if (!cpumask_equal(mc, mi) && cpumask_intersects(mc, mi))
 			return false;
 	}
 
-- 
2.34.1