linux-kernel - [tip: sched/core] sched: Improve wake_up_all_idle

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <163491732050.626.7063177885840173683.tip-bot2@tip-bot2>
Date:   Fri, 22 Oct 2021 15:42:00 -0000
From:   "tip-bot2 for Peter Zijlstra" <tip-bot2@...utronix.de>
To:     linux-tip-commits@...r.kernel.org
Cc:     syzbot+d5b23b18d2f4feae8a67@...kaller.appspotmail.com,
        Pavel Machek <pavel@....cz>,
        Qian Cai <quic_qiancai@...cinc.com>,
        "Peter Zijlstra (Intel)" <peterz@...radead.org>, x86@...nel.org,
        linux-kernel@...r.kernel.org
Subject: [tip: sched/core] sched: Improve wake_up_all_idle_cpus() take #2

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     96611c26dc351c33f73b48756a9feacc109e5bab
Gitweb:        https://git.kernel.org/tip/96611c26dc351c33f73b48756a9feacc109e5bab
Author:        Peter Zijlstra <peterz@...radead.org>
AuthorDate:    Mon, 18 Oct 2021 16:41:05 +02:00
Committer:     Peter Zijlstra <peterz@...radead.org>
CommitterDate: Fri, 22 Oct 2021 15:32:46 +02:00

sched: Improve wake_up_all_idle_cpus() take #2

As reported by syzbot and experienced by Pavel, using cpus_read_lock()
in wake_up_all_idle_cpus() generates lock inversion (against mmap_sem
and possibly others).

Instead, shrink the preempt disable region by iterating all CPUs and
checking the online status for each individual CPU while having
preemption disabled.

Fixes: 8850cb663b5c ("sched: Simplify wake_up_*idle*()")
Reported-by: syzbot+d5b23b18d2f4feae8a67@...kaller.appspotmail.com
Reported-by: Pavel Machek <pavel@....cz>
Reported-by: Qian Cai <quic_qiancai@...cinc.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@...radead.org>
Tested-by: Qian Cai <quic_qiancai@...cinc.com>
---
 kernel/smp.c | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/kernel/smp.c b/kernel/smp.c
index ad0b68a..01a7c17 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -1170,14 +1170,12 @@ void wake_up_all_idle_cpus(void)
 {
 	int cpu;
 
-	cpus_read_lock();
-	for_each_online_cpu(cpu) {
-		if (cpu == raw_smp_processor_id())
-			continue;
-
-		wake_up_if_idle(cpu);
+	for_each_possible_cpu(cpu) {
+		preempt_disable();
+		if (cpu != smp_processor_id() && cpu_online(cpu))
+			wake_up_if_idle(cpu);
+		preempt_enable();
 	}
-	cpus_read_unlock();
 }
 EXPORT_SYMBOL_GPL(wake_up_all_idle_cpus);