linux-kernel - [PATCH] arm64: kexec: Use smp_send_stop in machine

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <1604705148-1831-1-git-send-email-henry.willard@oracle.com>
Date:   Fri,  6 Nov 2020 15:25:48 -0800
From:   Henry Willard <henry.willard@...cle.com>
To:     <catalin.marinas@....com>, <will@...nel.org>,
        <linux-arm-kernel@...ts.infradead.org>,
        <linux-kernel@...r.kernel.org>, <qais.yousef@....com>
Subject: [PATCH] arm64: kexec: Use smp_send_stop in machine_shutdown

machine_shutdown() is called by kernel_kexec() to shutdown
the non-boot CPUs prior to starting the new kernel. The
implementation of machine_shutdown() varies by architecture.
Many make an interprocessor call, such as smp_send_stop(),
to stop the non-boot CPUs. On some architectures the CPUs make
some sort of firmware call to stop the CPU. On some architectures
without the necessary firmware support to stop the CPU, the CPUs
go into a disabled loop, which is not suitable for supporting
kexec. On Arm64 systems that support PSCI, CPUs can be stopped
with a PSCI CPU_OFF call.

Arm64 machine_shutdown() uses the CPU hotplug infrastructure via
smp_shutdown_nonboot_cpus() to stop each CPU. This is relatively
slow and takes a best case of .02 to .03 seconds per CPU which are
stopped sequentially. This can take the better part of a second for
all the CPUs to be stopped depending on how many CPUs are present.
If for some reason the CPUs are busy at the time of the kexec reboot,
it can take several seconds to shut them all down. Each CPU shuts
itself down by calling PSCI CPU_OFF.

In some applications such as embedded systems, which need a very
fast reboot (less than a second), this may be too slow.

This patch reverts to using smp_send_stop() to signal all
CPUs to stop immediately. Currently smp_send_stop() causes each cpu
to call local_cpu_stop(), which goes into a disabled loop. This patch
modifies local_cpu_stop() to call cpu_die() when kexec_in_progress
is true, so that the CPU calls PSCI CPU_OFF just as in the case of
smp_shutdown_nonboot_cpus(). Using smp_send_stop() instead of
smp_shutdown_nonboot_cpus() reduces the shutdown time for 23 CPUs
from about .65 seconds on an idle system to less than 5 msecs. On a
busy system smp_shutdown_nonboot_cpus() may take several seconds,
while smp_send_stop() needs only the 5 msecs.

Signed-off-by: Henry Willard <henry.willard@...cle.com>
---
 arch/arm64/kernel/process.c | 17 ++++++++++++++---
 arch/arm64/kernel/smp.c     |  8 +++++++-
 2 files changed, 21 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index 4784011cecac..2568452a2417 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -44,6 +44,7 @@
 #include <linux/percpu.h>
 #include <linux/thread_info.h>
 #include <linux/prctl.h>
+#include <linux/kexec.h>

 #include <asm/alternative.h>
 #include <asm/arch_gicv3.h>
@@ -142,12 +143,22 @@ void arch_cpu_idle_dead(void)
  * This must completely disable all secondary CPUs; simply causing those CPUs
  * to execute e.g. a RAM-based pin loop is not sufficient. This allows the
  * kexec'd kernel to use any and all RAM as it sees fit, without having to
- * avoid any code or data used by any SW CPU pin loop. The CPU hotplug
- * functionality embodied in smpt_shutdown_nonboot_cpus() to achieve this.
+ * avoid any code or data used by any SW CPU pin loop. The target stop function
+ * will call cpu_die() if kexec_in_progress is set.
  */
 void machine_shutdown(void)
 {
-	smp_shutdown_nonboot_cpus(reboot_cpu);
+	unsigned long timeout;
+
+	/*
+	 * Don't wait forever, but no longer than a second
+	 */
+	timeout = USEC_PER_SEC;
+
+	smp_send_stop();
+	while (num_online_cpus() > 1 && timeout--)
+		udelay(1);
+	return;
 }

 /*
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 09c96f57818c..310cdf327d91 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -373,7 +373,9 @@ void cpu_die(void)
 	unsigned int cpu = smp_processor_id();
 	const struct cpu_operations *ops = get_cpu_ops(cpu);

-	idle_task_exit();
+	/* Skip this if we are about to exit the machine */
+	if (!kexec_in_progress)
+		idle_task_exit();

 	local_daif_mask();

@@ -847,6 +849,10 @@ static void local_cpu_stop(void)

 	local_daif_mask();
 	sdei_mask_local_cpu();
+
+	if (kexec_in_progress)
+		cpu_die();
+
 	cpu_park_loop();
 }

-- 
1.8.3.1