linux-kernel - Re: [PATCH] Do not force shutdown/reboot to boot cpu.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130408155701.GB19974@gmail.com>
Date:	Mon, 8 Apr 2013 17:57:01 +0200
From:	Ingo Molnar <mingo@...nel.org>
To:	Robin Holt <holt@....com>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	linux-kernel@...r.kernel.org, Russ Anderson <rja@....com>,
	Shawn Guo <shawn.guo@...aro.org>,
	Thomas Gleixner <tglx@...utronix.de>,
	Ingo Molnar <mingo@...hat.com>,
	"H. Peter Anvin" <hpa@...or.com>, x86@...nel.org,
	Linus Torvalds <torvalds@...ux-foundation.org>
Subject: Re: [PATCH] Do not force shutdown/reboot to boot cpu.


* Robin Holt <holt@....com> wrote:

> We noticed that recently, reboot of a 1024 cpu machine takes approx 16
> minutes of just stopping the cpus.  The slowdown was tracked to commit
> f96972f which went into v3.7 and then to the stable trees.
> 
> x86 does not need to be running the boot cpu to pull reset and I don't
> think it is really needed for shutdown either.
> 
> I decided to go the "simple" way and make this a config option that is
> selected by the x86 arch.  I don't know which other arch's would also
> benefit, if any.
> 
> Signed-off-by: Robin Holt <holt@....com>
> To: Andrew Morton <akpm@...ux-foundation.org>
> Cc: Russ Anderson <rja@....com>
> Cc: Thomas Gleixner <tglx@...utronix.de>
> Cc: Ingo Molnar <mingo@...hat.com>
> Cc: "H. Peter Anvin" <hpa@...or.com>
> Cc: Shawn Guo <shawn.guo@...aro.org>
> Cc: <stable@...r.kernel.org>
> 
> ---
>  arch/x86/Kconfig        | 3 +++
>  kernel/Kconfig.shutdown | 3 +++
>  kernel/sys.c            | 4 ++++
>  3 files changed, 10 insertions(+)
>  create mode 100644 kernel/Kconfig.shutdown
> 
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index 70c0f3d..9611942 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -120,6 +120,7 @@ config X86
>  	select OLD_SIGSUSPEND3 if X86_32 || IA32_EMULATION
>  	select OLD_SIGACTION if X86_32
>  	select COMPAT_OLD_SIGACTION if IA32_EMULATION
> +	select ARCH_SHUTDOWN_TO_ANY_CPU
>  
>  config INSTRUCTION_DECODER
>  	def_bool y
> @@ -839,6 +840,8 @@ config SCHED_MC
>  	  making when dealing with multi-core CPU chips at a cost of slightly
>  	  increased overhead in some places. If unsure say N here.
>  
> +source "kernel/Kconfig.shutdown"
> +
>  source "kernel/Kconfig.preempt"
>  
>  config X86_UP_APIC
> diff --git a/kernel/Kconfig.shutdown b/kernel/Kconfig.shutdown
> new file mode 100644
> index 0000000..d79fc04
> --- /dev/null
> +++ b/kernel/Kconfig.shutdown
> @@ -0,0 +1,3 @@
> +
> +config ARCH_SHUTDOWN_TO_ANY_CPU
> +	bool
> diff --git a/kernel/sys.c b/kernel/sys.c
> index 39c9c4a..c0b8880 100644
> --- a/kernel/sys.c
> +++ b/kernel/sys.c
> @@ -369,7 +369,9 @@ EXPORT_SYMBOL(unregister_reboot_notifier);
>  void kernel_restart(char *cmd)
>  {
>  	kernel_restart_prepare(cmd);
> +#ifndef CONFIG_ARCH_SHUTDOWN_TO_ANY_CPU
>  	disable_nonboot_cpus();
> +#endif
>  	if (!cmd)
>  		printk(KERN_EMERG "Restarting system.\n");
>  	else
> @@ -413,7 +415,9 @@ void kernel_power_off(void)
>  	kernel_shutdown_prepare(SYSTEM_POWER_OFF);
>  	if (pm_power_off_prepare)
>  		pm_power_off_prepare();
> +#ifndef CONFIG_ARCH_SHUTDOWN_TO_ANY_CPU
>  	disable_nonboot_cpus();
> +#endif
>  	syscore_shutdown();
>  	printk(KERN_EMERG "Power down.\n");
>  	kmsg_dump(KMSG_DUMP_POWEROFF);

Hm, the 'fix' is a pretty ugly workaround that does not fix much IMHO.

I think the original commit:

  f96972f2dc63 kernel/sys.c: call disable_nonboot_cpus() in kernel_restart()

actually regressed your 1024 CPU systems, and should possibly be reverted or fixed 
in some other fashion - such as by migrating to the primary CPU (on architectures 
that require that), instead of hotplug offlining every secondary CPU on every 
architecture!

Alternatively, disable_nonboot_cpus() could perhaps be improved to down CPUs in 
parallel: issue the CPU-down requests to every CPU, then wait for them to complete 
- instead of the loop over every CPU?

This would be the conceptual counter part to parallel boot up of CPUs - something 
SGI might be interested in as well?

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/