linux-kernel - Re: [Patch -v4 1/4] Migrate shutdown/reboot to boot cpu.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130416113256.GA10452@gmail.com>
Date:	Tue, 16 Apr 2013 13:32:56 +0200
From:	Ingo Molnar <mingo@...nel.org>
To:	Robin Holt <holt@....com>
Cc:	Ingo Molnar <mingo@...hat.com>, Russ Anderson <rja@....com>,
	Shawn Guo <shawn.guo@...aro.org>,
	Oleg Nesterov <oleg@...hat.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	"H. Peter Anvin" <hpa@...or.com>, Joe Perches <joe@...ches.com>,
	Lai Jiangshan <laijs@...fujitsu.com>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	Michel Lespinasse <walken@...gle.com>,
	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
	Paul Mackerras <paulus@...ba.org>,
	Peter Zijlstra <peterz@...radead.org>,
	"rusty@...tcorp.com.au" <rusty@...tcorp.com.au>,
	Tejun Heo <tj@...nel.org>,
	the arch/x86 maintainers <x86@...nel.org>,
	Thomas Gleixner <tglx@...utronix.de>
Subject: Re: [Patch -v4 1/4] Migrate shutdown/reboot to boot cpu.


* Robin Holt <holt@....com> wrote:

> We recently noticed that reboot of a 1024 cpu machine takes approx 16
> minutes of just stopping the cpus.  The slowdown was tracked to commit
> f96972f.
> 
> The current implementation does all the work of hot removing the cpus
> before halting the system.  We are switching to just migrating to the
> boot cpu and then continuing with shutdown/reboot.
> 
> This also has the effect of not breaking x86's command line parameter for
> specifying the reboot cpu.  Note, this code was shamelessly copied from
> arch/x86/kernel/reboot.c with bits removed pertaining to the reboot_cpu
> command line parameter.
> 
> Signed-off-by: Robin Holt <holt@....com>
> Tested-by: Shawn Guo <shawn.guo@...aro.org>
> To: Ingo Molnar <mingo@...hat.com>
> To: Russ Anderson <rja@....com>
> To: Oleg Nesterov <oleg@...hat.com>
> Cc: Andrew Morton <akpm@...ux-foundation.org>
> Cc: "H. Peter Anvin" <hpa@...or.com>
> Cc: Lai Jiangshan <laijs@...fujitsu.com>
> Cc: Linus Torvalds <torvalds@...ux-foundation.org>
> Cc: Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
> Cc: Michel Lespinasse <walken@...gle.com>
> Cc: Oleg Nesterov <oleg@...hat.com>
> Cc: "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
> Cc: Paul Mackerras <paulus@...ba.org>
> Cc: Peter Zijlstra <peterz@...radead.org>
> Cc: Robin Holt <holt@....com>
> Cc: "rusty@...tcorp.com.au" <rusty@...tcorp.com.au>
> Cc: Tejun Heo <tj@...nel.org>
> Cc: the arch/x86 maintainers <x86@...nel.org>
> Cc: Thomas Gleixner <tglx@...utronix.de>
> Cc: <stable@...r.kernel.org>
> 
> ---
> 
> Changes since -v1.
> - Set PF_THREAD_BOUND before migrating to eliminate potential race.
> - Modified kernel_power_off to also migrate instead of using
>   disable_nonboot_cpus().
> ---
>  kernel/sys.c | 22 +++++++++++++++++++---
>  1 file changed, 19 insertions(+), 3 deletions(-)
> 
> diff --git a/kernel/sys.c b/kernel/sys.c
> index 0da73cf..5ef7aa2 100644
> --- a/kernel/sys.c
> +++ b/kernel/sys.c
> @@ -357,6 +357,22 @@ int unregister_reboot_notifier(struct notifier_block *nb)
>  }
>  EXPORT_SYMBOL(unregister_reboot_notifier);
>  
> +void migrate_to_reboot_cpu(void)

It appears to be file-scope, so should be static I guess?

> +{
> +	/* The boot cpu is always logical cpu 0 */
> +	int reboot_cpu_id = 0;
> +
> +	/* Make certain the cpu I'm about to reboot on is online */
> +	if (!cpu_online(reboot_cpu_id))
> +		reboot_cpu_id = smp_processor_id();

Shouldn't we pick the first online CPU instead, to make it deterministic?

Also, does this codepath prevent hotplug from going on in parallel?

( Plus, the smp_processor_id() is in a preemptible section AFAICS, so it will 
  throw a warning with preempt debug on. )

> +
> +	/* Prevent races with other tasks migrating this task. */

(I guess the colon can be dropped here, like in the other comments.)

> +	current->flags |= PF_THREAD_BOUND;
> +
> +	/* Make certain I only run on the appropriate processor */
> +	set_cpus_allowed_ptr(current, cpumask_of(reboot_cpu_id));
> +}

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/