linux-kernel - Re: [PATCH v3 10/52] arm, kvm: Fix CPU hotplug callback registration

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20140314191055.GF28661@cbox>
Date:	Fri, 14 Mar 2014 12:10:55 -0700
From:	Christoffer Dall <christoffer.dall@...aro.org>
To:	"Srivatsa S. Bhat" <srivatsa.bhat@...ux.vnet.ibm.com>
Cc:	paulus@...ba.org, oleg@...hat.com, mingo@...nel.org,
	rjw@...ysocki.net, rusty@...tcorp.com.au, peterz@...radead.org,
	tglx@...utronix.de, akpm@...ux-foundation.org,
	paulmck@...ux.vnet.ibm.com, tj@...nel.org, walken@...gle.com,
	ego@...ux.vnet.ibm.com, linux@....linux.org.uk,
	linux-kernel@...r.kernel.org, linux-arch@...r.kernel.org,
	linux-pm@...r.kernel.org, linuxppc-dev@...abs.org,
	Gleb Natapov <gleb@...nel.org>, kvmarm@...ts.cs.columbia.edu,
	kvm@...r.kernel.org, linux-arm-kernel@...ts.infradead.org,
	Paolo Bonzini <pbonzini@...hat.com>, marc.zyngier@....com
Subject: Re: [PATCH v3 10/52] arm, kvm: Fix CPU hotplug callback registration

On Fri, Mar 14, 2014 at 11:13:29AM +0530, Srivatsa S. Bhat wrote:
> On 03/13/2014 04:51 AM, Christoffer Dall wrote:
> > On Tue, Mar 11, 2014 at 02:05:38AM +0530, Srivatsa S. Bhat wrote:
> >> Subsystems that want to register CPU hotplug callbacks, as well as perform
> >> initialization for the CPUs that are already online, often do it as shown
> >> below:
> >>
> >> 	get_online_cpus();
> >>
> >> 	for_each_online_cpu(cpu)
> >> 		init_cpu(cpu);
> >>
> >> 	register_cpu_notifier(&foobar_cpu_notifier);
> >>
> >> 	put_online_cpus();
> >>
> >> This is wrong, since it is prone to ABBA deadlocks involving the
> >> cpu_add_remove_lock and the cpu_hotplug.lock (when running concurrently
> >> with CPU hotplug operations).
> >>
> >> Instead, the correct and race-free way of performing the callback
> >> registration is:
> >>
> >> 	cpu_notifier_register_begin();
> >>
> >> 	for_each_online_cpu(cpu)
> >> 		init_cpu(cpu);
> >>
> >> 	/* Note the use of the double underscored version of the API */
> >> 	__register_cpu_notifier(&foobar_cpu_notifier);
> >>
> >> 	cpu_notifier_register_done();
> >>
> >>
> >> Fix the kvm code in arm by using this latter form of callback registration.
> >>
> >> Cc: Christoffer Dall <christoffer.dall@...aro.org>
> >> Cc: Gleb Natapov <gleb@...nel.org>
> >> Cc: Russell King <linux@....linux.org.uk>
> >> Cc: Ingo Molnar <mingo@...nel.org>
> >> Cc: kvmarm@...ts.cs.columbia.edu
> >> Cc: kvm@...r.kernel.org
> >> Cc: linux-arm-kernel@...ts.infradead.org
> >> Acked-by: Paolo Bonzini <pbonzini@...hat.com>
> >> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@...ux.vnet.ibm.com>
> >> ---
> >>
> >>  arch/arm/kvm/arm.c |    7 ++++++-
> >>  1 file changed, 6 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> >> index bd18bb8..f0e50a0 100644
> >> --- a/arch/arm/kvm/arm.c
> >> +++ b/arch/arm/kvm/arm.c
> >> @@ -1051,21 +1051,26 @@ int kvm_arch_init(void *opaque)
> >>  		}
> >>  	}
> >>  
> >> +	cpu_notifier_register_begin();
> >> +
> >>  	err = init_hyp_mode();
> >>  	if (err)
> >>  		goto out_err;
> >>  
> >> -	err = register_cpu_notifier(&hyp_init_cpu_nb);
> >> +	err = __register_cpu_notifier(&hyp_init_cpu_nb);
> >>  	if (err) {
> >>  		kvm_err("Cannot register HYP init CPU notifier (%d)\n", err);
> >>  		goto out_err;
> >>  	}
> >>  
> >> +	cpu_notifier_register_done();
> >> +
> >>  	hyp_cpu_pm_init();
> >>  
> >>  	kvm_coproc_table_init();
> >>  	return 0;
> >>  out_err:
> >> +	cpu_notifier_register_done();
> >>  	return err;
> >>  }
> >>  
> >>
> > 
> > Just so we're clear, the existing code was simply racy as not prone to
> > deadlocks, right?
> > 
> > This makes it clear that the test above for compatible CPUs can be quite
> > easily evaded by using CPU hotplug, but we don't really have a good
> > solution for handling that yet...  Hmmm, grumble grumble, I guess if you
> > hotplug unsupported CPUs on a KVM/ARM system for now, stuff will break.
> > 
> 
> In this particular case, there was no deadlock possibility, rather the
> existing code had insufficient synchronization against CPU hotplug.
> 
> init_hyp_mode() would invoke cpu_init_hyp_mode() on currently online CPUs
> using on_each_cpu(). If a CPU came online after this point and before calling
> register_cpu_notifier(), that CPU would remain uninitialized because this
> subsystem would miss the hot-online event. This patch fixes this bug and
> also uses the new synchronization method (instead of get/put_online_cpus())
> to ensure that we don't deadlock with CPU hotplug.
> 

Yes, that was my conclusion as well.  Thanks for clarifying.  (It could
be noted in the commit message as well if you should feel so inclined).

> > In any case:
> > Acked-by: Christoffer Dall <christoffer.dall@...aro.org>
> > 
> 
> Thanks a lot!
> 
Thanks,
-Christoffer
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/