linux-kernel - Re: [PATCH 01/51] CPU hotplug: Provide lockless versions of callback registration functions

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1392136436.5612.131.camel@misato.fc.hp.com>
Date:	Tue, 11 Feb 2014 09:33:56 -0700
From:	Toshi Kani <toshi.kani@...com>
To:	"Srivatsa S. Bhat" <srivatsa.bhat@...ux.vnet.ibm.com>
Cc:	"paulus@...ba.org" <paulus@...ba.org>,
	"oleg@...hat.com" <oleg@...hat.com>,
	"rusty@...tcorp.com.au" <rusty@...tcorp.com.au>,
	"peterz@...radead.org" <peterz@...radead.org>,
	"tglx@...utronix.de" <tglx@...utronix.de>,
	"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
	"mingo@...nel.org" <mingo@...nel.org>,
	"paulmck@...ux.vnet.ibm.com" <paulmck@...ux.vnet.ibm.com>,
	"tj@...nel.org" <tj@...nel.org>,
	"walken@...gle.com" <walken@...gle.com>,
	"ego@...ux.vnet.ibm.com" <ego@...ux.vnet.ibm.com>,
	"linux@....linux.org.uk" <linux@....linux.org.uk>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"Rafael J. Wysocki" <rafael.j.wysocki@...el.com>
Subject: Re: [PATCH 01/51] CPU hotplug: Provide lockless versions of
 callback registration functions

On Tue, 2014-02-11 at 09:27 +0000, Srivatsa S. Bhat wrote:
> On 02/11/2014 06:56 AM, Toshi Kani wrote:
> > On Thu, 2014-02-06 at 03:34 +0530, Srivatsa S. Bhat wrote:
> >  :
> [...]
> >>
> >> Also, since cpu_maps_update_begin/done() is like a super-set of
> >> get/put_online_cpus(), the former naturally protects the critical sections
> >> from concurrent hotplug operations.
> > 
> > get/put_online_cpus() is a reader-lock and concurrent executions are
> > allowed among the readers.  They won't be serialized until a cpu
> > online/offline operation begins.  By replacing this lock with
> > cpu_maps_update_begin/done(), we now serialize all readers.  Isn't that
> > too restrictive?
> 
> That's an excellent line of thought! It doesn't really hurt at the moment
> because the for_each_online_cpu() kind of loop that the initcalls of various
> subsystems run (before registering the notifier) are really tiny (typically
> the loop runs for just 1 cpu, the boot-cpu). In other words, this change
> represents a tiny increase in the critical section size; so its effect
> shouldn't be noticeable. (Note that in the old model, register_cpu_notifier()
> already takes the cpu_add_remove_lock, so they will be serialized at that
> point, and this is necessary).
> 
> However, going forward, when we start using more aggressive CPU onlining
> techniques during boot (such as parallel CPU hotplug), the issue you pointed
> out can become a real bottleneck, since for_each_online_cpu() can become
> quite a large loop, and hence explicit (and unnecessary) mutual exclusion
> will start hurting.
>  
> >  Can we fix the issue with CPU_POST_DEAD and continue
> > to use get_online_cpus()?
> > 
> 
> We don't want to get rid of CPU_POST_DEAD, so unfortunately we can't continue
> to use get_online_cpus(). However, I am thinking of introducing a Reader-Writer
> semaphore for this purpose, so that the registration routines can run in
> parallel most of the time. (Basically, the rw-semaphore is like
> get/put_online_cpus(), except that it protects the full hotplug critical section,
> including the CPU_POST_DEAD stage.)

I agree that introducing a reader-writer semaphore allows concurrent
executions.  Adding yet another hotplug lock is a bit unfortunate,
though.

This may be a dumb question, but can't we simply do this way?

        get_online_cpus();

        for_each_online_cpu(cpu)
                init_cpu(cpu);

        put_online_cpus();

        register_cpu_notifier(&foobar_cpu_notifier);

Thanks,
-Toshi
 


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/