linux-kernel - Re: [PATCH RFC 5/9] RCU: CPU hotplug support for preemptible RCU

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20071001014142.GC12494@linux.vnet.ibm.com>
Date:	Sun, 30 Sep 2007 18:41:42 -0700
From:	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
To:	Oleg Nesterov <oleg@...sign.ru>
Cc:	linux-kernel@...r.kernel.org, linux-rt-users@...r.kernel.org,
	mingo@...e.hu, akpm@...ux-foundation.org, dipankar@...ibm.com,
	josht@...ux.vnet.ibm.com, tytso@...ibm.com, dvhltc@...ibm.com,
	tglx@...utronix.de, a.p.zijlstra@...llo.nl, bunk@...nel.org,
	ego@...ibm.com, srostedt@...hat.com
Subject: Re: [PATCH RFC 5/9] RCU: CPU hotplug support for preemptible RCU

On Sun, Sep 30, 2007 at 08:38:49PM +0400, Oleg Nesterov wrote:
> On 09/10, Paul E. McKenney wrote:
> >
> > --- linux-2.6.22-d-schedclassic/kernel/rcupreempt.c	2007-08-22 15:45:28.000000000 -0700
> > +++ linux-2.6.22-e-hotplugcpu/kernel/rcupreempt.c	2007-08-22 15:56:22.000000000 -0700
> > @@ -125,6 +125,8 @@ enum rcu_mb_flag_values {
> >  };
> >  static DEFINE_PER_CPU(enum rcu_mb_flag_values, rcu_mb_flag) = rcu_mb_done;
> >
> > +static cpumask_t rcu_cpu_online_map = CPU_MASK_NONE;
> 
> I'd suggest to append "__read_mostly"

Makes sense!

> > +void rcu_offline_cpu_rt(int cpu)
> > +{
> > [...snip...]
> > +	spin_lock_irqsave(&rcu_ctrlblk.fliplock, oldirq);
> > +	rcu_check_mb(cpu);
> > +	if (per_cpu(rcu_flip_flag, cpu) == rcu_flipped) {
> > +		smp_mb();  /* Subsequent counter accesses must see new value */
> > +		per_cpu(rcu_flip_flag, cpu) = rcu_flip_seen;
> > +		smp_mb();  /* Subsequent RCU read-side critical sections */
> > +			   /*  seen -after- acknowledgement. */
> 
> Imho, all these barriers are unneeded and confusing, we can't do them on behalf
> of a dead CPU anyway. Can't we just do
> 
> 	per_cpu(rcu_mb_flag, cpu) = rcu_mb_done;
> 	per_cpu(rcu_flip_flag, cpu) = rcu_flip_seen;
> ?

You are likely correct, but this is a slow path, extremely hard to
stress test, and I am freakin' paranoid about this sort of thing.

> Why can't we also do
> 
> 	__get_cpu_var(rcu_flipctr)[0] += per_cpu(rcu_flipctr, cpu)[0];
> 	per_cpu(rcu_flipctr, cpu)[0] = 0;	
> 	__get_cpu_var(rcu_flipctr)[1] += per_cpu(rcu_flipctr, cpu)[1];
> 	per_cpu(rcu_flipctr, cpu)[1] = 0;	
> 
> ? This way rcu_try_flip_waitzero() can also use rcu_cpu_online_map. This cpu
> is dead, nobody can modify per_cpu(rcu_flipctr, cpu). And we can't confuse
> rcu_try_flip_waitzero(), we are holding rcu_ctrlblk.fliplock.

Very good point!!!  This would reduce latencies on systems where
the number of possible CPUs greatly exceeds that of the number of
online CPUs, so seems quite worthwhile.

> > +void __devinit rcu_online_cpu_rt(int cpu)
> > +{
> > +	unsigned long oldirq;
> > +
> > +	spin_lock_irqsave(&rcu_ctrlblk.fliplock, oldirq);
> > +	cpu_set(cpu, rcu_cpu_online_map);
> 
> What if _cpu_up() fails? I think rcu_cpu_notify(CPU_UP_CANCELED) should call
> rcu_offline_cpu_rt() too.

Good catch, will fix!!!

							Thanx, Paul
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/