linux-kernel - Re: [PATCH 2/2] smp_call_function: use rwlocks on queues rather than rcu

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1219677736.8515.69.camel@twins>
Date:	Mon, 25 Aug 2008 17:22:16 +0200
From:	Peter Zijlstra <peterz@...radead.org>
To:	paulmck@...ux.vnet.ibm.com
Cc:	Christoph Lameter <cl@...ux-foundation.org>,
	Pekka Enberg <penberg@...helsinki.fi>,
	Ingo Molnar <mingo@...e.hu>,
	Jeremy Fitzhardinge <jeremy@...p.org>,
	Nick Piggin <nickpiggin@...oo.com.au>,
	Andi Kleen <andi@...stfloor.org>,
	"Pallipadi, Venkatesh" <venkatesh.pallipadi@...el.com>,
	Suresh Siddha <suresh.b.siddha@...el.com>,
	Jens Axboe <jens.axboe@...cle.com>,
	Rusty Russell <rusty@...tcorp.com.au>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 2/2] smp_call_function: use rwlocks on queues rather
	than rcu

On Mon, 2008-08-25 at 08:12 -0700, Paul E. McKenney wrote:
> On Mon, Aug 25, 2008 at 12:31:31PM +0200, Peter Zijlstra wrote:
> > On Fri, 2008-08-22 at 13:53 -0700, Paul E. McKenney wrote:
> > > On Fri, Aug 22, 2008 at 03:03:13PM -0500, Christoph Lameter wrote:
> > > > Paul E. McKenney wrote:
> > > > 
> > > > > I was indeed thinking in terms of the free from RCU being specially marked.
> > > > 
> > > > Isnt there some way to shorten the rcu periods significantly? Critical
> > > > sections do not take that long after all.
> > > 
> > > In theory, yes.  However, the shorter the grace period, the greater the
> > > per-update overhead of grace-period detection -- the general approach
> > > is to use a per-CPU high-resolution timer to force RCU grace period
> > > processing every 100 microseconds or so.  
> > 
> > You could of course also drive the rcu state machine from
> > rcu_read_unlock().
> 
> True, and Jim Houston implemented something similar to this some years
> back: http://marc.theaimsgroup.com/?l=linux-kernel&m=109387402400673&w=2
> 
> This of course greatly increases rcu_read_unlock() overhead.  But
> perhaps it is a good implementation for the workloads that Christoph is
> thinking of.
> 
> > > Also, by definition, the RCU
> > > grace period can be no shorter than the longest active RCU read-side
> > > critical section.  Nevertheless, I have designed my current hierarchical
> > > RCU patch with expedited grace periods in mind, though more for the
> > > purpose of reducing latency of long strings of operations that involve
> > > synchronize_rcu() than for cache locality.
> > 
> > Another thing that could be done is more often force a grace period by
> > flipping the counters.
> 
> Yep.  That is exactly what I was getting at with the high-resolution
> timer point above.  This seems to be a reasonable compromise, as it
> allows someone to specify how quickly the grace periods happen
> dynamically.
> 
> But I am not sure that this gets the grace periods to go fast enough to
> cover Christoph's use case -- he seems to be in a "faster is better"
> space rather than in an "at least this fast" space.  Still, it would
> likely help in some important cases.

If we combine these two cases, and flip the counter as soon as we've
enqueued one callback, unless we're already waiting for a grace period
to end - which gives us a longer window to collect callbacks.

And then the rcu_read_unlock() can do:

  if (dec_and_zero(my_counter) && my_index == dying)
    raise_softirq(RCU)

to fire off the callback stuff.

/me ponders - there must be something wrong with that...

Aaah, yes, the dec_and_zero is non trivial due to the fact that its a
distributed counter. Bugger..

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/