linux-kernel - Re: [PATCH] generic dynamic per cpu refcounting

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130128202214.GD26407@google.com>
Date:	Mon, 28 Jan 2013 12:22:14 -0800
From:	Kent Overstreet <koverstreet@...gle.com>
To:	Tejun Heo <tj@...nel.org>
Cc:	Oleg Nesterov <oleg@...hat.com>, srivatsa.bhat@...ux.vnet.ibm.com,
	rusty@...tcorp.com.au, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] generic dynamic per cpu refcounting

On Mon, Jan 28, 2013 at 10:55:52AM -0800, Tejun Heo wrote:
> Hey, Kent.
> 
> On Mon, Jan 28, 2013 at 10:49:33AM -0800, Kent Overstreet wrote:
> > Yeah. It'd be really nice if it was doable without synchronize_rcu(),
> > but it'd definitely make get/put heavier.
> > 
> > Though, re. close() - considering we only need a synchronize_rcu() if
> > the ref was in percpu mode, I wonder if that would be a dealbreaker. I
> > have no clue myself.
> 
> The problem is that the performance drop (or latency increase) in
> patheological cases would be catastrophic.  We're talking about
> possibly quite a few millisecs of delay between each close().  When
> done sequentially for large number of files, it gets ugly.  It becomes
> a dangerous optimization to make.

Yeah, I tend to agree.

> > Getting rid of synchronize_rcu would basically require turning get and
> > put into cmpxchg() loops - even in the percpu fastpath. However, percpu
> > mode would still be getting rid of the shared cacheline contention, we'd
> > just be adding another branch that can be safely marked unlikely() - and
> > my current version has one of those already, so two branches instead of
> > one in the fast path.
> 
> Or offer an asynchrnous interface so that high-frequency users don't
> end up inserting synchronize_sched() between each call.  It makes the
> interface more complex and further away from simple atomic_t
> replacement tho.

Could do that too, but then teardown gets really messy for the user - we
need two synchronize_rcu()s:

state := dying

synchronize_rcu()

/* Now nothing's changing the per cpu counters */

Add per cpu counters to atomic counter counter

/* Atomic counter is now consistent */

state := dead

synchronize_rcu()

/* Now percpu_ref_put will check for ref == 0 */

/* Drop initial ref */

percpu_ref_put()

And note that the first synchronize_rcu() is only needed when we had
allocated per cpu counters, my current code skips it otherwise.

(which is a reason for keeping dynamic allocation I hadn't thought of -
if we don't ever allocate percpu counters, teardown is faster)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/