[<prev] [next>] [day] [month] [year] [list]
Message-ID: <20090116220832.GB20653@elte.hu>
Date: Fri, 16 Jan 2009 23:08:32 +0100
From: Ingo Molnar <mingo@...e.hu>
To: Rusty Russell <rusty@...tcorp.com.au>
Cc: Herbert Xu <herbert@...dor.apana.org.au>,
akpm@...ux-foundation.org, tj@...nel.org, hpa@...or.com,
brgerst@...il.com, ebiederm@...ssion.com, cl@...ux-foundation.org,
travis@....com, linux-kernel@...r.kernel.org, steiner@....com,
hugh@...itas.com, "David S. Miller" <davem@...emloft.net>,
netdev@...r.kernel.org,
Mathieu Desnoyers <mathieu.desnoyers@...ymtl.ca>
Subject: Re: [PATCH] percpu: add optimized generic percpu accessors
* Rusty Russell <rusty@...tcorp.com.au> wrote:
> On Friday 16 January 2009 10:48:24 Herbert Xu wrote:
> > On Fri, Jan 16, 2009 at 01:15:44AM +0100, Ingo Molnar wrote:
> > >
> > > > So if you could design the API such that we have a variant of add/inc
> > > > that automatically disables/enables preemption then we can optimise that
> > > > away on x86.
> > >
> > > Yeah. percpu_add(var, 1) does exactly that on x86.
>
> <sigh>. No it doesn't.
What do you mean by "No it doesn't". It does exactly what i claimed it
does.
> It's really nice that everyone's excited about this, but it's more
> complex than this. Unf. I'm too busy preparing for linux.conf.au to
> explain it all properly right now, but here's the highlights:
>
> 1) This only works on static per-cpu vars.
> - We are working on fixing this, but it's non-trivial for large allocs like
> those in networking. Small allocs, we have patches for.
How do difficulties of dynamic percpu-alloc make my above suggestion
unsuitable for SNMP stats in networking? Most of those stats are not
dynamically allocated - they are plain straightforward percpu variables.
Plus the majority of percpu usage is static - just like the majority of
local variables is static, not dynamic. So any percpu-alloc complication
is a non-issue.
> 2) The generic versions of these as posted by Tejun are unsuitable for
> networking; they need to bh_disable. That would make networking less
> efficient than it is now for non-x86, and to be generic it would have
> to be local_irq_save/restore anyway.
The generic versions will not be used on 95%+ of the active Linux systems
out there, as they run on x86. If you worry about the remaining 5%, those
can be optimized too.
> 3) local_t was designed to do exactly this: a fast cpu-local counter
> implemented optimally for each arch. For sparc64, doing a trivalue version
> seems optimal, for s390 atomics, for x86 single-insn, for powerpc
> irq_save/restore, etc.
But local_t does not actually solve this problem at all - because one
still has to have per-cpu-ness.
> 4) Unfortunately, local_t has been extended beyond a simple counter, meaning
> it now has more complex requirements (eg. Mathieu wants nmi-safe, even
> though that's impossible on sparc and parisc, and percpu_counter wants
> local_add_return, which makes trival less desirable). These discussions
> are on the back burner at the moment, but ongoing.
In reality local_t has almost zero users in the kernel - despite being
with us at least since v2.6.12. That pretty much tells us all about its
utility.
The thing is, local_t without proper percpu integration is a toothless
tiger in the jungle. And our APIS do exactly that kind of integration and
i expect them to be more popular than local_t. There's already a dozen
usage sites of it in arch/x86.
Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists