linux-kernel - Re: per-cpu stats in block device: overkill?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <200812221426.09723.rusty@rustcorp.com.au>
Date:	Mon, 22 Dec 2008 14:26:09 +1030
From:	Rusty Russell <rusty@...tcorp.com.au>
To:	Tejun Heo <tj@...nel.org>
Cc:	Jens Axboe <jens.axboe@...cle.com>,
	Jerome Marchand <jmarchan@...hat.com>,
	linux-kernel@...r.kernel.org,
	Christoph Lameter <cl@...ux-foundation.org>,
	Mathieu Desnoyers <mathieu.desnoyers@...ymtl.ca>
Subject: Re: per-cpu stats in block device: overkill?

On Monday 22 December 2008 12:25:54 Tejun Heo wrote:
> I'm working on local counter (local_t) allocator which is used to
> replace percpu allocation in percpu_counter and used as basis for
> percpu_ref which replaces module ref counting and will be used to
> simplify block/char lifetime rules.

Hi Tejun,

   Interesting!  Thanks to Christoph's dynamic percpu efforts, I've
been revising efforts to make alloc_percpu use the same efficient
mechanism that static percpu vars use.  We actually have this code
already, tucked away in module.c.

   This work is basically complete; the step I started this morning
is to remove the per_cpu__ prefix hackery from the per-cpu ops (in favour
of sparse annotations).  This leads to cpu_local_inc et. al. being usable
for alloc_percpu-created percpu vars, not just static ones.

> The local counter allocator allocates per-cpu pages and the space
> overhead is minimal.  If per-cpu stats in genhd is necessary, I think
> converting it to percpu local counter allocation should do it.

   Interesting; an allyesconfig boot uses 194 per-cpu allocs from
lib/percpu_counter.c at the moment.  The module.c allocator is fairly space
efficient: 4 bytes per "block" (ie. each allocation or hole) but slow,
which I figure is OK.  Packing is good though.

> BTW, why make percpu area static?

   Good question.  Archs use a simple offset for per-cpu areas: some hold
this in a register (eg. %fs for x86-32).  This means that the layout must be
"congruent" (ie. have the same inter-cpu spacing) if we allocate a new
per-cpu area (hard for non-NUMA).

  For 5 years I waited for this to be fixed, and avoided exposing the per-cpu
core, and the alloc_percpu stuff was a standin implementation.  But Christoph
L. showed that even with the size limit, there are numerous places which want
small per-cpu allocations which are optimally accessed, so I restarted work.
See Message-Id: <20081117132630.33F09DDDF5@...abs.org> "[PATCH 1/7] Improve
alloc_percpu: make the per cpu reserve configurable and larger." and thread.

  In addition, Mathieu and I have been discussing local_t: it's wandered
off its original purpose and we're debating what to do about it.  See
Message-ID: <alpine.DEB.1.10.0812150823370.18692@...dalf.stny.rr.com>
"local_add_return" and thread.

I look forward to always-cogent your thoughts on these issues!
Rusty.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/