linux-kernel - Re: [RFC][PATCH] create workqueue threads only when needed

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 02 Feb 2009 10:58:47 +0100
From:	Peter Zijlstra <a.p.zijlstra@...llo.nl>
To:	Benjamin Herrenschmidt <benh@...nel.crashing.org>
Cc:	Oliver Neukum <oliver@...kum.org>,
	Stefan Richter <stefanr@...6.in-berlin.de>,
	Frederic Weisbecker <fweisbec@...il.com>,
	Arjan van de Ven <arjan@...radead.org>,
	Ingo Molnar <mingo@...e.hu>, linux-kernel@...r.kernel.org,
	Andrew Morton <akpm@...ux-foundation.org>,
	Lai Jiangshan <laijs@...fujitsu.com>,
	Steven Rostedt <rostedt@...dmis.org>,
	Chris Mason <chris.mason@...cle.com>
Subject: Re: [RFC][PATCH] create workqueue threads only when needed

On Mon, 2009-02-02 at 20:46 +1100, Benjamin Herrenschmidt wrote:
> On Mon, 2009-02-02 at 10:14 +0100, Oliver Neukum wrote:
> > Am Monday 02 February 2009 10:05:28 schrieb Benjamin Herrenschmidt:
> > > > Work which /may/ sleep longer, for example performs SCSI transactions,
> > > > needs to go into a private workqueue or other kind of context.
> > > 
> > > Well, it's a bit silly to allocate a private workqueue with all it's
> > > associated per CPU kernel threads for something as rare as resetting
> > > your eth NIC ... or even SCSI error handling in fact.
> > 
> > How do you avoid a deadlock if SCSI error handling doesn't use
> > a dedicated workqueue?
> 
> Something such as slow-work or async funcs (not sure about the later, I
> have to look at the implementation) but the basic idea is to have a pool
> of threads for "generic" delayed work, when its busy, pick another one,
> and the pool itself should resize if there's too much pressure.

One thing that comes to mind is that some people want strict cpu
affinity for their work, while others want the work to be freely
scheduled so that the load-balancer can get the most out of the system.

I think mason does something like that in btrfs, where he has likes to
keep all cpus busy doing checksumming or something or other.

One could of course layer these things so that the principal thread
pools are per-cpu and then let the next layer RR the work between all
cpus.

Furthermore, all cpus seems like too wide a mask, imagine what happens
on a 4k cpu system, do we really want a cpu half the world away from the
IO node doing checksumming? -- is there a nice solution?

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/