linux-kernel - Re: [RFC PATCHSET] workqueue: implement concurrency managed workqueue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 1 Oct 2009 10:40:40 +0200
From:	Ingo Molnar <mingo@...e.hu>
To:	Tejun Heo <tj@...nel.org>, Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Avi Kivity <avi@...hat.com>,
	Frédéric Weisbecker <fweisbec@...il.com>
Cc:	jeff@...zik.org, linux-kernel@...r.kernel.org,
	akpm@...ux-foundation.org, jens.axboe@...cle.com,
	rusty@...tcorp.com.au, cl@...ux-foundation.org,
	dhowells@...hat.com, arjan@...ux.intel.com
Subject: Re: [RFC PATCHSET] workqueue: implement concurrency managed
	workqueue


* Tejun Heo <tj@...nel.org> wrote:

> Hello, all.
> 
> This rather large patchset implements concurrency managed workqueue. 
> It's not complete yet.  Singlethread workqueue handling needs more 
> work and workqueue users need to be audited and simplified and async 
> and slow-work should be reimplemented in terms of workqueue.  Although 
> this patchset currently adds ~2000 lines of code, I'm fairly 
> optimistic that after the whole conversion is done, it would be a net 
> decrease in lines of code.
> 
> This patchset reimplements workqueue such that it auto-regulates 
> concurrency and thus relieves its users from the managing duty.  It 
> works by managing single shared pool of per-cpu workers and hooking 
> into the scheduler to get notifications about workers going to sleep 
> and waking up.  Using the mechanism, workqueue implementation keeps 
> track of the current level of concurrency and schedules only the 
> necessary number of workers to keep the cpu occupied.

Ok, this looks fairly interesting - and the way you reused scheduler 
classes to auto-regulate with no impact on regular performance is quite 
an ingenious idea as well. (KVM's preempt notifiers should probably use 
this trick too, instead of an ugly notifier in the scheduler hotpath)

This mechanism could be used to implement threadlets/syslets too btw., 
and other forms of asynchronous IO.

My main worry is that in practice workqueues arent all that performance 
critical - so we are shooting to optimize something that doesnt 
necessarily use all the potential goodness inherent in this approach.

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/