linux-kernel - Re: [RFC PATCHSET] workqueue: implement concurrency managed workqueue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1254645687.26976.9.camel@twins>
Date:	Sun, 04 Oct 2009 10:41:27 +0200
From:	Peter Zijlstra <peterz@...radead.org>
To:	Tejun Heo <tj@...nel.org>
Cc:	jeff@...zik.org, mingo@...e.hu, linux-kernel@...r.kernel.org,
	akpm@...ux-foundation.org, jens.axboe@...cle.com,
	rusty@...tcorp.com.au, cl@...ux-foundation.org,
	dhowells@...hat.com, arjan@...ux.intel.com
Subject: Re: [RFC PATCHSET] workqueue: implement concurrency managed
 workqueue

On Thu, 2009-10-01 at 17:08 +0900, Tejun Heo wrote:
> Hello, all.
> 
> This rather large patchset implements concurrency managed workqueue.
> It's not complete yet.  Singlethread workqueue handling needs more
> work and workqueue users need to be audited and simplified and async
> and slow-work should be reimplemented in terms of workqueue.  Although
> this patchset currently adds ~2000 lines of code, I'm fairly
> optimistic that after the whole conversion is done, it would be a net
> decrease in lines of code.
> 
> This patchset reimplements workqueue such that it auto-regulates
> concurrency and thus relieves its users from the managing duty.  It
> works by managing single shared pool of per-cpu workers and hooking
> into the scheduler to get notifications about workers going to sleep
> and waking up.  Using the mechanism, workqueue implementation keeps
> track of the current level of concurrency and schedules only the
> necessary number of workers to keep the cpu occupied.
> 
> Concurrency managed workqueue has the following benefits.
> 
> * Workqueue users no longer have to worry about managing concurrency
>   and, in most cases, deadlocks.  The workqueue will manage it
>   automatically and unless the deadlock chain involves many (currently
>   127) works, it won't happen.
> 
> * There's one single shared pool of workers per cpu and one rescuer
>   for each workqueue which requires it, so there are far fewer number
>   of kthreads.
> 
> * More efficient.  Although it adds considerable amount of code, the
>   code added to hot path isn't big and works will be executed on the
>   local cpu and in batch as much as possible using minimal number of
>   kthreads leading to fewer task switches and lower cache
>   footprint. <NEED SOME BACKING NUMBERS>
> 
> * As concurrency is no longer a problem, most types of asynchronous
>   jobs can be done using generic workqueue and other async mechanisms,
>   including slow-work, async and adhoc subsystem custom ones, can be
>   removed.  ie. It can serve as the unified async thread pool
>   mechanism.
> 
> Please read the patch description of the last patch for more details.
> 
> This patchset contains the following 19 patches and most of these are
> not signed off yet.

Like Linus, I dislike the sched_class bits (as in really hate them).

Also, from a quick look it looks like this scheme does not allow
priority inheritance of worklets, like we used to do in -rt.


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/