linux-kernel - Re: [RFC][PATCH] create workqueue threads only when needed

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090131180347.GC5884@nowhere>
Date:	Sat, 31 Jan 2009 19:03:49 +0100
From:	Frederic Weisbecker <fweisbec@...il.com>
To:	Arjan van de Ven <arjan@...radead.org>
Cc:	Ingo Molnar <mingo@...e.hu>, linux-kernel@...r.kernel.org,
	Andrew Morton <akpm@...ux-foundation.org>,
	Lai Jiangshan <laijs@...fujitsu.com>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	Steven Rostedt <rostedt@...dmis.org>
Subject: Re: [RFC][PATCH] create workqueue threads only when needed

On Mon, Jan 26, 2009 at 04:30:15PM -0800, Arjan van de Ven wrote:
> On Tue, 27 Jan 2009 01:17:11 +0100
> Frederic Weisbecker <fweisbec@...il.com> wrote:
> 
> > While looking at the statistics from the workqueue tracer, I've been
> > suprised by the number of useless workqueues I had:
> > 
> >  CPU  INSERTED  EXECUTED   NAME
> >  |      |         |          |
> > 
> >   *      0          0       kpsmoused
> >   *      0          0       ata_aux
> >   *      0          0       cqueue
> >   *      0          0       kacpi_notify
> >   *      0          0       kacpid
> >   *    998        998       khelper
> >   *      0          0       cpuset
> > 
> >   1      0          0       hda0/1
> >   1     42         42       reiserfs/1
> >   1      0          0       scsi_tgtd/1
> >   1      0          0       aio/1
> >   1      0          0       ata/1
> >   1    193        193       kblockd/1
> >   1      0          0       kintegrityd/1
> >   1      4          4       work_on_cpu/1
> >   1   1244       1244       events/1
> > 
> >   0      0          0       hda0/0
> >   0     63         63       reiserfs/0
> >   0      0          0       scsi_tgtd/0
> >   0      0          0       aio/0
> >   0      0          0       ata/0
> >   0    188        188       kblockd/0
> >   0      0          0       kintegrityd/0
> >   0     16         16       work_on_cpu/0
> >   0   1360       1360       events/0
> > 
> > 
> > All of the workqueues with 0 work inserted do nothing. 
> > For several reasons:
> > 
> > _ Unneeded built drivers for my system that create workqueue(s) when
> > they init _ Services which need their own workqueue, for several
> > reasons, but who receive very rare jobs (often never)
> > _ ...?
> > 
> > And the result of git-grep create_singlethread_workqueue is even more
> > surprising.
> > 
> > So I've started a patch which creates the workqueues by default
> > without thread except the kevents one.
> > They will have their thread created and started only when these
> > workqueues will receive a first work to do. This is performed by
> > submitting a task's creation work to the kevent workqueues which are
> > always there, and are the only one which have their thread started on
> > creation.
> > 
> > The result after this patch:
> > 
> > # CPU  INSERTED  EXECUTED   NAME
> > # |      |         |          |
> > 
> >   *    999       1000       khelper
> > 
> >   1      5          6       reiserfs/1
> >   1      0          2       work_on_cpu/1
> >   1     86         87       kblockd/1
> >   1     14         16       work_on_cpu/1
> >   1    149        149       events/1
> > 
> >   0     15         16       reiserfs/0
> >   0     85         86       kblockd/0
> >   0    146        146       events/0
> > 
> > 
> > Dropping 16 useless kernel threads in my system.
> > (Yes the inserted values are not synced with the executed one because
> > the tracers looses the first events. I just rewrote some parts to
> > make it work with this patch) .
> > I guess I will update this tracer to display the "shadow workqueues"
> > which have no threads too.
> > 
> > I hadn't any problems until now with this patch but I think it needs
> > more testing, like with cpu hotplug, and some renaming for its
> > functions and structures... And I would like to receive some comments
> > and feelings before continuing. So this is just an RFC :-)
> > 
> 
> one thing to look at for work queues that never get work is to see if
> they are appropriate for the async function call interface
> (the only requirement for that is that they need to cope with calling
> inline in exceptional cases)
> 


Hi Arjan,

There is one thing that make it hard to replace workqueues in such cases,
there is not guarantee the function will run in user context because of this
condition:

if (!async_enabled || !entry || atomic_read(&entry_count) > MAX_WORK)

I wanted to replace kpsmoused with an async function but I want to schedule
a slow work that can't be done from irq...

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/