linux-kernel - Re: [RFC PATCHSET] workqueue: implement concurrency managed workqueue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <31399.1254497883@redhat.com>
Date:	Fri, 02 Oct 2009 16:38:03 +0100
From:	David Howells <dhowells@...hat.com>
To:	Tejun Heo <tj@...nel.org>
Cc:	dhowells@...hat.com, jeff@...zik.org, mingo@...e.hu,
	linux-kernel@...r.kernel.org, akpm@...ux-foundation.org,
	jens.axboe@...cle.com, rusty@...tcorp.com.au,
	cl@...ux-foundation.org, arjan@...ux.intel.com
Subject: Re: [RFC PATCHSET] workqueue: implement concurrency managed workqueue

Tejun Heo <tj@...nel.org> wrote:

> Given that slow-work isn't being used too extensively yet, I was
> thinking whether that part could be pushed down to the caller.  Or, we
> can also wrap work and export an interface which supports the get/put
> reference.

The caller of what?

I found the refcounting much easier to manage in slow-work when slow-work
actively got/put refs on the work items it was queueing.  The reason for that
is that slow-work can handle the queue/queue races and the requeue/execute
races much more efficiently.

Part of this was due to the fact I wanted to prevent re-entry into the work
executor, and to do that I had maintenance flags in the work item struct - but
that meant that slow-work had to modify the work item after execution.

So I should adjust point 1 on my list.

  (1) Work items can be requeued whilst they are executing, but the execution
      function will not be re-entered until after the current execution
      completes, but rather the execution will be deferred.

One possible problem with assuming that you can no longer access the work item
after you call the execution function, is that it's slightly dodgy to retain
the pointer to it to prevent reentry as the item can be destroyed, reallocated
and queued before the execution function returns.

Anyway, don't let me put you off re-implementing the whole shebang - it needs
doing.

> Binding is usually beneficial and doesn't matter for IO intensive
> ones, so...

The scenario I'm thinking of is this: someone who has an NFS volume cached
through FS-Cache does a tar of a large tree of files (say a kernel source
tree).  FS-Cache adds a long duration work item for each of those files
(~32000) to create structure in the cache.  Will all of those wind up bound to
the same CPU as was running tar?

David
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/