Message-ID: <4C1901E9.2080907@kernel.org>
Date:	Wed, 16 Jun 2010 18:55:05 +0200
From:	Tejun Heo <tj@...nel.org>
To:	Daniel Walker <dwalker@...eaurora.org>
CC:	mingo@...e.hu, awalls@...ix.net, linux-kernel@...r.kernel.org,
	jeff@...zik.org, akpm@...ux-foundation.org, rusty@...tcorp.com.au,
	cl@...ux-foundation.org, dhowells@...hat.com,
	arjan@...ux.intel.com, johannes@...solutions.net, oleg@...hat.com,
	axboe@...nel.dk
Subject: Re: Overview of concurrency managed workqueue

Hello,

On 06/16/2010 06:30 PM, Daniel Walker wrote:
>> I don't know.  I suppose a high priority thread is trying to flush a
>> work item or workqueue thus causing priority inversion, right?  Maybe
>> we can add high priority emergency worker which gets triggered through
>> priority inversion detection or maybe the code shouldn't be flushing
>> in the critical path anyway.
> 
> It's not a flushing situation .. The high priority thread is a userspace
> thread so it , AFAIK, can't flush any workqueues.

So, how is it stalling?  How would I be able to tell anything about
the situation when all you're saying is doing this and that voodoo
thing made it go away for me?

>>>> Peter brought up the work priority issue previously and Linus was
>>>> pretty clear on the issue.  They're in the discussiosn for the first
>>>> or second take.
>>>
>>> Send us a link to this discussion.
>>
>>   http://thread.gmane.org/gmane.linux.kernel/929641
> 
> I didn't see anything in there related to this discussion.

It was about using wq for cpu intensive / RT stuff.  Linus said,

  So stop arguing about irrelevancies. Nobody uses workqueues for RT
  or for CPU-intensive crap. It's not what they were designed for, or
  used for.

>> Maybe it's me not understanding something but I don't really think
>> exposing workqueue priorities to userland is a good solution at all.
> 
> Why not? They currently are to no known ill effects (none that I know
> of).

Because...

* fragile as hell

* depends heavily on unrelated implementation details

* has extremely limited test coverage

* doesn't help progressing mainline at all

>>> You have no idea how many people are doing this, or in what
>>> circumstances .. Please don't make mass speculation over things you
>>> clearly are not aware of.
>>
>> Well, then please stop insisting it to be a feature to keep.  It's not
>> a feature.
> 
> It may not have been a feature in the past, but it's used as a feature
> now.. So it is a feature even tho you don't want it to be.

That's exactly like grepping /proc/kallsyms to determine some feature
and claiming it's a feature whether the kernel intends it or not.
Sure, use it all you want.  Just don't expect it to be there on the
next release.

>> Oh yeah, if you're not insisting fixed mapping, then I don't have any
>> problem with that.  As for what to do for priority inversions
>> involving workqueues, I don't have any concrete idea (it was not in
>> the mainline, so I didn't have to solve it) but one way would be
>> reserving/creating temporary high priority workers and use them to
>> process work items which the high priority thread is blocked on.
> 
> But it is in the mainline, that's why we're talking right now.

The problem is in the mainline.  Solution is not.  It's not a solved
problem.  It's also a pretty niche problem.

> What I was thinking is that you could have a debugfs interface which
> would list off what workqueues you system is processing and give the
> user the ability to pull one or more of those workqueues into individual
> threads for processing, just like it currently is. That way I can
> prioritize the work items with out you having to give priorities through
> your entire system.

Why would the user want to bother with all that?  Shouldn't the kernel
just do the right thing?  IIRC, people are working on priority
inheriting userland mutexes.  If your problem case can't be solved by
that, please try to fix it in proper way not through some random knobs
that nobody really can understand.

> The alternative is that you would give each work item a settable
> priority and your whole system would have to honor that, which would be
> a little like re-creating the scheduler.
>
>> But, really, without knowing details of those inversion cases, it
>> would be pretty difficult to design something which fits.  All that I
>> can say is having shared worker pool isn't exclusive with solving the
>> problem.
> 
> The cases are all in the mainline kernel, you just have to look at the
> code in a different way to understand them ..

Yeah, sure, all the problems in the world which haven't been solved
yet are there.  The _solution_ isn't there and solutions are what
people conserve when trying to improve things.

> If I have a userspace thread at a high priority and I'm making calls
> into the kernel, some of those call inevitably will put work items
> onto workqueues, right? I'm sure you can think of 100's of ways in
> which this could happen .. At that point my thread depends on the
> workqueue thread, since the workqueue thread is doing processing for
> which I've , in some way, requested from userspace.

You're basically saying that "I don't know how those inheritance
inversions are happening but if I turn these magic knobs they seem to
go away so I want those magic knobs".  Maybe the RT part of the code
shouldn't be depending on that many random things to begin with?  And
if there are actually things which are necessary, it's better idea to
solve it properly through identifying problem points and properly
inheriting priority instead of turning knobs until it somehow works?

If you wanna work on such things, be my guest.  I'll be happy to work
with you but please stop talking about setting priorities of
workqueues from userland.  That's just nuts.

Thanks.

-- 
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists