linux-kernel - Re: workqueue code needing preemption disabled

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130318182130.GA3042@htj.dyndns.org>
Date:	Mon, 18 Mar 2013 11:21:30 -0700
From:	Tejun Heo <tj@...nel.org>
To:	Steven Rostedt <rostedt@...dmis.org>
Cc:	LKML <linux-kernel@...r.kernel.org>,
	RT <linux-rt-users@...r.kernel.org>,
	Clark Williams <clark@...hat.com>,
	Thomas Gleixner <tglx@...utronix.de>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>
Subject: Re: workqueue code needing preemption disabled

On Mon, Mar 18, 2013 at 01:08:07PM -0400, Steven Rostedt wrote:
> On Mon, 2013-03-18 at 09:43 -0700, Tejun Heo wrote:
> 
> > Making gcwq locks disable preemption would be much safer / easier, but
> > if that's not desirable, anything touching gcwq->idle_list would be a
> > good place to start - worker_enter_idle() and worker_leave_idle().
> > Hmmm... ignoring CPU hotplug, I think those two might just do it.
> > Give it a try?  How reproducible is the problem?
> 
> Not very :-( I triggered it twice on a 40 CPU box. It can go
> approximately 1 month before it triggers. And the box we are testing on
> is currently a loaner, and we have it on extension right now. Which
> means we wont have it much longer.
> 
> But perhaps that's the place to fix things.

I've been thinking about it and AFAICS the only way that BUG_ON()
could trigger from preemption is if preemption happens while the
idle_list head is becoming or stopping being empty.
ie. pool->worklist is half updated so list_empty() isn't true but the
first next entry is already pointing back to itself.  If there's a
crashdump, it shouldn't be too difficult to verify and wrapping the
above two functions should resolve it.

Thanks.

-- 
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/