linux-kernel - Re: [PATCH, RFC] reimplement flush

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20070104142936.GA179@tv-sign.ru>
Date:	Thu, 4 Jan 2007 17:29:36 +0300
From:	Oleg Nesterov <oleg@...sign.ru>
To:	Srivatsa Vaddagiri <vatsa@...ibm.com>
Cc:	Andrew Morton <akpm@...l.org>, David Howells <dhowells@...hat.com>,
	Christoph Hellwig <hch@...radead.org>,
	Ingo Molnar <mingo@...e.hu>,
	Linus Torvalds <torvalds@...l.org>,
	linux-kernel@...r.kernel.org, Gautham shenoy <ego@...ibm.com>
Subject: Re: [PATCH, RFC] reimplement flush_workqueue()

On 01/04, Srivatsa Vaddagiri wrote:
>
> On Tue, Dec 19, 2006 at 03:43:19AM +0300, Oleg Nesterov wrote:
> > > Taking workqueue_mutex() unconditionally in flush_workqueue() means
> > > that we'll deadlock if a single-threaded workqueue callback handler calls
> > > flush_workqueue().
> > 
> > Well. But flush_workqueue() drops workqueue_mutex before going to sleep ?
> 
> ... and acquires it again after woken from sleep. That can be a problem, which 
> will lead to the problem described here:
> 
> 	http://lkml.org/lkml/2006/12/7/374
> 
> In brief:
> 
> keventd thread					hotplug thread
> --------------					--------------
> 
>   run_workqueue()
> 	|
>      work_fn()
> 	 |
> 	flush_workqueue()
> 	     |	
> 	   flush_cpu_workqueue
> 		|				cpu_down()
> 	     mutex_unlock(wq_mutex);		     |
> 	(above opens window for hotplug)	   mutex_lock(wq_mutex);
>     		|				   /* bring down cpu */	
> 	     wait_for_completition();		     notifier(CPU_DEAD, ..)
> 		| 				       workqueue_cpu_callback
> 		| 				        cleanup_workqueue_thread
> 		|					  kthread_stop()
> 		|
> 		|
> 	     mutex_lock(wq_mutex); <- Can deadlock
> 
> 
> The kthread_stop() will wait for keventd() thread to exit, but keventd()
> is blocked on mutex_lock(wq_mutex) leading to a deadlock.

Thanks, I need to think about this.

However I am not sure I fully understand the problem.

First, this deadlock was not introduced by recent changes (including "single
threaded flush_workqueue() takes workqueue_mutex too"), yes?

Also, it seems to me we have a much more simple scenario for deadlock.

events/0 runs run_workqueue(), work->func() sleeps or takes a preemtion. CPU 0
dies, keventd thread migrates to another CPU. CPU_DEAD calls kthread_stop() under
workqueue_mutex and waits for until kevents thread exits. Now, if this work (or
another work pending on cwq->worklist) takes workqueue_mutex (for example, does
flush_workqueue) we have a deadlock.

No?

Oleg.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/