linux-kernel - Re: Crash in schedule path after worker thread dies

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20110720170717.GF9924@mtj.dyndns.org>
Date:	Wed, 20 Jul 2011 19:07:17 +0200
From:	Tejun Heo <tj@...nel.org>
To:	Todd Poynor <toddpoynor@...gle.com>
Cc:	linux-kernel@...r.kernel.org
Subject: Re: Crash in schedule path after worker thread dies

Hello, Todd.

On Tue, Jul 19, 2011 at 05:07:38PM -0700, Todd Poynor wrote:
> After a worker thread died due to a bug in a work function, a NULL
> dereference was seen when schedule calls wq_worker_sleeping calls
> kthread_data:
> 
>           return to_kthread(task)->data;
> 
> mm_release has apparently already set the task's vfork_done = NULL,
> causing to_kthread to return a bad address (on 3.0-rc7 on ARM).
> 
> I haven't tried a fix because I'm not sure if avoiding this case is
> enough to properly recover from death of a worker thread, or if this
> has already been discussed and rejected in the past.  I searched
> around a little and found some mentions of problems in worker
> functions that were probably followed by the kthread_data crash,
> but didn't turn up any specific discussion of this crash.  So I
> thought I'd start by mentioning this here, and can help fix or test if
> needed.

Hmmm... the scheduler code is oopsing after the worker already oopsed,
escalating the failure mode, right?  Avoiding the scheduler oops can
be done simply by clearing PF_WQ_WORKER in do_exit(); however, it
would still leave workqueue code assuming the worker is busy running
and may lead to workqueue stall.

Cleaning that up would require more code but at that point it isn't
really clear what we would be buying with the added complexity.
Hmmm... maybe it would still be worthwhile to at least keep scheduler
working.  Something like this?

diff --git a/kernel/exit.c b/kernel/exit.c
index 6c7fbbe..412ead0 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -915,6 +915,7 @@ NORET_TYPE void do_exit(long code)
 	 * kernel address.
 	 */
 	set_fs(USER_DS);
+	tsk->flags &= ~PF_WQ_WORKER;
 
 	ptrace_event(PTRACE_EVENT_EXIT, code);
 
-- 
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/