linux-kernel - Re: [BUG] TASK_DEAD task is able to be woken up in special condition

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-Id: <20120117174031.3118.E1E9C6FF@jp.fujitsu.com>
Date:	Tue, 17 Jan 2012 17:40:31 +0900
From:	Yasunori Goto <y-goto@...fujitsu.com>
To:	Peter Zijlstra <peterz@...radead.org>
Cc:	Oleg Nesterov <oleg@...hat.com>, Ingo Molnar <mingo@...e.hu>,
	Hiroyuki KAMEZAWA <kamezawa.hiroyu@...fujitsu.com>,
	Motohiro Kosaki <kosaki.motohiro@...fujitsu.com>,
	Linux Kernel ML <linux-kernel@...r.kernel.org>
Subject: Re: [BUG] TASK_DEAD task is able to be woken up in special condition

> On Mon, 2012-01-16 at 20:51 +0900, Yasunori Goto wrote:
> 
> > try_to_wake_up() has a problem which may change status from TASK_DEAD to
> > TASK_RUNNING in race condition with SMI or guest environment of virtual
> > machine. (See: https://lkml.org/lkml/2011/12/21/523)
> > As a result, exited task is scheduled() again and panic occurs.
> > 
> > By this patch, do_exit() waits for releasing task->pi_lock which is used
> > in try_to_wake_up(). It guarantees the task becomes TASK_DEAD after
> > waking up.
> 
> 
> This Changelog isn't very good. Please spell out the problem instead of
> referring to it so that people using git-blame and the like don't then
> have to go look up some (possibly non-existent) web-resource.

Ahh, Ok.

I rewrote the description and comment in patch.
Please check them.

-----
try_to_wake_up() has a problem which may change status from TASK_DEAD to
TASK_RUNNING in race condition with SMI or guest environment of virtual
machine. As a result, exited task is scheduled() again and panic occurs.

Here is the sequence how it occurs.

----------------------------------+-----------------------------
                                  |
           CPU A                  |             CPU B
----------------------------------+-----------------------------
TASK A calls exit()....

do_exit()

  exit_mm()
    down_read(mm->mmap_sem);
    
    rwsem_down_failed_common()

      set TASK_UNINTERRUPTIBLE
      set waiter.task <= task A
      list_add to sem->wait_list
           :
      raw_spin_unlock_irq()
      (I/O interruption occured)

                                      __rwsem_do_wake(mmap_sem)

                                        list_del(&waiter->list);
                                        waiter->task = NULL
                                        wake_up_process(task A)
                                          try_to_wake_up()
                                             (task is still
                                               TASK_UNINTERRUPTIBLE)
                                              p->on_rq is still 1.)

                                              ttwu_do_wakeup()
                                                 (*A)
                                                   :
     (I/O interruption handler finished)

      if (!waiter.task) 
          schedule() is not called
          due to waiter.task is NULL.
      
      tsk->state = TASK_RUNNING

          :
                                              check_preempt_curr();
                                                  :
  task->state = TASK_DEAD
                                              (*B)
                                        <---    set TASK_RUNNING (*C)



     schedule()
     (exit task is running again)
     BUG_ON() is called!
--------------------------------------------------------


The execution time between (*A) and (*B) is usually very short,
because the interruption is disabled, and setting TASK_RUNNING at (*C)
must be executed before setting TASK_DEAD.

HOWEVER, if SMI is interrupted between (*A) and (*B), 
(*C) is able to execute AFTER setting TASK_DEAD!
Then, exited task is scheduled again, and BUG_ON() is called....

If the system works on guest system of virtual machine, the time
between (*A) and (*B) may be also long due to scheduling of hypervisor,
and same phenomenon can occur.



By this patch, do_exit() waits for releasing task->pi_lock which is used
in try_to_wake_up(). It guarantees the task becomes TASK_DEAD after
waking up.


Signed-off-by: Yasunori Goto <y-goto@...fujitsu.com>

---
 kernel/exit.c |   16 ++++++++++++++++
 1 file changed, 16 insertions(+)

Index: linux-3.2/kernel/exit.c
===================================================================
--- linux-3.2.orig/kernel/exit.c
+++ linux-3.2/kernel/exit.c
@@ -1038,6 +1038,22 @@ NORET_TYPE void do_exit(long code)
 
 	preempt_disable();
 	exit_rcu();
+
+	/*
+	 * The setting of TASK_RUNNING by try_to_wake_up() may be delayed
+	 * when the following two conditions become true.
+	 *   - There is race condition of mmap_sem (It is acquired by
+	 *     exit_mm()), and
+	 *   - SMI occurs before setting TASK_RUNINNG.
+	 *     (or hypervisor of virtual machine switches to other guest)
+	 *  As a result, we may become TASK_RUNNING after becoming TASK_DEAD
+	 *
+	 * To avoid it, we have to wait for releasing tsk->pi_lock which
+	 * is held by try_to_wake_up()
+	 */
+	smp_mb();
+	raw_spin_unlock_wait(&tsk->pi_lock);
+
 	/* causes final put_task_struct in finish_task_switch(). */
 	tsk->state = TASK_DEAD;
 	schedule();
-----


-- 
Yasunori Goto 


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/