linux-kernel - Re: [RFC][PATCH 17/18] sched: Move the second half of ttwu() to the remote cpu

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20110107152207.GA16341@redhat.com>
Date:	Fri, 7 Jan 2011 16:22:07 +0100
From:	Oleg Nesterov <oleg@...hat.com>
To:	Peter Zijlstra <a.p.zijlstra@...llo.nl>
Cc:	Chris Mason <chris.mason@...cle.com>,
	Frank Rowand <frank.rowand@...sony.com>,
	Ingo Molnar <mingo@...e.hu>,
	Thomas Gleixner <tglx@...utronix.de>,
	Mike Galbraith <efault@....de>, Paul Turner <pjt@...gle.com>,
	Jens Axboe <axboe@...nel.dk>,
	Yong Zhang <yong.zhang0@...il.com>,
	linux-kernel@...r.kernel.org
Subject: Re: [RFC][PATCH 17/18] sched: Move the second half of ttwu() to
	the remote cpu

On 01/06, Peter Zijlstra wrote:
>
> On Wed, 2011-01-05 at 22:07 +0100, Oleg Nesterov wrote:
>
> > I'll try to read it once again with the fresh head, though ;)
> > I also have a couple of very minor nits... In particular, perhaps
> > TASK_WAKING can die...
>
> I think it might.. I'll do a patch at the end removing it, lets see what
> happens.

Yes, ttwu can just set TASK_RUNNING. But, otoh, perhaps the
special state makes sense anyway, say, it can help to debug
the problems. We can even have TASK_WAKING_CONTRIBUTES_TO_LOAD
insetad of ->sched_contributes_to_load. But this all is very
minor.

A couple of questions...

Why sched_fork() does set_task_cpu() ? Just curious, it seems
that wake_up_new_task() does all we need.

ttwu_queue_remote() does "struct task_struct *next = NULL".
Probably "next = rq->wake_list" makes more sens. Otherwise the
first cmpxchg() always fails if rq->wake_list != NULL.

Doesn't __migrate_task() need pi_lock? Consider:

1. A task T runs on CPU_0, it does set_current_state(TASK_INTERRUBTIBLE)

2. some CPU does set_cpus_allowed_ptr(T, new_mask), new_mask doesn't
   include CPU_0.

   T is running, cpumask_any_and() picks CPU_1, set_cpus_allowed_ptr()
   drops pi_lock and rq->lock before stop_one_cpu().

3. T calls schedule() and becomes deactivated.

4. CPU_2 does try_to_wake_up(T, TASK_INTERRUPTIBLE), takes pi_lock
   and sees on_rq == F.

5. set_cpus_allowed_ptr() resumes and calls stop_one_cpu(cpu => 1).

6. cpu_stopper_thread() runs on CPU_1 and calls ____migrate_task().
   It locks CPU_0 and CPU_1 rq's and checks task_cpu() == src_cpu.

7. CPU_2 calls select_task_rq(), it returns (to simplify) 2.

   Now try_to_wake_up() does set_task_cpu(T, 2), and calls
   ttwu_queue()->ttwu_do_activate()->activate_task().

8. __migrate_task() on CPU_1 sees p->on_rq and starts the
   deactivate/activate dance racing with ttwu_do_activate()
   on CPU_2.

And a final question. This is really, really minor, but
activate_task/deactivate_task are not symmetric, the former
always sets p->on_rq. Looks correct, but imho a bit confusing and
can complicate the understanding. Since p->on_rq is cleared
explicitly by schedule(), perhaps it can be set explicitly to
in try_to_wake_up_*. Or, perhaps, activate/deactivate can check
ENQUEUE_WAKEUP/DEQUEUE_SLEEP and set/clear p->on_rq. Once again,
this is purely cosmetic issue.

Oleg.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/