linux-kernel - Re: sched: hang in migrate

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20140514102602.GJ30445@twins.programming.kicks-ass.net>
Date:	Wed, 14 May 2014 12:26:02 +0200
From:	Peter Zijlstra <peterz@...radead.org>
To:	Kirill Tkhai <tkhai@...dex.ru>
Cc:	Sasha Levin <sasha.levin@...cle.com>,
	Michael wang <wangyun@...ux.vnet.ibm.com>,
	"ktkhai@...allels.com" <ktkhai@...allels.com>,
	Ingo Molnar <mingo@...nel.org>,
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: sched: hang in migrate_swap

On Wed, May 14, 2014 at 02:21:04PM +0400, Kirill Tkhai wrote:
> 
> 
> 14.05.2014, 14:14, "Peter Zijlstra" <peterz@...radead.org>:
> > On Wed, May 14, 2014 at 01:42:32PM +0400, Kirill Tkhai wrote:
> >
> >>  Peter, do we have to queue stop works orderly?
> >>
> >>  Is there is not a possibility, when two pair of works queued different on
> >>  different cpus?
> >>
> >>   kernel/stop_machine.c | 10 ++++++++--
> >>   1 file changed, 8 insertions(+), 2 deletions(-)
> >>  diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c
> >>  index b6b67ec..29e221b 100644
> >>  --- a/kernel/stop_machine.c
> >>  +++ b/kernel/stop_machine.c
> >>  @@ -250,8 +250,14 @@ struct irq_cpu_stop_queue_work_info {
> >>   static void irq_cpu_stop_queue_work(void *arg)
> >>   {
> >>           struct irq_cpu_stop_queue_work_info *info = arg;
> >>  - cpu_stop_queue_work(info->cpu1, info->work1);
> >>  - cpu_stop_queue_work(info->cpu2, info->work2);
> >>  +
> >>  + if (info->cpu1 < info->cpu2) {
> >>  + cpu_stop_queue_work(info->cpu1, info->work1);
> >>  + cpu_stop_queue_work(info->cpu2, info->work2);
> >>  + } else {
> >>  + cpu_stop_queue_work(info->cpu2, info->work2);
> >>  + cpu_stop_queue_work(info->cpu1, info->work1);
> >>  + }
> >>   }
> >
> > I'm not sure, we already send the IPI to the first cpu of the pair, so
> > supposing we have 4 cpus, and get 4 pairs like:
> >
> > 0,1 1,2 2,3 3,0
> >
> > That would result in IPIs to 0, 1, 2, and 0 again, and since the IPI
> > function is serialized I don't immediately see a way for this to
> > deadlock.
> 
> It's about stop_two_cpus(), I have a distrust about other users of stop task:
> 
> queue_stop_cpus_work() queues work consequentially:
> 
> 0 1 2 4
> 
> stop_two_cpus() may queue:
> 
> 1 0
> 
> Looks like, stop thread on 0th and on 1th are waiting for wrong works.

so we serialize stop_cpus_work() vs stop_two_cpus() with an l/g lock.

Ah, but stop_cpus_work() only holds the global lock over queueing, it
doesn't wait for completion, that might indeed cause a problem.

Also, since its two different cpus queueing, the ordered queue doesn't
really matter, you can still interleave the all and two sets and get
into this state.

Content of type "application/pgp-signature" skipped