linux-kernel - Re: [PATCH 5/8] cpu: cpu-hotplug deadlock

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1209490317.6433.30.camel@lappy>
Date:	Tue, 29 Apr 2008 19:31:56 +0200
From:	Peter Zijlstra <a.p.zijlstra@...llo.nl>
To:	Oleg Nesterov <oleg@...sign.ru>
Cc:	Gautham R Shenoy <ego@...ibm.com>, linux-kernel@...r.kernel.org,
	Zdenek Kabelac <zdenek.kabelac@...il.com>,
	Heiko Carstens <heiko.carstens@...ibm.com>,
	"Rafael J. Wysocki" <rjw@...k.pl>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Ingo Molnar <mingo@...e.hu>,
	Srivatsa Vaddagiri <vatsa@...ibm.com>
Subject: Re: [PATCH 5/8] cpu: cpu-hotplug deadlock

On Tue, 2008-04-29 at 20:45 +0400, Oleg Nesterov wrote:
> On 04/29, Peter Zijlstra wrote:
> >
> > The only thing that changed is that the mutex is not held; so what we
> > change is:
> > 
> >  LOCK
> > 
> >  ... do the full hotplug thing ...
> > 
> >  UNLOCK
> > 
> > into
> > 
> >  LOCK
> >   set state
> >  UNLOCK
> > 
> >  ... do the full hotplug thing ...
> > 
> >  LOCK
> >   unset state
> >  UNLOCK
> > 
> > So that the lock isn't held over the hotplug operation.
> 
> Well, yes I see, but... Ugh, I have a a blind spot here ;)
> 
> why this makes any difference from the semantics POV ? why it is bad
> to hold the mutex throughout the "full hotplug thing" ?

Darn, now you make me think ;-)

Ok, I think I have it; the crux of the matter is that we want
reader-in-writer recursion for the cpu hotplug lock.

So we want:

 cpu_hotplug.write_lock()
   A.lock()
     cpu_hotplug.read_lock()

When - as it was - the write lock is implemented as keeping the lock
internal lock (the lock guarding the lock state) locked over the entire
write section, and the read lock side is, LOCK; change state; UNLOCK,
the above will result in a deadlock like:

  C.lock
    A.lock
      C.lock

By making both the read and write side work like:

 LOCK
  change state
 UNLOCK

the internal lock will not deadlock.

So what I did was promote cpu_hotplug to a full lock that handled
read-in-read and read-in-write recursion and made cpu_hotplug.lock the
lock internal lock.

> > > (actually, since write-locks should be very rare, perhaps we don't need
> > >  2 wait_queues ?)
> > 
> > And just let them race the wakeup race, sure that might work. Gautham
> > even pointed out that it never happens because there is another
> > exclusive lock on the write path.
> > 
> > But you say you like that it doesn't depend on that anymore - me too ;-)
> 
> Yes. but let's suppose we have the single wait_queue, this doesn't make
> any difference from the correctness POV, no?
> 
> To clarify: I am not arguing! this makes sense, but I'm asking to be sure
> I didn't miss a subtle reason why do we "really" need 2 wait_queues.
> 
> Also. Let's suppose we have both read- and write- waiters, and cpu_hotplug_done()
> does wake_up(writer_queue). It is possible that another reader comes and does
> get_online_cpus() and increments .refcount first. After that, cpu_hotplug
> is "opened" for the read-lock, but other read-waiters continue to sleep, and
> the final put_online_cpus() wakes up write-waiters only. Yes, this all is
> correct, but not "symmetrical", and leads to the question "do we really need
> 2 wait_queues" again.

I don't think we do. It just didn't occur to me to pile read-waiters and
write-waiters on the same waitqueue.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/