linux-kernel - PROBLEM:a bug about pi-futex maybe let the program going to hang

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Sat, 26 Mar 2011 12:19:03 +0800
From:	"scxby" <scxby@....com>
To:	<linux-kernel@...r.kernel.org>
Subject: PROBLEM:a bug about pi-futex maybe let the program going to hang

hi, all.

  Maybe, there is a bug about pi-futex, it would let the program in user-space going to hang.

  We have a board: CPU is powerpc 8572, two core. after ran one month, the state of pi-futex in user-space got bad: mutex->__data.__lock is 0x8000023e, mutex->__data.__count is 0, mutex->__data.__owner is 0.

  then, I review file "kernel/funtex.c"(the version is linux 2.6.38), found a case:

  if there are 3 thread, named threadA, threadB, threadC。thread A hold mutexM, threadB and threadC is waiting mutexM. They run as fllow steps:

  1. threadB and threadC sleep at line 1984.
  2. threadB receive a signal, then it will be wake up.
  3. threadA unlock mutexM, and give mutexM to threadB. 
  4. threadB call fixup_owner, try to give mutex to threadC.
  5. at line 1580, threadB trigger a addr-fault, then goto handle_fault.
  6. at line 1617, threadB release spinlock, then handle fault.
  7. threadC got spinlock, and call fixup_owner, and got mutexM.
  8. threadC give mutexM to threadB.
  9. threadB re-got spinlock, it will found "pi_state->owner == oldowner" and retry to fixup.
  10. threadB give mutexM to threadC, that's a bad thing.

  we have wrote a program, this program can prove all above.