linux-kernel - Weird hang with NPTL and SIGPROF.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [day] [month] [year] [list]

Date:	Wed, 06 Feb 2008 16:37:31 -0800
From:	Frank Mayhar <fmayhar@...gle.com>
To:	linux-kernel <linux-kernel@...r.kernel.org>
Subject: Weird hang with NPTL and SIGPROF.

I have a testcase that demonstrates a strange hang of the latest kernel
(as well as previous ones).  In the process of investigating the NPTL,
we wrote a test that just creates a bunch of threads, then does a
barrier wait to synchronize them all, after which everybody exits.
That's all it does.

This works fine under most circumstances.  Unfortunately, we also want
to do profiling, so we catch SIGPROF and turn on ITIMER_PROF.  In this
case, at somewhere between 4000 and 4500 threads, and using the NPTL,
the system hangs.  It's not a hard hang, interrupts are still working
and clocks are ticking, but nothing is making progress.  It becomes
noticeable when the softlockup_tick() warning goes off after the
watchdog has been starved long enough.

Sometimes the system recovers and gets going again.  Other times it
doesn't.  I've examined the state of things several times with kdb and
there's certainly nothing obvious going on.  Something, perhaps having
to do with the scheduler, is certainly getting into a bad state, but I
haven't yet been able to figure out what that is.  I've even run it with
KFT and have seen nothing obvious there, either, except for the fact
that when it hangs it becomes obvious that it stops making progress and
it begins to fill up with smp_apic_timer_interrupt() and do_softirq()
entries.  I've also seen smp_apic_timer_interrupt() appear twice or more
on the stack, as if the previous run(s) didn't finish before the next
tick happened.

Any hints would be very much appreciated.  If you want to try to
reproduce it yourself, I've attached the testcase; I built it (under
Ubuntu) with
	gcc -D_GNU_SOURCE -c hangc-2.c -o hangc-2.o
	gcc -lpthread -o hangc-2 hangc-2.o

Note that I've also opened a Bugzilla bug for this issue, with the same
information and testcase, at
	http://bugzilla.kernel.org/show_bug.cgi?id=9906
-- 
Frank Mayhar <fmayhar@...gle.com>
Google, Inc.

View attachment "hangc-2.c" of type "text/x-csrc" (2665 bytes)