linux-kernel - Re: 2.6.24 regression: deadlock on coredump of big process

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 29 Apr 2008 10:10:58 -0400
From:	Tony Battersby <tonyb@...ernetics.com>
To:	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
Cc:	linux-kernel@...r.kernel.org, linux-mm@...ck.org,
	Nick Piggin <npiggin@...e.de>
Subject: Re: 2.6.24 regression: deadlock on coredump of big process

KAMEZAWA Hiroyuki wrote:
> On Mon, 28 Apr 2008 11:11:46 -0400
> Tony Battersby <tonyb@...ernetics.com> wrote:
>
>   
>> Below is the program that triggers the deadlock; compile with
>> -D_REENTRANT -lpthread.
>>
>>     
> What happens if you changes size of stack (of pthreads) smaller ?
> (maybe ulimit -s will work also for threads.)
>
> Thanks,
> -Kame
>
>
>   

If I leave more memory free by changing the argument to
malloc_all_but_x_mb(), then I have to increase the number of threads
required to trigger the deadlock.  Changing the thread stack size via
setrlimit(RLIMIT_STACK) also changes the number of threads that are
required to trigger the deadlock.  For example, with
malloc_all_but_x_mb(16) and the default stack size of 8 MB, <= 5 threads
will coredump successfully, and >= 6 threads will deadlock.  With
malloc_all_but_x_mb(16) and a reduced stack size of 4096 bytes, <= 8
threads will coredump successfully, and >= 9 threads will deadlock.

Also note that the "free" command reports 10 MB free memory while the
program is running before the segfault is triggered.

Tony

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/