linux-kernel - Re: Linux 2.6.38

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20110315210855.GI21640@redhat.com>
Date:	Tue, 15 Mar 2011 22:08:55 +0100
From:	Oleg Nesterov <oleg@...hat.com>
To:	David Rientjes <rientjes@...gle.com>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	linux-kernel@...r.kernel.org, Rik van Riel <riel@...hat.com>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>
Subject: Re: Linux 2.6.38

On 03/14, David Rientjes wrote:
>
> He also presented a test case on linux-mm that caused the oom killer to 
> avoid acting if a thread is ptracing a thread in the exit path with 
> PTRACE_O_TRACEEXIT.  That should be fixed with
>
> 	http://marc.info/?l=linux-mm&m=129997893430351

I don't think it can fix this. I didn't verify this, but the slightly
different test-case below should have the same effect.

But this doesn't matter. We can fix this particular case, and we have
the problems with the coredump anyway.

What I can't understand is what exactly the first patch tries to fix.
When I ask you, you tell me that for_each_process() can miss the group
leader because it can exit before sub-threads. This must not happen,
or we have some serious bug triggered by your workload.

So, once again. Could you please explain the original problem and how
this patch helps?

Oleg.

#include <unistd.h>
#include <signal.h>
#include <pthread.h>
#include <sys/ptrace.h>
#include <sys/wait.h>
#include <assert.h>
#include <stdio.h>

void *tfunc(void* arg)
{
	if (arg) {
		ptrace(PTRACE_TRACEME, 0,0,0);
		raise(SIGSTOP);
		pthread_kill(*(pthread_t*)arg, SIGQUIT);
	}
	pause();
}

int main(void)
{
	int pid;

	if (!fork()) {
		pthread_t thread1, thread2;
		pthread_create(&thread1, NULL, tfunc, NULL);
		pthread_create(&thread2, NULL, tfunc, &thread1);
		pause();
		return 0;
	}

	assert((pid = waitpid(-1, NULL, __WALL)) > 0);
	assert(ptrace(PTRACE_SETOPTIONS, pid, 0, PTRACE_O_TRACEEXIT) == 0);
	assert(ptrace(PTRACE_CONT, pid, 0, 0) == 0);
	wait(NULL);

	pause();
	return 0;
}

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/