linux-kernel - Re: Linux 2.6.38

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 14 Mar 2011 21:50:24 -0700 (PDT)
From:	David Rientjes <rientjes@...gle.com>
To:	Andrew Morton <akpm@...ux-foundation.org>
cc:	Linus Torvalds <torvalds@...ux-foundation.org>,
	linux-kernel@...r.kernel.org, Rik van Riel <riel@...hat.com>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	Oleg Nesterov <oleg@...hat.com>
Subject: Re: Linux 2.6.38

On Mon, 14 Mar 2011, Andrew Morton wrote:

> > once oom-prevent-unnecessary-oom-kills-or-kernel-panics.patch is merged 
> > from -mm
> 
> Please (re)send the patches which you believe should be merged into
> 2.6.38 to address the problems which Oleg found, and any other critical
> problems.  Not in a huge rush - let's get this right.
> 

In my testing, Oleg's three test cases that he sent to the security list 
and cc'd us on get appropriately oom killed once swap is exhausted or 
swapoff -a is used on mmotm-2011-03-10-16-42 because of these two patches:

	oom-prevent-unnecessary-oom-kills-or-kernel-panics.patch
	oom-skip-zombies-when-iterating-tasklist.patch

He also presented a test case on linux-mm that caused the oom killer to 
avoid acting if a thread is ptracing a thread in the exit path with 
PTRACE_O_TRACEEXIT.  That should be fixed with

	http://marc.info/?l=linux-mm&m=129997893430351

that has yet to see -mm.  There are no other test cases that have been 
presented that cause undesired behavior.

That said, my approach to doing this has been to avoid arbitrary 
heuristics for special cases and address known issues by adding the 
appropriate logic in the oom killer.  For example, the ptrace problem that 
Oleg presented showed that the oom killer logic incorrectly deferred doing 
anything when an eligible thread was PF_EXITING.  It had done that 
believing that nothing would stop the thread from exiting or current 
would be given access to memory reserves itself and that assumption was 
broken for PTRACE_O_TRACEEXIT.  My patch above, in combination with 
Andrey's patch that only considers threads with valid mm's, fixes that 
issue because we'll now only defer if you still have an attached mm, are 
PF_EXITING, and are not being traced.

If, at some point, there is another gap in the exit code where a thread 
may hold PF_EXITING with a valid mm for an indefinite period, we'll need 
to address that in the oom killer as well.  We use PF_EXITING specifically 
in the oom killer to identify tasks that are going to exit soon and need 
handling for any case where that isn't guaranteed.  Anything else results 
in needlessly killing other tasks or, in the worst case, panicking when 
there is nothing left that is eligible.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/