linux-kernel - Re: [patch 4/7 -mm] oom: badness heuristic rewrite

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 17 Feb 2010 01:23:35 -0800 (PST)
From:	David Rientjes <rientjes@...gle.com>
To:	Minchan Kim <minchan.kim@...il.com>
cc:	Rik van Riel <riel@...hat.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	Nick Piggin <npiggin@...e.de>,
	Andrea Arcangeli <aarcange@...hat.com>,
	Balbir Singh <balbir@...ux.vnet.ibm.com>,
	Lubos Lunak <l.lunak@...e.cz>, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org
Subject: Re: [patch 4/7 -mm] oom: badness heuristic rewrite

On Wed, 17 Feb 2010, Minchan Kim wrote:

> >> Okay. I can think it of slight penalization in this patch.
> >> But in current OOM logic, we try to kill child instead of forkbomb
> >> itself. My concern was that.
> >
> > We still do with my rewrite, that is handled in oom_kill_process().  The
> > forkbomb penalization takes place in badness().
> 
> 
> I thought this patch is closely related to [patch  2/7].
> I can move this discussion to [patch 2/7] if you want.
> Another guys already pointed out why we care child.
> 

We have _always_ tried to kill a child of the selected task first if it 
has a seperate address space, patch 2 doesn't change that.  It simply 
tries to kill the child with the highest badness() score.

> I said this scenario is BUGGY forkbomb process. It will fork + exec continuously
> if it isn't killed. How does user intervene to fix the system?
> System was almost hang due to unresponsive.
> 

The user would need to kill the parent if it should be killed.  The 
unresponsiveness in this example, however, is not a question of the oom 
killer but rather the scheduler to provide interactivity to the user in 
forkbomb scenarios.  The oom killer should not create a policy that 
unfairly biases tasks that fork a large number of tasks, however, to 
provide interactivity since that task may be a vital system resource.

> For extreme example,
> User is writing some important document by OpenOffice and
> he decided to execute hackbench 1000000 process 1000000.
> 
> Could user save his important office data without halt if we kill
> child continuously?
> I think this scenario can be happened enough if the user didn't know
> parameter of hackbench.
> 

So what exactly are you proposing we do in the oom killer to distinguish 
between a user's mistake and a vital system resource?  I'm personally much 
more concerned with protecting system daemons that provide a service under 
heavyload than protecting against forkbombs in the oom killer.