linux-kernel - Re: [PATCH] mm,oom: Exclude TIF_MEMDIE processes from candidates.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 8 Jan 2016 22:14:54 +0900
From:	Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>
To:	mhocko@...nel.org, hannes@...xchg.org
Cc:	akpm@...ux-foundation.org, mgorman@...e.de, rientjes@...gle.com,
	torvalds@...ux-foundation.org, oleg@...hat.com, hughd@...gle.com,
	andrea@...nel.org, riel@...hat.com, linux-mm@...ck.org,
	linux-kernel@...r.kernel.org
Subject: Re: [PATCH] mm,oom: Exclude TIF_MEMDIE processes from candidates.

Michal Hocko wrote:
> On Thu 07-01-16 11:28:15, Johannes Weiner wrote:
> > On Tue, Dec 29, 2015 at 10:58:22PM +0900, Tetsuo Handa wrote:
> > > >From 8bb9e36891a803e82c589ef78077838026ce0f7d Mon Sep 17 00:00:00 2001
> > > From: Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>
> > > Date: Tue, 29 Dec 2015 22:20:58 +0900
> > > Subject: [PATCH] mm,oom: Exclude TIF_MEMDIE processes from candidates.
> > > 
> > > The OOM reaper kernel thread can reclaim OOM victim's memory before the victim
> > > terminates. But since oom_kill_process() tries to kill children of the memory
> > > hog process first, the OOM reaper can not reclaim enough memory for terminating
> > > the victim if the victim is consuming little memory. The result is OOM livelock
> > > as usual, for timeout based next OOM victim selection is not implemented.
> > 
> > What we should be doing is have the OOM reaper clear TIF_MEMDIE after
> > it's done. There is no reason to wait for and prioritize the exit of a
> > task that doesn't even have memory anymore. Once a task's memory has
> > been reaped, subsequent OOM invocations should evaluate anew the most
> > desirable OOM victim.
> 
> This is an interesting idea. It definitely sounds better than timeout
> based solutions. I will cook up a patch for this. The API between oom
> killer and the reaper has to change slightly but that shouldn't be a big
> deal.

That is part of what I suggested at
http://lkml.kernel.org/r/201512052133.IAE00551.LSOQFtMFFVOHOJ@I-love.SAKURA.ne.jp .
| What about marking current OOM victim unkillable by updating
| victim->signal->oom_score_adj to OOM_SCORE_ADJ_MIN and clearing victim's
| TIF_MEMDIE flag when the victim is still alive for a second after
| oom_reap_vmas() completed?

Can we update victim's oom_score_adj as well? Otherwise, the OOM killer
might choose the same victim if victim's oom_score_adj was set to 1000.