linux-kernel - [PATCH] mm: oom_kill_process: do not abort if the victim is exiting

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 24 May 2016 15:24:02 +0300
From:	Vladimir Davydov <vdavydov@...tuozzo.com>
To:	Andrew Morton <akpm@...ux-foundation.org>
CC:	Michal Hocko <mhocko@...nel.org>,
	Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>,
	David Rientjes <rientjes@...gle.com>,
	Johannes Weiner <hannes@...xchg.org>, <linux-mm@...ck.org>,
	<linux-kernel@...r.kernel.org>
Subject: [PATCH] mm: oom_kill_process: do not abort if the victim is exiting

After selecting an oom victim, we first check if it's already exiting
and if it is, we don't bother killing tasks sharing its mm. We do try to
reap its mm though, but we abort if any of the processes sharing it is
still alive. This might result in oom deadlock if an exiting task got
stuck trying to acquire a lock held by another task sharing the same mm
which needs memory to continue: if oom killer happens to keep selecting
the stuck task, we won't even try to kill other processes or reap the
mm.

The check in question was first introduced by commit 50ec3bbffbe8 ("oom:
handle current exiting"). Initially it worked in conjunction with
another check in select_bad_process() which forced selecting exiting
task. The goal of that patch was selecting the current task on oom if it
was exiting. Now, we select the current task if it's exiting or was
killed anyway. And the check in select_bad_process() was removed by
commit 6a618957ad17 ("mm: oom_kill: don't ignore oom score on exiting
tasks"), because it prevented oom reaper. This just makes the remaining
hunk in oom_kill_process pointless.

The only possible use of this check is avoiding a warning if oom killer
happens to select a dying process. This seems to be very unlikely,
because we should have spent a fair amount of time in reclaimer trying
to free some memory by the time we get to oom, so if the victim turns
out to be exiting, it is likely because of it is stuck on some lock. And
even if it is not, the check is racy anyway, because the warning is
still printed if the victim had managed to release its mm by the time we
entered oom_kill_process. So let's zap it to clean up the code.

Signed-off-by: Vladimir Davydov <vdavydov@...tuozzo.com>
---
 mm/oom_kill.c | 14 --------------
 1 file changed, 14 deletions(-)

diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index 03bf7a472296..66044153209e 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -747,20 +747,6 @@ void oom_kill_process(struct oom_control *oc, struct task_struct *p,
 					      DEFAULT_RATELIMIT_BURST);
 	bool can_oom_reap = true;

-	/*
-	 * If the task is already exiting, don't alarm the sysadmin or kill
-	 * its children or threads, just set TIF_MEMDIE so it can die quickly
-	 */
-	task_lock(p);
-	if (p->mm && task_will_free_mem(p)) {
-		mark_oom_victim(p);
-		try_oom_reaper(p);
-		task_unlock(p);
-		put_task_struct(p);
-		return;
-	}
-	task_unlock(p);
-
 	if (__ratelimit(&oom_rs))
 		dump_header(oc, p, memcg);

-- 
2.1.4