linux-kernel - [RFC 1/3] oom, sysrq: Skip over oom victims and killed tasks

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 12 Jan 2016 22:00:23 +0100
From:	Michal Hocko <mhocko@...nel.org>
To:	<linux-mm@...ck.org>
Cc:	Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>,
	David Rientjes <rientjes@...gle.com>,
	LKML <linux-kernel@...r.kernel.org>,
	Michal Hocko <mhocko@...e.com>
Subject: [RFC 1/3] oom, sysrq: Skip over oom victims and killed tasks

From: Michal Hocko <mhocko@...e.com>

When the OOM killer is issued by the administrator by sysrq+f it is
expected that a task will be killed to release the memory pressure.
Unlike the regular OOM killer the forced one doesn't abort when
there is an OOM victim selected. Instead oom_scan_process_thread
forces select_bad_process to check this thread. If this happens to be
the largest OOM hog then it will be selected again and the forced
OOM killer wouldn't make any change in case the current OOM victim
is not able terminate and free up resources it is sitting on.

This patch makes sure that the forced oom killer will skip over all
oom victims (with TIF_MEMDIE) and tasks with fatal_signal_pending
on basis that there is no guarantee those tasks are making progress
and there is no way to check they will ever make any. It is more
conservative to simply try another task.

Signed-off-by: Michal Hocko <mhocko@...e.com>
---
 mm/oom_kill.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index abefeeb42504..2b9dc5129a89 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -326,6 +326,17 @@ static struct task_struct *select_bad_process(struct oom_control *oc,
 		case OOM_SCAN_OK:
 			break;
 		};
+
+		/*
+		 * If we are doing sysrq+f then it doesn't make any sense to
+		 * check OOM victim or killed task because it might be stuck
+		 * and unable to terminate while the forced OOM might be the
+		 * only option left to get the system back to work.
+		 */
+		if (is_sysrq_oom(oc) && (test_tsk_thread_flag(p, TIF_MEMDIE) ||
+				fatal_signal_pending(p)))
+			continue;
+
 		points = oom_badness(p, NULL, oc->nodemask, totalpages);
 		if (!points || points < chosen_points)
 			continue;
-- 
2.6.4