linux-kernel - [PATCH 3/6] mm,oom: exclude oom_task_origin processes if they are OOM victims.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 17 Feb 2016 19:32:00 +0900
From:	Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>
To:	mhocko@...nel.org, akpm@...ux-foundation.org
Cc:	rientjes@...gle.com, mgorman@...e.de, oleg@...hat.com,
	torvalds@...ux-foundation.org, hughd@...gle.com, andrea@...nel.org,
	riel@...hat.com, linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: [PATCH 3/6] mm,oom: exclude oom_task_origin processes if they are OOM victims.

>>From f5531e726caad7431020c027b6900a8e2c678345 Mon Sep 17 00:00:00 2001
From: Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>
Date: Wed, 17 Feb 2016 16:32:37 +0900
Subject: [PATCH 3/6] mm,oom: exclude oom_task_origin processes if they are OOM victims.

Currently, oom_scan_process_thread() returns OOM_SCAN_SELECT when there
is a thread which returns oom_task_origin() == true. But it is possible
that that thread is sharing memory with OOM-unkillable processes or the
OOM reaper fails to reclaim enough memory. In that case, we must not
continue selecting such threads forever.

This patch changes oom_scan_process_thread() not to select a thread
which returns oom_task_origin() = true if TIF_MEMDIE is already set
because SysRq-f case can reach here. Since "mm,oom: exclude TIF_MEMDIE
processes from candidates." made sure that we will choose a !TIF_MEMDIE
thread when only some of threads are marked TIF_MEMDIE, we don't need to
check all threads which returns oom_task_origin() == true.

Signed-off-by: Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>
---
 mm/oom_kill.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index a3868fd..b0c327d 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -308,7 +308,7 @@ enum oom_scan_t oom_scan_process_thread(struct oom_control *oc,
 	 * If task is allocating a lot of memory and has been marked to be
 	 * killed first if it triggers an oom, then select it.
 	 */
-	if (oom_task_origin(task))
+	if (oom_task_origin(task) && !test_tsk_thread_flag(task, TIF_MEMDIE))
 		return OOM_SCAN_SELECT;

 	return OOM_SCAN_OK;
-- 
1.8.3.1