linux-kernel - [Patch] freezer: check OOM kill signal while being frozen

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Fri,  8 Aug 2014 17:46:38 -0700
From:	Cong Wang <xiyou.wangcong@...il.com>
To:	linux-kernel@...r.kernel.org
Cc:	David Rientjes <rientjes@...gle.com>,
	Michal Hocko <mhocko@...e.cz>,
	"Rafael J. Wysocki" <rjw@...ysocki.net>, Tejun Heo <tj@...nel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Cong Wang <xiyou.wangcong@...il.com>
Subject: [Patch] freezer: check OOM kill signal while being frozen

There is a race condition between OOM killer and freezer when
they try to operate on the same process, something like below:

	Process A	Process B		Process C
trigger oom
B=oom_scan_process_thread()
					cgroup_freezer_freeze(B)
			...
			try_to_freeze()
			stay in D state
oom_kill_process(B)

In this case, process A triggers OOM and kernel selects process B
as the victim, right before being killed process B was frozen by
process C therefore went to D state, then kernel sent SIGKILL but
it is already too late as process B will never care about pending
signals any more. Fix this straightly by checking fatal pending signal
from OOM killer, so that the frozen process will recover itself
and then be killed finally.

Cc: David Rientjes <rientjes@...gle.com>
Cc: Michal Hocko <mhocko@...e.cz>
Cc: "Rafael J. Wysocki" <rjw@...ysocki.net>
Cc: Tejun Heo <tj@...nel.org>
Cc: Andrew Morton <akpm@...ux-foundation.org>
Signed-off-by: Cong Wang <xiyou.wangcong@...il.com>
---
diff --git a/kernel/freezer.c b/kernel/freezer.c
index aa6a8aa..c6d189d 100644
--- a/kernel/freezer.c
+++ b/kernel/freezer.c
@@ -68,7 +68,9 @@ bool __refrigerator(bool check_kthr_stop)
 		spin_lock_irq(&freezer_lock);
 		current->flags |= PF_FROZEN;
 		if (!freezing(current) ||
-		    (check_kthr_stop && kthread_should_stop()))
+		    (check_kthr_stop && kthread_should_stop()) ||
+		    (test_tsk_thread_flag(current, TIF_MEMDIE) &&
+			 fatal_signal_pending(current)))
 			current->flags &= ~PF_FROZEN;
 		spin_unlock_irq(&freezer_lock);
 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/