linux-kernel - [PATCH] fix OOM killing of swapoff

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <Pine.LNX.4.64.0701052031260.3944@blonde.wat.veritas.com>
Date:	Fri, 5 Jan 2007 20:35:13 +0000 (GMT)
From:	Hugh Dickins <hugh@...itas.com>
To:	Andrew Morton <akpm@...l.org>
cc:	Nick Piggin <npiggin@...e.de>, linux-kernel@...r.kernel.org
Subject: [PATCH] fix OOM killing of swapoff

These days, if you swapoff when there isn't enough memory, OOM killer
gives "BUG: scheduling while atomic" and the machine hangs: badness()
needs to do its PF_SWAPOFF return after the task_unlock (tasklist_lock
is also held here, so p isn't going to be freed: PF_SWAPOFF might get
turned off at any moment, but that doesn't really matter).

Signed-off-by: Hugh Dickins <hugh@...itas.com>
---
Will be needed in 2.6.19-stable too.

 mm/oom_kill.c |   12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

--- 2.6.20-rc3/mm/oom_kill.c	2007-01-01 10:30:46.000000000 +0000
+++ linux/mm/oom_kill.c	2007-01-05 20:17:21.000000000 +0000
@@ -61,12 +61,6 @@ unsigned long badness(struct task_struct
 	}
 
 	/*
-	 * swapoff can easily use up all memory, so kill those first.
-	 */
-	if (p->flags & PF_SWAPOFF)
-		return ULONG_MAX;
-
-	/*
 	 * The memory size of the process is the basis for the badness.
 	 */
 	points = mm->total_vm;
@@ -77,6 +71,12 @@ unsigned long badness(struct task_struct
 	task_unlock(p);
 
 	/*
+	 * swapoff can easily use up all memory, so kill those first.
+	 */
+	if (p->flags & PF_SWAPOFF)
+		return ULONG_MAX;
+
+	/*
 	 * Processes which fork a lot of child processes are likely
 	 * a good choice. We add half the vmsize of the children if they
 	 * have an own mm. This prevents forking servers to flood the
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/