linux-kernel - [PATCH] mm/page_alloc: bail out on fatal signal during reclaim/compaction retry attempt

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20210519145014.3220164-1-atomlin@redhat.com>
Date:   Wed, 19 May 2021 15:50:14 +0100
From:   Aaron Tomlin <atomlin@...hat.com>
To:     linux-mm@...ck.org
Cc:     akpm@...ux-foundation.org, vbabka@...e.cz, mhocko@...e.com,
        linux-kernel@...r.kernel.org
Subject: [PATCH] mm/page_alloc: bail out on fatal signal during reclaim/compaction retry attempt

It does not make sense to retry compaction when the last known compact
result was skipped and a fatal signal is pending.

In the context of try_to_compact_pages(), indeed COMPACT_SKIPPED can be
returned; albeit, not every zone, on the zone list, would be considered
in the case a fatal signal is found to be pending.
Yet, in should_compact_retry(), given the last known compaction result,
each zone, on the zone list, can be considered/or checked
(see compaction_zonelist_suitable()). For example, if a zone was found
to succeed, then reclaim/compaction would be tried again
(notwithstanding the above).

This patch ensures that compaction is not needlessly retried when the
last known compaction result was skipped and in the unlikely case a
fatal signal is found pending. So, OOM is at least attempted.

Signed-off-by: Aaron Tomlin <atomlin@...hat.com>
---
 mm/page_alloc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index aaa1655cf682..5f9aac27a1b5 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -4268,6 +4268,8 @@ should_compact_retry(struct alloc_context *ac, int order, int alloc_flags,
 	 * to work with, so we retry only if it looks like reclaim can help.
 	 */
 	if (compaction_needs_reclaim(compact_result)) {
+		if (fatal_signal_pending(current))
+			goto out;
 		ret = compaction_zonelist_suitable(ac, order, alloc_flags);
 		goto out;
 	}
-- 
2.26.3