linux-kernel - [PATCH 1/1] signal: on exit skip waiting for an ack from the tracer if it is frozen

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20240628173247.3507846-1-surenb@google.com>
Date: Fri, 28 Jun 2024 10:32:47 -0700
From: Suren Baghdasaryan <surenb@...gle.com>
To: akpm@...ux-foundation.org
Cc: oleg@...hat.com, mhocko@...e.com, brauner@...nel.org, 
	tandersen@...flix.com, bigeasy@...utronix.de, vincent.whitchurch@...s.com, 
	ardb@...nel.org, linux-kernel@...r.kernel.org, 
	Suren Baghdasaryan <surenb@...gle.com>, Martin Liu <liumartin@...gle.com>, 
	Minchan Kim <minchan@...gle.com>
Subject: [PATCH 1/1] signal: on exit skip waiting for an ack from the tracer
 if it is frozen

When a process is being killed or exiting and it has a tracer, it will
notify the tracer and wait for an ack from the tracer to proceed. However
if the tracer is frozen, this ack will not arrive until the tracer gets
thawed. This poses a problem especially during memory pressure because
resources of the process are not released.
Things become even more interesting if OOM killer picks such tracee
and adds it into oom_victims. oom_victims counter will get incremented
and stay that way until tracee exits. In the meantime, if the system
tries to go into suspend, it will call oom_killer_disable() after
freezing all processes. That call will fail due to positive oom_victims,
but not until freeze_timeout_msecs passes. For the whole duration of the
freeze_timeout_msecs (20sec by default) the system will appear
unresponsive.
To fix this problem, skip the ack waiting step in the tracee when it's
exiting and the tracer is frozen. Per ptrace(2) manual, the tracer
cannot assume that the ptrace-stopped tracee exists. Therefore this
change does not break any valid assumptions.

Debugged-by: Martin Liu <liumartin@...gle.com>
Debugged-by: Minchan Kim <minchan@...gle.com>
Signed-off-by: Suren Baghdasaryan <surenb@...gle.com>
---
 kernel/signal.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/kernel/signal.c b/kernel/signal.c
index 1f9dd41c04be..dd9c18fdaaa5 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -2320,6 +2320,19 @@ static int ptrace_stop(int exit_code, int why, unsigned long message,
 	if (gstop_done && (!current->ptrace || ptrace_reparented(current)))
 		do_notify_parent_cldstop(current, false, why);

+	/*
+	 * If tracer is frozen, it won't ack until it gets unfrozen and if the
+	 * tracee is exiting this means its resources do not get freed until
+	 * the tracer is thawed. Skip waiting for the tracer. Per ptrace(2)
+	 * manual, the tracer cannot assume that the ptrace-stopped tracee
+	 * exists, so exiting now should not be an issue.
+	 */
+	if (current->ptrace && (exit_code >> 8) == PTRACE_EVENT_EXIT &&
+	    cgroup_task_frozen(current->parent)) {
+		read_unlock(&tasklist_lock);
+		goto skip_wait;
+	}
+
 	/*
 	 * The previous do_notify_parent_cldstop() invocation woke ptracer.
 	 * One a PREEMPTION kernel this can result in preemption requirement
@@ -2356,6 +2369,7 @@ static int ptrace_stop(int exit_code, int why, unsigned long message,
 	schedule();
 	cgroup_leave_frozen(true);

+skip_wait:
 	/*
 	 * We are back.  Now reacquire the siglock before touching
 	 * last_siginfo, so that we are sure to have synchronized with

base-commit: 6c0483dbfe7223f2b8390e3d5fe942629d3317b7
-- 
2.45.2.803.g4e1b14247a-goog