| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <jnvm77cg4egdxjcwn3kto6zrl4yqtlxf5bkzoy2ndhldu7vrwk@tewpuse6yp4g>
Date: Tue, 16 Sep 2025 09:42:29 -0400
From: "Liam R. Howlett" <Liam.Howlett@...cle.com>
To: zhongjinji <zhongjinji@...or.com>
Cc: mhocko@...e.com, rientjes@...gle.com, shakeel.butt@...ux.dev,
akpm@...ux-foundation.org, tglx@...utronix.de,
lorenzo.stoakes@...cle.com, surenb@...gle.com, lenb@...nel.org,
rafael@...nel.org, pavel@...nel.org, linux-mm@...ck.org,
linux-pm@...r.kernel.org, linux-kernel@...r.kernel.org,
liulu.liu@...or.com, feng.han@...or.com
Subject: Re: [PATCH v10 1/2] mm/oom_kill: Thaw the entire OOM victim process
* zhongjinji <zhongjinji@...or.com> [250915 12:30]:
> OOM killer is a mechanism that selects and kills processes when the system
> runs out of memory to reclaim resources and keep the system stable. But the
> oom victim cannot terminate on its own when it is frozen, even if the OOM
> victim task is thawed through __thaw_task(). This is because __thaw_task() can
> only thaw a single OOM victim thread, and cannot thaw the entire OOM victim
> process.
>
> In addition, freezing_slow_path() determines whether a task is an OOM victim
> by checking the task’s TIF_MEMDIE flag. When a task is identified as an OOM
> victim, the freezer bypasses both PM freezing and cgroup freezing states to
> thaw it.
>
> Historically, TIF_MEMDIE was a "this is the oom victim & it has access to
> memory reserves" flag in the past. It has that thread vs. process problems
> and tsk_is_oom_victim was introduced later to get rid of them and other
> issues as well as the guarantee that we can identify the oom victim's mm
> reliably for other oom_reaper.
>
> Therefore, thaw_process() is introduced to unfreeze all threads within the
> OOM victim process, ensuring that every thread is properly thawed. The freezer
> now uses tsk_is_oom_victim() to determine OOM victim status, allowing all
> victim threads to be unfrozen as necessary.
>
> With this change, the entire OOM victim process will be thawed when an OOM
> event occurs, ensuring that the victim can terminate on its own.
>
> Signed-off-by: zhongjinji <zhongjinji@...or.com>
> Reviewed-by: Suren Baghdasaryan <surenb@...gle.com>
>
> Acked-by: Shakeel Butt <shakeel.butt@...ux.dev>
> Acked-by: Michal Hocko <mhocko@...e.com>
Acked-by: Liam R. Howlett <Liam.Howlett@...cle.com>
> ---
> include/linux/freezer.h | 2 ++
> kernel/freezer.c | 20 +++++++++++++++++++-
> mm/oom_kill.c | 10 +++++-----
> 3 files changed, 26 insertions(+), 6 deletions(-)
>
> diff --git a/include/linux/freezer.h b/include/linux/freezer.h
> index b303472255be..32884c9721e5 100644
> --- a/include/linux/freezer.h
> +++ b/include/linux/freezer.h
> @@ -47,6 +47,7 @@ extern int freeze_processes(void);
> extern int freeze_kernel_threads(void);
> extern void thaw_processes(void);
> extern void thaw_kernel_threads(void);
> +extern void thaw_process(struct task_struct *p);
>
> static inline bool try_to_freeze(void)
> {
> @@ -80,6 +81,7 @@ static inline int freeze_processes(void) { return -ENOSYS; }
> static inline int freeze_kernel_threads(void) { return -ENOSYS; }
> static inline void thaw_processes(void) {}
> static inline void thaw_kernel_threads(void) {}
> +static inline void thaw_process(struct task_struct *p) {}
>
> static inline bool try_to_freeze(void) { return false; }
>
> diff --git a/kernel/freezer.c b/kernel/freezer.c
> index 6a96149aede9..ddc11a8bd2ea 100644
> --- a/kernel/freezer.c
> +++ b/kernel/freezer.c
> @@ -10,6 +10,7 @@
> #include <linux/export.h>
> #include <linux/syscalls.h>
> #include <linux/freezer.h>
> +#include <linux/oom.h>
> #include <linux/kthread.h>
>
> /* total number of freezing conditions in effect */
> @@ -40,7 +41,7 @@ bool freezing_slow_path(struct task_struct *p)
> if (p->flags & (PF_NOFREEZE | PF_SUSPEND_TASK))
> return false;
>
> - if (test_tsk_thread_flag(p, TIF_MEMDIE))
> + if (tsk_is_oom_victim(p))
> return false;
>
> if (pm_nosig_freezing || cgroup_freezing(p))
> @@ -206,6 +207,23 @@ void __thaw_task(struct task_struct *p)
> wake_up_state(p, TASK_FROZEN);
> }
>
> +/*
> + * thaw_process - Thaw a frozen process
> + * @p: the process to be thawed
> + *
> + * Iterate over all threads of @p and call __thaw_task() on each.
> + */
> +void thaw_process(struct task_struct *p)
> +{
> + struct task_struct *t;
> +
> + rcu_read_lock();
> + for_each_thread(p, t) {
> + __thaw_task(t);
> + }
> + rcu_read_unlock();
> +}
> +
> /**
> * set_freezable - make %current freezable
> *
> diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> index 25923cfec9c6..88356b66cc35 100644
> --- a/mm/oom_kill.c
> +++ b/mm/oom_kill.c
> @@ -772,12 +772,12 @@ static void mark_oom_victim(struct task_struct *tsk)
> mmgrab(tsk->signal->oom_mm);
>
> /*
> - * Make sure that the task is woken up from uninterruptible sleep
> - * if it is frozen because OOM killer wouldn't be able to free
> - * any memory and livelock. freezing_slow_path will tell the freezer
> - * that TIF_MEMDIE tasks should be ignored.
> + * Make sure that the process is woken up from uninterruptible sleep
> + * if it is frozen because OOM killer wouldn't be able to free any
> + * memory and livelock. The freezer will thaw the tasks that are OOM
> + * victims regardless of the PM freezing and cgroup freezing states.
> */
> - __thaw_task(tsk);
> + thaw_process(tsk);
> atomic_inc(&oom_victims);
> cred = get_task_cred(tsk);
> trace_mark_victim(tsk, cred->uid.val);
> --
> 2.17.1
>
Powered by blists - more mailing lists