| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <aMGWAg5jIitYCXdZ@tiehlicka>
Date: Wed, 10 Sep 2025 17:15:14 +0200
From: Michal Hocko <mhocko@...e.com>
To: zhongjinji <zhongjinji@...or.com>
Cc: rientjes@...gle.com, shakeel.butt@...ux.dev, akpm@...ux-foundation.org,
tglx@...utronix.de, liam.howlett@...cle.com,
lorenzo.stoakes@...cle.com, surenb@...gle.com, lenb@...nel.org,
rafael@...nel.org, pavel@...nel.org, linux-mm@...ck.org,
linux-pm@...r.kernel.org, linux-kernel@...r.kernel.org,
liulu.liu@...or.com, feng.han@...or.com
Subject: Re: [PATCH v9 1/2] mm/oom_kill: Thaw the entire OOM victim process
On Wed 10-09-25 22:37:25, zhongjinji wrote:
> OOM killer is a mechanism that selects and kills processes when the system
> runs out of memory to reclaim resources and keep the system stable. But the
> oom victim cannot terminate on its own when it is frozen, even if the OOM
> victim task is thawed through __thaw_task(). This is because __thaw_task() can
> only thaw a single OOM victim thread, and cannot thaw the entire OOM victim
> process.
>
> Also, freezing_slow_path() decides whether a task is an OOM victim by checking
> the task's TIF_MEMDIE flag. When a task is thawed, the freezer bypasses PM
> freezing and cgroup freezing states to thaw it. But TIF_MEMDIE is not a thread
> group shared flag, and only one thread is marked with TIF_MEMDIE. If other
> threads are thawed, they may still remain frozen due to PM freezing and cgroup
> freezing states.
>
> To solve this, thaw_process() is introduced to thaw all threads of the victim,
> ensuring every thread in the victim process can be thawed. The freezer uses
> tsk_is_oom_victim() to determine whether a task is an OOM victim, because
> tsk->signal->oom_mm is data shared by all threads. This allows all victim threads
> to rely on it to be thawed.
A history detour for future reference.
TIF_MEMDIE was a "this is the oom victim & it has access to memory
reserves" flag in the past. It has that thread vs. process problems and
tsk_is_oom_victim was introduced later to get rid of them and other
issues as well as the guarantee that we can identify the oom victim's mm reliably
for other oom_reaper. I recommend reading git log of mm/oom_kill.c to
get hairy history of that area and how tricky it is due all the subtle
interaction with process exit paths etc.
>
> This change will thaw the entire victim process when OOM occurs,
> ensuring that the oom victim can terminate on its own.
>
> Signed-off-by: zhongjinji <zhongjinji@...or.com>
Acked-by: Michal Hocko <mhocko@...e.com>
Thanks!
>
> Acked-by: Michal Hocko <mhocko@...e.com>
> ---
> include/linux/freezer.h | 2 ++
> kernel/freezer.c | 20 +++++++++++++++++++-
> mm/oom_kill.c | 10 +++++-----
> 3 files changed, 26 insertions(+), 6 deletions(-)
>
> diff --git a/include/linux/freezer.h b/include/linux/freezer.h
> index b303472255be..32884c9721e5 100644
> --- a/include/linux/freezer.h
> +++ b/include/linux/freezer.h
> @@ -47,6 +47,7 @@ extern int freeze_processes(void);
> extern int freeze_kernel_threads(void);
> extern void thaw_processes(void);
> extern void thaw_kernel_threads(void);
> +extern void thaw_process(struct task_struct *p);
>
> static inline bool try_to_freeze(void)
> {
> @@ -80,6 +81,7 @@ static inline int freeze_processes(void) { return -ENOSYS; }
> static inline int freeze_kernel_threads(void) { return -ENOSYS; }
> static inline void thaw_processes(void) {}
> static inline void thaw_kernel_threads(void) {}
> +static inline void thaw_process(struct task_struct *p) {}
>
> static inline bool try_to_freeze(void) { return false; }
>
> diff --git a/kernel/freezer.c b/kernel/freezer.c
> index 6a96149aede9..ddc11a8bd2ea 100644
> --- a/kernel/freezer.c
> +++ b/kernel/freezer.c
> @@ -10,6 +10,7 @@
> #include <linux/export.h>
> #include <linux/syscalls.h>
> #include <linux/freezer.h>
> +#include <linux/oom.h>
> #include <linux/kthread.h>
>
> /* total number of freezing conditions in effect */
> @@ -40,7 +41,7 @@ bool freezing_slow_path(struct task_struct *p)
> if (p->flags & (PF_NOFREEZE | PF_SUSPEND_TASK))
> return false;
>
> - if (test_tsk_thread_flag(p, TIF_MEMDIE))
> + if (tsk_is_oom_victim(p))
> return false;
>
> if (pm_nosig_freezing || cgroup_freezing(p))
> @@ -206,6 +207,23 @@ void __thaw_task(struct task_struct *p)
> wake_up_state(p, TASK_FROZEN);
> }
>
> +/*
> + * thaw_process - Thaw a frozen process
> + * @p: the process to be thawed
> + *
> + * Iterate over all threads of @p and call __thaw_task() on each.
> + */
> +void thaw_process(struct task_struct *p)
> +{
> + struct task_struct *t;
> +
> + rcu_read_lock();
> + for_each_thread(p, t) {
> + __thaw_task(t);
> + }
> + rcu_read_unlock();
> +}
> +
> /**
> * set_freezable - make %current freezable
> *
> diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> index 25923cfec9c6..88356b66cc35 100644
> --- a/mm/oom_kill.c
> +++ b/mm/oom_kill.c
> @@ -772,12 +772,12 @@ static void mark_oom_victim(struct task_struct *tsk)
> mmgrab(tsk->signal->oom_mm);
>
> /*
> - * Make sure that the task is woken up from uninterruptible sleep
> - * if it is frozen because OOM killer wouldn't be able to free
> - * any memory and livelock. freezing_slow_path will tell the freezer
> - * that TIF_MEMDIE tasks should be ignored.
> + * Make sure that the process is woken up from uninterruptible sleep
> + * if it is frozen because OOM killer wouldn't be able to free any
> + * memory and livelock. The freezer will thaw the tasks that are OOM
> + * victims regardless of the PM freezing and cgroup freezing states.
> */
> - __thaw_task(tsk);
> + thaw_process(tsk);
> atomic_inc(&oom_victims);
> cred = get_task_cred(tsk);
> trace_mark_victim(tsk, cred->uid.val);
> --
> 2.17.1
--
Michal Hocko
SUSE Labs
Powered by blists - more mailing lists