| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CAJuCfpEu_pq5kDgQ2jDnWVNuzVPrMKz2Gw0EeoPU6WD38aBXVA@mail.gmail.com>
Date: Wed, 10 Sep 2025 08:23:56 -0700
From: Suren Baghdasaryan <surenb@...gle.com>
To: Michal Hocko <mhocko@...e.com>
Cc: zhongjinji <zhongjinji@...or.com>, rientjes@...gle.com, shakeel.butt@...ux.dev,
akpm@...ux-foundation.org, tglx@...utronix.de, liam.howlett@...cle.com,
lorenzo.stoakes@...cle.com, lenb@...nel.org, rafael@...nel.org,
pavel@...nel.org, linux-mm@...ck.org, linux-pm@...r.kernel.org,
linux-kernel@...r.kernel.org, liulu.liu@...or.com, feng.han@...or.com
Subject: Re: [PATCH v9 1/2] mm/oom_kill: Thaw the entire OOM victim process
On Wed, Sep 10, 2025 at 8:15 AM Michal Hocko <mhocko@...e.com> wrote:
>
> On Wed 10-09-25 22:37:25, zhongjinji wrote:
> > OOM killer is a mechanism that selects and kills processes when the system
> > runs out of memory to reclaim resources and keep the system stable. But the
> > oom victim cannot terminate on its own when it is frozen, even if the OOM
> > victim task is thawed through __thaw_task(). This is because __thaw_task() can
> > only thaw a single OOM victim thread, and cannot thaw the entire OOM victim
> > process.
> >
> > Also, freezing_slow_path() decides whether a task is an OOM victim by checking
> > the task's TIF_MEMDIE flag. When a task is thawed, the freezer bypasses PM
> > freezing and cgroup freezing states to thaw it. But TIF_MEMDIE is not a thread
> > group shared flag, and only one thread is marked with TIF_MEMDIE. If other
> > threads are thawed, they may still remain frozen due to PM freezing and cgroup
> > freezing states.
> >
> > To solve this, thaw_process() is introduced to thaw all threads of the victim,
> > ensuring every thread in the victim process can be thawed. The freezer uses
> > tsk_is_oom_victim() to determine whether a task is an OOM victim, because
> > tsk->signal->oom_mm is data shared by all threads. This allows all victim threads
> > to rely on it to be thawed.
>
> A history detour for future reference.
> TIF_MEMDIE was a "this is the oom victim & it has access to memory
> reserves" flag in the past. It has that thread vs. process problems and
> tsk_is_oom_victim was introduced later to get rid of them and other
> issues as well as the guarantee that we can identify the oom victim's mm reliably
> for other oom_reaper. I recommend reading git log of mm/oom_kill.c to
> get hairy history of that area and how tricky it is due all the subtle
> interaction with process exit paths etc.
>
> >
> > This change will thaw the entire victim process when OOM occurs,
> > ensuring that the oom victim can terminate on its own.
> >
> > Signed-off-by: zhongjinji <zhongjinji@...or.com>
>
> Acked-by: Michal Hocko <mhocko@...e.com>
> Thanks!
Reviewed-by: Suren Baghdasaryan <surenb@...gle.com>
> >
> > Acked-by: Michal Hocko <mhocko@...e.com>
> > ---
> > include/linux/freezer.h | 2 ++
> > kernel/freezer.c | 20 +++++++++++++++++++-
> > mm/oom_kill.c | 10 +++++-----
> > 3 files changed, 26 insertions(+), 6 deletions(-)
> >
> > diff --git a/include/linux/freezer.h b/include/linux/freezer.h
> > index b303472255be..32884c9721e5 100644
> > --- a/include/linux/freezer.h
> > +++ b/include/linux/freezer.h
> > @@ -47,6 +47,7 @@ extern int freeze_processes(void);
> > extern int freeze_kernel_threads(void);
> > extern void thaw_processes(void);
> > extern void thaw_kernel_threads(void);
> > +extern void thaw_process(struct task_struct *p);
> >
> > static inline bool try_to_freeze(void)
> > {
> > @@ -80,6 +81,7 @@ static inline int freeze_processes(void) { return -ENOSYS; }
> > static inline int freeze_kernel_threads(void) { return -ENOSYS; }
> > static inline void thaw_processes(void) {}
> > static inline void thaw_kernel_threads(void) {}
> > +static inline void thaw_process(struct task_struct *p) {}
> >
> > static inline bool try_to_freeze(void) { return false; }
> >
> > diff --git a/kernel/freezer.c b/kernel/freezer.c
> > index 6a96149aede9..ddc11a8bd2ea 100644
> > --- a/kernel/freezer.c
> > +++ b/kernel/freezer.c
> > @@ -10,6 +10,7 @@
> > #include <linux/export.h>
> > #include <linux/syscalls.h>
> > #include <linux/freezer.h>
> > +#include <linux/oom.h>
> > #include <linux/kthread.h>
> >
> > /* total number of freezing conditions in effect */
> > @@ -40,7 +41,7 @@ bool freezing_slow_path(struct task_struct *p)
> > if (p->flags & (PF_NOFREEZE | PF_SUSPEND_TASK))
> > return false;
> >
> > - if (test_tsk_thread_flag(p, TIF_MEMDIE))
> > + if (tsk_is_oom_victim(p))
> > return false;
> >
> > if (pm_nosig_freezing || cgroup_freezing(p))
> > @@ -206,6 +207,23 @@ void __thaw_task(struct task_struct *p)
> > wake_up_state(p, TASK_FROZEN);
> > }
> >
> > +/*
> > + * thaw_process - Thaw a frozen process
> > + * @p: the process to be thawed
> > + *
> > + * Iterate over all threads of @p and call __thaw_task() on each.
> > + */
> > +void thaw_process(struct task_struct *p)
> > +{
> > + struct task_struct *t;
> > +
> > + rcu_read_lock();
> > + for_each_thread(p, t) {
> > + __thaw_task(t);
> > + }
> > + rcu_read_unlock();
> > +}
> > +
> > /**
> > * set_freezable - make %current freezable
> > *
> > diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> > index 25923cfec9c6..88356b66cc35 100644
> > --- a/mm/oom_kill.c
> > +++ b/mm/oom_kill.c
> > @@ -772,12 +772,12 @@ static void mark_oom_victim(struct task_struct *tsk)
> > mmgrab(tsk->signal->oom_mm);
> >
> > /*
> > - * Make sure that the task is woken up from uninterruptible sleep
> > - * if it is frozen because OOM killer wouldn't be able to free
> > - * any memory and livelock. freezing_slow_path will tell the freezer
> > - * that TIF_MEMDIE tasks should be ignored.
> > + * Make sure that the process is woken up from uninterruptible sleep
> > + * if it is frozen because OOM killer wouldn't be able to free any
> > + * memory and livelock. The freezer will thaw the tasks that are OOM
> > + * victims regardless of the PM freezing and cgroup freezing states.
> > */
> > - __thaw_task(tsk);
> > + thaw_process(tsk);
> > atomic_inc(&oom_victims);
> > cred = get_task_cred(tsk);
> > trace_mark_victim(tsk, cred->uid.val);
> > --
> > 2.17.1
>
> --
> Michal Hocko
> SUSE Labs
>
Powered by blists - more mailing lists