linux-kernel - Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170627144125.GP28072@dhcp22.suse.cz>
Date:   Tue, 27 Jun 2017 16:41:26 +0200
From:   Michal Hocko <mhocko@...nel.org>
To:     Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>
Cc:     linux-mm@...ck.org, rientjes@...gle.com, oleg@...hat.com,
        andrea@...nel.org, akpm@...ux-foundation.org,
        linux-kernel@...r.kernel.org
Subject: Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 27-06-17 23:26:22, Tetsuo Handa wrote:
> Michal Hocko wrote:
> > On Tue 27-06-17 22:31:58, Tetsuo Handa wrote:
[...]
> > > shouldn't we try __oom_reap_task_mm() before calling these down_write()
> > > if mm is OOM victim's?
> > 
> > This is what we try. We simply try to get mmap_sem for read and do our
> > work as soon as possible with the proposed patch. This is already an
> > improvement, no?
> 
> We can ask the OOM reaper kernel thread try to reap before the OOM killer
> releases oom_lock mutex. But that is not guaranteed. It is possible that
> the OOM victim thread is executed until down_write() in __ksm_exit() or
> __khugepaged_exit() and then the OOM reaper kernel thread starts calling
> down_read_trylock().

I strongly suspect we are getting tangent here. While I see your concern
and yes the approach can be probably improved, can we focus on one thing
at the time? I would like to fix the original problem first and only
then go deeper down the rat hole of other subtle details. Do you have
any fundamental objection to the suggested approach or see any issues
with it?

-- 
Michal Hocko
SUSE Labs