linux-kernel - Re: [PATCH] mm,oom: Bring OOM notifier callbacks to outside of OOM killer.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 29 Jun 2018 23:35:48 +0900
From:   Tetsuo Handa <penguin-kernel@...ove.sakura.ne.jp>
To:     paulmck@...ux.vnet.ibm.com, Michal Hocko <mhocko@...nel.org>
Cc:     David Rientjes <rientjes@...gle.com>, linux-mm@...ck.org,
        Andrew Morton <akpm@...ux-foundation.org>,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH] mm,oom: Bring OOM notifier callbacks to outside of OOM
 killer.

On 2018/06/29 21:52, Paul E. McKenney wrote:
> The effect of RCU's current OOM code is to speed up callback invocation
> by at most a few seconds (assuming no stalled CPUs, in which case
> it is not possible to speed up callback invocation).
> 
> Given that, I should just remove RCU's OOM code entirely?

out_of_memory() will start selecting an OOM victim without calling
get_page_from_freelist() since rcu_oom_notify() does not set non-zero
value to "freed" field.

I think that rcu_oom_notify() needs to wait for completion of callback
invocations (possibly with timeout in case there are stalling CPUs) and
set non-zero value to "freed" field if pending callbacks did release memory.

However, what will be difficult to tell is whether invocation of pending callbacks
did release memory. Lack of last second get_page_from_freelist() call after
blocking_notifier_call_chain(&oom_notify_list, 0, &freed) forces rcu_oom_notify()
to set appropriate value (i.e. zero or non-zero) to "freed" field.

We have tried to move really last second get_page_from_freelist() call to inside
out_of_memory() after blocking_notifier_call_chain(&oom_notify_list, 0, &freed).
But that proposal was not accepted...

We could move blocking_notifier_call_chain(&oom_notify_list, 0, &freed) to
before last second get_page_from_freelist() call (and this is what this patch
is trying to do) which would allow rcu_oom_notify() to always return 0...
or update rcu_oom_notify() to use shrinker API...