linux-kernel - Re: [PATCH] mm,oom: Bring OOM notifier callbacks to outside of OOM killer.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 27 Jun 2018 19:52:23 +0900
From:   Tetsuo Handa <penguin-kernel@...ove.sakura.ne.jp>
To:     paulmck@...ux.vnet.ibm.com
Cc:     Michal Hocko <mhocko@...nel.org>,
        David Rientjes <rientjes@...gle.com>, linux-mm@...ck.org,
        Andrew Morton <akpm@...ux-foundation.org>,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH] mm,oom: Bring OOM notifier callbacks to outside of OOM
 killer.

On 2018/06/27 8:50, Paul E. McKenney wrote:
> On Wed, Jun 27, 2018 at 05:10:48AM +0900, Tetsuo Handa wrote:
>> As far as I can see,
>>
>> -	atomic_set(&oom_callback_count, 1);
>> +	atomic_inc(&oom_callback_count);
>>
>> should be sufficient.
> 
> I don't see how that helps.  For example, suppose that two tasks
> invoked rcu_oom_notify() at about the same time.  Then they could
> both see oom_callback_count equal to zero, both atomically increment
> oom_callback_count, then both do the IPI invoking rcu_oom_notify_cpu()
> on each online CPU.
> 
> So far, so good.  But rcu_oom_notify_cpu() enqueues a per-CPU RCU
> callback, and enqueuing the same callback twice in quick succession
> would fatally tangle RCU's callback lists.
> 
> What am I missing here?
> 
> 							Thanx, Paul

You are pointing out that "number of rsp->call() is called" > "number of
rcu_oom_callback() is called" can happen if concurrently called, aren't you?
Then, you are not missing anything. You will need to use something equivalent
to oom_lock even if you can convert rcu_oom_notify() to use shrinkers.