linux-kernel - Re: [patch -mm] mm, oom: remove oom_lock from exit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 16 Jul 2018 19:38:21 +0900
From:   Tetsuo Handa <penguin-kernel@...ove.sakura.ne.jp>
To:     Michal Hocko <mhocko@...nel.org>
Cc:     David Rientjes <rientjes@...gle.com>,
        Andrew Morton <akpm@...ux-foundation.org>, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org
Subject: Re: [patch -mm] mm, oom: remove oom_lock from exit_mmap

On 2018/07/16 16:44, Michal Hocko wrote:
>> If setting MMF_OOM_SKIP is guarded by oom_lock, we can enforce
>> last second allocation attempt like below.
>>
>>   CPU 0                                   CPU 1
>>   
>>   mutex_trylock(&oom_lock) in __alloc_pages_may_oom() succeeds.
>>   get_page_from_freelist() fails.
>>   Enters out_of_memory().
>>
>>                                           __oom_reap_task_mm() reclaims some memory.
>>                                           mutex_lock(&oom_lock);
>>
>>   select_bad_process() does not select new victim because MMF_OOM_SKIP is not yet set.
>>   Leaves out_of_memory().
>>   mutex_unlock(&oom_lock) in __alloc_pages_may_oom() is called.
>>
>>                                           Sets MMF_OOM_SKIP.
>>                                           mutex_unlock(&oom_lock);
>>
>>   get_page_from_freelist() likely succeeds before reaching __alloc_pages_may_oom() again.
>>   Saved one OOM victim from being needlessly killed.
>>
>> That is, guarding setting MMF_OOM_SKIP works as if synchronize_rcu(); it waits for anybody
>> who already acquired (or started waiting for) oom_lock to release oom_lock, in order to
>> prevent select_bad_process() from needlessly selecting new OOM victim.
> 
> Hmm, is this a practical problem though? Do we really need to have a
> broader locking context just to defeat this race?

Yes, for you think that select_bad_process() might take long time. It is possible
that MMF_OOM_SKIP is set while the owner of oom_lock is preempted. It is not such
a small window that select_bad_process() finds an mm which got MMF_OOM_SKIP
immediately before examining that mm.

>                                                   How about this goes
> into a separate patch with some data justifying it?
> 

No. We won't be able to get data until we let people test using released
kernels. I don't like again getting reports like
http://lkml.kernel.org/r/1495034780-9520-1-git-send-email-guro@fb.com
by not guarding MMF_OOM_SKIP.