linux-kernel - Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <201706272231.ABH00025.FMOFOJSVLOQHFt@I-love.SAKURA.ne.jp>
Date:   Tue, 27 Jun 2017 22:31:58 +0900
From:   Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>
To:     mhocko@...nel.org
Cc:     linux-mm@...ck.org, rientjes@...gle.com, oleg@...hat.com,
        andrea@...nel.org, akpm@...ux-foundation.org,
        linux-kernel@...r.kernel.org
Subject: Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

Michal Hocko wrote:
> On Tue 27-06-17 20:39:28, Tetsuo Handa wrote:
> > Michal Hocko wrote:
> > > > I wonder why you prefer timeout based approach. Your patch will after all
> > > > set MMF_OOM_SKIP if operations between down_write() and up_write() took
> > > > more than one second.
> > > 
> > > if we reach down_write then we have unmapped the address space in
> > > exit_mmap and oom reaper cannot do much more.
> > 
> > So, by the time down_write() is called, majority of memory is already released, isn't it?
> 
> In most cases yes. To be put it in other words. By the time exit_mmap
> takes down_write there is nothing more oom reaper could reclaim.
> 
Then, aren't there two exceptions which your patch cannot guarantee;
down_write(&mm->mmap_sem) in __ksm_exit() and __khugepaged_exit() ?

Since for some reason exit_mmap() cannot be brought to before
ksm_exit(mm)/khugepaged_exit(mm) calls,

	ksm_exit(mm);
	khugepaged_exit(mm); /* must run before exit_mmap */
	exit_mmap(mm);

shouldn't we try __oom_reap_task_mm() before calling these down_write()
if mm is OOM victim's?