linux-kernel - Re: [PATCH 4/3] mm, oom: drop the last allocation attempt before out_of

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.DEB.2.10.1601281508380.31035@chino.kir.corp.google.com>
Date:	Thu, 28 Jan 2016 15:19:08 -0800 (PST)
From:	David Rientjes <rientjes@...gle.com>
To:	Johannes Weiner <hannes@...xchg.org>
cc:	Michal Hocko <mhocko@...nel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Mel Gorman <mgorman@...e.de>,
	Tetsuo Handa <penguin-kernel@...ove.sakura.ne.jp>,
	Hillf Danton <hillf.zj@...baba-inc.com>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	linux-mm@...ck.org, LKML <linux-kernel@...r.kernel.org>,
	Michal Hocko <mhocko@...e.com>
Subject: Re: [PATCH 4/3] mm, oom: drop the last allocation attempt before
 out_of_memory

On Thu, 28 Jan 2016, Johannes Weiner wrote:

> The check has to happen while holding the OOM lock, otherwise we'll
> end up killing much more than necessary when there are many racing
> allocations.
> 

Right, we need to try with ALLOC_WMARK_HIGH after oom_lock has been 
acquired.

The situation is still somewhat fragile, however, but I think it's 
tangential to this patch series.  If the ALLOC_WMARK_HIGH allocation fails 
because an oom victim hasn't freed its memory yet, and then the TIF_MEMDIE 
thread isn't visible during the oom killer's tasklist scan because it has 
exited, we still end up killing more than we should.  The likelihood of 
this happening grows with the length of the tasklist.

Perhaps we should try testing watermarks after a victim has been selected 
and immediately before killing?  (Aside: we actually carry an internal 
patch to test mem_cgroup_margin() in the memcg oom path after selecting a 
victim because we have been hit with this before in the memcg path.)

I would think that retrying with ALLOC_WMARK_HIGH would be enough memory 
to deem that we aren't going to immediately reenter an oom condition so 
the deferred killing is a waste of time.

The downside is how sloppy this would be because it's blurring the line 
between oom killer and page allocator.  We'd need the oom killer to return 
the selected victim to the page allocator, try the allocation, and then 
call oom_kill_process() if necessary.