linux-kernel - Re: [patch 1/2] mm, memcg: avoid oom notification when current needs access to memory reserves

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 4 Dec 2013 16:23:41 -0800 (PST)
From:	David Rientjes <rientjes@...gle.com>
To:	Michal Hocko <mhocko@...e.cz>
cc:	Johannes Weiner <hannes@...xchg.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	linux-kernel@...r.kernel.org, linux-mm@...ck.org,
	cgroups@...r.kernel.org
Subject: Re: [patch 1/2] mm, memcg: avoid oom notification when current needs
 access to memory reserves

On Wed, 4 Dec 2013, Michal Hocko wrote:

> > I'll repeat: Section 10 of Documentation/cgroups/memory.txt specifies what 
> > userspace should do when waking up; one of those options is not "check if 
> > the memcg is still actually oom in a short period of time once a charging 
> > task with a pending SIGKILL or in the exit path has been able to exit."
> > Users of this interface typically also disable the memcg oom killer 
> > through the same file, it's ludicrous to put the responsibility on 
> > userspace to determine if the wakeup is actionable and requires it to 
> > intervene in one of the methods listed in section 10.
> 
> David, you would need to show us that such a condition happens in real
> loads often enough that such a tweak is worth it. Repeating that a race
> exists doesn't help, because yeah it does and it will after your patch
> as well. So show us that it happens considerably less often with this
> check.
>  

Google depends on getting memory.oom_control notifications only when they 
are actionable, which is exactly how Documentation/cgroups/memory.txt 
describes how userspace should respond to such a notification.

"Actionable" here means that the kernel has exhausted its capabilities of 
allowing for future memory freeing, which is the entire premise of any oom 
killer.

Giving a dying process or a process that is going to subsequently die 
access to memory reserves is a capability the kernel users to ensure 
progress is made in oom conditions.  It is not an exhaustion of 
capabilities.

Yes, we all know that subsequent to the userspace notification that memory 
may be freed and the kill no longer becomes required.  There is nothing 
that can be done about that, and it has never been implied that a memcg is 
guaranteed to still be oom when the process wakes up.

I'm referring to a siutation that can manifest in a number of ways: 
coincidental process exit, coincidental process being killed, 
VMPRESSURE_CRITICAL notification that results in a process being killed, 
or memory threshold notification that results in a process being killed.  
Regardless, we're talking about a situation where something is already 
in the exit path or has been killed and is simply attempting to free its 
memory.

Such a process simply needs access to memory reserves to make progress and 
free its memory as part of the exit path.  The process waiting on 
memory.oom_control does _not_ need to do any of the actions mentioned in 
Documentation/cgroups/memory.txt: reduce usage, enlarge the limit, kill a 
process, or move a process with charge migration.

It would be ridiculous to require anybody implementing such a process to 
check if the oom condition still exists after a period of time before 
taking such an action.  It would be required to wait for any possible 
dying task or process with a pending SIGKILL to exit and there's no way to 
determine how long is long enough to wait or that it will get woken up 
again if it relies on a second signal for the same oom condition.  At the 
same time, the action taken by such a process would still be as racy as it 
would with the patch: we simply can't guarantee memory is not freed 
immediately after we issue the SIGKILL.

What we can control is that the kernel has exhausted its capabilities of 
allowing for future memory freeing at the time of notification.  That's 
the goal of the patch, at the same time making it consistent with the 
documentation.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/