linux-kernel - Re: Memory overcommit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 30 Oct 2009 14:53:33 +0100
From:	Vedran Furač <vedran.furac@...il.com>
To:	David Rientjes <rientjes@...gle.com>
CC:	Hugh Dickins <hugh.dickins@...cali.co.uk>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	linux-mm@...ck.org, linux-kernel@...r.kernel.org,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	minchan.kim@...il.com, Andrew Morton <akpm@...ux-foundation.org>,
	Andrea Arcangeli <aarcange@...hat.com>
Subject: Re: Memory overcommit

David Rientjes wrote:

> Ok, so this is the forkbomb problem by adding half of each child's 
> total_vm into the badness score of the parent.  We should address this 
> completely seperately by addressing that specific part of the heuristic, 
> not changing what we consider to be a baseline.
> thunderbird.
>
> You're making all these claims and assertions based _solely_ on the theory 
> that killing the application with the most resident RAM is always the 
> optimal solution.  That's just not true, especially if we're just 
> allocating small numbers of order-0 memory.

Well, you are kernel hacker, not me. You know how linux mm works much
more than I do. I just reported a, what I think is a big problem, which
needs to be solved ASAP (2.6.33). I'm afraid that we'll just talk much
and nothing will be done with solution/fix postponed indefinitely. Not
sure if you are interested, but I tested this on windowsxp also, and
nothing bad happens there, system continues to function properly.

For 2-3 years I had memory overcommit turn off. I didn't get any OOM,
but sometimes Java didn't work and it seems that because of some kernel
weirdness (or misunderstanding on my part) I couldn't use all the
available memory:

# echo 2 > /proc/sys/vm/overcommit_memory

# echo 95 > /proc/sys/vm/overcommit_ratio
% ./test  /* malloc in loop as before */
malloc: Cannot allocate memory /* Great, no OOM, but: */

% free -m
          total       used       free     shared    buffers     cached
Mem:      3458        3429         29          0        102       1119
-/+ buffers/cache:    2207       1251

There's plenty of memory available. Shouldn't cache be automatically
dropped (this question was in my original mail, hence the subject)?

All this frustrated not only me, but a great number of users on our
local Croatian linux usenet newsgroup with some of them pointing that as
the reason they use solaris. And so on...

> Much better is to allow the user to decide at what point, regardless of 
> swap usage, their application is using much more memory than expected or 
> required.  They can do that right now pretty well with /proc/pid/oom_adj 
> without this outlandish claim that they should be expected to know the rss 
> of their applications at the time of oom to effectively tune oom_adj.

Believe me, barely a few developers use oom_adj for their applications,
and probably almost none of the end users. What should they do, every
time they start an application, go to console and set the oom_adj. You
cannot expect them to do that.

> What would you suggest?  A script that sits in a loop checking each task's 
> current rss from /proc/pid/stat or their current oom priority though 
> /proc/pid/oom_score and adjusting oom_adj preemptively just in case the 
> oom killer is invoked in the next second?

:)

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/