linux-kernel - Re: Memory overcommit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20091013120840.a844052d.kamezawa.hiroyu@jp.fujitsu.com>
Date:	Tue, 13 Oct 2009 12:08:40 +0900
From:	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
To:	vedran.furac@...il.com
Cc:	Vedran Furač <vedranf@...ranf.mine.nu>,
	linux-kernel@...r.kernel.org
Subject: Re: Memory overcommit

On Mon, 12 Oct 2009 13:51:07 +0200
Vedran Furač <vedranf@...ranf.mine.nu> wrote:
> /proc/meminfo
> MemTotal:        3542532 kB
> MemFree:          892972 kB
> Buffers:            2664 kB
> Cached:           130940 kB
> 
> ...that there is almost 900MB free memory. But OK, I can live with it.
> 
> So, my question is: why today overcommit isn't turned off *by default*?
> I have it turned off for a few years now and only side effect is that I
> don't get processes killed randomly anymore, I don't loose valuable time
> and data.
> 
"isn't turned off" means "vm.overcommit_memory==2" ?
And...what's version your kernel is ?
oom-killer still finds "definitely-not-guilty" ones ?

I guess the reason of default value is that the kernel assume processes will
not always use all mmaped range. There will be unused range in process's virtual
memory and it can be big.

For example, typical case in a server, 
when you run multi-thread program (like java VM),

  - stack per thread
  - malloc() arena per thread 

can makes difference among size-of-mapped-range v.s. used-pages bigger.
I saw Gigabytes of unused range on ia64 host,...statck size was big.

IIUC, the size is determined by ulimit's stack size at default. it's 10M on
my x86-64 host.
You'll see 1G of commited usage when you run 100 no-op threads.

And if strict check(vm.ovecommit_memory=2) is used, mmap() return -ENOMEM
whenever it hits limit.
You have to find "which processs should be killed" by youself, anyway.

Against random-kill, you may have 2 choices.

1. use  /proc/<pid>/oom_adj
2. use  memory cgroup.

Something more easy-to-use method may be appriciated. We have above 2 now.

Thanks,
-Kame

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/