linux-kernel - [PATCH] fix vmaccnt at fork (Was Re: "heuristic overcommit" and fork())

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 13 Feb 2009 10:36:55 +0900
From:	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
To:	mel@....ul.ie
Cc:	<linux-kernel@...r.kernel.org>,
	"David CHAMPELOVIER" <david@...mpelovier.com>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>
Subject: [PATCH] fix vmaccnt at fork (Was Re: "heuristic overcommit" and
 fork())

On Wed, 11 Feb 2009 20:26:32 +0100
"David CHAMPELOVIER" <david@...mpelovier.com> wrote:

> Hi,
> 
> Recently, I was unable to fork() a 38 GB process on a system with 64 GB RAM
> and no swap.
> Having a look at the kernel source, I surprisingly found that in "heuristic
> overcommit" mode, fork() always checks that there is enough memory to
> duplicate process memory.
> 
> As far as I know, overcommit was introduced in the kernel for several
> reasons, and fork() was one of them, since applications often exec() just
> after fork(). I know fork() is not the most judicious choice in this case,
> but well, this is the way many applications are written.
> 
> Moreover, I can read in the proc man page that in "heuristic overcommit
> mode", "obvious overcommits of address space are refused". I do not think
> fork() is an obvious overcommit, that's why I would expect fork() to be
> always accepted in this mode.
> 
> So, is there a reason why fork() checks for available memory in "heuristic
> mode" ?
> 

fork() is used for duplicate process and it means to duplicate memory space.
Because of Copy-On-Write, the page will not be used acutally. But, it's not
different from mmap() case. In that case, overcommit_guess compares
requested size and size of free memory for all that we use demand paging.
So, the behavior is not surprizing.  For notifing the kernel can assume
exec-is-called-after-fork, we may need some flags or paramater.

But, hmm.., there is something strange, following. Mel, how do you think ?
==

From: KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>

Vm accounting at fork() should use the same logic as mmap().

Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
---
 include/linux/mm.h |    2 ++
 kernel/fork.c      |    3 ++-
 2 files changed, 4 insertions(+), 1 deletion(-)

Index: mmotm-2.6.29-Feb11/kernel/fork.c
===================================================================
--- mmotm-2.6.29-Feb11.orig/kernel/fork.c
+++ mmotm-2.6.29-Feb11/kernel/fork.c
@@ -301,7 +301,8 @@ static int dup_mmap(struct mm_struct *mm
 			continue;
 		}
 		charge = 0;
-		if (mpnt->vm_flags & VM_ACCOUNT) {
+		if (accountable_mapping(mpnt->vm_file, mpnt->vm_flags) &&
+			mpnt->vm_flags & VM_ACCOUNT) {
 			unsigned int len = (mpnt->vm_end - mpnt->vm_start) >> PAGE_SHIFT;
 			if (security_vm_enough_memory(len))
 				goto fail_nomem;
Index: mmotm-2.6.29-Feb11/include/linux/mm.h
===================================================================
--- mmotm-2.6.29-Feb11.orig/include/linux/mm.h
+++ mmotm-2.6.29-Feb11/include/linux/mm.h
@@ -1047,6 +1047,8 @@ extern void free_bootmem_with_active_reg
 typedef int (*work_fn_t)(unsigned long, unsigned long, void *);
 extern void work_with_active_regions(int nid, work_fn_t work_fn, void *data);
 extern void sparse_memory_present_with_active_regions(int nid);
+extern int accountable_mapping(struct file *file, unsigned int vmflags);
+
 #endif /* CONFIG_ARCH_POPULATES_NODE_MAP */
 
 #if !defined(CONFIG_ARCH_POPULATES_NODE_MAP) && \



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/