linux-kernel - [PATCH 1/2] mm: do not limit VmData with RLIMIT

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Sat, 23 Jan 2016 10:39:40 +0300
From:	Konstantin Khlebnikov <koct9i@...il.com>
To:	Cyrill Gorcunov <gorcunov@...il.com>, linux-mm@...ck.org,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Andrew Morton <akpm@...uxfoundation.org>,
	linux-kernel@...r.kernel.org
Cc:	Vegard Nossum <vegard.nossum@...cle.com>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	Vladimir Davydov <vdavydov@...tuozzo.com>,
	Andy Lutomirski <luto@...capital.net>,
	Quentin Casasnovas <quentin.casasnovas@...cle.com>,
	Kees Cook <keescook@...gle.com>, Willy Tarreau <w@....eu>,
	Pavel Emelyanov <xemul@...tuozzo.com>
Subject: [PATCH 1/2] mm: do not limit VmData with RLIMIT_DATA

This partially reverts 84638335900f ("mm: rework virtual memory accounting")

Before that commit RLIMIT_DATA have control only over size of the brk region.
But that change have caused problems with all existing versions of valgrind
because they set RLIMIT_DATA to zero for some reason.

More over, current check has a major flaw: RLIMIT_DATA in bytes,
not pages. So, some problems might have slipped through testing.
Let's revert it for now and put back in next release.

Signed-off-by: Konstantin Khlebnikov <koct9i@...il.com>
Link: http://lkml.kernel.org/r/20151228211015.GL2194@uranus
Reported-by: Christian Borntraeger <borntraeger@...ibm.com>
---
 mm/mmap.c |    4 ----
 1 file changed, 4 deletions(-)

diff --git a/mm/mmap.c b/mm/mmap.c
index 84b12624ceb0..e0cd98c510ba 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -2982,10 +2982,6 @@ bool may_expand_vm(struct mm_struct *mm, vm_flags_t flags, unsigned long npages)
 	if (mm->total_vm + npages > rlimit(RLIMIT_AS) >> PAGE_SHIFT)
 		return false;
 
-	if ((flags & (VM_WRITE | VM_SHARED | (VM_STACK_FLAGS &
-				(VM_GROWSUP | VM_GROWSDOWN)))) == VM_WRITE)
-		return mm->data_vm + npages <= rlimit(RLIMIT_DATA);
-
 	return true;
 }