linux-kernel - [PATCH v4 1/5] mm: Document lenient map

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20251028212528.681081-2-kaleshsingh@google.com>
Date: Tue, 28 Oct 2025 14:24:32 -0700
From: Kalesh Singh <kaleshsingh@...gle.com>
To: akpm@...ux-foundation.org, minchan@...nel.org, lorenzo.stoakes@...cle.com, 
	david@...hat.com, Liam.Howlett@...cle.com, rppt@...nel.org, pfalcato@...e.de
Cc: rostedt@...dmis.org, hughd@...gle.com, kernel-team@...roid.com, 
	android-mm@...gle.com, Kalesh Singh <kaleshsingh@...gle.com>, 
	Alexander Viro <viro@...iv.linux.org.uk>, Christian Brauner <brauner@...nel.org>, Jan Kara <jack@...e.cz>, 
	Kees Cook <kees@...nel.org>, Vlastimil Babka <vbabka@...e.cz>, Suren Baghdasaryan <surenb@...gle.com>, 
	Michal Hocko <mhocko@...e.com>, Jann Horn <jannh@...gle.com>, 
	Masami Hiramatsu <mhiramat@...nel.org>, Mathieu Desnoyers <mathieu.desnoyers@...icios.com>, 
	Ingo Molnar <mingo@...hat.com>, Peter Zijlstra <peterz@...radead.org>, 
	Juri Lelli <juri.lelli@...hat.com>, Vincent Guittot <vincent.guittot@...aro.org>, 
	Dietmar Eggemann <dietmar.eggemann@....com>, Ben Segall <bsegall@...gle.com>, 
	Mel Gorman <mgorman@...e.de>, Valentin Schneider <vschneid@...hat.com>, Shuah Khan <shuah@...nel.org>, 
	linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org, 
	linux-mm@...ck.org, linux-trace-kernel@...r.kernel.org, 
	linux-kselftest@...r.kernel.org
Subject: [PATCH v4 1/5] mm: Document lenient map_count checks

Add comments to the map_count limit checks in do_mmap() and
do_brk_flags() to clarify their intended behavior.

The use of a strict inequality ('>') in these checks is intentional but
non-obvious. It allows these functions to succeed when the VMA count is
exactly at the sysctl_max_map_count limit. This historical behavior
accounts for cases where the operation might not create a new VMA, but
instead merge with or expand an existing one, in which case the VMA
count does not increase.

These comments clarify the long-standing behavior and will help prevent
future misinterpretation as an off-by-one error.

Signed-off-by: Kalesh Singh <kaleshsingh@...gle.com>
---

Changes in v4:
 - Keep the existing lenient behavior, per Hugh
 - Document this is intended, per Lorenzo

Changes in v3:
 - Collect Reviewed-by and Acked-by tags.

Changes in v2:
 - Fix mmap check, per Pedro

 mm/mmap.c | 9 +++++++++
 mm/vma.c  | 6 ++++++
 2 files changed, 15 insertions(+)

diff --git a/mm/mmap.c b/mm/mmap.c
index 644f02071a41..78843a2fae42 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -374,6 +374,15 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
 		return -EOVERFLOW;
 
 	/* Too many mappings? */
+	/*
+	 * The check is intentionally lenient (>) to allow an mmap() at the limit
+	 * to succeed. This is for historical reasons, as the new mapping might
+	 * merge with an adjacent VMA and not increase the total VMA count.
+	 *
+	 * If a merge does not occur, the process is allowed to exceed the
+	 * sysctl_max_map_count limit by one. This behavior is preserved to
+	 * avoid breaking existing applications.
+	 */
 	if (mm->map_count > sysctl_max_map_count)
 		return -ENOMEM;
 
diff --git a/mm/vma.c b/mm/vma.c
index 919d1fc63a52..d0bb3127280e 100644
--- a/mm/vma.c
+++ b/mm/vma.c
@@ -2813,6 +2813,12 @@ int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *vma,
 	if (!may_expand_vm(mm, vm_flags, len >> PAGE_SHIFT))
 		return -ENOMEM;
 
+	/*
+	 * The check is intentionally lenient (>) to allow brk() to succeed at
+	 * the limit. This is for historical reasons, as expanding the heap
+	 * typically extends the existing brk VMA rather than creating a new one.
+	 * See also the comment in do_mmap().
+	 */
 	if (mm->map_count > sysctl_max_map_count)
 		return -ENOMEM;
 
-- 
2.51.1.851.g4ebd6896fd-goog