linux-kernel - Re: [PATCH] x86/mm: lower MAP_32BIT begin to reduce heap collisions

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <h6lip8qijg4dr.fsf@dev-dsk-simonlie-1b-d602a7e1.eu-west-1.amazon.com>
Date: Fri, 12 Sep 2025 13:57:36 +0000
From: Simon Liebold <simonlie@...zon.de>
To: Dave Hansen <dave.hansen@...el.com>
CC: Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar <mingo@...hat.com>,
	Borislav Petkov <bp@...en8.de>, Dave Hansen <dave.hansen@...ux.intel.com>,
	<x86@...nel.org>, "H. Peter Anvin" <hpa@...or.com>, Andrew Morton
	<akpm@...ux-foundation.org>, Mark Brown <broonie@...nel.org>, Lorenzo Stoakes
	<lorenzo.stoakes@...cle.com>, Helge Deller <deller@....de>, "Liam R. Howlett"
	<Liam.Howlett@...cle.com>, Simon Liebold <lieboldsimonpaul@...il.com>,
	<linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] x86/mm: lower MAP_32BIT begin to reduce heap collisions

Simon Liebold <simonlie@...zon.de> writes:

> Dave Hansen <dave.hansen@...el.com> writes:
>
>> On 8/25/25 03:48, Simon Liebold wrote:
>>> Commit 03475167fda5 ("x86: Increase brk randomness entropy for 64-bit
>>> systems") increased the brk randomness from 32 MiB to 1 GiB. MAP_32BIT
>>> looks between 1 GiB and 2 GiB for an unmapped area. Depending on the
>>> randomization, a heap starting high enough and being big enough can use
>>> up all the area that MAP_32BIT looks at, leading to allocation failures.
>>
>> Isn't that still a really unreasonably gigantic heap?
>>
>> Would you mind posting some actual /proc/$pid/maps output from one of
>> the failure cases?
>
> Hello Dave,
>
> we are seeing heaps this large on real workloads. This is a Lua
> application using LuaJit v2.0.
>
> This is an excerpt from the output of one of the failure cases:
>
>     00400000-00566000 r-xp 00000000 103:01 4476567               [...]/bin/httpd.orig
>     00765000-0076b000 r--p 00165000 103:01 4476567               [...]/bin/httpd.orig
>     0076b000-00772000 rw-p 0016b000 103:01 4476567               [...]/bin/httpd.orig
>     00772000-00778000 rw-p 00000000 00:00 0
>     34a21000-35021000 rw-p 00000000 00:00 0                      [heap]
>     35021000-82ea7000 rw-p 00000000 00:00 0                      [heap]
>     7fee7c0ba000-7fee7c11f000 r-xp 00000000 103:01 3836609       [...]/lib/libluajit-5.1.so
>     7fee7c11f000-7fee7c31f000 ---p 00065000 103:01 3836609       [...]/lib/libluajit-5.1.so
>     7fee7c31f000-7fee7c321000 r--p 00065000 103:01 3836609       [...]/lib/libluajit-5.1.so
>     7fee7c321000-7fee7c322000 rw-p 00067000 103:01 3836609       [...]/lib/libluajit-5.1.so
>     [Other maps at high addresses...]
>
> Regards,
>
> Simon Liebold

Hi Dave,

I wanted to follow up on our previous discussion about the brk
randomization and MAP_32BIT allocation failures.

The heap overflow we're seeing is standard LuaJIT v2.0 behavior under
certain workloads, not just custom programs with unusual memory
patterns. Given LuaJIT's widespread production use, these MAP_32BIT
allocation failures could impact many users.

While I understand this might be considered an edge case, it does make
MAP_32BIT allocations less reliable in practice.

Do you have any thoughts on this?

Regards,

Simon Liebold



Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597