linux-kernel - Re: [PATCH v4 3/3] mm/oom_kill: Have the OOM reaper and exit_mmap() traverse the maple tree in opposite orders

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <0A987E18-9E43-4ED9-BE02-EFCD8D6BC528@gmail.com>
Date: Fri, 15 Aug 2025 21:53:15 +0400
From: gio <giorgitchankvetadze1997@...il.com>
To: Andrew Morton <akpm@...ux-foundation.org>, zhongjinji@...or.com
CC: linux-mm@...ck.org, mhocko@...e.com, rientjes@...gle.com,
 shakeel.butt@...ux.dev, npache@...hat.com, linux-kernel@...r.kernel.org,
 tglx@...utronix.de, mingo@...hat.com, peterz@...radead.org,
 dvhart@...radead.org, dave@...olabs.net, andrealmeid@...lia.com,
 liam.howlett@...cle.com, liulu.liu@...or.com, feng.han@...or.com
Subject: Re: [PATCH v4 3/3] mm/oom_kill: Have the OOM reaper and exit_mmap() traverse the maple tree in opposite orders

I believe the patch should be accepted. While the race condition might be rare in simple workloads, it can become significant in containerized systems or Android devices. The proposed solution is simple, low-risk, and directly addresses the identified problem with minimal code changes.

On August 15, 2025 9:52:17 PM GMT+04:00, gio <giorgitchankvetadze1997@...il.com> wrote:
>I believe the patch should be accepted. While the race condition might be rare in simple workloads, it can become significant in containerized systems or Android devices. The proposed solution is simple, low-risk, and directly addresses the identified problem with minimal code changes.
>
>On August 15, 2025 3:09:14 AM GMT+04:00, Andrew Morton <akpm@...ux-foundation.org> wrote:
>>On Thu, 14 Aug 2025 21:55:55 +0800 <zhongjinji@...or.com> wrote:
>>
>>> When a process is OOM killed, if the OOM reaper and the thread running
>>> exit_mmap() execute at the same time, both will traverse the vma's maple
>>> tree along the same path. They may easily unmap the same vma, causing them
>>> to compete for the pte spinlock. This increases unnecessary load, causing
>>> the execution time of the OOM reaper and the thread running exit_mmap() to
>>> increase.
>>
>>Please tell me what I'm missing here.
>>
>>OOM kills are a rare event.  And this race sounds like it will rarely
>>occur even if an oom-killing is happening.  And the delay will be
>>relatively short.
>>
>>If I'm correct then we're addressing rare*rare*small, so why bother?
>>
>>> When a process exits, exit_mmap() traverses the vma's maple tree from low to high
>>> address. To reduce the chance of unmapping the same vma simultaneously,
>>> the OOM reaper should traverse vma's tree from high to low address. This reduces
>>> lock contention when unmapping the same vma.
>>
>>Sharing some before-and-after runtime measurements would be useful.  Or
>>at least, detailed anecdotes.
>>
>>