linux-kernel - Re: [PATCH v4] mm: use per_vma lock for MADV

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <f2a43ae1-6347-47e2-bcc4-845dc7e7ed87@linux.dev>
Date: Wed, 18 Jun 2025 10:25:36 +0800
From: Lance Yang <lance.yang@...ux.dev>
To: Barry Song <21cnbao@...il.com>
Cc: akpm@...ux-foundation.org, linux-mm@...ck.org,
 linux-kernel@...r.kernel.org, Barry Song <v-songbaohua@...o.com>,
 "Liam R. Howlett" <Liam.Howlett@...cle.com>,
 David Hildenbrand <david@...hat.com>, Vlastimil Babka <vbabka@...e.cz>,
 Jann Horn <jannh@...gle.com>, Suren Baghdasaryan <surenb@...gle.com>,
 Lokesh Gidra <lokeshgidra@...gle.com>,
 Tangquan Zheng <zhengtangquan@...o.com>,
 Qi Zheng <zhengqi.arch@...edance.com>, Lance Yang <ioworker0@...il.com>,
 Lorenzo Stoakes <lorenzo.stoakes@...cle.com>, Zi Li <zi.li@...ux.dev>
Subject: Re: [PATCH v4] mm: use per_vma lock for MADV_DONTNEED

Hi all,

Crazy, the per-VMA lock for madvise is an absolute game-changer ;)

On 2025/6/17 21:38, Lorenzo Stoakes wrote:
[...]
> 
> On Sun, Jun 08, 2025 at 10:01:50AM +1200, Barry Song wrote:
>> From: Barry Song <v-songbaohua@...o.com>
>>
>> Certain madvise operations, especially MADV_DONTNEED, occur far more
>> frequently than other madvise options, particularly in native and Java
>> heaps for dynamic memory management.
>>
>> Currently, the mmap_lock is always held during these operations, even when
>> unnecessary. This causes lock contention and can lead to severe priority
>> inversion, where low-priority threads—such as Android's HeapTaskDaemon—
>> hold the lock and block higher-priority threads.
>>
>> This patch enables the use of per-VMA locks when the advised range lies
>> entirely within a single VMA, avoiding the need for full VMA traversal. In
>> practice, userspace heaps rarely issue MADV_DONTNEED across multiple VMAs.
>>
>> Tangquan’s testing shows that over 99.5% of memory reclaimed by Android
>> benefits from this per-VMA lock optimization. After extended runtime,
>> 217,735 madvise calls from HeapTaskDaemon used the per-VMA path, while
>> only 1,231 fell back to mmap_lock.
>>
>> To simplify handling, the implementation falls back to the standard
>> mmap_lock if userfaultfd is enabled on the VMA, avoiding the complexity of
>> userfaultfd_remove().
>>
>> Many thanks to Lorenzo's work[1] on:
>> "Refactor the madvise() code to retain state about the locking mode
>> utilised for traversing VMAs.
>>
>> Then use this mechanism to permit VMA locking to be done later in the
>> madvise() logic and also to allow altering of the locking mode to permit
>> falling back to an mmap read lock if required."
>>
>> One important point, as pointed out by Jann[2], is that
>> untagged_addr_remote() requires holding mmap_lock. This is because
>> address tagging on x86 and RISC-V is quite complex.
>>
>> Until untagged_addr_remote() becomes atomic—which seems unlikely in
>> the near future—we cannot support per-VMA locks for remote processes.
>> So for now, only local processes are supported.

Just to put some numbers on it, I ran a micro-benchmark with 100
parallel threads, where each thread calls madvise() on its own 1GiB
chunk of 64KiB mTHP-backed memory. The performance gain is huge:

1) MADV_DONTNEED saw its average time drop from 0.0508s to 0.0270s (~47% 
faster)
2) MADV_FREE     saw its average time drop from 0.3078s to 0.1095s (~64% 
faster)

Thanks,
Lance