linux-kernel - Re: [PATCH v4] mm: use per_vma lock for MADV

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAGsJ_4zBKho=vdwfP89XvvouOytckBkFJc9h5G+-+DGDL803TA@mail.gmail.com>
Date: Wed, 18 Jun 2025 18:32:36 +0800
From: Barry Song <21cnbao@...il.com>
To: David Hildenbrand <david@...hat.com>
Cc: Lance Yang <lance.yang@...ux.dev>, akpm@...ux-foundation.org, linux-mm@...ck.org, 
	linux-kernel@...r.kernel.org, Barry Song <v-songbaohua@...o.com>, 
	"Liam R. Howlett" <Liam.Howlett@...cle.com>, Vlastimil Babka <vbabka@...e.cz>, Jann Horn <jannh@...gle.com>, 
	Suren Baghdasaryan <surenb@...gle.com>, Lokesh Gidra <lokeshgidra@...gle.com>, 
	Tangquan Zheng <zhengtangquan@...o.com>, Qi Zheng <zhengqi.arch@...edance.com>, 
	Lance Yang <ioworker0@...il.com>, Lorenzo Stoakes <lorenzo.stoakes@...cle.com>, 
	Zi Li <zi.li@...ux.dev>
Subject: Re: [PATCH v4] mm: use per_vma lock for MADV_DONTNEED

On Wed, Jun 18, 2025 at 6:30 PM Barry Song <21cnbao@...il.com> wrote:
>
> On Wed, Jun 18, 2025 at 6:18 PM David Hildenbrand <david@...hat.com> wrote:
> >
> > On 18.06.25 11:52, Barry Song wrote:
> > > On Wed, Jun 18, 2025 at 10:25 AM Lance Yang <lance.yang@...ux.dev> wrote:
> > >>
> > >> Hi all,
> > >>
> > >> Crazy, the per-VMA lock for madvise is an absolute game-changer ;)
> > >>
> > >> On 2025/6/17 21:38, Lorenzo Stoakes wrote:
> > >> [...]
> > >>>
> > >>> On Sun, Jun 08, 2025 at 10:01:50AM +1200, Barry Song wrote:
> > >>>> From: Barry Song <v-songbaohua@...o.com>
> > >>>>
> > >>>> Certain madvise operations, especially MADV_DONTNEED, occur far more
> > >>>> frequently than other madvise options, particularly in native and Java
> > >>>> heaps for dynamic memory management.
> > >>>>
> > >>>> Currently, the mmap_lock is always held during these operations, even when
> > >>>> unnecessary. This causes lock contention and can lead to severe priority
> > >>>> inversion, where low-priority threads—such as Android's HeapTaskDaemon—
> > >>>> hold the lock and block higher-priority threads.
> > >>>>
> > >>>> This patch enables the use of per-VMA locks when the advised range lies
> > >>>> entirely within a single VMA, avoiding the need for full VMA traversal. In
> > >>>> practice, userspace heaps rarely issue MADV_DONTNEED across multiple VMAs.
> > >>>>
> > >>>> Tangquan’s testing shows that over 99.5% of memory reclaimed by Android
> > >>>> benefits from this per-VMA lock optimization. After extended runtime,
> > >>>> 217,735 madvise calls from HeapTaskDaemon used the per-VMA path, while
> > >>>> only 1,231 fell back to mmap_lock.
> > >>>>
> > >>>> To simplify handling, the implementation falls back to the standard
> > >>>> mmap_lock if userfaultfd is enabled on the VMA, avoiding the complexity of
> > >>>> userfaultfd_remove().
> > >>>>
> > >>>> Many thanks to Lorenzo's work[1] on:
> > >>>> "Refactor the madvise() code to retain state about the locking mode
> > >>>> utilised for traversing VMAs.
> > >>>>
> > >>>> Then use this mechanism to permit VMA locking to be done later in the
> > >>>> madvise() logic and also to allow altering of the locking mode to permit
> > >>>> falling back to an mmap read lock if required."
> > >>>>
> > >>>> One important point, as pointed out by Jann[2], is that
> > >>>> untagged_addr_remote() requires holding mmap_lock. This is because
> > >>>> address tagging on x86 and RISC-V is quite complex.
> > >>>>
> > >>>> Until untagged_addr_remote() becomes atomic—which seems unlikely in
> > >>>> the near future—we cannot support per-VMA locks for remote processes.
> > >>>> So for now, only local processes are supported.
> > >>
> > >> Just to put some numbers on it, I ran a micro-benchmark with 100
> > >> parallel threads, where each thread calls madvise() on its own 1GiB
> > >> chunk of 64KiB mTHP-backed memory. The performance gain is huge:
> > >>
> > >> 1) MADV_DONTNEED saw its average time drop from 0.0508s to 0.0270s (~47%
> > >> faster)
> > >> 2) MADV_FREE     saw its average time drop from 0.3078s to 0.1095s (~64%
> > >> faster)
> > >
> > > Thanks for the report, Lance. I assume your micro-benchmark includes some
> > > explicit or implicit operations that may require mmap_write_lock().
> > > As  mmap_read_lock() only waits for writers and does not block other
> > > mmap_read_lock() calls.
> >
> > The number rather indicate that one test was run with (m)THPs enabled
> > and the other not? Just a thought. The locking overhead from my
> > experience is not that significant.
>
> Right. I don't expect pure madvise_dontneed/free—without any additional
> behavior requiring mmap_write_lock—to improve performance significantly.
> The main benefit would be avoiding contention on the write lock.
>
> Consider this scenario:
> timestamp1: Thread A acquires the read lock
> timestamp2: Thread B attempts to acquire the write lock
> timestamp3: Threads C, D, and E attempt to acquire the read lock
>
> In this case, thread B must wait for A, and threads C, D, and E will
> wait for both A and B. Any write lock request effectively blocks all
> subsequent read acquisitions.
>
> In the worst case, thread A might be a GC thread with a high nice value.
> If it's preempted by other threads, the delay can reach several
> milliseconds—as we've observed in some cases.

sorry for the typo. I mean a few hundred milliseconds.
>
> >
> > --
> > Cheers,
> >
> > David / dhildenb
> >
>
> Thanks
> Barry