linux-kernel - Re: [PATCH v1 1/2] mm/madvise: introduce MADV_TRY_COLLAPSE for attempted synchronous hugepage collapse

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87ededl9u3.fsf@linux.intel.com>
Date: Fri, 19 Jan 2024 05:41:08 -0800
From: Andi Kleen <ak@...ux.intel.com>
To: Lance Yang <ioworker0@...il.com>
Cc: akpm@...ux-foundation.org,  zokeefe@...gle.com,
  songmuchun@...edance.com,  linux-kernel@...r.kernel.org
Subject: Re: [PATCH v1 1/2] mm/madvise: introduce MADV_TRY_COLLAPSE for
 attempted synchronous hugepage collapse

Lance Yang <ioworker0@...il.com> writes:

> This idea was inspired by MADV_COLLAPSE introduced by Zach O'Keefe[1].
>
> Introduce a new madvise mode, MADV_TRY_COLLAPSE, that allows users to
> make a least-effort attempt at a synchronous collapse of memory at
> their own expense.
>
> The only difference from MADV_COLLAPSE is that the new hugepage allocation
> avoids direct reclaim and/or compaction, quickly failing on allocation errors.
>
> The benefits of this approach are:
>
> * CPU is charged to the process that wants to spend the cycles for the THP
> * Avoid unpredictable timing of khugepaged collapse
> * Prevent unpredictable stalls caused by direct reclaim and/or
> compaction

I haven't completely followed the discussion, but it seem your second
and third point could be addressed by a asynchronous THP fault without
any new APIs: allocate 2MB while failing quickly, then on failure get
a 4K page and provide it to the process, while asking khugepaged to
convert the page ASAP in the background, but only after
it managed to allocate a fresh 2MB page to minimize the process visible
down time.

I suppose that would be much more predictable, although there would be a
slightly risk of overwhelming khugepaged. The later could be
addressed by using a scalable workqueue that allocates more threads
when needed.

-Andi