linux-kernel - Re: [PATCH v7] mm: Add PM_THP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <FD103257-48FC-4441-BB32-58C14C0CDE3F@oracle.com>
Date:   Tue, 11 Jan 2022 23:35:02 +0000
From:   William Kucharski <william.kucharski@...cle.com>
To:     Mina Almasry <almasrymina@...gle.com>
CC:     Matthew Wilcox <willy@...radead.org>,
        Jonathan Corbet <corbet@....net>,
        David Hildenbrand <david@...hat.com>,
        "Paul E . McKenney" <paulmckrcu@...com>,
        Yu Zhao <yuzhao@...gle.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Peter Xu <peterx@...hat.com>,
        Ivan Teterevkov <ivan.teterevkov@...anix.com>,
        Florian Schmidt <florian.schmidt@...anix.com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "linux-doc@...r.kernel.org" <linux-doc@...r.kernel.org>
Subject: Re: [PATCH v7] mm: Add PM_THP_MAPPED to /proc/pid/pagemap



> On Jan 4, 2022, at 4:04 PM, Mina Almasry <almasrymina@...gle.com> wrote:
> 
> On Mon, Dec 13, 2021 at 4:22 PM Mina Almasry <almasrymina@...gle.com> wrote:
>> 
>> On Sat, Nov 27, 2021 at 8:10 PM Matthew Wilcox <willy@...radead.org> wrote:
>>> 
>>> On Mon, Nov 22, 2021 at 04:01:02PM -0800, Mina Almasry wrote:
>>>> Add PM_THP_MAPPED MAPPING to allow userspace to detect whether a given virt
>>>> address is currently mapped by a transparent huge page or not.  Example
>>>> use case is a process requesting THPs from the kernel (via a huge tmpfs
>>>> mount for example), for a performance critical region of memory.  The
>>>> userspace may want to query whether the kernel is actually backing this
>>>> memory by hugepages or not.
>>> 
>>> But what is userspace going to _do_ differently if the kernel hasn't
>>> backed the memory with huge pages?
>> 
>> Sorry for the late reply here.
>> 
>> My plan is to expose this information as metrics right now and:
>> 1. Understand the kind of hugepage backing we're actually getting if any.
>> 2. If there are drops in hugepage backing we can investigate the
>> cause, whether it's due to normal memory fragmentation or some
>> bug/issue.
>> 3. Schedule machines for reboots to defragment the memory if the
>> hugepage backing is too low.
>> 4. Possibly motivate future work to improve hugepage backing if our
>> numbers are too low.
> 
> Friendly ping on this. It has been reviewed by a few folks and after
> Matthew had questions about the use case which I've answered in the
> email above. Matthew, are you opposed to this patch?

I realize I'm jumping in late on this, but while (1) and (2) are
understandable, this bothers me:

> 3. Schedule machines for reboots to defragment the memory if the
> hugepage backing is too low.

If it's important enough to reboot the machine, shouldn't one be using
hugetlbfs instead?

It seems like user space is trying to track something it shouldn't be concerned
about, rather like if user space were concerned about precisely where in physical
memory its pages were mapped.