linux-kernel - Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <645b35a5-970d-dcfe-2b4a-04ebd4444756@redhat.com>
Date:   Fri, 2 Oct 2020 10:30:43 +0200
From:   David Hildenbrand <david@...hat.com>
To:     Michal Hocko <mhocko@...e.com>
Cc:     Zi Yan <ziy@...dia.com>, linux-mm@...ck.org,
        "Kirill A . Shutemov" <kirill.shutemov@...ux.intel.com>,
        Roman Gushchin <guro@...com>, Rik van Riel <riel@...riel.com>,
        Matthew Wilcox <willy@...radead.org>,
        Shakeel Butt <shakeelb@...gle.com>,
        Yang Shi <shy828301@...il.com>,
        Jason Gunthorpe <jgg@...dia.com>,
        Mike Kravetz <mike.kravetz@...cle.com>,
        William Kucharski <william.kucharski@...cle.com>,
        Andrea Arcangeli <aarcange@...hat.com>,
        John Hubbard <jhubbard@...dia.com>,
        David Nellans <dnellans@...dia.com>,
        linux-kernel@...r.kernel.org
Subject: Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

On 02.10.20 10:10, Michal Hocko wrote:
> On Fri 02-10-20 09:50:02, David Hildenbrand wrote:
>>>>> - huge page sizes controllable by the userspace?
>>>>
>>>> It might be good to allow advanced users to choose the page sizes, so they
>>>> have better control of their applications.
>>>
>>> Could you elaborate more? Those advanced users can use hugetlb, right?
>>> They get a very good control over page size and pool preallocation etc.
>>> So they can get what they need - assuming there is enough memory.
>>>
>>
>> I am still not convinced that 1G THP (TGP :) ) are really what we want
>> to support. I can understand that there are some use cases that might
>> benefit from it, especially:
> 
> Well, I would say that internal support for larger huge pages (e.g. 1GB)
> that can transparently split under memory pressure is a useful
> funtionality. I cannot really judge how complex that would be

Right, but that's then something different than serving (scarce,
unmovable) gigantic pages from CMA / reserved hugetlbfs pool. Nothing
wrong about *real* THP support, meaning, e.g., grouping consecutive
pages and converting them back and forth on demand. (E.g., 1GB ->
multiple 2MB -> multiple single pages), for example, when having to
migrate such a gigantic page. But that's very different from our
existing gigantic page code as far as I can tell.

> consideting that 2MB THP have turned out to be quite a pain but
> situation has settled over time. Maybe our current code base is prepared
> for that much better.
> 
> Exposing that interface to the userspace is a different story of course.
> I do agree that we likely do not want to be very explicit about that.
> E.g. an interface for address space defragmentation without any more
> specifics sounds like a useful feature to me. It will be up to the
> kernel to decide which huge pages to use.

Yes, I think one important feature would be that we don't end up placing
a gigantic page where only a handful of pages are actually populated
without green light from the application - because that's what some user
space applications care about (not consuming more memory than intended.
IIUC, this is also what this patch set does). I'm fine with placing
gigantic pages if it really just "defragments" the address space layout,
without filling unpopulated holes.

Then, this would be mostly invisible to user space, and we really
wouldn't have to care about any configuration.

-- 
Thanks,

David / dhildenb