linux-kernel - Re: [RFC PATCH 00/16] 1GB THP support on x86

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <7E20392E-5ED7-4C22-9555-F3BAABF3CBE9@nvidia.com>
Date:   Tue, 8 Sep 2020 10:05:11 -0400
From:   Zi Yan <ziy@...dia.com>
To:     David Hildenbrand <david@...hat.com>
CC:     Roman Gushchin <guro@...com>,
        "Kirill A. Shutemov" <kirill@...temov.name>, <linux-mm@...ck.org>,
        Rik van Riel <riel@...riel.com>,
        "Kirill A . Shutemov" <kirill.shutemov@...ux.intel.com>,
        Matthew Wilcox <willy@...radead.org>,
        Shakeel Butt <shakeelb@...gle.com>,
        Yang Shi <yang.shi@...ux.alibaba.com>,
        David Nellans <dnellans@...dia.com>,
        <linux-kernel@...r.kernel.org>
Subject: Re: [RFC PATCH 00/16] 1GB THP support on x86_64

On 8 Sep 2020, at 7:57, David Hildenbrand wrote:

> On 03.09.20 18:30, Roman Gushchin wrote:
>> On Thu, Sep 03, 2020 at 05:23:00PM +0300, Kirill A. Shutemov wrote:
>>> On Wed, Sep 02, 2020 at 02:06:12PM -0400, Zi Yan wrote:
>>>> From: Zi Yan <ziy@...dia.com>
>>>>
>>>> Hi all,
>>>>
>>>> This patchset adds support for 1GB THP on x86_64. It is on top of
>>>> v5.9-rc2-mmots-2020-08-25-21-13.
>>>>
>>>> 1GB THP is more flexible for reducing translation overhead and increasing the
>>>> performance of applications with large memory footprint without application
>>>> changes compared to hugetlb.
>>>
>>> This statement needs a lot of justification. I don't see 1GB THP as viable
>>> for any workload. Opportunistic 1GB allocation is very questionable
>>> strategy.
>>
>> Hello, Kirill!
>>
>> I share your skepticism about opportunistic 1 GB allocations, however it might be useful
>> if backed by an madvise() annotations from userspace application. In this case,
>> 1 GB THPs might be an alternative to 1 GB hugetlbfs pages, but with a more convenient
>> interface.
>
> I have concerns if we would silently use 1~GB THPs in most scenarios
> where be would have used 2~MB THP. I'd appreciate a trigger to
> explicitly enable that - MADV_HUGEPAGE is not sufficient because some
> applications relying on that assume that the THP size will be 2~MB
> (especially, if you want sparse, large VMAs).

This patchset is not intended to silently use 1GB THP in place of 2MB THP.
First of all, there is a knob /sys/kernel/mm/transparent_hugepage/enable_1GB
to enable 1GB THP explicitly. Also, 1GB THP is allocated from a reserved CMA
region (although I had alloc_contig_pages as a fallback, which can be removed
in next version), so users need to add hugepage_cma=nG kernel parameter to
enable 1GB THP allocation. If a finer control is necessary, we can add
a new MADV_HUGEPAGE_1GB for 1GB THP.


—
Best Regards,
Yan Zi

Download attachment "signature.asc" of type "application/pgp-signature" (855 bytes)