linux-kernel - Re: [RFC PATCH 00/16] 1GB THP support on x86

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 9 Sep 2020 15:54:40 +0200
From:   David Hildenbrand <david@...hat.com>
To:     Rik van Riel <riel@...riel.com>, Michal Hocko <mhocko@...e.com>
Cc:     Zi Yan <ziy@...dia.com>, Roman Gushchin <guro@...com>,
        "Kirill A. Shutemov" <kirill@...temov.name>, linux-mm@...ck.org,
        "Kirill A . Shutemov" <kirill.shutemov@...ux.intel.com>,
        Matthew Wilcox <willy@...radead.org>,
        Shakeel Butt <shakeelb@...gle.com>,
        Yang Shi <yang.shi@...ux.alibaba.com>,
        David Nellans <dnellans@...dia.com>,
        linux-kernel@...r.kernel.org
Subject: Re: [RFC PATCH 00/16] 1GB THP support on x86_64

On 09.09.20 15:49, Rik van Riel wrote:
> On Wed, 2020-09-09 at 15:43 +0200, David Hildenbrand wrote:
>> On 09.09.20 15:19, Rik van Riel wrote:
>>> On Wed, 2020-09-09 at 09:04 +0200, Michal Hocko wrote:
>>>
>>>> That CMA has to be pre-reserved, right? That requires a
>>>> configuration.
>>>
>>> To some extent, yes.
>>>
>>> However, because that pool can be used for movable
>>> 4kB and 2MB
>>> pages as well as for 1GB pages, it would be easy to just set
>>> the size of that pool to eg. 1/3 or even 1/2 of memory for every
>>> system.
>>>
>>> It isn't like the pool needs to be the exact right size. We
>>> just need to avoid the "highmem problem" of having too little
>>> memory for kernel allocations.
>>>
>>
>> I am not sure I like the trend towards CMA that we are seeing,
>> reserving
>> huge buffers for specific users (and eventually even doing it
>> automatically).
>>
>> What we actually want is ZONE_MOVABLE with relaxed guarantees, such
>> that
>> anybody who requires large, unmovable allocations can use it.
>>
>> I once played with the idea of having ZONE_PREFER_MOVABLE, which
>> a) Is the primary choice for movable allocations
>> b) Is allowed to contain unmovable allocations (esp., gigantic pages)
>> c) Is the fallback for ZONE_NORMAL for unmovable allocations, instead
>> of
>> running out of memory
>>
>> If someone messes up the zone ratio, issues known from zone
>> imbalances
>> are avoided - large allocations simply become less likely to succeed.
>> In
>> contrast to ZONE_MOVABLE, memory offlining is not guaranteed to work.
> 
> I really like that idea. This will be easier to deal with than
> a "just the right size" CMA area, and seems like it would be
> pretty forgiving in both directions.
> 

Yes, and can be extended using memory hotplug.

> Keeping unmovable allocations
> contained to one part of memory
> should also make compaction within the ZONE_PREFER_MOVABLE area
> a lot easier than compaction for higher order allocations is
> today.
> 
> I suspect your proposal solves a lot of issues at once.
> 
> For (c) from your proposal, we could even claim a whole
> 2MB or even 1GB area at once for unmovable allocations,
> keeping those contained in a limited amount of physical
> memory again, to make life easier on compaction.
> 

Exactly, locally limiting unmovable allocations to a sane minimum.

(with some smart extra work, we could even convert ZONE_PREFER_MOVABLE
to ZONE_NORMAL, one memory section/block at a time where needed, that
direction always works. But that's very tricky.)

-- 
Thanks,

David / dhildenb