linux-kernel - Re: [PATCH v5 1/5] mm,memory_hotplug: Allocate memmap from the added memory range

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <31110e58-c99a-8dee-6f6e-98f456b77759@redhat.com>
Date:   Thu, 25 Mar 2021 13:40:45 +0100
From:   David Hildenbrand <david@...hat.com>
To:     Michal Hocko <mhocko@...e.com>
Cc:     Oscar Salvador <osalvador@...e.de>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Anshuman Khandual <anshuman.khandual@....com>,
        Vlastimil Babka <vbabka@...e.cz>,
        Pavel Tatashin <pasha.tatashin@...een.com>, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH v5 1/5] mm,memory_hotplug: Allocate memmap from the added
 memory range

On 25.03.21 13:35, Michal Hocko wrote:
> On Thu 25-03-21 12:08:43, David Hildenbrand wrote:
>> On 25.03.21 11:55, Oscar Salvador wrote:
>>> On Thu, Mar 25, 2021 at 10:17:33AM +0100, Michal Hocko wrote:
>>>> Why do you think it is wrong to initialize/account pages when they are
>>>> used? Keep in mind that offline pages are not used until they are
>>>> onlined. But vmemmap pages are used since the vmemmap is established
>>>> which happens in the hotadd stage.
>>>
>>> Yes, that is true.
>>> vmemmap pages are used right when we populate the vmemmap space.
>>>
>>
>> Note: I once herd of a corner-case use case where people offline memory
>> blocks to then use the "free" memory via /dev/mem for other purposes ("large
>> physical memory"). Not that I encourage such use cases, but they would be
>> fundamentally broken if the vmemmap ends up on offline memory and is
>> supposed to keep its state ...
> 
> I am not aware of such a use case, it surely sounds, ehm creative, but
> nothing really new. But such a usecase sounds quite incompatible with
> this feature whether we account vmemmap at hotadd or hotremove because
> they would need to understand that part of the memory they have hotadded
> is not useable.

I think they can use it just fine via /dev/mem, which explicitly avoids 
any kind of "struct page" references IIRC. They would be overwriting the 
vmemmap, but that part scan happily read/write until onlining, where the 
vmemmap would get reinitialized and set online - from which point on 
pfn_to_online_page() would succeed also on the vmemmap itself.

> 
> [...]
>>> - When moving the initialization/accounting to hot-add/hot-remove,
>>>     the section containing the vmemmap pages will remain offline.
>>>     It might get onlined once the pages get online in online_pages(),
>>>     or not if vmemmap pages span a whole section.
>>>     I remember (but maybe David rmemeber better) that that was a problem
>>>     wrt. pfn_to_online_page() and hybernation/kdump.
>>>     So, if that is really a problem, we would have to care of ot setting
>>>     the section to the right state.
>>
>> Good memory. Indeed, hibernation/kdump won't save the state of the vmemmap,
>> because the memory is marked as offline and, thus, logically without any
>> valuable content.
> 
> Could you point me to the respective hibernation code please? I always
> get lost in that area. Anyway, we do have the same problem even if the
> whole accounting is handled during {on,off}lining, no?

kernel/power/snapshot.c:saveable_page().

> 
> I am not really worried about kdump though. As the memory is offline
> then losing itse vmemmap is a mere annoyance.

Yep, kdump was relevant in our previous discussions when we were talking 
about memory blocks having their altmap located on other memory blocks.

> 
> 
>>> - AFAICS, doing all the above brings us to former times were some
>>>     initialization/accounting was done in a previous stage, and I remember
>>>     it was pushed hard to move those in online/offline_pages().
>>>     Are we ok with that?
>>>     As I said, we might have to set the right zone in hot-add stage, as
>>>     otherwise problems might come up.
>>>     Being that case, would not that also be conflating different concepts
>>>     at a wrong phases?
>>>
>>
>> I expressed my opinion already, no need to repeat. Sub-section online maps
>> would make it cleaner, but I am still not convinced we want/need that.
> 
> Nah, subsections are more tricky than necessary and if we can live
> without them and have it just as pmem weirdness is more than enough ;)

Yes, absolutely :)

-- 
Thanks,

David / dhildenb