Message-ID: <cd0e727d-8ad6-2d74-55f5-498394aae297@redhat.com>
Date:   Wed, 8 Jun 2022 12:08:20 +0200
From:   David Hildenbrand <david@...hat.com>
To:     Mike Rapoport <rppt@...nel.org>, mawupeng <mawupeng1@...wei.com>
Cc:     ardb@...nel.org, corbet@....net, will@...nel.org,
        catalin.marinas@....com, tglx@...utronix.de, mingo@...hat.com,
        bp@...en8.de, dave.hansen@...ux.intel.com, x86@...nel.org,
        hpa@...or.com, dvhart@...radead.org, andy@...radead.org,
        akpm@...ux-foundation.org, paul.walmsley@...ive.com,
        palmer@...belt.com, aou@...s.berkeley.edu, paulmck@...nel.org,
        keescook@...omium.org, songmuchun@...edance.com,
        rdunlap@...radead.org, damien.lemoal@...nsource.wdc.com,
        swboyd@...omium.org, wei.liu@...nel.org, robin.murphy@....com,
        anshuman.khandual@....com, thunder.leizhen@...wei.com,
        wangkefeng.wang@...wei.com, gpiccoli@...lia.com,
        chenhuacai@...nel.org, geert@...ux-m68k.org, chenzhou10@...wei.com,
        vijayb@...ux.microsoft.com, linux-doc@...r.kernel.org,
        linux-kernel@...r.kernel.org, linux-arm-kernel@...ts.infradead.org,
        linux-efi@...r.kernel.org, platform-driver-x86@...r.kernel.org,
        linux-mm@...ck.org, linux-riscv@...ts.infradead.org
Subject: Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

On 08.06.22 12:02, Mike Rapoport wrote:
> On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
>>
>> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
>>> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@...hat.com> wrote:
>>>>
>>>> On 07.06.22 11:38, Wupeng Ma wrote:
>>>>> From: Ma Wupeng <mawupeng1@...wei.com>
>>>>>
>>>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
>>>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
>>>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
>>>>> the lower 4G range has some non-mirrored memory.
>>>>>
>>>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
>>>>> reinstalled if the origin memblock has this flag.
>>>>>
>>>>> Signed-off-by: Ma Wupeng <mawupeng1@...wei.com>
>>>>> ---
>>>>>   arch/arm64/mm/init.c     |  9 +++++++++
>>>>>   include/linux/memblock.h |  1 +
>>>>>   mm/memblock.c            | 20 ++++++++++++++++++++
>>>>>   3 files changed, 30 insertions(+)
>>>>>
>>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>>>>> index 339ee84e5a61..11641f924d08 100644
>>>>> --- a/arch/arm64/mm/init.c
>>>>> +++ b/arch/arm64/mm/init.c
>>>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
>>>>>                        "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
>>>>>                        phys_initrd_size = 0;
>>>>>                } else {
>>>>> +                     int flags, ret;
>>>>> +
>>>>> +                     ret = memblock_get_flags(base, &flags);
>>>>> +                     if (ret)
>>>>> +                             flags = 0;
>>>>> +
>>>>>                        memblock_remove(base, size); /* clear MEMBLOCK_ flags */
>>>>>                        memblock_add(base, size);
>>>>>                        memblock_reserve(base, size);
>>>>
>>>> Can you explain why we're removing+re-adding here exactly? Is it just to
>>>> clear flags as the comment indicates?
>>>>
>>>
>>> This should only happen if the placement of the initrd conflicts with
>>> a mem= command line parameter or it is not covered by memblock for
>>> some other reason.
>>>
>>> IOW, this should never happen, and if re-memblock_add'ing this memory
>>> unconditionally is causing problems, we should fix that instead of
>>> working around it.
>>
>> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
>> the 4G limit to test this scenario(just for testing, I have trouble to boot
>> qemu with initrd enabled and memory below 4G are all mirror memory).
>>
>> Re-memblock_add'ing this memory unconditionally seems fine but clear all
>> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
>>
>>>
>>>> If it's really just about clearing flags, I wonder if we rather want to
>>>> have an interface that does exactly that, and hides the way this is
>>>> actually implemented (obtain flags, remove, re-add ...), internally.
>>>>
>>>> But most probably there is more magic in the code and clearing flags
>>>> isn't all it ends up doing.
>>>>
>>>
>>> I don't remember exactly why we needed to clear the flags, but I think
>>> it had to do with some corner case we hit when the initrd was
>>> partially covered.
>> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
>> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
>> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
>>
>> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
>> MEMBLOCK_NOMAP of this initrd memblock.
>> So maybe the usage of memblock_remove() is just to avoid introducing new
>> function(memblock_clear_nomap)?
>>
>> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
>> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
>> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?
> 
> AFAICT, there are two corner cases that re-adding initrd memory covers:
> * initrd memory is not a part of the memory reported to memblock, either
> because of firmware weirdness or because it was cut out with mem=
> * initrd memory overlaps a NOMAP region
> 
> So to make sure initrd memory is mapped properly and retains
> MEMBLOCK_MIRROR I think the best we can do is
> 
> 	memblock_add();
> 	memblock_clear_nomap();
> 	memblock_reserve();

Would simply detect+rejecting to boot on such setups be an option? The
replies so far indicate to me that this is rather a corner case than a
reasonable use case.

-- 
Thanks,

David / dhildenb

Powered by blists - more mailing lists