linux-kernel - Re: [PATCH] arm64: configurable sparsemem section size

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CA+CK2bCD11x64pJj5gSnsu5jsUqJyU6o+=J4K8oYAsHqz9ULqQ@mail.gmail.com>
Date:   Wed, 24 Apr 2019 15:48:44 -0400
From:   Pavel Tatashin <patatash@...ux.microsoft.com>
To:     Anshuman Khandual <anshuman.khandual@....com>
Cc:     James Morris <jmorris@...ei.org>, Sasha Levin <sashal@...nel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        linux-mm <linux-mm@...ck.org>,
        linux-nvdimm <linux-nvdimm@...ts.01.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Michal Hocko <mhocko@...e.com>,
        Dave Hansen <dave.hansen@...ux.intel.com>,
        Dan Williams <dan.j.williams@...el.com>,
        Keith Busch <keith.busch@...el.com>,
        Vishal L Verma <vishal.l.verma@...el.com>,
        Dave Jiang <dave.jiang@...el.com>,
        Ross Zwisler <zwisler@...nel.org>,
        Tom Lendacky <thomas.lendacky@....com>,
        "Huang, Ying" <ying.huang@...el.com>,
        Fengguang Wu <fengguang.wu@...el.com>,
        Borislav Petkov <bp@...e.de>,
        Bjorn Helgaas <bhelgaas@...gle.com>,
        Yaowei Bai <baiyaowei@...s.chinamobile.com>,
        Takashi Iwai <tiwai@...e.de>,
        Jérôme Glisse <jglisse@...hat.com>,
        catalin.marinas@....com, Will Deacon <will.deacon@....com>,
        rppt@...ux.vnet.ibm.com,
        Ard Biesheuvel <ard.biesheuvel@...aro.org>,
        andrew.murray@....com, james.morse@....com,
        Marc Zyngier <marc.zyngier@....com>, sboyd@...nel.org,
        linux-arm-kernel@...ts.infradead.org
Subject: Re: [PATCH] arm64: configurable sparsemem section size

On Wed, Apr 24, 2019 at 5:07 AM Anshuman Khandual
<anshuman.khandual@....com> wrote:
>
> On 04/24/2019 02:08 AM, Pavel Tatashin wrote:
> > sparsemem section size determines the maximum size and alignment that
> > is allowed to offline/online memory block. The bigger the size the less
> > the clutter in /sys/devices/system/memory/*. On the other hand, however,
> > there is less flexability in what granules of memory can be added and
> > removed.
>
> Is there any scenario where less than a 1GB needs to be added on arm64 ?

Yes, DAX hotplug loses 1G of memory without allowing smaller sections.
Machines on which we are going to be using this functionality have 8G
of System RAM, therefore losing 1G is a big problem.

For details about using scenario see this cover letter:
https://lore.kernel.org/lkml/20190421014429.31206-1-pasha.tatashin@soleen.com/

>
> >
> > Recently, it was enabled in Linux to hotadd persistent memory that
> > can be either real NV device, or reserved from regular System RAM
> > and has identity of devdax.
>
> devdax (even ZONE_DEVICE) support has not been enabled on arm64 yet.

Correct, I use your patches to enable ZONE_DEVICE, and  thus devdax on ARM64:
https://lore.kernel.org/lkml/1554265806-11501-1-git-send-email-anshuman.khandual@arm.com/

>
> >
> > The problem is that because ARM64's section size is 1G, and devdax must
> > have 2M label section, the first 1G is always missed when device is
> > attached, because it is not 1G aligned.
>
> devdax has to be 2M aligned ? Does Linux enforce that right now ?

Unfortunately, there is no way around this. Part of the memory can be
reserved as persistent memory via device tree.
        memory@...00000 {
                device_type = "memory";
                reg = < 0x00000000 0x40000000
                        0x00000002 0x00000000 >;
        };

        pmem@...000000 {
                compatible = "pmem-region";
                reg = <0x00000001 0xc0000000
                       0x00000000 0x80000000>;
                volatile;
                numa-node-id = <0>;
        };

So, while pmem is section aligned, as it should be, the dax device is
going to be pmem start address + label size, which is 2M. The actual
DAX device starts at:
0x1c0000000 + 2M.

Because section size is 1G, the hotplug will able to add only memory
starting from
0x1c0000000 + 1G

> 27 and 28 do not even compile for ARM64_64_PAGES because of MAX_ORDER and
> SECTION_SIZE mismatch.

Can you please elaborate what configs are you using? I have no
problems compiling with 27 and 28 bit.

Thank you,
Pasha