linux-kernel - Re: [PATCH 1/2] arm64: mem-model: add flatmem model for arm64

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAKv+Gu9aqR=E3TmbPDFEUC+Q13bAJTU5wVTTHkOr6aX6BZ1OVA@mail.gmail.com>
Date:	Mon, 11 Apr 2016 10:00:14 +0200
From:	Ard Biesheuvel <ard.biesheuvel@...aro.org>
To:	Chen Feng <puck.chen@...ilicon.com>
Cc:	Will Deacon <will.deacon@....com>, mhocko@...e.com,
	Laura Abbott <labbott@...hat.com>,
	Mark Rutland <mark.rutland@....com>,
	Dan Zhao <dan.zhao@...ilicon.com>,
	Yiping Xu <xuyiping@...ilicon.com>, puck.chen@...mail.com,
	albert.lubing@...ilicon.com,
	Catalin Marinas <catalin.marinas@....com>,
	suzhuangluan@...ilicon.com,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	linuxarm@...wei.com, "linux-mm@...ck.org" <linux-mm@...ck.org>,
	kirill.shutemov@...ux.intel.com,
	David Rientjes <rientjes@...gle.com>, oliver.fu@...ilicon.com,
	Andrew Morton <akpm@...ux-foundation.org>,
	robin.murphy@....com, yudongbin@...licon.com,
	"linux-arm-kernel@...ts.infradead.org" 
	<linux-arm-kernel@...ts.infradead.org>, saberlily.xia@...ilicon.com
Subject: Re: [PATCH 1/2] arm64: mem-model: add flatmem model for arm64

On 11 April 2016 at 09:55, Chen Feng <puck.chen@...ilicon.com> wrote:
> Hi Ard,
>
> On 2016/4/11 15:35, Ard Biesheuvel wrote:
>> On 11 April 2016 at 04:49, Chen Feng <puck.chen@...ilicon.com> wrote:
>>> Hi will,
>>> Thanks for review.
>>>
>>> On 2016/4/7 22:21, Will Deacon wrote:
>>>> On Tue, Apr 05, 2016 at 04:22:51PM +0800, Chen Feng wrote:
>>>>> We can reduce the memory allocated at mem-map
>>>>> by flatmem.
>>>>>
>>>>> currently, the default memory-model in arm64 is
>>>>> sparse memory. The mem-map array is not freed in
>>>>> this scene. If the physical address is too long,
>>>>> it will reserved too much memory for the mem-map
>>>>> array.
>>>>
>>>> Can you elaborate a bit more on this, please? We use the vmemmap, so any
>>>> spaces between memory banks only burns up virtual space. What exactly is
>>>> the problem you're seeing that makes you want to use flatmem (which is
>>>> probably unsuitable for the majority of arm64 machines).
>>>>
>>> The root cause we want to use flat-mem is the mam_map alloced in sparse-mem
>>> is not freed.
>>>
>>> take a look at here:
>>> arm64/mm/init.c
>>> void __init mem_init(void)
>>> {
>>> #ifndef CONFIG_SPARSEMEM_VMEMMAP
>>>         free_unused_memmap();
>>> #endif
>>> }
>>>
>>> Memory layout (3GB)
>>>
>>>  0             1.5G    2G             3.5G            4G
>>>  |              |      |               |              |
>>>  +--------------+------+---------------+--------------+
>>>  |    MEM       | hole |     MEM       |   IO (regs)  |
>>>  +--------------+------+---------------+--------------+
>>>
>>>
>>> Memory layout (4GB)
>>>
>>>  0                                    3.5G            4G    4.5G
>>>  |                                     |              |       |
>>>  +-------------------------------------+--------------+-------+
>>>  |                   MEM               |   IO (regs)  |  MEM  |
>>>  +-------------------------------------+--------------+-------+
>>>
>>> Currently, the sparse memory section is 1GB.
>>>
>>> 3GB ddr: the 1.5 ~2G and 3.5 ~ 4G are holes.
>>> 3GB ddr: the 3.5 ~ 4G and 4.5 ~ 5G are holes.
>>>
>>> This will alloc 1G/4K * (struct page) memory for mem_map array.
>>>
>>
>> No, this is incorrect. Sparsemem vmemmap only allocates struct pages
>> for memory regions that are actually populated.
>>
>> For instance, on the Foundation model with 4 GB of memory, you may see
>> something like this in the boot log
>>
>> [    0.000000]     vmemmap : 0xffffffbdc0000000 - 0xffffffbfc0000000
>> (     8 GB maximum)
>> [    0.000000]               0xffffffbdc0000000 - 0xffffffbde2000000
>> (   544 MB actual)
>>
>> but in reality, only the following regions have been allocated
>>
>> ---[ vmemmap start ]---
>> 0xffffffbdc0000000-0xffffffbdc2000000          32M       RW NX SHD AF
>>       BLK UXN MEM/NORMAL
>> 0xffffffbde0000000-0xffffffbde2000000          32M       RW NX SHD AF
>>       BLK UXN MEM/NORMAL
>> ---[ vmemmap end ]---
>>
>> so only 64 MB is used to back 4 GB of RAM with struct pages, which is
>> minimal. Moving to flatmem will not reduce the memory footprint at
>> all.
>
> Yes,but the populate is section, which is 1GB. Take a look at the above
> memory layout.
>
> The section 1G ~ 2G is a section. But 1.5G ~ 2G is a hole.
>
> The section 3G ~ 4G is a section. But 3.5G ~ 4G is a hole.
>>>  0             1.5G    2G             3.5G            4G
>>>  |              |      |               |              |
>>>  +--------------+------+---------------+--------------+
>>>  |    MEM       | hole |     MEM       |   IO (regs)  |
>>>  +--------------+------+---------------+--------------+
> The hole in 1.5G ~ 2G is also allocated mem-map array. And also with the 3.5G ~ 4G.
>

No, it is not. It may be covered by a section, but that does not mean
sparsemem vmemmap will actually allocate backing for it. The
granularity used by sparsemem vmemmap on a 4k pages kernel is 128 MB,
due to the fact that the backing is performed at PMD granularity.

Please, could you share the contents of the vmemmap section in
/sys/kernel/debug/kernel_page_tables of your system running with
sparsemem vmemmap enabled? You will need to set CONFIG_ARM64_PTDUMP=y