linux-kernel - Re: [PATCH 0/2] mm: Enable page parallel initialisation for Power

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <56DEE59F.7020602@gmail.com>
Date:	Wed, 9 Mar 2016 01:45:51 +1100
From:	Balbir Singh <bsingharora@...il.com>
To:	Li Zhang <zhlcindy@...il.com>, akpm@...ux-foundation.org,
	vbabka@...e.cz, mgorman@...hsingularity.net, mpe@...erman.id.au,
	khandual@...ux.vnet.ibm.com, aneesh.kumar@...ux.vnet.ibm.com
Cc:	linux-mm@...ck.org, linuxppc-dev@...ts.ozlabs.org,
	linux-kernel@...r.kernel.org,
	Li Zhang <zhlcindy@...ux.vnet.ibm.com>
Subject: Re: [PATCH 0/2] mm: Enable page parallel initialisation for Power



On 08/03/16 14:55, Li Zhang wrote:
> From: Li Zhang <zhlcindy@...ux.vnet.ibm.com>
>
> Uptream has supported page parallel initialisation for X86 and the
> boot time is improved greately. Some tests have been done for Power.
>
> Here is the result I have done with different memory size.
>
> * 4GB memory:
>     boot time is as the following: 
>     with patch vs without patch: 10.4s vs 24.5s
>     boot time is improved 57%
> * 200GB memory: 
>     boot time looks the same with and without patches.
>     boot time is about 38s
> * 32TB memory: 
>     boot time looks the same with and without patches 
>     boot time is about 160s.
>     The boot time is much shorter than X86 with 24TB memory.
>     From community discussion, it costs about 694s for X86 24T system.
>
> From code view, parallel initialisation improve the performance by
> deferring memory initilisation to kswap with N kthreads, it should
> improve the performance therotically. 
>
> From the test result, On X86, performance is improved greatly with huge
> memory. But on Power platform, it is improved greatly with less than 
> 100GB memory. For huge memory, it is not improved greatly. But it saves 
> the time with several threads at least, as the following information 
> shows(32TB system log):
>
> [   22.648169] node 9 initialised, 16607461 pages in 280ms
> [   22.783772] node 3 initialised, 23937243 pages in 410ms
> [   22.858877] node 6 initialised, 29179347 pages in 490ms
> [   22.863252] node 2 initialised, 29179347 pages in 490ms
> [   22.907545] node 0 initialised, 32049614 pages in 540ms
> [   22.920891] node 15 initialised, 32212280 pages in 550ms
> [   22.923236] node 4 initialised, 32306127 pages in 550ms
> [   22.923384] node 12 initialised, 32314319 pages in 550ms
> [   22.924754] node 8 initialised, 32314319 pages in 550ms
> [   22.940780] node 13 initialised, 33353677 pages in 570ms
> [   22.940796] node 11 initialised, 33353677 pages in 570ms
> [   22.941700] node 5 initialised, 33353677 pages in 570ms
> [   22.941721] node 10 initialised, 33353677 pages in 570ms
> [   22.941876] node 7 initialised, 33353677 pages in 570ms
> [   22.944946] node 14 initialised, 33353677 pages in 570ms
> [   22.946063] node 1 initialised, 33345485 pages in 580ms
>
> It saves the time about 550*16 ms at least, although it can be ignore to compare 
> the boot time about 160 seconds. What's more, the boot time is much shorter 
> on Power even without patches than x86 for huge memory machine. 
>
> So this patchset is still necessary to be enabled for Power. 
>
>

The patchset looks good, two questions

1. The patchset is still necessary for
    a. systems with smaller amount of RAM?
    b. Theoretically it improves boot time?
2. the pgdat->node_spanned_pages >> 8 sounds arbitrary
    On a system with 2TB*16 nodes, it would initialize about 8GB before calling deferred init?
    Don't we need at-least 32GB + space for other early hash allocations
    BTW, My expectation was that 32TB would imply 32GB+32GB of large hash allocations early on

Balbir Singh.