lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  PHC 
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Thu, 30 Apr 2020 18:09:35 -0700
From:   Josh Triplett <>
To:     Daniel Jordan <>
Cc:     Andrew Morton <>,
        Herbert Xu <>,
        Steffen Klassert <>,
        Alex Williamson <>,
        Alexander Duyck <>,
        Dan Williams <>,
        Dave Hansen <>,
        David Hildenbrand <>,
        Jason Gunthorpe <>,
        Jonathan Corbet <>,
        Kirill Tkhai <>,
        Michal Hocko <>, Pavel Machek <>,
        Pavel Tatashin <>,
        Peter Zijlstra <>,
        Randy Dunlap <>,
        Shile Zhang <>,
        Tejun Heo <>, Zi Yan <>,,,
Subject: Re: [PATCH 0/7] padata: parallelize deferred page init

On Thu, Apr 30, 2020 at 04:11:18PM -0400, Daniel Jordan wrote:
> Sometimes the kernel doesn't take full advantage of system memory
> bandwidth, leading to a single CPU spending excessive time in
> initialization paths where the data scales with memory size.
> Multithreading naturally addresses this problem, and this series is the
> first step.
> It extends padata, a framework that handles many parallel singlethreaded
> jobs, to handle multithreaded jobs as well by adding support for
> splitting up the work evenly, specifying a minimum amount of work that's
> appropriate for one helper thread to do, load balancing between helpers,
> and coordinating them.  More documentation in patches 4 and 7.
> The first user is deferred struct page init, a large bottleneck in
> kernel boot--actually the largest for us and likely others too.  This
> path doesn't require concurrency limits, resource control, or priority
> adjustments like future users will (vfio, hugetlb fallocate, munmap)
> because it happens during boot when the system is otherwise idle and
> waiting on page init to finish.
> This has been tested on a variety of x86 systems and speeds up kernel
> boot by 6% to 49% by making deferred init 63% to 91% faster.  Patch 6
> has detailed numbers.  Test results from other systems appreciated.
> This series is based on v5.6 plus these three from mmotm:
>   mm-call-touch_nmi_watchdog-on-max-order-boundaries-in-deferred-init.patch
>   mm-initialize-deferred-pages-with-interrupts-enabled.patch
>   mm-call-cond_resched-from-deferred_init_memmap.patch
> All of the above can be found in this branch:
>   git:// padata-mt-definit-v1

For the series (and the three prerequisite patches):

Tested-by: Josh Triplett <>

Thank you for writing this, and thank you for working towards
upstreaming it!

Powered by blists - more mailing lists