linux-kernel - Re: [PATCH v4 6/7] hugetlb: parallelize 2M hugetlb allocation and initialization

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 22 Jan 2024 15:10:07 +0800
From: Muchun Song <muchun.song@...ux.dev>
To: Gang Li <gang.li@...ux.dev>, David Hildenbrand <david@...hat.com>,
 David Rientjes <rientjes@...gle.com>, Mike Kravetz
 <mike.kravetz@...cle.com>, Andrew Morton <akpm@...ux-foundation.org>,
 Tim Chen <tim.c.chen@...ux.intel.com>
Cc: linux-mm@...ck.org, linux-kernel@...r.kernel.org,
 ligang.bdlg@...edance.com
Subject: Re: [PATCH v4 6/7] hugetlb: parallelize 2M hugetlb allocation and
 initialization



On 2024/1/18 20:39, Gang Li wrote:
> By distributing both the allocation and the initialization tasks across
> multiple threads, the initialization of 2M hugetlb will be faster,
> thereby improving the boot speed.
>
> Here are some test results:
>          test          no patch(ms)   patched(ms)   saved
>   ------------------- -------------- ------------- --------
>    256c2t(4 node) 2M           3336          1051   68.52%
>    128c1t(2 node) 2M           1943           716   63.15%
>
> Signed-off-by: Gang Li <gang.li@...ux.dev>
> Tested-by: David Rientjes <rientjes@...gle.com>
> ---
>   mm/hugetlb.c | 70 ++++++++++++++++++++++++++++++++++++++--------------
>   1 file changed, 52 insertions(+), 18 deletions(-)
>
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index effe5539e545..9b348ba418f5 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -35,6 +35,7 @@
>   #include <linux/delayacct.h>
>   #include <linux/memory.h>
>   #include <linux/mm_inline.h>
> +#include <linux/padata.h>
>   
>   #include <asm/page.h>
>   #include <asm/pgalloc.h>
> @@ -3510,43 +3511,76 @@ static void __init hugetlb_hstate_alloc_pages_errcheck(unsigned long allocated,
>   	}
>   }
>   
> -static unsigned long __init hugetlb_gigantic_pages_alloc_boot(struct hstate *h)
> +static void __init hugetlb_alloc_node(unsigned long start, unsigned long end, void *arg)
>   {
> -	unsigned long i;
> +	struct hstate *h = (struct hstate *)arg;
> +	int i, num = end - start;
> +	nodemask_t node_alloc_noretry;
> +	unsigned long flags;
> +	int next_node = 0;

This should be first_online_node which may be not zero.

>   
> -	for (i = 0; i < h->max_huge_pages; ++i) {
> -		if (!alloc_bootmem_huge_page(h, NUMA_NO_NODE))
> +	/* Bit mask controlling how hard we retry per-node allocations.*/
> +	nodes_clear(node_alloc_noretry);
> +
> +	for (i = 0; i < num; ++i) {
> +		struct folio *folio = alloc_pool_huge_folio(h, &node_states[N_MEMORY],
> +						&node_alloc_noretry, &next_node);
> +		if (!folio)
>   			break;
> +		spin_lock_irqsave(&hugetlb_lock, flags);

I suspect there will more contention on this lock when parallelizing.
I want to know why you chose to drop prep_and_add_allocated_folios()
call in the original hugetlb_pages_alloc_boot()?

> +		__prep_account_new_huge_page(h, folio_nid(folio));
> +		enqueue_hugetlb_folio(h, folio);
> +		spin_unlock_irqrestore(&hugetlb_lock, flags);
>   		cond_resched();
>   	}
> +}
>   
> -	return i;
> +static void __init hugetlb_vmemmap_optimize_node(unsigned long start, unsigned long end, void *arg)
> +{
> +	struct hstate *h = (struct hstate *)arg;
> +	int nid = start;
> +
> +	hugetlb_vmemmap_optimize_folios(h, &h->hugepage_freelists[nid]);
>   }
>   
> -static unsigned long __init hugetlb_pages_alloc_boot(struct hstate *h)
> +static unsigned long __init hugetlb_gigantic_pages_alloc_boot(struct hstate *h)
>   {
>   	unsigned long i;
> -	struct folio *folio;
> -	LIST_HEAD(folio_list);
> -	nodemask_t node_alloc_noretry;
> -
> -	/* Bit mask controlling how hard we retry per-node allocations.*/
> -	nodes_clear(node_alloc_noretry);
>   
>   	for (i = 0; i < h->max_huge_pages; ++i) {
> -		folio = alloc_pool_huge_folio(h, &node_states[N_MEMORY],
> -						&node_alloc_noretry);
> -		if (!folio)
> +		if (!alloc_bootmem_huge_page(h, NUMA_NO_NODE))
>   			break;
> -		list_add(&folio->lru, &folio_list);
>   		cond_resched();
>   	}
>   
> -	prep_and_add_allocated_folios(h, &folio_list);
> -
>   	return i;
>   }
>   
> +static unsigned long __init hugetlb_pages_alloc_boot(struct hstate *h)
> +{
> +	struct padata_mt_job job = {
> +		.fn_arg		= h,
> +		.align		= 1,
> +		.numa_aware	= true
> +	};
> +
> +	job.thread_fn	= hugetlb_alloc_node;
> +	job.start	= 0;
> +	job.size	= h->max_huge_pages;
> +	job.min_chunk	= h->max_huge_pages / num_node_state(N_MEMORY) / 2;
> +	job.max_threads	= num_node_state(N_MEMORY) * 2;

I am curious the magic number of 2 used in assignments of ->min_chunk
and ->max_threads, does it from your experiment? I thinke it should
be a comment here.

And I am also sceptical about the optimization for a small amount of
allocation of hugepages. Given 4 hugepags needed to be allocated on UMA
system, job.min_chunk will be 2, job.max_threads will be 2. Then, 2
workers will be scheduled, however each worker will just allocate 2 pages,
how much the cost of scheduling? What if allocate 4 pages in single
worker? Do you have any numbers on parallelism vs non-parallelism in
a small allocation case? If we cannot gain from this case, I think we shold
assign a reasonable value to ->min_chunk based on experiment.

Thanks.

> +	padata_do_multithreaded(&job);
> +
> +	job.thread_fn	= hugetlb_vmemmap_optimize_node;
> +	job.start	= 0;
> +	job.size	= num_node_state(N_MEMORY);
> +	job.min_chunk	= 1;
> +	job.max_threads	= num_node_state(N_MEMORY);
> +	padata_do_multithreaded(&job);
> +
> +	return h->nr_huge_pages;
> +}
> +
>   /*
>    * NOTE: this routine is called in different contexts for gigantic and
>    * non-gigantic pages.