linux-kernel - [PATCH] mm/percpu.c: use smarter memory allocation for struct pcpu_alloc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 3 Oct 2017 16:57:44 -0400 (EDT)
From:   Nicolas Pitre <nicolas.pitre@...aro.org>
To:     Tejun Heo <tj@...nel.org>, Christoph Lameter <cl@...ux.com>
cc:     linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: [PATCH] mm/percpu.c: use smarter memory allocation for struct
 pcpu_alloc_info

This can be much smaller than a page on very small memory systems. 
Always rounding up the size to a page is wasteful in that case, and 
required alignment is smaller than the memblock default. Let's round 
things up to a page size only when the actual size is >= page size, and 
then it makes sense to page-align for a nicer allocation pattern.

Signed-off-by: Nicolas Pitre <nico@...aro.org>

diff --git a/mm/percpu.c b/mm/percpu.c
index 434844415d..fe37f85cc2 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -1410,13 +1410,17 @@ struct pcpu_alloc_info * __init pcpu_alloc_alloc_info(int nr_groups,
 	struct pcpu_alloc_info *ai;
 	size_t base_size, ai_size;
 	void *ptr;
-	int unit;
+	int unit, align;
 
-	base_size = ALIGN(sizeof(*ai) + nr_groups * sizeof(ai->groups[0]),
-			  __alignof__(ai->groups[0].cpu_map[0]));
+	align = __alignof__(ai->groups[0].cpu_map[0]);
+	base_size = ALIGN(sizeof(*ai) + nr_groups * sizeof(ai->groups[0]), align);
 	ai_size = base_size + nr_units * sizeof(ai->groups[0].cpu_map[0]);
+	if (ai_size >= PAGE_SIZE) {
+		ai_size = PFN_ALIGN(ai_size);
+		align = PAGE_SIZE;
+	}
 
-	ptr = memblock_virt_alloc_nopanic(PFN_ALIGN(ai_size), 0);
+	ptr = memblock_virt_alloc_nopanic(ai_size, align);
 	if (!ptr)
 		return NULL;
 	ai = ptr;
@@ -1428,7 +1432,7 @@ struct pcpu_alloc_info * __init pcpu_alloc_alloc_info(int nr_groups,
 		ai->groups[0].cpu_map[unit] = NR_CPUS;
 
 	ai->nr_groups = nr_groups;
-	ai->__ai_size = PFN_ALIGN(ai_size);
+	ai->__ai_size = ai_size;
 
 	return ai;
 }