linux-kernel - [PATCH] iommu/dma: speedup memory allocation in __iommu_dma_alloc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date:   Tue, 6 Sep 2022 10:39:02 +0800
From:   Chen Wandun <chenwandun@...wei.com>
To:     <joro@...tes.org>, <will@...nel.org>,
        <iommu@...ts.linux-foundation.org>, <linux-kernel@...r.kernel.org>
CC:     <wangkefeng.wang@...wei.com>, <chenwandun@...wei.com>
Subject: [PATCH] iommu/dma: speedup memory allocation in __iommu_dma_alloc_pages

We found kcompactd was excessively running in Android, after
some debug, found some order-9 allocations in iommu/dma.

It splits contiguous page to single page in dma allocation,
that means it is not necessary to alloc contiguous page, what
is more, allocation for high order may cause direct memory
reclaim and compaction, result in poor performance.

In this patch, try to alloc memory by alloc_pages_bulk_array_node
first, speedup memory allocation by saving unnecessary direct
memory reclaim and compaction, fallback to original path when
failed, beside remove __GFP_DIRECT_RECLAIM for costly order.

Signed-off-by: Chen Wandun <chenwandun@...wei.com>
---
 drivers/iommu/dma-iommu.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index f90251572a5d..b8463934d806 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -720,7 +720,7 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev,
 		unsigned int count, unsigned long order_mask, gfp_t gfp)
 {
 	struct page **pages;
-	unsigned int i = 0, nid = dev_to_node(dev);
+	unsigned int i, nid = dev_to_node(dev);
 
 	order_mask &= (2U << MAX_ORDER) - 1;
 	if (!order_mask)
@@ -736,6 +736,11 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev,
 	/* It makes no sense to muck about with huge pages */
 	gfp &= ~__GFP_COMP;
 
+	i = alloc_pages_bulk_array_node(gfp, nid, count, pages);
+	if (count == i)
+		return pages;
+	count -= i;
+
 	while (count) {
 		struct page *page = NULL;
 		unsigned int order_size;
@@ -753,6 +758,10 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev,
 			order_size = 1U << order;
 			if (order_mask > order_size)
 				alloc_flags |= __GFP_NORETRY;
+
+			if (order > PAGE_ALLOC_COSTLY_ORDER)
+				alloc_flags &= ~__GFP_DIRECT_RECLAIM;
+
 			page = alloc_pages_node(nid, alloc_flags, order);
 			if (!page)
 				continue;
-- 
2.25.1