linux-kernel - [PATCH 35/35] Allow up to 4MB PCP lists due to compound pages

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1237196790-7268-36-git-send-email-mel@csn.ul.ie>
Date:	Mon, 16 Mar 2009 09:46:30 +0000
From:	Mel Gorman <mel@....ul.ie>
To:	Mel Gorman <mel@....ul.ie>,
	Linux Memory Management List <linux-mm@...ck.org>
Cc:	Pekka Enberg <penberg@...helsinki.fi>,
	Rik van Riel <riel@...hat.com>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	Christoph Lameter <cl@...ux-foundation.org>,
	Johannes Weiner <hannes@...xchg.org>,
	Nick Piggin <npiggin@...e.de>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	Lin Ming <ming.m.lin@...el.com>,
	Zhang Yanmin <yanmin_zhang@...ux.intel.com>,
	Peter Zijlstra <peterz@...radead.org>
Subject: [PATCH 35/35] Allow up to 4MB PCP lists due to compound pages

Compound pages from SLUB on the free lists can occupy a fair percentage of
the 512K that is currently allowed on the PCP lists. This can push out cache
hot order-0 pages even though the compound page may be relatively sparsely
used in the short term. This patch changes pcp->count to count pages (1
per page regardless of order) instead of accounting for the number of base
pages on the list. This keeps cache hot pages on the list at the cost of
the PCP lists being up to 4MB in size instead of 512K.

Signed-off-by: Mel Gorman <mel@....ul.ie>
---
 mm/page_alloc.c |   23 +++++++++--------------
 1 files changed, 9 insertions(+), 14 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 1ac4c3d..d5161cf 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -572,11 +572,10 @@ static inline int check_freepage_flags(struct page *page,
 static inline void rmv_pcp_page(struct per_cpu_pages *pcp, struct page *page)
 {
 	int migratetype = page_private(page);
-	int basepage_count = 1 << page->index;
 
 	list_del(&page->lru);
-	pcp->count -= basepage_count;
-	pcp->mocount[migratetype][page->index] -= basepage_count;
+	pcp->count--;
+	pcp->mocount[migratetype][page->index]--;
 }
 
 static inline void add_pcp_page(struct per_cpu_pages *pcp,
@@ -584,22 +583,20 @@ static inline void add_pcp_page(struct per_cpu_pages *pcp,
 					int cold)
 {
 	int migratetype = page_private(page);
-	int basepage_count = 1 << page->index;
 
 	if (cold)
 		list_add_tail(&page->lru, &pcp->lists[migratetype]);
 	else
 		list_add(&page->lru, &pcp->lists[migratetype]);
-	pcp->count += basepage_count;
-	pcp->mocount[migratetype][page->index] += basepage_count;
+	pcp->count++;
+	pcp->mocount[migratetype][page->index]++;
 }
 
 static inline void bulk_add_pcp_page(struct per_cpu_pages *pcp,
 					int migratetype, int order, int count)
 {
-	int basepage_count = count << order;
-	pcp->count += basepage_count;
-	pcp->mocount[migratetype][order] += basepage_count;
+	pcp->count += count;
+	pcp->mocount[migratetype][order] += count;
 }
 
 /*
@@ -627,9 +624,8 @@ static void free_pcppages_bulk(struct zone *zone, int count,
 
 	list = &pcp->lists[migratetype];
 	bulkcount = 1 + (count / (MIGRATE_PCPTYPES * 2));
-	while (freed < count) {
+	while (count--) {
 		struct page *page;
-		int thisfreed;
 
 		/*
 		 * Move to another migratetype if this list is depleted or
@@ -645,9 +641,8 @@ static void free_pcppages_bulk(struct zone *zone, int count,
 		/* Remove from list and update counters */
 		page = list_entry(list->prev, struct page, lru);
 		rmv_pcp_page(pcp, page);
-		thisfreed = 1 << page->index;
-		freed += thisfreed;
-		bulkcount -= thisfreed;
+		freed += 1 << page->index;
+		bulkcount--;
 
 		__free_one_page(page, zone, page->index, migratetype);
 	}
-- 
1.5.6.5

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/