linux-kernel - Re: [ 101/175] mm: vmscan: forcibly scan highmem if there are too many buffer

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20120406061248.GA16549@pcnci2.linuxbox.cz>
Date:	Fri, 6 Apr 2012 08:13:05 +0200
From:	Nikola Ciprich <nikola.ciprich@...uxbox.cz>
To:	Mel Gorman <mgorman@...e.de>
Cc:	Ben Hutchings <ben@...adent.org.uk>, linux-kernel@...r.kernel.org,
	stable@...r.kernel.org, torvalds@...ux-foundation.org,
	akpm@...ux-foundation.org, alan@...rguk.ukuu.org.uk,
	Stuart Foster <smf.linux@...world.com>,
	Hugh Dickins <hughd@...gle.com>,
	Johannes Weiner <hannes@...xchg.org>,
	Rik van Riel <riel@...hat.com>,
	Christoph Lameter <cl@...ux.com>,
	Greg KH <gregkh@...uxfoundation.org>,
	nikola.ciprich@...uxbox.cz
Subject: Re: [ 101/175] mm: vmscan: forcibly scan highmem if there are too
	many buffer_heads pinning highmem

Hi, sorry it took me a bit longer.

here's my backport, compiles fine, kernel boots without any problems.
please review.
n.

Signed-off-by: Nikola Ciprich <nikola.ciprich@...uxbox.cz

(backport of upstream commit cc715d99e529d470dde2f33a6614f255adea71f3)

    mm: vmscan: forcibly scan highmem if there are too many buffer_heads pinning highmem
    
    Stuart Foster reported on bugzilla that copying large amounts of data
    from NTFS caused an OOM kill on 32-bit X86 with 16G of memory.  Andrew
    Morton correctly identified that the problem was NTFS was using 512
    blocks meaning each page had 8 buffer_heads in low memory pinning it.
    
    In the past, direct reclaim used to scan highmem even if the allocating
    process did not specify __GFP_HIGHMEM but not any more.  kswapd no longer
    will reclaim from zones that are above the high watermark.  The intention
    in both cases was to minimise unnecessary reclaim.  The downside is on
    machines with large amounts of highmem that lowmem can be fully consumed
    by buffer_heads with nothing trying to free them.
    
    The following patch is based on a suggestion by Andrew Morton to extend
    the buffer_heads_over_limit case to force kswapd and direct reclaim to
    scan the highmem zone regardless of the allocation request or watermarks.
    
    Addresses https://bugzilla.kernel.org/show_bug.cgi?id=42578

---

diff -Naur linux-3.0/mm/vmscan.c linux-3.0-cc715d99e529d470dde2f33a6614f255adea71f3-backport/mm/vmscan.c
--- linux-3.0/mm/vmscan.c	2012-04-05 23:09:28.364000004 +0200
+++ linux-3.0-cc715d99e529d470dde2f33a6614f255adea71f3-backport/mm/vmscan.c	2012-04-05 23:25:30.989968627 +0200
@@ -1581,6 +1581,14 @@
 			putback_lru_page(page);
 			continue;
 		}
+		
+		if (unlikely(buffer_heads_over_limit)) {
+			if (page_has_private(page) && trylock_page(page)) {
+				if (page_has_private(page))
+					try_to_release_page(page, 0);
+				unlock_page(page);
+			}
+		}
 
 		if (page_referenced(page, 0, sc->mem_cgroup, &vm_flags)) {
 			nr_rotated += hpage_nr_pages(page);
@@ -2053,6 +2061,14 @@
 	struct zoneref *z;
 	struct zone *zone;
 
+	/*
+	 * If the number of buffer_heads in the machine exceeds the maximum
+	 * allowed level, force direct reclaim to scan the highmem zone as
+	 * highmem pages could be pinning lowmem pages storing buffer_heads
+	 */
+	if (buffer_heads_over_limit)
+		sc->gfp_mask |= __GFP_HIGHMEM;
+
 	for_each_zone_zonelist_nodemask(zone, z, zonelist,
 			gfp_zone(sc->gfp_mask), sc->nodemask) {
 		if (!populated_zone(zone))
@@ -2514,7 +2530,8 @@
 				(zone->present_pages +
 					KSWAPD_ZONE_BALANCE_GAP_RATIO-1) /
 				KSWAPD_ZONE_BALANCE_GAP_RATIO);
-			if (!zone_watermark_ok_safe(zone, order,
+			if ((buffer_heads_over_limit && is_highmem_idx(i)) ||
+				!zone_watermark_ok_safe(zone, order,
 					high_wmark_pages(zone) + balance_gap,
 					end_zone, 0)) {
 				shrink_zone(priority, zone, &sc);
@@ -2543,6 +2560,17 @@
 				continue;
 			}
 
+			/*
+			 * If the number of buffer_heads in the machine
+			 * exceeds the maximum allowed level and this node
+			 * has a highmem zone, force kswapd to reclaim from
+			 * it to relieve lowmem pressure.
+			 */
+			if (buffer_heads_over_limit && is_highmem_idx(i)) {
+				end_zone = i;
+				break;
+			}
+
 			if (!zone_watermark_ok_safe(zone, order,
 					high_wmark_pages(zone), end_zone, 0)) {
 				all_zones_ok = 0;

Content of type "application/pgp-signature" skipped