linux-kernel - Re: [PATCH] page-allocator: Ensure that processes that have been OOM killed exit the page allocator (resend)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.DEB.2.00.0907161202500.27201@chino.kir.corp.google.com>
Date:	Thu, 16 Jul 2009 12:14:13 -0700 (PDT)
From:	David Rientjes <rientjes@...gle.com>
To:	Mel Gorman <mel@....ul.ie>
cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Nick Piggin <npiggin@...e.de>, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org
Subject: Re: [PATCH] page-allocator: Ensure that processes that have been
 OOM killed exit the page allocator (resend)

On Thu, 16 Jul 2009, Mel Gorman wrote:

> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 4b8552e..b381a6b 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -1830,8 +1830,6 @@ rebalance:
>  			if (order > PAGE_ALLOC_COSTLY_ORDER &&
>  						!(gfp_mask & __GFP_NOFAIL))
>  				goto nopage;
> -
> -			goto restart;
>  		}
>  	}
>  
> 

This isn't right (and not only because it'll add a compiler warning 
because `restart' is now unused).

This would immediately fail any allocation that triggered the oom killer 
and ended up being selected that isn't __GFP_NOFAIL, even if it would have 
succeeded without even killing any task simply because it allocates 
without watermarks.

It will also, coupled with your earlier patch, inappropriately warn about 
an infinite loop with __GFP_NOFAIL even though it hasn't even attempted to 
loop once since that decision is now handled by should_alloc_retry().

The liklihood of such an infinite loop, considering only one thread per 
system (or cpuset) can be TIF_MEMDIE at a time, is very low.  I've never 
seen memory reserves completely depleted such that the next high-priority 
allocation wouldn't succeed so that current could handle its pending 
SIGKILL.

You get the same behavior with my patch, but are allowed to try the high 
priority allocation again for the attempt that triggered the oom killer 
(and not only subsequent ones).
---
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1789,6 +1789,10 @@ rebalance:
 	if (p->flags & PF_MEMALLOC)
 		goto nopage;

+	/* Avoid allocations with no watermarks from looping endlessly */
+	if (test_thread_flag(TIF_MEMDIE) && !(gfp_mask & __GFP_NOFAIL))
+		goto nopage;
+
 	/* Try direct reclaim and then allocating */
 	page = __alloc_pages_direct_reclaim(gfp_mask, order,
 					zonelist, high_zoneidx,
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/