linux-kernel - Re: [PATCH 0/3] OOM detection rework v4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160302095056.GB26701@dhcp22.suse.cz>
Date:	Wed, 2 Mar 2016 10:50:56 +0100
From:	Michal Hocko <mhocko@...nel.org>
To:	Joonsoo Kim <iamjoonsoo.kim@....com>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Hugh Dickins <hughd@...gle.com>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Johannes Weiner <hannes@...xchg.org>,
	Mel Gorman <mgorman@...e.de>,
	David Rientjes <rientjes@...gle.com>,
	Tetsuo Handa <penguin-kernel@...ove.sakura.ne.jp>,
	Hillf Danton <hillf.zj@...baba-inc.com>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	linux-mm@...ck.org, LKML <linux-kernel@...r.kernel.org>,
	Sergey Senozhatsky <sergey.senozhatsky.work@...il.com>
Subject: Re: [PATCH 0/3] OOM detection rework v4

On Wed 02-03-16 11:19:54, Joonsoo Kim wrote:
> On Mon, Feb 29, 2016 at 10:02:13PM +0100, Michal Hocko wrote:
[...]
> > > +	/*
> > > +	 * OK, so the watermak check has failed. Make sure we do all the
> > > +	 * retries for !costly high order requests and hope that multiple
> > > +	 * runs of compaction will generate some high order ones for us.
> > > +	 *
> > > +	 * XXX: ideally we should teach the compaction to try _really_ hard
> > > +	 * if we are in the retry path - something like priority 0 for the
> > > +	 * reclaim
> > > +	 */
> > > +	if (order && order <= PAGE_ALLOC_COSTLY_ORDER)
> > > +		return true;
> > > +
> > >  	return false;
> 
> This seems not a proper fix. Checking watermark with high order has
> another meaning that there is high order page or not. This isn't
> what we want here.

Why not? Why should we retry the reclaim if we do not have >=order page
available? Reclaim itself doesn't guarantee any of the freed pages will
form the requested order. The ordering on the LRU lists is pretty much
random wrt. pfn ordering. On the other hand if we have a page available
which is just hidden by watermarks then it makes perfect sense to retry
and free even order-0 pages.

> So, following fix is needed.

> 'if (order)' check isn't needed. It is used to clarify the meaning of
> this fix. You can remove it.
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 1993894..8c80375 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -3125,6 +3125,10 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order,
>         if (order > PAGE_ALLOC_COSTLY_ORDER && !(gfp_mask & __GFP_REPEAT))
>                 return false;
>  
> +       /* To check whether compaction is available or not */
> +       if (order)
> +               order = 0;
> +

This would enforce the order 0 wmark check which is IMHO not correct as
per above.

>         /*
>          * Keep reclaiming pages while there is a chance this will lead
>          * somewhere.  If none of the target zones can satisfy our allocation
> 
> > >  }
> > >  
> > > @@ -3281,11 +3293,11 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
> > >  		goto noretry;
> > >  
> > >  	/*
> > > -	 * Costly allocations might have made a progress but this doesn't mean
> > > -	 * their order will become available due to high fragmentation so do
> > > -	 * not reset the no progress counter for them
> > > +	 * High order allocations might have made a progress but this doesn't
> > > +	 * mean their order will become available due to high fragmentation so
> > > +	 * do not reset the no progress counter for them
> > >  	 */
> > > -	if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER)
> > > +	if (did_some_progress && !order)
> > >  		no_progress_loops = 0;
> > >  	else
> > >  		no_progress_loops++;
> 
> This unconditionally increases no_progress_loops for high order
> allocation, so, after 16 iterations, it will fail. If compaction isn't
> enabled in Kconfig, 16 times reclaim attempt would not be sufficient
> to make high order page. Should we consider this case also?

How many retries would help? I do not think any number will work
reliably. Configurations without compaction enabled are asking for
problems by definition IMHO. Relying on order-0 reclaim for high order
allocations simply cannot work.

-- 
Michal Hocko
SUSE Labs