linux-kernel - Re: [PATCH -mm] throttle direct reclaim when too many pages are isolated already

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20090716124255.3d601efb.kamezawa.hiroyu@jp.fujitsu.com>
Date:	Thu, 16 Jul 2009 12:42:55 +0900
From:	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
To:	Rik van Riel <riel@...hat.com>
Cc:	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	LKML <linux-kernel@...r.kernel.org>,
	linux-mm <linux-mm@...ck.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Wu Fengguang <fengguang.wu@...el.com>
Subject: Re: [PATCH -mm] throttle direct reclaim when too many pages are
 isolated already

On Wed, 15 Jul 2009 23:32:13 -0400
Rik van Riel <riel@...hat.com> wrote:

> KAMEZAWA Hiroyuki wrote:
> > On Wed, 15 Jul 2009 22:38:53 -0400
> > Rik van Riel <riel@...hat.com> wrote:
> > 
> >> When way too many processes go into direct reclaim, it is possible
> >> for all of the pages to be taken off the LRU.  One result of this
> >> is that the next process in the page reclaim code thinks there are
> >> no reclaimable pages left and triggers an out of memory kill.
> >>
> >> One solution to this problem is to never let so many processes into
> >> the page reclaim path that the entire LRU is emptied.  Limiting the
> >> system to only having half of each inactive list isolated for
> >> reclaim should be safe.
> >>
> >> Signed-off-by: Rik van Riel <riel@...hat.com>
> >> ---
> >> This patch goes on top of Kosaki's "Account the number of isolated pages"
> >> patch series.
> >>
> >>  mm/vmscan.c |   25 +++++++++++++++++++++++++
> >>  1 file changed, 25 insertions(+)
> >>
> >> Index: mmotm/mm/vmscan.c
> >> ===================================================================
> >> --- mmotm.orig/mm/vmscan.c	2009-07-08 21:37:01.000000000 -0400
> >> +++ mmotm/mm/vmscan.c	2009-07-08 21:39:02.000000000 -0400
> >> @@ -1035,6 +1035,27 @@ int isolate_lru_page(struct page *page)
> >>  }
> >>  
> >>  /*
> >> + * Are there way too many processes in the direct reclaim path already?
> >> + */
> >> +static int too_many_isolated(struct zone *zone, int file)
> >> +{
> >> +	unsigned long inactive, isolated;
> >> +
> >> +	if (current_is_kswapd())
> >> +		return 0;
> >> +
> >> +	if (file) {
> >> +		inactive = zone_page_state(zone, NR_INACTIVE_FILE);
> >> +		isolated = zone_page_state(zone, NR_ISOLATED_FILE);
> >> +	} else {
> >> +		inactive = zone_page_state(zone, NR_INACTIVE_ANON);
> >> +		isolated = zone_page_state(zone, NR_ISOLATED_ANON);
> >> +	}
> >> +
> >> +	return isolated > inactive;
> >> +}
> > 
> > Why this means "too much" ?
> 
> This triggers when most of the pages in the zone (in the
> category we are trying to reclaim) have already been
> isolated by other tasks, to be reclaimed.  There is really
> no need to reclaim all of the pages in a zone all at once,
> plus it can cause false OOM kills.
> 
> Setting the threshold at isolated > inactive gives us
> enough of a safety margin that we can do this comparison
> lockless.
> 
> > And, could you put this check under scanning_global_lru(sc) ?
> 
> When most of the pages in a zone have been isolated from
> the LRU already by page reclaim, chances are that cgroup
> reclaim will suffer from the same problem.
> 
> Am I overlooking something?
> 
Reclaim from cgorup doesn't come from memory shortage but from
"it hits limit". Then, it doen't necessary to reclaim pages from
this zone. fallback to other zone is always ok.
This will trigger unnecessary wait, I think.

Thanks,
-Kame



> -- 
> All rights reversed.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/