linux-kernel - Re: Found the commit that causes the OOMs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090701035415.GA22364@localhost>
Date:	Wed, 1 Jul 2009 11:54:15 +0800
From:	Wu Fengguang <fengguang.wu@...il.com>
To:	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>
Cc:	David Woodhouse <dwmw2@...radead.org>,
	David Howells <dhowells@...hat.com>,
	Minchan Kim <minchan.kim@...il.com>,
	Mel Gorman <mel@....ul.ie>,
	Johannes Weiner <hannes@...xchg.org>,
	"riel@...hat.com" <riel@...hat.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	LKML <linux-kernel@...r.kernel.org>,
	Christoph Lameter <cl@...ux-foundation.org>,
	"peterz@...radead.org" <peterz@...radead.org>,
	"tytso@....edu" <tytso@....edu>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"elladan@...imo.com" <elladan@...imo.com>,
	"npiggin@...e.de" <npiggin@...e.de>,
	"Barnes, Jesse" <jesse.barnes@...el.com>
Subject: Re: Found the commit that causes the OOMs

On Wed, Jul 01, 2009 at 11:51:54AM +0900, KOSAKI Motohiro wrote:
> > > > What is "hidden" anon pages?
> > > > each shrink_{in}active_list isolate 32 pages from lru. it mean anon or file lru
> > > > accounting decrease temporary.
> > > > 
> > > > if system have plenty thread or process, heavy memory pressure makes 
> > > > #-of-thread x 32pages isolation.
> > > > 
> > > > msgctl11 makes >10K processes.
> > > 
> > > More exactly, ~16K processes:
> > > 
> > >         msgctl11    0  INFO  :  Using upto 16298 pids
> > > 
> > > So the maximum number of isolated pages is 16K * 32 = 512K, or 2GiB.
> > > 
> > > > I have debugging patch for this case.
> > > > Wu, Can you please try this patch?
> > > 
> > > OK. But the OOM is not quite reproducible. Sometimes it produces these
> > > messages:
> > 
> > This time I got the OOM: there are 69817 isolated pages (just as expected)!
> > 
> (snip)
> 
> > [ 1522.019259] Active_anon:11 active_file:6 inactive_anon:0
> > [ 1522.019260]  inactive_file:0 unevictable:0 dirty:0 writeback:0 unstable:0
> > [ 1522.019261]  free:1985 slab:44399 mapped:132 pagetables:61830 bounce:0
> > [ 1522.019262]  isolate:69817
> 
> OK. thanks.
> I plan to submit this patch after small more tests. it is useful for OOM analysis.

Other counters to consider are NR_ANON_PAGES/NR_FILE_PAGES.

If they were showed in the oom message, this problem could be found
much earlier.  In this case, we'll find that the total file+anon pages
outnumbered the active+inactive file/anon pages.

Thanks,
Fengguang
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/