linux-kernel - RE: [PATCH] mm: implement WasActive page flag (for improving cleancache)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1327686876.2977.37.camel@dabdike.int.hansenpartnership.com>
Date:	Fri, 27 Jan 2012 11:54:36 -0600
From:	James Bottomley <James.Bottomley@...senPartnership.com>
To:	Dan Magenheimer <dan.magenheimer@...cle.com>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Dave Hansen <dave@...ux.vnet.ibm.com>,
	linux-kernel@...r.kernel.org, linux-mm@...ck.org,
	Konrad Wilk <konrad.wilk@...cle.com>,
	Seth Jennings <sjenning@...ux.vnet.ibm.com>,
	Nitin Gupta <ngupta@...are.org>,
	Nebojsa Trpkovic <trx.lists@...il.com>, minchan@...nel.org,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	riel@...hat.com, Chris Mason <chris.mason@...cle.com>,
	lsf-pc@...ts.linux-foundation.org
Subject: RE: [PATCH] mm: implement WasActive page flag (for improving
 cleancache)

On Fri, 2012-01-27 at 09:32 -0800, Dan Magenheimer wrote:
> > From: James Bottomley [mailto:James.Bottomley@...senPartnership.com]
> > What I was wondering was instead of using a flag, could we make the LRU
> > lists do this for us ... something like have a special LRU list for
> > pages added to the page cache but never referenced since added?  It
> > sounds like you can get your WasActive information from the same type of
> > LRU list tricks (assuming we can do them).
> 
> Hmmm... I think this would mean more LRU queues but that may be
> the right long term answer.  Something like?
> 
> 	read but not used yet LRU
> 	readahead but not used yet LRU

What's the difference between these two?  I think read but not used is
some form of readahead regardless of where it came from.

> 	active LRU
> 	previously active LRU

I don't quite understand why you need two queues here, either.  Surely
active is logically at the bottom of the LRU and previously active at
the top (assuming we've separated the unused pages to a different LRU
list).

> Naturally, this then complicates the eviction selection process.
> 
> > I think the memory pressure eviction heuristic is: referenced but not
> > recently used pages first followed by unreferenced and not recently used
> > readahead pages.  The key being to keep recently read in readahead pages
> > until last because there's a time between doing readahead and getting
> > the page accessed and we don't want to trash a recently red in readahead
> > page only to have the process touch it and find it has to be read in
> > again.
> 
> I suspect that any further progress on the page replacement heuristics
> is going to require more per-page data to be stored.  Which means
> that it probably won't work under the constraints of 32-bit systems.
> So it might also make sense to revisit when/whether to allow the
> heuristics to be better on a 64-bit system than on a 32-bit.
> (Even ARM now has 64-bit processors!)  I put this on my topic list
> for LSF/MM, though I have no history of previous discussion so
> this may already have previously been decided.

I've got to say why? on this.  The object is to find a simple solution
that's good enough.  I think separating the LRU list into two based on
once referenced/never referenced might be enough to derive all the
information we need.  Until that theory is disproved, there's not much
benefit to developing ever more complex heuristics.

James


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/