linux-kernel - Re: [PATCH 10/10] vmscan: Kick flusher threads to clean pages when reclaim is encountering dirty pages

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20100913141046.GI23508@csn.ul.ie>
Date:	Mon, 13 Sep 2010 15:10:46 +0100
From:	Mel Gorman <mel@....ul.ie>
To:	Wu Fengguang <fengguang.wu@...el.com>
Cc:	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
	Linux Kernel List <linux-kernel@...r.kernel.org>,
	Rik van Riel <riel@...hat.com>,
	Johannes Weiner <hannes@...xchg.org>,
	Minchan Kim <minchan.kim@...il.com>,
	Andrea Arcangeli <aarcange@...hat.com>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	Dave Chinner <david@...morbit.com>,
	Chris Mason <chris.mason@...cle.com>,
	Christoph Hellwig <hch@....de>,
	Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: [PATCH 10/10] vmscan: Kick flusher threads to clean pages when
	reclaim is encountering dirty pages

On Mon, Sep 13, 2010 at 09:48:45PM +0800, Wu Fengguang wrote:
> > +	/*
> > +	 * If reclaim is encountering dirty pages, it may be because
> > +	 * dirty pages are reaching the end of the LRU even though the
> > +	 * dirty_ratio may be satisified. In this case, wake flusher
> > +	 * threads to pro-actively clean up to a maximum of
> > +	 * 4 * SWAP_CLUSTER_MAX amount of data (usually 1/2MB) unless
> > +	 * !may_writepage indicates that this is a direct reclaimer in
> > +	 * laptop mode avoiding disk spin-ups
> > +	 */
> > +	if (file && nr_dirty_seen && sc->may_writepage)
> > +		wakeup_flusher_threads(nr_writeback_pages(nr_dirty));
> 
> wakeup_flusher_threads() works, but seems not the pertinent one.
> 
> - locally, it needs some luck to clean the pages that direct reclaim is waiting on

There is a certain amount of luck involved but it's depending on there being a
correlation between old inodes and old pages on the LRU list. As long as that
correlation is accurate, some relevant pages will get cleaned.  Testing on
previously released versions of this patch did show that the percentage of
dirty pages encountered during reclaim were reduced as a result of this patch.

> - globally, it cleans up some dirty pages, however some heavy dirtier
>   may quickly create new ones..
> 
> So how about taking the approaches in these patches?
> 
> - "[PATCH 4/4] vmscan: transfer async file writeback to the flusher"
> - "[PATCH 15/17] mm: lower soft dirty limits on memory pressure"
> 

There is a lot going on in those patches. It's going to take me a while to
figure them out and formulate an opinion.

> In particular the first patch should work very nicely with memcg, as
> all pages of an inode typically belong to the same memcg. So doing
> write-around helps clean lots of dirty pages in the target LRU list in
> one shot.
> 

It might but as there is also a correlation between old dirty inodes and
the location of dirty pages, it is tricky to predict if it is better and
if so, by how much.

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/