linux-kernel - Re: [PATCH 17/18] writeback: fix dirtied pages accounting on redirty

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20110907011846.GB6513@localhost>
Date:	Wed, 7 Sep 2011 09:18:46 +0800
From:	Wu Fengguang <fengguang.wu@...el.com>
To:	Jan Kara <jack@...e.cz>
Cc:	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	"linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Christoph Hellwig <hch@....de>,
	Dave Chinner <david@...morbit.com>,
	Greg Thelen <gthelen@...gle.com>,
	Minchan Kim <minchan.kim@...il.com>,
	Vivek Goyal <vgoyal@...hat.com>,
	Andrea Righi <arighi@...eler.com>,
	linux-mm <linux-mm@...ck.org>,
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 17/18] writeback: fix dirtied pages accounting on
 redirty

On Wed, Sep 07, 2011 at 08:22:22AM +0800, Jan Kara wrote:
> On Tue 06-09-11 18:18:56, Peter Zijlstra wrote:
> > On Sun, 2011-09-04 at 09:53 +0800, Wu Fengguang wrote:
> > > De-account the accumulative dirty counters on page redirty.
> > > 
> > > Page redirties (very common in ext4) will introduce mismatch between
> > > counters (a) and (b)
> > > 
> > > a) NR_DIRTIED, BDI_DIRTIED, tsk->nr_dirtied
> > > b) NR_WRITTEN, BDI_WRITTEN
> > > 
> > > This will introduce systematic errors in balanced_rate and result in
> > > dirty page position errors (ie. the dirty pages are no longer balanced
> > > around the global/bdi setpoints).
> > > 
> > 
> > So wtf is ext4 doing? Shouldn't a page stay dirty until its written out?
> > 
> > That is, should we really frob around this behaviour or fix ext4 because
> > its on crack?
>   Fengguang, could you please verify your findings with recent kernel? I
> believe ext4 got fixed in this regard some time ago already (and yes, old
> delalloc writeback code in ext4 was terrible).

Jan, attached is the results for 3.1-rc4, before and after this patchset.
The test case is ext4, 1 dd, bs=4k, dirty_bytes=1GB.

Judging from global_dirtied_written.png, the dirtied/written lines are
still departing away from each other...

Thanks,
Fengguang

Download attachment "global_dirtied_written.png" of type "image/png" (38142 bytes)

Download attachment "balance_dirty_pages-pause.png" of type "image/png" (22406 bytes)

Download attachment "balance_dirty_pages-pages.png" of type "image/png" (41955 bytes)

Download attachment "global_dirtied_written.png" of type "image/png" (38597 bytes)

Download attachment "balance_dirty_pages-pause.png" of type "image/png" (28829 bytes)

Download attachment "balance_dirty_pages-pages.png" of type "image/png" (71535 bytes)