linux-kernel - Re: [PATCH 01/45] writeback: reduce calls to global_page_state in balance_dirty

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20091012012609.GA7553@localhost>
Date:	Mon, 12 Oct 2009 09:26:09 +0800
From:	Wu Fengguang <fengguang.wu@...el.com>
To:	Peter Zijlstra <a.p.zijlstra@...llo.nl>
Cc:	Jan Kara <jack@...e.cz>, Andrew Morton <akpm@...ux-foundation.org>,
	Theodore Tso <tytso@....edu>,
	Christoph Hellwig <hch@...radead.org>,
	Dave Chinner <david@...morbit.com>,
	Chris Mason <chris.mason@...cle.com>,
	"Li, Shaohua" <shaohua.li@...el.com>,
	Myklebust Trond <Trond.Myklebust@...app.com>,
	"jens.axboe@...cle.com" <jens.axboe@...cle.com>,
	Nick Piggin <npiggin@...e.de>,
	"linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
	Richard Kennedy <richard@....demon.co.uk>,
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 01/45] writeback: reduce calls to global_page_state in
	balance_dirty_pages()

On Sun, Oct 11, 2009 at 07:25:17PM +0800, Peter Zijlstra wrote:
> On Sun, 2009-10-11 at 18:50 +0800, Wu Fengguang wrote:
> > 
> > Sorry for the confusion, but I mean, filesystems have to limit
> > nr_writeback (directly or indirectly via the block io queue),
> > otherwise it either hit nr_dirty to 0 (with the loop), or let
> > nr_writeback grow out of control (without the loop).
> 
> Doesn't this require the writeback queue to have a limit < dirty_thresh?

Yes, this is the key (open) issue. For now we have nothing to limit

        nr_writeback < dirty_thresh

> Or more specifically, for the bdi case:
> 
>  bdi_dirty + bdi_writeback + bdi_unstable <= bdi_thresh
> 
> we require that the writeback queue be smaller than bdi_thresh, which
> could be quite difficult, since bdi_thresh can easily be 0.

We could apply a MIN_BDI_DIRTY_THRESH. Because the bdi threshold is
estimated from writeback events, so bdi_thresh must be non-zero to
allow some writeback pages in flight :)

> Without observing the bdi_thresh constraint we can have:
> 
>   \Sum_(over bdis) writeback_queue_size
> 
> dirty pages outstanding, which could be significantly higher than
> dirty_thresh.

Yes.  Maybe we could do some per-bdi and/or global writeback wait
queue (ie. some generalized version of the patch 20: NFS: introduce
writeback wait queue).

The per-bdi writeback queue size should ideally be proportional to its
available writeback bandwidth. MIN_BDI_DIRTY_THRESH could be defined
to (2*bdi_writeback_bandwidth) or something close. And if the resulted
bdi limits turn out to be too large for a small memory system, we just
let the global limit kick in. For such small memory systems, it is
very likely there are only one bdi. So it is not likely to lose
fairness to base its limits on available memory instead of device
capability.

Thanks,
Fengguang
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/