linux-kernel - Re: regression in page writeback

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090922114143.GA6175@sli10-desk.sh.intel.com>
Date:	Tue, 22 Sep 2009 19:41:43 +0800
From:	Shaohua Li <shaohua.li@...el.com>
To:	"Wu, Fengguang" <fengguang.wu@...el.com>
Cc:	Richard Kennedy <richard@....demon.co.uk>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"jens.axboe@...cle.com" <jens.axboe@...cle.com>,
	"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
	Chris Mason <chris.mason@...cle.com>
Subject: Re: regression in page writeback

On Tue, Sep 22, 2009 at 05:05:01PM +0800, Wu, Fengguang wrote:
> On Tue, Sep 22, 2009 at 04:52:48PM +0800, Richard Kennedy wrote:
> > On Tue, 2009-09-22 at 10:32 +0200, Peter Zijlstra wrote:
> > > On Tue, 2009-09-22 at 16:24 +0800, Wu Fengguang wrote:
> > > > On Tue, Sep 22, 2009 at 04:09:25PM +0800, Peter Zijlstra wrote:
> > > > > On Tue, 2009-09-22 at 16:05 +0800, Wu Fengguang wrote:
> > > > > > 
> > > > > > I'm not sure how this patch stopped the "overshooting" behavior.
> > > > > > Maybe it managed to not start the background pdflush, or the started
> > > > > > pdflush thread exited because it found writeback is in progress by
> > > > > > someone else?
> > > > > > 
> > > > > > -               if (bdi_nr_reclaimable) {
> > > > > > +               if (bdi_nr_reclaimable > bdi_thresh) {
> > > > > 
> > > > > The idea is that we shouldn't move more pages from dirty -> writeback
> > > > > when there's not actually that much dirty left.
> > > > 
> > > > IMHO this makes little sense given that pdflush will move all dirty
> > > > pages anyway. pdflush should already be started to do background
> > > > writeback before the process is throttled, and it is designed to sync
> > > > all current dirty pages as quick as possible and as much as possible.
> > > 
> > > Not so, pdflush (or now the bdi writer thread thingies) should not
> > > deplete all dirty pages but should stop writing once they are below the
> > > background limit.
> > > 
> > > > > Now, I'm not sure about the > bdi_thresh part, I've suggested to maybe
> > > > > use bdi_thresh/2 a few times, but it generally didn't seem to make much
> > > > > of a difference.
> > > > 
> > > > One possible difference is, the process may end up waiting longer time
> > > > in order to sync write_chunk pages and quit the throttle. This could
> > > > hurt the responsiveness of the throttled process.
> > > 
> > > Well, that's all because this congestion_wait stuff is borken..
> > > 
> > 
> > The problem occurred as pdflush stopped when the number of dirty pages
> > reached the background threshold but balance_dirty_pages kept moving
> > pages to writeback because the total of dirty + writeback was over the
> > limit. 
> 
> Ah yes it is possible. The pdflush started by balance_dirty_pages()
> does stop at the background threshold (sorry for the confusion!),
> and then balance_dirty_pages() continue to sync pages in _smaller_
> chunk sizes, which should be suboptimal..
This is possible. Without the patch, balance_dirty_pages() can move some pages
to writeback and don't need do congestion_wait(), so the task can continue doing
write. The patch seems to break this.
I tried to set dirty_exceeded only when bdi_nr_reclaimable > bdi_thresh, this helps
a little in my test, but still not reach the best.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/