linux-kernel - Re: [PATCH 10/18] writeback: dirty position control

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 28 Sep 2011 16:50:35 +0200
From:	Peter Zijlstra <a.p.zijlstra@...llo.nl>
To:	Wu Fengguang <fengguang.wu@...el.com>
Cc:	"linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Jan Kara <jack@...e.cz>, Christoph Hellwig <hch@....de>,
	Dave Chinner <david@...morbit.com>,
	Greg Thelen <gthelen@...gle.com>,
	Minchan Kim <minchan.kim@...il.com>,
	Vivek Goyal <vgoyal@...hat.com>,
	Andrea Righi <arighi@...eler.com>,
	linux-mm <linux-mm@...ck.org>,
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 10/18] writeback: dirty position control - bdi reserve
 area

On Wed, 2011-09-28 at 22:02 +0800, Wu Fengguang wrote:

/me attempts to swap back neurons related to writeback

> After lots of experiments, I end up with this bdi reserve point
> 
> +       x_intercept = bdi_thresh / 2 + MIN_WRITEBACK_PAGES;
> 
> together with this chunk to avoid a bdi stuck in bdi_thresh=0 state:
> 
> @@ -590,6 +590,7 @@ static unsigned long bdi_position_ratio(
>          */
>         if (unlikely(bdi_thresh > thresh))
>                 bdi_thresh = thresh;
> +       bdi_thresh = max(bdi_thresh, (limit - dirty) / 8);
>         /*
>          * scale global setpoint to bdi's:
>          *      bdi_setpoint = setpoint * bdi_thresh / thresh

So you cap bdi_thresh at a minimum of (limit-dirty)/8 which can be
pretty close to 0 if we have a spike in dirty or a negative spike in
writeout bandwidth (sudden seeks or whatnot).


> The above changes are good enough to keep reasonable amount of bdi
> dirty pages, so the bdi underrun flag ("[PATCH 11/18] block: add bdi
> flag to indicate risk of io queue underrun") is dropped.

That sounds like goodness ;-)

> I also tried various bdi freerun patches, however the results are not
> satisfactory. Basically the bdi reserve area approach (this patch)
> yields noticeably more smooth/resilient behavior than the
> freerun/underrun approaches. I noticed that the bdi underrun flag
> could lead to sudden surge of dirty pages (especially if not
> safeguarded by the dirty_exceeded condition) in the very small
> window.. 

OK, so let me try and parse this magic:

+       x_intercept = bdi_thresh / 2 + MIN_WRITEBACK_PAGES;
+       if (bdi_dirty < x_intercept) {
+               if (bdi_dirty > x_intercept / 8) {
+                       pos_ratio *= x_intercept;
+                       do_div(pos_ratio, bdi_dirty);
+               } else
+                       pos_ratio *= 8;
+       }

So we set our target some place north of MIN_WRITEBACK_PAGES: if we're
short we add a factor of: x_intercept/bdi_dirty. 

Now, since bdi_dirty < x_intercept, this is > 1 and thus we promote more
dirties.

Additionally we don't let the factor get larger than 8 to avoid silly
large fluctuations (8 already seems quite generous to me).


Now I guess the only problem is when nr_bdi * MIN_WRITEBACK_PAGES ~
limit, at which point things go pear shaped.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/