linux-kernel - [RFC PATCH] mm: let the bdi_writeout fraction respond more quickly

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <1276523894.1980.85.camel@castor.rsk>
Date:	Mon, 14 Jun 2010 14:58:14 +0100
From:	Richard Kennedy <richard@....demon.co.uk>
To:	Jens Axboe <jens.axboe@...cle.com>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Wu Fengguang <fengguang.wu@...el.com>
Cc:	lkml <linux-kernel@...r.kernel.org>, linux-mm <linux-mm@...ck.org>
Subject: [RFC PATCH] mm: let the bdi_writeout fraction respond more quickly

Hi all,
The fraction of vm cache allowed to each BDI as calculated by
get_dirty_limits (mm/page-writeback.c) respond very slowly to changes in
workload.

Running a simple test that alternately writes 1Gb to sda then sdb,
twice, shows the bdi_threshold taking approximately 15 seconds to reach
a steady state value. This prevents a application from using all of the
available cache and forces it to write to the physical disk earlier than
strictly necessary.  
As you can see from the attached graph, bdi_thresh_before.png, our
current control system responds to this kind of workload very slowly.

The below patch speeds up the recalculation and lets it reach a steady
state value in a couple of seconds. see bdi_thresh_after.png.

I get better throughput with this patch applied and have been running
some variation of this on and off for some months without any obvious
problems.

(These tests were all run on 2.6.35-rc3,
where dm-2 is a sata drive lvm/ext4 and sdb is ide ext4.
I've got lots more results and graphs but won't bore you all with
them ;) )

I see this as a considerable improvement but I have found the magic
number of -4 empirically so it may just be tuned to my system. I'm not
sure how to decide on a value that is suitable for everyone. 

Does anyone have any suggestions or thoughts?

Unfortunately I don't have any other hardware to try this on, so I would
be very interest to hear if anyone tries this on their favourite
workload.

regards
Richard

patch against 2.6.35-rc3

diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 2fdda90..315dd04 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -144,7 +144,7 @@ static int calc_period_shift(void)
 	else
 		dirty_total = (vm_dirty_ratio * determine_dirtyable_memory()) /
 				100;
-	return 2 + ilog2(dirty_total - 1);
+	return ilog2(dirty_total - 1) - 4;
 }

 /*

Download attachment "bdi_thresh_before.png" of type "image/png" (4098 bytes)

Download attachment "bdi_thresh_after.png" of type "image/png" (2398 bytes)