linux-ext4 - Re: [PATCH, RFC] vm: Add an tuning knob for vm.max_writeback

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090901180052.GA7885@think>
Date:	Tue, 1 Sep 2009 14:00:52 -0400
From:	Chris Mason <chris.mason@...cle.com>
To:	Theodore Tso <tytso@....edu>
Cc:	Christoph Hellwig <hch@...radead.org>, linux-mm@...ck.org,
	Ext4 Developers List <linux-ext4@...r.kernel.org>,
	linux-fsdevel@...r.kernel.org, jens.axboe@...cle.com
Subject: Re: [PATCH, RFC] vm: Add an tuning knob for vm.max_writeback_pages

On Sun, Aug 30, 2009 at 02:17:31PM -0400, Theodore Tso wrote:
[ ... ]

> Jens?  What do you think?  Fixing MAX_WRITEBACK_PAGES was something I
> really wanted to merge in 2.6.32 since it makes a huge difference for
> the block allocation layout for a "rsync -avH /old-fs /new-fs" when we
> are copying bunch of large files (say, 800 meg iso images) and so the
> fact that the writeback routine is writing out 4 megs at a time, means
> that our files get horribly interleaved and thus get fragmented.

I did some runs comparing mainline with Jens' current writeback queue.
This is just btrfs, but its a good example of how things are improving.

These graphs show us the 'compile' phase of compilebench, where it is
writing all the .o files into the 30 kernel trees.  Basically we have
one thread, creating a bunch of files based on the sizes of all the .o
files in a compiled kernel.  They are created in random order, similar
to the files produced from a make -j.

I haven't yet tried this without the max_writeback_pages patch, but the
graphs clearly show a speed improvement, and that the mainline code is
smearing writes across the drive while Jens' work is writing
sequentially.

Jens' writeback branch:
http://oss.oracle.com/~mason/seekwatcher/compilebench-writes-axboe.png

Mainline
http://oss.oracle.com/~mason/seekwatcher/compilebench-writes-pdflush.png

-chris

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html