linux-kernel - [GIT PULL] writeback rewrite

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <20090911143944.GJ14984@kernel.dk>
Date:	Fri, 11 Sep 2009 16:39:44 +0200
From:	Jens Axboe <jens.axboe@...cle.com>
To:	Linus Torvalds <torvalds@...ux-foundation.org>
Cc:	Linux Kernel <linux-kernel@...r.kernel.org>, hch@...radead.org,
	jack@...e.cz
Subject: [GIT PULL] writeback rewrite

Hi Linus,

This is something that I have been working on for many months now.
Originally I wanted to merge it for .31, but it was a bit too early. At
least now Jan Kara and Christoph is happy with it, so it should be good
to go.

Essentially this gets rid of pdflush for writeback completely. pdflush
has a number of issues that largely stem from the fact that it has to
work multiple queues at once. So it has to back off on congestion, which
can cause queue access starvation and lumpy writeback. The latter is
very apparent on many setups.

This patchset adds a thread per backing device. The thread will exit if
it has been idle for too long, and will get re-created if we start
seeing dirty inodes on the bdi again. Care is taken to ensure that we
don't get stuck doing that.

Writeback intensive behaviour is much better. I recently generated a
graph with seekwatcher of XFS writing 2 20G files using normal buffered
writeout. The existing code does:

http://kernel.dk/dd-md0-xfs-pdflush.png

while with the patchset we get both a faster and more smooth writeout:

http://kernel.dk/dd-md0-xfs-flush.png

The patchset has been well tested and has been in -next for 3-4 months
or so.

I have a follow-up writeback-postmerge branch with some good cleanups
and performance enhancements too, but I'd like to get this in a day or
two first since I didn't want to mess with the core of these bits
without doing a lot more testing again. That testing can happen while
this is out in the wild.

So please pull this patchset, I think we should merge it early so we
have a good chance to ensure we fixup any regressions should they occur.

  git://git.kernel.dk/linux-2.6-block.git writeback

Jens Axboe (7):
      writeback: get rid of generic_sync_sb_inodes() export
      writeback: move dirty inodes from super_block to backing_dev_info
      writeback: switch to per-bdi threads for flushing data
      writeback: get rid of pdflush completely
      writeback: add some debug inode list counters to bdi stats
      writeback: add name to backing_dev_info
      writeback: check for registered bdi in flusher add and inode dirty

 block/blk-core.c                 |    1 +
 drivers/block/aoe/aoeblk.c       |    1 +
 drivers/char/mem.c               |    1 +
 drivers/staging/pohmelfs/inode.c |    9 +-
 fs/btrfs/disk-io.c               |    1 +
 fs/buffer.c                      |    2 +-
 fs/char_dev.c                    |    1 +
 fs/configfs/inode.c              |    1 +
 fs/fs-writeback.c                | 1065 ++++++++++++++++++++++++++++----------
 fs/fuse/inode.c                  |    1 +
 fs/hugetlbfs/inode.c             |    1 +
 fs/nfs/client.c                  |    1 +
 fs/ocfs2/dlm/dlmfs.c             |    1 +
 fs/ramfs/inode.c                 |    1 +
 fs/super.c                       |    5 +-
 fs/sync.c                        |   20 +-
 fs/sysfs/inode.c                 |    1 +
 fs/ubifs/budget.c                |   16 +-
 fs/ubifs/super.c                 |    9 +-
 include/linux/backing-dev.h      |   55 ++-
 include/linux/fs.h               |    9 +-
 include/linux/writeback.h        |   23 +-
 kernel/cgroup.c                  |    1 +
 mm/Makefile                      |    2 +-
 mm/backing-dev.c                 |  381 +++++++++++++-
 mm/page-writeback.c              |  182 +------
 mm/pdflush.c                     |  269 ----------
 mm/swap_state.c                  |    1 +
 mm/vmscan.c                      |    2 +-
 29 files changed, 1285 insertions(+), 778 deletions(-)
 delete mode 100644 mm/pdflush.c

-- 
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/