lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date:	Wed, 16 Jan 2008 13:30:43 -0800
From:	"Valerie Henson" <>
Cc:	"Theodore Ts'o" <>,
	"Andreas Dilger" <>,
	"Ric Wheeler" <>
Subject: [RFC] Parallelize IO for e2fsck

Hi y'all,

This is a request for comments on the rewrite of the e2fsck IO
parallelization patches I sent out a few months ago.  The mechanism is
totally different.  Previously IO was parallelized by issuing IOs from
multiple threads; now a single thread issues fadvise(WILLNEED) and
then uses read() to complete the IO.

Single disk performance doesn't change, but elapsed time drops by
about 50% on a big RAID-5 box.  Passes 1 and 2 are parallelized.  Pass
5 is left as an exercise for the reader.

Many thanks to the Lustre folks for their fadvise readahead patch
which this patch uses and for comments and help in general.  Our good
friends at EMC Centera funded this work.

Here are the top things I'd like feedback on:

How to split up the patch?  My take:

* Indirect block only flag for iterate
* IO manager readahead/release functions
* Readahead infrastructure (readahead.c and related)
* Readahead calls for pass 1
* Readahead calls for pass 2

Killing readahead properly is hard.  I implemented it several ways and
didn't like any of them.  The current solution is still racy and
completely untested.

The whole thing needs to be autoconfed correctly.  Bah.

The user interface kinda sucks.  It should at least take arguments of
the form "128KB" or "52m" instead of number of file system blocks.
Guessing the right amount of buffer cache to use and io requests to
issue would also be good.

ext2fs_get_next_inode_ptr() - With readahead, copying the inode in
ext2fs_get_next_inode_full() costs about 2-3% of elapsed time.  This
is a hacked up version that just returns a pointer to the inode.

The patch is against e2fsprogs 1.40.4 and is attached.  Future patches
will be split up and sent via quilt.



 e2fsck/                      |    6
 e2fsck/e2fsck.h                         |    5
 e2fsck/pass1.c                          |   36
 e2fsck/pass2.c                          |   12
 e2fsck/unix.c                           |   18
 lib/ext2fs/readahead.c                  | 1607 ++++++++++++++++++++++++++++++++
 lib/ext2fs/                  |    1
 lib/ext2fs/block.c                      |   20
 lib/ext2fs/dosio.c                      |    2
 lib/ext2fs/ext2_io.h                    |   10
 lib/ext2fs/ext2fs.h                     |   42
 lib/ext2fs/inode.c                      |   70 +
 lib/ext2fs/inode_io.c                   |    2
 lib/ext2fs/io_manager.c                 |   12
 lib/ext2fs/nt_io.c                      |    2
 lib/ext2fs/test_io.c                    |    2
 lib/ext2fs/unix_io.c                    |   41
 17 files changed, 1873 insertions(+), 15 deletions(-)

Download attachment "readahead" of type "application/octet-stream" (68339 bytes)

Powered by blists - more mailing lists