linux-ext4 - The changes I made to the undo-mgr (Re: What's cooking in e2fsprogs.git (topics))

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20080428194434.GA9399@mit.edu>
Date:	Mon, 28 Apr 2008 15:44:34 -0400
From:	Theodore Tso <tytso@....edu>
To:	"Aneesh Kumar K.V" <aneesh.kumar@...ux.vnet.ibm.com>
Cc:	linux-ext4@...r.kernel.org
Subject: The changes I made to the undo-mgr (Re: What's cooking in
	e2fsprogs.git (topics))

So let me talk about what were some of the last couple of things that
I did before I folded the undo-mgr into the tree, since it might be
helpful for other people who are into hacking e2fsprogs.

1) I used "make gcc-wall" (or if you only want to scan for changes in
e2undo.c, you can use the command; "rm e2undo.o; make gcc-wall-new")
to scan for warnings; this turned up some interesting bugs.  Please,
make sure your code is gcc-Wall clean.  In some cases the pre-existing
code isn't, due to folks being lazy or gcc -Wall becoming more strict.
But your patch shouldn't make things worse.

2) Compiling with the --enable-testio-debug configure option, I used
the command "export TEST_IO_FLAGS=0xffff" to set the environment
variable to see the I/O pattern of various programs like "mke2fs" with
the config option defaults.force_undo=1 set.  This is how I discovered
that we were using a pathetically small tdb_data_size, and changing it
to 32768 cut the mke2fs time in half.  Some more investigation showed
that part of the problem was we were initializing the journal one
block at a time, causing significant overhead.  When I changed
lib/ext2fs/mkjournal.c to zero out the journal 8 blocks at a time,
aligned on 8 block boundary to make life easier for the undo manager,
and made a similar change when zero'ing out the inode table so it
would also be issue writes aligned on 8 block boundaries, I was able
to cut the mke2fs time in half again.  

So for everyone who had been complaining about mke2fs taking too much
time with the undo option --- we do have tools in e2fsprogs that can
be used to help profile its operation and speed things up.  You can
also use seekwatcher/blktrace, of course, but there are also other
userspace tools that can be used. 

3) I also changed the name of the files created by the undo manager to
include an .e2undo suffix.  This makes it much more likely that system
administrators will understand what the file is and what it is used
for.

If people are looking for things to do with the undo manager --- it
would be good to wire up e2fsck and resize2fs to use it --- and at
this point, it would probably make sense to factor out common code for
setting up the undo manager into undo_mgr.c itself (although remember
the rules, nothing in libext2fs should try to write to stdout; library
routines must return error codes, and leave the printing of any status
or error messages to the application).

						- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html