linux-ext4 - Re: [PATCH 1/3] ext4: Fix insertion point of extent in mext_insert_across

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87f94c371003032150q64046db9h815d4cb630bb7a12@mail.gmail.com>
Date:	Thu, 4 Mar 2010 00:50:33 -0500
From:	Greg Freemyer <greg.freemyer@...il.com>
To:	tytso@....edu
Cc:	Akira Fujita <a-fujita@...jp.nec.com>,
	ext4 development <linux-ext4@...r.kernel.org>,
	OHSM-DEV <ohsm-devel@...ts.sourceforge.net>
Subject: Re: [PATCH 1/3] ext4: Fix insertion point of extent in 
	mext_insert_across_blocks()

>
> P.S.  Here's another random idea for how we might aggressively test
> the EXT4_IOC_MOVE_EXT ioctl: (1) create an empty filesystem, (2)
> create a tool which randomly sets 50% of the bits in the block
> allocation bitmap, marking them as in use, and making the free space
> look very badly fragmented.  (3) write a large number of files into
> the filesystem.  (4) calculate the checksums for all of the files.
> (5) run e2fsck on the filesystem to fix up the block allocation
> bitmap.  (6) defrag all of the files on the filesystem.  (7) use
> e2fsck to make sure the filesystem is still consistent.  (8) calculate
> the checksums for all of the files to make sure they still contain
> their original data.

Even that does not address issues with files in use during defrag.

ie. Defrag'ing a mysql database file while in use seems like an
important test case that is missing above.

Also, one issue with repetitive testing via the e4defrag tool, is you
only end up moving everything once and then in theory extra passes
have little to do.

The ohsm project has written a userspace "relocate" tool that calls
ext4_ioc_move_ext() to move files around on the filesystem.

In the absense of any ext4 ohsm kernel patches the blocks allocated to
the donor file would just use the normal block allocators.  Therefore
it should be relatively easy to introduce an effect of just randomly
using ext4_ioc_move_ext() to change out the underlying blocks.  It
maybe a useful in building up a test suite for ext4_ioc_move_ext().

In addition, for static files such as you describe above, we plan to
use the 60 GB or so of real world public domain data at
http://edrm.net/activities/projects/dataset as potential well known /
well defined real world data.  That data already has published MD5
values available, so data corruption at any point in the process
should be readily identifiable.

Greg
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html