linux-ext4 - Re: [RFC] Ext3 online defrag

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20061026033316.GC27858@thunk.org>
Date:	Wed, 25 Oct 2006 23:33:16 -0400
From:	Theodore Tso <tytso@....edu>
To:	David Chinner <dgc@....com>
Cc:	Jeff Garzik <jeff@...zik.org>,
	Barry Naujok <bnaujok@...bourne.sgi.com>,
	'Dave Kleikamp' <shaggy@...tin.ibm.com>,
	'Alex Tomas' <alex@...sterfs.com>, 'Jan Kara' <jack@...e.cz>,
	linux-fsdevel@...r.kernel.org, linux-ext4@...r.kernel.org
Subject: Re: [RFC] Ext3 online defrag

On Thu, Oct 26, 2006 at 11:40:20AM +1000, David Chinner wrote:
> We don't need to expose anything filesystem specific to userspace to
> implement this.  Online data movement (i.e. the defrag mechanism)
> becomes something like:
> 
> 	do {
> 		get_free_list(dst_fd, location, len, list)
> 		/* select extent to use */
> 		alloc_from_list(dst_fd, list[X], off, len)
> 	} while (ENOALLOC)
> 	move_data(src_fd, dst_fd, off, len);
> 
> And this would work on any filesystem type that implemented these
> interfaces. Hence tools like a startup file optimiser would
> only need to be written once, rather than needing a different
> tool for every different filesystem type.....

Yeah, but that's simply not enough.  A good defragger needs to know
about a filesystem's allocation policies, and move files so they are
optimally located, given the filesystem layout.  For example, in
ext2/3/4 we will want to move blocks so they in the same block group
as the inode.  That's filesystem specific information; other
filesystems will require different policies.

> Remember, I'm not just talking about defrag - I'm talking about
> an interface that is actually useful to apps that might care
> about how data is laid out on disk but the applications writers
> don't know anyhting about how filesystem X or Y or Z is
> implemented. Putting the burden of learning about fileystem
> internals on application developers is not the correct solution.

Unfortunately, if you want to do a good job, a defragger *has* to know
about some very low-level filesystem specific information, if it wants
to do a good job.

						- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html