linux-ext4 - Re: Online defragmentation and ext4migrate

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 18 May 2007 13:19:54 -0700
From:	Eric <erpo41@...il.com>
To:	linux-ext4 <linux-ext4@...r.kernel.org>
Subject: Re: Online defragmentation and ext4migrate

On Fri, 2007-05-18 at 18:36 +0530, Aneesh Kumar K.V wrote:
> The reason why i am asking this is to understand the
> usefulness of doing a ext4migrate followed by defrag. 
> [...]
> Also looking at the version 0.4 I see that defrag ioctl only work if we 
> have EXT4_EXTENTS_FL flag set. 

ext4migrate is necessary because the current ext4 defrag routines will
only defragment files stored as extents. AFAIK, converting a file to
extents does not allow the defrag routine to defragment it "better" than
an indirect block map inode, but converting any file to extents has
performance benefits regardless of whether it is later defragmented.

> What are the plans for making defrag work 
> with indirect block map inode ?

I think there is a second set of patches to defragment non-extent
files. 

When I started investigating this topic, I would have preferred a defrag
routine for indirect block map inodes since it would work with the
filesystems that I and others are using right now. However, as I read
more code and documentation, I'm beginning to warm up to extents.

A defragmentation routine makes files contiguous on disk. A better
defragmentation routine intelligently locates data structures on the
disk so that files and directories are placed to minimize latency and
maximize throughput now, AND so that this will continue to happen in the
future. Typically this means not only making files contiguous, but also
consolidating free space at the end of the volume so that the block
allocator can pick contiguous blocks for new files. An even better
defragmentation routine knows how to balance the time lost to
defragmentation with the performance gained from a defragmented
filesystem. IMHO, this requires detailed knowledge of the layout of a
file's blocks on the disk. Right now, we get this information by looping
over the FIBMAP ioctl, which I understand can take quite a long time.
But on an extent file there is a logical, high-performance mapping
between the on-disk structures that keep track of which blocks belong to
which files and the data returned by the as-yet-to-be-implemented FIEMAP
ioctl, which could make defragging faster and more fun.

http://www.mail-archive.com/linux-ext4@vger.kernel.org/msg01434.html

Cheers,

Eric

Download attachment "signature.asc" of type "application/pgp-signature" (190 bytes)