Message-ID: <49019EDA.2080400@rs.jp.nec.com>
Date:	Fri, 24 Oct 2008 19:09:30 +0900
From:	Akira Fujita <a-fujita@...jp.nec.com>
To:	linux-ext4@...r.kernel.org, Theodore Tso <tytso@....edu>,
	Mingming Cao <cmm@...ibm.com>
CC:	linux-fsdevel@...r.kernel.org
Subject: [RFC][PATCH 0/9]ext4: online defrag (ver 0.97)

Hello,

I've updated ext4 online defrag patches.
Changes are as followings (Ted pointed 1-4 out before).
 1. Remove unneeded ioctls and related functions.
   - EXT4_IOC_FIBMAP
   - EXT4_IOC_GROUP_INFO
   - EXT4_IOC_RESERVE_BLOCK
   - EXT4_IOC_BLOCK_RELEASE
 2. Implement new ioctl (EXT4_IOC_SUPER_INFO) which gets ext4_super_block.
 3. Remove ext4_defrag_ioctl() not to have a double layer of ioctl's dispatch.
 4. Fix defrag not to use block reservation in force defrag mode (-f),
    because the legacy block allocator have been removed from ext4.
 5. Add permission check to ext4_defrag_fiemap_ino and ext4_defag_move_victim.
    Because non-permitted file should not be selected as victim file
    in force defrag mode (-f).

Changelog:
- 0.97(Oct. 24, 2008)
  - Remove unneeded ioctls.
  - Add EXT4_IOC_SUPER_INFO which gets ext4_super_block.
  - Fix defrag not to use block reservation in force defrag mode (-f).
  - Remove ext4_defrag_ioctl() not to have a double layer of ioctl's dispatch.
  - Some cleanups.
- 0.95(Sep. 26, 2008)
  - Support 1KB and 2KB block size.
  - Implement EXT4_IOC_FIEMAP_INO which calls ext4_fiemap() for specified inode number
    instead of EXT4_IOC_EXTENTS_INFO to get extents information.
  - Merge "JC6-defrag-alloc-contiguous-blks-credit" and
    "ext4-request-for-blocks-with-ar.excepted_group-1.patch" in the ext4 patch queue
    into this version.
  - Handle s_dirtyblocks_counter correctly in defrag when delalloc enabled.
  - Remove unneeded copy_from_user() in EXT4_IOC_GROUP_INFO ioctl.  Ted pointed this out.
- 0.9 (May 30, 2008)
  - Create some new functions (ext4_defrag_fill_ar(),
    ext4_defrag_check_phase() ...) to separate block allocation function,
    since the phase mode plug into the allocation function isn't good.
  - Add the description of ext4_defrag() which is main function.
  - Add the capability check.
  - Some cleanups.

Outline for ext4 online defragmentation:
Ext4 online defrag has the following three functions.

 no option  Solving a single file fragmentation.
            Single file fragmentation is solved by moving file
            data to contiguous free blocks.

    -r      Solving a relevant file fragmentation.
            Relevant file fragmentation could be solved by moving
            the files under the specified directory close together with
            the block containing the directory data.

    -f      Solving a free space fragmentation.
            If there is no contiguous free blocks in the filesystem,
            the other files are moved to make sufficient space to allocate
            contiguous blocks for the target file.

Next step:
 - Support flex_bg.
 Address the following items that Ted pointed out before.
 These are no functional impact.
 - Rebuild defrag patches construction.
   (low-level functions will be first)
 - Move the defrag functions that are related to extents or mballoc
   to the appropriate files.
 - Add more descriptions each of defrag functions.

Summary of patches:
* The followings are new ext4 online defrag patches and they consist
  of ioctl unit except 1-4.  Because the EXT4_IOC_DEFRAG is too big for reviewing,
  I divided it into 4 patches (I'll consider about appropriate construction
  of defrag patches later).

[PATCH 1/9] EXT4_IOC_DEFRAG ioctl and  defrag main function
- Create the temporary inode and do defrag per
  defrag_size (defalut 64MB).

[PATCH 2/9] Allocate new contiguous blocks with mballoc
- Search contiguous free blocks with mutil-block allocation
  and allocate them for the temporary inode.

[PATCH 3/9]  Exchange the extents between two inodes
- For each page, exchange the extents between the temporary inode
  and the original inode, and then write them.

[PATCH 4/9] Inside extents management and relevant defrag
- Merge extents of target inode which is increased by defragmentation.
- For relevant defrag, add the goal argument to related functions
  to allocate data block to the specified offset.

[PATCH 5/9] EXT4_IOC_SUPER_INFO ioctl
- EXT4_IOC_SUPER_INFO gets super block information on disk.

  Super block information is necessary for relevant (-r) and force (-f) defrag.
  Non-root user can't get this information by opening the block device directly,
  so implement this ioctl newly.

[PATCH 6/9] EXT4_IOC_FREE_BLOCKS_INFO ioctl
- The EXT4_IOC_FREE_BLOCKS_INFO ioctl gets free extents
  information of the target block group.
  This ioctl is used only in the force defrag (-f).

  ext4_defrag_fblocks_distribution() was renamed as ext4_mb_search_free_extents()
  and moved into mballoc.c.  Because it uses ext4_buddy structure.

[PATCH 7/9] EXT4_IOC_FIEMAP_INO ioctl
- The EXT4_IOC_FIEMAP_INO is used to get extents information of
  inode which set to ioctl.
  The defragger uses this ioctl to check the fragment condition
  and to get extents information in the specified block group.

  Andreas Dilger suggested that implement EXT4_IOC_WRAPPER to call fiemap
  for arbitrary inodes instead of this ioctl.
  But there is a matter for consideration to use EXT4_IOC_WRAPPER,
  so I passed up it in this time.

[PATCH 8/9] EXT4_IOC_MOVE_VICTIM ioctl
- The EXT4_IOC_MOVE_VICTIM moves the victim extents into other block group.
  Therefore the contiguous free space is made in the target block group.
  This ioctl is used only in the force defrag (-f).

[PATCH 9/9] Online defrag command
- The defrag command. Usage is as follows:
  - Defrag for a single file.
    # e4defrag file-name
  - Defrag for all files on ext4.
    # e4defrag device-name
  - Put the multiple files closer together.
    # e4defrag -r directory-name
  - Defrag for free space fragmentation.
    # e4defrag -f file-name

Any comments are welcome.

Best regards,
Akira Fujita
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists