linux-ext4 - [RFC][PATCH 0/3] ext4: online defrag (ver 1.0)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <49829A1D.5090002@rs.jp.nec.com>
Date:	Fri, 30 Jan 2009 15:11:41 +0900
From:	Akira Fujita <a-fujita@...jp.nec.com>
To:	Theodore Tso <tytso@....edu>, linux-ext4@...r.kernel.org
CC:	linux-fsdevel@...r.kernel.org
Subject: [RFC][PATCH 0/3] ext4: online defrag (ver 1.0)

Hi,

I have rewritten ext4 online defrag patches based on the comments from Ted.
In the new defrag, create donor inode in the user space instead of kernel space,
and then allocate contiguous blocks to it with fallocate().
In kernel space, exchange the blocks between target inode and donor inode,
and then copy the file data of target inode to donor inode every 64MB.
The EXT4_IOC_DEFRAG ioctl becomes simpler than the old one,
so it may be useful for other purposes.

#define EXT4_IOC_DEFRAG                 _IOW('f', 15, struct move_extent)

struct move_extent {
        int org_fd;             /* original file descriptor */
        int dest_fd;            /* destination file descriptor */
        ext4_lblk_t start;      /* logical offset of org_fd and dest_fd */
        ext4_lblk_t len;        /* exchange block length */
};

Ext4 online defrag is done in the following steps:
(U:User spcace K:Kernel space)

1:U  Create a donor inode (destination file) and then unlink it.

2:U  Allocate contiguous blocks to donor inode with fallocate().

3:U  Call the FS_IOC_FIEMAP ioctl to get the extents information of donor inode.
     And check the extents of donor inode are less than the defrag target inode's.

4:U  Call the EXT4_IOC_DEFRAG ioctl every 64MB.

5:K  In kernel space, the EXT4_IOC_DEFRAG ioctl calls ext4_defrag()
     to exchange the blocks between org_fd and dest_fd (donor inode).
     Then write the file data of org_fd to dest_fd.

6:U  Close donor inode, then it will be deleted.

The new defrag has not implemented -r and -f options
which were in the old version (0.97) yet.
If defrag implements these options, probably it will be necessary
to change the block allocation method and add new ioctls.
I will address it later.

Summary of patches:
* The followings are new ext4 online defrag patches.
  They are applied to the ext4 patch queue (2.6.29-rc3).

[PATCH 1/3] Add EXT4_IOC_DEFRAG ioctl and basic defrag functions
- Add EXT4_IOC_DEFRAG ioctl and basic defrag functions.

[PATCH 2/3] Exchange the blocks between two inodes
- For each page, exchange the extents between original inode
  and destination inode, and then write the file data of
  the original inode to destination inode.

[PATCH 3/3] Ext4 online defrag command
- Usage is as follows:
  - Defrag for a single file.
    # e4defrag file, directory, device
  - Check the file fragments on ext4.
    # e4defrag -c file, directory, device

Best regards,
Akira Fujita

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html