lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Message-Id: <1371967642-3116-1-git-send-email-linkinjeon@gmail.com> Date: Sun, 23 Jun 2013 15:07:22 +0900 From: Namjae Jeon <linkinjeon@...il.com> To: tytso@....edu, adilger.kernel@...ger.ca Cc: linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org, linux-ext4@...r.kernel.org, a.sangwan@...sung.com, Namjae Jeon <linkinjeon@...il.com>, Namjae Jeon <namjae.jeon@...sung.com> Subject: [PATCH 0/3] ext4: introduce two new ioctls From: Namjae Jeon <namjae.jeon@...sung.com> This patch series introduces 2 new ioctls for ext4. Truncate_block_range ioctl truncates blocks from source file. Transfer_block_range ioctl transfers data blocks from source file and append them at the end of destination file. Ioctl1: EXT4_IOC_TRUNCATE_BLOCK_RANGE: This ioctl truncates a range of data blocks from file. It is useful to remove easily and quickly the garbage data at the middle of file. e.g. we have a movie file and there is long advertisement in movie file. user want to remove only advertisement range. 1) Movie file (8GB), There is the adverisement of 500MB size. ____________________________________________________________________ | | | | | a) range | b) Advertisement | c) range | | | (unneeded data) | | |_____________________|___________________|_________________________| 2) Currently if user want to remove portion b), the conventional way would be to copy a) and c) (7.5GB) to new file by reading data from original file and writing to new file, followed up by delete original file and rename new file. It will take long time. When we measure time, it takes around 3 minutes. 3) If we use EXT4_IOC_TRUNCATE_BLOCK_RANGE, we can have garbage data removed in less than a second. Also, no need to perform deletion and rename. _______________________________________________ | | | | a) range | c) range | | | | |_____________________|________________________| #define EXT4_IOC_TRUNCATE_BLOCK_RANGE _IOW('f', 18, struct truncate_range) struct truncate_range { __u32 start_block; __u32 length; }; example => Originally the file "abc" has the below extent tree: debugfs: ex abc Level Entries Logical Physical Length Flags 0/ 0 1/ 3 0 - 4 33615 - 33619 5 0/ 0 2/ 3 5 - 9 33855 - 33859 5 0/ 0 3/ 3 10 - 14 69657 - 69661 5 ls -lh abc -rw-r--r-- 1 root 0 60.0K Jan 1 00:01 abc du -h abc 60.0K abc e4_truncate_block_range abc 2 10 Return: : Success After executing truncate_block_range ioctl, the extent tree: ex abc Level Entries Logical Physical Length Flags 0/ 0 1/ 2 0 - 1 33615 - 33616 2 0/ 0 2/ 2 2 - 4 69659 - 69661 3 ls -lh abc -rw-r--r-- 1 root 0 20.0K Jan 1 00:08 abc du -h abc 20.0K abc This ioctl works in 2 parts: 1) remove _only_ data blocks that resides within specified range. If the entire range is a hole than nothing is removed. 2) update file's logical block offsets ranging from block number "start_block + length" to last logical block of file such that lblk_number = lblk_number - length; This is done by updating starting block of all the extents that resides within the range. If "start_block + length" is already equal to the last block of file than no block is updated. This case is similar to convential truncate. In the above example: The data blocks ranging from [2 - 11] have been removed and the logical offsets of the file beyond block number 12 till last block of file are updated by subtracting length from each of logical numbers. This gives a contiguous logical space to the file. Also, the logical size and disksize of the file are updated accordingly. Ioctl2: EXT4_IOC_TRANSFER_BLOCK_RANGE: This ioctl transfers a range of data blocks from source file and append them at the end of the destination file. This is not actual data transfer but only metadata is moved. ____________________________________________________________________ | | | | | a) range | b) range | c) range | | | | | |_____________________|___________________|_________________________| If user does not want b) in the orig file but wants to make a new file comprising only b) OR wants b) at the end of an already existing file, the conventional way of doing it would be to: 1) Copy b) to new file 2) Copy c) to temp file 3) Truncate orig file to a) 4) Copy c) from temp file to the end of orig file. 5) Delete temp file. After this operations => orig_file: __________________________________________ | | | | a) range | c) range | | | | |_____________________|___________________| new_file: _______________________ | | | b) range | | | |_____________________| Again, this operation would take a long time (depending on the sizes of range) if done using conventional way while using transfer_block_range ioctl reduces the time within a second. #define EXT4_IOC_TRANSFER_BLOCK_RANGE _IOW('f', 19, struct transfer_range) struct transfer_range { __u32 dest_fd; __u32 start_block; __u32 length; }; example=> debugfs: ex source Level Entries Logical Physical Length Flags 0/ 1 1/ 1 0 - 24 32809 25 1/ 1 1/ 5 0 - 4 4071 - 4075 5 1/ 1 2/ 5 5 - 9 4081 - 4085 5 1/ 1 3/ 5 10 - 14 4091 - 4095 5 1/ 1 4/ 5 15 - 19 4101 - 4105 5 1/ 1 5/ 5 20 - 24 4151 - 4155 5 debugfs: ex dest Level Entries Logical Physical Length Flags 0/ 0 1/ 3 0 - 4 32825 - 32829 5 0/ 0 2/ 3 5 - 9 33545 - 33549 5 0/ 0 3/ 3 10 - 14 33615 - 33619 5 ls -lh source -rw-r--r-- 1 root 0 100.0K Jan 1 00:01 source ls -lh dest -rw-r--r-- 1 root 0 60.0K Jan 1 00:01 dest du -h source 104.0K source du -h dest 60.0K dest e4_transfer_block_range source dest 2 10 Return: : Success debugfs: ex source Level Entries Logical Physical Length Flags 0/ 1 1/ 1 0 - 24 32809 25 1/ 1 1/ 4 0 - 1 4071 - 4072 2 1/ 1 2/ 4 12 - 14 4093 - 4095 3 1/ 1 3/ 4 15 - 19 4101 - 4105 5 1/ 1 4/ 4 20 - 24 4151 - 4155 5 debugfs: ex dest Level Entries Logical Physical Length Flags 0/ 1 1/ 1 0 - 24 32835 25 1/ 1 1/ 6 0 - 4 32825 - 32829 5 1/ 1 2/ 6 5 - 9 33545 - 33549 5 1/ 1 3/ 6 10 - 14 33615 - 33619 5 1/ 1 4/ 6 15 - 17 4073 - 4075 3 1/ 1 5/ 6 18 - 22 4081 - 4085 5 1/ 1 6/ 6 23 - 24 4091 - 4092 2 ls -lh source -rw-r--r-- 1 root 0 100.0K Jan 1 00:04 source ls -lh dest -rw-r--r-- 1 root 0 100.0K Jan 1 00:04 dest du -h source 64.0K source du -h dest 104.0K dest The data blocks lying between [start_block to start_block + length) are appended contiguously at the end of destination file. The block transfer leaves a hole in the source file. If any hole is encountered in the range, it is ommited. This ioctl does not change the logical size of the source file hence leaves a hole in place of transfered range. If user want contiguous logical space for source file, it can truncate the hole by calling truncate_range_ioctl for source file. Example for above "source" file: e4_truncate_block_range source 2 10 Return: : Success debugfs: ex source Level Entries Logical Physical Length Flags 0/ 1 1/ 1 0 - 14 32809 15 1/ 1 1/ 4 0 - 1 4071 - 4072 2 1/ 1 2/ 4 2 - 4 4093 - 4095 3 1/ 1 3/ 4 5 - 9 4101 - 4105 5 1/ 1 4/ 4 10 - 14 4151 - 4155 5 Namjae Jeon (3): ext4: Add EXT4_IOC_TRUNCATE_BLOCK_RANGE ioctl ext4: make mext_next_extent non static and move get_ext_path ext4: Add EXT4_IOC_TRANSFER_BLOCK_RANGE ioctl -- 1.7.9.5 -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists