linux-ext4 - Re: Overwrite faster than fallocate

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YrTCbPK94Ejh4ei3@mit.edu>
Date:   Thu, 23 Jun 2022 15:43:40 -0400
From:   "Theodore Ts'o" <tytso@....edu>
To:     Santosh S <santosh.letterz@...il.com>
Cc:     Andreas Dilger <adilger@...ger.ca>, linux-ext4@...r.kernel.org
Subject: Re: Overwrite faster than fallocate

On Thu, Jun 23, 2022 at 02:28:47PM -0400, Santosh S wrote:
> 
> What kind of write will stop an uninitialized extent from splitting?
> For example, I want to create a file, fallocate 512MB, and zero-fill
> it. But I want the file system to only create 4 extents so they all
> reside in the inode itself, and each extent represents the entire
> 128MB (so no splitting).

If you write into an unitialized extent, it *has* to be split, since
we have to record what has been initialized, and what has not.  So for
example:

root@...-xfstests:/vdc# fallocate  -l 1M test-file
root@...-xfstests:/vdc# filefrag -vs test-file
Filesystem type is: ef53
File size of test-file is 1048576 (256 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..     255:      68864..     69119:    256:             last,unwritten,eof
test-file: 1 extent found
root@...-xfstests:/vdc# dd if=/dev/zero of=test-file bs=1k conv=notrunc bs=4k count=1 seek=10
1+0 records in
1+0 records out
4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000252186 s, 16.2 MB/s
root@...-xfstests:/vdc# filefrag -vs test-file
Filesystem type is: ef53
File size of test-file is 1048576 (256 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       9:      68864..     68873:     10:             unwritten
   1:       10..      10:      68874..     68874:      1:            
   2:       11..     255:      68875..     69119:    245:             last,unwritten,eof
test-file: 1 extent found

However, if you write to an adjacent block, the extent will get split
--- and then we will merge it to the initialized block.  So for
example, if we write to block 9:

root@...-xfstests:/vdc# dd if=/dev/zero of=test-file bs=1k conv=notrunc bs=4k count=1 seek=9
1+0 records in
1+0 records out
4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000205357 s, 19.9 MB/s
root@...-xfstests:/vdc# filefrag -vs test-file
Filesystem type is: ef53
File size of test-file is 1048576 (256 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       8:      68864..     68872:      9:             unwritten
   1:        9..      10:      68873..     68874:      2:            
   2:       11..     255:      68875..     69119:    245:             last,unwritten,eof
test-file: 1 extent found

So if you eventually write all of the blocks, because of the split and
the merging behavior, eventually the extent tree will be put into an efficient state:

root@...-xfstests:/vdc# dd if=/dev/zero of=test-file bs=1k conv=notrunc bs=4k count=9 seek=0
    ...
root@...-xfstests:/vdc# filefrag -vs test-file
Filesystem type is: ef53
File size of test-file is 1048576 (256 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..      10:      68864..     68874:     11:            
   1:       11..     255:      68875..     69119:    245:             last,unwritten,eof
test-file: 1 extent found
root@...-xfstests:/vdc# dd if=/dev/zero of=test-file bs=1k conv=notrunc bs=4k count=240 seek=11
    ...
root@...-xfstests:/vdc# filefrag -vs test-file
Filesystem type is: ef53
File size of test-file is 1048576 (256 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..     250:      68864..     69114:    251:            
   1:      251..     255:      69115..     69119:      5:             last,unwritten,eof
test-file: 1 extent found
root@...-xfstests:/vdc# dd if=/dev/zero of=test-file bs=1k conv=notrunc bs=4k count=5 seek=251
    ...
root@...-xfstests:/vdc# filefrag -vs test-file
Filesystem type is: ef53
File size of test-file is 1048576 (256 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..     255:      68864..     69119:    256:             last,eof
test-file: 1 extent found
root@...-xfstests:/vdc# 

Bottom-line, there isn't just splitting, but there is also merging
going on.  So it's not really something that you need to worry about.

Cheers,

						- Ted