linux-ext4 - Re: fallocate creating fragmented files

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <5108B833.6010004@redhat.com>
Date:	Wed, 30 Jan 2013 00:05:39 -0600
From:	Eric Sandeen <sandeen@...hat.com>
To:	Bron Gondwana <brong@...tmail.fm>
CC:	linux-ext4@...r.kernel.org, Rob Mueller <robm@...tmail.fm>
Subject: Re: fallocate creating fragmented files

On 1/29/13 11:46 PM, Bron Gondwana wrote:
> Hi All,
> 
> I'm trying to understand why my ext4 filesystem is creating highly fragmented files even though it's only just over 50% full.

It's at least possible that freespace is very fragmented; you could try the "e2freefrag" command to see.

<backgroun>

> Now we've got some background, let's create two files using the fallocate command line tool, and see where the blocks wound up.  This mirrors exactly the behaviour we have seen with both using posix_fallocate to pre-allocate space, or just using seek and writev to write individual records out to the file.  Except with the slower writes of a real application, we get closer to 9000 extents for a file this size.
> 
> [brong@...p14 conf]$ fallocate -l 20m testfile 
> [brong@...p14 conf]$ fallocate -l 20m testfile2
> [brong@...p14 conf]$ filefrag testfile
> testfile: 421 extents found
> [brong@...p14 conf]$ filefrag testfile2
> testfile2: 306 extents found
> 
> Now looking at the verbose output, we can see that there are many extents of just 3 or 4 blocks:
> 
> [brong@...p14 conf]$ filefrag -v testfile | awk '{print $5}' | sort -n | uniq -c | head
>       2 
>       1 is
>       1 length
>       1 unwritten
>       6 3
>      10 4
>       6 5
>       5 6
>       3 7
>       1 8

But longer extents too, right:

$ filefrag -v testfile | awk '{print $5}' | sort -n | uniq -c | tail
      1 162
      1 164
      1 179
      1 188
      1 215
      1 231
      1 233
      1 255
      1 322
      1 357

> Yet looking at the next file,
> 
> [brong@...p14 conf]$ filefrag -v testfile2 | awk '{print $5}' | sort -n | uniq -c | tail
>       1 173
>       1 175
>       1 178
>       1 184
>       1 187
>       1 189
>       1 194
>       1 289
>       1 321
>       1 330
> 

and presumably shorter extents at the beginning?

So it sounds like both files are a mix of long & short extents.

> There are multiple extents of hundreds of blocks in length.  Why weren't they used in allocating the first file?

I'm not sure, offhand.  But just to be clear, while contiguous allocations are usually a nice side-effect of fallocate, nothing at all guarantees it.  It only guarantees that you'll have that space available for future writes.

Still, it'd be interesting to figure out why the allocator is behaving this way.
It'd be interesting to see the freefrag info, the allocator might really be in scavenger mode.

-Eric

> This filesystem is quite busy all the time.  There are hundreds of imapd processes all locking and writing to it, including a lot of fdatasync and fsync calls.  During the time it took to run this command, there would have been multiple fsyncs.  I can't see why that would affect the allocator in this way for a single fallocate call though.
> 
> Regards,
> 
> Bron.


--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html