linux-ext4 - Re: Minimizing fragmentation in ext4, fallocate not enough?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 27 Sep 2010 14:10:52 -0700
From:	Taras Glek <tglek@...illa.com>
To:	Eric Sandeen <sandeen@...hat.com>
CC:	Andreas Dilger <adilger.kernel@...ger.ca>,
	linux-ext4@...r.kernel.org
Subject: Re: Minimizing fragmentation in ext4, fallocate not enough?

On 09/25/2010 10:26 AM, Eric Sandeen wrote:
> Andreas Dilger wrote:
>> On 2010-09-24, at 18:05, Taras Glek wrote:
>>> I noticed that several random IO-heavy Firefox files got fragmented
>>> easily. Our cache suffers most. The cache works by creating a flat
>>> file and storing fixed-size entries in it. I though if I
>>> fallocate() the file first, then all of the writes within the
>>> allocated area would not cause additional fragmentation.
>>>
>>> This doesn't seem to completely cure fragmentation with ext4 in
>>> 2.6.33. If I allocate a 4mb file, it gets more and more fragmented
>>> over time. fallocate() does reduce fragmentation, but not as much
>>> as I expected.
>> Have you checked filefrag immediately after fallocating the file?  Is
>> it OK?
>>
>> It may be that the issue is that an fallocate()'d file is using
>> "unwritten extents" and converting these extents to "normal" extents
>> may cause apparent fragmentation.  However, depending on which
>> version of e2fsprogs/filefrag you are using, it may well be that
>> these extents only appear to be fragmented due to the different
>> extent types.
> Agreed, please include filefrag (-v) output right after it's fallocated,
> and also when you see this fragmentation, and then we'll have a better idea
> about what you're seeing.  And, the newer the filefrag the better.  :)
Thanks for clarification. Turns out ext4 is performing as expected, 
nevermind my previous message.

I was confused by discrepancy  in number of extents reported by filefrag 
1.41.10 with/without -v flag.

filefrag _CACHE_003_
_CACHE_003_: 17 extents found
filefrag -v _CACHE_003_
Filesystem type is: ef53
File size of _CACHE_003_ is 4194304 (1024 blocks, blocksize 4096)
  ext logical physical expected length flags
    0       0   232448             128
    1     128   232576               1 unwritten
    2     129   232577              95
    3     224   232672               1 unwritten
    4     225   232673              31
    5     256   232704               1 unwritten
    6     257   232705              63
    7     320   232768               1 unwritten
    8     321   232769              95
    9     416   232864               1 unwritten
   10     417   232865             255
   11     672   233120               1 unwritten
   12     673   233121             191
   13     864   233312               1 unwritten
   14     865   233313             127
   15     992   233440               3
   16     995   233443              29 unwritten,eof
_CACHE_003_: 1 extent found

Thanks,
Taras
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html