linux-ext4 - Re: Minimizing fragmentation in ext4, fallocate not enough?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sat, 25 Sep 2010 12:26:15 -0500
From:	Eric Sandeen <sandeen@...hat.com>
To:	Andreas Dilger <adilger.kernel@...ger.ca>
CC:	Taras Glek <tglek@...illa.com>, linux-ext4@...r.kernel.org
Subject: Re: Minimizing fragmentation in ext4, fallocate not enough?

Andreas Dilger wrote:
> On 2010-09-24, at 18:05, Taras Glek wrote:
>> I noticed that several random IO-heavy Firefox files got fragmented
>> easily. Our cache suffers most. The cache works by creating a flat
>> file and storing fixed-size entries in it. I though if I
>> fallocate() the file first, then all of the writes within the
>> allocated area would not cause additional fragmentation.
>> 
>> This doesn't seem to completely cure fragmentation with ext4 in
>> 2.6.33. If I allocate a 4mb file, it gets more and more fragmented
>> over time. fallocate() does reduce fragmentation, but not as much
>> as I expected.
> 
> Have you checked filefrag immediately after fallocating the file?  Is
> it OK?
> 
> It may be that the issue is that an fallocate()'d file is using
> "unwritten extents" and converting these extents to "normal" extents
> may cause apparent fragmentation.  However, depending on which
> version of e2fsprogs/filefrag you are using, it may well be that
> these extents only appear to be fragmented due to the different
> extent types.

Agreed, please include filefrag (-v) output right after it's fallocated,
and also when you see this fragmentation, and then we'll have a better idea
about what you're seeing.  And, the newer the filefrag the better.  :)

-Eric

>> ps2. Does running filefrag on a directory mean anything in ext4?
> 
> With newer e2fsprogs using FIEMAP and kernels it should be possible
> to get useful filefrag data from a directory.  Older
> kernels/e2fsprogs using FIBMAP will just fail outright. 

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html