linux-ext4 - Re: Minimizing fragmentation in ext4, fallocate not enough?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <7A56949B-99B6-439F-A015-1C0398A8B525@dilger.ca>
Date:	Fri, 24 Sep 2010 19:07:03 -0600
From:	Andreas Dilger <adilger.kernel@...ger.ca>
To:	Taras Glek <tglek@...illa.com>
Cc:	linux-ext4@...r.kernel.org
Subject: Re: Minimizing fragmentation in ext4, fallocate not enough?

On 2010-09-24, at 18:05, Taras Glek wrote:
> I noticed that several random IO-heavy Firefox files got fragmented easily. Our cache suffers most.
> The cache works by creating a flat file and storing fixed-size entries in it. I though if I fallocate() the file first, then all of the writes within the allocated area would not cause additional fragmentation.
> 
> This doesn't seem to completely cure fragmentation with ext4 in 2.6.33. If I allocate a 4mb file, it gets more and more fragmented over time. fallocate() does reduce fragmentation, but not as much as I expected.

Have you checked filefrag immediately after fallocating the file?  Is it OK?

It may be that the issue is that an fallocate()'d file is using "unwritten extents" and converting these extents to "normal" extents may cause apparent fragmentation.  However, depending on which version of e2fsprogs/filefrag you are using, it may well be that these extents only appear to be fragmented due to the different extent types.

> ps2. Does running filefrag on a directory mean anything in ext4?

With newer e2fsprogs using FIEMAP and kernels it should be possible to get useful filefrag data from a directory.  Older kernels/e2fsprogs using FIBMAP will just fail outright.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html