linux-ext4 - Re: inconsistent file placement

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20100706220101.GA6603@thunk.org>
Date:	Tue, 6 Jul 2010 18:01:01 -0400
From:	tytso@....edu
To:	Eric Sandeen <sandeen@...hat.com>
Cc:	Daniel Taylor <Daniel.Taylor@....com>, linux-ext4@...r.kernel.org
Subject: Re: inconsistent file placement

On Tue, Jul 06, 2010 at 01:59:34PM -0500, Eric Sandeen wrote:
> However, from the test description it looks like it is writing
> a file to the root dir, so there should be no parent-dir random spreading,
> right?

Hmm, yes, I missed that part of Daniel's e-mail.  He's just writing a
single file.  In that case, Amir is right, the only thing which would
be causing this is the colour offset, at least for ext2 and ext3.
This is avoid fragmented files caused by two or more processes running
on different CPU's all writing into the same block group.

In the case of ext4, we don't use a pid-determined colour algorithm if
delayed allocation is used, and the randomness is caused by the
writeback system deciding to write out different chunks of pages
first.  The way to fix this when writing large files is to use
fallocate(2) when writing a large file, so it can be allocated
contiguously.

In any case, Daniel, if you want the best results for your benchmark,
use ext4, and tweak the script slightly:

touch /DataVolume/hex.txt
fallocate -l 5G /DataVolume/hex.txt
for i in 0 1 2 3 4
do
    dd if=/hex.txt of=/DataVolume/hex.txt bs=64k conv=notrunc \
       oflag=direct,append
done

Best regards,

					- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html