linux-ext4 - Re: question about file space preallocation with fallocate

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 27 Dec 2010 11:11:14 -0800
From:	Sunil Mushran <sunil.mushran@...cle.com>
To:	Filipe David Manana <fdmanana@...che.org>
CC:	linux-ext4@...r.kernel.org
Subject: Re: question about file space preallocation with fallocate

On 12/27/2010 09:57 AM, Filipe David Manana wrote:
> On Mon, Dec 27, 2010 at 5:17 PM, Sunil Mushran<sunil.mushran@...cle.com>  wrote:
>> fallocate() gives users the ability to allocate space instantly. One way
>> to compare would be to time just fallocate() with another program
>> writing zeros for that length.
>>
>> But that's not the aim of the syscall. The aim is to allow the fs to
>> allocate
>> the space in as large chunks as possible to allow for better read
>> performance.
>>
>> If you don't do fallocate() and allow writes to allocate in small chunks,
>> as you are doing, the allocations on disks could be interleaved in face of
>> multiple processes doing the same. Fragmented allocations can only hurt
>> read performance.
>>
> Thanks for the clarification Sunil. But preallocation of blocks
> shouldn't also improve write operations? Since each write operation
> will no longer cause the OS/filesystem to allocate blocks for the
> file, therefore should be faster.
>
> Also, any particular advice for improving write performance when all
> the writes are done in append-only fashion?

Even with meta-data journaling, the allocation overhead is tiny
compared to the 1G data write overhead.

Considering you are using ext4, you should benefit from delayed
allocation. But for that you'll need to have enough memory and
be running a 64-bit kernel. That way you wont be limited by the
speed of the disk.

Other option is submitting writes in larger chunks. Say 1MB rather
than 1KB.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html