linux-ext4 - Re: [RFC] ext4: block reservation allocation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4F4BA33C.1050303@redhat.com>
Date:	Mon, 27 Feb 2012 09:37:32 -0600
From:	Eric Sandeen <sandeen@...hat.com>
To:	linux-ext4@...r.kernel.org
Subject: Re: [RFC] ext4: block reservation allocation

On 2/27/12 3:09 AM, Zheng Liu wrote:
> Hi list,
> 
> Now, in ext4, we have multi-block allocation and delay allocation. They work
> well for most scenarios. However, in some specific scenarios, they cannot help
> us to optimize block allocation. For example, the user may want to indicate some
> file set to be allocated at the beginning of the disk because its speed in this
> position is faster than its speed at the end of disk.

I agree with Lukas - please, no.

You can play tricks with your storage to accomplish much the same thing,
by making filesystems on faster & slower devices and mounting them on
directories which your application can recognize as faster/slower.

If you want "fast" for metadata, adilger has a recipe out there for using
lvm to interleave ssd blocks with spinning blocks to get metadata to line
up on the ssd.

A filesystem-specific hack for a custom application has no place in EXT4,
IMHO, sorry.

Essentially this would move allocation decisions to userspace, and I don't
think that sounds like a good idea.  If nothing else, the application shouldn't
assume that it "knows" anything at all about which regions of a filesystem may
be faster or slower...

-Eric


> I have done the following experiment. The experiment is on my own server, which
> has 16 Intel(R) Xeon(R) CPU E5620 @ 2.40GHz, 48G memory and a 1T sas disk. I
> split this disk into two partitions, one has 900G, and another has 100G. Then I
> use dd to get the speed of read/write. The result is as following.
> 
> [READ]
> # dd if=/dev/sdk1 of=/dev/null bs=128k count=10000 iflag=direct
> 1310720000 bytes (1.3 GB) copied, 9.41151 s, 139 MB/s
> 
> # dd if=/dev/sdk2 of=/dev/null bs=128k count=10000 iflag=direct
> 1310720000 bytes (1.3 GB) copied, 17.952 s, 73.0 MB/s
> 
> [WRITE]
> # dd if=/dev/zero of=/dev/sdk1 bs=128k count=10000 oflag=direct
> 1310720000 bytes (1.3 GB) copied, 8.46005 s, 155 MB/s
> 
> # dd if=/dev/zero of=/dev/sdk2 bs=128k count=10000 oflag=direct
> 1310720000 bytes (1.3 GB) copied, 15.8493 s, 82.7 MB/s
> 
> So filesystem can provide a new feature to let the user to indicate a value
> for reserving some blocks from the beginning of the disk. When the user needs
> to allocate some blocks for an important file that needs to be read/write as
> quick as possible, the user can use ioctl(2) and/or other ways to notify
> filesystem to allocate these blocks in the reservation area. Thereby, the user
> can obtain the higher performance for manipulating this file set.
> 
> This idea is very trivial. So any comments or suggestions are appreciated.
> 
> Regards,
> Zheng
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html