linux-kernel - Re: Ext3 vs NTFS performance

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Sat, 5 May 2007 09:45:05 -0400
From:	Theodore Tso <tytso@....edu>
To:	Xu CanHao <xucanhao@...il.com>
Cc:	linux-kernel@...r.kernel.org
Subject: Re: Ext3 vs NTFS performance

On Sat, May 05, 2007 at 11:13:36AM +0800, Xu CanHao wrote:
> On 5 Mai, 10:20, Theodore Tso <t...@....edu> wrote:
> >
> >This is being worked on already.  XFS has a per-filesystem ioctl, but
> >we want to create a filesystem-independent system call,
> >sys_fallocate(), that would wired into the already existing
> >posix_fallocate() function exported by glibc.
> 
> The story told us: an application must look to the file-systems, ext3
> is good at aaa, is not good at bbb; XFS is good at ccc, is not good at
> ddd; reiserfs is good at eee, is not good at fff........
> 
> For this scenario, XFS is good at dealing with fragmentation while ext3 not.

That's true.  XFS has the ability to do delayed allocations, so that
the blocks don't get allocated until they are written out.  Hence, a
workload that writes a pattern which uses random access writes in
strides of 128k, and then goes back to fill them in, will result in
fragmentation given ext3's current block reservation allocation
algorithm --- but, as long as the system isn't under high memory
pressure, XFS will do better in this particular scenario.

Actually, ext3 does have a block reservation system, which will
prevent this scenario if the random access writes are within a range
of 32k or so --- which is enough to protect against the bad effects of
more common random access write patterns, such as those used when
writing out ELF object files, for example.  Increasing
EXT3_DEFAULT_RESERVE_BLOCKS by a factor of 4 would adaopt the ext3
block reservation system to this pathalogical workload, and we could
easily add a tunable mount option to change the reservation size used
by ext3.  Unfortunately, this could make fragmentation work for other
workloads.  So adding delayed allocation to ext4 is a better solution.

But as has already been discussed on this thread, in situations where
the fileserver is under high memory pressure, any filesystem (XFS or
ext4) would still end up allocating blocks out of order, resulting in
fragmentation.  Explicit preallocation, as opposed to delayed
allocation, is really the best long-term solution; and in order to do
that, Samba needs to detect this scenario --- which as has been noted,
there appears to be no good reason for the Windows CIFS client (or any
other application)to be doing this, other than perhaps to deliberate
trigger a worst case allocation pattern in ext3 --- and translate it
into a explicit preallocation request.

Regards,

						- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/