linux-kernel - Re: EXT4 is ~2X as slow as XFS (593MB/s vs 304MB/s) for writes?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <4B8C5DD3.4060804@redhat.com>
Date:	Mon, 01 Mar 2010 18:37:39 -0600
From:	Eric Sandeen <sandeen@...hat.com>
To:	Justin Piszcz <jpiszcz@...idpixels.com>
CC:	linux-ext4@...r.kernel.org, linux-kernel@...r.kernel.org,
	Alan Piszcz <ap@...arrain.com>
Subject: Re: EXT4 is ~2X as slow as XFS (593MB/s vs 304MB/s) for writes?

Eric Sandeen wrote:
> Justin Piszcz wrote:
>> Hello,
>>
>> Is it possible to 'optimize' ext4 so it is as fast as XFS for writes?
>> I see about half the performance as XFS for sequential writes.
>>
>> I have checked the doc and tried several options, a few of which are shown
>> below (I have also tried the commit/journal_async/etc options but none
>> of them get the write speeds anywhere near XFS)?
>>
>> Sure 'dd' is not a real benchmark, etc, etc, but with 10Gbps between 2
>> hosts I get 550MiB/s+ on reads from EXT4 but only 100-200MiB/s write.
>>
>> When it was XFS I used to get 400-600MiB/s for writes for the same RAID
>> volume.
>>
>> How do I 'speed' up ext4?  Is it possible?
>>
> 
> FWIW I'm seeing similar things on fast storage (Fusion IO),
> though this is under 2.6.31.  500MB/s+ for xfs, 300 for ext4.
> 
> Overwriting an existing file is no faster.   I don't think this
> driver is blktraceable but I'll try a newer driver that should be I think.

FWIW, blktrace (I'm still on 2.6.31) is enlightening:

Total (xfs):
 Reads Queued:           4,       16KiB	 Writes Queued:     122,567,   10,485MiB
 Read Dispatches:        4,       16KiB	 Write Dispatches:   83,219,   10,485MiB
 Reads Requeued:         0		 Writes Requeued:         0
 Reads Completed:        4,       16KiB	 Writes Completed:   83,219,   10,485MiB
 Read Merges:            0,        0KiB	 Write Merges:       39,348,  314,804KiB
 IO unplugs:           344        	 Timer unplugs:         338

Total (ext4):
 Reads Queued:          14,       56KiB	 Writes Queued:       2,621K,   10,486MiB
 Read Dispatches:       14,       56KiB	 Write Dispatches:  107,944,   10,486MiB
 Reads Requeued:         0		 Writes Requeued:         0
 Reads Completed:       14,       56KiB	 Writes Completed:  107,944,   10,486MiB
 Read Merges:            0,        0KiB	 Write Merges:        2,513K,   10,054MiB
 IO unplugs:         2,461        	 Timer unplugs:       2,020


See "Writes Queued"  See also submit_bio() calls in xfs.

ext4 doing things a block at a time is certainly giving the elevator a workout...
I'd tend to chalk it up to that at first glance.

-Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/