linux-kernel - Re: mdadm software raid + ext4, capped at ~350MiB/s limitation/bug?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20100228080100.092c24c2@notabene.brown>
Date:	Sun, 28 Feb 2010 08:01:00 +1100
From:	Neil Brown <neilb@...e.de>
To:	Justin Piszcz <jpiszcz@...idpixels.com>
Cc:	linux-kernel@...r.kernel.org, linux-raid@...r.kernel.org,
	linux-ext4@...r.kernel.org, Alan Piszcz <ap@...arrain.com>
Subject: Re: mdadm software raid + ext4, capped at ~350MiB/s limitation/bug?

On Sat, 27 Feb 2010 08:47:48 -0500 (EST)
Justin Piszcz <jpiszcz@...idpixels.com> wrote:

> Hello,
> 
> I have two separate systems and with ext4 I cannot get speeds greater than 
> ~350MiB/s when using ext4 as the filesystem on top of a raid5 or raid0. 
> It appears to be a bug with ext4 (or its just that ext4 is slower for this
> test)?
> 
> Each system runs 2.6.33 x86_64.

Could be related to the recent implementation of IO barriers in md.
Can you try mounting your filesystem with
   -o barrier=0

and see how that changes the result.

NeilBrown


> 
> Can someone please confirm?
> 
> Here is ext4:
> 
> # dd if=/dev/zero of=bigfile bs=1M count=10240
> 10240+0 records in
> 10240+0 records out
> 10737418240 bytes (11 GB) copied, 29.8556 s, 360 MB/s
> 
> The result is the same regardless of the RAID type (RAID-5 or RAID-0)
> 
> Note, this is not a bandwidth problem:
> 
> # dd if=/dev/zero of=/dev/md0 bs=1M count=10240
> 10240+0 records in
> 10240+0 records out
> 10737418240 bytes (11 GB) copied, 17.6871 s, 607 MB/s
> 
> With XFS:
> 
> p63:~# mkfs.xfs -f /dev/md0
> p63:~# mount /dev/md0 /r1
> p63:~# cd /r1
> 10240+0 records in
> 10240+0 records out
> 10737418240 bytes (11 GB) copied, 17.6078 s, 610 MB/s
> 
> NOTE: With a HW raid controller (OR using XFS), I can get > 500 MiB/s, 
> this problem only occurs with SW raid (Linux/mdadm).
> 
> Example (3ware 9650SE-16PML RAID-6, 15 drives (using EXT4)
> $ dd if=/dev/zero of=bigfile bs=1M count=10240
> 10240+0 records in
> 10240+0 records out
> 10737418240 bytes (11 GB) copied, 21.1729 s, 507 MB/s
> 
> Justin.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/