linux-kernel - Re: Supermicro X8DTH-6: Only ~250MiB/s from RAID<->RAID over 10GbE?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.DEB.2.00.1102051729010.8518@p34.internal.lan>
Date:	Sat, 5 Feb 2011 17:30:33 -0500 (EST)
From:	Justin Piszcz <jpiszcz@...idpixels.com>
To:	"Dr. David Alan Gilbert" <linux@...blig.org>
cc:	Emmanuel Florac <eflorac@...ellique.com>,
	linux-kernel@...r.kernel.org, linux-scsi@...r.kernel.org,
	linux-net@...r.kernel.org, Alan Piszcz <ap@...arrain.com>
Subject: Re: Supermicro X8DTH-6: Only ~250MiB/s from RAID<->RAID over
 10GbE?



On Sat, 5 Feb 2011, Dr. David Alan Gilbert wrote:

> * Justin Piszcz (jpiszcz@...idpixels.com) wrote:
>>
>>
>> On Sat, 5 Feb 2011, Emmanuel Florac wrote:
>>
>>> Le Sat, 5 Feb 2011 14:35:52 -0500 (EST) vous écriviez:
>>>
>>
>> To respond to everyone:
>>
>>> Did you try launching 4 simultaneous cp operations over nfs to get to
>>> 1.2 GB/s?
>>>  I've witnessed single stream copy performance with Samba being less than
>>> maximum due to Samba limitations.  Running multiple copy ops in parallel then
>>> usually saturates the pipe.
>>
>>
>> I tried 4 simultaenous cp's and there was little change, 250-320MiB/s.
>
> So I think you've said network benchmarks are OK, disc benchmarks are OK, but
> a copy over the network is slow.
>
> What happens if you run a disc benchmark at the same time as a network benchmark;
> even though the two aren't related? Can you keep the disc busy writing even when
> the network is being pushed?

Hi,

Still OK when reading or writing & iperf test, so it does not appear to be
I/O or backplane bound.  I also put both cards on the same CPU (lane-wise)
and it made no difference.

Device eth0 [10.0.1.4] (1/1):
================================================================================
Incoming:
                                                   ###
                                                   ###
                                                   ###
                                                   ###
                                                   ###
                                                   ###  Curr: 1188.85 MByte/s
                                                   ###  Avg: 680.11 MByte/s
                                                   ###  Min: 0.00 MByte/s
                                                   ###  Max: 1188.88 MByte/s
                                                   ###  Ttl: 18.32 GByte
Outgoing: