linux-kernel - Re: Fast network file copy; "recvfile()" ?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <alpine.DEB.0.999999.0801211021200.22574@twinlark.arctic.org>
Date:	Mon, 21 Jan 2008 10:29:52 -0800 (PST)
From:	dean gaudet <dean@...tic.org>
To:	"Patrick J. LoPresti" <lopresti@...il.com>
cc:	linux-kernel <linux-kernel@...r.kernel.org>
Subject: Re: Fast network file copy; "recvfile()" ?

On Thu, 17 Jan 2008, Patrick J. LoPresti wrote:

> I need to copy large (> 100GB) files between machines on a fast
> network.  Both machines have reasonably fast disk subsystems, with
> read/write performance benchmarked at > 800 MB/sec. Using 10GigE cards
> and the usual tweaks to tcp_rmem etc., I am getting single-stream TCP
> throughput better than 600 MB/sec.
> 
> My question is how best to move the actual file.  NFS writes appear to
> max out at a little over 100 MB/sec on this configuration.

did your "usual tweaks" include mounting with -o tcp,rsize=262144,wsize=262144 ?

i should have kept better notes last time i was experimenting with this,
but from memory here's what i found:

- if i used three NFS clients and was reading from page cache on the
  server i hit 1.2GB/s total throughput from the server.  the client
  NFS code was maxing out one CPU on each of the client machines.

- disk subsystem (sw raid10 far2) was capable of 600MB/s+ when read
  locally on the NFS server, but topped out around ~250MB/s when read
  remotely (no matter how many clients).

my workload was read-intensive so i didn't experiment with writes...

-dean
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/