linux-kernel - Re: [RFC] extending splice for copy offloading

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAJfpegseipg6DAseBTG4Z6024noQa7owv48rOJznCPiXK+hezw@mail.gmail.com>
Date:	Fri, 27 Sep 2013 16:39:15 +0200
From:	Miklos Szeredi <miklos@...redi.hu>
To:	Ric Wheeler <rwheeler@...hat.com>
Cc:	Zach Brown <zab@...hat.com>,
	"J. Bruce Fields" <bfields@...ldses.org>,
	Anna Schumaker <schumaker.anna@...il.com>,
	Kernel Mailing List <linux-kernel@...r.kernel.org>,
	Linux-Fsdevel <linux-fsdevel@...r.kernel.org>,
	"linux-nfs@...r.kernel.org" <linux-nfs@...r.kernel.org>,
	Trond Myklebust <Trond.Myklebust@...app.com>,
	Bryan Schumaker <bjschuma@...app.com>,
	"Martin K. Petersen" <mkp@....net>, Jens Axboe <axboe@...nel.dk>,
	Mark Fasheh <mfasheh@...e.com>,
	Joel Becker <jlbec@...lplan.org>,
	Eric Wong <normalperson@...t.net>
Subject: Re: [RFC] extending splice for copy offloading

On Fri, Sep 27, 2013 at 4:00 PM, Ric Wheeler <rwheeler@...hat.com> wrote:

> I think that you are an order of magnitude off here in thinking about the
> scale of the operations.
>
> An enabled, synchronize copy offload to an array (or one that turns into a
> reflink locally) is effectively the cost of the call itself. Let's say no
> slower than one IO to a S-ATA disk (10ms?) as a pessimistic guess.
> Realistically, that call is much faster than that worst case number.
>
> Copying any substantial amount of data - like the target workload of VM
> images or media files - would be hundreds of MB's per copy and that would
> take seconds or minutes.

Will a single splice-copy operation be interruptible/restartable?  If
not, how should apps size one request so that it doesn't take too much
time?  Even for slow devices (usb stick)?  If it will be restartable,
how?   Can remote copy be done with this?  Over a high latency
network?

Those are the questions I'm worried about.

>
> We should really work on getting the basic mechanism working and robust
> without any complications, then we can look at real, measured performance
> and see if there is any justification for adding complexity.

Go for that.  But don't forget that at the end of the day actual apps
will need to be converted like file managers and "dd" and "cp" and we
definitely don't wont a userspace library to be able to figure out how
the copy is done most efficiently; it's something for the kernel to
figure out.

Thanks,
Miklos
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/