linux-kernel - Re: New copyfile system call

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <6CEF377E-088F-4A79-896E-074CBC61CD8E@dilger.ca>
Date:	Thu, 21 Feb 2013 09:36:12 -0700
From:	Andreas Dilger <adilger@...ger.ca>
To:	Ric Wheeler <rwheeler@...hat.com>
Cc:	"Myklebust, Trond" <Trond.Myklebust@...app.com>,
	Linux FS Devel <linux-fsdevel@...r.kernel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"Chris L. Mason" <clmason@...ionio.com>,
	Christoph Hellwig <hch@...radead.org>,
	Alexander Viro <aviro@...hat.com>,
	"Martin K. Petersen" <mkp@....net>, Hannes Reinecke <hare@...e.de>,
	Joel Becker <jlbec@...lplan.org>
Subject: Re: New copyfile system call - discuss before LSF?

On 2013-02-21, at 7:57 AM, Ric Wheeler wrote:
> On 02/21/2013 02:51 PM, Myklebust, Trond wrote:
>> On Thu, 2013-02-21 at 12:37 +0100, Ric Wheeler wrote:
>>> We have debated the need to have a system call to allow for offloading copy
>>> operations, for example to an NFS server (part to the new NFS 4.2
>>> specification), SCSI target device (two different SCSI commands do this), local
>>> file systems (reflink, etc) and I suspect many other possible parts of the stack
>>> could implement this.
>> sendfile64() pretty much already has the right arguments for a
>> "copyfile", however it would be nice to add a 'flags' parameter: the
>> NFSv4.2 version would use that to specify whether or not to copy file
>> metadata.
> 
> That would seem to be enough to me and has the advantage that it is an relatively obvious extension to something that is at least not totally unknown to developers.
> 
> Do we need more than that for non-NFS paths I wonder? What does reflink need or the SCSI mechanism?

IMHO, the critical part about a copy syscall is avoiding the data
copy to/from userspace.  Copying file attributes opens up a huge
morass of issues related to which attrs/xattrs/ACLs are copied,
yet those don't cost nearly so much as the data copies.

We definitely want the API to be flexible enough to do server-side
copies (e.g. NFS and CIFS), but we also need to allow data copies
for regular files between different local and/or network filesystems
within the VFS.

Cheers, Andreas

>>> The earliest discussion of such a system call I saw happened back in 2001, I
>>> know we had another more recent flurry (2-3 years back?) as well that got
>>> tangled up and died away.
>>> 
>>> Given the new popularity of this in storage devices and the use case for virt
>>> guests, any chance to get a proposal floated this year that might be able to
>>> land upstream in our life times :) ?
>> I'm planning on soon dusting off the NFS prototype that NetApp wrote 3
>> years ago and converting at least the client implementation into
>> something that can go upstream. We do also have a server prototype for
>> Linux, but the copy offload between 2 different servers is a hack and
>> would need significant work.
>> 
> 
> That would be really interesting, thanks!
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/