linux-kernel - Re: New copyfile system call

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <512BD44C.40907@amacapital.net>
Date:	Mon, 25 Feb 2013 13:14:52 -0800
From:	Andy Lutomirski <luto@...capital.net>
To:	Zach Brown <zab@...hat.com>
CC:	"Myklebust, Trond" <Trond.Myklebust@...app.com>,
	Paolo Bonzini <pbonzini@...hat.com>,
	Ric Wheeler <rwheeler@...hat.com>,
	Linux FS Devel <linux-fsdevel@...r.kernel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"Chris L. Mason" <clmason@...ionio.com>,
	Christoph Hellwig <hch@...radead.org>,
	Alexander Viro <aviro@...hat.com>,
	"Martin K. Petersen" <mkp@....net>, Hannes Reinecke <hare@...e.de>,
	Joel Becker <jlbec@...lplan.org>
Subject: Re: New copyfile system call - discuss before LSF?

On 02/21/2013 02:24 PM, Zach Brown wrote:
> On Thu, Feb 21, 2013 at 08:50:27PM +0000, Myklebust, Trond wrote:
>> On Thu, 2013-02-21 at 21:00 +0100, Paolo Bonzini wrote:
>>> Il 21/02/2013 15:57, Ric Wheeler ha scritto:
>>>>>>
>>>>> sendfile64() pretty much already has the right arguments for a
>>>>> "copyfile", however it would be nice to add a 'flags' parameter: the
>>>>> NFSv4.2 version would use that to specify whether or not to copy file
>>>>> metadata.
>>>>
>>>> That would seem to be enough to me and has the advantage that it is an
>>>> relatively obvious extension to something that is at least not totally
>>>> unknown to developers.
>>>>
>>>> Do we need more than that for non-NFS paths I wonder? What does reflink
>>>> need or the SCSI mechanism?
>>>
>>> For virt we would like to be able to specify arbitrary block ranges.
>>> Copying an entire file helps some copy operations like storage
>>> migration.  However, it is not enough to convert the guest's offloaded
>>> copies to host-side offloaded copies.
>>
>> So how would a system call based on sendfile64() plus my flag parameter
>> prevent an underlying implementation from meeting your criterion?
> 
> If I'm guessing correctly, sendfile64()+flags would be annoying because
> it's missing an out_fd_offset.  The host will want to offload the
> guest's copies by calling sendfile on block ranges of a guest disk image
> file that correspond to the mappings of the in and out files in the
> guest.
> 
> You could make it work with some locking and out_fd seeking to set the
> write offset before calling sendfile64()+flags, but ugh.
> 
>  ssize_t sendfile(int out_fd, int in_fd, off_t in_offset, off_t
>                   out_offset, size_t count, int flags);
> 
> That seems closer.
> 
> We might also want to pre-emptively offer iovs instead of offsets,
> because that's the very first thing that's going to be requested after
> people prototype having to iterate calling sendfile() for each
> contiguous copy region. 

I thought the first thing people would ask for is to atomically create a
new file and copy the old file into it (at least on local file systems).
 The idea is that nothing should see an empty destination file, either
by race or by crash.  (This feature would perhaps be described as a
pony, but it should be implementable.)

This would be like a better link(2).

--Andy
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/