linux-kernel - Re: [RFC] extending splice for copy offloading

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130919085924.09b2a25c@tlielax.poochiereds.net>
Date:	Thu, 19 Sep 2013 08:59:24 -0400
From:	Jeff Layton <jlayton@...hat.com>
To:	Eric Wong <normalperson@...t.net>
Cc:	Zach Brown <zab@...hat.com>, linux-kernel@...r.kernel.org,
	linux-fsdevel@...r.kernel.org, linux-nfs@...r.kernel.org,
	Trond Myklebust <Trond.Myklebust@...app.com>,
	Bryan Schumaker <bjschuma@...app.com>,
	"Martin K. Petersen" <mkp@....net>, Jens Axboe <axboe@...nel.dk>,
	Mark Fasheh <mfasheh@...e.com>,
	Joel Becker <jlbec@...lplan.org>
Subject: Re: [RFC] extending splice for copy offloading

On Wed, 11 Sep 2013 21:17:23 +0000
Eric Wong <normalperson@...t.net> wrote:

> Zach Brown <zab@...hat.com> wrote:
> > Towards the end of that thread Eric Wong asked why we didn't just
> > extend splice.  I immediately replied with some dumb dismissive
> > answer.  Once I sat down and looked at it, though, it does make a
> > lot of sense.  So good job, Eric.  +10 Dummie points for me.
> 
> Thanks for revisiting that :>
> 
> > Some things to talk about:
> > - I really don't care about the naming here.  If you do, holler.
> 
> Exposing "DIRECT" to userspace now might confuse users into expecting
> O_DIRECT behavior.  I say this as an easily-confused user.
> 
> In the future, perhaps O_DIRECT behavior can become per-splice (instead
> of just per-open) and can save SPLICE_F_DIRECT for that.
> 
> > - We might want different flags for file-to-file splicing and acceleration
> > - We might want flags to require or forbid acceleration
> 

Do we need new flags at all? If both fds refer to files, then perhaps
we can just take it that SPLICE_F_DIRECT behavior is implied?

I'd probably suggest that we not add any more flags than are necessary
until use-cases for them become clear.

> > - We might want to provide all these flags to sendfile, too
> 
> Another syscall?  I prefer not.  Better to just maintain the sendfile
> API as-is for compatibility reasons and nudge users towards splice.
> 

Agreed.

> > Thoughts?  Objections?
> 
> I'll try to test/comment more in a week or two (not much time for
> computing until then).

On the whole, the concept looks sound.

I'll note too that by simply lifting the restriction that one of the
fd's to splice must always be a pipe, that may also give us a relatively
simple way to add recvfile() as well, even if only as a macro wrapper
around splice(). That's been a long sought-after feature of the samba
developers...

Just allow userland to do a splice straight from a socket fd to a file.
We may end up having to copy data if the alignment isn't right, but it'd
still be valuable to do that directly in the kernel in a single syscall.

-- 
Jeff Layton <jlayton@...hat.com>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/