linux-kernel - Re: Unexpected splice "always copy" behavior observed

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20100519160135.GC2039@Krystal>
Date:	Wed, 19 May 2010 12:01:35 -0400
From:	Mathieu Desnoyers <mathieu.desnoyers@...icios.com>
To:	Nick Piggin <npiggin@...e.de>
Cc:	Steven Rostedt <rostedt@...dmis.org>,
	Miklos Szeredi <miklos@...redi.hu>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	peterz@...radead.org, fweisbec@...il.com, tardyp@...il.com,
	mingo@...e.hu, acme@...hat.com, tzanussi@...il.com,
	paulus@...ba.org, linux-kernel@...r.kernel.org,
	arjan@...radead.org, ziga.mahkovec@...il.com, davem@...emloft.net,
	linux-mm@...ck.org, akpm@...ux-foundation.org,
	kosaki.motohiro@...fujitsu.com, cl@...ux-foundation.org,
	tj@...nel.org, jens.axboe@...cle.com
Subject: Re: Unexpected splice "always copy" behavior observed

* Nick Piggin (npiggin@...e.de) wrote:
> On Wed, May 19, 2010 at 11:45:42AM -0400, Steven Rostedt wrote:
> > On Wed, 2010-05-19 at 17:33 +0200, Miklos Szeredi wrote:
> > > On Wed, 19 May 2010, Linus Torvalds wrote:
> > > > Btw, since you apparently have a real case - is the "splice to file" 
> > > > always just an append? IOW, if I'm not right in assuming that the only 
> > > > sane thing people would reasonable care about is "append to a file", then 
> > > > holler now.
> > > 
> > > Virtual machines might reasonably need this for splicing to a disk
> > > image.
> > 
> > This comes down to balancing speed and complexity. Perhaps a copy is
> > fine in this case.
> > 
> > I'm concerned about high speed tracing, where we are always just taking
> > pages from the trace ring buffer and appending them to a file or sending
> > them off to the network. The slower this is, the more likely you will
> > lose events.
> > 
> > If the "move only on append to file" is easy to implement, I would
> > really like to see that happen. The speed of splicing a disk image for a
> > virtual machine only impacts the patience of the user. The speed of
> > splicing tracing output, impacts how much you can trace without losing
> > events.
> 
> It's not "easy" to implement :) What's your ring buffer look like?
> Is it a normal user address which the kernel does copy_to_user()ish
> things into? Or a mmapped special driver?
> 
> If the latter, it get's even harder again. But either way if the
> source pages just have to be regenerated anyway (eg. via page fault
> on next access), then it might not even be worthwhile to do the
> splice move.

Steven and I use pages to which we write directly by using the page address from
the linear memory mapping returned by page_address(). These pages have no other
mapping. They are moved to the pipe, and then from the pipe to a file (or to the
network). It's possibly the simplest scenario you could think of for splice().

Thanks,

Mathieu


-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/