linux-kernel - maximum buffer size for splice(2) tcp->pipe?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <E1LKrp3-004Tub-IR@intern.SerNet.DE>
Date:	Thu, 8 Jan 2009 11:13:51 +0100
From:	Volker Lendecke <Volker.Lendecke@...Net.DE>
To:	linux-kernel@...r.kernel.org
Subject: maximum buffer size for splice(2) tcp->pipe?

Hi!

While implementing splice support in Samba for better
performance I found it blocking when trying to pull data off
tcp into a pipe when the recvq was full. Attached find a
test program that shows this behaviour, on another host I
started

netcat 192.168.19.10 4711 < /dev/zero

vlendec@...ny:~$ uname -a
Linux lenny 2.6.28-06857-g5cbd04a #7 Wed Jan 7 10:10:42 CET 2009 x86_64 = GNU/Linux
vlendec@...ny:~$ gcc -o splicetest /host/home/vlendec/splicetest.c -O3 -Wall
vlendec@...ny:~$ ./splicetest out 65536 &
[1] 697
vlendec@...ny:~$ strace -p 697
Process 697 attached - interrupt to quit
splice(0x3, 0, 0x5, 0, 0x56a0, 0x1)     = 22176
splice(0x7, 0, 0x4, 0, 0x10000, 0x1^C <unfinished ...>
Process 697 detached
vlendec@...ny:~$ netstat -nt | grep 4711
tcp    69272      0 192.168.19.10:4711 192.168.19.1:33773 ESTABLISHED
vlendec@...ny:~$

Interestingly, whenever I start the strace, it gets another
chunk of data and then blocks in the next splice call.

If I start splicetest with a buffer size of 16384 instead of
65536, it does not block. I could not find a way to ask the
kernel for the tipping point below which it does not block.

What is a safe buffer size to use with splice?

BTW, this kernel is from Steve French's linux-cifs.git repo.

Thanks,

Volker Lendecke

Samba Team

P.S: I'm not subscribed to linux-kernel, so if possible
please CC me directly. If this is inappropriate behaviour,
please give me a quick hint :-)

-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen

View attachment "splicetest.c" of type "text/x-c++src" (2005 bytes)

Content of type "application/pgp-signature" skipped