linux-kernel - Re[2]: sendfile to nonblocking socket

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1211570752.20070424143348@amhost.net>
Date:	Tue, 24 Apr 2007 14:33:48 +0400
From:	Alex Vorona <voron@...ost.net>
To:	linux-kernel@...r.kernel.org
Subject: Re[2]: sendfile to nonblocking socket

Hello David,

Tuesday, April 24, 2007, 1:19:49 PM, you wrote:

>> sendfile function is not just a more efficient version of a read 
>> followed by a write.  It reads from one fd and write to another at tha
>> same time. Please try to read 2G, and then write 2G - and how much 
>> memory you will be need and how much time you will loose while reading
>> 2G from disk, but not writing them to socket.

DS> You are correct. What I meant to say was that it's just a
DS> more efficient version of 'mmap'ing a file and then 'write'ing
DS> from the 'mmap'. The 'write' to a non-blocking socket can still
DS> 'block' on disk I/O.

How can I avoid that blocking? Or maybe another question - how can I
deliver data from disk to network with minimal copy operations, etc.

>> If you know more 
>> efficient method to transfer file from disk to network - please advise.
>> Now all I want is really non-blocking sendfile. Currently sendfile is
>> non-blocking on network, but not on disk i/o. And when I have network
>> faster than disk - I get block.

DS> There are many different techniques and which is correct
DS> depends on what direction you want to go.
_very_ fast frontend web-server :) using as much kernel features as possible
DS> POSIX asynchronous I/O is one possibility.
aio does not support direct transfer file->socket. Using some hints,
like aio reading into shared memory and then sendfile from shared
memory, like lighttpd does, is not what I want. My tests showing
that aio-sendfile realization in lighttpd is slower than sendfile.
I think, sendfile uses less copy operations(maybe even zerocopy), than current aio
realizations in kernel.
DS> Threads plus epoll is another.
20k threads and maybe more is too much :). Look at http://nginx.net/
senction "Architecture and scalability" for example.
DS> It really depends upon how much performance you need
all, that hardware can take and hold :)

-- 
Best regards,
 Alex                            mailto:voron@...ost.net

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/