netdev - Re: Fwd: Packet mmap: TX RING and zero copy

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 5 Sep 2008 14:44:47 +0200
From:	"Johann Baudy" <johaahn@...il.com>
To:	"Evgeniy Polyakov" <johnpol@....mipt.ru>
Cc:	netdev@...r.kernel.org
Subject: Re: Fwd: Packet mmap: TX RING and zero copy

Hi Evgeniy,

>> vmsplice() is short in comparison to splice()  ~ 200us !
>> This was just to show you that even this vmpslice duration of 80us
>> that is needed for each packet is too long to send only 1 packet.
>> I really need a mechanism that allow sending of ~ 40 packets of 7200K
>> in one system call to keep some cpu ressources to do other things.
>> (Not spending time in kernel layers :))
>
> Hmmm... splice()/sendfile() shuold be able to send the whole file in
> single syscall. This looks like a problem in the userspace.
>

I was talking about vmsplice()/splice() which seems to me the only way
to send user buffer (pages) to socket without copy.

>> > kill_fasync() also took too much time (top CPU user
>> > is at bottom I suppose?), do you use SIGIO? Also vma traveling and page
>> > checking is not what will be done in network code and your project, so
>> > it also adds an overhead.
>>
>> Between kill_fasync() sys_gettimeofday() , I thought that we returned
>> to user space.
>> No SIGIO. But FYI, I use PREEMPT_RT patch.
>
> Does it also push softirq processing into threads?
>
I don't understand your point, how can I check?
I'm not handling any IRQ in this test software.

>> >Please try without vmsplice() at all, usual
>> > splice()/sendfile() _has_ to saturate the link, otherwise we have a
>> > serious problem.
>>
>> I've already tried sendfile only with standard TCP/UDP socket. I've
>> not saturated the link.
>> Around same bitrate.
>
> This worries me a lot: sendfile should be a single syscall which very
> optimally creates network packets getting into account MTU and hardware
> capabilities. I do belive it is a problem with userspace code.
>

Yes, and this is what it does (only one single syscall).
No printf, only one sendfile of 10MB file over TCP socket

To resume ongoing test status:
with vmsplice()/splice() I need to do multiple call of vmsplice() and
one call of splice() - ratio seems to be limited to the pipe capacity
(16 pages: 64K)
                                      - vmsplice call specify the size
of the udp packet which means 1 syscall per packet :(
                                      - In UDP, Bitrate is  < 20MB/s


with sendfile() only one system call of 10MB  in TCP (in UDP I have to
split in 61440 bytes).
                                      - In TCP bitrate is limited due to remote
                                      - In UDP,  this 61440 bytes
limit which  is really inferior to my 7200*40 packets that allows me
to saturate the link (during 2ms only...) with my circular buffer.
                                     - Bitrate is  < 20MB/s





Thanks,
Johann

-- 
Johann Baudy
johaahn@...il.com
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html