netdev - Re: TCP communication for raw image transmission

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 2 Jan 2012 17:08:26 +0100
From:	Jean-Michel Hautbois <jhautbois@...il.com>
To:	Eric Dumazet <eric.dumazet@...il.com>
Cc:	netdev@...r.kernel.org
Subject: Re: TCP communication for raw image transmission

2012/1/2 Eric Dumazet <eric.dumazet@...il.com>:
> Le lundi 02 janvier 2012 à 16:21 +0100, Jean-Michel Hautbois a écrit :
>> Hi all,
>>
>> I am currently working on a way to send raw images (~400*300 YUV
>> 4:2:0) at a relative high speed (raw image and not compressed image
>> due to specific codec issues).
>> This would represent about 57MBps/sec on a 100Mbps ethernet link.
>> This link is direct, nothing else will pass through it.
>>
>> My question is a performance and efficiency question. Here is what I
>> am thinking about :
>> - I have the image on one computer at a known memory adress
>> - I want top transmit this picture using TCP packets with the rule
>> "1row = 1packet", maybe "2rows=1packet"
>> - All these packets have to go on an ethernet link and will be
>> received on another computer starting at a known memory adress
>> Something like this :
>> http://imageshack.us/photo/my-images/15/tcpmemory.png/
>>
>> I was thinking about using TCP socket (of course) and the sendfile
>> syscall on sender side, splice syscall on the receiver side, but I
>> don't know if this is the best (and fastest) way to do it ?
>> I would like to avoid memory copy as much as possible as the systems
>> on which it will be done are embedded arm computers.
>> TCP is chosen in order to get ordered packets, but maybe am I wrong ?
>>
>> Thanks for any clues you can give !
>> Regards,
>> JM
>> --
>
> Do you really need full image, or are some missing lines allowed ?
>
> What is the length in bytes for a line ?

I can't accept any missing line, and line length in bytes is to be
decided but I think it would be something like 672bytes. This is why I
would put two lines per packet.

> TCP stream and your bandwidth requirements would work only if you
> guarantee not a single packet loss.
>
> Using UDP (RTP) would allow you to respect any time limits at the
> sender, yet allowing some drops on the network.

Yes, and again, I don't want any missing lines :).

> If you use a direct link, packets wont be re-ordered.
>
> Is your ARM platform so slow it cannot afford copying 10 Mbytes per
> second in skb ?

I don't really know what it can do, but it has to do encoding (even if
it is a SoC, with dedicated devices, memory has to be used).
The main problem I have, is that I don't have the two boards with me
today, and even if I can prototype on x86, I can't conclude as far as
I don't test on the real HW.
This is also why I want to prototype my best bet in the first try :).
And this is mainly a networking/memory problem.

Thanks for answering.
JM
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html