netdev - Re: Packet mmap: TX RING and zero copy

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <7e0dd21a0809031330i1363c4aema9c57ef06a3d5279@mail.gmail.com>
Date:	Wed, 3 Sep 2008 22:30:14 +0200
From:	"Johann Baudy" <johaahn@...il.com>
To:	"Evgeniy Polyakov" <johnpol@....mipt.ru>
Cc:	"David Miller" <davem@...emloft.net>, netdev@...r.kernel.org
Subject: Re: Packet mmap: TX RING and zero copy

Hi Evgeniy,

> So small PLB? Not OPB? Weird hardware :)
> But nevertheless at most 400 MB/s with 100mhz, so looks like either
> there is no burst mode or weird NIC hardware (or something else :)
> I used to easily saturate 100mbit channel with 405gp(r) and emac driver,
> which are better numbers than what you have with gige and sockets...
> Actually even 405gp had much wider plb, so this could be an issue.
>
> Likley your project will just dma data from some sensor to the
> preallocated buffer, you will add headers and send the data, so very
> small memory bus speed will not allow to use sockets and thus TCP.
> Having splice-friendly setup is possible, but I think raw socket
> approach is simpler for you.

Yes, this is a custom hardware (FPGA :)). There is no combo IPLB / DPLB,
only one and small PLB bus at 100Mhz.

> No, it will use sendpage() if hardware and driver support scatter/gather
> and checksumm ofloading. Since you say they do, then there should be no
> copies at all.
>

> It will endup with generic_splice_sendpage() and pipe_to_sendpage().
>
Indeed, I've double checked, but pipe_to_sendpage() will end up with
packet_sendmsg()

.splice_write = generic_splice_sendpage,
generic_splice_sendpage()
splice_from_pipe();
pipe_to_sendpage() from err = actor(pipe, buf, sd);
sock_sendpage() from ile->f_op->sendpage()
sock_no_sendpage() from sock->ops->sendpage()
kernel_sendmsg()
sock_sendmsg();
packet_sendmsg() from sock->ops->sendmsg();
memcpy() :'(

I think a non-generic splice_write function should do the job.
What do you think?

>
> What do you mean by packet size? MTU/MSS? In pktgen it means size of the
> allocated skb, so it will be eventually split into smaller chunks and the
> bigger size you have, the less allocations will be performed. Actually
> the fact, that 7200 works at all, is a bit surprising: your small
> machine has lots of ram and is effectively unused during tests (i.e. no
> other allocations). Changing it do 4k should not decrease performance at
> all... Do you have jumbo frames enabled?
>

I mean the transfer unit size (ethernet frame length) that must be <= MTU.
Jumbo frames are enabled in the driver and mtu size is set to 7200.
I'm currently using wireshark on a remote pc to check bitrate and format.
I think performance can decrease because CPU will spend the same time
to send 7200 or 4096 bytes but not the DMA.(~50µs for 7200, ~30µs for
4096)


Thanks,
Johann

-- 
Johann Baudy
johaahn@...il.com
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html