netdev - Re: TCP higher throughput that UDP?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CADVnQyn-+bbvhnx7PvstUNNBWVkTGd+TLPfeJxgy+BGT_or_-Q@mail.gmail.com>
Date:   Tue, 31 Aug 2021 16:40:53 -0400
From:   Neal Cardwell <ncardwell@...gle.com>
To:     Eric Curtin <ericcurtin17@...il.com>
Cc:     netdev@...r.kernel.org, linux-newbie@...r.kernel.org,
        Eric Dumazet <edumazet@...gle.com>,
        Willem de Bruijn <willemb@...gle.com>
Subject: Re: TCP higher throughput that UDP?

On Tue, Aug 31, 2021 at 4:25 PM Eric Curtin <ericcurtin17@...il.com> wrote:
>
> Hi Guys,
>
> I've been researching quantitatively various protocols QUIC, TCP, UDP,
> SCTP, etc. on modern hardware. And some TLS, DTLS and plain text
> variants of these protocols. I've avoided tuning kernel parameters so
> much and focused more on adjusting what I can in user space on an out
> of the box kernel, as if I was deploying on a machine where I did not
> have root access (lets say an Android phone, even though these results
> come from a laptop for simplicity). I did make a minor exception in
> that I had to turn on SCTP.
>
> I have a small client/server binary that implements bare bones
> versions of TCP, UDP and SCTP. The server allows you to specify which
> file clients will pull. And the client lets you specify which file to
> write the pulled data to (often I use /dev/null here). I run both on
> 127.0.0.1 . I would have thought (obviously I was wrong) that UDP
> would be faster, given no acks, retransmits, checksums, etc. as many
> laymen would tell you. But on this machine TCP is faster more often
> than not. This has been answered online with different answers of the
> reasoning (hardware offload, how packet fragmentation is handled, TCP
> tuned more for throughput, UDP tuned more for latency, etc.). I'm just
> curious if someone could let me know the most significant factor (or
> factors) here these days? Things like strace don't reveal much.
> Changing buffer size to the read/write/send/recv system calls alters
> things but TCP seems to still win regardless of buffer size.

These articles have some nice discussion and description of
experimental results that shed some light on the differences between
UDP performance and TCP performance, and what it takes to make UDP
performance approximate TCP on Linux:

  Optimizing UDP for content delivery: GSO, pacing and zerocopy
  http://vger.kernel.org/lpc_net2018_talks/willemdebruijn-lpc2018-udpgso-paper-DRAFT-1.pdf

  Can QUIC match TCP’s computational efficiency?
  https://www.fastly.com/blog/measuring-quic-vs-tcp-computational-efficiency

I would guess segmentation offload for TCP is probably the biggest
factor in your experiments.

best,
neal