netdev - RE: Kernel UDP behavior with missing destinations

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 17 May 2019 12:57:36 +0000
From:   David Laight <David.Laight@...LAB.COM>
To:     'Willem de Bruijn' <willemdebruijn.kernel@...il.com>,
        Adam Urban <adam.urban@...leguru.org>
CC:     Eric Dumazet <eric.dumazet@...il.com>,
        Network Development <netdev@...r.kernel.org>
Subject: RE: Kernel UDP behavior with missing destinations

From: Willem de Bruijn
> Sent: 17 May 2019 04:23
> On Thu, May 16, 2019 at 8:27 PM Adam Urban <adam.urban@...leguru.org> wrote:
> >
> > And replying to your earlier comment about TTL, yes I think a TTL on
> > arp_queues would be hugely helpful.
> >
> > In any environment where you are streaming time-sensitive UDP traffic,
> > you really want the kernel to be tuned to immediately drop the
> > outgoing packet if the destination isn't yet known/in the arp table
> > already...

I suspect we may suffer from the same problems when sending out a lot
of RTP (think of sending 1000s of UDP messages to different addresses
every 20ms).
For various reasons the sends are done from a single raw socket (rather
than 'connected' UDP sockets).

> For packets that need to be sent immediately or not at all, you
> probably do not want a TTL, but simply for the send call to fail
> immediately with EAGAIN instead of queuing the packet for ARP
> resolution at all. Which is approximated with unres_qlen 0.
> 
> The relation between unres_qlen_bytes, arp_queue and SO_SNDBUF is
> pretty straightforward in principal. Packets can be queued on the arp
> queue until the byte limit is reached. Any packets on this queue still
> have their memory counted towards their socket send budget. If a
> packet is queued that causes to exceed the threshold, older packets
> are freed and dropped as needed. Calculating the exact numbers is not
> as straightforward, as, for instance, skb->truesize is a kernel
> implementation detail.

But 'fiddling' with the arp queue will affect all traffic.
So you'd need it to be per socket option so that it is a property
of the message by the time it reaches the arp code.

> The simple solution is just to overprovision the socket SO_SNDBUF. If
> there are few sockets in the system that perform this role, that seems
> perfectly fine.

That depends on how often you are sending messages compared to the
arp timeout. If you are sending 50 messages a second to each of 1000
destinations the over provisioning of SO_SNDBUF would have to be extreme.

FWIW we do sometimes see sendmsg() taking much longer than expected,
but haven't get tracked down why.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)