lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Date:   Fri, 17 May 2019 09:20:23 -0400
From:   Willem de Bruijn <>
To:     David Laight <>
Cc:     Adam Urban <>,
        Eric Dumazet <>,
        Network Development <>
Subject: Re: Kernel UDP behavior with missing destinations

On Fri, May 17, 2019 at 8:57 AM David Laight <> wrote:
> From: Willem de Bruijn
> > Sent: 17 May 2019 04:23
> > On Thu, May 16, 2019 at 8:27 PM Adam Urban <> wrote:
> > >
> > > And replying to your earlier comment about TTL, yes I think a TTL on
> > > arp_queues would be hugely helpful.
> > >
> > > In any environment where you are streaming time-sensitive UDP traffic,
> > > you really want the kernel to be tuned to immediately drop the
> > > outgoing packet if the destination isn't yet known/in the arp table
> > > already...
> I suspect we may suffer from the same problems when sending out a lot
> of RTP (think of sending 1000s of UDP messages to different addresses
> every 20ms).
> For various reasons the sends are done from a single raw socket (rather
> than 'connected' UDP sockets).
> > For packets that need to be sent immediately or not at all, you
> > probably do not want a TTL, but simply for the send call to fail
> > immediately with EAGAIN instead of queuing the packet for ARP
> > resolution at all. Which is approximated with unres_qlen 0.
> >
> > The relation between unres_qlen_bytes, arp_queue and SO_SNDBUF is
> > pretty straightforward in principal. Packets can be queued on the arp
> > queue until the byte limit is reached. Any packets on this queue still
> > have their memory counted towards their socket send budget. If a
> > packet is queued that causes to exceed the threshold, older packets
> > are freed and dropped as needed. Calculating the exact numbers is not
> > as straightforward, as, for instance, skb->truesize is a kernel
> > implementation detail.
> But 'fiddling' with the arp queue will affect all traffic.
> So you'd need it to be per socket option so that it is a property
> of the message by the time it reaches the arp code.

A per socket or even datagram do-not-queue signal would be
interesting. Where any queuing would instead result in send failure
(though this feedback signal does not work for secondary qdiscs).

The recent SCM_TXTIME cmsg has a deadline mode that might implement
this. In which case we would only have to check for it in the neighbor

> > The simple solution is just to overprovision the socket SO_SNDBUF. If
> > there are few sockets in the system that perform this role, that seems
> > perfectly fine.
> That depends on how often you are sending messages compared to the
> arp timeout. If you are sending 50 messages a second to each of 1000
> destinations the over provisioning of SO_SNDBUF would have to be extreme.
> FWIW we do sometimes see sendmsg() taking much longer than expected,
> but haven't get tracked down why.

I've observed this problem with health checks under particular ARP
settings as well.

Powered by blists - more mailing lists