lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Message-ID: <878snxo5kq.fsf@cloudflare.com> Date: Sat, 30 Nov 2019 14:29:41 +0100 From: Jakub Sitnicki <jakub@...udflare.com> To: David Laight <David.Laight@...LAB.COM> Cc: Eric Dumazet <eric.dumazet@...il.com>, 'Paolo Abeni' <pabeni@...hat.com>, Jesper Dangaard Brouer <brouer@...hat.com>, 'Marek Majkowski' <marek@...udflare.com>, linux-kernel <linux-kernel@...r.kernel.org>, network dev <netdev@...r.kernel.org>, kernel-team <kernel-team@...udflare.com> Subject: Re: epoll_wait() performance On Sat, Nov 30, 2019 at 02:07 AM CET, Eric Dumazet wrote: > On 11/28/19 2:17 AM, David Laight wrote: >> From: Eric Dumazet >>> Sent: 27 November 2019 17:47 >> ... >>> A QUIC server handles hundred of thousands of ' UDP flows' all using only one UDP socket >>> per cpu. >>> >>> This is really the only way to scale, and does not need kernel changes to efficiently >>> organize millions of UDP sockets (huge memory footprint even if we get right how >>> we manage them) >>> >>> Given that UDP has no state, there is really no point trying to have one UDP >>> socket per flow, and having to deal with epoll()/poll() overhead. >> >> How can you do that when all the UDP flows have different destination port numbers? >> These are message flows not idempotent requests. >> I don't really want to collect the packets before they've been processed by IP. >> >> I could write a driver that uses kernel udp sockets to generate a single message queue >> than can be efficiently processed from userspace - but it is a faff compiling it for >> the systems kernel version. > > Well if destinations ports are not under your control, > you also could use AF_PACKET sockets, no need for 'UDP sockets' to receive UDP traffic, > especially it the rate is small. Alternatively, you could steer UDP flows coming to a certain port range to one UDP socket using TPROXY [0, 1]. TPROXY has the same downside as AF_PACKET, meaning that it requires at least CAP_NET_RAW to create/set up the socket. OTOH, with TPROXY you can gracefully co-reside with other services, filering on just the destination addresses you want in iptables/nftables. Fan-out / load-balancing with reuseport to have one socket per CPU is not possible, though. You would need to do that with Netfilter. -Jakub [0] https://www.kernel.org/doc/Documentation/networking/tproxy.txt [1] https://blog.cloudflare.com/how-we-built-spectrum/
Powered by blists - more mailing lists