netdev - Re: UDP multi-core performance on a single socket and SO

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CA+mtBx_+xR5Y5zEn6nxQ=++FEWKUw+zHzP4MamM_EEF-mMXB1Q@mail.gmail.com>
Date:	Fri, 4 Jan 2013 12:47:04 -0800
From:	Tom Herbert <therbert@...gle.com>
To:	Eric Dumazet <eric.dumazet@...il.com>
Cc:	Mark Zealey <netdev@...kandruth.co.uk>, netdev@...r.kernel.org
Subject: Re: UDP multi-core performance on a single socket and SO_REUSEPORT

I believe the hard part of making SO_REUSEPORT was on the TCP side in
dealing with state in req structs which we have not resolved.  UDP
SO_REUSEPORT seems to be working pretty well.

Tom

On Fri, Jan 4, 2013 at 11:37 AM, Eric Dumazet <eric.dumazet@...il.com> wrote:
> On Fri, 2013-01-04 at 18:50 +0000, Mark Zealey wrote:
>> I have written two small test scripts now which can be found at
>> http://mark.zealey.org/uploads/ - one launches 16 listening threads for
>> a single UDP socket, the other needs to be run as
>>
>> for i in `seq 16`; do ./udp_test_client & done
>>
>> On my test server (32-core), stock kernel 3.7.1, 90% of the time is
>> spent in the kernel waiting on spinlocks. Perf output:
>
> Mark
>
> We know the scalability issue of using a single socket and many threads.
>
> The send path was somehow fixed to not require socket lock.
>
> But the receive path uses a single receive_queue, protected by a
> spinlock.
>
> SO_REUSEPORT would be nice, but had known issues.
>
> af_packet fanout implementation was nicer.
> You could try :
>
> 1) Use af_packet FANOUT instead of UDP sockets
>
> 2) rewrite SO_REUSEPORT to use a FANOUT like implementation
>
> 3) Extend UDP sockets to be able to use a configurable number of receive
> queues instead of a single one.
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html