netdev - Re: SO_REUSEPORT?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <489B578A.9030505@hp.com>
Date:	Thu, 07 Aug 2008 13:14:02 -0700
From:	Rick Jones <rick.jones2@...com>
To:	Tom Herbert <therbert@...gle.com>
CC:	Stephen Hemminger <stephen.hemminger@...tta.com>,
	netdev@...r.kernel.org
Subject: Re: SO_REUSEPORT?

> I'm not sure that's applicable for us since the server application and
> networking will max out all the CPUs on host anyway; one way or
> another we need to dispatch the work of incoming connections to
> threads on different CPUs.  If we do this in user space and do all
> accepts in one thread, the CPU of that  thread becomes the bottleneck
> (we're accepting about 40,000 connections per second).  If we have
> multiple accept threads running on different CPUs, this helps some,
> but the load is spread unevenly across the CPUs and we still can't get
> the highest connection rate.  So it seems we're looking for a method
> that distributes the incoming connection load across CPUs pretty
> evenly.

Well, if you _really_ want the load spread, you may need to use a 
multiqueue (at least inbound if not also later outbound) interface, 
"know" how the NIC will hash and then have N distinct port numbers each 
assigned to a LISTEN endpoint.  The old song and dance about making an N 
CPU system look as much like N single-CPU systems and all that...

Unless there are NICs you can "tell" where to send the interrupts, which 
IMO is preferable - I have a preference for the application/scheduler 
telling "networking" where to work rather than networking (or the NIC) 
telling the scheduler where to run a thread - the archives of either 
here or netnews will probalby pull-up stuff were I've talked about 
Inbound Packet Scheduling (IPS) vs Thread Optimized Packet Scheduling 
(TOPS) and limitations of simplistic address hashing to pick a 
queue/processor/whatnot :)

rick jones
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html