netdev - Re: [PATCH net-next 1/4] net: SO_INCOMING

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1444321775.27760.19.camel@edumazet-glaptop2.roam.corp.google.com>
Date:	Thu, 08 Oct 2015 09:29:35 -0700
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	Tom Herbert <tom@...bertland.com>
Cc:	Eric Dumazet <edumazet@...gle.com>,
	"David S . Miller" <davem@...emloft.net>,
	netdev <netdev@...r.kernel.org>
Subject: Re: [PATCH net-next 1/4] net: SO_INCOMING_CPU setsockopt() support

On Thu, 2015-10-08 at 09:03 -0700, Tom Herbert wrote:
> On Thu, Oct 8, 2015 at 8:37 AM, Eric Dumazet <edumazet@...gle.com> wrote:
> > SO_INCOMING_CPU as added in commit 2c8c56e15df3 was a getsockopt() command
> > to fetch incoming cpu handling a particular TCP flow after accept()
> >
> > This commits adds setsockopt() support and extends SO_REUSEPORT selection
> > logic : If a TCP listener or UDP socket has this option set, a packet is
> > delivered to this socket only if CPU handling the packet matches the specified one.
> >
> > This allows to build very efficient TCP servers, using one thread per cpu,
> > as the associated TCP listener should only accept flows handled in softirq
> > by the same cpu. This provides optimal NUMA/SMP behavior and keep cpu caches hot.
> >
> Please look again at my SO_INCOMING_CPU_MASK patches to see it these
> will work. I believe SO_INCOMING_CPU setsockopt is probably a subset
> of that functionality. A single CPU assigned to socket forces an
> application design to have one thread per CPU-- this may be overkill.
> It's probably be sufficient in many cases to have just one listener
> thread per NUMA node.

I think you misunderstood my patch.

For optimal behavior against DDOS, you need one TCP _listener_ per RX
queue on the NIC.

Not one listener per cpu !

You can then have multiple threads servicing requests coming for a
particular listener. Or one thread (cpu) servicing requests coming from
multiple TCP listeners. It's as you said an application choice/design.

Say you have 32 cpu (hyper threads) and 8 RX queues, you can create
groups of 4 threads per RX queue. Or one thread servicing all accept()

If you use 2 listeners (one per NUMA node), performance is not so good,
and SO_INCOMING_CPU_MASK adds an extra indirection, scaling poorly with
say 64 listeners.

Ying Cai has a patch to add an array selection (instead of having to
parse the list, yet to be rebased) 

I am afraid your choice of SO_INCOMING_CPU_MASK might have been driven
by the necessity of keeping the list short, but this is about to be a
non issue.

Thanks.

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html