netdev - Re: [PATCH net-next] rps: overflow prevention for saturated cpus

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CA+FuTSc=-TE4X=VWmdXMLhE_jn6CtjUpauznt4evrRLi-DOd2g@mail.gmail.com>
Date:	Thu, 6 Dec 2012 18:04:23 -0500
From:	Willem de Bruijn <willemb@...gle.com>
To:	Rick Jones <rick.jones2@...com>
Cc:	netdev@...r.kernel.org, David Miller <davem@...emloft.net>,
	Eric Dumazet <edumazet@...gle.com>,
	Tom Herbert <therbert@...gle.com>
Subject: Re: [PATCH net-next] rps: overflow prevention for saturated cpus

On Thu, Dec 6, 2012 at 5:25 PM, Rick Jones <rick.jones2@...com> wrote:
> On 12/06/2012 12:36 PM, Willem de Bruijn wrote:
>>
>> RPS and RFS balance load across cpus with flow affinity. This can
>> cause local bottlenecks, where a small number or single large flow
>> (DoS) can saturate one CPU while others are idle.
>>
>> This patch maintains flow affinity in normal conditions, but
>> trades it for throughput when a cpu becomes saturated. Then, packets
>> destined to that cpu (only) are redirected to the lightest loaded cpu
>> in the rxqueue's rps_map. This breaks flow affinity under high load
>> for some flows, in favor of processing packets up to the capacity
>> of the complete rps_map cpuset in all circumstances.
>
>
> I thought (one of) the ideas behind RFS at least was to give the CPU
> scheduler control over where network processing took place instead of it
> being dictated solely by the addressing. I would have expected the CPU
> scheduler to migrate some work off the saturated CPU.  Or will this only
> affect RPS and not RFS?

I wrote it with RPS in mind, indeed. With RFS, for sufficiently
multithreaded applications that are unpinned, the scheduler will
likely spread the threads across as many cpus as possible. In that
case, the mechanism will not kick in, or as quickly. Even with RFS,
pinned threads and single-threaded applications will likely also
benefit during high load from redirecting kernel receive
processing away from the cpu that runs the application thread. I
haven't tested that case independently.

> Allowing individual flows to straddle the CPUs - won't that be somewhat like
> what happens in bonding with mode-rr in the outbound case - packet
> reordering evil?

Yes, that's the main drawback.

> What kind of workload is this targeting that calls for
> such intra-flow parallelism?

Packet processing middeboxes that rather operate in degraded mode
(reordering) than drop packets. Intrusion detection systems and proxies,
for instance. These boxes are actually likely to have RPS enabled and
RFS disabled.

> With respect to the examples given, what happens when it is TCP traffic
> rather than UDP?

That should be identical. RFS is supported for both protocols. In the test, it
is turned off to demonstrate the effect solely with RPS.

Restricting this to only RPS is easy, btw: get_rps_overflow_cpu() is called
one for RFS and once for RPS.

> happy benchmarking,
>
> rick jones
>
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html