netdev - Re: [RFC PATCH v2 5/5] Add sample for adding simple drop program to link

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <57093B67.1080604@mojatatu.com>
Date:	Sat, 9 Apr 2016 13:27:03 -0400
From:	Jamal Hadi Salim <jhs@...atatu.com>
To:	Brenden Blanco <bblanco@...mgrid.com>
Cc:	davem@...emloft.net, netdev@...r.kernel.org, tom@...bertland.com,
	alexei.starovoitov@...il.com, ogerlitz@...lanox.com,
	daniel@...earbox.net, brouer@...hat.com, eric.dumazet@...il.com,
	ecree@...arflare.com, john.fastabend@...il.com, tgraf@...g.ch,
	johannes@...solutions.net, eranlinuxmellanox@...il.com,
	lorenzo@...gle.com
Subject: Re: [RFC PATCH v2 5/5] Add sample for adding simple drop program to
 link

On 16-04-09 12:43 PM, Brenden Blanco wrote:
> On Sat, Apr 09, 2016 at 10:48:05AM -0400, Jamal Hadi Salim wrote:

>> Ok, sorry - should have looked this far before sending earlier email.
>> So when you run concurently you see about 5Mpps per core but if you
>> shoot all traffic at a single core you see 20Mpps?
> No, only sender is multiple, receiver is still single core. The flow is
> the same in all 4 of the send threads. Note that only ksoftirqd/6 is
> active.

Got it.
The sender was limited to the 20Mpps and you are able to keep up
if i understand correctly.

>>
>> Devil's advocate question:
>> If the bottleneck is the driver - is there an advantage in adding the
>> bpf code at all in the driver?
> Only by adding this hook into the driver has it become the bottleneck.
 >
> Prior to this, the bottleneck was later in the codepath, primarily in
> allocations.
>

Maybe useful in your commit log to show the prior and after.
Looking at both your and Daniel's profile you show in this email
mlx4_en_process_rx_cq() seems to be where the action is on both, no?

> If a packet is to be dropped, and a determination can be made with fewer
> cpu cycles spent, then there is more time for the goodput.
>

Agreed.

> Beyond that, even if the skb allocation gets 10x or 100x or whatever
> improvement, there is still a non-zero cost associated, and dropping bad
> packets with minimal time spent has value. The same argument holds for
> physical nic forwarding decisions.
>

I always go for the lowest hanging fruit.
It seemed it was the driver path in your case. When we removed
the driver overhead (as demoed at the tc workshop in netdev11) we saw
__netif_receive_skb_core() at the top of the profile.
So in this case seems it was mlx4_en_process_rx_cq() - thats why i
was saying the bottleneck is the driver.
Having said that: I agree that early drop is useful if not for anything
else to avoid the longer code path (but was worried after reading on
thread this was going to get into a messy stack-in-the-driver and i am
not sure it is avoidable either given a new ops interface is showing
  up).

>> I am curious than before to see the comparison for the same bpf code
>> running at tc level vs in the driver..
> Here is a perf report for drop in the clsact qdisc with direct-action,
> which Daniel earlier showed to have the best performance to-date. On my
> machine, this gets about 6.5Mpps drop single core. Drop due to failed
> IP lookup (not shown here) is worse @4.5Mpps.
>

Nice.
However, still for this to be orange/orange comparison you have to
run it on the _same receiver machine_ as opposed to Daniel doing
it on his for the one case. And two different kernels booted up
one patched  with your changes and another virgin without them.

cheers,
jamal