netdev - Re: [RFC v2 bpf-next 8/9] bpf: Provide helper to do lookups in kernel FIB table

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 2 May 2018 13:27:36 +0200
From:   Jesper Dangaard Brouer <brouer@...hat.com>
To:     David Ahern <dsahern@...il.com>
Cc:     netdev@...r.kernel.org, borkmann@...earbox.net, ast@...nel.org,
        davem@...emloft.net, shm@...ulusnetworks.com,
        roopa@...ulusnetworks.com, toke@...e.dk, john.fastabend@...il.com,
        brouer@...hat.com, Vincent Bernat <bernat@...fy.cx>
Subject: Re: [RFC v2 bpf-next 8/9] bpf: Provide helper to do lookups in
 kernel FIB table

On Sun, 29 Apr 2018 11:07:51 -0700
David Ahern <dsahern@...il.com> wrote:

> Initial performance numbers collected by Jesper, forwarded packets/sec:
> 
>        Full stack    XDP FIB lookup    XDP Direct lookup
> IPv4   1,947,969       7,074,156          7,415,333
> IPv6   1,728,000       6,165,504          7,262,720

Do notice these number is single CPU core forwarding performance!
On a Broadwell E5-1650 v4 @ 3.60GHz.

Another interesting data point is that xdp_redirect_map performance is
13,365,161 pps, which allow us to calculate/isolate the overhead/cost
of the FIB lookup.

(1/13365161-1/7074156)*10^9 = -66.5 ns
(1/13365161-1/7415333)*10^9 = -60.0 ns

Which is very close to the measured 50 ns cost of the FIB lookup, done
by Vincent Bernat.
  See: https://vincent.bernat.im/en/blog/2017-ipv4-route-lookup-linux

Another way I calculate this is by (ran a new benchmark):

Performance: 7641593 (7,641,593) <= tx_unicast /sec
 * Packet-gap: (1/7641593*10^9)  = 130.86 ns

Find all FIB related lookup functions in perf-report::

 Samples: 93K of event 'cycles:ppp', Event count (approx.): 88553104553
   Overhead   Cost      CPU  Command      Symbol
     20.63 %  26.99 ns  002  ksoftirqd/2  [k] fib_table_lookup
     12.92 %  16.90 ns  002  ksoftirqd/2  [k] bpf_fib_lookup
      2.40 %   3.14 ns  002  ksoftirqd/2  [k] fib_select_path
      0.83 %   1.09 ns  002  ksoftirqd/2  [k] fib_get_table
      0.40 %   0.52 ns  002  ksoftirqd/2  [k] l3mdev_fib_table_rcu
 -----------
 Tot:37.18 % (20.63+12.92+2.40+0.83+0.40)
 ===========

Cost of FIB lookup:
 - 130.86/100*37.18 = 48.65 ns overhead by FIB lookup.

Again very close to Vincent's IPv4 measurements of ~50 ns.

Notice that the IPv6 measurements does not match up:
 https://vincent.bernat.im/en/blog/2017-ipv6-route-lookup-linux
This is because, we/I'm just testing the IPv6 route cache here...

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer