netdev - Re: Kernel 4.19 network performance - forwarding/routing normal users traffic

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Sun, 11 Nov 2018 09:56:27 +0100
From:   Jesper Dangaard Brouer <brouer@...hat.com>
To:     Paweł Staszewski <pstaszewski@...are.pl>
Cc:     Saeed Mahameed <saeedm@...lanox.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        brouer@...hat.com
Subject: Re: Kernel 4.19 network performance - forwarding/routing normal
 users traffic

On Sat, 10 Nov 2018 22:53:53 +0100 Paweł Staszewski <pstaszewski@...are.pl> wrote:

> Now im messing with ring configuration for connectx5 nics.
> And after reading that paper:
> https://netdevconf.org/2.1/slides/apr6/network-performance/04-amir-RX_and_TX_bulking_v2.pdf
>

Do notice that some of the ideas in that slide deck, was never
implemented. But they are still on my todo list ;-).

Notice how that it show that TX bulking is very important, but based on
your ethtool_stats.pl, I can see that not much TX bulking is happening
in your case.  This is indicated via the xmit_more counters.

 Ethtool(enp175s0) stat:    2630 (     2,630) <= tx_xmit_more /sec
 Ethtool(enp175s0) stat: 4956995 ( 4,956,995) <= tx_packets /sec

And the per queue levels are also avail:

 Ethtool(enp175s0) stat: 184845 ( 184,845) <= tx7_packets /sec
 Ethtool(enp175s0) stat:     78 (      78) <= tx7_xmit_more /sec

This means that you are doing too many doorbell's to the NIC hardware
at TX time, which I worry could be what cause the NIC and PCIe hardware
not to operate at optimal speeds.

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer