netdev - [RFC] performance regression with commit-id<adb03115f459> ("net: get rid of an signed integer overflow in ip_idents

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 24 Jul 2019 16:38:30 +0800
From:   Zhangshaokun <zhangshaokun@...ilicon.com>
To:     Eric Dumazet <edumazet@...gle.com>, Jiri Pirko <jiri@...nulli.us>,
        <netdev@...r.kernel.org>, <linux-kernel@...r.kernel.org>
CC:     "David S. Miller" <davem@...emloft.net>,
        "guoyang (C)" <guoyang2@...wei.com>,
        "zhudacai@...ilicon.com" <zhudacai@...ilicon.com>
Subject: [RFC] performance regression with commit-id<adb03115f459> ("net: get
 rid of an signed integer overflow in ip_idents_reserve()")

Hi,

I've observed an significant performance regression with the following commit-id <adb03115f459>
("net: get rid of an signed integer overflow in ip_idents_reserve()").

Here are my test scenes:
----Server----
Cmd: iperf3 -s xxx.xxx.xxxx.xxx -p 10000 -i 0 -A 0
Kenel: 4.19.34
Server number: 32
Port: 10000 – 10032
CPU affinity: 0 – 32
CPU architecture: aarch64
NUMA node0 CPU(s): 0-23
NUMA node1 CPU(s): 24-47

----Client----
Cmd: iperf3 -u -c xxx.xxx.xxxx.xxx -p 10000 -l 16 -b 0 -t 0 -i 0 -A 8
Kenel: 4.19.34
Client number: 32
Port: 10000 – 10032
CPU affinity: 0 – 32
CPU architecture: aarch64
NUMA node0 CPU(s): 0-23
NUMA node1 CPU(s): 24-47

Firstly, With patch <adb03115f459> ("net: get rid of an signed integer overflow in ip_idents_reserve()") ,
client’s cpu is 100%, and function ip_idents_reserve() cpu usage is very high, but the result is not good.
03:08:32 AM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
03:08:33 AM      eth0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:08:33 AM      eth1      0.00 3461296.00      0.00 196049.97      0.00      0.00      0.00      0.00
03:08:33 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

Secondly, revert that patch, use atomic_add_return() instead, the result is better, as below:
03:23:24 AM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
03:23:25 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:23:25 AM      eth1      0.00 12834590.00      0.00 726959.20      0.00      0.00      0.00      0.00
03:23:25 AM      eth0      7.00     11.00      0.40      2.95      0.00      0.00      0.00      0.00

Thirdly, atomic is not used in ip_idents_reserve() completely ,while each cpu core allocates its own ID segment,
Such as: cpu core0 allocate ID 0 – 1023, cpu core1 allocate 1024 – 2047, …,etc
the result is the best:
03:27:06 AM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
03:27:07 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:27:07 AM      eth1      0.00 14275505.00      0.00 808573.53      0.00      0.00      0.00      0.00
03:27:07 AM      eth0      0.00      2.00      0.00      0.18      0.00      0.00      0.00      0.00

Because atomic operation performance is bottleneck when cpu core number increase, Can we revert the patch or
use ID segment for each cpu core instead?

Thanks in advance,
Shaokun