| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Thu, 20 Jun 2019 10:09:56 +0800
From: kernel test robot <rong.a.chen@...el.com>
To: Eric Dumazet <edumazet@...gle.com>
Cc: "David S. Miller" <davem@...emloft.net>,
LKML <linux-kernel@...r.kernel.org>,
Linus Torvalds <torvalds@...ux-foundation.org>, lkp@...org
Subject: [tcp] ede61ca474: netperf.Throughput_Mbps 36.5% improvement
Greeting,
FYI, we noticed a 36.5% improvement of netperf.Throughput_Mbps due to commit:
commit: ede61ca474a0348b975d9824565b66c7595461de ("tcp: add tcp_rx_skb_cache sysctl")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: netperf
on test machine: 72 threads Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz with 128G memory
with following parameters:
ip: ipv4
runtime: 600s
nr_threads: 200%
cluster: cs-localhost
test: TCP_STREAM
cpufreq_governor: performance
test-description: Netperf is a benchmark that can be use to measure various aspect of networking performance.
test-url: http://www.netperf.org/netperf/
In addition to that, the commit also has significant impact on the following tests:
+------------------+------------------------------------------------------------+
| testcase: change | netperf: |
| test machine | 96 threads Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory |
| test parameters | cluster=cs-localhost |
| | cpufreq_governor=performance |
| | ip=ipv4 |
| | nr_threads=200% |
| | runtime=600s |
| | test=TCP_STREAM |
| | ucode=0x400001c |
+------------------+------------------------------------------------------------+
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase:
cs-localhost/gcc-7/performance/ipv4/x86_64-rhel-7.6/200%/debian-x86_64-2018-04-03.cgz/600s/lkp-skl-2sp7/TCP_STREAM/netperf
commit:
a8e11e5c56 ("sysctl: define proc_do_static_key()")
ede61ca474 ("tcp: add tcp_rx_skb_cache sysctl")
a8e11e5c5611a9f7 ede61ca474a0348b975d9824565
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
2:4 -50% :5 dmesg.WARNING:at#for_ip_interrupt_entry/0x
:4 50% 2:5 dmesg.WARNING:at_ip__netif_receive_skb_core/0x
:4 25% 1:5 dmesg.WARNING:at_ip_ip_finish_output2/0x
%stddev %change %stddev
\ | \
3310 +36.5% 4518 netperf.Throughput_Mbps
476765 +36.5% 650735 netperf.Throughput_total_Mbps
1.443e+09 +36.0% 1.962e+09 netperf.time.involuntary_context_switches
26037 -1.9% 25552 netperf.time.system_time
926.25 +65.4% 1532 ± 2% netperf.time.user_time
4705 ± 4% +883.6% 46279 ± 5% netperf.time.voluntary_context_switches
2.182e+09 +36.5% 2.979e+09 netperf.workload
317804 ± 6% -7.3% 294661 ± 6% meminfo.DirectMap4k
2925 -5.2% 2774 turbostat.Avg_MHz
22.65 -7.2 15.43 ± 3% mpstat.cpu.all.soft%
4.78 +2.2 6.93 ± 2% mpstat.cpu.all.usr%
5.39e+08 -8.6% 4.928e+08 ± 2% numa-numastat.node0.local_node
5.39e+08 -8.6% 4.928e+08 ± 2% numa-numastat.node0.numa_hit
2.691e+08 -9.2% 2.443e+08 numa-vmstat.node0.numa_hit
2.691e+08 -9.2% 2.442e+08 numa-vmstat.node0.numa_local
94.00 -2.1% 92.00 vmstat.cpu.sy
4796080 +35.7% 6509475 vmstat.system.cs
153412 +1.3% 155421 vmstat.system.in
1.092e+09 -8.4% 1e+09 proc-vmstat.numa_hit
1.092e+09 -8.4% 1e+09 proc-vmstat.numa_local
8.728e+09 -8.4% 7.993e+09 proc-vmstat.pgalloc_normal
8.727e+09 -8.4% 7.992e+09 proc-vmstat.pgfree
811.55 -9.8% 732.06 sched_debug.cfs_rq:/.util_est_enqueued.avg
1586 ± 3% -10.6% 1417 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.max
293.47 ± 2% -15.6% 247.77 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.stddev
4.36 +10.4% 4.82 ± 2% sched_debug.cpu.cpu_load[2].min
7098 ± 3% -11.8% 6262 ± 11% sched_debug.cpu.load.min
20026979 +35.1% 27057938 sched_debug.cpu.nr_switches.avg
20909879 +36.3% 28506716 sched_debug.cpu.nr_switches.max
18629543 +31.2% 24444109 sched_debug.cpu.nr_switches.min
521083 ± 8% +81.1% 943750 ± 8% sched_debug.cpu.nr_switches.stddev
24.09 -24.2% 18.26 perf-stat.i.MPKI
1.233e+10 +27.5% 1.572e+10 perf-stat.i.branch-instructions
1.93 +0.0 1.97 perf-stat.i.branch-miss-rate%
2.338e+08 +31.0% 3.064e+08 perf-stat.i.branch-misses
1.452e+09 -1.9% 1.424e+09 perf-stat.i.cache-references
4822346 +35.7% 6545267 perf-stat.i.context-switches
3.48 -26.7% 2.55 ± 2% perf-stat.i.cpi
2.099e+11 -5.3% 1.989e+11 perf-stat.i.cpu-cycles
1.04 ± 4% +0.1 1.16 ± 2% perf-stat.i.dTLB-load-miss-rate%
1.904e+08 ± 4% +44.6% 2.753e+08 ± 2% perf-stat.i.dTLB-load-misses
1.818e+10 +29.5% 2.355e+10 perf-stat.i.dTLB-loads
36389994 ± 6% +36.7% 49728544 ± 6% perf-stat.i.dTLB-store-misses
1.035e+10 +32.0% 1.367e+10 perf-stat.i.dTLB-stores
49.33 +2.6 51.92 perf-stat.i.iTLB-load-miss-rate%
40247503 ± 2% +43.2% 57619885 ± 2% perf-stat.i.iTLB-load-misses
41247559 +28.9% 53170095 perf-stat.i.iTLB-loads
6.12e+10 +28.1% 7.842e+10 perf-stat.i.instructions
1550 ± 2% -8.5% 1418 ± 2% perf-stat.i.instructions-per-iTLB-miss
0.29 +35.3% 0.40 ± 2% perf-stat.i.ipc
676689 ± 22% +161.1% 1766997 ± 15% perf-stat.i.node-loads
23.72 -23.5% 18.15 perf-stat.overall.MPKI
1.90 +0.1 1.95 perf-stat.overall.branch-miss-rate%
3.43 -26.1% 2.54 ± 2% perf-stat.overall.cpi
1.04 ± 4% +0.1 1.16 ± 2% perf-stat.overall.dTLB-load-miss-rate%
49.38 +2.6 52.00 perf-stat.overall.iTLB-load-miss-rate%
1521 ± 2% -10.5% 1361 ± 2% perf-stat.overall.instructions-per-iTLB-miss
0.29 +35.3% 0.39 ± 2% perf-stat.overall.ipc
71.87 ± 5% -23.4 48.48 ± 25% perf-stat.overall.node-load-miss-rate%
16855 -6.1% 15820 perf-stat.overall.path-length
1.231e+10 +27.5% 1.569e+10 perf-stat.ps.branch-instructions
2.334e+08 +31.0% 3.059e+08 perf-stat.ps.branch-misses
1.449e+09 -1.9% 1.421e+09 perf-stat.ps.cache-references
4814104 +35.7% 6533972 perf-stat.ps.context-switches
2.096e+11 -5.3% 1.985e+11 perf-stat.ps.cpu-cycles
1.901e+08 ± 4% +44.6% 2.748e+08 ± 2% perf-stat.ps.dTLB-load-misses
1.815e+10 +29.5% 2.351e+10 perf-stat.ps.dTLB-loads
36327516 ± 6% +36.7% 49642182 ± 6% perf-stat.ps.dTLB-store-misses
1.033e+10 +32.0% 1.364e+10 perf-stat.ps.dTLB-stores
40178575 ± 2% +43.2% 57520127 ± 2% perf-stat.ps.iTLB-load-misses
41177027 +28.9% 53078387 perf-stat.ps.iTLB-loads
6.109e+10 +28.1% 7.829e+10 perf-stat.ps.instructions
675658 ± 22% +161.4% 1765986 ± 15% perf-stat.ps.node-loads
3.679e+13 +28.1% 4.713e+13 perf-stat.total.instructions
1.00 ± 70% +12880.0% 129.80 ±117% interrupts.65:PCI-MSI.31981598-edge.i40e-eth0-TxRx-29
57.25 ±124% -91.3% 5.00 ±190% interrupts.90:PCI-MSI.31981623-edge.i40e-eth0-TxRx-54
7755 -12.9% 6754 ± 6% interrupts.CPU0.NMI:Non-maskable_interrupts
7755 -12.9% 6754 ± 6% interrupts.CPU0.PMI:Performance_monitoring_interrupts
61810 ± 3% +33.3% 82407 ± 18% interrupts.CPU10.RES:Rescheduling_interrupts
56578 ± 4% +54.8% 87606 ± 10% interrupts.CPU12.RES:Rescheduling_interrupts
62386 ± 10% +26.5% 78916 ± 11% interrupts.CPU15.RES:Rescheduling_interrupts
65540 ± 6% +26.3% 82769 ± 7% interrupts.CPU19.RES:Rescheduling_interrupts
66735 ± 5% +24.4% 83044 ± 6% interrupts.CPU21.RES:Rescheduling_interrupts
58038 ± 4% +44.3% 83734 ± 9% interrupts.CPU22.RES:Rescheduling_interrupts
66281 ± 6% +31.2% 86973 ± 13% interrupts.CPU23.RES:Rescheduling_interrupts
69758 ± 15% +26.2% 88045 ± 7% interrupts.CPU24.RES:Rescheduling_interrupts
64356 ± 4% +20.5% 77519 ± 9% interrupts.CPU25.RES:Rescheduling_interrupts
67092 ± 7% +30.4% 87484 ± 11% interrupts.CPU27.RES:Rescheduling_interrupts
64721 ± 8% +31.0% 84808 ± 15% interrupts.CPU28.RES:Rescheduling_interrupts
67218 ± 4% +20.2% 80771 ± 7% interrupts.CPU30.RES:Rescheduling_interrupts
8009 -23.3% 6139 ± 20% interrupts.CPU33.NMI:Non-maskable_interrupts
8009 -23.3% 6139 ± 20% interrupts.CPU33.PMI:Performance_monitoring_interrupts
68432 ± 6% +27.8% 87485 ± 6% interrupts.CPU33.RES:Rescheduling_interrupts
62253 ± 6% +47.1% 91571 ± 9% interrupts.CPU34.RES:Rescheduling_interrupts
7961 -23.3% 6110 ± 20% interrupts.CPU35.NMI:Non-maskable_interrupts
7961 -23.3% 6110 ± 20% interrupts.CPU35.PMI:Performance_monitoring_interrupts
7775 -22.0% 6065 ± 20% interrupts.CPU36.NMI:Non-maskable_interrupts
7775 -22.0% 6065 ± 20% interrupts.CPU36.PMI:Performance_monitoring_interrupts
7791 -22.0% 6074 ± 20% interrupts.CPU37.NMI:Non-maskable_interrupts
7791 -22.0% 6074 ± 20% interrupts.CPU37.PMI:Performance_monitoring_interrupts
7750 -22.1% 6034 ± 20% interrupts.CPU38.NMI:Non-maskable_interrupts
7750 -22.1% 6034 ± 20% interrupts.CPU38.PMI:Performance_monitoring_interrupts
60522 ± 9% +40.2% 84834 ± 5% interrupts.CPU38.RES:Rescheduling_interrupts
7749 -22.1% 6035 ± 20% interrupts.CPU39.NMI:Non-maskable_interrupts
7749 -22.1% 6035 ± 20% interrupts.CPU39.PMI:Performance_monitoring_interrupts
62257 ± 4% +25.4% 78063 ± 13% interrupts.CPU39.RES:Rescheduling_interrupts
62146 ± 16% +32.0% 82038 ± 11% interrupts.CPU41.RES:Rescheduling_interrupts
7783 -22.3% 6050 ± 20% interrupts.CPU42.NMI:Non-maskable_interrupts
7783 -22.3% 6050 ± 20% interrupts.CPU42.PMI:Performance_monitoring_interrupts
7793 -22.3% 6058 ± 20% interrupts.CPU43.NMI:Non-maskable_interrupts
7793 -22.3% 6058 ± 20% interrupts.CPU43.PMI:Performance_monitoring_interrupts
58804 ± 18% +42.2% 83622 ± 10% interrupts.CPU43.RES:Rescheduling_interrupts
7721 -12.1% 6785 ± 6% interrupts.CPU44.NMI:Non-maskable_interrupts
7721 -12.1% 6785 ± 6% interrupts.CPU44.PMI:Performance_monitoring_interrupts
7774 -22.5% 6026 ± 20% interrupts.CPU45.NMI:Non-maskable_interrupts
7774 -22.5% 6026 ± 20% interrupts.CPU45.PMI:Performance_monitoring_interrupts
7771 -22.1% 6051 ± 20% interrupts.CPU46.NMI:Non-maskable_interrupts
7771 -22.1% 6051 ± 20% interrupts.CPU46.PMI:Performance_monitoring_interrupts
59920 ± 12% +35.0% 80893 ± 15% interrupts.CPU46.RES:Rescheduling_interrupts
7785 -22.4% 6040 ± 20% interrupts.CPU47.NMI:Non-maskable_interrupts
7785 -22.4% 6040 ± 20% interrupts.CPU47.PMI:Performance_monitoring_interrupts
62454 ± 14% +26.8% 79204 ± 11% interrupts.CPU47.RES:Rescheduling_interrupts
7752 -12.5% 6783 ± 6% interrupts.CPU48.NMI:Non-maskable_interrupts
7752 -12.5% 6783 ± 6% interrupts.CPU48.PMI:Performance_monitoring_interrupts
7789 -22.4% 6045 ± 20% interrupts.CPU49.NMI:Non-maskable_interrupts
7789 -22.4% 6045 ± 20% interrupts.CPU49.PMI:Performance_monitoring_interrupts
56431 ± 14% +45.7% 82247 ± 17% interrupts.CPU49.RES:Rescheduling_interrupts
7750 -22.2% 6027 ± 20% interrupts.CPU50.NMI:Non-maskable_interrupts
7750 -22.2% 6027 ± 20% interrupts.CPU50.PMI:Performance_monitoring_interrupts
61121 ± 5% +31.5% 80349 ± 19% interrupts.CPU50.RES:Rescheduling_interrupts
7812 -22.4% 6058 ± 20% interrupts.CPU51.NMI:Non-maskable_interrupts
7812 -22.4% 6058 ± 20% interrupts.CPU51.PMI:Performance_monitoring_interrupts
61379 ± 10% +47.1% 90306 ± 6% interrupts.CPU51.RES:Rescheduling_interrupts
56451 ± 8% +23.5% 69695 ± 3% interrupts.CPU53.RES:Rescheduling_interrupts
62541 ± 6% +46.5% 91595 ± 10% interrupts.CPU54.RES:Rescheduling_interrupts
60908 ± 9% +55.6% 94803 ± 16% interrupts.CPU56.RES:Rescheduling_interrupts
65323 ± 12% +27.5% 83269 ± 5% interrupts.CPU57.RES:Rescheduling_interrupts
60774 ± 10% +33.3% 80996 ± 8% interrupts.CPU6.RES:Rescheduling_interrupts
62070 ± 8% +26.8% 78725 ± 2% interrupts.CPU60.RES:Rescheduling_interrupts
65281 ± 3% +30.3% 85086 ± 13% interrupts.CPU61.RES:Rescheduling_interrupts
61530 ± 9% +36.1% 83722 ± 11% interrupts.CPU62.RES:Rescheduling_interrupts
59129 ± 5% +37.6% 81379 ± 18% interrupts.CPU67.RES:Rescheduling_interrupts
67413 ± 8% +16.6% 78602 ± 10% interrupts.CPU69.RES:Rescheduling_interrupts
67134 ± 11% +20.2% 80710 ± 13% interrupts.CPU70.RES:Rescheduling_interrupts
57213 ± 8% +50.6% 86168 ± 11% interrupts.CPU71.RES:Rescheduling_interrupts
4630804 ± 3% +26.4% 5851310 ± 2% interrupts.RES:Rescheduling_interrupts
17331944 +37.9% 23900673 softirqs.CPU0.NET_RX
17530939 +36.3% 23889797 softirqs.CPU1.NET_RX
17136982 ± 2% +39.0% 23820881 ± 2% softirqs.CPU10.NET_RX
58917 +34.3% 79149 ± 24% softirqs.CPU10.RCU
17571701 ± 2% +37.4% 24147419 softirqs.CPU11.NET_RX
17672278 +31.9% 23306041 ± 4% softirqs.CPU12.NET_RX
17325581 +38.4% 23975105 ± 2% softirqs.CPU13.NET_RX
17294123 ± 2% +38.9% 24026366 ± 2% softirqs.CPU14.NET_RX
17581330 +32.6% 23306553 ± 4% softirqs.CPU15.NET_RX
17744380 +35.3% 24005037 ± 2% softirqs.CPU16.NET_RX
17366681 +36.0% 23625553 ± 3% softirqs.CPU17.NET_RX
17513047 +34.6% 23580015 ± 2% softirqs.CPU18.NET_RX
63886 +30.7% 83523 ± 22% softirqs.CPU18.RCU
17950167 +36.1% 24429312 ± 3% softirqs.CPU19.NET_RX
17280755 ± 2% +38.0% 23846236 ± 3% softirqs.CPU2.NET_RX
18132078 +33.8% 24265893 softirqs.CPU20.NET_RX
18027898 +36.2% 24548533 softirqs.CPU21.NET_RX
17696115 +38.9% 24572590 ± 2% softirqs.CPU22.NET_RX
17847777 +34.1% 23940312 ± 2% softirqs.CPU23.NET_RX
18099147 +37.0% 24789584 softirqs.CPU24.NET_RX
18223713 +36.3% 24845983 softirqs.CPU25.NET_RX
17512311 ± 2% +37.4% 24069333 ± 3% softirqs.CPU26.NET_RX
17841600 +33.6% 23839917 ± 2% softirqs.CPU27.NET_RX
17601089 ± 3% +35.4% 23828320 ± 4% softirqs.CPU28.NET_RX
18169931 +36.7% 24835918 softirqs.CPU29.NET_RX
17445850 ± 2% +36.1% 23739675 ± 4% softirqs.CPU3.NET_RX
17722005 ± 2% +29.8% 23008127 softirqs.CPU30.NET_RX
17663859 ± 2% +38.4% 24447008 softirqs.CPU31.NET_RX
17928131 +34.5% 24108420 ± 2% softirqs.CPU32.NET_RX
62457 +29.9% 81131 ± 22% softirqs.CPU32.RCU
18220692 +34.7% 24536407 ± 2% softirqs.CPU33.NET_RX
18286650 +34.3% 24565350 softirqs.CPU34.NET_RX
17304413 ± 2% +40.8% 24363651 softirqs.CPU35.NET_RX
17173191 +33.4% 22916547 ± 4% softirqs.CPU36.NET_RX
17563105 +34.5% 23628157 ± 3% softirqs.CPU37.NET_RX
17435903 ± 2% +35.1% 23564057 ± 3% softirqs.CPU38.NET_RX
62230 ± 4% +34.3% 83598 ± 24% softirqs.CPU38.RCU
17272182 ± 2% +35.1% 23336729 ± 5% softirqs.CPU39.NET_RX
17335637 +34.0% 23236226 ± 3% softirqs.CPU4.NET_RX
17164703 ± 2% +36.6% 23445940 ± 2% softirqs.CPU40.NET_RX
62436 ± 2% +34.6% 84012 ± 23% softirqs.CPU40.RCU
17030886 ± 2% +31.7% 22422929 ± 4% softirqs.CPU41.NET_RX
17349878 ± 2% +39.2% 24157883 softirqs.CPU42.NET_RX
61585 ± 3% +37.2% 84468 ± 23% softirqs.CPU42.RCU
17775923 +36.6% 24279566 ± 2% softirqs.CPU43.NET_RX
62964 ± 2% +35.3% 85174 ± 23% softirqs.CPU43.RCU
17268058 ± 2% +36.6% 23592781 ± 2% softirqs.CPU44.NET_RX
63331 ± 2% +33.2% 84384 ± 23% softirqs.CPU44.RCU
17148500 ± 2% +38.8% 23796364 ± 2% softirqs.CPU45.NET_RX
17615847 +35.1% 23796420 softirqs.CPU46.NET_RX
63792 ± 2% +34.6% 85859 ± 23% softirqs.CPU46.RCU
17153040 ± 2% +38.6% 23779686 ± 3% softirqs.CPU47.NET_RX
17324943 ± 2% +38.8% 24050281 ± 2% softirqs.CPU48.NET_RX
63757 +34.1% 85479 ± 23% softirqs.CPU48.RCU
17102014 +32.3% 22627699 ± 3% softirqs.CPU49.NET_RX
64602 ± 2% +30.1% 84027 ± 22% softirqs.CPU49.RCU
17338693 +36.7% 23705985 softirqs.CPU5.NET_RX
59132 +33.0% 78620 ± 24% softirqs.CPU5.RCU
17425587 ± 2% +32.7% 23117622 ± 4% softirqs.CPU50.NET_RX
17618503 +36.0% 23964794 ± 3% softirqs.CPU51.NET_RX
17354234 +37.8% 23913189 ± 4% softirqs.CPU52.NET_RX
63888 +33.3% 85175 ± 23% softirqs.CPU52.RCU
17115148 ± 2% +36.2% 23311982 ± 4% softirqs.CPU53.NET_RX
17735102 +35.2% 23986709 softirqs.CPU54.NET_RX
18057531 +34.2% 24239055 ± 2% softirqs.CPU55.NET_RX
17551567 +39.3% 24454599 ± 4% softirqs.CPU56.NET_RX
18150174 +34.1% 24345105 softirqs.CPU57.NET_RX
17949707 +36.3% 24462203 ± 2% softirqs.CPU58.NET_RX
17612693 ± 2% +40.8% 24794166 softirqs.CPU59.NET_RX
17495928 ± 2% +36.3% 23839559 ± 2% softirqs.CPU6.NET_RX
58659 +33.9% 78562 ± 24% softirqs.CPU6.RCU
18113824 +37.0% 24822642 softirqs.CPU60.NET_RX
18138838 +35.8% 24629071 ± 4% softirqs.CPU61.NET_RX
17915148 +35.7% 24309691 ± 3% softirqs.CPU62.NET_RX
17681225 +39.1% 24596506 softirqs.CPU63.NET_RX
17853174 +36.3% 24329595 ± 4% softirqs.CPU64.NET_RX
17861934 ± 2% +35.7% 24231143 ± 2% softirqs.CPU65.NET_RX
17843127 ± 3% +39.6% 24906670 softirqs.CPU66.NET_RX
17240363 ± 2% +38.3% 23839354 softirqs.CPU67.NET_RX
17773093 +37.0% 24341283 ± 2% softirqs.CPU68.NET_RX
18018477 ± 2% +37.9% 24840396 softirqs.CPU69.NET_RX
17692495 +36.6% 24168650 ± 2% softirqs.CPU7.NET_RX
17975980 ± 2% +35.3% 24319019 ± 2% softirqs.CPU70.NET_RX
17796849 +31.9% 23479744 ± 3% softirqs.CPU71.NET_RX
17621674 +34.0% 23613890 ± 2% softirqs.CPU8.NET_RX
17292292 ± 2% +32.6% 22931589 ± 3% softirqs.CPU9.NET_RX
1.269e+09 +36.0% 1.726e+09 softirqs.NET_RX
61.47 -6.8 54.67 perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64
61.67 -6.7 54.96 perf-profile.calltrace.cycles-pp.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
48.45 -5.5 42.98 perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto
8.37 ± 14% -4.4 4.00 ± 6% perf-profile.calltrace.cycles-pp.sk_page_frag_refill.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
8.34 ± 14% -4.4 3.98 ± 6% perf-profile.calltrace.cycles-pp.skb_page_frag_refill.sk_page_frag_refill.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
8.11 ± 15% -4.3 3.82 ± 7% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.skb_page_frag_refill.sk_page_frag_refill.tcp_sendmsg_locked.tcp_sendmsg
7.84 ± 16% -4.2 3.67 ± 7% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.skb_page_frag_refill.sk_page_frag_refill.tcp_sendmsg_locked
6.56 ± 18% -4.0 2.55 ± 10% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.get_page_from_freelist.__alloc_pages_nodemask.skb_page_frag_refill.sk_page_frag_refill
6.37 ± 19% -4.0 2.38 ± 10% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.get_page_from_freelist.__alloc_pages_nodemask.skb_page_frag_refill
22.37 ± 2% -3.7 18.63 ± 3% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames
28.02 ± 3% -3.5 24.55 ± 3% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit
26.62 ± 3% -3.5 23.17 ± 3% perf-profile.calltrace.cycles-pp.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv
26.79 ± 3% -3.4 23.35 ± 3% perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core
26.82 ± 3% -3.4 23.39 ± 3% perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog
26.92 ± 3% -3.4 23.49 ± 3% perf-profile.calltrace.cycles-pp.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog.net_rx_action
27.21 ± 2% -3.4 23.83 ± 3% perf-profile.calltrace.cycles-pp.ip_rcv.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start
27.46 ± 2% -3.2 24.24 ± 3% perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start.do_softirq_own_stack
27.77 ± 2% -3.2 24.61 ± 3% perf-profile.calltrace.cycles-pp.process_backlog.net_rx_action.__softirqentry_text_start.do_softirq_own_stack.do_softirq
27.97 ± 2% -3.1 24.87 ± 3% perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.do_softirq_own_stack.do_softirq.__local_bh_enable_ip
28.20 ± 2% -3.1 25.13 ± 3% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2
28.28 ± 2% -3.1 25.21 ± 3% perf-profile.calltrace.cycles-pp.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output
28.45 ± 2% -3.0 25.43 ± 3% perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit
28.49 ± 2% -3.0 25.48 ± 3% perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb
6.62 ± 7% -2.9 3.67 ± 2% perf-profile.calltrace.cycles-pp.sk_stream_alloc_skb.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
9.31 -2.9 6.38 ± 3% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established
9.47 -2.9 6.56 ± 3% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv
9.68 -2.9 6.80 ± 2% perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv.__release_sock
9.68 -2.9 6.81 ± 2% perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock
4.28 ± 17% -2.9 1.41 ± 16% perf-profile.calltrace.cycles-pp.___pskb_trim.sk_stream_alloc_skb.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
4.07 ± 18% -2.8 1.25 ± 17% perf-profile.calltrace.cycles-pp.__free_pages_ok.___pskb_trim.sk_stream_alloc_skb.tcp_sendmsg_locked.tcp_sendmsg
3.90 ± 18% -2.8 1.14 ± 18% perf-profile.calltrace.cycles-pp.free_one_page.__free_pages_ok.___pskb_trim.sk_stream_alloc_skb.tcp_sendmsg_locked
3.66 ± 20% -2.7 0.99 ± 19% perf-profile.calltrace.cycles-pp._raw_spin_lock.free_one_page.__free_pages_ok.___pskb_trim.sk_stream_alloc_skb
3.53 ± 20% -2.6 0.91 ± 20% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_one_page.__free_pages_ok.___pskb_trim
12.40 -1.4 10.96 perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto
12.55 -1.4 11.18 ± 2% perf-profile.calltrace.cycles-pp.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto
2.19 ± 38% -1.2 0.97 ± 25% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_one_page.__free_pages_ok.skb_release_data
2.28 ± 37% -1.0 1.25 ± 28% perf-profile.calltrace.cycles-pp._raw_spin_lock.free_one_page.__free_pages_ok.skb_release_data.__kfree_skb
6.30 ± 61% -1.0 5.34 ±181% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.30 ± 61% -1.0 5.35 ±181% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
4.00 ± 61% -0.9 3.12 ±183% perf-profile.calltrace.cycles-pp.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.00 ± 61% -0.9 3.12 ±183% perf-profile.calltrace.cycles-pp.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.95 ± 10% -0.8 4.14 ± 6% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg
4.68 ± 13% -0.7 4.00 ± 6% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg
0.26 ±100% +0.4 0.69 ± 10% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.send.send_omni_inner.send_tcp_stream.main
0.00 +0.7 0.68 ± 14% perf-profile.calltrace.cycles-pp.__free_pages_ok.skb_release_data.__kfree_skb.tcp_recvmsg.inet_recvmsg
2.54 ± 4% +0.7 3.25 ± 3% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg
3.33 ± 4% +0.8 4.12 ± 3% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
2.95 ± 4% +0.8 3.74 ± 3% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg.inet_recvmsg
0.00 +0.8 0.83 ± 14% perf-profile.calltrace.cycles-pp.skb_release_data.__kfree_skb.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
3.62 ± 4% +0.8 4.47 ± 3% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
0.00 +1.0 1.05 ± 18% perf-profile.calltrace.cycles-pp.free_one_page.__free_pages_ok.skb_release_data.__kfree_skb.tcp_clean_rtx_queue
0.00 +1.1 1.13 ± 18% perf-profile.calltrace.cycles-pp.__free_pages_ok.skb_release_data.__kfree_skb.tcp_clean_rtx_queue.tcp_ack
0.00 +1.2 1.16 ± 14% perf-profile.calltrace.cycles-pp.__kfree_skb.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
0.00 +1.4 1.38 ± 16% perf-profile.calltrace.cycles-pp.skb_release_data.__kfree_skb.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established
9.54 +1.4 10.94 perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg
10.05 +1.4 11.47 perf-profile.calltrace.cycles-pp._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
9.70 +1.4 11.13 perf-profile.calltrace.cycles-pp.copyin._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
0.00 +1.4 1.45 ± 16% perf-profile.calltrace.cycles-pp.__kfree_skb.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv
0.00 +1.8 1.78 ± 15% perf-profile.calltrace.cycles-pp.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock
0.14 ±173% +2.0 2.13 ± 8% perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock
5.82 +2.8 8.59 ± 2% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
5.92 +2.8 8.72 ± 2% perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg
6.04 +2.8 8.89 ± 2% perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg
6.59 +3.0 9.60 ± 2% perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
6.61 +3.0 9.62 ± 2% perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
26.70 ± 3% +4.9 31.60 perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64
26.74 ± 3% +4.9 31.65 perf-profile.calltrace.cycles-pp.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
12.98 ± 18% -8.1 4.85 ± 10% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
61.75 -6.8 54.98 perf-profile.children.cycles-pp.tcp_sendmsg
61.95 -6.7 55.27 perf-profile.children.cycles-pp.sock_sendmsg
62.43 -6.5 55.90 perf-profile.children.cycles-pp.__sys_sendto
62.53 -6.5 56.02 perf-profile.children.cycles-pp.__x64_sys_sendto
48.75 ± 2% -5.6 43.19 perf-profile.children.cycles-pp.tcp_sendmsg_locked
8.40 ± 15% -4.4 4.04 ± 7% perf-profile.children.cycles-pp.sk_page_frag_refill
8.37 ± 15% -4.4 4.01 ± 7% perf-profile.children.cycles-pp.skb_page_frag_refill
8.06 ± 15% -4.3 3.75 ± 7% perf-profile.children.cycles-pp.get_page_from_freelist
8.21 ± 15% -4.3 3.90 ± 7% perf-profile.children.cycles-pp.__alloc_pages_nodemask
7.43 ± 16% -4.1 3.29 ± 8% perf-profile.children.cycles-pp.__free_pages_ok
7.10 ± 17% -4.1 2.98 ± 9% perf-profile.children.cycles-pp.free_one_page
6.99 ± 17% -4.1 2.93 ± 9% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
7.08 ± 17% -4.0 3.08 ± 8% perf-profile.children.cycles-pp._raw_spin_lock
26.97 ± 3% -3.5 23.46 ± 3% perf-profile.children.cycles-pp.tcp_v4_rcv
27.13 ± 3% -3.5 23.64 ± 3% perf-profile.children.cycles-pp.ip_protocol_deliver_rcu
27.17 ± 3% -3.5 23.68 ± 3% perf-profile.children.cycles-pp.ip_local_deliver_finish
27.26 ± 3% -3.5 23.80 ± 3% perf-profile.children.cycles-pp.ip_local_deliver
27.56 ± 2% -3.4 24.15 ± 3% perf-profile.children.cycles-pp.ip_rcv
27.82 ± 2% -3.3 24.48 ± 3% perf-profile.children.cycles-pp.__netif_receive_skb_one_core
24.03 ± 2% -3.3 20.74 ± 2% perf-profile.children.cycles-pp.__tcp_push_pending_frames
28.14 ± 2% -3.3 24.86 ± 3% perf-profile.children.cycles-pp.process_backlog
28.36 ± 2% -3.2 25.12 ± 3% perf-profile.children.cycles-pp.net_rx_action
28.62 ± 2% -3.2 25.42 ± 3% perf-profile.children.cycles-pp.__softirqentry_text_start
28.67 ± 2% -3.2 25.47 ± 3% perf-profile.children.cycles-pp.do_softirq_own_stack
28.86 ± 2% -3.2 25.70 ± 3% perf-profile.children.cycles-pp.do_softirq
29.01 ± 2% -3.1 25.88 ± 3% perf-profile.children.cycles-pp.__local_bh_enable_ip
4.56 ± 13% -3.1 1.49 ± 10% perf-profile.children.cycles-pp.___pskb_trim
6.88 ± 7% -3.0 3.84 ± 2% perf-profile.children.cycles-pp.sk_stream_alloc_skb
30.92 ± 2% -2.9 28.03 ± 2% perf-profile.children.cycles-pp.ip_finish_output2
31.95 ± 2% -2.7 29.24 ± 2% perf-profile.children.cycles-pp.ip_output
30.90 ± 2% -2.7 28.21 ± 2% perf-profile.children.cycles-pp.tcp_write_xmit
32.87 ± 2% -2.7 30.21 ± 2% perf-profile.children.cycles-pp.__ip_queue_xmit
33.82 ± 2% -2.4 31.37 ± 2% perf-profile.children.cycles-pp.__tcp_transmit_skb
12.58 -1.3 11.24 perf-profile.children.cycles-pp.__release_sock
12.88 -1.2 11.63 perf-profile.children.cycles-pp.release_sock
3.44 ± 18% -1.0 2.43 ± 5% perf-profile.children.cycles-pp.skb_release_data
95.46 -0.9 94.54 perf-profile.children.cycles-pp.do_syscall_64
3.88 ± 16% -0.9 2.97 ± 5% perf-profile.children.cycles-pp.__kfree_skb
95.56 -0.9 94.68 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
0.21 ± 5% -0.1 0.15 ± 9% perf-profile.children.cycles-pp.prep_new_page
0.08 ± 8% -0.0 0.04 ± 50% perf-profile.children.cycles-pp.kfree_skbmem
0.09 ± 13% -0.0 0.05 ± 52% perf-profile.children.cycles-pp.alloc_pages_vma
0.09 ± 13% -0.0 0.05 ± 52% perf-profile.children.cycles-pp.shmem_alloc_page
0.15 ± 3% -0.0 0.12 ± 9% perf-profile.children.cycles-pp.kfree
0.08 ± 5% -0.0 0.06 ± 6% perf-profile.children.cycles-pp.sock_rfree
0.07 -0.0 0.05 ± 9% perf-profile.children.cycles-pp.tcp_cleanup_rbuf
0.23 ± 3% -0.0 0.21 ± 2% perf-profile.children.cycles-pp.__list_add_valid
0.05 +0.0 0.06 perf-profile.children.cycles-pp.bictcp_cong_avoid
0.05 +0.0 0.06 perf-profile.children.cycles-pp.__sock_wfree
0.09 ± 4% +0.0 0.10 ± 3% perf-profile.children.cycles-pp.sock_recvmsg
0.20 ± 2% +0.0 0.22 ± 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.08 +0.0 0.10 ± 8% perf-profile.children.cycles-pp.add_wait_queue
0.05 +0.0 0.07 ± 7% perf-profile.children.cycles-pp.sock_put
0.16 ± 2% +0.0 0.18 ± 4% perf-profile.children.cycles-pp.skb_entail
0.10 ± 7% +0.0 0.12 ± 6% perf-profile.children.cycles-pp.selinux_ipv4_output
0.06 ± 7% +0.0 0.08 ± 6% perf-profile.children.cycles-pp.tcp_stream_memory_free
0.05 ± 8% +0.0 0.07 ± 10% perf-profile.children.cycles-pp.rcu_all_qs
0.06 ± 9% +0.0 0.08 ± 6% perf-profile.children.cycles-pp.lock_timer_base
0.36 +0.0 0.38 ± 3% perf-profile.children.cycles-pp.__switch_to_asm
0.17 ± 2% +0.0 0.19 ± 5% perf-profile.children.cycles-pp.__calc_delta
0.09 ± 8% +0.0 0.12 ± 3% perf-profile.children.cycles-pp.kmem_cache_free
0.08 ± 5% +0.0 0.11 ± 6% perf-profile.children.cycles-pp.import_single_range
0.06 ± 6% +0.0 0.09 ± 8% perf-profile.children.cycles-pp.tcp_event_data_recv
0.09 ± 4% +0.0 0.11 ± 4% perf-profile.children.cycles-pp.tcp_v4_send_check
0.12 ± 11% +0.0 0.14 ± 3% perf-profile.children.cycles-pp.ksoftirqd_running
0.10 ± 5% +0.0 0.12 ± 3% perf-profile.children.cycles-pp.tcp_wfree
0.05 ± 9% +0.0 0.08 ± 4% perf-profile.children.cycles-pp.tcp_rearm_rto
0.09 ± 4% +0.0 0.12 perf-profile.children.cycles-pp.tcp_release_cb
0.08 +0.0 0.11 ± 10% perf-profile.children.cycles-pp.rb_erase_cached
0.12 ± 5% +0.0 0.15 ± 9% perf-profile.children.cycles-pp.find_next_bit
0.04 ± 57% +0.0 0.07 ± 12% perf-profile.children.cycles-pp.__tcp_select_window
0.12 +0.0 0.15 ± 5% perf-profile.children.cycles-pp.ip_rcv_core
0.12 ± 5% +0.0 0.15 ± 9% perf-profile.children.cycles-pp.validate_xmit_skb
0.06 ± 7% +0.0 0.09 ± 8% perf-profile.children.cycles-pp.tcp_queue_rcv
0.26 +0.0 0.30 ± 4% perf-profile.children.cycles-pp.enqueue_to_backlog
0.16 ± 4% +0.0 0.19 ± 4% perf-profile.children.cycles-pp.tcp_rcv_space_adjust
0.08 ± 6% +0.0 0.11 ± 6% perf-profile.children.cycles-pp.ip_send_check
0.22 +0.0 0.26 ± 2% perf-profile.children.cycles-pp.pick_next_entity
0.12 ± 8% +0.0 0.16 ± 4% perf-profile.children.cycles-pp.__skb_clone
0.18 ± 4% +0.0 0.22 ± 6% perf-profile.children.cycles-pp.ip_finish_output
0.33 ± 4% +0.0 0.37 ± 2% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.18 ± 2% +0.0 0.22 ± 3% perf-profile.children.cycles-pp.___might_sleep
0.31 ± 4% +0.0 0.35 ± 3% perf-profile.children.cycles-pp.___perf_sw_event
0.42 ± 4% +0.0 0.46 ± 3% perf-profile.children.cycles-pp.update_cfs_group
0.19 ± 2% +0.0 0.23 ± 3% perf-profile.children.cycles-pp.tcp_schedule_loss_probe
0.22 ± 4% +0.0 0.27 ± 5% perf-profile.children.cycles-pp.selinux_parse_skb
0.01 ±173% +0.0 0.06 perf-profile.children.cycles-pp.rb_next
0.00 +0.1 0.05 perf-profile.children.cycles-pp.ip_queue_xmit
0.23 +0.1 0.28 ± 4% perf-profile.children.cycles-pp.check_preempt_wakeup
0.00 +0.1 0.05 ± 7% perf-profile.children.cycles-pp.__tcp_ack_snd_check
0.00 +0.1 0.05 ± 7% perf-profile.children.cycles-pp.__put_compound_page
0.29 +0.1 0.34 ± 5% perf-profile.children.cycles-pp.set_next_entity
0.33 ± 2% +0.1 0.38 ± 3% perf-profile.children.cycles-pp.native_write_msr
0.23 +0.1 0.28 ± 3% perf-profile.children.cycles-pp.__netif_receive_skb_core
0.26 ± 2% +0.1 0.32 ± 4% perf-profile.children.cycles-pp.check_preempt_curr
0.18 +0.1 0.24 ± 2% perf-profile.children.cycles-pp.sock_has_perm
0.00 +0.1 0.06 ± 8% perf-profile.children.cycles-pp.sk_stream_write_space
0.49 +0.1 0.54 ± 3% perf-profile.children.cycles-pp.switch_fpu_return
0.00 +0.1 0.06 ± 6% perf-profile.children.cycles-pp.tcp_data_queue
0.34 ± 2% +0.1 0.40 ± 5% perf-profile.children.cycles-pp.__tcp_send_ack
0.30 ± 2% +0.1 0.36 ± 3% perf-profile.children.cycles-pp.tcp_mstamp_refresh
0.47 +0.1 0.54 ± 3% perf-profile.children.cycles-pp.selinux_ip_postroute_compat
0.00 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.tcp_v4_fill_cb
0.67 +0.1 0.73 ± 3% perf-profile.children.cycles-pp.load_new_mm_cr3
0.31 ± 2% +0.1 0.38 ± 2% perf-profile.children.cycles-pp.__virt_addr_valid
0.14 +0.1 0.21 ± 7% perf-profile.children.cycles-pp.security_socket_sendmsg
0.28 +0.1 0.35 ± 3% perf-profile.children.cycles-pp.sk_reset_timer
0.56 ± 2% +0.1 0.63 ± 3% perf-profile.children.cycles-pp.__switch_to
0.30 ± 3% +0.1 0.38 ± 3% perf-profile.children.cycles-pp.__ip_local_out
0.47 ± 3% +0.1 0.55 ± 3% perf-profile.children.cycles-pp.netif_rx
0.46 ± 4% +0.1 0.54 ± 3% perf-profile.children.cycles-pp.netif_rx_internal
0.33 ± 3% +0.1 0.41 ± 4% perf-profile.children.cycles-pp.ip_local_out
0.25 +0.1 0.33 ± 3% perf-profile.children.cycles-pp.mod_timer
0.50 ± 2% +0.1 0.59 perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.43 +0.1 0.52 ± 2% perf-profile.children.cycles-pp.selinux_sock_rcv_skb_compat
0.40 +0.1 0.49 ± 2% perf-profile.children.cycles-pp.__fget_light
0.31 +0.1 0.40 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_bh
0.58 +0.1 0.68 ± 3% perf-profile.children.cycles-pp.selinux_ip_postroute
0.28 ± 14% +0.1 0.38 ± 10% perf-profile.children.cycles-pp.skb_release_all
0.00 +0.1 0.10 ± 3% perf-profile.children.cycles-pp.tcp_check_space
0.36 +0.1 0.47 perf-profile.children.cycles-pp.simple_copy_to_iter
0.27 ± 14% +0.1 0.37 ± 10% perf-profile.children.cycles-pp.skb_release_head_state
0.35 +0.1 0.45 ± 2% perf-profile.children.cycles-pp.__inet_lookup_established
0.37 +0.1 0.48 ± 3% perf-profile.children.cycles-pp.lock_sock_nested
0.43 ± 3% +0.1 0.54 ± 3% perf-profile.children.cycles-pp.read_tsc
0.44 ± 10% +0.1 0.55 ± 8% perf-profile.children.cycles-pp.tcp_send_mss
0.24 ± 3% +0.1 0.36 ± 2% perf-profile.children.cycles-pp.tcp_event_new_data_sent
0.45 ± 10% +0.1 0.56 ± 8% perf-profile.children.cycles-pp.tcp_current_mss
0.46 +0.1 0.58 ± 2% perf-profile.children.cycles-pp.sockfd_lookup_light
1.33 +0.1 1.44 ± 2% perf-profile.children.cycles-pp.pick_next_task_fair
0.54 ± 2% +0.1 0.66 ± 2% perf-profile.children.cycles-pp.__check_object_size
0.54 ± 4% +0.1 0.67 ± 3% perf-profile.children.cycles-pp.ktime_get
0.65 +0.1 0.78 ± 2% perf-profile.children.cycles-pp.selinux_socket_sock_rcv_skb
0.85 +0.1 1.00 ± 3% perf-profile.children.cycles-pp.nf_hook_slow
0.71 +0.1 0.86 perf-profile.children.cycles-pp.security_sock_rcv_skb
0.77 +0.1 0.92 perf-profile.children.cycles-pp.sk_filter_trim_cap
1.23 +0.2 1.38 ± 3% perf-profile.children.cycles-pp.switch_mm_irqs_off
0.98 ± 2% +0.2 1.16 ± 3% perf-profile.children.cycles-pp.loopback_xmit
0.00 +0.2 0.20 ± 6% perf-profile.children.cycles-pp.__sk_mem_raise_allocated
1.05 +0.2 1.25 ± 3% perf-profile.children.cycles-pp.dev_hard_start_xmit
0.00 +0.2 0.20 ± 7% perf-profile.children.cycles-pp.__sk_mem_schedule
1.63 +0.2 1.87 ± 3% perf-profile.children.cycles-pp.__dev_queue_xmit
0.91 +0.3 1.16 ± 2% perf-profile.children.cycles-pp.entry_SYSCALL_64
0.79 +0.3 1.06 ± 3% perf-profile.children.cycles-pp.syscall_return_via_sysret
10.09 +1.5 11.56 ± 2% perf-profile.children.cycles-pp._copy_from_iter_full
9.86 +1.5 11.35 perf-profile.children.cycles-pp.copyin
0.47 ± 3% +1.6 2.09 ± 8% perf-profile.children.cycles-pp.tcp_clean_rtx_queue
0.81 ± 2% +1.7 2.49 ± 7% perf-profile.children.cycles-pp.tcp_ack
6.04 ± 2% +2.8 8.79 ± 2% perf-profile.children.cycles-pp.copyout
6.16 +2.8 8.96 ± 2% perf-profile.children.cycles-pp._copy_to_iter
6.73 +3.0 9.68 ± 2% perf-profile.children.cycles-pp.skb_copy_datagram_iter
6.71 +3.0 9.67 ± 2% perf-profile.children.cycles-pp.__skb_datagram_iter
15.78 +4.2 20.00 ± 2% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
26.84 ± 3% +4.9 31.76 perf-profile.children.cycles-pp.tcp_recvmsg
26.87 ± 3% +4.9 31.79 perf-profile.children.cycles-pp.inet_recvmsg
27.28 ± 3% +5.0 32.28 perf-profile.children.cycles-pp.__sys_recvfrom
27.35 ± 3% +5.0 32.36 perf-profile.children.cycles-pp.__x64_sys_recvfrom
12.97 ± 18% -8.1 4.85 ± 10% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
1.16 ± 2% -0.1 1.07 perf-profile.self.cycles-pp.tcp_sendmsg_locked
0.20 ± 5% -0.1 0.12 ± 6% perf-profile.self.cycles-pp.skb_page_frag_refill
0.18 ± 2% -0.1 0.12 ± 8% perf-profile.self.cycles-pp.___pskb_trim
0.20 ± 4% -0.0 0.15 ± 7% perf-profile.self.cycles-pp.prep_new_page
0.08 ± 5% -0.0 0.03 ± 81% perf-profile.self.cycles-pp.kfree_skbmem
0.30 ± 2% -0.0 0.26 ± 9% perf-profile.self.cycles-pp.update_rq_clock
0.15 ± 7% -0.0 0.11 ± 11% perf-profile.self.cycles-pp.__sk_dst_check
0.55 ± 2% -0.0 0.52 perf-profile.self.cycles-pp.get_page_from_freelist
0.15 ± 2% -0.0 0.12 ± 9% perf-profile.self.cycles-pp.kfree
0.44 -0.0 0.41 ± 3% perf-profile.self.cycles-pp.__list_del_entry_valid
0.23 ± 3% -0.0 0.20 ± 4% perf-profile.self.cycles-pp.free_one_page
0.22 ± 3% -0.0 0.20 ± 5% perf-profile.self.cycles-pp.enqueue_entity
0.08 ± 5% -0.0 0.06 ± 6% perf-profile.self.cycles-pp.sock_rfree
0.07 ± 6% -0.0 0.05 perf-profile.self.cycles-pp.tcp_cleanup_rbuf
0.23 ± 2% -0.0 0.21 ± 3% perf-profile.self.cycles-pp.__list_add_valid
0.05 +0.0 0.06 perf-profile.self.cycles-pp.bictcp_cong_avoid
0.06 +0.0 0.07 ± 6% perf-profile.self.cycles-pp.woken_wake_function
0.06 ± 7% +0.0 0.07 ± 5% perf-profile.self.cycles-pp.tcp_queue_rcv
0.10 ± 5% +0.0 0.11 perf-profile.self.cycles-pp.ip_local_deliver
0.05 ± 8% +0.0 0.07 ± 5% perf-profile.self.cycles-pp.tcp_stream_memory_free
0.05 +0.0 0.07 ± 7% perf-profile.self.cycles-pp.sock_put
0.06 +0.0 0.08 ± 6% perf-profile.self.cycles-pp.selinux_ipv4_postroute
0.07 ± 7% +0.0 0.08 ± 4% perf-profile.self.cycles-pp.tcp_event_new_data_sent
0.07 ± 6% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.release_sock
0.08 ± 5% +0.0 0.10 ± 8% perf-profile.self.cycles-pp.tcp_sendmsg
0.09 ± 5% +0.0 0.10 ± 4% perf-profile.self.cycles-pp.tcp_v4_send_check
0.06 ± 6% +0.0 0.08 ± 9% perf-profile.self.cycles-pp.import_single_range
0.05 ± 8% +0.0 0.07 ± 5% perf-profile.self.cycles-pp.sockfd_lookup_light
0.15 ± 5% +0.0 0.17 ± 2% perf-profile.self.cycles-pp.dequeue_task_fair
0.09 +0.0 0.11 ± 5% perf-profile.self.cycles-pp.sk_stream_alloc_skb
0.06 ± 11% +0.0 0.08 perf-profile.self.cycles-pp.security_sock_rcv_skb
0.09 ± 5% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.ip_protocol_deliver_rcu
0.10 ± 5% +0.0 0.12 ± 6% perf-profile.self.cycles-pp.__sys_recvfrom
0.07 ± 6% +0.0 0.09 ± 22% perf-profile.self.cycles-pp.dev_hard_start_xmit
0.21 ± 3% +0.0 0.23 ± 5% perf-profile.self.cycles-pp.tcp_v4_rcv
0.18 ± 3% +0.0 0.20 ± 4% perf-profile.self.cycles-pp.select_task_rq_fair
0.17 ± 2% +0.0 0.19 ± 5% perf-profile.self.cycles-pp.__calc_delta
0.10 ± 4% +0.0 0.12 ± 6% perf-profile.self.cycles-pp.check_preempt_wakeup
0.09 ± 4% +0.0 0.11 ± 3% perf-profile.self.cycles-pp.tcp_wfree
0.10 ± 5% +0.0 0.12 ± 5% perf-profile.self.cycles-pp.__x64_sys_sendto
0.04 ± 58% +0.0 0.07 ± 7% perf-profile.self.cycles-pp.perf_swevent_get_recursion_context
0.09 ± 4% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.pick_next_entity
0.10 ± 4% +0.0 0.12 ± 3% perf-profile.self.cycles-pp.sock_def_readable
0.09 ± 9% +0.0 0.11 ± 4% perf-profile.self.cycles-pp._copy_to_iter
0.12 ± 9% +0.0 0.14 ± 2% perf-profile.self.cycles-pp.ksoftirqd_running
0.14 ± 3% +0.0 0.17 ± 5% perf-profile.self.cycles-pp.enqueue_to_backlog
0.06 +0.0 0.09 ± 8% perf-profile.self.cycles-pp.tcp_event_data_recv
0.14 ± 3% +0.0 0.16 ± 6% perf-profile.self.cycles-pp.__alloc_skb
0.09 ± 5% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.tcp_release_cb
0.11 ± 4% +0.0 0.14 ± 3% perf-profile.self.cycles-pp.ip_rcv_core
0.15 ± 2% +0.0 0.18 ± 2% perf-profile.self.cycles-pp.__softirqentry_text_start
0.10 ± 10% +0.0 0.13 ± 5% perf-profile.self.cycles-pp.tcp_v4_do_rcv
0.10 ± 4% +0.0 0.13 ± 3% perf-profile.self.cycles-pp.mod_timer
0.04 ± 57% +0.0 0.07 ± 17% perf-profile.self.cycles-pp.rb_erase_cached
0.10 ± 11% +0.0 0.13 ± 3% perf-profile.self.cycles-pp.__skb_clone
0.10 ± 4% +0.0 0.14 ± 7% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.11 ± 3% +0.0 0.14 ± 3% perf-profile.self.cycles-pp.selinux_ip_postroute
0.20 ± 2% +0.0 0.23 ± 2% perf-profile.self.cycles-pp.process_backlog
0.15 ± 3% +0.0 0.19 ± 5% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.12 ± 4% +0.0 0.16 ± 3% perf-profile.self.cycles-pp.try_to_wake_up
0.12 ± 4% +0.0 0.16 perf-profile.self.cycles-pp.selinux_sock_rcv_skb_compat
0.18 ± 6% +0.0 0.22 ± 2% perf-profile.self.cycles-pp.__ip_queue_xmit
0.16 ± 5% +0.0 0.20 ± 3% perf-profile.self.cycles-pp.ip_output
0.17 ± 2% +0.0 0.20 ± 3% perf-profile.self.cycles-pp.net_rx_action
0.18 ± 4% +0.0 0.21 ± 4% perf-profile.self.cycles-pp.selinux_ip_postroute_compat
0.07 ± 5% +0.0 0.11 ± 8% perf-profile.self.cycles-pp.ip_send_check
0.24 +0.0 0.28 ± 6% perf-profile.self.cycles-pp.loopback_xmit
0.28 ± 5% +0.0 0.31 ± 5% perf-profile.self.cycles-pp.___perf_sw_event
0.03 ±100% +0.0 0.06 ± 7% perf-profile.self.cycles-pp.rcu_all_qs
0.17 +0.0 0.21 ± 3% perf-profile.self.cycles-pp.___might_sleep
0.22 ± 3% +0.0 0.26 ± 4% perf-profile.self.cycles-pp.selinux_parse_skb
0.42 ± 5% +0.0 0.46 ± 4% perf-profile.self.cycles-pp.update_cfs_group
0.16 ± 6% +0.0 0.20 ± 5% perf-profile.self.cycles-pp.tcp_current_mss
0.21 ± 2% +0.0 0.26 ± 4% perf-profile.self.cycles-pp.selinux_socket_sock_rcv_skb
0.15 ± 3% +0.0 0.20 ± 6% perf-profile.self.cycles-pp.tcp_ack
0.14 ± 5% +0.0 0.18 ± 4% perf-profile.self.cycles-pp.__local_bh_enable_ip
0.21 ± 8% +0.0 0.25 ± 3% perf-profile.self.cycles-pp.ktime_get
0.15 ± 5% +0.0 0.20 perf-profile.self.cycles-pp.tcp_clean_rtx_queue
0.14 ± 5% +0.0 0.19 ± 3% perf-profile.self.cycles-pp.__skb_datagram_iter
0.18 ± 8% +0.0 0.23 ± 2% perf-profile.self.cycles-pp.__check_object_size
0.00 +0.1 0.05 ± 7% perf-profile.self.cycles-pp.netif_rx_internal
0.16 ± 2% +0.1 0.22 ± 2% perf-profile.self.cycles-pp.__sys_sendto
0.00 +0.1 0.05 ± 9% perf-profile.self.cycles-pp.rb_next
0.33 ± 2% +0.1 0.38 ± 4% perf-profile.self.cycles-pp.native_write_msr
0.18 ± 3% +0.1 0.24 ± 3% perf-profile.self.cycles-pp.tcp_rcv_established
0.49 +0.1 0.54 ± 3% perf-profile.self.cycles-pp.switch_fpu_return
0.22 +0.1 0.28 ± 2% perf-profile.self.cycles-pp.__netif_receive_skb_core
0.17 ± 2% +0.1 0.23 ± 2% perf-profile.self.cycles-pp.sock_has_perm
0.00 +0.1 0.06 ± 6% perf-profile.self.cycles-pp.tcp_v4_fill_cb
0.31 ± 4% +0.1 0.37 ± 2% perf-profile.self.cycles-pp.__virt_addr_valid
0.67 +0.1 0.73 ± 3% perf-profile.self.cycles-pp.load_new_mm_cr3
0.50 ± 2% +0.1 0.57 ± 3% perf-profile.self.cycles-pp.__switch_to
0.45 ± 3% +0.1 0.51 perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.26 +0.1 0.34 ± 2% perf-profile.self.cycles-pp.__inet_lookup_established
0.29 +0.1 0.38 ± 3% perf-profile.self.cycles-pp._raw_spin_lock_bh
0.56 ± 2% +0.1 0.65 ± 3% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.39 ± 2% +0.1 0.48 ± 2% perf-profile.self.cycles-pp.__fget_light
0.36 +0.1 0.46 ± 3% perf-profile.self.cycles-pp.tcp_recvmsg
0.32 ± 2% +0.1 0.42 ± 2% perf-profile.self.cycles-pp.skb_release_data
0.41 ± 2% +0.1 0.51 ± 3% perf-profile.self.cycles-pp.read_tsc
0.65 ± 2% +0.1 0.76 ± 4% perf-profile.self.cycles-pp.__schedule
0.33 ± 2% +0.1 0.44 ± 3% perf-profile.self.cycles-pp.tcp_write_xmit
0.41 +0.1 0.53 ± 4% perf-profile.self.cycles-pp.__tcp_transmit_skb
0.00 +0.2 0.19 ± 7% perf-profile.self.cycles-pp.__sk_mem_raise_allocated
0.81 ± 2% +0.2 1.03 perf-profile.self.cycles-pp.entry_SYSCALL_64
0.79 +0.3 1.06 ± 3% perf-profile.self.cycles-pp.syscall_return_via_sysret
1.71 +0.4 2.11 ± 3% perf-profile.self.cycles-pp.do_syscall_64
15.64 +4.2 19.83 ± 2% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
netperf.Throughput_Mbps
5000 +-+------------------------------------------------------------------+
4500 O-+ O O |
| O O |
4000 +-+ |
3500 +-+ |
|..+...+..+...+..+...+..+...+..+...+ +...+..+...+..+...+..+...+..|
3000 +-+ : : |
2500 +-+ : : |
2000 +-+ : : |
| : : |
1500 +-+ : : |
1000 +-+ : : |
| : : |
500 +-+ : |
0 +-+------------------------------------------------------------------+
netperf.Throughput_total_Mbps
700000 +-+----------------------------------------------------------------+
O O O O O |
600000 +-+ |
| |
500000 +-++...+..+..+...+..+.. ..+..+... +...+..+..+...+..+..+...+..|
| +. + : |
400000 +-+ : : |
| : : |
300000 +-+ : : |
| : : |
200000 +-+ : : |
| : : |
100000 +-+ : : |
| : |
0 +-+----------------------------------------------------------------+
netperf.workload
3.5e+09 +-+---------------------------------------------------------------+
| |
3e+09 O-+O O O O |
| |
2.5e+09 +-+ |
|..+...+..+..+...+..+..+..+...+..+ +..+..+...+..+..+..+...+..|
2e+09 +-+ : : |
| : : |
1.5e+09 +-+ : : |
| : : |
1e+09 +-+ : : |
| : : |
5e+08 +-+ : : |
| : |
0 +-+---------------------------------------------------------------+
netperf.time.user_time
1600 +-+-------O---O------------------------------------------------------+
O O O |
1400 +-+ |
1200 +-+ |
| |
1000 +-++...+..+... |
| +..+...+..+...+..+...+ +...+..+...+..+...+..+...+..|
800 +-+ : : |
| : : |
600 +-+ : : |
400 +-+ : : |
| : : |
200 +-+ : : |
| : |
0 +-+------------------------------------------------------------------+
netperf.time.voluntary_context_switches
50000 +-+----O------O-----------------------------------------------------+
45000 +-+ O |
O O |
40000 +-+ |
35000 +-+ |
| |
30000 +-+ |
25000 +-+ |
20000 +-+ + + |
| + : : : : |
15000 +-+ + : : : : : |
10000 +-+ + : : : : : |
| .+...+.. + : : : .+ : |
5000 +-+ + +..+ +...+..+.. ..+..+...+. +..+...+..|
0 +-+-----------------------------------------------------------------+
netperf.time.involuntary_context_switches
2.5e+09 +-+---------------------------------------------------------------+
| |
| |
2e+09 O-+O O O O |
| |
| |
1.5e+09 +-++...+..+..+...+..+..+..+...+..+ +..+..+...+..+..+..+...+..|
| : : |
1e+09 +-+ : : |
| : : |
| : : |
5e+08 +-+ : : |
| : : |
| :: |
0 +-+---------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-csl-2sp4: 96 threads Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory
=========================================================================================
cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase/ucode:
cs-localhost/gcc-7/performance/ipv4/x86_64-rhel-7.6/200%/debian-x86_64-2018-04-03.cgz/600s/lkp-csl-2sp4/TCP_STREAM/netperf/0x400001c
commit:
a8e11e5c56 ("sysctl: define proc_do_static_key()")
ede61ca474 ("tcp: add tcp_rx_skb_cache sysctl")
a8e11e5c5611a9f7 ede61ca474a0348b975d9824565
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:1 100% 1:1 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
1:1 -100% :1 dmesg.WARNING:at_ip__netif_receive_skb_core/0x
:1 100% 1:1 dmesg.WARNING:stack_recursion
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
View attachment "config-5.2.0-rc3-00259-gede61ca" of type "text/plain" (196402 bytes)
View attachment "job-script" of type "text/plain" (7730 bytes)
View attachment "job.yaml" of type "text/plain" (5002 bytes)
View attachment "reproduce" of type "text/plain" (8051 bytes)
Powered by blists - more mailing lists