| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20201004132716.GS393@shao2-debian>
Date: Sun, 4 Oct 2020 21:27:16 +0800
From: kernel test robot <rong.a.chen@...el.com>
To: Vincent Guittot <vincent.guittot@...aro.org>
Cc: Ingo Molnar <mingo@...nel.org>, Rik van Riel <riel@...riel.com>,
Ben Segall <bsegall@...gle.com>,
Dietmar Eggemann <dietmar.eggemann@....com>,
Juri Lelli <juri.lelli@...hat.com>,
Linus Torvalds <torvalds@...ux-foundation.org>,
Mel Gorman <mgorman@...e.de>, Mike Galbraith <efault@....de>,
Peter Zijlstra <peterz@...radead.org>,
Steven Rostedt <rostedt@...dmis.org>,
Thomas Gleixner <tglx@...utronix.de>,
LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org,
lkp@...el.com, ying.huang@...el.com, feng.tang@...el.com,
zhengjun.xing@...el.com, aubrey.li@...ux.intel.com,
yu.c.chen@...el.com
Subject: [sched/fair] fcf0553db6: netperf.Throughput_Mbps -30.8% regression
Greeting,
FYI, we noticed a -30.8% regression of netperf.Throughput_Mbps due to commit:
commit: fcf0553db6f4c79387864f6e4ab4a891601f395e ("sched/fair: Remove meaningless imbalance calculation")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: netperf
on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
with following parameters:
ip: ipv4
runtime: 900s
nr_threads: 50%
cluster: cs-localhost
test: TCP_STREAM
cpufreq_governor: performance
ucode: 0x5002f01
test-description: Netperf is a benchmark that can be use to measure various aspect of networking performance.
test-url: http://www.netperf.org/netperf/
In addition to that, the commit also has significant impact on the following tests:
+------------------+------------------------------------------------------------------------+
| testcase: change | vm-scalability: vm-scalability.throughput 1.5% improvement |
| test machine | 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory |
| test parameters | cpufreq_governor=performance |
| | runtime=300s |
| | size=8T |
| | test=anon-w-seq |
| | ucode=0x16 |
+------------------+------------------------------------------------------------------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen@...el.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase/ucode:
cs-localhost/gcc-9/performance/ipv4/x86_64-rhel-8.3/50%/debian-10.4-x86_64-20200603.cgz/900s/lkp-csl-2ap3/TCP_STREAM/netperf/0x5002f01
commit:
a349834703 ("sched/fair: Rename sg_lb_stats::sum_nr_running to sum_h_nr_running")
fcf0553db6 ("sched/fair: Remove meaningless imbalance calculation")
a349834703010183 fcf0553db6f4c79387864f6e4ab
---------------- ---------------------------
%stddev %change %stddev
\ | \
3242 ± 9% -30.8% 2243 ± 5% netperf.Throughput_Mbps
311318 ± 9% -30.8% 215389 ± 5% netperf.Throughput_total_Mbps
1481632 ± 44% +121.8% 3286883 ± 29% netperf.time.involuntary_context_switches
43693 ± 3% -23.1% 33615 ± 2% netperf.time.minor_page_faults
2.138e+09 ± 9% -30.8% 1.479e+09 ± 5% netperf.workload
1042132 ± 8% +13.6% 1183437 ± 8% meminfo.DirectMap4k
148685 ± 3% +8.0% 160562 ± 3% numa-meminfo.node3.Shmem
24.60 ± 4% -52.8% 11.60 ± 22% vmstat.cpu.id
74.40 +16.9% 87.00 ± 2% vmstat.cpu.sy
143.80 +17.0% 168.20 ± 2% vmstat.procs.r
1085464 ± 17% -75.7% 263351 ± 30% vmstat.system.cs
24.75 ± 3% -12.8 11.94 ± 21% mpstat.cpu.all.idle%
1.18 ± 5% +0.2 1.41 ± 3% mpstat.cpu.all.irq%
1.56 ± 8% -0.7 0.87 ± 12% mpstat.cpu.all.soft%
72.26 +13.3 85.59 ± 3% mpstat.cpu.all.sys%
0.26 ± 10% -0.1 0.19 ± 9% mpstat.cpu.all.usr%
3.401e+09 ± 44% -91.1% 3.012e+08 ± 32% cpuidle.C1.time
1.436e+08 ± 65% -89.6% 14954139 ± 34% cpuidle.C1.usage
3.536e+10 ± 5% -45.3% 1.932e+10 ± 24% cpuidle.C1E.time
4.126e+08 ± 7% -66.1% 1.399e+08 ± 31% cpuidle.C1E.usage
5.227e+08 ±138% -85.7% 74659149 ± 22% cpuidle.POLL.time
4914892 ± 99% -86.6% 657190 ± 25% cpuidle.POLL.usage
332.80 ± 20% -38.5% 204.80 ± 7% slabinfo.biovec-64.active_objs
332.80 ± 20% -38.5% 204.80 ± 7% slabinfo.biovec-64.num_objs
47399 ± 2% +12.9% 53505 ± 3% slabinfo.skbuff_fclone_cache.active_objs
743.80 ± 2% +12.8% 838.80 ± 3% slabinfo.skbuff_fclone_cache.active_slabs
47625 ± 2% +12.8% 53730 ± 3% slabinfo.skbuff_fclone_cache.num_objs
743.80 ± 2% +12.8% 838.80 ± 3% slabinfo.skbuff_fclone_cache.num_slabs
2.361e+08 ± 5% -20.7% 1.872e+08 ± 5% numa-numastat.node0.local_node
2.361e+08 ± 5% -20.7% 1.872e+08 ± 5% numa-numastat.node0.numa_hit
2.767e+08 ± 13% -34.5% 1.812e+08 ± 3% numa-numastat.node1.local_node
2.767e+08 ± 13% -34.5% 1.812e+08 ± 3% numa-numastat.node1.numa_hit
2.555e+08 ± 8% -29.6% 1.798e+08 ± 3% numa-numastat.node2.local_node
2.555e+08 ± 8% -29.6% 1.799e+08 ± 3% numa-numastat.node2.numa_hit
3.029e+08 ± 40% -35.6% 1.952e+08 ± 9% numa-numastat.node3.local_node
3.029e+08 ± 40% -35.6% 1.952e+08 ± 9% numa-numastat.node3.numa_hit
1.197e+08 ± 7% -19.7% 96054758 ± 5% numa-vmstat.node0.numa_hit
1.196e+08 ± 7% -19.8% 95968475 ± 5% numa-vmstat.node0.numa_local
1.418e+08 ± 17% -35.8% 90977174 ± 4% numa-vmstat.node1.numa_hit
1.418e+08 ± 17% -35.9% 90926215 ± 4% numa-vmstat.node1.numa_local
1.252e+08 ± 6% -28.4% 89645786 ± 5% numa-vmstat.node2.numa_hit
1.25e+08 ± 6% -28.4% 89551187 ± 5% numa-vmstat.node2.numa_local
1.487e+08 ± 38% -34.0% 98073664 ± 9% numa-vmstat.node3.numa_hit
1.486e+08 ± 38% -34.1% 97983383 ± 9% numa-vmstat.node3.numa_local
125155 +2.2% 127953 proc-vmstat.nr_active_anon
264519 +1.2% 267816 proc-vmstat.nr_file_pages
6252 ± 2% +6.3% 6647 ± 2% proc-vmstat.nr_inactive_anon
10301 ± 2% +2.3% 10542 proc-vmstat.nr_mapped
43102 +7.7% 46401 ± 3% proc-vmstat.nr_shmem
125155 +2.2% 127953 proc-vmstat.nr_zone_active_anon
6252 ± 2% +6.3% 6647 ± 2% proc-vmstat.nr_zone_inactive_anon
213957 ± 11% -82.5% 37345 ± 20% proc-vmstat.numa_hint_faults
178876 ± 10% -88.9% 19891 ± 40% proc-vmstat.numa_hint_faults_local
1.071e+09 ± 9% -30.6% 7.43e+08 ± 5% proc-vmstat.numa_hit
1.071e+09 ± 9% -30.6% 7.429e+08 ± 5% proc-vmstat.numa_local
237444 ± 15% -72.4% 65597 ± 25% proc-vmstat.numa_pte_updates
40514 ± 3% +31.5% 53282 ± 7% proc-vmstat.pgactivate
8.553e+09 ± 9% -30.7% 5.929e+09 ± 5% proc-vmstat.pgalloc_normal
3599397 -5.3% 3407537 proc-vmstat.pgfault
8.553e+09 ± 9% -30.7% 5.929e+09 ± 5% proc-vmstat.pgfree
657361 ± 12% -54.2% 301036 ± 26% sched_debug.cfs_rq:/.MIN_vruntime.avg
5121779 ± 9% -34.6% 3348814 ± 20% sched_debug.cfs_rq:/.MIN_vruntime.stddev
73.53 ± 29% +45.6% 107.09 ± 30% sched_debug.cfs_rq:/.load_avg.max
657361 ± 12% -54.2% 301036 ± 26% sched_debug.cfs_rq:/.max_vruntime.avg
5121779 ± 9% -34.6% 3348814 ± 20% sched_debug.cfs_rq:/.max_vruntime.stddev
50650339 ± 3% +47.8% 74859779 ± 6% sched_debug.cfs_rq:/.min_vruntime.avg
62450655 ± 2% +33.7% 83491916 ± 4% sched_debug.cfs_rq:/.min_vruntime.max
36914060 ± 5% +61.8% 59739601 ± 13% sched_debug.cfs_rq:/.min_vruntime.min
0.73 +15.9% 0.84 ± 2% sched_debug.cfs_rq:/.nr_running.avg
0.40 -27.8% 0.29 ± 9% sched_debug.cfs_rq:/.nr_running.stddev
-994538 -957.6% 8528746 ± 32% sched_debug.cfs_rq:/.spread0.avg
10807042 ± 22% +58.8% 17160375 ± 19% sched_debug.cfs_rq:/.spread0.max
-14731891 -55.3% -6592382 sched_debug.cfs_rq:/.spread0.min
741.02 +16.5% 862.98 ± 2% sched_debug.cfs_rq:/.util_avg.avg
0.57 ± 82% +4756.2% 27.44 ± 38% sched_debug.cfs_rq:/.util_avg.min
367.90 ± 2% -29.3% 260.00 ± 10% sched_debug.cfs_rq:/.util_avg.stddev
642383 +21.0% 777315 ± 3% sched_debug.cpu.avg_idle.avg
412048 -20.4% 328111 ± 5% sched_debug.cpu.avg_idle.stddev
2319 ± 32% -31.8% 1582 ± 15% sched_debug.cpu.clock_task.stddev
2673 +17.7% 3147 ± 2% sched_debug.cpu.curr->pid.avg
1885 -16.0% 1583 ± 2% sched_debug.cpu.curr->pid.stddev
0.72 +16.5% 0.84 ± 2% sched_debug.cpu.nr_running.avg
0.41 -27.7% 0.30 ± 8% sched_debug.cpu.nr_running.stddev
2491773 ± 18% -76.8% 578096 ± 29% sched_debug.cpu.nr_switches.avg
6795283 ± 36% -74.9% 1707607 ± 36% sched_debug.cpu.nr_switches.max
478409 ± 15% -96.4% 17371 ±105% sched_debug.cpu.nr_switches.min
1299648 ± 48% -73.8% 340921 ± 32% sched_debug.cpu.nr_switches.stddev
27.38 ± 3% -25.7% 20.35 ± 5% perf-stat.i.MPKI
9.812e+09 ± 4% -5.5% 9.272e+09 perf-stat.i.branch-instructions
0.81 ± 6% -0.4 0.46 ± 10% perf-stat.i.branch-miss-rate%
80638412 ± 11% -47.7% 42189389 ± 9% perf-stat.i.branch-misses
40.15 ± 8% +20.5 60.60 ± 5% perf-stat.i.cache-miss-rate%
4.562e+08 +5.2% 4.798e+08 perf-stat.i.cache-misses
1.207e+09 ± 9% -33.2% 8.064e+08 ± 5% perf-stat.i.cache-references
1096573 ± 17% -75.8% 265735 ± 29% perf-stat.i.context-switches
10.52 ± 5% +25.7% 13.22 ± 2% perf-stat.i.cpi
4.508e+11 +15.6% 5.21e+11 ± 2% perf-stat.i.cpu-cycles
13047 ± 7% -81.6% 2396 ± 23% perf-stat.i.cpu-migrations
992.80 +13.6% 1128 ± 5% perf-stat.i.cycles-between-cache-misses
0.02 ± 14% -0.0 0.02 ± 9% perf-stat.i.dTLB-load-miss-rate%
2798302 ± 12% -31.0% 1930060 ± 9% perf-stat.i.dTLB-load-misses
1.179e+10 ± 5% -12.2% 1.036e+10 perf-stat.i.dTLB-loads
0.13 ± 15% +0.1 0.19 ± 15% perf-stat.i.dTLB-store-miss-rate%
4296580 ± 7% -17.8% 3530350 ± 14% perf-stat.i.dTLB-store-misses
3.57e+09 ± 11% -46.7% 1.901e+09 ± 8% perf-stat.i.dTLB-stores
78.03 +8.8 86.79 ± 2% perf-stat.i.iTLB-load-miss-rate%
51762078 ± 12% -54.6% 23480553 ± 12% perf-stat.i.iTLB-load-misses
15349414 ± 16% -74.0% 3985887 ± 27% perf-stat.i.iTLB-loads
4.351e+10 ± 4% -9.3% 3.947e+10 perf-stat.i.instructions
935.21 ± 8% +90.8% 1784 ± 11% perf-stat.i.instructions-per-iTLB-miss
0.10 ± 5% -21.5% 0.08 ± 2% perf-stat.i.ipc
2.35 +15.6% 2.71 ± 2% perf-stat.i.metric.GHz
0.12 ± 5% +148.4% 0.31 ± 37% perf-stat.i.metric.K/sec
139.00 ± 5% -15.2% 117.90 perf-stat.i.metric.M/sec
3921 -5.5% 3706 perf-stat.i.minor-faults
82.40 -30.1 52.35 ± 8% perf-stat.i.node-load-miss-rate%
1.602e+08 ± 2% -30.0% 1.122e+08 ± 6% perf-stat.i.node-load-misses
34641208 ± 8% +202.8% 1.049e+08 ± 10% perf-stat.i.node-loads
95.43 -8.8 86.63 perf-stat.i.node-store-miss-rate%
95463384 ± 3% -24.8% 71816176 ± 6% perf-stat.i.node-store-misses
4504280 ± 4% +136.0% 10631682 ± 8% perf-stat.i.node-stores
3921 -5.5% 3706 perf-stat.i.page-faults
27.61 ± 4% -26.1% 20.40 ± 5% perf-stat.overall.MPKI
0.82 ± 7% -0.4 0.45 ± 10% perf-stat.overall.branch-miss-rate%
38.34 ± 9% +21.5 59.81 ± 5% perf-stat.overall.cache-miss-rate%
10.42 ± 5% +26.7% 13.21 ± 2% perf-stat.overall.cpi
989.04 +9.8% 1085 ± 2% perf-stat.overall.cycles-between-cache-misses
0.02 ± 14% -0.0 0.02 ± 9% perf-stat.overall.dTLB-load-miss-rate%
0.12 ± 16% +0.1 0.19 ± 15% perf-stat.overall.dTLB-store-miss-rate%
77.26 +8.6 85.86 ± 2% perf-stat.overall.iTLB-load-miss-rate%
853.07 ± 7% +100.6% 1711 ± 12% perf-stat.overall.instructions-per-iTLB-miss
0.10 ± 5% -21.3% 0.08 ± 2% perf-stat.overall.ipc
82.05 -30.3 51.72 ± 8% perf-stat.overall.node-load-miss-rate%
95.46 -8.4 87.03 perf-stat.overall.node-store-miss-rate%
18379 ± 4% +31.1% 24093 ± 5% perf-stat.overall.path-length
9.785e+09 ± 4% -5.4% 9.259e+09 perf-stat.ps.branch-instructions
79985971 ± 11% -47.5% 41978792 ± 9% perf-stat.ps.branch-misses
4.56e+08 +5.2% 4.795e+08 perf-stat.ps.cache-misses
1.2e+09 ± 9% -33.0% 8.039e+08 ± 5% perf-stat.ps.cache-references
1084462 ± 17% -75.8% 262972 ± 30% perf-stat.ps.context-switches
4.509e+11 +15.5% 5.206e+11 ± 2% perf-stat.ps.cpu-cycles
12921 ± 8% -81.6% 2375 ± 25% perf-stat.ps.cpu-migrations
2775208 ± 11% -30.8% 1921057 ± 9% perf-stat.ps.dTLB-load-misses
1.175e+10 ± 5% -12.0% 1.034e+10 perf-stat.ps.dTLB-loads
3.542e+09 ± 11% -46.5% 1.894e+09 ± 8% perf-stat.ps.dTLB-stores
51329779 ± 12% -54.5% 23375015 ± 12% perf-stat.ps.iTLB-load-misses
15181049 ± 16% -74.1% 3937533 ± 27% perf-stat.ps.iTLB-loads
4.337e+10 ± 4% -9.1% 3.941e+10 perf-stat.ps.instructions
3905 -5.5% 3690 perf-stat.ps.minor-faults
1.598e+08 ± 2% -29.9% 1.121e+08 ± 6% perf-stat.ps.node-load-misses
34957327 ± 8% +200.2% 1.049e+08 ± 10% perf-stat.ps.node-loads
95360155 ± 3% -24.7% 71759787 ± 6% perf-stat.ps.node-store-misses
4532465 ± 4% +134.6% 10630923 ± 8% perf-stat.ps.node-stores
3905 -5.5% 3690 perf-stat.ps.page-faults
3.911e+13 ± 4% -9.1% 3.554e+13 perf-stat.total.instructions
20018 ± 25% -65.9% 6832 ± 32% interrupts.CPU10.RES:Rescheduling_interrupts
6930 ± 22% +20.4% 8344 ± 3% interrupts.CPU100.NMI:Non-maskable_interrupts
6930 ± 22% +20.4% 8344 ± 3% interrupts.CPU100.PMI:Performance_monitoring_interrupts
26175 ± 14% -67.9% 8402 ± 48% interrupts.CPU100.RES:Rescheduling_interrupts
26256 ± 18% -69.9% 7911 ± 26% interrupts.CPU101.RES:Rescheduling_interrupts
23953 ± 16% -64.2% 8581 ± 24% interrupts.CPU102.RES:Rescheduling_interrupts
22838 ± 15% -52.8% 10770 ± 55% interrupts.CPU103.RES:Rescheduling_interrupts
22692 ± 9% -63.1% 8375 ± 60% interrupts.CPU104.RES:Rescheduling_interrupts
22355 ± 16% -76.8% 5197 ± 73% interrupts.CPU105.RES:Rescheduling_interrupts
18887 ± 17% -74.0% 4919 ± 27% interrupts.CPU107.RES:Rescheduling_interrupts
17905 ± 15% -71.2% 5158 ± 70% interrupts.CPU108.RES:Rescheduling_interrupts
17658 ± 19% -80.4% 3468 ± 65% interrupts.CPU109.RES:Rescheduling_interrupts
16137 ± 10% -53.2% 7549 ± 47% interrupts.CPU11.RES:Rescheduling_interrupts
18899 ± 11% -65.1% 6597 ± 32% interrupts.CPU110.RES:Rescheduling_interrupts
6906 ± 23% +23.3% 8515 interrupts.CPU111.NMI:Non-maskable_interrupts
6906 ± 23% +23.3% 8515 interrupts.CPU111.PMI:Performance_monitoring_interrupts
18144 ± 12% -66.2% 6134 ± 82% interrupts.CPU111.RES:Rescheduling_interrupts
15208 ± 12% -70.4% 4509 ± 65% interrupts.CPU112.RES:Rescheduling_interrupts
19107 ± 17% -62.4% 7181 ± 63% interrupts.CPU113.RES:Rescheduling_interrupts
17575 ± 9% -56.2% 7690 ± 69% interrupts.CPU114.RES:Rescheduling_interrupts
17096 ± 8% -65.5% 5889 ± 85% interrupts.CPU115.RES:Rescheduling_interrupts
15285 ± 19% -57.1% 6556 ± 26% interrupts.CPU116.RES:Rescheduling_interrupts
16701 ± 17% -76.0% 4009 ± 42% interrupts.CPU117.RES:Rescheduling_interrupts
14198 ± 18% -78.7% 3029 ± 43% interrupts.CPU118.RES:Rescheduling_interrupts
12635 ± 6% -59.9% 5071 ± 42% interrupts.CPU119.RES:Rescheduling_interrupts
111.20 ± 39% -66.0% 37.80 ± 54% interrupts.CPU119.TLB:TLB_shootdowns
21097 ± 21% -67.3% 6890 ± 47% interrupts.CPU12.RES:Rescheduling_interrupts
38899 ± 13% -77.7% 8664 ± 36% interrupts.CPU120.RES:Rescheduling_interrupts
35324 ± 16% -78.9% 7468 ± 53% interrupts.CPU121.RES:Rescheduling_interrupts
30246 ± 15% -77.3% 6857 ± 36% interrupts.CPU122.RES:Rescheduling_interrupts
29316 ± 15% -76.0% 7025 ± 58% interrupts.CPU123.RES:Rescheduling_interrupts
26502 ± 17% -63.4% 9695 ± 38% interrupts.CPU124.RES:Rescheduling_interrupts
25454 ± 13% -65.1% 8892 ± 58% interrupts.CPU125.RES:Rescheduling_interrupts
25998 ± 28% -76.8% 6034 ± 30% interrupts.CPU126.RES:Rescheduling_interrupts
27086 ± 15% -70.2% 8084 ± 45% interrupts.CPU127.RES:Rescheduling_interrupts
21401 ± 19% -72.4% 5899 ± 39% interrupts.CPU128.RES:Rescheduling_interrupts
23975 ± 18% -71.6% 6800 ± 68% interrupts.CPU129.RES:Rescheduling_interrupts
19847 ± 17% -65.9% 6763 ± 65% interrupts.CPU13.RES:Rescheduling_interrupts
20441 ± 23% -66.6% 6836 ± 45% interrupts.CPU130.RES:Rescheduling_interrupts
20268 ± 28% -67.9% 6508 ± 39% interrupts.CPU131.RES:Rescheduling_interrupts
19568 ± 17% -77.4% 4420 ± 58% interrupts.CPU132.RES:Rescheduling_interrupts
18921 ± 22% -63.0% 6991 ± 19% interrupts.CPU133.RES:Rescheduling_interrupts
19231 ± 18% -70.1% 5746 ± 50% interrupts.CPU134.RES:Rescheduling_interrupts
5876 ± 27% +43.5% 8430 ± 2% interrupts.CPU135.NMI:Non-maskable_interrupts
5876 ± 27% +43.5% 8430 ± 2% interrupts.CPU135.PMI:Performance_monitoring_interrupts
18139 ± 15% -64.8% 6388 ± 60% interrupts.CPU135.RES:Rescheduling_interrupts
17565 ± 22% -68.7% 5505 ± 47% interrupts.CPU136.RES:Rescheduling_interrupts
18447 ± 14% -80.5% 3592 ± 78% interrupts.CPU137.RES:Rescheduling_interrupts
15921 ± 28% -62.6% 5948 ± 53% interrupts.CPU138.RES:Rescheduling_interrupts
15194 ± 24% -60.3% 6029 ± 33% interrupts.CPU139.RES:Rescheduling_interrupts
19339 ± 21% -70.7% 5667 ± 21% interrupts.CPU14.RES:Rescheduling_interrupts
6659 ± 25% +23.0% 8190 ± 5% interrupts.CPU140.NMI:Non-maskable_interrupts
6659 ± 25% +23.0% 8190 ± 5% interrupts.CPU140.PMI:Performance_monitoring_interrupts
15306 ± 19% -62.2% 5782 ± 49% interrupts.CPU140.RES:Rescheduling_interrupts
14340 ± 11% -72.7% 3913 ± 38% interrupts.CPU141.RES:Rescheduling_interrupts
10236 ± 26% -60.4% 4056 ± 60% interrupts.CPU142.RES:Rescheduling_interrupts
9089 ± 9% -44.5% 5045 ± 41% interrupts.CPU143.RES:Rescheduling_interrupts
36285 ± 13% -82.8% 6248 ± 73% interrupts.CPU144.RES:Rescheduling_interrupts
35513 ± 12% -75.3% 8776 ± 42% interrupts.CPU145.RES:Rescheduling_interrupts
31846 ± 10% -81.0% 6036 ± 38% interrupts.CPU146.RES:Rescheduling_interrupts
27606 ± 15% -81.5% 5099 ± 61% interrupts.CPU147.RES:Rescheduling_interrupts
28526 ± 17% -77.8% 6335 ± 54% interrupts.CPU148.RES:Rescheduling_interrupts
5629 ± 35% +43.8% 8093 ± 5% interrupts.CPU149.NMI:Non-maskable_interrupts
5629 ± 35% +43.8% 8093 ± 5% interrupts.CPU149.PMI:Performance_monitoring_interrupts
23694 ± 9% -65.5% 8176 ± 52% interrupts.CPU149.RES:Rescheduling_interrupts
17594 ± 23% -60.3% 6977 ± 57% interrupts.CPU15.RES:Rescheduling_interrupts
25905 ± 15% -73.6% 6827 ± 59% interrupts.CPU150.RES:Rescheduling_interrupts
24007 ± 17% -83.4% 3992 ± 35% interrupts.CPU151.RES:Rescheduling_interrupts
26038 ± 21% -69.0% 8076 ± 66% interrupts.CPU152.RES:Rescheduling_interrupts
22935 ± 7% -66.6% 7668 ± 32% interrupts.CPU153.RES:Rescheduling_interrupts
62.60 ± 45% -65.2% 21.80 ± 47% interrupts.CPU153.TLB:TLB_shootdowns
22072 ± 14% -79.7% 4472 ± 57% interrupts.CPU154.RES:Rescheduling_interrupts
21397 ± 16% -71.6% 6070 ± 41% interrupts.CPU155.RES:Rescheduling_interrupts
19854 ± 21% -65.7% 6815 ± 72% interrupts.CPU156.RES:Rescheduling_interrupts
21814 ± 21% -71.5% 6218 ± 95% interrupts.CPU157.RES:Rescheduling_interrupts
22265 ± 24% -75.8% 5390 ± 46% interrupts.CPU158.RES:Rescheduling_interrupts
21240 ± 20% -58.8% 8747 ± 52% interrupts.CPU159.RES:Rescheduling_interrupts
17437 ± 21% -63.7% 6334 ± 29% interrupts.CPU16.RES:Rescheduling_interrupts
65.60 ± 56% -67.1% 21.60 ± 67% interrupts.CPU16.TLB:TLB_shootdowns
17052 ± 9% -62.5% 6396 ± 27% interrupts.CPU160.RES:Rescheduling_interrupts
15195 ± 6% -72.1% 4245 ± 61% interrupts.CPU161.RES:Rescheduling_interrupts
14960 ± 24% -55.0% 6730 ± 55% interrupts.CPU162.RES:Rescheduling_interrupts
17994 ± 16% -64.3% 6426 ± 68% interrupts.CPU163.RES:Rescheduling_interrupts
14710 ± 13% -58.7% 6076 ± 43% interrupts.CPU164.RES:Rescheduling_interrupts
11712 ± 14% -48.4% 6041 ± 56% interrupts.CPU165.RES:Rescheduling_interrupts
11431 ± 14% -42.6% 6562 ± 27% interrupts.CPU167.RES:Rescheduling_interrupts
37298 ± 7% -76.3% 8847 ± 19% interrupts.CPU168.RES:Rescheduling_interrupts
35906 ± 10% -68.3% 11386 ± 70% interrupts.CPU169.RES:Rescheduling_interrupts
16528 ± 34% -65.4% 5723 ± 52% interrupts.CPU17.RES:Rescheduling_interrupts
33673 ± 15% -76.9% 7761 ± 73% interrupts.CPU170.RES:Rescheduling_interrupts
65.40 ± 38% -55.0% 29.40 ± 67% interrupts.CPU170.TLB:TLB_shootdowns
29357 ± 15% -69.6% 8935 ± 39% interrupts.CPU171.RES:Rescheduling_interrupts
28550 ± 27% -67.3% 9324 ± 78% interrupts.CPU172.RES:Rescheduling_interrupts
23625 ± 21% -79.5% 4850 ± 71% interrupts.CPU173.RES:Rescheduling_interrupts
24733 ± 10% -70.5% 7294 ± 74% interrupts.CPU174.RES:Rescheduling_interrupts
25344 ± 14% -74.7% 6408 ± 68% interrupts.CPU175.RES:Rescheduling_interrupts
20908 ± 29% -66.4% 7021 ± 45% interrupts.CPU177.RES:Rescheduling_interrupts
21900 ± 27% -71.7% 6199 ± 75% interrupts.CPU178.RES:Rescheduling_interrupts
20672 ± 15% -60.3% 8201 ± 57% interrupts.CPU179.RES:Rescheduling_interrupts
19053 ± 20% -64.5% 6769 ± 39% interrupts.CPU18.RES:Rescheduling_interrupts
24900 ± 26% -70.4% 7361 ± 71% interrupts.CPU180.RES:Rescheduling_interrupts
23027 ± 18% -63.1% 8499 ± 59% interrupts.CPU181.RES:Rescheduling_interrupts
19105 ± 25% -66.5% 6409 ± 46% interrupts.CPU183.RES:Rescheduling_interrupts
18049 ± 6% -77.8% 4015 ± 59% interrupts.CPU184.RES:Rescheduling_interrupts
17829 ± 34% -79.1% 3731 ± 97% interrupts.CPU185.RES:Rescheduling_interrupts
17896 ± 21% -56.7% 7741 ± 64% interrupts.CPU187.RES:Rescheduling_interrupts
19615 ± 25% -68.4% 6205 ± 45% interrupts.CPU188.RES:Rescheduling_interrupts
5762 ± 37% +20.6% 6948 ± 28% interrupts.CPU189.NMI:Non-maskable_interrupts
5762 ± 37% +20.6% 6948 ± 28% interrupts.CPU189.PMI:Performance_monitoring_interrupts
18061 ± 16% -65.4% 6251 ± 53% interrupts.CPU19.RES:Rescheduling_interrupts
20448 ± 24% -68.6% 6426 ± 41% interrupts.CPU20.RES:Rescheduling_interrupts
60.60 ± 57% -74.6% 15.40 ± 68% interrupts.CPU20.TLB:TLB_shootdowns
18718 ± 26% -77.9% 4146 ± 35% interrupts.CPU21.RES:Rescheduling_interrupts
19195 ± 24% -72.9% 5208 ± 33% interrupts.CPU22.RES:Rescheduling_interrupts
18435 ± 24% -59.7% 7435 ± 30% interrupts.CPU23.RES:Rescheduling_interrupts
12421 ± 21% -61.9% 4733 ± 29% interrupts.CPU25.RES:Rescheduling_interrupts
14604 ± 24% -54.6% 6631 ± 34% interrupts.CPU28.RES:Rescheduling_interrupts
16075 ± 26% -61.1% 6249 ± 38% interrupts.CPU29.RES:Rescheduling_interrupts
903.60 +13.5% 1026 ± 9% interrupts.CPU31.CAL:Function_call_interrupts
17567 ± 29% -62.3% 6617 ± 46% interrupts.CPU32.RES:Rescheduling_interrupts
15244 ± 21% -68.5% 4809 ± 50% interrupts.CPU33.RES:Rescheduling_interrupts
16526 ± 16% -63.8% 5988 ± 71% interrupts.CPU35.RES:Rescheduling_interrupts
19889 ± 6% -75.0% 4966 ± 51% interrupts.CPU36.RES:Rescheduling_interrupts
19404 ± 22% -66.2% 6563 ± 32% interrupts.CPU37.RES:Rescheduling_interrupts
19446 ± 11% -62.8% 7240 ± 60% interrupts.CPU38.RES:Rescheduling_interrupts
19790 ± 25% -63.1% 7297 ± 21% interrupts.CPU39.RES:Rescheduling_interrupts
8407 ± 4% -22.2% 6541 ± 29% interrupts.CPU4.NMI:Non-maskable_interrupts
8407 ± 4% -22.2% 6541 ± 29% interrupts.CPU4.PMI:Performance_monitoring_interrupts
21624 ± 20% -60.7% 8506 ± 59% interrupts.CPU40.RES:Rescheduling_interrupts
19575 ± 15% -72.3% 5430 ± 35% interrupts.CPU41.RES:Rescheduling_interrupts
22512 ± 21% -76.1% 5370 ± 65% interrupts.CPU42.RES:Rescheduling_interrupts
20342 ± 11% -78.7% 4337 ± 27% interrupts.CPU43.RES:Rescheduling_interrupts
71.00 ± 77% -66.5% 23.80 ± 53% interrupts.CPU43.TLB:TLB_shootdowns
22717 ± 16% -80.8% 4363 ± 42% interrupts.CPU44.RES:Rescheduling_interrupts
22427 ± 25% -87.0% 2919 ± 29% interrupts.CPU45.RES:Rescheduling_interrupts
25129 ± 26% -75.6% 6131 ± 35% interrupts.CPU46.RES:Rescheduling_interrupts
25311 ± 27% -69.7% 7679 ± 46% interrupts.CPU47.RES:Rescheduling_interrupts
12818 ± 23% -45.9% 6933 ± 36% interrupts.CPU5.RES:Rescheduling_interrupts
12087 ± 43% -56.6% 5248 ± 43% interrupts.CPU50.RES:Rescheduling_interrupts
15964 ± 15% -66.9% 5290 ± 59% interrupts.CPU51.RES:Rescheduling_interrupts
12712 ± 16% -51.1% 6220 ± 56% interrupts.CPU52.RES:Rescheduling_interrupts
12931 ± 8% -48.8% 6621 ± 52% interrupts.CPU53.RES:Rescheduling_interrupts
15241 ± 29% -46.6% 8135 ± 61% interrupts.CPU54.RES:Rescheduling_interrupts
18851 ± 10% -75.1% 4691 ± 68% interrupts.CPU55.RES:Rescheduling_interrupts
16114 ± 20% -72.6% 4418 ± 49% interrupts.CPU56.RES:Rescheduling_interrupts
14867 ± 36% -46.9% 7901 ± 54% interrupts.CPU57.RES:Rescheduling_interrupts
18445 ± 19% -60.3% 7329 ± 57% interrupts.CPU60.RES:Rescheduling_interrupts
16799 ± 19% -56.2% 7353 ± 56% interrupts.CPU61.RES:Rescheduling_interrupts
16466 ± 9% -49.0% 8398 ± 41% interrupts.CPU63.RES:Rescheduling_interrupts
19173 ± 7% -51.3% 9329 ± 22% interrupts.CPU64.RES:Rescheduling_interrupts
19019 ± 15% -65.0% 6662 ± 70% interrupts.CPU65.RES:Rescheduling_interrupts
24865 ± 14% -70.3% 7383 ± 31% interrupts.CPU66.RES:Rescheduling_interrupts
20350 ± 29% -52.0% 9765 ± 17% interrupts.CPU67.RES:Rescheduling_interrupts
21037 ± 25% -67.7% 6793 ± 55% interrupts.CPU68.RES:Rescheduling_interrupts
25286 ± 17% -70.7% 7398 ± 46% interrupts.CPU69.RES:Rescheduling_interrupts
17345 ± 17% -58.4% 7218 ± 40% interrupts.CPU7.RES:Rescheduling_interrupts
23913 ± 12% -67.2% 7844 ± 39% interrupts.CPU70.RES:Rescheduling_interrupts
22159 ± 29% -72.9% 5997 ± 42% interrupts.CPU71.RES:Rescheduling_interrupts
16007 ± 38% -54.6% 7270 ± 33% interrupts.CPU75.RES:Rescheduling_interrupts
16100 ± 29% -62.5% 6037 ± 61% interrupts.CPU76.RES:Rescheduling_interrupts
17691 ± 13% -71.8% 4992 ± 66% interrupts.CPU77.RES:Rescheduling_interrupts
19181 ± 29% -55.9% 8452 ± 34% interrupts.CPU78.RES:Rescheduling_interrupts
16480 ± 42% -66.8% 5468 ± 41% interrupts.CPU79.RES:Rescheduling_interrupts
14797 ± 21% -38.0% 9169 ± 15% interrupts.CPU8.RES:Rescheduling_interrupts
16987 ± 18% -55.3% 7601 ± 76% interrupts.CPU80.RES:Rescheduling_interrupts
20522 ± 15% -58.5% 8522 ± 32% interrupts.CPU81.RES:Rescheduling_interrupts
18750 ± 24% -71.3% 5386 ± 91% interrupts.CPU82.RES:Rescheduling_interrupts
21485 ± 34% -70.6% 6317 ±102% interrupts.CPU83.RES:Rescheduling_interrupts
15390 ± 19% -57.3% 6576 ± 66% interrupts.CPU84.RES:Rescheduling_interrupts
16251 ± 35% -60.0% 6503 ± 9% interrupts.CPU85.RES:Rescheduling_interrupts
22661 ± 20% -61.2% 8781 ± 43% interrupts.CPU86.RES:Rescheduling_interrupts
21182 ± 37% -72.1% 5915 ± 34% interrupts.CPU87.RES:Rescheduling_interrupts
20482 ± 27% -66.2% 6913 ± 55% interrupts.CPU88.RES:Rescheduling_interrupts
22303 ± 18% -74.8% 5616 ± 97% interrupts.CPU89.RES:Rescheduling_interrupts
22674 ± 39% -72.3% 6285 ± 45% interrupts.CPU90.RES:Rescheduling_interrupts
22527 ± 38% -66.7% 7509 ± 78% interrupts.CPU91.RES:Rescheduling_interrupts
24202 ± 41% -77.5% 5449 ± 75% interrupts.CPU93.RES:Rescheduling_interrupts
24056 ± 26% -62.5% 9013 ± 50% interrupts.CPU94.RES:Rescheduling_interrupts
25980 ± 26% -59.7% 10460 ± 31% interrupts.CPU96.RES:Rescheduling_interrupts
33285 ± 20% -78.3% 7213 ± 45% interrupts.CPU97.RES:Rescheduling_interrupts
88.00 ± 53% -77.5% 19.80 ± 63% interrupts.CPU97.TLB:TLB_shootdowns
30452 ± 14% -66.5% 10214 ± 57% interrupts.CPU98.RES:Rescheduling_interrupts
27805 ± 7% -67.8% 8944 ± 60% interrupts.CPU99.RES:Rescheduling_interrupts
3779530 ± 8% -64.4% 1345454 ± 26% interrupts.RES:Rescheduling_interrupts
23.08 -16.2 6.89 ± 45% perf-profile.calltrace.cycles-pp.secondary_startup_64
22.99 -16.1 6.85 ± 45% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
22.99 -16.1 6.85 ± 45% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
22.98 -16.1 6.85 ± 45% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
22.44 -15.7 6.73 ± 45% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
22.43 -15.7 6.73 ± 45% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
21.98 -15.4 6.62 ± 45% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
14.81 ± 3% -8.9 5.95 ± 34% perf-profile.calltrace.cycles-pp.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto
14.74 ± 3% -8.9 5.89 ± 35% perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto
14.11 ± 3% -8.4 5.72 ± 34% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg
14.09 ± 3% -8.4 5.71 ± 34% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg
12.73 ± 3% -8.2 4.55 ± 43% perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock
12.68 ± 3% -8.2 4.51 ± 43% perf-profile.calltrace.cycles-pp.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock
12.37 ± 4% -8.1 4.25 ± 46% perf-profile.calltrace.cycles-pp.skb_release_data.__kfree_skb.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established
12.42 ± 4% -8.1 4.30 ± 46% perf-profile.calltrace.cycles-pp.__kfree_skb.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv
11.93 ± 4% -8.0 3.95 ± 49% perf-profile.calltrace.cycles-pp.__free_pages_ok.skb_release_data.__kfree_skb.tcp_clean_rtx_queue.tcp_ack
11.79 ± 4% -7.9 3.90 ± 49% perf-profile.calltrace.cycles-pp.free_one_page.__free_pages_ok.skb_release_data.__kfree_skb.tcp_clean_rtx_queue
51.48 -1.5 50.03 perf-profile.calltrace.cycles-pp.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
51.47 -1.5 50.02 perf-profile.calltrace.cycles-pp.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
51.38 -1.4 49.95 perf-profile.calltrace.cycles-pp.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
51.33 -1.4 49.90 perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64
1.75 ± 5% -0.6 1.17 ± 6% perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock
1.75 ± 5% -0.6 1.17 ± 5% perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv.__release_sock
1.29 ± 8% -0.6 0.71 ± 10% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established
1.54 ± 6% -0.6 0.97 ± 6% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv
7.37 ± 3% -0.6 6.81 perf-profile.calltrace.cycles-pp._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
7.11 ± 2% -0.5 6.57 perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg
7.22 ± 3% -0.5 6.68 perf-profile.calltrace.cycles-pp.copyin._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
0.00 +0.5 0.52 ± 3% perf-profile.calltrace.cycles-pp.release_sock.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
8.04 ± 2% +1.0 9.09 ± 2% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
8.23 ± 2% +1.1 9.35 ± 2% perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg
8.63 ± 2% +1.1 9.76 ± 2% perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
8.62 ± 2% +1.1 9.75 ± 2% perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
8.19 ± 2% +1.1 9.32 ± 2% perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg
36.28 +7.5 43.79 ± 4% perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto
24.55 +8.2 32.71 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.rmqueue.get_page_from_freelist.__alloc_pages_nodemask
24.99 +8.2 33.18 ± 5% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.rmqueue.get_page_from_freelist.__alloc_pages_nodemask.skb_page_frag_refill
26.00 +8.2 34.25 ± 5% perf-profile.calltrace.cycles-pp.rmqueue.get_page_from_freelist.__alloc_pages_nodemask.skb_page_frag_refill.sk_page_frag_refill
26.80 +8.3 35.14 ± 5% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.skb_page_frag_refill.sk_page_frag_refill.tcp_sendmsg_locked.tcp_sendmsg
26.89 +8.3 35.23 ± 5% perf-profile.calltrace.cycles-pp.sk_page_frag_refill.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
26.73 +8.3 35.07 ± 5% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.skb_page_frag_refill.sk_page_frag_refill.tcp_sendmsg_locked
26.91 +8.3 35.25 ± 5% perf-profile.calltrace.cycles-pp.skb_page_frag_refill.sk_page_frag_refill.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
24.02 +8.8 32.85 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_one_page.__free_pages_ok.skb_release_data
24.44 +8.9 33.32 ± 5% perf-profile.calltrace.cycles-pp._raw_spin_lock.free_one_page.__free_pages_ok.skb_release_data.__kfree_skb
76.53 +16.3 92.79 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
76.51 +16.3 92.78 ± 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
13.05 ± 3% +16.7 29.76 ± 11% perf-profile.calltrace.cycles-pp.free_one_page.__free_pages_ok.skb_release_data.__kfree_skb.tcp_recvmsg
13.42 ± 3% +17.0 30.47 ± 11% perf-profile.calltrace.cycles-pp.__free_pages_ok.skb_release_data.__kfree_skb.tcp_recvmsg.inet_recvmsg
13.93 ± 3% +17.3 31.25 ± 11% perf-profile.calltrace.cycles-pp.skb_release_data.__kfree_skb.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
14.12 ± 3% +17.4 31.48 ± 11% perf-profile.calltrace.cycles-pp.__kfree_skb.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
24.95 ± 2% +17.7 42.70 ± 7% perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
24.95 ± 2% +17.7 42.70 ± 7% perf-profile.calltrace.cycles-pp.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
24.89 ± 2% +17.8 42.68 ± 7% perf-profile.calltrace.cycles-pp.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
24.89 ± 2% +17.8 42.67 ± 7% perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64
23.08 -16.2 6.89 ± 45% perf-profile.children.cycles-pp.secondary_startup_64
23.08 -16.2 6.89 ± 45% perf-profile.children.cycles-pp.cpu_startup_entry
23.07 -16.2 6.89 ± 45% perf-profile.children.cycles-pp.do_idle
22.99 -16.1 6.85 ± 45% perf-profile.children.cycles-pp.start_secondary
22.53 -15.8 6.77 ± 44% perf-profile.children.cycles-pp.cpuidle_enter
22.53 -15.8 6.77 ± 44% perf-profile.children.cycles-pp.cpuidle_enter_state
22.07 -15.4 6.66 ± 44% perf-profile.children.cycles-pp.intel_idle
15.38 ± 3% -8.9 6.48 ± 32% perf-profile.children.cycles-pp.tcp_rcv_established
15.42 ± 3% -8.9 6.52 ± 32% perf-profile.children.cycles-pp.tcp_v4_do_rcv
15.17 ± 3% -8.7 6.48 ± 31% perf-profile.children.cycles-pp.release_sock
15.06 ± 3% -8.7 6.38 ± 32% perf-profile.children.cycles-pp.__release_sock
12.87 ± 3% -8.3 4.62 ± 43% perf-profile.children.cycles-pp.tcp_ack
12.81 ± 3% -8.2 4.57 ± 43% perf-profile.children.cycles-pp.tcp_clean_rtx_queue
51.48 -1.5 50.03 perf-profile.children.cycles-pp.__x64_sys_sendto
51.47 -1.5 50.02 perf-profile.children.cycles-pp.__sys_sendto
51.38 -1.4 49.95 perf-profile.children.cycles-pp.sock_sendmsg
51.33 -1.4 49.90 perf-profile.children.cycles-pp.tcp_sendmsg
2.82 ± 7% -1.2 1.64 ± 12% perf-profile.children.cycles-pp.__tcp_transmit_skb
2.35 ± 8% -1.1 1.27 ± 14% perf-profile.children.cycles-pp.__ip_queue_xmit
2.03 ± 10% -1.1 0.97 ± 19% perf-profile.children.cycles-pp.ip_output
1.95 ± 10% -1.0 0.93 ± 19% perf-profile.children.cycles-pp.ip_finish_output2
1.69 ± 10% -0.9 0.79 ± 21% perf-profile.children.cycles-pp.__local_bh_enable_ip
1.66 ± 10% -0.9 0.78 ± 21% perf-profile.children.cycles-pp.do_softirq
1.69 ± 10% -0.9 0.81 ± 20% perf-profile.children.cycles-pp.__softirqentry_text_start
1.64 ± 10% -0.9 0.77 ± 21% perf-profile.children.cycles-pp.do_softirq_own_stack
2.39 ± 5% -0.9 1.53 ± 8% perf-profile.children.cycles-pp.tcp_write_xmit
1.57 ± 10% -0.8 0.73 ± 21% perf-profile.children.cycles-pp.net_rx_action
1.54 ± 10% -0.8 0.71 ± 22% perf-profile.children.cycles-pp.process_backlog
1.49 ± 10% -0.8 0.70 ± 22% perf-profile.children.cycles-pp.__netif_receive_skb_one_core
1.45 ± 10% -0.8 0.67 ± 22% perf-profile.children.cycles-pp.ip_rcv
1.41 ± 10% -0.8 0.65 ± 23% perf-profile.children.cycles-pp.ip_local_deliver
1.40 ± 10% -0.8 0.65 ± 23% perf-profile.children.cycles-pp.ip_local_deliver_finish
1.39 ± 10% -0.8 0.64 ± 23% perf-profile.children.cycles-pp.ip_protocol_deliver_rcu
1.37 ± 10% -0.7 0.63 ± 23% perf-profile.children.cycles-pp.tcp_v4_rcv
2.03 ± 4% -0.6 1.42 ± 5% perf-profile.children.cycles-pp.__tcp_push_pending_frames
7.38 ± 3% -0.6 6.82 perf-profile.children.cycles-pp._copy_from_iter_full
7.25 ± 3% -0.5 6.72 perf-profile.children.cycles-pp.copyin
0.69 ± 12% -0.5 0.19 ± 43% perf-profile.children.cycles-pp.sock_def_readable
0.67 ± 12% -0.5 0.18 ± 44% perf-profile.children.cycles-pp.__wake_up_common_lock
0.66 ± 17% -0.5 0.18 ± 31% perf-profile.children.cycles-pp.__schedule
0.64 ± 12% -0.5 0.18 ± 44% perf-profile.children.cycles-pp.__wake_up_common
0.63 ± 11% -0.5 0.17 ± 43% perf-profile.children.cycles-pp.try_to_wake_up
0.51 ± 17% -0.4 0.10 ± 47% perf-profile.children.cycles-pp.sk_wait_data
0.46 ± 16% -0.3 0.13 ± 29% perf-profile.children.cycles-pp.wait_woken
0.43 ± 16% -0.3 0.13 ± 26% perf-profile.children.cycles-pp.schedule_timeout
0.43 ± 17% -0.3 0.13 ± 25% perf-profile.children.cycles-pp.schedule
0.34 ± 12% -0.3 0.08 ± 67% perf-profile.children.cycles-pp.ttwu_do_activate
0.34 ± 13% -0.3 0.08 ± 67% perf-profile.children.cycles-pp.activate_task
0.34 ± 13% -0.3 0.08 ± 65% perf-profile.children.cycles-pp.enqueue_task_fair
0.27 ± 14% -0.2 0.05 ± 90% perf-profile.children.cycles-pp.enqueue_entity
0.20 ± 9% -0.1 0.07 ± 34% perf-profile.children.cycles-pp.lock_sock_nested
0.24 ± 6% -0.1 0.14 ± 19% perf-profile.children.cycles-pp._raw_spin_lock_bh
0.23 ± 8% -0.1 0.13 ± 11% perf-profile.children.cycles-pp.__dev_queue_xmit
0.18 ± 15% -0.1 0.09 ± 18% perf-profile.children.cycles-pp.pick_next_task_fair
0.23 ± 12% -0.1 0.14 ± 12% perf-profile.children.cycles-pp.update_load_avg
0.17 ± 27% -0.1 0.08 ± 20% perf-profile.children.cycles-pp.update_cfs_group
0.33 ± 2% -0.1 0.24 ± 6% perf-profile.children.cycles-pp.__list_del_entry_valid
0.12 ± 6% -0.1 0.04 ± 83% perf-profile.children.cycles-pp.__switch_to
0.15 ± 5% -0.1 0.08 ± 12% perf-profile.children.cycles-pp.dev_hard_start_xmit
0.11 ± 12% -0.1 0.05 ± 52% perf-profile.children.cycles-pp.update_rq_clock
0.13 ± 8% -0.1 0.07 ± 10% perf-profile.children.cycles-pp.loopback_xmit
0.08 ± 10% -0.0 0.03 ± 82% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.11 ± 16% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.update_curr
0.07 -0.0 0.03 ± 81% perf-profile.children.cycles-pp.__ksize
0.35 -0.0 0.31 ± 4% perf-profile.children.cycles-pp.__free_one_page
0.12 ± 10% -0.0 0.09 ± 11% perf-profile.children.cycles-pp.__inet_lookup_established
0.10 ± 4% -0.0 0.07 ± 12% perf-profile.children.cycles-pp.skb_release_all
0.10 ± 7% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.__list_add_valid
0.09 ± 8% -0.0 0.06 ± 14% perf-profile.children.cycles-pp.__tcp_send_ack
0.10 ± 6% -0.0 0.07 ± 12% perf-profile.children.cycles-pp.skb_release_head_state
0.06 ± 6% -0.0 0.04 ± 50% perf-profile.children.cycles-pp.tcp_event_new_data_sent
0.06 ± 6% -0.0 0.04 ± 50% perf-profile.children.cycles-pp.sockfd_lookup_light
0.07 ± 5% -0.0 0.05 ± 7% perf-profile.children.cycles-pp.tcp_current_mss
0.09 ± 5% -0.0 0.08 ± 6% perf-profile.children.cycles-pp.__skb_clone
0.07 ± 11% -0.0 0.05 ± 7% perf-profile.children.cycles-pp.tcp_send_mss
0.08 ± 4% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.sk_reset_timer
0.08 -0.0 0.07 ± 7% perf-profile.children.cycles-pp.mod_timer
0.09 +0.0 0.11 ± 4% perf-profile.children.cycles-pp.kmem_cache_free
0.19 +0.0 0.21 ± 3% perf-profile.children.cycles-pp.__slab_free
0.09 ± 5% +0.0 0.11 ± 4% perf-profile.children.cycles-pp.__mod_zone_page_state
0.11 ± 6% +0.0 0.13 ± 4% perf-profile.children.cycles-pp.prep_new_page
0.27 +0.0 0.30 ± 3% perf-profile.children.cycles-pp.simple_copy_to_iter
0.08 ± 20% +0.0 0.11 ± 10% perf-profile.children.cycles-pp.ksys_write
0.08 ± 22% +0.0 0.11 ± 8% perf-profile.children.cycles-pp.vfs_write
0.08 ± 22% +0.0 0.11 ± 8% perf-profile.children.cycles-pp.__libc_write
0.08 ± 19% +0.0 0.11 ± 6% perf-profile.children.cycles-pp.generic_file_write_iter
0.08 ± 19% +0.0 0.11 ± 6% perf-profile.children.cycles-pp.__generic_file_write_iter
0.08 ± 19% +0.0 0.11 ± 6% perf-profile.children.cycles-pp.generic_perform_write
0.08 ± 19% +0.0 0.11 ± 8% perf-profile.children.cycles-pp.new_sync_write
0.12 ± 4% +0.0 0.15 ± 5% perf-profile.children.cycles-pp.skb_clone
0.11 ± 4% +0.0 0.15 ± 8% perf-profile.children.cycles-pp.__kmalloc_reserve
0.11 ± 3% +0.0 0.15 ± 7% perf-profile.children.cycles-pp.__kmalloc_node_track_caller
0.36 ± 3% +0.0 0.41 ± 5% perf-profile.children.cycles-pp.sk_stream_alloc_skb
0.06 ± 6% +0.1 0.12 ± 13% perf-profile.children.cycles-pp.alloc_slab_page
0.06 ± 7% +0.1 0.13 ± 11% perf-profile.children.cycles-pp.new_slab
0.06 ± 7% +0.1 0.13 ± 11% perf-profile.children.cycles-pp.allocate_slab
0.16 ± 2% +0.1 0.23 ± 3% perf-profile.children.cycles-pp.skb_try_coalesce
0.09 ± 4% +0.1 0.16 ± 9% perf-profile.children.cycles-pp.___slab_alloc
0.06 +0.1 0.13 ± 10% perf-profile.children.cycles-pp.put_cpu_partial
0.17 ± 2% +0.1 0.24 ± 2% perf-profile.children.cycles-pp.tcp_try_coalesce
0.06 ± 8% +0.1 0.13 ± 9% perf-profile.children.cycles-pp.unfreeze_partials
0.22 ± 2% +0.1 0.29 ± 2% perf-profile.children.cycles-pp.tcp_queue_rcv
0.04 ± 51% +0.1 0.13 ± 14% perf-profile.children.cycles-pp.native_irq_return_iret
0.20 ± 20% +0.1 0.34 ± 9% perf-profile.children.cycles-pp.task_tick_fair
0.75 ± 9% +0.2 0.90 ± 6% perf-profile.children.cycles-pp.smp_apic_timer_interrupt
0.26 ± 19% +0.2 0.42 ± 7% perf-profile.children.cycles-pp.scheduler_tick
0.33 ± 12% +0.2 0.49 ± 7% perf-profile.children.cycles-pp.update_process_times
0.36 ± 12% +0.2 0.53 ± 6% perf-profile.children.cycles-pp.tick_sched_timer
0.33 ± 12% +0.2 0.50 ± 7% perf-profile.children.cycles-pp.tick_sched_handle
0.45 ± 8% +0.2 0.64 ± 7% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.62 ± 10% +0.2 0.81 ± 6% perf-profile.children.cycles-pp.hrtimer_interrupt
1.04 ± 12% +0.4 1.43 ± 4% perf-profile.children.cycles-pp.apic_timer_interrupt
15.41 ± 2% +0.6 16.01 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
8.23 ± 2% +1.1 9.35 ± 2% perf-profile.children.cycles-pp._copy_to_iter
8.63 ± 2% +1.1 9.76 ± 2% perf-profile.children.cycles-pp.skb_copy_datagram_iter
8.19 ± 2% +1.1 9.32 ± 2% perf-profile.children.cycles-pp.copyout
8.62 ± 2% +1.1 9.75 ± 2% perf-profile.children.cycles-pp.__skb_datagram_iter
36.35 +7.5 43.87 ± 4% perf-profile.children.cycles-pp.tcp_sendmsg_locked
25.15 +8.2 33.34 ± 5% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
26.91 +8.3 35.25 ± 5% perf-profile.children.cycles-pp.skb_page_frag_refill
26.93 +8.3 35.27 ± 5% perf-profile.children.cycles-pp.sk_page_frag_refill
26.13 +8.4 34.51 ± 5% perf-profile.children.cycles-pp.rmqueue
26.82 +8.4 35.26 ± 5% perf-profile.children.cycles-pp.get_page_from_freelist
26.89 +8.4 35.33 ± 5% perf-profile.children.cycles-pp.__alloc_pages_nodemask
24.99 +8.8 33.83 ± 4% perf-profile.children.cycles-pp.free_one_page
24.77 +8.8 33.61 ± 4% perf-profile.children.cycles-pp._raw_spin_lock
26.43 +9.1 35.56 ± 4% perf-profile.children.cycles-pp.skb_release_data
25.55 +9.1 34.69 ± 4% perf-profile.children.cycles-pp.__free_pages_ok
26.76 +9.2 35.91 ± 4% perf-profile.children.cycles-pp.__kfree_skb
76.67 +16.3 92.96 ± 3% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
76.65 +16.3 92.95 ± 3% perf-profile.children.cycles-pp.do_syscall_64
48.94 +17.0 65.94 ± 5% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
24.96 ± 2% +17.7 42.70 ± 7% perf-profile.children.cycles-pp.__x64_sys_recvfrom
24.95 ± 2% +17.7 42.70 ± 7% perf-profile.children.cycles-pp.__sys_recvfrom
24.89 ± 2% +17.8 42.68 ± 7% perf-profile.children.cycles-pp.inet_recvmsg
24.89 ± 2% +17.8 42.67 ± 7% perf-profile.children.cycles-pp.tcp_recvmsg
22.07 -15.4 6.66 ± 44% perf-profile.self.cycles-pp.intel_idle
0.17 ± 27% -0.1 0.08 ± 20% perf-profile.self.cycles-pp.update_cfs_group
0.33 ± 2% -0.1 0.24 ± 6% perf-profile.self.cycles-pp.__list_del_entry_valid
0.21 ± 3% -0.1 0.13 ± 12% perf-profile.self.cycles-pp.__tcp_transmit_skb
0.11 ± 8% -0.1 0.04 ± 83% perf-profile.self.cycles-pp.__switch_to
0.13 ± 5% -0.1 0.08 ± 14% perf-profile.self.cycles-pp.tcp_recvmsg
0.08 ± 10% -0.0 0.03 ± 82% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.07 ± 5% -0.0 0.03 ± 81% perf-profile.self.cycles-pp.__ksize
0.13 ± 3% -0.0 0.10 ± 7% perf-profile.self.cycles-pp.tcp_clean_rtx_queue
0.21 ± 2% -0.0 0.18 ± 6% perf-profile.self.cycles-pp.__free_one_page
0.11 ± 13% -0.0 0.08 ± 9% perf-profile.self.cycles-pp.__inet_lookup_established
0.10 ± 8% -0.0 0.07 ± 7% perf-profile.self.cycles-pp.__list_add_valid
0.08 ± 9% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.tcp_v4_rcv
0.11 ± 9% -0.0 0.09 ± 8% perf-profile.self.cycles-pp._raw_spin_lock_bh
0.08 ± 4% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.__skb_clone
0.16 ± 3% -0.0 0.14 ± 6% perf-profile.self.cycles-pp.__ip_queue_xmit
0.12 ± 3% -0.0 0.10 ± 4% perf-profile.self.cycles-pp.__alloc_skb
0.19 +0.0 0.21 ± 3% perf-profile.self.cycles-pp.__slab_free
0.11 ± 6% +0.0 0.13 ± 5% perf-profile.self.cycles-pp.prep_new_page
0.09 ± 5% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.__mod_zone_page_state
0.27 +0.0 0.30 ± 3% perf-profile.self.cycles-pp.__check_object_size
0.11 ± 4% +0.0 0.15 ± 5% perf-profile.self.cycles-pp.skb_clone
0.90 +0.0 0.94 ± 3% perf-profile.self.cycles-pp.skb_release_data
0.00 +0.1 0.07 ± 17% perf-profile.self.cycles-pp.task_tick_fair
0.16 ± 2% +0.1 0.23 ± 2% perf-profile.self.cycles-pp.skb_try_coalesce
0.04 ± 51% +0.1 0.13 ± 14% perf-profile.self.cycles-pp.native_irq_return_iret
0.33 ± 5% +0.1 0.46 ± 5% perf-profile.self.cycles-pp.__free_pages_ok
15.30 ± 2% +0.6 15.89 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
48.94 +17.0 65.94 ± 5% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
2411060 ± 11% -17.0% 2001744 ± 6% softirqs.CPU0.NET_RX
2718610 ± 11% -37.5% 1698432 ± 8% softirqs.CPU10.NET_RX
2668197 ± 9% -36.9% 1683516 ± 17% softirqs.CPU100.NET_RX
55543 ± 7% -50.3% 27591 ± 36% softirqs.CPU100.SCHED
2531285 ± 9% -38.1% 1565973 ± 12% softirqs.CPU101.NET_RX
58736 ± 13% -55.5% 26130 ± 13% softirqs.CPU101.SCHED
2418334 ± 13% -27.0% 1765607 ± 15% softirqs.CPU102.NET_RX
53198 ± 15% -50.3% 26416 ± 20% softirqs.CPU102.SCHED
2606464 ± 9% -32.2% 1766378 ± 10% softirqs.CPU103.NET_RX
12406 ± 4% -7.9% 11428 ± 2% softirqs.CPU103.RCU
51062 ± 9% -41.0% 30107 ± 35% softirqs.CPU103.SCHED
2493152 ± 9% -41.8% 1450692 ± 20% softirqs.CPU104.NET_RX
53419 ± 11% -48.2% 27660 ± 47% softirqs.CPU104.SCHED
2480381 ± 9% -48.6% 1273839 ± 40% softirqs.CPU105.NET_RX
12399 ± 4% -10.0% 11163 ± 3% softirqs.CPU105.RCU
53177 ± 12% -64.1% 19113 ± 53% softirqs.CPU105.SCHED
2568408 ± 10% -34.3% 1687652 ± 20% softirqs.CPU106.NET_RX
2449975 ± 10% -39.4% 1483795 ± 15% softirqs.CPU107.NET_RX
48717 ± 12% -61.8% 18588 ± 16% softirqs.CPU107.SCHED
2465363 ± 11% -49.6% 1241743 ± 40% softirqs.CPU108.NET_RX
12381 ± 5% -9.2% 11237 ± 3% softirqs.CPU108.RCU
46096 ± 11% -58.0% 19358 ± 52% softirqs.CPU108.SCHED
2273723 ± 7% -47.0% 1205867 ± 28% softirqs.CPU109.NET_RX
12299 ± 4% -9.0% 11198 ± 2% softirqs.CPU109.RCU
48456 ± 18% -67.7% 15656 ± 39% softirqs.CPU109.SCHED
2665035 ± 8% -39.7% 1607028 ± 11% softirqs.CPU11.NET_RX
2402488 ± 8% -46.9% 1274744 ± 22% softirqs.CPU110.NET_RX
48708 ± 10% -55.3% 21768 ± 23% softirqs.CPU110.SCHED
2294041 ± 9% -40.9% 1355094 ± 23% softirqs.CPU111.NET_RX
49685 ± 11% -57.8% 20983 ± 43% softirqs.CPU111.SCHED
2359170 ± 9% -35.3% 1526656 ± 17% softirqs.CPU112.NET_RX
12723 ± 3% -9.4% 11526 ± 3% softirqs.CPU112.RCU
46602 ± 12% -58.6% 19273 ± 42% softirqs.CPU112.SCHED
2340351 ± 8% -35.8% 1501914 ± 20% softirqs.CPU113.NET_RX
50881 ± 11% -52.5% 24187 ± 49% softirqs.CPU113.SCHED
2256188 ± 10% -32.8% 1515043 ± 24% softirqs.CPU114.NET_RX
12786 ± 3% -10.6% 11428 ± 2% softirqs.CPU114.RCU
47771 ± 7% -47.1% 25263 ± 50% softirqs.CPU114.SCHED
2270439 ± 7% -43.9% 1274147 ± 25% softirqs.CPU115.NET_RX
47755 ± 8% -60.1% 19048 ± 50% softirqs.CPU115.SCHED
2301318 ± 10% -29.8% 1614857 ± 18% softirqs.CPU116.NET_RX
44792 ± 17% -50.2% 22306 ± 20% softirqs.CPU116.SCHED
2148563 ± 11% -52.7% 1016630 ± 27% softirqs.CPU117.NET_RX
12487 ± 2% -9.3% 11322 ± 2% softirqs.CPU117.RCU
46927 ± 10% -65.7% 16097 ± 30% softirqs.CPU117.SCHED
2051462 ± 7% -36.9% 1293612 ± 36% softirqs.CPU118.NET_RX
42919 ± 10% -64.4% 15274 ± 29% softirqs.CPU118.SCHED
2051750 ± 8% -32.4% 1386457 ± 22% softirqs.CPU119.NET_RX
39861 ± 6% -50.3% 19821 ± 23% softirqs.CPU119.SCHED
2863159 ± 10% -48.2% 1484251 ± 13% softirqs.CPU12.NET_RX
35633 ± 14% -40.8% 21081 ± 31% softirqs.CPU12.SCHED
3570263 ± 17% -54.2% 1635090 ± 6% softirqs.CPU120.NET_RX
64626 ± 8% -58.7% 26662 ± 26% softirqs.CPU120.SCHED
3702963 ± 17% -59.9% 1484709 ± 17% softirqs.CPU121.NET_RX
64426 ± 9% -61.3% 24913 ± 37% softirqs.CPU121.SCHED
3530917 ± 17% -54.1% 1622018 ± 11% softirqs.CPU122.NET_RX
59735 ± 10% -58.8% 24608 ± 26% softirqs.CPU122.SCHED
3562489 ± 20% -60.7% 1398515 ± 22% softirqs.CPU123.NET_RX
62125 ± 7% -61.1% 24139 ± 35% softirqs.CPU123.SCHED
3394123 ± 16% -50.6% 1678076 ± 16% softirqs.CPU124.NET_RX
59949 ± 9% -48.9% 30638 ± 18% softirqs.CPU124.SCHED
3315251 ± 18% -52.8% 1563380 ± 20% softirqs.CPU125.NET_RX
57125 ± 8% -45.2% 31315 ± 42% softirqs.CPU125.SCHED
3336327 ± 17% -58.0% 1402064 ± 20% softirqs.CPU126.NET_RX
60326 ± 12% -59.2% 24601 ± 21% softirqs.CPU126.SCHED
3355759 ± 20% -56.1% 1471860 ± 24% softirqs.CPU127.NET_RX
60251 ± 8% -53.4% 28094 ± 34% softirqs.CPU127.SCHED
3067020 ± 12% -58.0% 1288984 ± 24% softirqs.CPU128.NET_RX
56485 ± 16% -59.8% 22700 ± 26% softirqs.CPU128.SCHED
3137842 ± 15% -57.0% 1349344 ± 33% softirqs.CPU129.NET_RX
59946 ± 10% -60.5% 23697 ± 48% softirqs.CPU129.SCHED
2857070 ± 8% -45.3% 1562616 ± 20% softirqs.CPU13.NET_RX
3019596 ± 23% -58.6% 1249743 ± 19% softirqs.CPU130.NET_RX
55289 ± 14% -50.3% 27497 ± 40% softirqs.CPU130.SCHED
3156549 ± 22% -59.9% 1265006 ± 21% softirqs.CPU131.NET_RX
55181 ± 12% -56.0% 24264 ± 30% softirqs.CPU131.SCHED
3040632 ± 24% -58.8% 1252943 ± 29% softirqs.CPU132.NET_RX
53604 ± 9% -64.8% 18845 ± 37% softirqs.CPU132.SCHED
2941914 ± 19% -48.4% 1519254 ± 15% softirqs.CPU133.NET_RX
51831 ± 12% -52.8% 24441 ± 11% softirqs.CPU133.SCHED
3004762 ± 16% -50.0% 1501441 ± 28% softirqs.CPU134.NET_RX
53490 ± 8% -58.3% 22283 ± 32% softirqs.CPU134.SCHED
3036502 ± 16% -61.9% 1155804 ± 23% softirqs.CPU135.NET_RX
53052 ± 7% -55.5% 23632 ± 38% softirqs.CPU135.SCHED
3058234 ± 15% -48.1% 1587797 ± 22% softirqs.CPU136.NET_RX
52750 ± 13% -55.9% 23276 ± 26% softirqs.CPU136.SCHED
2834945 ± 21% -57.8% 1195389 ± 30% softirqs.CPU137.NET_RX
53108 ± 5% -67.9% 17056 ± 50% softirqs.CPU137.SCHED
2664120 ± 22% -46.5% 1424697 ± 38% softirqs.CPU138.NET_RX
46323 ± 19% -54.5% 21075 ± 39% softirqs.CPU138.SCHED
2610691 ± 23% -49.8% 1310524 ± 17% softirqs.CPU139.NET_RX
47434 ± 10% -54.0% 21820 ± 20% softirqs.CPU139.SCHED
2806472 ± 11% -49.1% 1428968 ± 15% softirqs.CPU14.NET_RX
32233 ± 11% -38.8% 19727 ± 11% softirqs.CPU14.SCHED
2505133 ± 12% -44.3% 1396093 ± 15% softirqs.CPU140.NET_RX
45572 ± 9% -51.9% 21928 ± 19% softirqs.CPU140.SCHED
2393157 ± 12% -52.9% 1127286 ± 28% softirqs.CPU141.NET_RX
44422 ± 7% -58.4% 18491 ± 28% softirqs.CPU141.SCHED
2020765 ± 13% -41.5% 1182979 ± 30% softirqs.CPU142.NET_RX
36011 ± 18% -49.0% 18360 ± 36% softirqs.CPU142.SCHED
2104843 ± 7% -35.5% 1357127 ± 24% softirqs.CPU143.NET_RX
3276208 ± 14% -62.9% 1215092 ± 24% softirqs.CPU144.NET_RX
62909 ± 12% -67.4% 20530 ± 50% softirqs.CPU144.SCHED
3287089 ± 11% -49.0% 1677726 ± 7% softirqs.CPU145.NET_RX
61959 ± 5% -53.3% 28959 ± 29% softirqs.CPU145.SCHED
3078579 ± 10% -54.6% 1397375 ± 17% softirqs.CPU146.NET_RX
59372 ± 7% -60.6% 23390 ± 27% softirqs.CPU146.SCHED
3099385 ± 16% -56.2% 1356855 ± 32% softirqs.CPU147.NET_RX
55073 ± 8% -62.1% 20887 ± 42% softirqs.CPU147.SCHED
3058536 ± 14% -57.4% 1303914 ± 19% softirqs.CPU148.NET_RX
57666 ± 11% -59.4% 23388 ± 36% softirqs.CPU148.SCHED
2797210 ± 15% -44.9% 1541878 ± 14% softirqs.CPU149.NET_RX
54769 ± 5% -48.0% 28475 ± 36% softirqs.CPU149.SCHED
2753938 ± 13% -46.4% 1476205 ± 26% softirqs.CPU15.NET_RX
2773837 ± 8% -49.8% 1392482 ± 25% softirqs.CPU150.NET_RX
56481 ± 11% -57.9% 23792 ± 43% softirqs.CPU150.SCHED
2913248 ± 12% -57.9% 1227126 ± 20% softirqs.CPU151.NET_RX
54119 ± 6% -67.3% 17686 ± 18% softirqs.CPU151.SCHED
2962237 ± 14% -55.4% 1322170 ± 23% softirqs.CPU152.NET_RX
57848 ± 12% -55.2% 25899 ± 46% softirqs.CPU152.SCHED
2824322 ± 8% -49.7% 1421240 ± 16% softirqs.CPU153.NET_RX
57214 ± 5% -55.0% 25730 ± 23% softirqs.CPU153.SCHED
3017257 ± 14% -59.8% 1212020 ± 31% softirqs.CPU154.NET_RX
54493 ± 10% -65.8% 18622 ± 37% softirqs.CPU154.SCHED
2808497 ± 10% -53.7% 1299347 ± 32% softirqs.CPU155.NET_RX
55357 ± 9% -58.1% 23189 ± 31% softirqs.CPU155.SCHED
2864825 ± 15% -47.3% 1510474 ± 23% softirqs.CPU156.NET_RX
52853 ± 13% -52.1% 25309 ± 48% softirqs.CPU156.SCHED
2822646 ± 14% -54.4% 1287032 ± 26% softirqs.CPU157.NET_RX
55730 ± 10% -60.7% 21925 ± 62% softirqs.CPU157.SCHED
2509321 ± 9% -41.5% 1467265 ± 33% softirqs.CPU158.NET_RX
55135 ± 11% -64.5% 19590 ± 29% softirqs.CPU158.SCHED
2638967 ± 14% -40.1% 1580103 ± 16% softirqs.CPU159.NET_RX
56293 ± 8% -49.9% 28192 ± 31% softirqs.CPU159.SCHED
2695457 ± 15% -48.0% 1401454 ± 13% softirqs.CPU16.NET_RX
31877 ± 14% -31.0% 21980 ± 23% softirqs.CPU16.SCHED
2330445 ± 8% -42.0% 1352543 ± 13% softirqs.CPU160.NET_RX
50654 ± 8% -53.1% 23760 ± 15% softirqs.CPU160.SCHED
2659502 ± 12% -48.1% 1380199 ± 22% softirqs.CPU161.NET_RX
47876 ± 6% -58.2% 20010 ± 39% softirqs.CPU161.SCHED
2477864 ± 11% -37.5% 1549170 ± 15% softirqs.CPU162.NET_RX
45515 ± 17% -45.6% 24752 ± 36% softirqs.CPU162.SCHED
2382583 ± 6% -30.8% 1649222 ± 18% softirqs.CPU163.NET_RX
50905 ± 11% -50.0% 25461 ± 39% softirqs.CPU163.SCHED
45620 ± 11% -50.7% 22490 ± 25% softirqs.CPU164.SCHED
2003025 ± 16% -38.9% 1224234 ± 21% softirqs.CPU165.NET_RX
38487 ± 12% -45.7% 20897 ± 33% softirqs.CPU165.SCHED
2029079 ± 20% -36.7% 1283536 ± 11% softirqs.CPU167.NET_RX
37271 ± 11% -36.3% 23745 ± 18% softirqs.CPU167.SCHED
3788471 ± 46% -58.0% 1592793 ± 13% softirqs.CPU168.NET_RX
64500 ± 9% -58.1% 27032 ± 14% softirqs.CPU168.SCHED
4160271 ± 58% -59.3% 1693815 ± 21% softirqs.CPU169.NET_RX
65543 ± 12% -54.3% 29953 ± 44% softirqs.CPU169.SCHED
2704755 ± 9% -39.3% 1640754 ± 16% softirqs.CPU17.NET_RX
4010274 ± 54% -61.1% 1561586 ± 35% softirqs.CPU170.NET_RX
65461 ± 15% -64.6% 23155 ± 40% softirqs.CPU170.SCHED
4005888 ± 56% -53.5% 1863617 ± 17% softirqs.CPU171.NET_RX
61007 ± 8% -52.1% 29231 ± 18% softirqs.CPU171.SCHED
3794731 ± 58% -56.2% 1663773 ± 29% softirqs.CPU172.NET_RX
60746 ± 20% -53.5% 28242 ± 51% softirqs.CPU172.SCHED
3830799 ± 53% -63.3% 1404468 ± 39% softirqs.CPU173.NET_RX
54674 ± 20% -65.0% 19126 ± 49% softirqs.CPU173.SCHED
3538477 ± 42% -52.8% 1668621 ± 15% softirqs.CPU174.NET_RX
55423 ± 9% -55.9% 24447 ± 50% softirqs.CPU174.SCHED
57534 ± 7% -60.0% 22985 ± 45% softirqs.CPU175.SCHED
3887126 ± 65% -55.5% 1727897 ± 27% softirqs.CPU176.NET_RX
57205 ± 18% -44.7% 31630 ± 36% softirqs.CPU176.SCHED
3714857 ± 57% -49.2% 1887234 ± 22% softirqs.CPU177.NET_RX
53869 ± 20% -52.6% 25530 ± 32% softirqs.CPU177.SCHED
3795497 ± 55% -63.1% 1401327 ± 35% softirqs.CPU178.NET_RX
11681 ± 2% -8.1% 10736 ± 4% softirqs.CPU178.RCU
55747 ± 19% -60.3% 22144 ± 45% softirqs.CPU178.SCHED
3395540 ± 44% -54.3% 1552463 ± 33% softirqs.CPU179.NET_RX
52609 ± 15% -48.1% 27319 ± 42% softirqs.CPU179.SCHED
2752010 ± 11% -40.6% 1634235 ± 22% softirqs.CPU18.NET_RX
33098 ± 14% -32.8% 22230 ± 32% softirqs.CPU18.SCHED
3390488 ± 47% -60.0% 1356411 ± 38% softirqs.CPU180.NET_RX
61426 ± 14% -58.8% 25303 ± 52% softirqs.CPU180.SCHED
58005 ± 13% -52.8% 27403 ± 42% softirqs.CPU181.SCHED
3456590 ± 52% -49.9% 1732729 ± 17% softirqs.CPU182.NET_RX
3186026 ± 47% -58.2% 1332986 ± 20% softirqs.CPU183.NET_RX
53111 ± 13% -52.3% 25319 ± 37% softirqs.CPU183.SCHED
49775 ± 5% -63.8% 18007 ± 40% softirqs.CPU184.SCHED
3305651 ± 51% -63.5% 1205891 ± 49% softirqs.CPU185.NET_RX
11475 -8.2% 10528 softirqs.CPU185.RCU
48770 ± 14% -66.5% 16357 ± 57% softirqs.CPU185.SCHED
3150235 ± 44% -48.0% 1638477 ± 13% softirqs.CPU186.NET_RX
48194 ± 19% -40.9% 28494 ± 28% softirqs.CPU186.SCHED
2753794 ± 35% -55.6% 1223044 ± 28% softirqs.CPU187.NET_RX
48711 ± 12% -52.0% 23387 ± 46% softirqs.CPU187.SCHED
53989 ± 16% -60.2% 21466 ± 31% softirqs.CPU188.SCHED
2384076 ± 28% -40.8% 1412505 ± 19% softirqs.CPU189.NET_RX
42698 ± 22% -46.4% 22902 ± 44% softirqs.CPU189.SCHED
2686332 ± 8% -46.4% 1441162 ± 36% softirqs.CPU19.NET_RX
32263 ± 13% -38.1% 19986 ± 36% softirqs.CPU19.SCHED
2573552 ± 32% -35.3% 1664384 ± 20% softirqs.CPU190.NET_RX
2641313 ± 36% -30.6% 1833708 ± 8% softirqs.CPU191.NET_RX
2504611 ± 8% -41.0% 1477849 ± 18% softirqs.CPU2.NET_RX
2845833 ± 9% -41.6% 1663141 ± 14% softirqs.CPU20.NET_RX
36451 ± 18% -45.1% 20023 ± 31% softirqs.CPU20.SCHED
2659091 ± 9% -49.7% 1337253 ± 18% softirqs.CPU21.NET_RX
34403 ± 16% -55.6% 15274 ± 24% softirqs.CPU21.SCHED
2743681 ± 9% -46.0% 1480961 ± 26% softirqs.CPU22.NET_RX
36228 ± 16% -47.7% 18943 ± 26% softirqs.CPU22.SCHED
2679441 ± 12% -39.4% 1625081 ± 11% softirqs.CPU23.NET_RX
35508 ± 19% -25.9% 26299 ± 19% softirqs.CPU23.SCHED
3079975 ± 21% -43.4% 1742003 ± 6% softirqs.CPU24.NET_RX
3147505 ± 25% -55.7% 1394293 ± 16% softirqs.CPU25.NET_RX
27604 ± 11% -34.8% 17997 ± 25% softirqs.CPU25.SCHED
3142164 ± 19% -44.0% 1759559 ± 9% softirqs.CPU26.NET_RX
3139990 ± 25% -55.5% 1397428 ± 25% softirqs.CPU27.NET_RX
3423302 ± 24% -52.3% 1633571 ± 21% softirqs.CPU28.NET_RX
3183269 ± 18% -48.2% 1648689 ± 22% softirqs.CPU29.NET_RX
32549 ± 18% -30.2% 22733 ± 22% softirqs.CPU29.SCHED
2360622 ± 7% -36.9% 1490348 ± 28% softirqs.CPU3.NET_RX
3341152 ± 14% -55.1% 1500296 ± 14% softirqs.CPU30.NET_RX
29104 ± 21% -22.7% 22486 ± 8% softirqs.CPU30.SCHED
3265019 ± 15% -49.5% 1649112 ± 21% softirqs.CPU31.NET_RX
3477261 ± 11% -60.8% 1364031 ± 19% softirqs.CPU32.NET_RX
3477803 ± 15% -59.0% 1426551 ± 25% softirqs.CPU33.NET_RX
28698 ± 13% -38.2% 17743 ± 31% softirqs.CPU33.SCHED
3454786 ± 21% -55.8% 1527719 ± 30% softirqs.CPU34.NET_RX
3187114 ± 17% -59.1% 1303766 ± 30% softirqs.CPU35.NET_RX
32320 ± 8% -36.3% 20591 ± 44% softirqs.CPU35.SCHED
3826701 ± 17% -68.9% 1189508 ± 19% softirqs.CPU36.NET_RX
35974 ± 5% -46.3% 19331 ± 34% softirqs.CPU36.SCHED
3493579 ± 17% -56.6% 1516418 ± 15% softirqs.CPU37.NET_RX
33361 ± 13% -30.8% 23072 ± 22% softirqs.CPU37.SCHED
3520932 ± 16% -59.7% 1418104 ± 36% softirqs.CPU38.NET_RX
35750 ± 5% -36.2% 22801 ± 38% softirqs.CPU38.SCHED
3565998 ± 12% -61.2% 1385115 ± 18% softirqs.CPU39.NET_RX
2482668 ± 9% -40.3% 1482219 ± 19% softirqs.CPU4.NET_RX
3626850 ± 21% -60.4% 1437274 ± 27% softirqs.CPU40.NET_RX
3214798 ± 13% -56.9% 1384251 ± 22% softirqs.CPU41.NET_RX
35949 ± 11% -38.8% 21996 ± 30% softirqs.CPU41.SCHED
3589512 ± 16% -60.8% 1406472 ± 36% softirqs.CPU42.NET_RX
39104 ± 19% -47.8% 20421 ± 41% softirqs.CPU42.SCHED
3501848 ± 16% -58.4% 1455130 ± 21% softirqs.CPU43.NET_RX
38844 ± 9% -54.1% 17814 ± 17% softirqs.CPU43.SCHED
3638528 ± 21% -65.0% 1272753 ± 25% softirqs.CPU44.NET_RX
41268 ± 12% -52.8% 19458 ± 31% softirqs.CPU44.SCHED
3459596 ± 16% -68.4% 1092443 ± 17% softirqs.CPU45.NET_RX
40479 ± 17% -63.4% 14820 ± 15% softirqs.CPU45.SCHED
3411025 ± 20% -54.3% 1558432 ± 16% softirqs.CPU46.NET_RX
44947 ± 20% -47.7% 23520 ± 19% softirqs.CPU46.SCHED
3482646 ± 16% -55.4% 1552959 ± 18% softirqs.CPU47.NET_RX
45972 ± 17% -41.2% 27049 ± 28% softirqs.CPU47.SCHED
2523850 ± 15% -48.7% 1293986 ± 33% softirqs.CPU48.NET_RX
2552666 ± 11% -35.1% 1655564 ± 14% softirqs.CPU49.NET_RX
2571090 ± 10% -40.1% 1539608 ± 14% softirqs.CPU5.NET_RX
2581711 ± 13% -51.6% 1250613 ± 19% softirqs.CPU50.NET_RX
2884667 ± 11% -47.1% 1525563 ± 31% softirqs.CPU51.NET_RX
3079250 ± 12% -52.2% 1473165 ± 11% softirqs.CPU52.NET_RX
2840747 ± 21% -44.5% 1576170 ± 8% softirqs.CPU53.NET_RX
2921278 ± 14% -52.0% 1401696 ± 16% softirqs.CPU54.NET_RX
3169937 ± 17% -60.8% 1241156 ± 25% softirqs.CPU55.NET_RX
36307 ± 8% -52.5% 17262 ± 38% softirqs.CPU55.SCHED
3056456 ± 6% -53.4% 1422908 ± 25% softirqs.CPU56.NET_RX
31795 ± 10% -44.7% 17568 ± 29% softirqs.CPU56.SCHED
3141468 ± 13% -52.7% 1487422 ± 9% softirqs.CPU57.NET_RX
3221467 ± 8% -56.5% 1402480 ± 25% softirqs.CPU58.NET_RX
3502440 ± 14% -60.5% 1385062 ± 27% softirqs.CPU59.NET_RX
2633501 ± 8% -37.2% 1653750 ± 21% softirqs.CPU6.NET_RX
3262285 ± 10% -55.1% 1463693 ± 26% softirqs.CPU60.NET_RX
3039300 ± 5% -51.3% 1479816 ± 22% softirqs.CPU61.NET_RX
2817826 ± 17% -51.1% 1376685 ± 25% softirqs.CPU62.NET_RX
3057336 ± 13% -50.4% 1516165 ± 18% softirqs.CPU63.NET_RX
3067351 ± 11% -53.7% 1421032 ± 20% softirqs.CPU64.NET_RX
34915 ± 7% -22.2% 27175 ± 15% softirqs.CPU64.SCHED
3255527 ± 11% -58.0% 1368668 ± 25% softirqs.CPU65.NET_RX
3245829 ± 14% -50.6% 1602539 ± 15% softirqs.CPU66.NET_RX
43367 ± 10% -40.4% 25864 ± 25% softirqs.CPU66.SCHED
3054422 ± 17% -45.0% 1681270 ± 14% softirqs.CPU67.NET_RX
3024427 ± 10% -55.6% 1343967 ± 17% softirqs.CPU68.NET_RX
3359067 ± 14% -60.1% 1340486 ± 20% softirqs.CPU69.NET_RX
45511 ± 11% -44.8% 25131 ± 31% softirqs.CPU69.SCHED
2581762 ± 12% -30.1% 1804333 ± 13% softirqs.CPU7.NET_RX
3185973 ± 12% -56.5% 1384547 ± 8% softirqs.CPU70.NET_RX
44949 ± 6% -42.2% 25997 ± 29% softirqs.CPU70.SCHED
3100967 ± 12% -55.4% 1384032 ± 13% softirqs.CPU71.NET_RX
42515 ± 18% -49.1% 21654 ± 28% softirqs.CPU71.SCHED
3332116 ± 63% -50.5% 1649135 ± 22% softirqs.CPU72.NET_RX
3459284 ± 55% -46.1% 1865592 ± 16% softirqs.CPU73.NET_RX
3593017 ± 41% -54.6% 1630199 ± 31% softirqs.CPU74.NET_RX
3908371 ± 64% -52.2% 1866996 ± 17% softirqs.CPU75.NET_RX
3424152 ± 38% -58.6% 1416916 ± 28% softirqs.CPU76.NET_RX
3523798 ± 49% -68.5% 1110373 ± 34% softirqs.CPU77.NET_RX
3782170 ± 56% -59.1% 1547780 ± 22% softirqs.CPU78.NET_RX
3831396 ± 58% -60.5% 1511650 ± 40% softirqs.CPU79.NET_RX
2544287 ± 15% -32.8% 1709146 ± 14% softirqs.CPU8.NET_RX
3777653 ± 52% -57.6% 1603569 ± 40% softirqs.CPU80.NET_RX
4121863 ± 60% -58.1% 1727891 ± 23% softirqs.CPU81.NET_RX
38262 ± 10% -27.5% 27747 ± 17% softirqs.CPU81.SCHED
4008537 ± 53% -67.8% 1291794 ± 23% softirqs.CPU82.NET_RX
3925610 ± 57% -60.9% 1533230 ± 40% softirqs.CPU83.NET_RX
3675683 ± 56% -59.6% 1483266 ± 39% softirqs.CPU84.NET_RX
4013276 ± 59% -51.5% 1946438 ± 19% softirqs.CPU85.NET_RX
3980480 ± 59% -57.3% 1697784 ± 13% softirqs.CPU86.NET_RX
3903322 ± 48% -61.2% 1516172 ± 27% softirqs.CPU87.NET_RX
39408 ± 36% -44.2% 21980 ± 21% softirqs.CPU87.SCHED
3820578 ± 42% -68.5% 1204526 ± 47% softirqs.CPU89.NET_RX
41951 ± 21% -54.5% 19089 ± 62% softirqs.CPU89.SCHED
2771428 ± 7% -39.0% 1690687 ± 19% softirqs.CPU9.NET_RX
3937814 ± 53% -58.8% 1623879 ± 35% softirqs.CPU90.NET_RX
4063598 ± 54% -66.5% 1359492 ± 28% softirqs.CPU91.NET_RX
41868 ± 33% -46.8% 22255 ± 44% softirqs.CPU91.SCHED
3610446 ± 40% -55.8% 1597481 ± 47% softirqs.CPU92.NET_RX
4116107 ± 52% -65.1% 1434570 ± 33% softirqs.CPU93.NET_RX
48062 ± 36% -57.8% 20260 ± 47% softirqs.CPU93.SCHED
3649362 ± 39% -53.6% 1692783 ± 13% softirqs.CPU94.NET_RX
46932 ± 22% -40.0% 28178 ± 34% softirqs.CPU94.SCHED
3674896 ± 48% -53.8% 1695998 ± 19% softirqs.CPU95.NET_RX
2906346 ± 10% -36.3% 1850842 ± 12% softirqs.CPU96.NET_RX
47371 ± 17% -34.1% 31198 ± 25% softirqs.CPU96.SCHED
2823944 ± 6% -46.6% 1506891 ± 27% softirqs.CPU97.NET_RX
12566 -8.4% 11511 ± 2% softirqs.CPU97.RCU
58287 ± 16% -60.5% 23031 ± 34% softirqs.CPU97.SCHED
2778341 ± 8% -41.0% 1640505 ± 21% softirqs.CPU98.NET_RX
12108 ± 4% -5.9% 11391 ± 3% softirqs.CPU98.RCU
57815 ± 13% -49.7% 29081 ± 43% softirqs.CPU98.SCHED
2754057 ± 9% -43.6% 1553858 ± 22% softirqs.CPU99.NET_RX
55507 ± 4% -52.1% 26578 ± 38% softirqs.CPU99.SCHED
5.877e+08 ± 12% -51.4% 2.859e+08 ± 10% softirqs.NET_RX
8311241 ± 4% -44.9% 4578168 ± 15% softirqs.SCHED
netperf.Throughput_Mbps
4000 +--------------------------------------------------------------------+
3800 |-+ .+... |
| .... ... |
3600 |-+ ... .. |
3400 |-+ ... .. |
| .... ... |
3200 |................+.................+. .. |
3000 |-+ .|
2800 |-+ |
| |
2600 |-+ |
2400 |-+ |
| |
2200 |-+ O O O |
2000 +--------------------------------------------------------------------+
netperf.time.minor_page_faults
48000 +-------------------------------------------------------------------+
|........ |
46000 |-+ ...... |
44000 |-+ .. ..|
| +................ ...... |
42000 |-+ +................ ...... |
| +.. |
40000 |-+ |
| |
38000 |-+ |
36000 |-+ |
| |
34000 |-+ O |
| O O |
32000 +-------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-hsw-4ex1: 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-8.3/debian-10.4-x86_64-20200603.cgz/300s/8T/lkp-hsw-4ex1/anon-w-seq/vm-scalability/0x16
commit:
a349834703 ("sched/fair: Rename sg_lb_stats::sum_nr_running to sum_h_nr_running")
fcf0553db6 ("sched/fair: Remove meaningless imbalance calculation")
a349834703010183 fcf0553db6f4c79387864f6e4ab
---------------- ---------------------------
%stddev %change %stddev
\ | \
595857 +4.6% 623549 vm-scalability.median
8.22 ± 7% -3.2 5.01 ± 5% vm-scalability.median_stddev%
88608356 +1.5% 89976003 vm-scalability.throughput
498624 ± 2% -9.5% 451343 vm-scalability.time.involuntary_context_switches
9335 -1.3% 9212 vm-scalability.time.percent_of_cpu_this_job_got
12408855 ± 9% -17.1% 10291775 ± 8% meminfo.DirectMap2M
20311 ± 40% +60.3% 32564 ± 16% numa-numastat.node0.other_node
1759 ± 8% +62.0% 2850 ± 6% syscalls.sys_read.med
6827 ± 2% -2.8% 6636 vmstat.system.cs
12092 ± 18% -43.8% 6792 ± 30% numa-vmstat.node0.nr_slab_reclaimable
6233 ± 26% +29.4% 8069 ± 23% numa-vmstat.node3.nr_slab_reclaimable
14990 ± 12% +45.0% 21732 ± 18% numa-vmstat.node3.nr_slab_unreclaimable
48371 ± 18% -43.8% 27169 ± 30% numa-meminfo.node0.KReclaimable
48371 ± 18% -43.8% 27169 ± 30% numa-meminfo.node0.SReclaimable
167341 ± 16% -32.6% 112749 ± 31% numa-meminfo.node0.Slab
24931 ± 26% +29.5% 32277 ± 23% numa-meminfo.node3.KReclaimable
24931 ± 26% +29.5% 32277 ± 23% numa-meminfo.node3.SReclaimable
59961 ± 12% +45.0% 86933 ± 18% numa-meminfo.node3.SUnreclaim
84893 ± 15% +40.4% 119211 ± 13% numa-meminfo.node3.Slab
4627 ± 9% +25.4% 5802 ± 4% slabinfo.eventpoll_pwq.active_objs
4627 ± 9% +25.4% 5802 ± 4% slabinfo.eventpoll_pwq.num_objs
2190 ± 14% +26.1% 2762 ± 4% slabinfo.kmem_cache.active_objs
2190 ± 14% +26.1% 2762 ± 4% slabinfo.kmem_cache.num_objs
5823 ± 12% +22.2% 7118 ± 4% slabinfo.kmem_cache_node.active_objs
9665 ± 5% +10.3% 10663 ± 3% slabinfo.shmem_inode_cache.active_objs
9775 ± 5% +10.1% 10762 ± 3% slabinfo.shmem_inode_cache.num_objs
322.79 -7.9% 297.22 perf-stat.i.cpu-migrations
21.89 ± 3% -1.2 20.69 perf-stat.i.node-load-miss-rate%
13112232 ± 3% +3.4% 13562841 perf-stat.i.node-loads
21.44 ± 2% -1.1 20.29 perf-stat.overall.node-load-miss-rate%
3160 +1.9% 3219 perf-stat.overall.path-length
6748 ± 2% -2.6% 6570 perf-stat.ps.context-switches
319.09 -8.2% 292.97 perf-stat.ps.cpu-migrations
12905267 ± 3% +3.4% 13345546 perf-stat.ps.node-loads
12193 ± 5% +8.6% 13239 ± 4% softirqs.CPU105.RCU
12791 ± 6% +12.4% 14382 ± 7% softirqs.CPU22.RCU
12781 ± 5% +12.3% 14351 ± 4% softirqs.CPU28.RCU
12870 ± 4% +9.0% 14033 ± 7% softirqs.CPU31.RCU
11451 ± 5% +11.7% 12785 ± 4% softirqs.CPU90.RCU
11449 ± 6% +9.1% 12497 ± 6% softirqs.CPU91.RCU
11486 ± 6% +8.9% 12510 softirqs.CPU93.RCU
12197 ± 5% +10.4% 13462 ± 5% softirqs.CPU97.RCU
3899 ±100% -78.7% 830.89 ± 67% sched_debug.cfs_rq:/.load_avg.max
386.54 ± 91% -76.1% 92.43 ± 61% sched_debug.cfs_rq:/.load_avg.stddev
747.05 ± 7% -26.9% 546.12 ± 31% sched_debug.cfs_rq:/.runnable_load_avg.max
-203275 +540.7% -1302453 sched_debug.cfs_rq:/.spread0.avg
1142 ± 5% +9.6% 1251 ± 6% sched_debug.cfs_rq:/.util_avg.max
1.00 ±163% +2117.5% 22.17 ±100% sched_debug.cfs_rq:/.util_avg.min
212.47 ± 14% -16.9% 176.57 ± 4% sched_debug.cfs_rq:/.util_est_enqueued.stddev
0.00 ± 7% +226.7% 0.00 ± 18% sched_debug.cpu.next_balance.stddev
2.48 ± 20% -24.6% 1.87 ± 11% sched_debug.cpu.nr_running.max
0.64 ± 29% -49.1% 0.33 ± 11% sched_debug.cpu.nr_running.stddev
137.50 ± 54% +194.2% 404.50 ± 78% interrupts.CPU109.RES:Rescheduling_interrupts
822.50 ± 22% -39.4% 498.75 ± 49% interrupts.CPU114.CAL:Function_call_interrupts
126.00 ± 72% +161.1% 329.00 ± 48% interrupts.CPU117.RES:Rescheduling_interrupts
153.00 ± 69% +121.6% 339.00 ± 51% interrupts.CPU119.RES:Rescheduling_interrupts
162.75 ± 83% +170.8% 440.75 ± 36% interrupts.CPU121.RES:Rescheduling_interrupts
798.00 ± 30% -45.0% 438.75 ± 28% interrupts.CPU126.CAL:Function_call_interrupts
3831 ± 37% -96.4% 138.50 ±106% interrupts.CPU128.NMI:Non-maskable_interrupts
3831 ± 37% -96.4% 138.50 ±106% interrupts.CPU128.PMI:Performance_monitoring_interrupts
218.25 ± 44% -51.7% 105.50 ± 4% interrupts.CPU128.RES:Rescheduling_interrupts
453.00 ± 74% -74.3% 116.50 ± 26% interrupts.CPU130.RES:Rescheduling_interrupts
2382 ± 15% -96.6% 80.75 ±133% interrupts.CPU138.NMI:Non-maskable_interrupts
2382 ± 15% -96.6% 80.75 ±133% interrupts.CPU138.PMI:Performance_monitoring_interrupts
557.25 ±155% +341.2% 2458 ± 79% interrupts.CPU14.NMI:Non-maskable_interrupts
557.25 ±155% +341.2% 2458 ± 79% interrupts.CPU14.PMI:Performance_monitoring_interrupts
782.00 ± 31% -37.0% 492.50 ± 50% interrupts.CPU17.CAL:Function_call_interrupts
779.75 ± 31% -36.8% 492.50 ± 50% interrupts.CPU18.CAL:Function_call_interrupts
782.25 ± 31% -37.2% 491.00 ± 49% interrupts.CPU20.CAL:Function_call_interrupts
5855 ± 20% -75.7% 1420 ±106% interrupts.CPU23.NMI:Non-maskable_interrupts
5855 ± 20% -75.7% 1420 ±106% interrupts.CPU23.PMI:Performance_monitoring_interrupts
785.00 ± 30% -38.0% 487.00 ± 51% interrupts.CPU3.CAL:Function_call_interrupts
1266 ±166% +241.5% 4325 ± 38% interrupts.CPU34.NMI:Non-maskable_interrupts
1266 ±166% +241.5% 4325 ± 38% interrupts.CPU34.PMI:Performance_monitoring_interrupts
784.00 ± 30% -37.0% 493.75 ± 49% interrupts.CPU36.CAL:Function_call_interrupts
783.25 ± 31% -37.6% 489.00 ± 51% interrupts.CPU37.CAL:Function_call_interrupts
325.75 ± 11% +80.6% 588.25 ± 45% interrupts.CPU37.RES:Rescheduling_interrupts
782.00 ± 31% -37.0% 492.50 ± 50% interrupts.CPU39.CAL:Function_call_interrupts
781.50 ± 31% -38.0% 484.50 ± 52% interrupts.CPU41.CAL:Function_call_interrupts
292.50 ± 21% +59.2% 465.75 ± 26% interrupts.CPU43.RES:Rescheduling_interrupts
233.00 ± 12% +59.4% 371.50 ± 20% interrupts.CPU48.RES:Rescheduling_interrupts
883.00 ±160% -97.1% 25.25 ±150% interrupts.CPU50.NMI:Non-maskable_interrupts
883.00 ±160% -97.1% 25.25 ±150% interrupts.CPU50.PMI:Performance_monitoring_interrupts
265.25 ± 28% +69.8% 450.50 ± 27% interrupts.CPU50.RES:Rescheduling_interrupts
249.75 ± 19% +40.8% 351.75 ± 11% interrupts.CPU51.RES:Rescheduling_interrupts
785.50 ± 30% -37.2% 493.00 ± 49% interrupts.CPU53.CAL:Function_call_interrupts
1.75 ±116% +1.2e+05% 2020 ±108% interrupts.CPU53.NMI:Non-maskable_interrupts
1.75 ±116% +1.2e+05% 2020 ±108% interrupts.CPU53.PMI:Performance_monitoring_interrupts
826.75 ± 79% +429.5% 4377 ± 27% interrupts.CPU56.NMI:Non-maskable_interrupts
826.75 ± 79% +429.5% 4377 ± 27% interrupts.CPU56.PMI:Performance_monitoring_interrupts
782.50 ± 31% -37.6% 488.00 ± 51% interrupts.CPU57.CAL:Function_call_interrupts
32.25 ±164% -100.0% 0.00 interrupts.CPU57.TLB:TLB_shootdowns
782.50 ± 31% -36.9% 494.00 ± 50% interrupts.CPU6.CAL:Function_call_interrupts
781.00 ± 31% -36.8% 493.50 ± 50% interrupts.CPU61.CAL:Function_call_interrupts
658.75 ±169% +429.9% 3490 ± 43% interrupts.CPU61.NMI:Non-maskable_interrupts
658.75 ±169% +429.9% 3490 ± 43% interrupts.CPU61.PMI:Performance_monitoring_interrupts
405.75 ± 20% -35.6% 261.50 ± 17% interrupts.CPU62.RES:Rescheduling_interrupts
781.75 ± 31% -36.8% 494.00 ± 50% interrupts.CPU65.CAL:Function_call_interrupts
782.50 ± 31% -37.0% 492.75 ± 50% interrupts.CPU66.CAL:Function_call_interrupts
838.25 ±121% +513.8% 5145 ± 30% interrupts.CPU66.NMI:Non-maskable_interrupts
838.25 ±121% +513.8% 5145 ± 30% interrupts.CPU66.PMI:Performance_monitoring_interrupts
782.00 ± 31% -37.1% 492.25 ± 50% interrupts.CPU67.CAL:Function_call_interrupts
782.25 ± 31% -37.0% 492.75 ± 49% interrupts.CPU69.CAL:Function_call_interrupts
780.00 ± 31% -37.3% 489.25 ± 49% interrupts.CPU71.CAL:Function_call_interrupts
784.25 ± 31% -38.0% 486.50 ± 51% interrupts.CPU75.CAL:Function_call_interrupts
783.25 ± 31% -36.9% 494.25 ± 50% interrupts.CPU77.CAL:Function_call_interrupts
822.00 ± 33% -39.9% 494.25 ± 50% interrupts.CPU83.CAL:Function_call_interrupts
789.75 ± 29% -37.1% 496.50 ± 49% interrupts.CPU90.CAL:Function_call_interrupts
5966 ± 15% -53.5% 2777 ± 62% interrupts.CPU94.NMI:Non-maskable_interrupts
5966 ± 15% -53.5% 2777 ± 62% interrupts.CPU94.PMI:Performance_monitoring_interrupts
46379 ± 9% -10.1% 41674 ± 4% interrupts.RES:Rescheduling_interrupts
68.78 -10.0 58.75 ± 8% perf-profile.calltrace.cycles-pp.do_access
30.04 ± 4% -4.3 25.69 ± 9% perf-profile.calltrace.cycles-pp.do_page_fault.page_fault.do_access
30.06 ± 4% -4.3 25.71 ± 9% perf-profile.calltrace.cycles-pp.page_fault.do_access
30.00 ± 4% -4.3 25.66 ± 9% perf-profile.calltrace.cycles-pp.do_user_addr_fault.do_page_fault.page_fault.do_access
29.93 ± 4% -4.3 25.59 ± 9% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.do_page_fault.page_fault.do_access
26.63 ± 2% -2.7 23.88 ± 6% perf-profile.calltrace.cycles-pp.do_rw_once
0.57 ± 5% -0.3 0.27 ±100% perf-profile.calltrace.cycles-pp.rmqueue.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.do_huge_pmd_anonymous_page
0.63 ± 6% -0.2 0.43 ± 58% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.alloc_pages_vma.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault
0.63 ± 6% -0.2 0.43 ± 58% perf-profile.calltrace.cycles-pp.alloc_pages_vma.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
0.62 ± 5% -0.2 0.42 ± 58% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.do_huge_pmd_anonymous_page.__handle_mm_fault
28.25 ± 5% +6.6 34.88 ± 6% perf-profile.calltrace.cycles-pp.clear_page_erms.clear_subpage.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault
29.30 ± 5% +6.7 36.03 ± 6% perf-profile.calltrace.cycles-pp.clear_subpage.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault
30.21 ± 5% +7.0 37.23 ± 6% perf-profile.calltrace.cycles-pp.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
31.31 ± 5% +7.3 38.61 ± 6% perf-profile.calltrace.cycles-pp.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.do_page_fault
31.39 ± 5% +7.3 38.70 ± 6% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.do_page_fault.page_fault
1.50 ±106% +11.7 13.17 ± 35% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.do_page_fault.page_fault
1.50 ±106% +11.7 13.20 ± 35% perf-profile.calltrace.cycles-pp.do_user_addr_fault.do_page_fault.page_fault
1.51 ±106% +11.7 13.21 ± 35% perf-profile.calltrace.cycles-pp.page_fault
1.50 ±106% +11.7 13.21 ± 35% perf-profile.calltrace.cycles-pp.do_page_fault.page_fault
65.44 -9.4 56.04 ± 8% perf-profile.children.cycles-pp.do_access
30.08 -3.4 26.70 ± 7% perf-profile.children.cycles-pp.do_rw_once
0.06 ± 11% +0.0 0.08 ± 10% perf-profile.children.cycles-pp.pagevec_lru_move_fn
0.14 ± 6% +0.0 0.16 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.13 ± 10% +0.0 0.16 ± 9% perf-profile.children.cycles-pp.pte_alloc_one
0.12 ± 12% +0.0 0.15 ± 8% perf-profile.children.cycles-pp.prep_new_page
0.12 ± 10% +0.0 0.16 ± 20% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.12 ± 9% +0.0 0.15 ± 10% perf-profile.children.cycles-pp.prepare_exit_to_usermode
0.14 ± 8% +0.0 0.19 ± 9% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.irq_work_interrupt
0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.smp_irq_work_interrupt
0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.irq_work_run
0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.printk
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.irq_work_run_list
0.00 +0.1 0.07 ± 31% perf-profile.children.cycles-pp.__intel_pmu_enable_all
0.35 ± 3% +0.1 0.43 ± 7% perf-profile.children.cycles-pp._cond_resched
0.38 ± 6% +0.1 0.48 ± 6% perf-profile.children.cycles-pp.___might_sleep
0.46 ± 8% +0.1 0.60 ± 13% perf-profile.children.cycles-pp.scheduler_tick
0.66 ± 8% +0.2 0.81 ± 8% perf-profile.children.cycles-pp.rmqueue
0.70 ± 8% +0.2 0.87 ± 8% perf-profile.children.cycles-pp.alloc_pages_vma
0.79 ± 9% +0.2 0.97 ± 8% perf-profile.children.cycles-pp.get_page_from_freelist
0.83 ± 8% +0.2 1.03 ± 7% perf-profile.children.cycles-pp.__alloc_pages_nodemask
1.09 ± 6% +0.3 1.38 ± 11% perf-profile.children.cycles-pp.__hrtimer_run_queues
2.11 ± 3% +0.5 2.63 ± 12% perf-profile.children.cycles-pp.apic_timer_interrupt
28.56 ± 5% +6.6 35.18 ± 6% perf-profile.children.cycles-pp.clear_page_erms
29.39 ± 5% +6.8 36.14 ± 6% perf-profile.children.cycles-pp.clear_subpage
30.34 ± 5% +7.0 37.30 ± 6% perf-profile.children.cycles-pp.clear_huge_page
31.39 ± 5% +7.2 38.61 ± 6% perf-profile.children.cycles-pp.do_huge_pmd_anonymous_page
31.47 ± 5% +7.2 38.72 ± 6% perf-profile.children.cycles-pp.__handle_mm_fault
31.52 ± 5% +7.3 38.77 ± 6% perf-profile.children.cycles-pp.handle_mm_fault
31.60 ± 5% +7.3 38.87 ± 6% perf-profile.children.cycles-pp.do_user_addr_fault
31.67 ± 5% +7.3 38.95 ± 6% perf-profile.children.cycles-pp.page_fault
31.63 ± 5% +7.3 38.91 ± 6% perf-profile.children.cycles-pp.do_page_fault
29.16 ± 4% -4.1 25.05 ± 7% perf-profile.self.cycles-pp.do_access
27.38 -3.1 24.28 ± 7% perf-profile.self.cycles-pp.do_rw_once
0.05 ± 8% +0.0 0.07 ± 12% perf-profile.self.cycles-pp.prep_new_page
0.09 ± 9% +0.0 0.12 ± 11% perf-profile.self.cycles-pp.prepare_exit_to_usermode
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.do_huge_pmd_anonymous_page
0.00 +0.1 0.07 ± 31% perf-profile.self.cycles-pp.__intel_pmu_enable_all
0.28 ± 3% +0.1 0.35 ± 7% perf-profile.self.cycles-pp._cond_resched
0.35 ± 4% +0.1 0.43 ± 6% perf-profile.self.cycles-pp.___might_sleep
0.48 ± 8% +0.1 0.60 ± 9% perf-profile.self.cycles-pp.rmqueue
0.88 ± 4% +0.2 1.11 ± 3% perf-profile.self.cycles-pp.clear_subpage
28.18 ± 5% +6.4 34.57 ± 6% perf-profile.self.cycles-pp.clear_page_erms
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
View attachment "config-5.4.0-rc1-00009-gfcf0553db6f4c" of type "text/plain" (152748 bytes)
View attachment "job-script" of type "text/plain" (7804 bytes)
View attachment "job.yaml" of type "text/plain" (5406 bytes)
View attachment "reproduce" of type "text/plain" (5459 bytes)
Powered by blists - more mailing lists