| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Tue, 22 Nov 2022 13:52:29 +0800
From: kernel test robot <yujie.liu@...el.com>
To: Andrei Vagin <avagin@...gle.com>
CC: <oe-lkp@...ts.linux.dev>, <lkp@...el.com>,
Ingo Molnar <mingo@...hat.com>,
Peter Zijlstra <peterz@...radead.org>,
Juri Lelli <juri.lelli@...hat.com>,
Vincent Guittot <vincent.guittot@...aro.org>,
Dietmar Eggemann <dietmar.eggemann@....com>,
Steven Rostedt <rostedt@...dmis.org>,
Ben Segall <bsegall@...gle.com>, Mel Gorman <mgorman@...e.de>,
"Daniel Bristot de Oliveira" <bristot@...hat.com>,
Valentin Schneider <vschneid@...hat.com>,
<linux-kernel@...r.kernel.org>, <ying.huang@...el.com>,
<feng.tang@...el.com>, <zhengjun.xing@...ux.intel.com>,
<fengwei.yin@...el.com>, Andrei Vagin <avagin@...il.com>,
Chen Yu <yu.c.chen@...el.com>
Subject: Re: [PATCH] sched: consider WF_SYNC to find idle siblings
Please be noted that we observed various performance changes in
different benchmarks. The impact is well discussed in the mail thread,
e.g. "strictly synchronous jobs may benefit, but netperf will get
punished". We are just sending this report to provide some additional
test data for your reference. Thanks.
Greeting,
FYI, we noticed a -66.0% regression of netperf.Throughput_Mbps due to commit:
commit: b6aabb01e3004b0285d953761e1592974b4b50fa ("[PATCH] sched: consider WF_SYNC to find idle siblings")
url: https://github.com/intel-lab-lkp/linux/commits/Andrei-Vagin/sched-consider-WF_SYNC-to-find-idle-siblings/20221028-042823
base: https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git c36eae5a86d15a694968be35d7ff056854997a72
patch link: https://lore.kernel.org/all/20221027202603.670616-1-avagin@google.com/
patch subject: [PATCH] sched: consider WF_SYNC to find idle siblings
in testcase: netperf
on test machine: 128 threads 2 sockets Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz (Ice Lake) with 128G memory
with following parameters:
ip: ipv4
runtime: 300s
nr_threads: 50%
cluster: cs-localhost
send_size: 10K
test: SCTP_STREAM_MANY
cpufreq_governor: performance
test-description: Netperf is a benchmark that can be use to measure various aspect of networking performance.
test-url: http://www.netperf.org/netperf/
In addition to that, the commit also has significant impact on the following tests:
+------------------+------------------------------------------------------------------------------------------------+
| testcase: change | stress-ng: stress-ng.sctp.ops_per_sec -48.4% regression |
| test machine | 128 threads 2 sockets Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz (Ice Lake) with 128G memory |
| test parameters | class=network |
| | cpufreq_governor=performance |
| | nr_threads=100% |
| | test=sctp |
| | testtime=60s |
+------------------+------------------------------------------------------------------------------------------------+
| testcase: change | unixbench: unixbench.score 36.2% improvement |
| test machine | 16 threads 1 sockets Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz (Coffee Lake) with 32G memory |
| test parameters | cpufreq_governor=performance |
| | nr_task=30% |
| | runtime=300s |
| | test=context1 |
+------------------+------------------------------------------------------------------------------------------------+
| testcase: change | netperf: netperf.Throughput_tps 53.3% improvement |
| test machine | 144 threads 4 sockets Intel(R) Xeon(R) Gold 5318H CPU @ 2.50GHz (Cooper Lake) with 128G memory |
| test parameters | cluster=cs-localhost |
| | cpufreq_governor=performance |
| | ip=ipv4 |
| | nr_threads=25% |
| | runtime=300s |
| | test=TCP_RR |
+------------------+------------------------------------------------------------------------------------------------+
Details are as below:
=========================================================================================
cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/send_size/tbox_group/test/testcase:
cs-localhost/gcc-11/performance/ipv4/x86_64-rhel-8.3/50%/debian-11.1-x86_64-20220510.cgz/300s/10K/lkp-icl-2sp4/SCTP_STREAM_MANY/netperf
commit:
c36eae5a86 ("sched/psi: Fix possible missing or delayed pending event")
b6aabb01e3 ("sched: consider WF_SYNC to find idle siblings")
c36eae5a86d15a69 b6aabb01e3004b0285d953761e1
---------------- ---------------------------
%stddev %change %stddev
\ | \
7844 -66.0% 2669 ± 23% netperf.Throughput_Mbps
502046 -66.0% 170845 ± 23% netperf.Throughput_total_Mbps
96866 +87957.6% 85298188 ± 23% netperf.time.involuntary_context_switches
22262 -5.6% 21022 netperf.time.minor_page_faults
1674 -79.6% 341.00 ± 23% netperf.time.percent_of_cpu_this_job_got
4913 -79.7% 995.08 ± 23% netperf.time.system_time
133.78 ± 2% -74.6% 34.03 ± 23% netperf.time.user_time
345136 +237.6% 1165282 ± 9% netperf.time.voluntary_context_switches
1.839e+09 -66.0% 6.257e+08 ± 23% netperf.workload
34569 +22.0% 42159 uptime.idle
2.841e+10 +27.0% 3.608e+10 cpuidle..time
5.644e+08 -86.9% 73779743 cpuidle..usage
1.201e+09 ± 2% -66.7% 4.002e+08 ± 21% numa-numastat.node0.local_node
1.201e+09 ± 2% -66.7% 4.003e+08 ± 21% numa-numastat.node0.numa_hit
1.171e+09 ± 3% -65.3% 4.064e+08 ± 25% numa-numastat.node1.local_node
1.171e+09 ± 3% -65.3% 4.064e+08 ± 25% numa-numastat.node1.numa_hit
74.00 +25.0% 92.50 vmstat.cpu.id
4716732 -15.7% 3974601 vmstat.memory.cache
31.83 -65.4% 11.00 ± 21% vmstat.procs.r
3346244 -82.4% 589168 ± 23% vmstat.system.cs
74.13 +19.0 93.17 mpstat.cpu.all.idle%
0.60 ± 2% +0.3 0.95 ± 24% mpstat.cpu.all.irq%
3.85 -2.9 0.93 ± 22% mpstat.cpu.all.soft%
20.66 -15.9 4.72 ± 23% mpstat.cpu.all.sys%
0.77 -0.5 0.24 ± 20% mpstat.cpu.all.usr%
147463 ± 10% +15.6% 170513 ± 6% numa-meminfo.node0.SUnreclaim
803425 ± 5% -95.6% 35689 ± 56% numa-meminfo.node1.Active
803380 ± 5% -95.6% 35646 ± 56% numa-meminfo.node1.Active(anon)
330592 ± 9% -90.1% 32819 ± 45% numa-meminfo.node1.Mapped
815993 ± 5% -90.1% 80396 ± 35% numa-meminfo.node1.Shmem
832708 ± 2% -95.0% 41879 ± 52% meminfo.Active
832610 ± 2% -95.0% 41793 ± 52% meminfo.Active(anon)
4598130 -16.3% 3847596 meminfo.Cached
1436261 -53.0% 674663 ± 3% meminfo.Committed_AS
381642 ± 3% -81.9% 69247 ± 10% meminfo.Mapped
6510432 -10.7% 5812535 meminfo.Memused
846123 ± 2% -88.6% 96290 ± 24% meminfo.Shmem
10463627 -32.4% 7073752 ± 3% meminfo.max_used_kB
36887 ± 10% +15.4% 42580 ± 6% numa-vmstat.node0.nr_slab_unreclaimable
1.201e+09 ± 2% -66.7% 4.003e+08 ± 21% numa-vmstat.node0.numa_hit
1.201e+09 ± 2% -66.7% 4.002e+08 ± 21% numa-vmstat.node0.numa_local
200557 ± 5% -95.5% 8951 ± 56% numa-vmstat.node1.nr_active_anon
83108 ± 9% -90.0% 8342 ± 44% numa-vmstat.node1.nr_mapped
203712 ± 5% -90.1% 20228 ± 35% numa-vmstat.node1.nr_shmem
200557 ± 5% -95.5% 8951 ± 56% numa-vmstat.node1.nr_zone_active_anon
1.171e+09 ± 3% -65.3% 4.064e+08 ± 25% numa-vmstat.node1.numa_hit
1.171e+09 ± 3% -65.3% 4.064e+08 ± 25% numa-vmstat.node1.numa_local
957.83 -73.0% 258.33 ± 21% turbostat.Avg_MHz
29.11 -21.2 7.89 ± 20% turbostat.Busy%
4.303e+08 -100.0% 107318 ± 26% turbostat.C1
5.52 -5.5 0.03 ±100% turbostat.C1%
1.317e+08 ± 2% -62.3% 49626627 ± 35% turbostat.C1E
74.33 -24.7% 56.00 ± 4% turbostat.CoreTmp
0.30 +20.0% 0.36 ± 4% turbostat.IPC
1105923 -99.4% 6441 ± 46% turbostat.POLL
0.03 -0.0 0.00 turbostat.POLL%
74.33 ± 2% -24.4% 56.17 ± 4% turbostat.PkgTmp
455.30 -38.7% 279.16 ± 5% turbostat.PkgWatt
46.51 -6.1% 43.69 turbostat.RAMWatt
208073 ± 2% -95.0% 10499 ± 52% proc-vmstat.nr_active_anon
101503 -1.2% 100305 proc-vmstat.nr_anon_pages
1149456 -16.3% 962060 proc-vmstat.nr_file_pages
106787 +8.5% 115818 proc-vmstat.nr_inactive_anon
95661 ± 3% -81.7% 17510 ± 10% proc-vmstat.nr_mapped
2428 ± 2% -6.0% 2282 proc-vmstat.nr_page_table_pages
211454 ± 2% -88.5% 24233 ± 24% proc-vmstat.nr_shmem
31087 -2.0% 30451 proc-vmstat.nr_slab_reclaimable
74377 +7.9% 80233 proc-vmstat.nr_slab_unreclaimable
208073 ± 2% -95.0% 10499 ± 52% proc-vmstat.nr_zone_active_anon
106787 +8.5% 115818 proc-vmstat.nr_zone_inactive_anon
2.372e+09 -66.0% 8.068e+08 ± 23% proc-vmstat.numa_hit
2.372e+09 -66.0% 8.066e+08 ± 23% proc-vmstat.numa_local
242358 ± 2% +95.5% 473925 ± 7% proc-vmstat.pgactivate
1.366e+10 -66.0% 4.643e+09 ± 23% proc-vmstat.pgalloc_normal
1461389 -16.3% 1222803 ± 4% proc-vmstat.pgfault
1.366e+10 -66.0% 4.643e+09 ± 23% proc-vmstat.pgfree
12556 ± 50% -99.9% 17.47 ±159% sched_debug.cfs_rq:/.MIN_vruntime.avg
766470 ± 38% -99.7% 2236 ±159% sched_debug.cfs_rq:/.MIN_vruntime.max
93742 ± 44% -99.8% 196.88 ±159% sched_debug.cfs_rq:/.MIN_vruntime.stddev
0.24 ± 6% -58.4% 0.10 ± 37% sched_debug.cfs_rq:/.h_nr_running.avg
1.06 ± 7% +66.3% 1.76 ± 5% sched_debug.cfs_rq:/.h_nr_running.max
218.12 ± 7% +153.6% 553.21 ± 76% sched_debug.cfs_rq:/.load_avg.max
12556 ± 50% -99.9% 17.47 ±159% sched_debug.cfs_rq:/.max_vruntime.avg
766470 ± 38% -99.7% 2236 ±159% sched_debug.cfs_rq:/.max_vruntime.max
93742 ± 44% -99.8% 196.88 ±159% sched_debug.cfs_rq:/.max_vruntime.stddev
998821 ± 10% -90.9% 90606 ± 16% sched_debug.cfs_rq:/.min_vruntime.avg
1154960 ± 10% -88.8% 128907 ± 11% sched_debug.cfs_rq:/.min_vruntime.max
874022 ± 13% -92.2% 67934 ± 21% sched_debug.cfs_rq:/.min_vruntime.min
49122 ± 17% -80.4% 9626 ± 13% sched_debug.cfs_rq:/.min_vruntime.stddev
0.24 ± 6% -70.0% 0.07 ± 32% sched_debug.cfs_rq:/.nr_running.avg
0.42 ± 2% -40.3% 0.25 ± 14% sched_debug.cfs_rq:/.nr_running.stddev
269.97 ± 2% -53.3% 126.18 ± 19% sched_debug.cfs_rq:/.runnable_avg.avg
1166 ± 4% +24.6% 1453 ± 4% sched_debug.cfs_rq:/.runnable_avg.max
116926 ± 27% -84.1% 18583 ± 50% sched_debug.cfs_rq:/.spread0.max
-164017 -74.1% -42414 sched_debug.cfs_rq:/.spread0.min
49129 ± 17% -80.4% 9632 ± 13% sched_debug.cfs_rq:/.spread0.stddev
269.72 ± 2% -61.9% 102.78 ± 15% sched_debug.cfs_rq:/.util_avg.avg
1166 ± 4% -7.0% 1084 ± 3% sched_debug.cfs_rq:/.util_avg.max
323.32 ± 2% -32.6% 217.82 ± 13% sched_debug.cfs_rq:/.util_avg.stddev
146.06 ± 6% -77.0% 33.66 ± 44% sched_debug.cfs_rq:/.util_est_enqueued.avg
989.44 -13.3% 857.96 ± 8% sched_debug.cfs_rq:/.util_est_enqueued.max
276.89 ± 2% -50.8% 136.28 ± 23% sched_debug.cfs_rq:/.util_est_enqueued.stddev
574938 ± 3% +26.2% 725536 ± 3% sched_debug.cpu.avg_idle.avg
3945 ± 3% +867.5% 38177 ± 80% sched_debug.cpu.avg_idle.min
406249 ± 2% -21.9% 317312 ± 6% sched_debug.cpu.avg_idle.stddev
3.57 ± 5% +54.4% 5.51 ± 6% sched_debug.cpu.clock.stddev
1612 ± 2% -11.4% 1429 sched_debug.cpu.clock_task.stddev
1391 ± 5% -69.5% 424.83 ± 33% sched_debug.cpu.curr->pid.avg
8941 ± 4% -17.6% 7367 ± 4% sched_debug.cpu.curr->pid.max
2392 ± 2% -39.0% 1459 ± 14% sched_debug.cpu.curr->pid.stddev
0.00 ± 6% -27.8% 0.00 ± 7% sched_debug.cpu.next_balance.stddev
0.24 ± 5% -56.8% 0.10 ± 37% sched_debug.cpu.nr_running.avg
1.06 ± 7% +86.8% 1.97 ± 3% sched_debug.cpu.nr_running.max
3581710 ± 11% -84.7% 549192 ± 15% sched_debug.cpu.nr_switches.avg
4987997 ± 13% -85.8% 708118 ± 15% sched_debug.cpu.nr_switches.max
2372208 ± 19% -82.6% 411758 ± 18% sched_debug.cpu.nr_switches.min
497412 ± 10% -88.1% 59277 ± 14% sched_debug.cpu.nr_switches.stddev
950.00 -100.0% 0.00 sched_debug.rt_rq:/.rt_runtime.avg
950.00 -100.0% 0.00 sched_debug.rt_rq:/.rt_runtime.max
950.00 -100.0% 0.00 sched_debug.rt_rq:/.rt_runtime.min
25.06 -64.2% 8.97 ± 18% perf-stat.i.MPKI
2.221e+10 -67.9% 7.14e+09 ± 22% perf-stat.i.branch-instructions
1.197e+08 -59.5% 48489409 ± 32% perf-stat.i.branch-misses
1.48 ± 3% +1.5 2.96 ± 20% perf-stat.i.cache-miss-rate%
39815902 ± 3% -78.9% 8385204 ± 16% perf-stat.i.cache-misses
2.787e+09 -88.6% 3.168e+08 ± 30% perf-stat.i.cache-references
3386965 -82.4% 596733 ± 23% perf-stat.i.context-switches
1.17 -19.7% 0.94 ± 4% perf-stat.i.cpi
1.291e+11 -74.7% 3.267e+10 ± 21% perf-stat.i.cpu-cycles
744.79 +888.7% 7363 ± 16% perf-stat.i.cpu-migrations
3317 ± 2% +18.6% 3933 ± 5% perf-stat.i.cycles-between-cache-misses
3.178e+10 -67.6% 1.029e+10 ± 22% perf-stat.i.dTLB-loads
1.882e+10 -67.8% 6.057e+09 ± 22% perf-stat.i.dTLB-stores
1.11e+11 -67.7% 3.585e+10 ± 22% perf-stat.i.instructions
0.86 +24.8% 1.07 ± 4% perf-stat.i.ipc
1.01 -74.7% 0.26 ± 21% perf-stat.i.metric.GHz
590.56 -68.5% 185.89 ± 22% perf-stat.i.metric.M/sec
4085 -19.2% 3300 ± 6% perf-stat.i.minor-faults
91.59 -1.9 89.65 perf-stat.i.node-load-miss-rate%
3843758 ± 8% -56.4% 1676424 ± 17% perf-stat.i.node-load-misses
345198 ± 4% -42.8% 197478 ± 17% perf-stat.i.node-loads
70.70 -31.5 39.23 ± 6% perf-stat.i.node-store-miss-rate%
3113677 ± 4% -64.0% 1121141 ± 17% perf-stat.i.node-store-misses
4085 -19.2% 3300 ± 6% perf-stat.i.page-faults
25.10 -65.2% 8.73 ± 16% perf-stat.overall.MPKI
1.43 ± 3% +1.4 2.79 ± 17% perf-stat.overall.cache-miss-rate%
1.16 -21.4% 0.91 ± 4% perf-stat.overall.cpi
3244 ± 2% +19.3% 3871 ± 5% perf-stat.overall.cycles-between-cache-misses
0.86 +27.4% 1.10 ± 4% perf-stat.overall.ipc
91.72 -2.3 89.42 perf-stat.overall.node-load-miss-rate%
68.26 -29.8 38.51 ± 6% perf-stat.overall.node-store-miss-rate%
18276 -5.2% 17333 perf-stat.overall.path-length
2.214e+10 -67.8% 7.118e+09 ± 22% perf-stat.ps.branch-instructions
1.192e+08 -59.5% 48334433 ± 32% perf-stat.ps.branch-misses
39676032 ± 3% -78.9% 8357755 ± 16% perf-stat.ps.cache-misses
2.777e+09 -88.6% 3.158e+08 ± 30% perf-stat.ps.cache-references
3374855 -82.4% 594908 ± 23% perf-stat.ps.context-switches
1.286e+11 -74.7% 3.257e+10 ± 21% perf-stat.ps.cpu-cycles
742.42 +888.6% 7339 ± 16% perf-stat.ps.cpu-migrations
3.166e+10 -67.6% 1.026e+10 ± 22% perf-stat.ps.dTLB-loads
1.876e+10 -67.8% 6.038e+09 ± 22% perf-stat.ps.dTLB-stores
1.106e+11 -67.7% 3.574e+10 ± 22% perf-stat.ps.instructions
4069 -19.2% 3290 ± 6% perf-stat.ps.minor-faults
3830257 ± 8% -56.4% 1670802 ± 17% perf-stat.ps.node-load-misses
344097 ± 4% -42.8% 196864 ± 17% perf-stat.ps.node-loads
3102660 ± 4% -64.0% 1117149 ± 17% perf-stat.ps.node-store-misses
4069 -19.2% 3290 ± 6% perf-stat.ps.page-faults
3.36e+13 -67.8% 1.082e+13 ± 22% perf-stat.total.instructions
22.54 -9.4 13.14 ± 5% perf-profile.calltrace.cycles-pp.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg.__sys_recvmsg
22.62 -9.4 13.26 ± 5% perf-profile.calltrace.cycles-pp.inet_recvmsg.____sys_recvmsg.___sys_recvmsg.__sys_recvmsg.do_syscall_64
23.55 -9.2 14.36 ± 5% perf-profile.calltrace.cycles-pp.____sys_recvmsg.___sys_recvmsg.__sys_recvmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe
24.92 -9.0 15.96 ± 4% perf-profile.calltrace.cycles-pp.___sys_recvmsg.__sys_recvmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.recvmsg
25.25 -8.9 16.31 ± 4% perf-profile.calltrace.cycles-pp.__sys_recvmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.recvmsg
26.64 -8.9 17.72 ± 4% perf-profile.calltrace.cycles-pp.recvmsg
25.74 -8.9 16.87 ± 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.recvmsg
25.56 -8.8 16.74 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.recvmsg
8.60 -8.6 0.00 perf-profile.calltrace.cycles-pp.intel_idle_irq.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
8.45 -8.5 0.00 perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle_irq.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
10.10 -4.9 5.19 ± 3% perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.sctp_recvmsg.inet_recvmsg.____sys_recvmsg
10.13 -4.9 5.24 ± 3% perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg
8.62 -4.3 4.35 ± 3% perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.sctp_recvmsg
8.58 -4.3 4.30 ± 3% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
8.87 -4.2 4.66 ± 3% perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.sctp_recvmsg.inet_recvmsg
4.71 -2.5 2.22 ± 4% perf-profile.calltrace.cycles-pp.kfree_skb_reason.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg
3.72 ± 2% -1.9 1.83 ± 4% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages.__kmalloc_large_node.__kmalloc_node_track_caller.kmalloc_reserve
4.19 ± 2% -1.8 2.36 ± 4% perf-profile.calltrace.cycles-pp.__alloc_pages.__kmalloc_large_node.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
2.89 ± 2% -1.8 1.09 ± 5% perf-profile.calltrace.cycles-pp.skb_release_data.kfree_skb_reason.sctp_recvmsg.inet_recvmsg.____sys_recvmsg
10.94 -1.8 9.18 ± 4% perf-profile.calltrace.cycles-pp.sctp_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog
11.10 -1.7 9.37 ± 4% perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog.__napi_poll
11.08 -1.7 9.35 ± 3% perf-profile.calltrace.cycles-pp.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_primitive_SEND
11.12 -1.7 9.44 ± 4% perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action
13.03 -1.6 11.41 ± 3% perf-profile.calltrace.cycles-pp.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_primitive_SEND.sctp_sendmsg_to_asoc
11.43 -1.6 9.85 ± 4% perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start
11.76 -1.5 10.22 ± 4% perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2
11.65 -1.5 10.11 ± 4% perf-profile.calltrace.cycles-pp.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip
11.62 -1.5 10.10 ± 4% perf-profile.calltrace.cycles-pp.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq
12.02 -1.5 10.50 ± 4% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit
12.10 -1.5 10.59 ± 4% perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit.sctp_packet_transmit
12.16 -1.5 10.67 ± 4% perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit.sctp_packet_transmit.sctp_outq_flush
2.44 ± 2% -1.4 1.00 ± 17% perf-profile.calltrace.cycles-pp.__alloc_skb.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm
12.44 -1.4 11.00 ± 4% perf-profile.calltrace.cycles-pp.ip_finish_output2.__ip_queue_xmit.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter
14.88 -1.4 13.45 ± 3% perf-profile.calltrace.cycles-pp.sctp_primitive_SEND.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg.____sys_sendmsg
12.62 -1.4 11.21 ± 4% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm
8.62 -1.4 7.21 ± 4% perf-profile.calltrace.cycles-pp.sctp_do_sm.sctp_assoc_bh_rcv.sctp_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish
2.17 ± 2% -1.4 0.79 ± 4% perf-profile.calltrace.cycles-pp.kmalloc_reserve.__alloc_skb.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter
2.16 ± 2% -1.4 0.78 ± 4% perf-profile.calltrace.cycles-pp.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb.sctp_packet_transmit.sctp_outq_flush
2.15 ± 2% -1.4 0.77 ± 4% perf-profile.calltrace.cycles-pp.__kmalloc_large_node.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb.sctp_packet_transmit
9.20 -1.4 7.83 ± 4% perf-profile.calltrace.cycles-pp.sctp_assoc_bh_rcv.sctp_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core
5.66 -1.3 4.33 ± 5% perf-profile.calltrace.cycles-pp.sctp_ulpq_tail_data.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv.sctp_rcv
2.18 ± 2% -1.3 0.87 ± 24% perf-profile.calltrace.cycles-pp.rmqueue.get_page_from_freelist.__alloc_pages.__kmalloc_large_node.__kmalloc_node_track_caller
7.72 -1.3 6.46 ± 5% perf-profile.calltrace.cycles-pp.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv.sctp_rcv.ip_protocol_deliver_rcu
10.04 -1.2 8.80 ± 3% perf-profile.calltrace.cycles-pp.sctp_cmd_interpreter.sctp_do_sm.sctp_primitive_SEND.sctp_sendmsg_to_asoc.sctp_sendmsg
2.33 -1.2 1.14 ± 5% perf-profile.calltrace.cycles-pp.sctp_ulpq_tail_event.sctp_ulpq_tail_data.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv
9.78 -1.1 8.67 ± 3% perf-profile.calltrace.cycles-pp.sctp_do_sm.sctp_primitive_SEND.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg
2.14 -1.1 1.08 ± 5% perf-profile.calltrace.cycles-pp.sctp_data_ready.sctp_ulpq_tail_event.sctp_ulpq_tail_data.sctp_cmd_interpreter.sctp_do_sm
4.22 ± 2% -0.9 3.28 ± 4% perf-profile.calltrace.cycles-pp.sctp_chunk_put.sctp_outq_sack.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv
19.14 -0.9 18.21 ± 3% perf-profile.calltrace.cycles-pp.sctp_backlog_rcv.__release_sock.release_sock.sctp_sendmsg.sock_sendmsg
3.72 ± 2% -0.8 2.90 ± 4% perf-profile.calltrace.cycles-pp.consume_skb.sctp_chunk_put.sctp_outq_sack.sctp_cmd_interpreter.sctp_do_sm
1.77 -0.8 1.00 ± 4% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sctp_data_ready.sctp_ulpq_tail_event.sctp_ulpq_tail_data.sctp_cmd_interpreter
1.69 -0.7 0.96 ± 4% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sctp_data_ready.sctp_ulpq_tail_event.sctp_ulpq_tail_data
5.43 ± 2% -0.7 4.72 ± 3% perf-profile.calltrace.cycles-pp.sctp_outq_sack.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv.sctp_backlog_rcv
1.35 -0.7 0.65 ± 9% perf-profile.calltrace.cycles-pp.__sk_mem_raise_allocated.__sk_mem_schedule.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg
1.80 -0.7 1.10 ± 4% perf-profile.calltrace.cycles-pp.skb_release_head_state.kfree_skb_reason.sctp_recvmsg.inet_recvmsg.____sys_recvmsg
1.38 -0.7 0.68 ± 9% perf-profile.calltrace.cycles-pp.__sk_mem_schedule.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg.____sys_sendmsg
0.98 ± 2% -0.6 0.35 ± 70% perf-profile.calltrace.cycles-pp.mem_cgroup_charge_skmem.__sk_mem_raise_allocated.__sk_mem_schedule.sctp_sendmsg_to_asoc.sctp_sendmsg
1.29 ± 2% -0.6 0.69 ± 5% perf-profile.calltrace.cycles-pp.__sk_mem_reduce_allocated.skb_release_head_state.kfree_skb_reason.sctp_recvmsg.inet_recvmsg
2.42 ± 2% -0.6 1.84 ± 4% perf-profile.calltrace.cycles-pp.__kmalloc_large_node.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb._sctp_make_chunk
2.48 ± 2% -0.6 1.92 ± 4% perf-profile.calltrace.cycles-pp.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb._sctp_make_chunk.sctp_make_datafrag_empty
2.52 ± 2% -0.6 1.97 ± 4% perf-profile.calltrace.cycles-pp.kmalloc_reserve.__alloc_skb._sctp_make_chunk.sctp_make_datafrag_empty.sctp_datamsg_from_user
13.06 -0.5 12.53 ± 3% perf-profile.calltrace.cycles-pp.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv
1.04 ± 2% -0.5 0.54 ± 5% perf-profile.calltrace.cycles-pp.__sk_mem_reduce_allocated.sctp_wfree.skb_release_head_state.consume_skb.sctp_chunk_put
1.42 -0.5 0.94 ± 4% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sctp_data_ready.sctp_ulpq_tail_event
1.40 -0.5 0.92 ± 4% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sctp_data_ready
1.58 ± 2% -0.4 1.14 ± 5% perf-profile.calltrace.cycles-pp.sctp_wfree.skb_release_head_state.consume_skb.sctp_chunk_put.sctp_outq_sack
1.74 ± 2% -0.4 1.32 ± 4% perf-profile.calltrace.cycles-pp.skb_release_head_state.consume_skb.sctp_chunk_put.sctp_outq_sack.sctp_cmd_interpreter
3.02 ± 2% -0.4 2.61 ± 4% perf-profile.calltrace.cycles-pp.sctp_ulpevent_make_rcvmsg.sctp_ulpq_tail_data.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv
3.27 -0.4 2.87 ± 4% perf-profile.calltrace.cycles-pp.__alloc_skb._sctp_make_chunk.sctp_make_datafrag_empty.sctp_datamsg_from_user.sctp_sendmsg_to_asoc
1.42 ± 2% -0.4 1.04 ± 5% perf-profile.calltrace.cycles-pp.free_unref_page.skb_release_data.consume_skb.sctp_chunk_put.sctp_outq_sack
1.92 ± 2% -0.4 1.55 ± 5% perf-profile.calltrace.cycles-pp.skb_release_data.consume_skb.sctp_chunk_put.sctp_outq_sack.sctp_cmd_interpreter
0.91 -0.3 0.60 ± 4% perf-profile.calltrace.cycles-pp.sctp_outq_select_transport.sctp_outq_flush_data.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm
0.75 -0.3 0.47 ± 44% perf-profile.calltrace.cycles-pp.sctp_chunk_put.sctp_ulpevent_free.sctp_recvmsg.inet_recvmsg.____sys_recvmsg
3.92 -0.2 3.70 ± 4% perf-profile.calltrace.cycles-pp._sctp_make_chunk.sctp_make_datafrag_empty.sctp_datamsg_from_user.sctp_sendmsg_to_asoc.sctp_sendmsg
1.38 -0.2 1.23 ± 3% perf-profile.calltrace.cycles-pp.sctp_ulpevent_free.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg
0.52 ± 2% +0.1 0.64 ± 4% perf-profile.calltrace.cycles-pp.sock_kmalloc.____sys_sendmsg.___sys_sendmsg.__sys_sendmsg.do_syscall_64
0.59 ± 3% +0.1 0.74 ± 5% perf-profile.calltrace.cycles-pp.__import_iovec.import_iovec.copy_msghdr_from_user.___sys_sendmsg.__sys_sendmsg
0.58 ± 4% +0.2 0.73 ± 5% perf-profile.calltrace.cycles-pp.__import_iovec.import_iovec.copy_msghdr_from_user.___sys_recvmsg.__sys_recvmsg
0.61 ± 2% +0.2 0.77 ± 4% perf-profile.calltrace.cycles-pp.import_iovec.copy_msghdr_from_user.___sys_sendmsg.__sys_sendmsg.do_syscall_64
0.62 ± 3% +0.2 0.78 ± 5% perf-profile.calltrace.cycles-pp.__copy_msghdr.copy_msghdr_from_user.___sys_sendmsg.__sys_sendmsg.do_syscall_64
0.60 ± 4% +0.2 0.76 ± 5% perf-profile.calltrace.cycles-pp.import_iovec.copy_msghdr_from_user.___sys_recvmsg.__sys_recvmsg.do_syscall_64
0.62 ± 5% +0.2 0.78 ± 9% perf-profile.calltrace.cycles-pp.__rhashtable_lookup.sctp_epaddr_lookup_transport.sctp_endpoint_lookup_assoc.sctp_sendmsg.sock_sendmsg
0.79 +0.2 0.97 ± 4% perf-profile.calltrace.cycles-pp.sctp_outq_flush_data.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv
0.68 ± 4% +0.2 0.86 ± 8% perf-profile.calltrace.cycles-pp.sctp_epaddr_lookup_transport.sctp_endpoint_lookup_assoc.sctp_sendmsg.sock_sendmsg.____sys_sendmsg
0.72 ± 4% +0.2 0.92 ± 8% perf-profile.calltrace.cycles-pp.sctp_endpoint_lookup_assoc.sctp_sendmsg.sock_sendmsg.____sys_sendmsg.___sys_sendmsg
0.45 ± 44% +0.2 0.67 ± 4% perf-profile.calltrace.cycles-pp.move_addr_to_kernel.__copy_msghdr.copy_msghdr_from_user.___sys_sendmsg.__sys_sendmsg
1.20 +0.2 1.43 ± 3% perf-profile.calltrace.cycles-pp.copy_msghdr_from_user.___sys_recvmsg.__sys_recvmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.34 ± 70% +0.3 0.64 ± 7% perf-profile.calltrace.cycles-pp.sctp_chunkify._sctp_make_chunk.sctp_make_datafrag_empty.sctp_datamsg_from_user.sctp_sendmsg_to_asoc
1.82 ± 2% +0.4 2.18 ± 4% perf-profile.calltrace.cycles-pp.schedule_timeout.sctp_skb_recv_datagram.sctp_recvmsg.inet_recvmsg.____sys_recvmsg
0.26 ±100% +0.4 0.64 ± 6% perf-profile.calltrace.cycles-pp.iovec_from_user.__import_iovec.import_iovec.copy_msghdr_from_user.___sys_sendmsg
1.69 ± 2% +0.4 2.14 ± 3% perf-profile.calltrace.cycles-pp.copy_msghdr_from_user.___sys_sendmsg.__sys_sendmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.09 ±223% +0.5 0.62 ± 5% perf-profile.calltrace.cycles-pp.iovec_from_user.__import_iovec.import_iovec.copy_msghdr_from_user.___sys_recvmsg
0.00 +0.6 0.56 ± 6% perf-profile.calltrace.cycles-pp.release_sock.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg
0.00 +0.6 0.59 ± 5% perf-profile.calltrace.cycles-pp.sctp_chunk_free.sctp_outq_sack.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv
0.00 +0.6 0.62 ± 3% perf-profile.calltrace.cycles-pp.__schedule.__cond_resched.__release_sock.release_sock.sctp_sendmsg
0.00 +0.6 0.62 ± 3% perf-profile.calltrace.cycles-pp.update_sg_lb_stats.update_sd_lb_stats.find_busiest_group.load_balance.newidle_balance
0.00 +0.6 0.62 ± 5% perf-profile.calltrace.cycles-pp.sctp_addto_chunk.sctp_make_datafrag_empty.sctp_datamsg_from_user.sctp_sendmsg_to_asoc.sctp_sendmsg
1.50 ± 2% +0.6 2.13 ± 3% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.sctp_skb_recv_datagram.sctp_recvmsg.inet_recvmsg
1.44 ± 2% +0.6 2.08 ± 3% perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.sctp_skb_recv_datagram.sctp_recvmsg
0.00 +0.7 0.67 ± 3% perf-profile.calltrace.cycles-pp.update_sd_lb_stats.find_busiest_group.load_balance.newidle_balance.pick_next_task_fair
0.00 +0.7 0.67 ± 4% perf-profile.calltrace.cycles-pp.find_busiest_group.load_balance.newidle_balance.pick_next_task_fair.__schedule
0.00 +0.7 0.68 ± 14% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry
0.00 +0.7 0.68 ± 3% perf-profile.calltrace.cycles-pp.__cond_resched.__release_sock.release_sock.sctp_sendmsg.sock_sendmsg
0.00 +0.7 0.73 ± 18% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
0.00 +0.7 0.74 ± 4% perf-profile.calltrace.cycles-pp.load_balance.newidle_balance.pick_next_task_fair.__schedule.schedule
0.00 +0.8 0.79 ± 4% perf-profile.calltrace.cycles-pp.newidle_balance.pick_next_task_fair.__schedule.schedule.schedule_timeout
3.36 +0.9 4.22 ± 3% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin._copy_from_iter.sctp_user_addto_chunk.sctp_datamsg_from_user
3.40 +0.9 4.27 ± 3% perf-profile.calltrace.cycles-pp.copyin._copy_from_iter.sctp_user_addto_chunk.sctp_datamsg_from_user.sctp_sendmsg_to_asoc
0.68 ± 2% +0.9 1.55 ± 19% perf-profile.calltrace.cycles-pp.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
0.00 +0.9 0.90 ± 15% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
0.00 +0.9 0.93 ± 3% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.schedule_timeout.sctp_skb_recv_datagram
3.65 +0.9 4.59 ± 3% perf-profile.calltrace.cycles-pp._copy_from_iter.sctp_user_addto_chunk.sctp_datamsg_from_user.sctp_sendmsg_to_asoc.sctp_sendmsg
4.08 +1.1 5.13 ± 3% perf-profile.calltrace.cycles-pp.sctp_user_addto_chunk.sctp_datamsg_from_user.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg
0.00 +1.1 1.14 ± 14% perf-profile.calltrace.cycles-pp.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
0.00 +1.4 1.36 ± 26% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
9.56 +1.4 10.94 ± 3% perf-profile.calltrace.cycles-pp.sctp_datamsg_from_user.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg.____sys_sendmsg
0.00 +1.5 1.49 ± 30% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
0.00 +1.7 1.66 ± 27% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
0.00 +2.5 2.46 ± 26% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +3.5 3.48 ± 19% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
0.00 +3.6 3.57 ± 20% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
0.66 ± 10% +4.9 5.60 ± 18% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
0.75 ± 9% +5.9 6.66 ± 17% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
17.20 +6.5 23.70 ± 13% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
17.19 +6.5 23.70 ± 13% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
17.15 +6.5 23.67 ± 13% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
17.32 +6.6 23.88 ± 13% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
13.07 +7.7 20.79 ± 12% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
13.17 +8.2 21.38 ± 12% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
14.06 +9.2 23.24 ± 13% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
3.18 ± 4% +10.6 13.78 ± 12% perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
3.20 ± 4% +10.6 13.84 ± 12% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
22.86 -9.4 13.47 ± 5% perf-profile.children.cycles-pp.sctp_recvmsg
22.63 -9.4 13.27 ± 5% perf-profile.children.cycles-pp.inet_recvmsg
23.57 -9.2 14.38 ± 5% perf-profile.children.cycles-pp.____sys_recvmsg
24.96 -9.0 16.00 ± 4% perf-profile.children.cycles-pp.___sys_recvmsg
25.27 -8.9 16.34 ± 4% perf-profile.children.cycles-pp.__sys_recvmsg
26.87 -8.9 18.01 ± 4% perf-profile.children.cycles-pp.recvmsg
8.67 -8.7 0.00 perf-profile.children.cycles-pp.intel_idle_irq
79.11 -7.0 72.07 ± 4% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
78.72 -7.0 71.69 ± 4% perf-profile.children.cycles-pp.do_syscall_64
10.11 -4.9 5.20 ± 3% perf-profile.children.cycles-pp.__skb_datagram_iter
10.14 -4.9 5.25 ± 3% perf-profile.children.cycles-pp.skb_copy_datagram_iter
8.64 -4.3 4.36 ± 3% perf-profile.children.cycles-pp.copyout
8.88 -4.2 4.68 ± 3% perf-profile.children.cycles-pp._copy_to_iter
13.72 -2.9 10.78 ± 3% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
4.73 -2.5 2.25 ± 4% perf-profile.children.cycles-pp.kfree_skb_reason
33.66 -2.3 31.40 ± 3% perf-profile.children.cycles-pp.sctp_cmd_interpreter
33.91 -2.2 31.71 ± 3% perf-profile.children.cycles-pp.sctp_do_sm
4.98 -2.1 2.88 ± 4% perf-profile.children.cycles-pp.skb_release_data
2.42 ± 2% -2.0 0.45 ± 71% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
24.48 -1.9 22.58 ± 3% perf-profile.children.cycles-pp.sctp_packet_transmit
27.60 -1.6 25.95 ± 3% perf-profile.children.cycles-pp.sctp_outq_flush
3.79 -1.6 2.16 ± 4% perf-profile.children.cycles-pp.get_page_from_freelist
4.66 -1.6 3.04 ± 3% perf-profile.children.cycles-pp.__kmalloc_large_node
4.81 -1.6 3.25 ± 3% perf-profile.children.cycles-pp.__kmalloc_node_track_caller
4.87 -1.5 3.33 ± 3% perf-profile.children.cycles-pp.kmalloc_reserve
4.28 ± 2% -1.5 2.76 ± 3% perf-profile.children.cycles-pp.__alloc_pages
15.05 -1.5 13.56 ± 4% perf-profile.children.cycles-pp.sctp_primitive_SEND
2.22 ± 2% -1.5 0.75 ± 8% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
11.15 -1.4 9.72 ± 4% perf-profile.children.cycles-pp.sctp_rcv
2.74 ± 2% -1.4 1.34 ± 5% perf-profile.children.cycles-pp.free_unref_page
11.28 -1.4 9.89 ± 4% perf-profile.children.cycles-pp.ip_protocol_deliver_rcu
6.14 -1.4 4.76 ± 3% perf-profile.children.cycles-pp.__alloc_skb
11.30 -1.4 9.96 ± 4% perf-profile.children.cycles-pp.ip_local_deliver_finish
5.81 -1.3 4.49 ± 5% perf-profile.children.cycles-pp.sctp_ulpq_tail_data
1.36 ± 2% -1.3 0.06 ± 6% perf-profile.children.cycles-pp.__slab_free
2.67 ± 2% -1.3 1.40 ± 4% perf-profile.children.cycles-pp.rmqueue
11.61 -1.2 10.38 ± 4% perf-profile.children.cycles-pp.__netif_receive_skb_one_core
12.28 -1.2 11.06 ± 4% perf-profile.children.cycles-pp.do_softirq
11.95 -1.2 10.76 ± 4% perf-profile.children.cycles-pp.net_rx_action
11.83 -1.2 10.64 ± 4% perf-profile.children.cycles-pp.__napi_poll
11.81 -1.2 10.63 ± 4% perf-profile.children.cycles-pp.process_backlog
5.74 -1.2 4.56 ± 4% perf-profile.children.cycles-pp.sctp_chunk_put
12.49 -1.2 11.32 ± 4% perf-profile.children.cycles-pp.__local_bh_enable_ip
1.62 ± 2% -1.2 0.45 ± 8% perf-profile.children.cycles-pp._raw_spin_lock
2.60 -1.2 1.43 ± 7% perf-profile.children.cycles-pp.__sk_mem_raise_allocated
3.83 -1.2 2.66 ± 4% perf-profile.children.cycles-pp.skb_release_head_state
2.66 -1.2 1.51 ± 6% perf-profile.children.cycles-pp.__sk_mem_schedule
24.83 -1.1 23.72 ± 4% perf-profile.children.cycles-pp.sctp_assoc_bh_rcv
2.37 -1.1 1.27 ± 5% perf-profile.children.cycles-pp.__sk_mem_reduce_allocated
12.86 -1.1 11.80 ± 4% perf-profile.children.cycles-pp.ip_finish_output2
1.10 -1.1 0.04 ± 44% perf-profile.children.cycles-pp.sched_ttwu_pending
13.14 -1.0 12.13 ± 4% perf-profile.children.cycles-pp.__ip_queue_xmit
2.38 -1.0 1.41 ± 5% perf-profile.children.cycles-pp.sctp_ulpq_tail_event
19.52 -1.0 18.56 ± 3% perf-profile.children.cycles-pp.sctp_backlog_rcv
2.02 -0.9 1.09 ± 5% perf-profile.children.cycles-pp.mem_cgroup_charge_skmem
2.17 -0.9 1.25 ± 5% perf-profile.children.cycles-pp.sctp_data_ready
4.27 -0.9 3.40 ± 4% perf-profile.children.cycles-pp.consume_skb
0.94 ± 2% -0.8 0.12 ± 17% perf-profile.children.cycles-pp.schedule_idle
1.10 ± 3% -0.8 0.34 ± 7% perf-profile.children.cycles-pp.try_charge_memcg
1.80 -0.7 1.07 ± 4% perf-profile.children.cycles-pp.__wake_up_common_lock
1.72 -0.7 1.02 ± 4% perf-profile.children.cycles-pp.__wake_up_common
5.54 -0.6 4.92 ± 3% perf-profile.children.cycles-pp.sctp_outq_sack
1.21 ± 3% -0.6 0.65 ± 7% perf-profile.children.cycles-pp.__mod_memcg_state
1.06 ± 2% -0.5 0.51 ± 9% perf-profile.children.cycles-pp.__free_pages_ok
0.90 ± 2% -0.5 0.44 ± 6% perf-profile.children.cycles-pp.refill_stock
12.34 -0.5 11.88 ± 2% perf-profile.children.cycles-pp.__softirqentry_text_start
0.62 ± 3% -0.5 0.17 ± 20% perf-profile.children.cycles-pp.ipv4_dst_check
1.45 -0.5 1.00 ± 3% perf-profile.children.cycles-pp.autoremove_wake_function
1.09 -0.4 0.65 ± 6% perf-profile.children.cycles-pp.sctp_packet_config
1.43 -0.4 0.99 ± 3% perf-profile.children.cycles-pp.try_to_wake_up
0.77 ± 2% -0.4 0.33 ± 7% perf-profile.children.cycles-pp.free_unref_page_commit
1.24 -0.4 0.82 ± 5% perf-profile.children.cycles-pp.sctp_outq_select_transport
1.33 -0.4 0.92 ± 5% perf-profile.children.cycles-pp.kmem_cache_free
1.62 -0.4 1.22 ± 4% perf-profile.children.cycles-pp.sctp_wfree
3.15 -0.4 2.76 ± 5% perf-profile.children.cycles-pp.sctp_ulpevent_make_rcvmsg
0.40 -0.3 0.07 ± 6% perf-profile.children.cycles-pp.select_task_rq
0.46 ± 2% -0.3 0.16 ± 7% perf-profile.children.cycles-pp.sctp_inet_skb_msgname
0.57 ± 2% -0.3 0.29 ± 8% perf-profile.children.cycles-pp.dequeue_entity
0.33 -0.3 0.06 ± 9% perf-profile.children.cycles-pp.select_task_rq_fair
0.49 ± 2% -0.3 0.23 ± 8% perf-profile.children.cycles-pp.__mod_node_page_state
0.76 ± 3% -0.2 0.51 ± 7% perf-profile.children.cycles-pp.mem_cgroup_uncharge_skmem
0.55 ± 3% -0.2 0.33 ± 16% perf-profile.children.cycles-pp.dst_release
0.49 -0.2 0.28 ± 6% perf-profile.children.cycles-pp.enqueue_entity
0.38 ± 3% -0.2 0.18 ± 9% perf-profile.children.cycles-pp.__free_one_page
4.26 -0.2 4.07 ± 4% perf-profile.children.cycles-pp._sctp_make_chunk
0.70 ± 2% -0.2 0.51 ± 5% perf-profile.children.cycles-pp.sctp_eat_data
0.79 ± 2% -0.2 0.61 ± 4% perf-profile.children.cycles-pp.sctp_sf_eat_data_6_2
0.25 ± 3% -0.2 0.07 ± 11% perf-profile.children.cycles-pp.prepare_to_wait_exclusive
0.60 ± 2% -0.2 0.42 ± 8% perf-profile.children.cycles-pp.simple_copy_to_iter
0.58 ± 4% -0.2 0.42 ± 9% perf-profile.children.cycles-pp.__copy_skb_header
0.79 ± 3% -0.2 0.64 ± 7% perf-profile.children.cycles-pp.__skb_clone
1.40 -0.1 1.26 ± 3% perf-profile.children.cycles-pp.sctp_ulpevent_free
0.76 ± 2% -0.1 0.63 ± 5% perf-profile.children.cycles-pp.sctp_addrs_lookup_transport
0.79 ± 2% -0.1 0.66 ± 5% perf-profile.children.cycles-pp.__sctp_rcv_lookup
0.45 ± 2% -0.1 0.32 ± 4% perf-profile.children.cycles-pp.free_large_kmalloc
0.53 ± 2% -0.1 0.40 ± 4% perf-profile.children.cycles-pp.__zone_watermark_ok
0.28 -0.1 0.16 ± 8% perf-profile.children.cycles-pp.prepare_task_switch
0.60 -0.1 0.49 ± 3% perf-profile.children.cycles-pp.ttwu_do_activate
0.84 -0.1 0.72 ± 6% perf-profile.children.cycles-pp.check_heap_object
0.58 -0.1 0.49 ± 3% perf-profile.children.cycles-pp.enqueue_task_fair
0.62 ± 2% -0.1 0.52 ± 6% perf-profile.children.cycles-pp.dequeue_task_fair
0.33 -0.1 0.24 ± 5% perf-profile.children.cycles-pp.__switch_to
0.43 -0.1 0.34 ± 4% perf-profile.children.cycles-pp.skb_set_owner_w
0.20 -0.1 0.12 ± 12% perf-profile.children.cycles-pp.___perf_sw_event
0.20 ± 2% -0.1 0.13 ± 7% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
0.13 -0.1 0.06 ± 9% perf-profile.children.cycles-pp.perf_tp_event
0.60 -0.1 0.53 ± 5% perf-profile.children.cycles-pp.sctp_inq_pop
0.12 ± 3% -0.0 0.08 ± 9% perf-profile.children.cycles-pp.__wrgsbase_inactive
0.20 ± 3% -0.0 0.16 ± 6% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.15 ± 3% -0.0 0.11 ± 8% perf-profile.children.cycles-pp.__build_skb_around
0.10 ± 3% -0.0 0.08 ± 6% perf-profile.children.cycles-pp.check_new_pages
0.05 +0.0 0.07 ± 7% perf-profile.children.cycles-pp.sctp_sendmsg_check_sflags
0.06 ± 7% +0.0 0.08 ± 7% perf-profile.children.cycles-pp.sctp_assoc_rwnd_decrease
0.05 ± 7% +0.0 0.07 ± 10% perf-profile.children.cycles-pp.skb_pull
0.10 ± 3% +0.0 0.12 ± 8% perf-profile.children.cycles-pp.validate_xmit_skb
0.07 ± 5% +0.0 0.09 ± 7% perf-profile.children.cycles-pp.ip_send_check
0.05 +0.0 0.07 ± 9% perf-profile.children.cycles-pp.sctp_tsnmap_check
0.08 ± 5% +0.0 0.10 ± 4% perf-profile.children.cycles-pp.sctp_validate_data
0.14 +0.0 0.16 ± 3% perf-profile.children.cycles-pp.__mod_lruvec_page_state
0.09 ± 5% +0.0 0.11 ± 9% perf-profile.children.cycles-pp.syscall_enter_from_user_mode
0.07 ± 7% +0.0 0.09 ± 11% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
0.09 ± 5% +0.0 0.12 ± 8% perf-profile.children.cycles-pp.__ip_local_out
0.12 ± 4% +0.0 0.14 ± 5% perf-profile.children.cycles-pp.sctp_sock_rfree
0.17 ± 2% +0.0 0.20 ± 5% perf-profile.children.cycles-pp.sctp_ulpq_order
0.06 ± 11% +0.0 0.09 ± 19% perf-profile.children.cycles-pp.sctp_get_af_specific
0.08 ± 4% +0.0 0.11 ± 8% perf-profile.children.cycles-pp.ip_rcv_core
0.11 ± 6% +0.0 0.14 ± 9% perf-profile.children.cycles-pp.ip_local_out
0.14 ± 5% +0.0 0.16 ± 7% perf-profile.children.cycles-pp.check_stack_object
0.03 ± 70% +0.0 0.06 ± 7% perf-profile.children.cycles-pp.sctp_auth_send_cid
0.10 +0.0 0.13 ± 9% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare
0.11 ± 3% +0.0 0.14 ± 9% perf-profile.children.cycles-pp.__netif_receive_skb_core
0.30 ± 2% +0.0 0.33 ± 6% perf-profile.children.cycles-pp.sctp_transport_put
0.07 +0.0 0.10 ± 7% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.16 ± 2% +0.0 0.20 ± 5% perf-profile.children.cycles-pp.__put_user_nocheck_8
0.12 ± 5% +0.0 0.16 ± 7% perf-profile.children.cycles-pp.os_xsave
0.04 ± 44% +0.0 0.08 ± 41% perf-profile.children.cycles-pp.inet_sendmsg
0.12 ± 6% +0.0 0.16 ± 7% perf-profile.children.cycles-pp.sctp_cmp_addr_exact
0.27 ± 2% +0.0 0.31 ± 6% perf-profile.children.cycles-pp.sctp_make_control
0.25 ± 3% +0.0 0.29 ± 6% perf-profile.children.cycles-pp.sctp_outq_tail
0.12 ± 4% +0.0 0.16 ± 8% perf-profile.children.cycles-pp.sctp_chunk_assign_ssn
0.02 ±141% +0.0 0.06 ± 11% perf-profile.children.cycles-pp.update_min_vruntime
0.05 ± 8% +0.0 0.10 ± 8% perf-profile.children.cycles-pp.check_preempt_curr
0.11 ± 5% +0.0 0.15 ± 22% perf-profile.children.cycles-pp.sctp_hash_key
0.08 ± 5% +0.0 0.13 ± 9% perf-profile.children.cycles-pp.__sctp_write_space
0.24 ± 4% +0.0 0.29 ± 3% perf-profile.children.cycles-pp.skb_clone
0.01 ±223% +0.0 0.06 ± 9% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
0.18 ± 2% +0.0 0.23 ± 7% perf-profile.children.cycles-pp.rcu_all_qs
0.16 ± 3% +0.0 0.20 ± 3% perf-profile.children.cycles-pp.ip_rcv
0.20 +0.0 0.25 ± 9% perf-profile.children.cycles-pp.sctp_datamsg_destroy
0.23 ± 3% +0.1 0.28 ± 7% perf-profile.children.cycles-pp.sctp_association_hold
0.18 ± 5% +0.1 0.24 ± 7% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.01 ±223% +0.1 0.06 ± 9% perf-profile.children.cycles-pp.netif_skb_features
0.34 ± 3% +0.1 0.39 ± 5% perf-profile.children.cycles-pp.sctp_sched_fcfs_dequeue
0.00 +0.1 0.05 ± 13% perf-profile.children.cycles-pp.sctp_ulpq_retrieve_ordered
0.23 ± 4% +0.1 0.29 ± 5% perf-profile.children.cycles-pp.sctp_chunk_hold
0.41 ± 4% +0.1 0.46 ± 5% perf-profile.children.cycles-pp.sock_recvmsg
0.39 +0.1 0.45 ± 5% perf-profile.children.cycles-pp.send_sctp_stream_1toMany
0.06 +0.1 0.12 ± 10% perf-profile.children.cycles-pp.reweight_entity
0.00 +0.1 0.06 ± 9% perf-profile.children.cycles-pp.sctp_transport_burst_reset
0.45 +0.1 0.51 ± 7% perf-profile.children.cycles-pp.sched_clock_cpu
0.15 ± 2% +0.1 0.21 ± 9% perf-profile.children.cycles-pp.__mod_timer
0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.asm_sysvec_call_function_single
0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.error_entry
0.17 ± 2% +0.1 0.24 ± 6% perf-profile.children.cycles-pp.__netif_rx
0.15 ± 4% +0.1 0.21 ± 6% perf-profile.children.cycles-pp.enqueue_to_backlog
0.16 ± 4% +0.1 0.22 ± 6% perf-profile.children.cycles-pp.netif_rx_internal
0.27 ± 3% +0.1 0.34 ± 8% perf-profile.children.cycles-pp.sctp_sendmsg_parse
0.00 +0.1 0.07 ± 16% perf-profile.children.cycles-pp.perf_trace_sched_switch
0.46 ± 2% +0.1 0.53 ± 6% perf-profile.children.cycles-pp.sctp_check_transmitted
0.21 ± 2% +0.1 0.28 ± 8% perf-profile.children.cycles-pp.__put_user_nocheck_4
0.12 ± 36% +0.1 0.19 ± 25% perf-profile.children.cycles-pp.start_kernel
0.12 ± 36% +0.1 0.19 ± 25% perf-profile.children.cycles-pp.arch_call_rest_init
0.12 ± 36% +0.1 0.19 ± 25% perf-profile.children.cycles-pp.rest_init
0.24 ± 3% +0.1 0.31 ± 4% perf-profile.children.cycles-pp.kfree
0.25 ± 2% +0.1 0.33 ± 6% perf-profile.children.cycles-pp.sctp_datamsg_put
0.08 ± 4% +0.1 0.16 ± 9% perf-profile.children.cycles-pp.sctp_transport_reset_t3_rtx
0.59 +0.1 0.66 ± 3% perf-profile.children.cycles-pp.kmem_cache_alloc_node
0.28 ± 3% +0.1 0.36 ± 6% perf-profile.children.cycles-pp.__genradix_ptr
0.00 +0.1 0.08 ± 6% perf-profile.children.cycles-pp.check_preempt_wakeup
0.48 ± 2% +0.1 0.56 ± 3% perf-profile.children.cycles-pp.free_pcp_prepare
0.33 +0.1 0.41 ± 6% perf-profile.children.cycles-pp.kmalloc_trace
0.30 +0.1 0.38 ± 5% perf-profile.children.cycles-pp.sctp_association_put
0.00 +0.1 0.09 ± 7% perf-profile.children.cycles-pp.put_prev_entity
0.01 ±223% +0.1 0.10 ± 17% perf-profile.children.cycles-pp.ret_from_fork
0.01 ±223% +0.1 0.10 ± 17% perf-profile.children.cycles-pp.kthread
0.03 ±100% +0.1 0.12 ± 28% perf-profile.children.cycles-pp.arch_cpu_idle_enter
0.40 +0.1 0.49 ± 6% perf-profile.children.cycles-pp.sctp_ulpevent_receive_data
0.30 +0.1 0.40 ± 8% perf-profile.children.cycles-pp.__virt_addr_valid
0.00 +0.1 0.10 ± 17% perf-profile.children.cycles-pp.tick_sched_do_timer
0.42 ± 2% +0.1 0.52 ± 5% perf-profile.children.cycles-pp.__kmalloc
0.33 ± 2% +0.1 0.44 ± 8% perf-profile.children.cycles-pp.__sctp_packet_append_chunk
0.33 ± 2% +0.1 0.43 ± 7% perf-profile.children.cycles-pp.loopback_xmit
0.27 ± 4% +0.1 0.38 ± 7% perf-profile.children.cycles-pp.update_curr
0.00 +0.1 0.10 ± 7% perf-profile.children.cycles-pp._find_next_and_bit
0.01 ±223% +0.1 0.11 ± 27% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
0.26 ± 3% +0.1 0.37 ± 7% perf-profile.children.cycles-pp.sock_kfree_s
0.04 ± 45% +0.1 0.15 ± 35% perf-profile.children.cycles-pp.tick_nohz_irq_exit
0.00 +0.1 0.11 ± 21% perf-profile.children.cycles-pp.__intel_pmu_enable_all
0.11 +0.1 0.22 ± 29% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.28 ± 3% +0.1 0.39 ± 6% perf-profile.children.cycles-pp.__kmem_cache_free
0.29 ± 2% +0.1 0.40 ± 48% perf-profile.children.cycles-pp.security_socket_sendmsg
0.79 ± 2% +0.1 0.91 ± 4% perf-profile.children.cycles-pp.__copy_msghdr
0.00 +0.1 0.11 ± 17% perf-profile.children.cycles-pp.update_irq_load_avg
0.36 ± 2% +0.1 0.48 ± 4% perf-profile.children.cycles-pp.__fdget
0.34 ± 2% +0.1 0.45 ± 8% perf-profile.children.cycles-pp._raw_spin_trylock
0.36 ± 2% +0.1 0.48 ± 8% perf-profile.children.cycles-pp.dev_hard_start_xmit
0.43 ± 3% +0.1 0.55 ± 6% perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
0.02 ± 99% +0.1 0.15 ± 33% perf-profile.children.cycles-pp.__hrtimer_next_event_base
0.54 +0.1 0.66 ± 5% perf-profile.children.cycles-pp.sock_kmalloc
0.33 +0.1 0.46 ± 3% perf-profile.children.cycles-pp.read_tsc
0.00 +0.1 0.12 ± 40% perf-profile.children.cycles-pp.hrtimer_update_next_event
0.44 ± 2% +0.1 0.57 ± 4% perf-profile.children.cycles-pp.sockfd_lookup_light
0.00 +0.1 0.13 ± 42% perf-profile.children.cycles-pp.wait_for_lsr
0.50 +0.1 0.63 ± 4% perf-profile.children.cycles-pp.sctp_chunk_free
0.00 +0.1 0.13 ± 27% perf-profile.children.cycles-pp.ktime_get_update_offsets_now
0.00 +0.1 0.13 ± 43% perf-profile.children.cycles-pp.serial8250_console_write
0.00 +0.1 0.14 ± 43% perf-profile.children.cycles-pp.irq_work_single
0.00 +0.1 0.14 ± 43% perf-profile.children.cycles-pp.console_unlock
0.00 +0.1 0.14 ± 43% perf-profile.children.cycles-pp.console_emit_next_record
0.04 ± 71% +0.1 0.17 ± 67% perf-profile.children.cycles-pp.timekeeping_max_deferment
0.00 +0.1 0.14 ± 40% perf-profile.children.cycles-pp.irq_work_run_list
0.56 ± 2% +0.2 0.71 ± 4% perf-profile.children.cycles-pp.move_addr_to_kernel
0.48 ± 2% +0.2 0.65 ± 19% perf-profile.children.cycles-pp.aa_sk_perm
0.65 +0.2 0.81 ± 3% perf-profile.children.cycles-pp.__entry_text_start
0.54 +0.2 0.70 perf-profile.children.cycles-pp.__might_sleep
0.78 +0.2 0.94 ± 5% perf-profile.children.cycles-pp.sctp_chunkify
0.60 ± 2% +0.2 0.76 ± 5% perf-profile.children.cycles-pp.sctp_addto_chunk
0.66 +0.2 0.82 ± 3% perf-profile.children.cycles-pp.__might_fault
0.50 +0.2 0.66 ± 6% perf-profile.children.cycles-pp.memset_erms
0.60 ± 2% +0.2 0.77 ± 4% perf-profile.children.cycles-pp.__dev_queue_xmit
0.00 +0.2 0.17 ± 52% perf-profile.children.cycles-pp.rcu_pending
0.00 +0.2 0.17 ± 29% perf-profile.children.cycles-pp.__update_blocked_fair
0.57 ± 2% +0.2 0.75 ± 6% perf-profile.children.cycles-pp.sctp_packet_append_chunk
0.18 ± 3% +0.2 0.36 ± 5% perf-profile.children.cycles-pp.irqtime_account_irq
0.05 ± 8% +0.2 0.24 ± 21% perf-profile.children.cycles-pp.lapic_next_deadline
0.89 +0.2 1.08 ± 4% perf-profile.children.cycles-pp.kmem_cache_alloc
0.70 ± 3% +0.2 0.89 ± 8% perf-profile.children.cycles-pp.sctp_epaddr_lookup_transport
0.00 +0.2 0.20 ± 26% perf-profile.children.cycles-pp.arch_scale_freq_tick
0.73 ± 3% +0.2 0.93 ± 9% perf-profile.children.cycles-pp.sctp_endpoint_lookup_assoc
0.74 +0.2 0.94 ± 4% perf-profile.children.cycles-pp.__kmem_cache_alloc_node
0.06 ± 14% +0.2 0.28 ± 30% perf-profile.children.cycles-pp.update_blocked_averages
0.00 +0.2 0.21 ± 51% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.07 +0.2 0.30 ± 15% perf-profile.children.cycles-pp.native_irq_return_iret
0.00 +0.2 0.24 ± 8% perf-profile.children.cycles-pp.idle_cpu
0.06 ± 6% +0.2 0.31 ± 8% perf-profile.children.cycles-pp.sctp_assoc_rwnd_increase
0.09 ± 6% +0.3 0.34 ± 8% perf-profile.children.cycles-pp.switch_fpu_return
0.00 +0.3 0.26 ± 32% perf-profile.children.cycles-pp.run_rebalance_domains
1.04 ± 2% +0.3 1.31 ± 4% perf-profile.children.cycles-pp.iovec_from_user
0.05 ± 7% +0.3 0.32 ± 15% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.00 +0.3 0.27 ± 7% perf-profile.children.cycles-pp.restore_fpregs_from_fpstate
0.24 ± 3% +0.3 0.52 ± 18% perf-profile.children.cycles-pp.tick_nohz_next_event
0.06 ± 7% +0.3 0.34 ± 15% perf-profile.children.cycles-pp.sctp_wait_for_sndbuf
0.18 ± 5% +0.3 0.46 ± 6% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
1.20 ± 2% +0.3 1.51 ± 4% perf-profile.children.cycles-pp.__import_iovec
1.24 ± 2% +0.3 1.56 ± 4% perf-profile.children.cycles-pp.import_iovec
0.36 ± 4% +0.3 0.69 ± 14% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.35 ± 2% +0.4 0.71 ± 5% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.09 ± 6% +0.4 0.45 ± 10% perf-profile.children.cycles-pp.rebalance_domains
0.09 ± 30% +0.4 0.48 ± 32% perf-profile.children.cycles-pp.tick_irq_enter
0.10 ± 27% +0.4 0.49 ± 31% perf-profile.children.cycles-pp.irq_enter_rcu
0.65 ± 2% +0.4 1.08 ± 6% perf-profile.children.cycles-pp.sctp_packet_transmit_chunk
2.88 +0.4 3.31 ± 4% perf-profile.children.cycles-pp.sctp_outq_flush_data
0.18 ± 12% +0.5 0.63 ± 21% perf-profile.children.cycles-pp.clockevents_program_event
0.71 ± 7% +0.5 1.20 ± 27% perf-profile.children.cycles-pp.ktime_get
0.01 ±223% +0.5 0.51 ± 4% perf-profile.children.cycles-pp.switch_mm_irqs_off
1.84 ± 2% +0.5 2.37 ± 2% perf-profile.children.cycles-pp.schedule_timeout
0.17 ± 4% +0.6 0.78 ± 16% perf-profile.children.cycles-pp.scheduler_tick
2.44 +0.7 3.10 ± 3% perf-profile.children.cycles-pp._copy_from_user
2.95 +0.7 3.64 ± 3% perf-profile.children.cycles-pp.copy_msghdr_from_user
0.23 ± 10% +0.7 0.96 ± 2% perf-profile.children.cycles-pp.newidle_balance
0.12 ± 9% +0.8 0.89 ± 2% perf-profile.children.cycles-pp.update_sg_lb_stats
1.53 ± 2% +0.8 2.35 ± 2% perf-profile.children.cycles-pp.schedule
0.13 ± 9% +0.9 0.98 ± 2% perf-profile.children.cycles-pp.update_sd_lb_stats
3.45 +0.9 4.31 ± 3% perf-profile.children.cycles-pp.copyin
0.14 ± 10% +0.9 1.00 ± 2% perf-profile.children.cycles-pp.find_busiest_group
0.70 ± 2% +0.9 1.58 ± 19% perf-profile.children.cycles-pp.menu_select
0.47 ± 4% +0.9 1.36 ± 2% perf-profile.children.cycles-pp.pick_next_task_fair
2.38 +0.9 3.28 ± 3% perf-profile.children.cycles-pp.__schedule
3.70 +0.9 4.65 ± 2% perf-profile.children.cycles-pp._copy_from_iter
0.38 +1.0 1.36 ± 4% perf-profile.children.cycles-pp.__cond_resched
0.18 ± 4% +1.0 1.16 ± 14% perf-profile.children.cycles-pp.__irq_exit_rcu
0.16 ± 8% +1.0 1.20 ± 3% perf-profile.children.cycles-pp.load_balance
4.16 +1.0 5.20 ± 2% perf-profile.children.cycles-pp.sctp_user_addto_chunk
0.24 ± 5% +1.2 1.44 ± 25% perf-profile.children.cycles-pp.update_process_times
0.25 ± 6% +1.3 1.56 ± 28% perf-profile.children.cycles-pp.tick_sched_handle
9.70 +1.3 11.05 ± 3% perf-profile.children.cycles-pp.sctp_datamsg_from_user
0.29 ± 5% +1.4 1.74 ± 25% perf-profile.children.cycles-pp.tick_sched_timer
0.38 ± 4% +2.2 2.57 ± 25% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.63 ± 5% +3.0 3.63 ± 18% perf-profile.children.cycles-pp.hrtimer_interrupt
0.64 ± 5% +3.1 3.72 ± 19% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.99 ± 7% +4.8 5.79 ± 18% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
1.24 ± 5% +5.3 6.52 ± 16% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
17.20 +6.5 23.70 ± 13% perf-profile.children.cycles-pp.start_secondary
17.32 +6.6 23.88 ± 13% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
17.32 +6.6 23.88 ± 13% perf-profile.children.cycles-pp.cpu_startup_entry
17.29 +6.6 23.88 ± 13% perf-profile.children.cycles-pp.do_idle
13.26 +8.3 21.54 ± 12% perf-profile.children.cycles-pp.cpuidle_enter_state
13.27 +8.3 21.56 ± 12% perf-profile.children.cycles-pp.cpuidle_enter
14.17 +9.3 23.44 ± 13% perf-profile.children.cycles-pp.cpuidle_idle_call
3.22 ± 4% +10.7 13.95 ± 12% perf-profile.children.cycles-pp.intel_idle
13.62 -2.9 10.68 ± 3% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
2.42 ± 2% -2.0 0.45 ± 71% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
1.36 ± 2% -1.3 0.06 ± 8% perf-profile.self.cycles-pp.__slab_free
1.03 ± 3% -0.6 0.43 ± 10% perf-profile.self.cycles-pp.__mod_memcg_state
0.63 ± 2% -0.5 0.11 ± 11% perf-profile.self.cycles-pp.__skb_datagram_iter
0.61 ± 2% -0.5 0.16 ± 23% perf-profile.self.cycles-pp.ipv4_dst_check
1.11 ± 2% -0.4 0.66 ± 6% perf-profile.self.cycles-pp.rmqueue
0.74 ± 2% -0.4 0.30 ± 8% perf-profile.self.cycles-pp.free_unref_page_commit
1.09 -0.4 0.66 ± 7% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
1.30 -0.4 0.90 ± 5% perf-profile.self.cycles-pp.kmem_cache_free
0.73 -0.4 0.34 ± 7% perf-profile.self.cycles-pp.__sk_mem_reduce_allocated
0.79 -0.4 0.42 ± 7% perf-profile.self.cycles-pp._raw_spin_lock
0.44 ± 2% -0.3 0.14 ± 7% perf-profile.self.cycles-pp.sctp_inet_skb_msgname
0.45 -0.3 0.16 ± 5% perf-profile.self.cycles-pp.sctp_skb_recv_datagram
0.86 -0.3 0.59 ± 8% perf-profile.self.cycles-pp.sctp_recvmsg
0.47 -0.3 0.21 ± 9% perf-profile.self.cycles-pp.__mod_node_page_state
0.32 -0.3 0.07 ± 11% perf-profile.self.cycles-pp.schedule_timeout
0.56 ± 2% -0.2 0.31 ± 5% perf-profile.self.cycles-pp.sctp_eat_data
0.58 ± 2% -0.2 0.34 ± 13% perf-profile.self.cycles-pp.__sk_mem_raise_allocated
0.60 -0.2 0.37 ± 6% perf-profile.self.cycles-pp.get_page_from_freelist
0.53 ± 2% -0.2 0.30 ± 7% perf-profile.self.cycles-pp.try_charge_memcg
0.55 ± 3% -0.2 0.32 ± 17% perf-profile.self.cycles-pp.dst_release
0.50 -0.2 0.30 ± 5% perf-profile.self.cycles-pp.check_heap_object
0.31 -0.2 0.13 ± 12% perf-profile.self.cycles-pp.sctp_hash_cmp
0.36 -0.2 0.19 ± 11% perf-profile.self.cycles-pp.sctp_data_ready
0.57 ± 4% -0.2 0.41 ± 9% perf-profile.self.cycles-pp.__copy_skb_header
0.52 ± 2% -0.1 0.40 ± 4% perf-profile.self.cycles-pp.__zone_watermark_ok
0.31 ± 2% -0.1 0.20 ± 5% perf-profile.self.cycles-pp.sctp_rcv
0.17 ± 2% -0.1 0.06 ± 14% perf-profile.self.cycles-pp.enqueue_entity
0.18 ± 4% -0.1 0.08 ± 8% perf-profile.self.cycles-pp.consume_skb
0.13 ± 5% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.__free_one_page
0.24 ± 2% -0.1 0.15 ± 22% perf-profile.self.cycles-pp.update_rq_clock
0.32 -0.1 0.24 ± 4% perf-profile.self.cycles-pp.__switch_to
0.18 -0.1 0.09 ± 10% perf-profile.self.cycles-pp.___perf_sw_event
0.42 -0.1 0.33 ± 4% perf-profile.self.cycles-pp.skb_set_owner_w
0.38 ± 2% -0.1 0.30 ± 4% perf-profile.self.cycles-pp.skb_release_data
0.15 ± 4% -0.1 0.07 ± 24% perf-profile.self.cycles-pp.do_idle
0.20 ± 4% -0.1 0.14 ± 8% perf-profile.self.cycles-pp.sctp_ulpq_tail_event
0.29 ± 3% -0.1 0.24 ± 7% perf-profile.self.cycles-pp.__schedule
0.11 ± 6% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.enqueue_task_fair
0.14 ± 3% -0.0 0.10 ± 7% perf-profile.self.cycles-pp.__build_skb_around
0.12 ± 4% -0.0 0.07 ± 10% perf-profile.self.cycles-pp.__wrgsbase_inactive
0.09 ± 5% -0.0 0.04 ± 44% perf-profile.self.cycles-pp.try_to_wake_up
0.19 ± 3% -0.0 0.15 ± 7% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.14 ± 3% -0.0 0.10 ± 10% perf-profile.self.cycles-pp.__sys_recvmsg
0.26 ± 2% -0.0 0.22 ± 3% perf-profile.self.cycles-pp.sctp_assoc_bh_rcv
0.23 ± 4% -0.0 0.19 ± 7% perf-profile.self.cycles-pp.__copy_msghdr
0.08 ± 6% -0.0 0.04 ± 45% perf-profile.self.cycles-pp.prepare_task_switch
0.10 ± 5% -0.0 0.07 ± 10% perf-profile.self.cycles-pp.security_socket_recvmsg
0.10 ± 3% -0.0 0.07 ± 9% perf-profile.self.cycles-pp.check_new_pages
0.08 ± 6% -0.0 0.06 ± 9% perf-profile.self.cycles-pp.finish_task_switch
0.06 ± 6% +0.0 0.08 ± 9% perf-profile.self.cycles-pp.kmalloc_reserve
0.06 ± 6% +0.0 0.08 ± 6% perf-profile.self.cycles-pp.__sk_mem_schedule
0.16 ± 2% +0.0 0.18 ± 2% perf-profile.self.cycles-pp.sctp_inq_pop
0.06 ± 7% +0.0 0.08 ± 5% perf-profile.self.cycles-pp.sctp_epaddr_lookup_transport
0.12 ± 3% +0.0 0.14 ± 5% perf-profile.self.cycles-pp.sock_kmalloc
0.09 ± 6% +0.0 0.11 ± 10% perf-profile.self.cycles-pp.__sys_sendmsg
0.07 ± 6% +0.0 0.09 ± 10% perf-profile.self.cycles-pp.syscall_enter_from_user_mode
0.07 ± 8% +0.0 0.09 ± 11% perf-profile.self.cycles-pp.__sctp_write_space
0.09 ± 5% +0.0 0.12 ± 6% perf-profile.self.cycles-pp.___sys_sendmsg
0.08 ± 6% +0.0 0.10 ± 6% perf-profile.self.cycles-pp.exit_to_user_mode_prepare
0.11 ± 3% +0.0 0.13 ± 4% perf-profile.self.cycles-pp.sctp_sock_rfree
0.08 ± 5% +0.0 0.10 ± 9% perf-profile.self.cycles-pp.lock_sock_nested
0.08 ± 4% +0.0 0.10 ± 9% perf-profile.self.cycles-pp.move_addr_to_kernel
0.07 ± 5% +0.0 0.09 ± 10% perf-profile.self.cycles-pp.sctp_outq_flush_ctrl
0.11 ± 5% +0.0 0.13 ± 5% perf-profile.self.cycles-pp.check_stack_object
0.08 +0.0 0.10 ± 7% perf-profile.self.cycles-pp.sctp_packet_transmit_chunk
0.07 ± 5% +0.0 0.09 ± 7% perf-profile.self.cycles-pp.__kmalloc_node_track_caller
0.05 ± 8% +0.0 0.08 ± 13% perf-profile.self.cycles-pp.sctp_assoc_rwnd_increase
0.12 +0.0 0.14 ± 8% perf-profile.self.cycles-pp.__softirqentry_text_start
0.08 ± 6% +0.0 0.10 ± 8% perf-profile.self.cycles-pp.ip_rcv_core
0.07 ± 5% +0.0 0.10 ± 25% perf-profile.self.cycles-pp.sock_sendmsg
0.12 ± 3% +0.0 0.15 ± 4% perf-profile.self.cycles-pp.__mod_lruvec_page_state
0.15 ± 3% +0.0 0.18 ± 5% perf-profile.self.cycles-pp.update_load_avg
0.10 ± 4% +0.0 0.13 ± 7% perf-profile.self.cycles-pp.process_backlog
0.06 ± 11% +0.0 0.09 ± 11% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
0.11 ± 4% +0.0 0.14 ± 8% perf-profile.self.cycles-pp.__netif_receive_skb_core
0.08 ± 4% +0.0 0.11 ± 7% perf-profile.self.cycles-pp.sctp_chunk_assign_ssn
0.20 ± 3% +0.0 0.23 ± 7% perf-profile.self.cycles-pp.sctp_check_transmitted
0.05 ± 8% +0.0 0.08 ± 11% perf-profile.self.cycles-pp.sctp_sched_fcfs_dequeue
0.05 +0.0 0.08 ± 12% perf-profile.self.cycles-pp.inet_recvmsg
0.05 +0.0 0.08 ± 10% perf-profile.self.cycles-pp.enqueue_to_backlog
0.12 +0.0 0.15 ± 3% perf-profile.self.cycles-pp.copy_msghdr_from_user
0.11 ± 4% +0.0 0.14 ± 9% perf-profile.self.cycles-pp._sctp_make_chunk
0.19 ± 3% +0.0 0.22 ± 8% perf-profile.self.cycles-pp.free_unref_page
0.14 ± 3% +0.0 0.18 ± 5% perf-profile.self.cycles-pp.sctp_user_addto_chunk
0.08 ± 6% +0.0 0.11 ± 9% perf-profile.self.cycles-pp.ip_protocol_deliver_rcu
0.08 ± 8% +0.0 0.11 ± 6% perf-profile.self.cycles-pp.sctp_cmp_addr_exact
0.13 ± 3% +0.0 0.16 ± 6% perf-profile.self.cycles-pp.__ip_queue_xmit
0.12 ± 4% +0.0 0.15 ± 5% perf-profile.self.cycles-pp.__might_fault
0.16 ± 3% +0.0 0.19 ± 6% perf-profile.self.cycles-pp.__put_user_nocheck_8
0.05 +0.0 0.08 ± 11% perf-profile.self.cycles-pp.pick_next_task_fair
0.13 ± 4% +0.0 0.16 ± 7% perf-profile.self.cycles-pp.skb_put
0.12 ± 5% +0.0 0.16 ± 7% perf-profile.self.cycles-pp.os_xsave
0.36 +0.0 0.40 ± 6% perf-profile.self.cycles-pp.native_sched_clock
0.27 ± 2% +0.0 0.31 ± 3% perf-profile.self.cycles-pp.recvmsg
0.18 ± 2% +0.0 0.21 ± 6% perf-profile.self.cycles-pp.__import_iovec
0.08 ± 5% +0.0 0.12 ± 9% perf-profile.self.cycles-pp.sctp_make_datafrag_empty
0.18 ± 2% +0.0 0.21 ± 6% perf-profile.self.cycles-pp._copy_to_iter
0.22 ± 4% +0.0 0.26 ± 7% perf-profile.self.cycles-pp.do_syscall_64
0.10 ± 4% +0.0 0.14 ± 24% perf-profile.self.cycles-pp.sctp_hash_key
0.13 ± 3% +0.0 0.17 ± 6% perf-profile.self.cycles-pp.sctp_ulpevent_free
0.11 ± 3% +0.0 0.16 ± 10% perf-profile.self.cycles-pp.sctp_outq_sack
0.11 ± 6% +0.0 0.16 ± 6% perf-profile.self.cycles-pp.update_curr
0.12 ± 4% +0.0 0.17 ± 4% perf-profile.self.cycles-pp.ip_finish_output2
0.09 ± 7% +0.0 0.14 ± 11% perf-profile.self.cycles-pp.sock_kfree_s
0.19 ± 3% +0.0 0.24 ± 5% perf-profile.self.cycles-pp.__local_bh_enable_ip
0.32 ± 2% +0.0 0.36 ± 5% perf-profile.self.cycles-pp.__alloc_pages
0.23 ± 2% +0.0 0.27 ± 5% perf-profile.self.cycles-pp.__entry_text_start
0.02 ±141% +0.0 0.06 ± 11% perf-profile.self.cycles-pp.sctp_tsnmap_check
0.01 ±223% +0.0 0.06 ± 9% perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
0.16 ± 2% +0.0 0.21 ± 10% perf-profile.self.cycles-pp.__sctp_packet_append_chunk
0.16 +0.0 0.21 ± 5% perf-profile.self.cycles-pp._copy_from_iter
0.14 ± 5% +0.0 0.19 ± 5% perf-profile.self.cycles-pp.iovec_from_user
0.22 ± 4% +0.1 0.27 ± 5% perf-profile.self.cycles-pp.sctp_chunk_hold
0.31 ± 5% +0.1 0.36 ± 8% perf-profile.self.cycles-pp.sctp_transport_hold
0.34 ± 3% +0.1 0.40 ± 4% perf-profile.self.cycles-pp.sctp_packet_pack
0.21 ± 2% +0.1 0.26 ± 5% perf-profile.self.cycles-pp.____sys_sendmsg
0.21 ± 3% +0.1 0.26 ± 6% perf-profile.self.cycles-pp.sctp_association_hold
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.skb_pull
0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.sctp_sendmsg_check_sflags
0.20 ± 3% +0.1 0.25 ± 5% perf-profile.self.cycles-pp.kfree
0.00 +0.1 0.06 ± 13% perf-profile.self.cycles-pp.error_entry
0.17 ± 2% +0.1 0.23 ± 8% perf-profile.self.cycles-pp.mem_cgroup_uncharge_skmem
0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.loopback_xmit
0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.ip_local_deliver_finish
0.12 ± 4% +0.1 0.18 ± 9% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.24 +0.1 0.30 ± 6% perf-profile.self.cycles-pp.sctp_ulpevent_receive_data
0.23 ± 3% +0.1 0.29 ± 6% perf-profile.self.cycles-pp.sctp_datamsg_put
0.38 +0.1 0.44 ± 6% perf-profile.self.cycles-pp.send_sctp_stream_1toMany
0.35 ± 2% +0.1 0.41 ± 4% perf-profile.self.cycles-pp.kmem_cache_alloc_node
0.06 ± 7% +0.1 0.13 ± 7% perf-profile.self.cycles-pp.syscall_exit_to_user_mode
0.23 ± 3% +0.1 0.29 ± 7% perf-profile.self.cycles-pp.__genradix_ptr
0.21 ± 2% +0.1 0.27 ± 8% perf-profile.self.cycles-pp.__put_user_nocheck_4
0.00 +0.1 0.06 ± 19% perf-profile.self.cycles-pp.perf_trace_sched_switch
0.00 +0.1 0.06 ± 17% perf-profile.self.cycles-pp.update_sd_lb_stats
0.00 +0.1 0.07 ± 11% perf-profile.self.cycles-pp.__netif_receive_skb_one_core
0.26 ± 3% +0.1 0.33 ± 8% perf-profile.self.cycles-pp.sctp_sendmsg_parse
0.00 +0.1 0.07 ± 25% perf-profile.self.cycles-pp.intel_idle
0.01 ±223% +0.1 0.08 ± 34% perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
0.00 +0.1 0.07 ± 18% perf-profile.self.cycles-pp.tick_sched_do_timer
0.23 ± 5% +0.1 0.30 ± 3% perf-profile.self.cycles-pp.sctp_packet_append_chunk
0.29 ± 3% +0.1 0.36 ± 4% perf-profile.self.cycles-pp.sctp_do_sm
0.00 +0.1 0.07 ± 15% perf-profile.self.cycles-pp.load_balance
0.22 ± 2% +0.1 0.30 ± 4% perf-profile.self.cycles-pp.sctp_outq_flush
0.29 +0.1 0.37 ± 5% perf-profile.self.cycles-pp.sctp_association_put
0.06 ± 11% +0.1 0.14 ± 15% perf-profile.self.cycles-pp.irqtime_account_irq
0.47 ± 2% +0.1 0.55 ± 4% perf-profile.self.cycles-pp.free_pcp_prepare
0.00 +0.1 0.09 ± 40% perf-profile.self.cycles-pp.ktime_get_update_offsets_now
0.30 ± 2% +0.1 0.40 ± 4% perf-profile.self.cycles-pp._copy_from_user
0.29 +0.1 0.38 ± 9% perf-profile.self.cycles-pp.__virt_addr_valid
0.00 +0.1 0.09 ± 32% perf-profile.self.cycles-pp.hrtimer_interrupt
0.26 ± 2% +0.1 0.36 ± 8% perf-profile.self.cycles-pp.sctp_wfree
0.00 +0.1 0.10 ± 8% perf-profile.self.cycles-pp._find_next_and_bit
0.34 +0.1 0.44 ± 7% perf-profile.self.cycles-pp.__check_object_size
0.29 ± 3% +0.1 0.39 ± 5% perf-profile.self.cycles-pp.sendmsg
0.00 +0.1 0.10 ± 7% perf-profile.self.cycles-pp.__release_sock
0.00 +0.1 0.10 ± 12% perf-profile.self.cycles-pp.asm_sysvec_apic_timer_interrupt
0.29 ± 5% +0.1 0.39 ± 25% perf-profile.self.cycles-pp.aa_sk_perm
0.56 ± 2% +0.1 0.66 ± 4% perf-profile.self.cycles-pp.sctp_sendmsg
0.32 ± 2% +0.1 0.42 ± 4% perf-profile.self.cycles-pp.mem_cgroup_charge_skmem
0.27 ± 3% +0.1 0.37 ± 6% perf-profile.self.cycles-pp.__kmem_cache_free
0.11 ± 3% +0.1 0.21 ± 31% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.32 ± 2% +0.1 0.43 ± 8% perf-profile.self.cycles-pp._raw_spin_trylock
0.00 +0.1 0.10 ± 33% perf-profile.self.cycles-pp.__update_blocked_fair
0.46 +0.1 0.56 ± 4% perf-profile.self.cycles-pp.kmem_cache_alloc
0.23 ± 2% +0.1 0.34 ± 4% perf-profile.self.cycles-pp.__cond_resched
0.00 +0.1 0.11 ± 32% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler
0.00 +0.1 0.11 ± 21% perf-profile.self.cycles-pp.__intel_pmu_enable_all
0.40 ± 3% +0.1 0.51 ± 9% perf-profile.self.cycles-pp.sctp_cmd_interpreter
0.00 +0.1 0.11 ± 19% perf-profile.self.cycles-pp.update_irq_load_avg
0.35 ± 3% +0.1 0.46 ± 4% perf-profile.self.cycles-pp.__fdget
0.32 +0.1 0.44 ± 3% perf-profile.self.cycles-pp.read_tsc
0.42 +0.1 0.54 ± 4% perf-profile.self.cycles-pp.__alloc_skb
0.41 ± 2% +0.1 0.52 ± 3% perf-profile.self.cycles-pp.sctp_sendmsg_to_asoc
0.42 ± 2% +0.1 0.54 ± 5% perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
0.01 ±223% +0.1 0.13 ± 23% perf-profile.self.cycles-pp.tick_nohz_next_event
0.01 ±223% +0.1 0.13 ± 32% perf-profile.self.cycles-pp.__hrtimer_next_event_base
0.04 ± 71% +0.1 0.16 ± 71% perf-profile.self.cycles-pp.timekeeping_max_deferment
0.44 +0.1 0.58 ± 4% perf-profile.self.cycles-pp.__kmem_cache_alloc_node
0.55 +0.1 0.69 ± 3% perf-profile.self.cycles-pp.sctp_chunk_put
0.44 ± 2% +0.1 0.59 perf-profile.self.cycles-pp.__might_sleep
0.48 ± 2% +0.2 0.64 ± 5% perf-profile.self.cycles-pp.memset_erms
0.47 +0.2 0.64 ± 6% perf-profile.self.cycles-pp.sctp_outq_flush_data
0.57 +0.2 0.75 ± 6% perf-profile.self.cycles-pp._raw_spin_lock_bh
0.05 +0.2 0.24 ± 21% perf-profile.self.cycles-pp.lapic_next_deadline
0.00 +0.2 0.20 ± 26% perf-profile.self.cycles-pp.arch_scale_freq_tick
0.00 +0.2 0.23 ± 7% perf-profile.self.cycles-pp.idle_cpu
0.07 +0.2 0.30 ± 15% perf-profile.self.cycles-pp.native_irq_return_iret
0.63 +0.2 0.88 ± 3% perf-profile.self.cycles-pp.sctp_datamsg_from_user
0.00 +0.3 0.27 ± 7% perf-profile.self.cycles-pp.restore_fpregs_from_fpstate
0.38 ± 13% +0.4 0.78 ± 42% perf-profile.self.cycles-pp.ktime_get
0.24 ± 3% +0.5 0.71 ± 30% perf-profile.self.cycles-pp.menu_select
0.00 +0.5 0.50 ± 4% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.10 ± 11% +0.6 0.70 perf-profile.self.cycles-pp.update_sg_lb_stats
0.28 ± 3% +0.7 0.97 ± 14% perf-profile.self.cycles-pp.cpuidle_enter_state
***************************************************************************************************
lkp-icl-2sp6: 128 threads 2 sockets Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz (Ice Lake) with 128G memory
=========================================================================================
class/compiler/cpufreq_governor/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime:
network/gcc-11/performance/x86_64-rhel-8.3/100%/debian-11.1-x86_64-20220510.cgz/lkp-icl-2sp6/sctp/stress-ng/60s
commit:
c36eae5a86 ("sched/psi: Fix possible missing or delayed pending event")
b6aabb01e3 ("sched: consider WF_SYNC to find idle siblings")
c36eae5a86d15a69 b6aabb01e3004b0285d953761e1
---------------- ---------------------------
%stddev %change %stddev
\ | \
9399721 -48.4% 4850253 ± 5% stress-ng.sctp.ops
156661 -48.4% 80837 ± 5% stress-ng.sctp.ops_per_sec
514.00 ± 3% +41004.5% 211277 ± 10% stress-ng.time.involuntary_context_switches
59.67 ± 2% -51.1% 29.17 ± 10% stress-ng.time.percent_of_cpu_this_job_got
36.02 ± 2% -50.6% 17.80 ± 10% stress-ng.time.system_time
970641 -63.3% 356134 ± 5% stress-ng.time.voluntary_context_switches
31732 -60.7% 12482 ± 3% vmstat.system.cs
0.43 -0.2 0.22 ± 9% mpstat.cpu.all.sys%
0.10 ± 3% -0.0 0.09 ± 9% mpstat.cpu.all.usr%
1544 ± 2% +16.4% 1797 ± 11% numa-meminfo.node0.Active
3914 ± 5% -16.9% 3253 ± 12% numa-meminfo.node1.Active
3834 ± 5% -15.6% 3237 ± 12% numa-meminfo.node1.Active(anon)
1154373 ± 2% -31.7% 788904 ± 8% numa-numastat.node0.local_node
1239775 ± 2% -32.6% 835766 ± 7% numa-numastat.node0.numa_hit
1300505 ± 2% -40.6% 772809 ± 12% numa-numastat.node1.local_node
1330886 ± 2% -36.8% 841742 ± 10% numa-numastat.node1.numa_hit
213291 ± 17% -99.3% 1496 ± 33% turbostat.C1
0.35 ± 78% -0.3 0.00 ±141% turbostat.C1%
389879 ±129% +1006.6% 4314548 ± 91% turbostat.C6
16517 ± 3% -97.4% 425.50 ± 54% turbostat.POLL
1239630 ± 2% -32.6% 835988 ± 7% numa-vmstat.node0.numa_hit
1154228 ± 2% -31.6% 789126 ± 8% numa-vmstat.node0.numa_local
958.33 ± 5% -15.6% 809.17 ± 12% numa-vmstat.node1.nr_active_anon
958.33 ± 5% -15.6% 809.17 ± 12% numa-vmstat.node1.nr_zone_active_anon
1330563 ± 2% -36.7% 841771 ± 10% numa-vmstat.node1.numa_hit
1300182 ± 2% -40.6% 772838 ± 12% numa-vmstat.node1.numa_local
2.67 ± 35% +59431.2% 1587 ±116% proc-vmstat.numa_hint_faults_local
2580484 -34.9% 1679518 ± 2% proc-vmstat.numa_hit
2464701 -36.6% 1563724 ± 2% proc-vmstat.numa_local
14670 ± 55% +64.3% 24106 ± 21% proc-vmstat.numa_pte_updates
18610165 -48.5% 9577564 ± 4% proc-vmstat.pgalloc_normal
18478395 -48.9% 9435687 ± 4% proc-vmstat.pgfree
18554 -3.3% 17937 proc-vmstat.pgreuse
6103 ± 43% -72.4% 1682 ± 36% sched_debug.cfs_rq:/.load.avg
50624 ± 58% -81.4% 9439 ± 83% sched_debug.cfs_rq:/.load.stddev
853.50 ± 14% +3161.6% 27837 ±136% sched_debug.cfs_rq:/.load_avg.max
965.00 ± 5% +52.0% 1466 ± 18% sched_debug.cfs_rq:/.runnable_avg.max
223.02 ± 23% +36.3% 303.95 ± 15% sched_debug.cfs_rq:/.runnable_avg.stddev
964.33 ± 5% +51.4% 1460 ± 18% sched_debug.cfs_rq:/.util_avg.max
222.86 ± 23% +35.4% 301.73 ± 15% sched_debug.cfs_rq:/.util_avg.stddev
351971 ± 38% -41.6% 205390 ± 25% sched_debug.cpu.avg_idle.stddev
4.02 ± 8% +18.1% 4.75 ± 5% sched_debug.cpu.clock.stddev
950.00 -100.0% 0.00 sched_debug.rt_rq:/.rt_runtime.avg
950.00 -100.0% 0.00 sched_debug.rt_rq:/.rt_runtime.max
950.00 -100.0% 0.00 sched_debug.rt_rq:/.rt_runtime.min
7.447e+08 -27.7% 5.387e+08 ± 3% perf-stat.i.branch-instructions
0.79 ± 20% +2.6 3.35 ± 75% perf-stat.i.branch-miss-rate%
4421153 ± 3% -29.9% 3100372 ± 5% perf-stat.i.cache-misses
32967 -61.7% 12612 ± 4% perf-stat.i.context-switches
1.67 +50.8% 2.52 ± 21% perf-stat.i.cpi
313.73 ± 3% +253.6% 1109 ± 4% perf-stat.i.cpu-migrations
1477 ± 4% +55.0% 2289 ± 19% perf-stat.i.cycles-between-cache-misses
0.06 ± 42% +0.3 0.36 ± 77% perf-stat.i.dTLB-load-miss-rate%
1.027e+09 -28.5% 7.34e+08 ± 2% perf-stat.i.dTLB-loads
5.512e+08 -31.7% 3.762e+08 ± 2% perf-stat.i.dTLB-stores
3.762e+09 -27.1% 2.742e+09 ± 3% perf-stat.i.instructions
0.61 -29.2% 0.43 ± 21% perf-stat.i.ipc
393.08 ± 7% -73.0% 106.27 ± 76% perf-stat.i.metric.K/sec
18.00 -26.4% 13.25 ± 5% perf-stat.i.metric.M/sec
51.73 ± 4% +22.8 74.57 ± 6% perf-stat.i.node-load-miss-rate%
225775 ± 5% +85.3% 418438 ± 3% perf-stat.i.node-load-misses
14.54 ± 2% +17.0 31.53 ± 11% perf-stat.i.node-store-miss-rate%
213471 ± 5% +19.7% 255432 ± 7% perf-stat.i.node-store-misses
1489600 -47.6% 780285 ± 7% perf-stat.i.node-stores
1.03 ± 14% +2.2 3.19 ± 64% perf-stat.overall.branch-miss-rate%
1.61 +42.0% 2.29 ± 19% perf-stat.overall.cpi
1370 +46.6% 2008 ± 17% perf-stat.overall.cycles-between-cache-misses
0.06 ± 42% +0.3 0.31 ± 76% perf-stat.overall.dTLB-load-miss-rate%
0.62 -26.7% 0.46 ± 20% perf-stat.overall.ipc
53.37 ± 3% +20.7 74.08 ± 5% perf-stat.overall.node-load-miss-rate%
12.54 ± 4% +12.2 24.74 ± 9% perf-stat.overall.node-store-miss-rate%
7.319e+08 -27.8% 5.286e+08 ± 3% perf-stat.ps.branch-instructions
4344607 ± 3% -29.8% 3051326 ± 5% perf-stat.ps.cache-misses
32395 -61.7% 12403 ± 3% perf-stat.ps.context-switches
308.45 ± 4% +253.7% 1091 ± 4% perf-stat.ps.cpu-migrations
1.009e+09 -28.6% 7.207e+08 ± 2% perf-stat.ps.dTLB-loads
5.417e+08 -31.8% 3.695e+08 ± 2% perf-stat.ps.dTLB-stores
3.697e+09 -27.2% 2.691e+09 ± 3% perf-stat.ps.instructions
222119 ± 5% +85.2% 411273 ± 3% perf-stat.ps.node-load-misses
209895 ± 5% +19.8% 251498 ± 7% perf-stat.ps.node-store-misses
1463189 -47.5% 768638 ± 7% perf-stat.ps.node-stores
2.374e+11 -26.9% 1.736e+11 ± 2% perf-stat.total.instructions
25.09 ± 2% -14.3 10.78 ± 29% perf-profile.calltrace.cycles-pp.stress_sctp
22.22 ± 2% -12.6 9.64 ± 30% perf-profile.calltrace.cycles-pp.sendmsg.stress_sctp
21.90 ± 2% -12.4 9.50 ± 29% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.sendmsg.stress_sctp
21.80 ± 2% -12.3 9.47 ± 30% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendmsg.stress_sctp
21.67 ± 2% -12.3 9.40 ± 30% perf-profile.calltrace.cycles-pp.__sys_sendmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendmsg.stress_sctp
21.51 ± 2% -12.2 9.32 ± 30% perf-profile.calltrace.cycles-pp.___sys_sendmsg.__sys_sendmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendmsg
20.62 ± 2% -11.7 8.96 ± 30% perf-profile.calltrace.cycles-pp.____sys_sendmsg.___sys_sendmsg.__sys_sendmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe
19.56 ± 2% -11.1 8.47 ± 30% perf-profile.calltrace.cycles-pp.sock_sendmsg.____sys_sendmsg.___sys_sendmsg.__sys_sendmsg.do_syscall_64
19.33 ± 2% -11.0 8.37 ± 30% perf-profile.calltrace.cycles-pp.sctp_sendmsg.sock_sendmsg.____sys_sendmsg.___sys_sendmsg.__sys_sendmsg
11.82 ± 4% -8.0 3.85 ± 29% perf-profile.calltrace.cycles-pp.recvmsg
11.46 ± 4% -7.8 3.68 ± 29% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.recvmsg
11.40 ± 4% -7.8 3.65 ± 29% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.recvmsg
11.23 ± 5% -7.7 3.58 ± 29% perf-profile.calltrace.cycles-pp.__sys_recvmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.recvmsg
11.04 ± 4% -7.5 3.50 ± 29% perf-profile.calltrace.cycles-pp.___sys_recvmsg.__sys_recvmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe.recvmsg
13.15 ± 3% -7.2 5.97 ± 32% perf-profile.calltrace.cycles-pp.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg.____sys_sendmsg.___sys_sendmsg
10.28 ± 4% -7.2 3.13 ± 29% perf-profile.calltrace.cycles-pp.____sys_recvmsg.___sys_recvmsg.__sys_recvmsg.do_syscall_64.entry_SYSCALL_64_after_hwframe
9.78 ± 4% -6.9 2.89 ± 29% perf-profile.calltrace.cycles-pp.inet_recvmsg.____sys_recvmsg.___sys_recvmsg.__sys_recvmsg.do_syscall_64
9.74 ± 4% -6.9 2.85 ± 29% perf-profile.calltrace.cycles-pp.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg.__sys_recvmsg
6.57 ± 5% -4.1 2.45 ± 30% perf-profile.calltrace.cycles-pp.sctp_primitive_SEND.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg.____sys_sendmsg
5.83 ± 5% -3.7 2.14 ± 31% perf-profile.calltrace.cycles-pp.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_primitive_SEND.sctp_sendmsg_to_asoc
5.48 ± 2% -3.5 2.00 ± 26% perf-profile.calltrace.cycles-pp.sctp_backlog_rcv.__release_sock.release_sock.sctp_sendmsg.sock_sendmsg
5.59 ± 2% -3.5 2.13 ± 26% perf-profile.calltrace.cycles-pp.release_sock.sctp_sendmsg.sock_sendmsg.____sys_sendmsg.___sys_sendmsg
5.49 ± 2% -3.4 2.09 ± 26% perf-profile.calltrace.cycles-pp.__release_sock.release_sock.sctp_sendmsg.sock_sendmsg.____sys_sendmsg
4.61 -3.0 1.66 ± 27% perf-profile.calltrace.cycles-pp.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv.sctp_backlog_rcv.__release_sock
4.57 -2.9 1.62 ± 27% perf-profile.calltrace.cycles-pp.sctp_assoc_bh_rcv.sctp_backlog_rcv.__release_sock.release_sock.sctp_sendmsg
4.49 -2.9 1.61 ± 27% perf-profile.calltrace.cycles-pp.sctp_do_sm.sctp_assoc_bh_rcv.sctp_backlog_rcv.__release_sock.release_sock
4.22 ± 2% -2.9 1.34 ± 53% perf-profile.calltrace.cycles-pp.sctp_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog
5.33 -2.8 2.50 ± 35% perf-profile.calltrace.cycles-pp.sctp_datamsg_from_user.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg.____sys_sendmsg
4.28 ± 2% -2.8 1.45 ± 54% perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action
4.27 ± 2% -2.8 1.44 ± 54% perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog.__napi_poll
4.36 ± 2% -2.8 1.57 ± 46% perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start
4.32 ± 4% -2.7 1.58 ± 30% perf-profile.calltrace.cycles-pp.sctp_cmd_interpreter.sctp_do_sm.sctp_primitive_SEND.sctp_sendmsg_to_asoc.sctp_sendmsg
3.96 ± 2% -2.7 1.24 ± 49% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm
3.92 ± 2% -2.7 1.23 ± 50% perf-profile.calltrace.cycles-pp.ip_finish_output2.__ip_queue_xmit.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter
3.67 ± 2% -2.7 0.98 ± 67% perf-profile.calltrace.cycles-pp.sctp_assoc_bh_rcv.sctp_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core
3.72 ± 3% -2.7 1.06 ± 52% perf-profile.calltrace.cycles-pp.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip
3.72 ± 3% -2.7 1.06 ± 53% perf-profile.calltrace.cycles-pp.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq
4.00 ± 3% -2.7 1.34 ± 30% perf-profile.calltrace.cycles-pp.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_primitive_SEND
3.83 ± 3% -2.6 1.19 ± 51% perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit.sctp_packet_transmit.sctp_outq_flush
3.81 ± 3% -2.6 1.19 ± 50% perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit.sctp_packet_transmit
3.47 -2.6 0.84 ± 83% perf-profile.calltrace.cycles-pp.sctp_do_sm.sctp_assoc_bh_rcv.sctp_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish
3.80 ± 3% -2.6 1.18 ± 50% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit
3.21 ± 2% -2.6 0.61 ± 55% perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.sctp_recvmsg.inet_recvmsg.____sys_recvmsg
4.13 ± 4% -2.6 1.54 ± 30% perf-profile.calltrace.cycles-pp.sctp_do_sm.sctp_primitive_SEND.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg
3.74 ± 3% -2.6 1.15 ± 50% perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2
3.23 ± 2% -2.5 0.71 ± 29% perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg
3.09 ± 2% -2.4 0.71 ±100% perf-profile.calltrace.cycles-pp.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv.sctp_rcv.ip_protocol_deliver_rcu
2.51 ± 2% -2.2 0.36 ±103% perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.sctp_recvmsg.inet_recvmsg
2.38 ± 2% -2.1 0.32 ±101% perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.sctp_recvmsg
2.35 ± 2% -2.0 0.31 ±102% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
2.85 -1.8 1.02 ± 23% perf-profile.calltrace.cycles-pp.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv.sctp_backlog_rcv
2.56 -1.7 0.87 ± 25% perf-profile.calltrace.cycles-pp.sctp_packet_transmit.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv
2.24 ± 8% -1.7 0.57 ± 52% perf-profile.calltrace.cycles-pp.sctp_skb_recv_datagram.sctp_recvmsg.inet_recvmsg.____sys_recvmsg.___sys_recvmsg
2.40 ± 3% -1.5 0.89 ± 31% perf-profile.calltrace.cycles-pp.sctp_outq_sack.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv.sctp_backlog_rcv
1.72 ± 4% -1.4 0.37 ±104% perf-profile.calltrace.cycles-pp.sctp_chunk_put.sctp_outq_sack.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv
2.57 ± 2% -1.3 1.24 ± 42% perf-profile.calltrace.cycles-pp.sctp_make_datafrag_empty.sctp_datamsg_from_user.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg
1.77 ± 9% -1.2 0.60 ± 75% perf-profile.calltrace.cycles-pp.sctp_outq_flush_data.sctp_outq_flush.sctp_cmd_interpreter.sctp_do_sm.sctp_primitive_SEND
2.05 ± 3% -1.1 0.95 ± 38% perf-profile.calltrace.cycles-pp._sctp_make_chunk.sctp_make_datafrag_empty.sctp_datamsg_from_user.sctp_sendmsg_to_asoc.sctp_sendmsg
1.47 ± 4% -1.0 0.48 ±105% perf-profile.calltrace.cycles-pp.__alloc_skb._sctp_make_chunk.sctp_make_datafrag_empty.sctp_datamsg_from_user.sctp_sendmsg_to_asoc
1.54 ± 3% -0.9 0.60 ± 53% perf-profile.calltrace.cycles-pp.sctp_user_addto_chunk.sctp_datamsg_from_user.sctp_sendmsg_to_asoc.sctp_sendmsg.sock_sendmsg
1.19 ± 4% -0.9 0.32 ±102% perf-profile.calltrace.cycles-pp._copy_from_iter.sctp_user_addto_chunk.sctp_datamsg_from_user.sctp_sendmsg_to_asoc.sctp_sendmsg
1.22 ± 3% +0.2 1.39 ± 7% perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
1.02 ± 5% +0.5 1.47 ± 6% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.19 ±141% +0.5 0.67 ± 14% perf-profile.calltrace.cycles-pp.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call
0.17 ±141% +0.5 0.69 ± 19% perf-profile.calltrace.cycles-pp.find_busiest_group.load_balance.rebalance_domains.__softirqentry_text_start.__irq_exit_rcu
0.40 ± 71% +0.6 0.95 ± 18% perf-profile.calltrace.cycles-pp.load_balance.rebalance_domains.__softirqentry_text_start.__irq_exit_rcu.sysvec_apic_timer_interrupt
2.27 ± 14% +0.6 2.82 ± 10% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry
0.00 +0.6 0.57 ± 3% perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +0.6 0.60 ± 7% perf-profile.calltrace.cycles-pp.update_rq_clock.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer
0.00 +0.6 0.64 ± 18% perf-profile.calltrace.cycles-pp.update_sd_lb_stats.find_busiest_group.load_balance.rebalance_domains.__softirqentry_text_start
0.00 +0.6 0.64 ± 6% perf-profile.calltrace.cycles-pp.ret_from_fork
0.00 +0.6 0.64 ± 6% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
0.44 ± 72% +0.8 1.22 ± 39% perf-profile.calltrace.cycles-pp.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
1.48 ± 3% +0.8 2.27 ± 11% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
3.20 ± 3% +0.8 4.00 ± 7% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
3.70 ± 2% +1.1 4.80 ± 7% perf-profile.calltrace.cycles-pp.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
2.36 ± 4% +1.9 4.25 ± 23% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
2.45 ± 5% +2.2 4.70 ± 28% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
3.99 ± 8% +2.4 6.41 ± 15% perf-profile.calltrace.cycles-pp.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
2.99 ± 2% +2.4 5.42 ± 24% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
4.96 ± 2% +3.6 8.54 ± 25% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
9.02 ± 2% +3.7 12.76 ± 12% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
9.12 ± 2% +3.9 13.04 ± 12% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
14.90 +6.0 20.88 ± 12% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
20.77 +6.3 27.04 ± 7% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
30.56 +12.3 42.87 ± 5% perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
30.80 +12.4 43.18 ± 5% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
54.38 +19.1 73.50 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
53.23 +19.3 72.57 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
60.54 +21.9 82.44 ± 5% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
60.62 +21.9 82.56 ± 5% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
60.62 +21.9 82.56 ± 5% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
59.19 +22.2 81.34 ± 5% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
61.07 +22.3 83.39 ± 5% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
36.24 ± 2% -21.4 14.80 ± 28% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
36.09 ± 2% -21.4 14.74 ± 28% perf-profile.children.cycles-pp.do_syscall_64
25.09 ± 2% -14.3 10.78 ± 29% perf-profile.children.cycles-pp.stress_sctp
22.40 ± 2% -12.6 9.77 ± 30% perf-profile.children.cycles-pp.sendmsg
21.69 ± 2% -12.3 9.44 ± 30% perf-profile.children.cycles-pp.__sys_sendmsg
21.54 ± 2% -12.2 9.36 ± 30% perf-profile.children.cycles-pp.___sys_sendmsg
20.65 ± 2% -11.7 9.00 ± 30% perf-profile.children.cycles-pp.____sys_sendmsg
19.57 ± 2% -11.1 8.50 ± 30% perf-profile.children.cycles-pp.sctp_sendmsg
19.56 ± 2% -11.1 8.50 ± 30% perf-profile.children.cycles-pp.sock_sendmsg
14.14 -8.2 5.89 ± 26% perf-profile.children.cycles-pp.sctp_do_sm
13.90 -8.1 5.79 ± 26% perf-profile.children.cycles-pp.sctp_cmd_interpreter
11.97 ± 4% -8.1 3.92 ± 29% perf-profile.children.cycles-pp.recvmsg
11.24 ± 4% -7.7 3.58 ± 29% perf-profile.children.cycles-pp.__sys_recvmsg
11.06 ± 4% -7.6 3.51 ± 29% perf-profile.children.cycles-pp.___sys_recvmsg
13.19 ± 3% -7.2 6.00 ± 32% perf-profile.children.cycles-pp.sctp_sendmsg_to_asoc
10.29 ± 4% -7.2 3.13 ± 29% perf-profile.children.cycles-pp.____sys_recvmsg
9.97 ± 4% -7.0 2.95 ± 29% perf-profile.children.cycles-pp.sctp_recvmsg
9.79 ± 4% -6.9 2.89 ± 29% perf-profile.children.cycles-pp.inet_recvmsg
10.33 ± 2% -6.1 4.23 ± 27% perf-profile.children.cycles-pp.sctp_outq_flush
9.94 -5.7 4.22 ± 26% perf-profile.children.cycles-pp.sctp_assoc_bh_rcv
8.54 -5.0 3.51 ± 27% perf-profile.children.cycles-pp.sctp_packet_transmit
7.06 ± 2% -4.3 2.76 ± 27% perf-profile.children.cycles-pp.release_sock
6.76 ± 2% -4.2 2.52 ± 27% perf-profile.children.cycles-pp.sctp_backlog_rcv
6.79 ± 2% -4.2 2.64 ± 27% perf-profile.children.cycles-pp.__release_sock
6.58 ± 5% -4.1 2.45 ± 31% perf-profile.children.cycles-pp.sctp_primitive_SEND
4.28 ± 2% -3.0 1.32 ± 29% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
5.24 ± 2% -2.9 2.37 ± 29% perf-profile.children.cycles-pp.__ip_queue_xmit
5.36 -2.8 2.51 ± 35% perf-profile.children.cycles-pp.sctp_datamsg_from_user
5.15 ± 3% -2.8 2.32 ± 29% perf-profile.children.cycles-pp.ip_finish_output2
5.03 ± 3% -2.8 2.25 ± 31% perf-profile.children.cycles-pp.__local_bh_enable_ip
5.57 ± 3% -2.8 2.80 ± 26% perf-profile.children.cycles-pp.net_rx_action
5.53 ± 3% -2.8 2.78 ± 26% perf-profile.children.cycles-pp.process_backlog
5.54 ± 3% -2.8 2.78 ± 26% perf-profile.children.cycles-pp.__napi_poll
4.93 ± 3% -2.7 2.20 ± 30% perf-profile.children.cycles-pp.do_softirq
5.47 ± 3% -2.7 2.74 ± 26% perf-profile.children.cycles-pp.__netif_receive_skb_one_core
5.35 ± 3% -2.7 2.68 ± 26% perf-profile.children.cycles-pp.ip_protocol_deliver_rcu
5.35 ± 3% -2.7 2.68 ± 26% perf-profile.children.cycles-pp.ip_local_deliver_finish
5.30 ± 3% -2.6 2.65 ± 26% perf-profile.children.cycles-pp.sctp_rcv
3.24 ± 2% -2.5 0.71 ± 29% perf-profile.children.cycles-pp.skb_copy_datagram_iter
3.22 ± 2% -2.5 0.70 ± 30% perf-profile.children.cycles-pp.__skb_datagram_iter
2.36 ± 7% -2.0 0.35 ± 36% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
2.54 ± 2% -2.0 0.57 ± 33% perf-profile.children.cycles-pp._copy_to_iter
8.20 -1.9 6.30 ± 7% perf-profile.children.cycles-pp.__softirqentry_text_start
2.40 ± 2% -1.9 0.49 ± 34% perf-profile.children.cycles-pp.copyout
2.14 ± 6% -1.7 0.46 ± 30% perf-profile.children.cycles-pp._raw_spin_lock_bh
2.25 ± 8% -1.6 0.65 ± 25% perf-profile.children.cycles-pp.sctp_skb_recv_datagram
2.01 ± 5% -1.5 0.46 ± 30% perf-profile.children.cycles-pp.lock_sock_nested
2.60 ± 3% -1.5 1.05 ± 27% perf-profile.children.cycles-pp.sctp_ulpq_tail_data
2.52 ± 3% -1.5 1.00 ± 25% perf-profile.children.cycles-pp.sctp_chunk_put
2.70 ± 4% -1.5 1.23 ± 24% perf-profile.children.cycles-pp.sctp_outq_sack
2.44 ± 3% -1.4 1.00 ± 28% perf-profile.children.cycles-pp.sctp_packet_pack
2.33 ± 6% -1.4 0.97 ± 34% perf-profile.children.cycles-pp.__alloc_skb
2.58 ± 2% -1.3 1.25 ± 42% perf-profile.children.cycles-pp.sctp_make_datafrag_empty
2.10 ± 4% -1.2 0.93 ± 32% perf-profile.children.cycles-pp.memcpy_erms
2.19 ± 4% -1.2 1.03 ± 38% perf-profile.children.cycles-pp._sctp_make_chunk
1.76 ± 5% -1.1 0.68 ± 27% perf-profile.children.cycles-pp.sctp_ulpevent_make_rcvmsg
2.17 ± 7% -1.1 1.11 ± 27% perf-profile.children.cycles-pp.sctp_outq_flush_data
1.74 ± 3% -1.1 0.67 ± 26% perf-profile.children.cycles-pp.skb_release_head_state
1.75 ± 3% -1.0 0.74 ± 24% perf-profile.children.cycles-pp.consume_skb
1.38 ± 7% -1.0 0.39 ± 34% perf-profile.children.cycles-pp.kfree_skb_reason
1.60 ± 8% -1.0 0.62 ± 41% perf-profile.children.cycles-pp.kmalloc_reserve
1.59 ± 9% -1.0 0.60 ± 41% perf-profile.children.cycles-pp.__kmalloc_node_track_caller
1.57 ± 3% -0.9 0.68 ± 29% perf-profile.children.cycles-pp.sctp_user_addto_chunk
1.27 ± 2% -0.8 0.43 ± 31% perf-profile.children.cycles-pp.skb_release_data
1.49 ± 4% -0.8 0.67 ± 26% perf-profile.children.cycles-pp.copy_msghdr_from_user
1.30 ± 9% -0.7 0.60 ± 28% perf-profile.children.cycles-pp._copy_from_user
1.20 ± 5% -0.7 0.52 ± 27% perf-profile.children.cycles-pp._copy_from_iter
1.32 ± 5% -0.7 0.65 ± 39% perf-profile.children.cycles-pp.__kmem_cache_alloc_node
1.52 ± 9% -0.7 0.86 ± 22% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
1.05 ± 5% -0.6 0.40 ± 25% perf-profile.children.cycles-pp.copyin
0.96 ± 5% -0.6 0.34 ± 32% perf-profile.children.cycles-pp.__sk_mem_schedule
0.92 ± 5% -0.6 0.32 ± 33% perf-profile.children.cycles-pp.__sk_mem_raise_allocated
0.90 ± 8% -0.6 0.34 ± 23% perf-profile.children.cycles-pp.__check_object_size
0.83 ± 3% -0.5 0.28 ± 28% perf-profile.children.cycles-pp.__sk_mem_reduce_allocated
0.72 ± 13% -0.5 0.19 ± 28% perf-profile.children.cycles-pp.__alloc_pages
0.68 ± 12% -0.5 0.16 ± 26% perf-profile.children.cycles-pp.__kmalloc_large_node
0.90 ± 6% -0.5 0.38 ± 28% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.77 ± 5% -0.5 0.27 ± 38% perf-profile.children.cycles-pp.__close
0.72 ± 4% -0.5 0.22 ± 33% perf-profile.children.cycles-pp.kmem_cache_free
0.75 ± 5% -0.5 0.26 ± 38% perf-profile.children.cycles-pp.task_work_run
0.65 ± 16% -0.5 0.16 ± 33% perf-profile.children.cycles-pp.get_page_from_freelist
0.74 ± 6% -0.5 0.26 ± 39% perf-profile.children.cycles-pp.__fput
0.83 ± 5% -0.5 0.35 ± 29% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.75 ± 6% -0.5 0.27 ± 36% perf-profile.children.cycles-pp.exit_to_user_mode_loop
0.71 ± 5% -0.5 0.23 ± 34% perf-profile.children.cycles-pp.__slab_free
0.64 ± 10% -0.5 0.17 ± 23% perf-profile.children.cycles-pp.sctp_outq_select_transport
0.68 ± 5% -0.5 0.22 ± 46% perf-profile.children.cycles-pp.sock_close
0.68 ± 5% -0.5 0.22 ± 46% perf-profile.children.cycles-pp.__sock_release
0.68 ± 5% -0.5 0.22 ± 46% perf-profile.children.cycles-pp.inet_release
0.82 ± 10% -0.5 0.36 ± 30% perf-profile.children.cycles-pp.connect
0.81 ± 9% -0.5 0.36 ± 30% perf-profile.children.cycles-pp.__x64_sys_connect
0.81 ± 9% -0.5 0.36 ± 30% perf-profile.children.cycles-pp.__sys_connect
0.68 ± 5% -0.5 0.22 ± 46% perf-profile.children.cycles-pp.sctp_close
0.81 ± 9% -0.5 0.36 ± 30% perf-profile.children.cycles-pp.sctp_inet_connect
0.81 ± 9% -0.5 0.36 ± 30% perf-profile.children.cycles-pp.__sctp_connect
0.81 ± 3% -0.4 0.37 ± 21% perf-profile.children.cycles-pp.sctp_wfree
0.55 ± 17% -0.4 0.12 ± 38% perf-profile.children.cycles-pp.rmqueue
0.55 ± 15% -0.4 0.13 ± 29% perf-profile.children.cycles-pp.sctp_packet_config
0.78 ± 2% -0.4 0.36 ± 25% perf-profile.children.cycles-pp.import_iovec
0.66 ± 11% -0.4 0.25 ± 34% perf-profile.children.cycles-pp.mem_cgroup_charge_skmem
0.75 ± 3% -0.4 0.35 ± 24% perf-profile.children.cycles-pp.__import_iovec
0.69 ± 6% -0.4 0.29 ± 27% perf-profile.children.cycles-pp.sctp_ulpq_tail_event
0.69 ± 5% -0.4 0.30 ± 32% perf-profile.children.cycles-pp.kmem_cache_alloc
0.64 ± 10% -0.4 0.27 ± 28% perf-profile.children.cycles-pp.__might_resched
0.60 ± 6% -0.4 0.25 ± 36% perf-profile.children.cycles-pp.sctp_chunkify
0.38 ± 4% -0.4 0.03 ±102% perf-profile.children.cycles-pp.flush_smp_call_function_queue
0.63 ± 3% -0.3 0.29 ± 25% perf-profile.children.cycles-pp.iovec_from_user
0.50 ± 7% -0.3 0.16 ± 30% perf-profile.children.cycles-pp.check_heap_object
0.56 ± 6% -0.3 0.25 ± 44% perf-profile.children.cycles-pp.___slab_alloc
0.51 ± 14% -0.3 0.20 ± 28% perf-profile.children.cycles-pp.__skb_clone
0.94 ± 8% -0.3 0.65 ± 29% perf-profile.children.cycles-pp.sctp_packet_transmit_chunk
0.41 ± 3% -0.3 0.13 ± 23% perf-profile.children.cycles-pp.sctp_sf_eat_data_6_2
0.53 ± 5% -0.3 0.25 ± 33% perf-profile.children.cycles-pp.sctp_data_ready
0.33 ± 5% -0.3 0.05 ± 59% perf-profile.children.cycles-pp.__free_pages_ok
0.40 ± 2% -0.3 0.13 ± 37% perf-profile.children.cycles-pp.__mod_memcg_state
0.35 ± 4% -0.3 0.10 ± 21% perf-profile.children.cycles-pp.sctp_eat_data
0.51 ± 12% -0.2 0.26 ± 48% perf-profile.children.cycles-pp.sctp_outq_flush_ctrl
0.43 ± 12% -0.2 0.19 ± 28% perf-profile.children.cycles-pp.__entry_text_start
0.43 ± 5% -0.2 0.19 ± 27% perf-profile.children.cycles-pp.__list_del_entry_valid
0.33 ± 7% -0.2 0.10 ± 19% perf-profile.children.cycles-pp.simple_copy_to_iter
0.32 ± 30% -0.2 0.09 ± 36% perf-profile.children.cycles-pp.try_charge_memcg
0.39 ± 16% -0.2 0.16 ± 24% perf-profile.children.cycles-pp.__might_sleep
0.32 ± 11% -0.2 0.10 ± 16% perf-profile.children.cycles-pp.schedule_idle
0.34 ± 20% -0.2 0.13 ± 22% perf-profile.children.cycles-pp.__copy_skb_header
0.44 ± 4% -0.2 0.24 ± 22% perf-profile.children.cycles-pp.__wake_up_common
0.29 ± 7% -0.2 0.08 ± 21% perf-profile.children.cycles-pp.sctp_inq_pop
0.37 ± 9% -0.2 0.16 ± 38% perf-profile.children.cycles-pp.memset_erms
0.38 ± 12% -0.2 0.18 ± 29% perf-profile.children.cycles-pp.__might_fault
0.25 ± 8% -0.2 0.05 ± 72% perf-profile.children.cycles-pp.dst_release
0.46 ± 5% -0.2 0.26 ± 20% perf-profile.children.cycles-pp.__wake_up_common_lock
0.36 ± 3% -0.2 0.16 ± 25% perf-profile.children.cycles-pp.sock_kmalloc
0.29 ± 14% -0.2 0.10 ± 42% perf-profile.children.cycles-pp.mem_cgroup_uncharge_skmem
0.33 ± 5% -0.2 0.15 ± 41% perf-profile.children.cycles-pp.sctp_wait_for_connect
0.29 ± 12% -0.2 0.12 ± 33% perf-profile.children.cycles-pp.sctp_ulpevent_receive_data
0.28 ± 16% -0.2 0.11 ± 31% perf-profile.children.cycles-pp.refill_stock
0.39 ± 3% -0.2 0.22 ± 20% perf-profile.children.cycles-pp.autoremove_wake_function
0.32 ± 9% -0.2 0.15 ± 22% perf-profile.children.cycles-pp.kmem_cache_alloc_node
0.34 ± 4% -0.2 0.18 ± 31% perf-profile.children.cycles-pp.sctp_packet_append_chunk
0.42 ± 3% -0.2 0.26 ± 17% perf-profile.children.cycles-pp.try_to_wake_up
0.29 ± 8% -0.2 0.13 ± 31% perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
0.35 ± 4% -0.2 0.19 ± 47% perf-profile.children.cycles-pp.sctp_endpoint_bh_rcv
0.18 ± 10% -0.1 0.04 ±101% perf-profile.children.cycles-pp.free_unref_page
0.23 ± 4% -0.1 0.09 ± 36% perf-profile.children.cycles-pp.stress_sctp_server
0.28 ± 8% -0.1 0.13 ± 31% perf-profile.children.cycles-pp.__kmalloc
0.26 ± 9% -0.1 0.12 ± 33% perf-profile.children.cycles-pp.sockfd_lookup_light
0.25 ± 3% -0.1 0.12 ± 42% perf-profile.children.cycles-pp.kmalloc_trace
0.31 ± 12% -0.1 0.18 ± 19% perf-profile.children.cycles-pp.sctp_chunk_free
0.20 ± 16% -0.1 0.07 ± 69% perf-profile.children.cycles-pp.skb_set_owner_w
0.27 ± 4% -0.1 0.14 ± 33% perf-profile.children.cycles-pp.aa_sk_perm
0.24 ± 3% -0.1 0.12 ± 19% perf-profile.children.cycles-pp.__kmem_cache_free
0.18 ± 11% -0.1 0.06 ± 84% perf-profile.children.cycles-pp.__genradix_ptr
0.20 ± 12% -0.1 0.08 ± 25% perf-profile.children.cycles-pp.skb_clone
0.28 ± 10% -0.1 0.16 ± 27% perf-profile.children.cycles-pp.sctp_ulpevent_read_sndrcvinfo
0.21 ± 5% -0.1 0.10 ± 39% perf-profile.children.cycles-pp.__fdget
0.19 ± 9% -0.1 0.07 ± 21% perf-profile.children.cycles-pp.setsockopt
0.25 ± 8% -0.1 0.14 ± 28% perf-profile.children.cycles-pp.sctp_check_transmitted
0.15 ± 14% -0.1 0.04 ±101% perf-profile.children.cycles-pp.do_accept
0.18 ± 26% -0.1 0.06 ± 79% perf-profile.children.cycles-pp.sctp_sendmsg_parse
0.16 ± 13% -0.1 0.05 ± 76% perf-profile.children.cycles-pp.accept
0.16 ± 16% -0.1 0.05 ± 74% perf-profile.children.cycles-pp.__x64_sys_accept
0.16 ± 16% -0.1 0.05 ± 74% perf-profile.children.cycles-pp.__sys_accept4
0.16 ± 16% -0.1 0.05 ± 85% perf-profile.children.cycles-pp.get_partial_node
0.20 ± 6% -0.1 0.09 ± 29% perf-profile.children.cycles-pp.__sctp_packet_append_chunk
0.15 ± 14% -0.1 0.04 ± 73% perf-profile.children.cycles-pp.__x64_sys_setsockopt
0.13 ± 12% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__copy_msghdr
0.18 ± 16% -0.1 0.08 ± 29% perf-profile.children.cycles-pp.sock_kfree_s
0.15 ± 17% -0.1 0.04 ± 73% perf-profile.children.cycles-pp.__sys_setsockopt
0.21 ± 2% -0.1 0.10 ± 24% perf-profile.children.cycles-pp.sock_recvmsg
0.19 ± 5% -0.1 0.08 ± 14% perf-profile.children.cycles-pp.enqueue_entity
0.24 ± 12% -0.1 0.14 ± 29% perf-profile.children.cycles-pp.put_cmsg
0.22 ± 12% -0.1 0.12 ± 14% perf-profile.children.cycles-pp.__dev_queue_xmit
0.16 ± 6% -0.1 0.06 ± 47% perf-profile.children.cycles-pp.sock_wfree
0.19 ± 10% -0.1 0.09 ± 43% perf-profile.children.cycles-pp.__virt_addr_valid
0.18 ± 5% -0.1 0.09 ± 26% perf-profile.children.cycles-pp.sctp_sched_fcfs_dequeue
0.20 ± 15% -0.1 0.10 ± 32% perf-profile.children.cycles-pp.dequeue_entity
0.21 ± 8% -0.1 0.11 ± 18% perf-profile.children.cycles-pp.ttwu_do_activate
0.21 ± 9% -0.1 0.11 ± 37% perf-profile.children.cycles-pp.sctp_association_put
0.19 ± 7% -0.1 0.10 ± 29% perf-profile.children.cycles-pp.skb_put
0.17 ± 8% -0.1 0.08 ± 31% perf-profile.children.cycles-pp.kfree
0.16 ± 5% -0.1 0.07 ± 56% perf-profile.children.cycles-pp.security_socket_sendmsg
0.15 ± 5% -0.1 0.06 ± 51% perf-profile.children.cycles-pp.sctp_sched_dequeue_common
0.18 ± 2% -0.1 0.09 ± 22% perf-profile.children.cycles-pp.security_socket_recvmsg
0.12 ± 11% -0.1 0.03 ±101% perf-profile.children.cycles-pp.inet_accept
0.13 ± 16% -0.1 0.04 ± 73% perf-profile.children.cycles-pp.sctp_setsockopt
0.13 ± 18% -0.1 0.04 ± 77% perf-profile.children.cycles-pp.sctp_ulpq_order
0.12 ± 16% -0.1 0.03 ±106% perf-profile.children.cycles-pp.sctp_ulpevent_init
0.14 ± 17% -0.1 0.05 ± 55% perf-profile.children.cycles-pp.sctp_connect_new_asoc
0.14 ± 3% -0.1 0.05 ± 73% perf-profile.children.cycles-pp.sctp_outq_tail
0.11 ± 12% -0.1 0.03 ±100% perf-profile.children.cycles-pp.sctp_accept
0.21 ± 8% -0.1 0.13 ± 17% perf-profile.children.cycles-pp.enqueue_task_fair
0.15 ± 10% -0.1 0.08 ± 33% perf-profile.children.cycles-pp.sctp_association_hold
0.13 ± 3% -0.1 0.06 ± 53% perf-profile.children.cycles-pp.__ksize
0.17 ± 12% -0.1 0.10 ± 34% perf-profile.children.cycles-pp.memcg_slab_post_alloc_hook
0.12 ± 27% -0.1 0.05 ± 46% perf-profile.children.cycles-pp.__check_heap_object
0.14 ± 21% -0.1 0.07 ± 18% perf-profile.children.cycles-pp.dev_hard_start_xmit
0.13 ± 16% -0.1 0.06 ± 78% perf-profile.children.cycles-pp.sctp_chunk_hold
0.12 ± 3% -0.1 0.05 ± 49% perf-profile.children.cycles-pp.__x64_sys_getsockopt
0.11 ± 11% -0.1 0.04 ±109% perf-profile.children.cycles-pp.__put_user_nocheck_8
0.15 ± 8% -0.1 0.08 ± 17% perf-profile.children.cycles-pp.getsockopt
0.13 ± 26% -0.1 0.06 ± 20% perf-profile.children.cycles-pp.loopback_xmit
0.10 ± 8% -0.1 0.03 ±108% perf-profile.children.cycles-pp.cgroup_rstat_updated
0.15 ± 14% -0.1 0.08 ± 24% perf-profile.children.cycles-pp.sctp_gen_sack
0.09 ± 13% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__socket
0.09 ± 13% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__sys_socket
0.09 ± 13% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__x64_sys_socket
0.21 ± 2% -0.1 0.15 ± 8% perf-profile.children.cycles-pp.update_load_avg
0.12 ± 14% -0.1 0.05 ± 76% perf-profile.children.cycles-pp.rcu_all_qs
0.11 ± 8% -0.1 0.05 ± 49% perf-profile.children.cycles-pp.__sys_getsockopt
0.15 ± 11% -0.1 0.09 ± 29% perf-profile.children.cycles-pp.sctp_make_sack
0.09 ± 9% -0.1 0.03 ±103% perf-profile.children.cycles-pp.sctp_chunk_abandoned
0.09 ± 30% -0.1 0.04 ± 75% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.09 ± 10% -0.1 0.04 ± 75% perf-profile.children.cycles-pp.sctp_getsockopt
0.10 ± 9% -0.0 0.05 ± 45% perf-profile.children.cycles-pp.sctp_primitive_SHUTDOWN
0.08 ± 5% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.prepare_task_switch
0.13 ± 12% +0.0 0.16 ± 12% perf-profile.children.cycles-pp.calc_global_load_tick
0.04 ± 71% +0.1 0.11 ± 17% perf-profile.children.cycles-pp.smpboot_thread_fn
0.02 ±141% +0.1 0.09 ± 14% perf-profile.children.cycles-pp.switch_mm_irqs_off
0.23 ± 5% +0.1 0.31 ± 11% perf-profile.children.cycles-pp.ct_kernel_exit_state
0.02 ±141% +0.1 0.10 ± 33% perf-profile.children.cycles-pp.update_rt_rq_load_avg
0.07 ± 18% +0.1 0.16 ± 52% perf-profile.children.cycles-pp.ct_nmi_enter
0.25 ± 13% +0.1 0.35 ± 14% perf-profile.children.cycles-pp.memcpy_toio
0.25 ± 13% +0.1 0.35 ± 15% perf-profile.children.cycles-pp.drm_fb_helper_damage_work
0.25 ± 13% +0.1 0.35 ± 15% perf-profile.children.cycles-pp.drm_fb_helper_damage_blit_real
0.11 ± 4% +0.1 0.21 ± 33% perf-profile.children.cycles-pp.sctp_assoc_rwnd_increase
0.30 ± 13% +0.1 0.40 ± 13% perf-profile.children.cycles-pp.process_one_work
0.09 ± 15% +0.1 0.20 ± 28% perf-profile.children.cycles-pp._find_next_and_bit
0.32 ± 8% +0.1 0.44 ± 10% perf-profile.children.cycles-pp.worker_thread
0.25 +0.1 0.38 ± 22% perf-profile.children.cycles-pp.ct_kernel_enter
0.32 ± 19% +0.1 0.45 ± 9% perf-profile.children.cycles-pp.io_serial_in
0.46 ± 16% +0.1 0.60 ± 10% perf-profile.children.cycles-pp.rcu_core
0.23 ± 7% +0.1 0.38 ± 14% perf-profile.children.cycles-pp.arch_cpu_idle_enter
0.18 ± 9% +0.2 0.33 ± 20% perf-profile.children.cycles-pp.idle_cpu
0.26 +0.2 0.42 ± 21% perf-profile.children.cycles-pp.update_irq_load_avg
0.25 ± 10% +0.2 0.41 ± 13% perf-profile.children.cycles-pp.note_gp_changes
0.20 ± 10% +0.2 0.36 ± 17% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
0.43 ± 3% +0.2 0.60 ± 4% perf-profile.children.cycles-pp.ktime_get_update_offsets_now
1.27 ± 3% +0.2 1.44 ± 8% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.58 ± 4% +0.2 0.76 ± 8% perf-profile.children.cycles-pp.update_rq_clock
0.34 +0.2 0.52 ± 26% perf-profile.children.cycles-pp.ct_idle_exit
0.44 ± 18% +0.2 0.63 ± 12% perf-profile.children.cycles-pp.wait_for_lsr
0.45 ± 9% +0.2 0.64 ± 6% perf-profile.children.cycles-pp.ret_from_fork
0.44 ± 10% +0.2 0.64 ± 6% perf-profile.children.cycles-pp.kthread
0.45 ± 16% +0.2 0.65 ± 13% perf-profile.children.cycles-pp.serial8250_console_write
0.50 ± 12% +0.2 0.71 ± 13% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.46 ± 17% +0.2 0.66 ± 11% perf-profile.children.cycles-pp.asm_sysvec_irq_work
0.46 ± 17% +0.2 0.66 ± 11% perf-profile.children.cycles-pp.sysvec_irq_work
0.46 ± 17% +0.2 0.66 ± 11% perf-profile.children.cycles-pp.__sysvec_irq_work
0.46 ± 17% +0.2 0.66 ± 11% perf-profile.children.cycles-pp.irq_work_run
0.46 ± 17% +0.2 0.66 ± 11% perf-profile.children.cycles-pp.irq_work_single
0.46 ± 17% +0.2 0.66 ± 11% perf-profile.children.cycles-pp._printk
0.46 ± 17% +0.2 0.66 ± 11% perf-profile.children.cycles-pp.vprintk_emit
0.46 ± 17% +0.2 0.66 ± 11% perf-profile.children.cycles-pp.console_unlock
0.46 ± 17% +0.2 0.66 ± 11% perf-profile.children.cycles-pp.console_emit_next_record
0.40 ± 10% +0.2 0.62 ± 23% perf-profile.children.cycles-pp.hrtimer_next_event_without
1.05 ± 3% +0.2 1.28 ± 7% perf-profile.children.cycles-pp.native_sched_clock
0.48 ± 19% +0.2 0.71 ± 11% perf-profile.children.cycles-pp.irq_work_run_list
1.29 ± 2% +0.2 1.54 ± 9% perf-profile.children.cycles-pp.sched_clock_cpu
0.08 ± 20% +0.3 0.37 ± 63% perf-profile.children.cycles-pp.check_cpu_stall
0.71 ± 4% +0.3 1.01 ± 11% perf-profile.children.cycles-pp.irqtime_account_irq
0.74 ± 4% +0.3 1.06 ± 24% perf-profile.children.cycles-pp.schedule
0.34 ± 2% +0.3 0.65 ± 39% perf-profile.children.cycles-pp.__hrtimer_next_event_base
0.46 ± 34% +0.4 0.83 ± 33% perf-profile.children.cycles-pp.start_kernel
0.46 ± 34% +0.4 0.83 ± 33% perf-profile.children.cycles-pp.arch_call_rest_init
0.46 ± 34% +0.4 0.83 ± 33% perf-profile.children.cycles-pp.rest_init
0.37 ± 8% +0.4 0.76 ± 25% perf-profile.children.cycles-pp.pick_next_task_fair
0.19 ± 16% +0.4 0.58 ± 26% perf-profile.children.cycles-pp.sctp_wait_for_sndbuf
0.31 ± 9% +0.4 0.75 ± 27% perf-profile.children.cycles-pp.newidle_balance
0.26 ± 6% +0.4 0.70 ± 40% perf-profile.children.cycles-pp.rcu_pending
1.04 ± 4% +0.5 1.50 ± 6% perf-profile.children.cycles-pp.rebalance_domains
0.51 ± 15% +0.5 1.03 ± 14% perf-profile.children.cycles-pp.update_sg_lb_stats
2.33 ± 14% +0.6 2.89 ± 10% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.34 ± 2% +0.6 0.91 ± 41% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.62 ± 12% +0.6 1.26 ± 9% perf-profile.children.cycles-pp.update_sd_lb_stats
0.64 ± 18% +0.6 1.27 ± 37% perf-profile.children.cycles-pp.tick_irq_enter
0.65 ± 10% +0.6 1.30 ± 8% perf-profile.children.cycles-pp.find_busiest_group
1.59 ± 3% +0.8 2.36 ± 10% perf-profile.children.cycles-pp.scheduler_tick
0.85 ± 8% +0.8 1.66 ± 7% perf-profile.children.cycles-pp.load_balance
3.82 ± 3% +1.1 4.90 ± 7% perf-profile.children.cycles-pp.__irq_exit_rcu
2.48 ± 5% +1.9 4.36 ± 23% perf-profile.children.cycles-pp.update_process_times
2.56 ± 5% +2.2 4.76 ± 28% perf-profile.children.cycles-pp.tick_sched_handle
3.13 ± 3% +2.4 5.52 ± 24% perf-profile.children.cycles-pp.tick_sched_timer
4.07 ± 8% +2.5 6.53 ± 15% perf-profile.children.cycles-pp.menu_select
5.17 ± 3% +3.5 8.70 ± 25% perf-profile.children.cycles-pp.__hrtimer_run_queues
9.32 ± 2% +3.7 12.99 ± 11% perf-profile.children.cycles-pp.hrtimer_interrupt
9.38 ± 2% +3.8 13.22 ± 12% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
15.33 ± 2% +5.9 21.22 ± 12% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
18.70 +6.0 24.66 ± 9% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
31.40 +11.9 43.30 ± 5% perf-profile.children.cycles-pp.mwait_idle_with_hints
31.04 +12.6 43.59 ± 5% perf-profile.children.cycles-pp.intel_idle
54.74 +19.4 74.17 ± 4% perf-profile.children.cycles-pp.cpuidle_enter_state
54.81 +19.5 74.31 ± 4% perf-profile.children.cycles-pp.cpuidle_enter
60.62 +21.9 82.56 ± 5% perf-profile.children.cycles-pp.start_secondary
61.07 +22.3 83.39 ± 5% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
61.07 +22.3 83.39 ± 5% perf-profile.children.cycles-pp.cpu_startup_entry
61.07 +22.3 83.38 ± 5% perf-profile.children.cycles-pp.do_idle
59.68 +22.6 82.26 ± 5% perf-profile.children.cycles-pp.cpuidle_idle_call
4.21 ± 2% -2.9 1.30 ± 29% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
2.35 ± 7% -2.0 0.35 ± 36% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
2.09 ± 4% -1.2 0.90 ± 29% perf-profile.self.cycles-pp.memcpy_erms
0.99 ± 6% -0.5 0.47 ± 28% perf-profile.self.cycles-pp.sctp_datamsg_from_user
0.68 ± 7% -0.5 0.21 ± 33% perf-profile.self.cycles-pp.kmem_cache_free
0.69 ± 5% -0.5 0.22 ± 34% perf-profile.self.cycles-pp.__slab_free
0.58 ± 10% -0.3 0.24 ± 29% perf-profile.self.cycles-pp.__might_resched
0.45 ± 6% -0.3 0.14 ± 39% perf-profile.self.cycles-pp.sctp_recvmsg
0.50 ± 4% -0.3 0.24 ± 28% perf-profile.self.cycles-pp.__kmem_cache_alloc_node
0.30 ± 10% -0.2 0.07 ± 47% perf-profile.self.cycles-pp.check_heap_object
0.38 ± 10% -0.2 0.16 ± 32% perf-profile.self.cycles-pp._raw_spin_lock_bh
0.34 ± 20% -0.2 0.12 ± 23% perf-profile.self.cycles-pp.__copy_skb_header
0.38 ± 3% -0.2 0.17 ± 28% perf-profile.self.cycles-pp.__list_del_entry_valid
0.37 ± 9% -0.2 0.16 ± 40% perf-profile.self.cycles-pp.memset_erms
0.25 ± 6% -0.2 0.05 ± 72% perf-profile.self.cycles-pp.dst_release
0.32 ± 7% -0.2 0.13 ± 25% perf-profile.self.cycles-pp.sctp_chunk_put
0.26 ± 15% -0.2 0.07 ± 31% perf-profile.self.cycles-pp.__sk_mem_raise_allocated
0.27 ± 6% -0.2 0.08 ± 36% perf-profile.self.cycles-pp.__mod_memcg_state
0.27 ± 3% -0.2 0.08 ± 30% perf-profile.self.cycles-pp.sctp_packet_pack
0.30 ± 8% -0.2 0.12 ± 38% perf-profile.self.cycles-pp.sctp_sendmsg
0.25 ± 11% -0.2 0.08 ± 27% perf-profile.self.cycles-pp.__sk_mem_reduce_allocated
0.22 ± 11% -0.2 0.04 ± 72% perf-profile.self.cycles-pp.sctp_eat_data
0.24 ± 13% -0.2 0.06 ± 47% perf-profile.self.cycles-pp.sctp_packet_config
0.31 ± 3% -0.2 0.14 ± 31% perf-profile.self.cycles-pp.sctp_outq_flush_data
0.28 ± 7% -0.2 0.13 ± 29% perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
0.30 ± 4% -0.2 0.15 ± 52% perf-profile.self.cycles-pp.___slab_alloc
0.28 -0.2 0.13 ± 28% perf-profile.self.cycles-pp.kmem_cache_alloc
0.29 ± 2% -0.2 0.14 ± 21% perf-profile.self.cycles-pp.__might_sleep
0.22 ± 3% -0.1 0.08 ± 54% perf-profile.self.cycles-pp.stress_sctp_server
0.22 ± 4% -0.1 0.08 ± 75% perf-profile.self.cycles-pp.mem_cgroup_charge_skmem
0.27 ± 10% -0.1 0.13 ± 26% perf-profile.self.cycles-pp.sctp_sendmsg_to_asoc
0.20 ± 16% -0.1 0.06 ± 88% perf-profile.self.cycles-pp.skb_set_owner_w
0.17 ± 12% -0.1 0.03 ±102% perf-profile.self.cycles-pp.sctp_skb_recv_datagram
0.25 ± 11% -0.1 0.12 ± 34% perf-profile.self.cycles-pp.__alloc_skb
0.24 ± 5% -0.1 0.11 ± 20% perf-profile.self.cycles-pp.__kmem_cache_free
0.20 ± 12% -0.1 0.08 ± 41% perf-profile.self.cycles-pp.sendmsg
0.18 ± 9% -0.1 0.06 ± 54% perf-profile.self.cycles-pp.sctp_ulpevent_receive_data
0.20 ± 12% -0.1 0.08 ± 52% perf-profile.self.cycles-pp._copy_from_user
0.16 ± 6% -0.1 0.04 ± 73% perf-profile.self.cycles-pp.sock_wfree
0.15 ± 21% -0.1 0.04 ±102% perf-profile.self.cycles-pp.sctp_chunkify
0.21 ± 6% -0.1 0.10 ± 41% perf-profile.self.cycles-pp.__fdget
0.24 ± 13% -0.1 0.13 ± 24% perf-profile.self.cycles-pp.sctp_cmd_interpreter
0.19 ± 18% -0.1 0.08 ± 39% perf-profile.self.cycles-pp.__list_add_valid
0.17 ± 17% -0.1 0.06 ± 68% perf-profile.self.cycles-pp.try_charge_memcg
0.17 ± 24% -0.1 0.06 ± 79% perf-profile.self.cycles-pp.sctp_sendmsg_parse
0.24 ± 12% -0.1 0.13 ± 26% perf-profile.self.cycles-pp.put_cmsg
0.16 ± 5% -0.1 0.06 ± 47% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.19 ± 2% -0.1 0.08 ± 22% perf-profile.self.cycles-pp.kmem_cache_alloc_node
0.15 ± 17% -0.1 0.04 ±101% perf-profile.self.cycles-pp.sctp_data_ready
0.19 ± 17% -0.1 0.09 ± 23% perf-profile.self.cycles-pp.__check_object_size
0.13 ± 9% -0.1 0.03 ±100% perf-profile.self.cycles-pp.__copy_msghdr
0.19 ± 22% -0.1 0.09 ± 27% perf-profile.self.cycles-pp.refill_stock
0.17 ± 9% -0.1 0.08 ± 60% perf-profile.self.cycles-pp.__virt_addr_valid
0.16 ± 17% -0.1 0.06 ± 60% perf-profile.self.cycles-pp.sctp_do_sm
0.16 ± 7% -0.1 0.07 ± 58% perf-profile.self.cycles-pp.__skb_clone
0.19 ± 6% -0.1 0.09 ± 38% perf-profile.self.cycles-pp.sctp_wfree
0.17 ± 4% -0.1 0.08 ± 28% perf-profile.self.cycles-pp.recvmsg
0.15 ± 3% -0.1 0.06 ± 82% perf-profile.self.cycles-pp.__genradix_ptr
0.16 ± 10% -0.1 0.07 ± 62% perf-profile.self.cycles-pp.skb_release_data
0.16 ± 10% -0.1 0.07 ± 52% perf-profile.self.cycles-pp.__cond_resched
0.11 ± 14% -0.1 0.03 ±102% perf-profile.self.cycles-pp.do_syscall_64
0.16 ± 10% -0.1 0.08 ± 51% perf-profile.self.cycles-pp.skb_put
0.12 ± 26% -0.1 0.04 ± 72% perf-profile.self.cycles-pp.__check_heap_object
0.13 ± 22% -0.1 0.05 ± 56% perf-profile.self.cycles-pp.__entry_text_start
0.14 ± 8% -0.1 0.07 ± 31% perf-profile.self.cycles-pp.kfree
0.12 ± 20% -0.1 0.05 ± 78% perf-profile.self.cycles-pp.sctp_chunk_hold
0.13 ± 6% -0.1 0.06 ± 51% perf-profile.self.cycles-pp.__import_iovec
0.10 ± 9% -0.1 0.03 ±111% perf-profile.self.cycles-pp.__put_user_nocheck_8
0.13 ± 7% -0.1 0.06 ± 53% perf-profile.self.cycles-pp.__ksize
0.14 ± 9% -0.1 0.07 ± 52% perf-profile.self.cycles-pp.memcg_slab_post_alloc_hook
0.09 ± 26% -0.1 0.03 ±100% perf-profile.self.cycles-pp.__sctp_packet_append_chunk
0.14 ± 6% -0.1 0.08 ± 39% perf-profile.self.cycles-pp.aa_sk_perm
0.13 ± 18% -0.1 0.07 ± 35% perf-profile.self.cycles-pp.sctp_association_hold
0.16 ± 13% -0.1 0.10 ± 34% perf-profile.self.cycles-pp.sctp_association_put
0.10 ± 9% -0.1 0.04 ± 75% perf-profile.self.cycles-pp.sctp_outq_tail
0.09 ± 14% -0.1 0.03 ±105% perf-profile.self.cycles-pp._sctp_make_chunk
0.10 ± 21% -0.1 0.04 ± 76% perf-profile.self.cycles-pp.__local_bh_enable_ip
0.09 ± 30% -0.1 0.04 ± 75% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.12 ± 6% -0.1 0.06 ± 24% perf-profile.self.cycles-pp.sctp_check_transmitted
0.10 ± 12% -0.0 0.06 ± 49% perf-profile.self.cycles-pp.____sys_sendmsg
0.12 ± 13% +0.0 0.15 ± 11% perf-profile.self.cycles-pp.calc_global_load_tick
0.09 ± 24% +0.0 0.12 ± 22% perf-profile.self.cycles-pp.tick_sched_timer
0.07 ± 23% +0.0 0.11 ± 17% perf-profile.self.cycles-pp.trigger_load_balance
0.02 ±141% +0.0 0.06 ± 11% perf-profile.self.cycles-pp.hrtimer_forward
0.22 ± 5% +0.0 0.27 ± 9% perf-profile.self.cycles-pp.ct_kernel_exit_state
0.15 ± 17% +0.1 0.22 ± 19% perf-profile.self.cycles-pp.update_rq_clock
0.02 ±141% +0.1 0.09 ± 14% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.02 ±141% +0.1 0.10 ± 35% perf-profile.self.cycles-pp.update_rt_rq_load_avg
0.14 ± 11% +0.1 0.22 ± 23% perf-profile.self.cycles-pp.ct_kernel_enter
0.07 ± 11% +0.1 0.15 ± 42% perf-profile.self.cycles-pp.update_sd_lb_stats
0.09 ± 14% +0.1 0.17 ± 10% perf-profile.self.cycles-pp.note_gp_changes
0.08 ± 14% +0.1 0.17 ± 43% perf-profile.self.cycles-pp.tick_irq_enter
0.00 +0.1 0.09 ± 37% perf-profile.self.cycles-pp.rcu_core
0.11 ± 25% +0.1 0.20 ± 22% perf-profile.self.cycles-pp.rebalance_domains
0.18 ± 9% +0.1 0.28 ± 11% perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
0.02 ±141% +0.1 0.12 ± 28% perf-profile.self.cycles-pp.cpuidle_enter
0.24 ± 7% +0.1 0.35 ± 4% perf-profile.self.cycles-pp.ktime_get_update_offsets_now
0.09 ± 15% +0.1 0.20 ± 14% perf-profile.self.cycles-pp.load_balance
0.08 ± 22% +0.1 0.19 ± 31% perf-profile.self.cycles-pp._find_next_and_bit
0.17 ± 12% +0.1 0.29 ± 20% perf-profile.self.cycles-pp.irq_work_tick
0.29 ± 4% +0.1 0.42 ± 16% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler
0.17 ± 7% +0.1 0.30 ± 17% perf-profile.self.cycles-pp.idle_cpu
0.32 ± 19% +0.1 0.45 ± 9% perf-profile.self.cycles-pp.io_serial_in
0.26 +0.2 0.41 ± 22% perf-profile.self.cycles-pp.update_irq_load_avg
0.12 ± 17% +0.2 0.27 ± 40% perf-profile.self.cycles-pp.get_next_timer_interrupt
0.16 ± 5% +0.2 0.32 ± 18% perf-profile.self.cycles-pp.rcu_pending
0.10 ± 12% +0.2 0.28 ± 54% perf-profile.self.cycles-pp.update_process_times
1.01 ± 3% +0.2 1.21 ± 6% perf-profile.self.cycles-pp.native_sched_clock
0.26 ± 12% +0.2 0.46 ± 14% perf-profile.self.cycles-pp.tick_nohz_next_event
0.09 ± 18% +0.3 0.34 ± 67% perf-profile.self.cycles-pp.timerqueue_add
0.08 ± 20% +0.3 0.36 ± 65% perf-profile.self.cycles-pp.check_cpu_stall
0.58 +0.3 0.87 ± 7% perf-profile.self.cycles-pp.cpuidle_idle_call
0.35 ± 8% +0.4 0.78 ± 16% perf-profile.self.cycles-pp.update_sg_lb_stats
4.33 +0.4 4.78 ± 9% perf-profile.self.cycles-pp.cpuidle_enter_state
1.43 ± 34% +1.6 3.07 ± 33% perf-profile.self.cycles-pp.menu_select
31.31 +12.0 43.29 ± 5% perf-profile.self.cycles-pp.mwait_idle_with_hints
***************************************************************************************************
lkp-cfl-e1: 16 threads 1 sockets Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz (Coffee Lake) with 32G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase:
gcc-11/performance/x86_64-rhel-8.3/30%/debian-11.1-x86_64-20220510.cgz/300s/lkp-cfl-e1/context1/unixbench
commit:
c36eae5a86 ("sched/psi: Fix possible missing or delayed pending event")
b6aabb01e3 ("sched: consider WF_SYNC to find idle siblings")
c36eae5a86d15a69 b6aabb01e3004b0285d953761e1
---------------- ---------------------------
%stddev %change %stddev
\ | \
3547 +36.2% 4834 unixbench.score
1202 ± 4% +3.2e+05% 3838646 ± 17% unixbench.time.involuntary_context_switches
146.17 +4.7% 153.00 unixbench.time.percent_of_cpu_this_job_got
533.06 +1.7% 542.03 unixbench.time.system_time
40.02 +40.3% 56.14 unixbench.time.user_time
4.156e+08 +34.9% 5.604e+08 unixbench.time.voluntary_context_switches
5.537e+08 +36.3% 7.548e+08 unixbench.workload
1.53 +0.5 2.01 mpstat.cpu.all.usr%
4.434e+09 +13.2% 5.022e+09 cpuidle..time
8.352e+08 -98.7% 11131648 ± 12% cpuidle..usage
3.00 +33.3% 4.00 vmstat.procs.r
4212747 -32.1% 2861762 vmstat.system.cs
40960 -29.9% 28703 vmstat.system.in
161422 -32.1% 109641 meminfo.Active
161345 -32.1% 109569 meminfo.Active(anon)
530075 -10.9% 472307 meminfo.Committed_AS
80181 -27.2% 58384 meminfo.Mapped
162280 -32.2% 110057 meminfo.Shmem
40427 -31.8% 27576 proc-vmstat.nr_active_anon
978441 -1.3% 965308 proc-vmstat.nr_file_pages
20137 -26.7% 14769 proc-vmstat.nr_mapped
40661 -31.9% 27701 proc-vmstat.nr_shmem
13329 -0.9% 13206 proc-vmstat.nr_slab_unreclaimable
40427 -31.8% 27576 proc-vmstat.nr_zone_active_anon
1751058 -14.2% 1502883 proc-vmstat.numa_hit
1751425 -14.1% 1504002 proc-vmstat.numa_local
92726 -17.3% 76685 proc-vmstat.pgactivate
1819543 -13.7% 1570329 proc-vmstat.pgalloc_normal
684646 -2.4% 668006 proc-vmstat.pgfault
1807696 -13.7% 1559418 proc-vmstat.pgfree
40680 -1.8% 39968 proc-vmstat.pgreuse
1740 -47.5% 912.83 turbostat.Avg_MHz
38.54 -18.6 19.90 turbostat.Busy%
4515 +1.6% 4586 turbostat.Bzy_MHz
3885351 ± 9% -82.8% 667034 ± 73% turbostat.C1
26.41 ± 6% +50.7% 39.80 ± 19% turbostat.CPU%c6
74.05 -31.1% 50.99 ± 2% turbostat.CorWatt
60.17 -18.3% 49.17 ± 2% turbostat.CoreTmp
0.22 ± 2% +83.6% 0.41 turbostat.IPC
16161117 -30.0% 11319624 turbostat.IRQ
8.241e+08 -100.0% 3881 ± 26% turbostat.POLL
9.50 -9.5 0.00 turbostat.POLL%
58.67 ± 2% -17.0% 48.67 ± 2% turbostat.PkgTmp
74.52 -30.9% 51.46 ± 2% turbostat.PkgWatt
489.94 -35.7% 315.24 turbostat.Totl%C0
4851 ± 56% -100.0% 0.00 sched_debug.cfs_rq:/.MIN_vruntime.avg
56902 ± 44% -100.0% 0.00 sched_debug.cfs_rq:/.MIN_vruntime.max
15406 ± 47% -100.0% 0.00 sched_debug.cfs_rq:/.MIN_vruntime.stddev
0.44 ± 9% -14.4% 0.38 ± 2% sched_debug.cfs_rq:/.h_nr_running.avg
127353 +10.5% 140708 ± 3% sched_debug.cfs_rq:/.load.stddev
120.76 ± 8% +25.4% 151.41 ± 17% sched_debug.cfs_rq:/.load_avg.avg
4851 ± 56% -100.0% 0.00 sched_debug.cfs_rq:/.max_vruntime.avg
56902 ± 44% -100.0% 0.00 sched_debug.cfs_rq:/.max_vruntime.max
15406 ± 47% -100.0% 0.00 sched_debug.cfs_rq:/.max_vruntime.stddev
197189 -9.1% 179171 ± 2% sched_debug.cfs_rq:/.min_vruntime.max
116754 ± 6% +12.9% 131851 ± 3% sched_debug.cfs_rq:/.min_vruntime.min
21217 ± 10% -39.9% 12759 ± 18% sched_debug.cfs_rq:/.min_vruntime.stddev
0.44 ± 7% -18.5% 0.36 ± 3% sched_debug.cfs_rq:/.nr_running.avg
366.23 +21.1% 443.60 ± 3% sched_debug.cfs_rq:/.runnable_avg.avg
714.74 ± 4% +81.8% 1299 ± 3% sched_debug.cfs_rq:/.runnable_avg.max
246.75 ± 5% +101.3% 496.81 ± 3% sched_debug.cfs_rq:/.runnable_avg.stddev
51496 ± 44% -66.9% 17048 ± 49% sched_debug.cfs_rq:/.spread0.max
21217 ± 10% -39.9% 12760 ± 18% sched_debug.cfs_rq:/.spread0.stddev
708.50 ± 4% +49.8% 1061 ± 4% sched_debug.cfs_rq:/.util_avg.max
245.43 ± 5% +63.7% 401.71 ± 3% sched_debug.cfs_rq:/.util_avg.stddev
107.96 ± 11% +42.0% 153.34 ± 4% sched_debug.cfs_rq:/.util_est_enqueued.avg
499.12 ± 2% +65.3% 825.00 ± 4% sched_debug.cfs_rq:/.util_est_enqueued.max
181.05 ± 4% +58.9% 287.67 ± 4% sched_debug.cfs_rq:/.util_est_enqueued.stddev
407511 ± 4% +110.5% 857930 sched_debug.cpu.avg_idle.avg
104621 +462.1% 588066 ± 11% sched_debug.cpu.avg_idle.min
350206 ± 2% -68.5% 110282 ± 21% sched_debug.cpu.avg_idle.stddev
0.35 ± 18% +89.2% 0.67 ± 3% sched_debug.cpu.clock.stddev
0.00 ± 37% +68.1% 0.00 ± 9% sched_debug.cpu.next_balance.stddev
1.07 ± 6% +55.6% 1.67 ± 9% sched_debug.cpu.nr_running.max
0.46 ± 3% +24.0% 0.57 ± 4% sched_debug.cpu.nr_running.stddev
47178388 -32.4% 31875847 sched_debug.cpu.nr_switches.avg
56953632 ± 2% -36.6% 36093819 ± 2% sched_debug.cpu.nr_switches.max
31935712 ± 9% -13.3% 27680720 ± 3% sched_debug.cpu.nr_switches.min
6579393 ± 14% -64.6% 2330670 ± 21% sched_debug.cpu.nr_switches.stddev
950.00 -100.0% 0.00 sched_debug.rt_rq:/.rt_runtime.avg
950.00 -100.0% 0.00 sched_debug.rt_rq:/.rt_runtime.max
950.00 -100.0% 0.00 sched_debug.rt_rq:/.rt_runtime.min
65.64 -30.8% 45.42 perf-stat.i.MPKI
4.603e+09 -7.6% 4.252e+09 perf-stat.i.branch-instructions
2.81 +0.1 2.89 perf-stat.i.branch-miss-rate%
49172917 -1.4% 48506007 perf-stat.i.branch-misses
0.79 +2.6 3.39 ± 7% perf-stat.i.cache-miss-rate%
1887466 -4.2% 1808731 ± 2% perf-stat.i.cache-misses
5.404e+08 -94.1% 32037290 ± 3% perf-stat.i.cache-references
4247507 -32.1% 2884302 perf-stat.i.context-switches
1.99 -24.4% 1.50 perf-stat.i.cpi
2.784e+10 -47.7% 1.457e+10 perf-stat.i.cpu-cycles
52.17 ± 2% +634.1% 383.01 ± 4% perf-stat.i.cpu-migrations
56513 ± 2% -59.7% 22750 ± 9% perf-stat.i.cycles-between-cache-misses
410439 ± 2% +23.7% 507559 ± 7% perf-stat.i.dTLB-load-misses
6.418e+09 -3.1% 6.22e+09 perf-stat.i.dTLB-loads
3.826e+09 -4.0% 3.674e+09 perf-stat.i.dTLB-stores
95.43 -38.9 56.58 perf-stat.i.iTLB-load-miss-rate%
16498563 -5.9% 15523524 perf-stat.i.iTLB-load-misses
239550 ± 3% +5782.0% 14090457 perf-stat.i.iTLB-loads
2.197e+10 -4.8% 2.092e+10 perf-stat.i.instructions
0.70 +78.1% 1.25 perf-stat.i.ipc
1.74 -47.7% 0.91 perf-stat.i.metric.GHz
404.88 ± 2% -4.8% 385.49 ± 2% perf-stat.i.metric.K/sec
961.54 -7.8% 886.77 perf-stat.i.metric.M/sec
1575 -2.7% 1533 perf-stat.i.minor-faults
0.40 ± 4% +0.1 0.47 ± 7% perf-stat.i.node-load-miss-rate%
261859 -14.5% 223787 perf-stat.i.node-stores
1575 -2.7% 1533 perf-stat.i.page-faults
24.60 -93.8% 1.53 ± 3% perf-stat.overall.MPKI
1.07 +0.1 1.14 perf-stat.overall.branch-miss-rate%
0.35 +5.3 5.67 ± 5% perf-stat.overall.cache-miss-rate%
1.27 -45.0% 0.70 perf-stat.overall.cpi
14726 -45.3% 8048 ± 2% perf-stat.overall.cycles-between-cache-misses
0.01 ± 2% +0.0 0.01 ± 7% perf-stat.overall.dTLB-load-miss-rate%
98.57 -46.1 52.42 perf-stat.overall.iTLB-load-miss-rate%
1331 +1.2% 1347 perf-stat.overall.instructions-per-iTLB-miss
0.79 +81.9% 1.44 perf-stat.overall.ipc
0.00 ± 11% +0.0 0.01 ± 10% perf-stat.overall.node-store-miss-rate%
15528 -30.1% 10847 perf-stat.overall.path-length
4.591e+09 -7.6% 4.24e+09 perf-stat.ps.branch-instructions
49052787 -1.4% 48388408 perf-stat.ps.branch-misses
1885579 -4.2% 1807271 ± 2% perf-stat.ps.cache-misses
5.389e+08 -94.1% 31961586 ± 3% perf-stat.ps.cache-references
4235546 -32.1% 2876157 perf-stat.ps.context-switches
2.777e+10 -47.6% 1.454e+10 perf-stat.ps.cpu-cycles
52.05 ± 2% +633.9% 381.95 ± 4% perf-stat.ps.cpu-migrations
409410 ± 2% +23.7% 506263 ± 7% perf-stat.ps.dTLB-load-misses
6.4e+09 -3.1% 6.203e+09 perf-stat.ps.dTLB-loads
3.815e+09 -4.0% 3.664e+09 perf-stat.ps.dTLB-stores
16452159 -5.9% 15479733 perf-stat.ps.iTLB-load-misses
238902 ± 3% +5781.4% 14050699 perf-stat.ps.iTLB-loads
2.19e+10 -4.8% 2.086e+10 perf-stat.ps.instructions
1571 -2.7% 1529 perf-stat.ps.minor-faults
261165 -14.5% 223199 perf-stat.ps.node-stores
1571 -2.7% 1529 perf-stat.ps.page-faults
8.598e+12 -4.8% 8.188e+12 perf-stat.total.instructions
57.34 -35.5 21.87 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
53.70 ± 2% -32.8 20.91 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
53.66 ± 2% -32.8 20.91 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
53.51 ± 2% -32.6 20.91 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
21.60 -21.6 0.00 perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
38.36 ± 2% -17.5 20.86 perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
35.56 ± 2% -14.1 21.44 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
33.37 ± 2% -12.8 20.57 perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
7.88 ± 2% -7.9 0.00 perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
5.64 -5.6 0.00 perf-profile.calltrace.cycles-pp.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary
3.63 ± 24% -2.7 0.96 ± 31% perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64_no_verify
3.63 ± 24% -2.7 0.96 ± 31% perf-profile.calltrace.cycles-pp.arch_call_rest_init.start_kernel.secondary_startup_64_no_verify
3.63 ± 24% -2.7 0.96 ± 31% perf-profile.calltrace.cycles-pp.rest_init.arch_call_rest_init.start_kernel.secondary_startup_64_no_verify
3.63 ± 24% -2.7 0.96 ± 31% perf-profile.calltrace.cycles-pp.cpu_startup_entry.rest_init.arch_call_rest_init.start_kernel.secondary_startup_64_no_verify
3.62 ± 24% -2.7 0.96 ± 31% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.rest_init.arch_call_rest_init.start_kernel
2.64 ± 18% -1.7 0.96 ± 31% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.rest_init.arch_call_rest_init
2.31 ± 15% -1.4 0.94 ± 31% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.rest_init
4.08 ± 2% -1.4 2.73 perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_read
1.90 ± 3% -0.8 1.11 ± 5% perf-profile.calltrace.cycles-pp.select_task_rq_fair.select_task_rq.try_to_wake_up.autoremove_wake_function.__wake_up_common
2.12 ± 2% -0.8 1.32 ± 3% perf-profile.calltrace.cycles-pp.select_task_rq.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.86 ± 4% -0.6 0.26 ±100% perf-profile.calltrace.cycles-pp.copy_user_short_string.copyout._copy_to_iter.copy_page_to_iter.pipe_read
1.60 ± 2% -0.4 1.16 ± 3% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
1.12 ± 2% -0.4 0.70 ± 3% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.vfs_write.ksys_write.do_syscall_64
0.97 ± 4% -0.2 0.75 ± 5% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule
1.20 ± 4% -0.2 1.00 ± 5% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.prepare_to_wait_event.pipe_read.vfs_read.ksys_read
1.04 ± 4% -0.1 0.90 ± 5% perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.copy_page_to_iter.pipe_read.vfs_read
1.31 ± 3% +0.1 1.40 ± 2% perf-profile.calltrace.cycles-pp._copy_to_iter.copy_page_to_iter.pipe_read.vfs_read.ksys_read
1.41 +0.1 1.53 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.vfs_read.ksys_read.do_syscall_64
1.04 +0.2 1.21 ± 4% perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
2.68 ± 2% +0.2 2.85 perf-profile.calltrace.cycles-pp.prepare_to_wait_event.pipe_read.vfs_read.ksys_read.do_syscall_64
0.56 ± 4% +0.2 0.80 ± 7% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.42 ± 2% +0.3 1.70 perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.vfs_read.ksys_read.do_syscall_64
0.26 ±100% +0.4 0.69 ± 6% perf-profile.calltrace.cycles-pp.apparmor_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64
0.56 ± 3% +0.5 1.07 ± 8% perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule.pipe_read.vfs_read
0.00 +0.6 0.56 ± 11% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +0.6 0.56 ± 7% perf-profile.calltrace.cycles-pp.__might_resched.__might_fault._copy_from_iter.copy_page_from_iter.pipe_write
0.00 +0.6 0.58 ± 3% perf-profile.calltrace.cycles-pp.apparmor_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64
0.00 +0.6 0.58 ± 5% perf-profile.calltrace.cycles-pp.native_sched_clock.sched_clock_cpu.update_rq_clock.__schedule.schedule
0.00 +0.6 0.59 ± 3% perf-profile.calltrace.cycles-pp.native_sched_clock.sched_clock_cpu.update_rq_clock.try_to_wake_up.autoremove_wake_function
0.00 +0.6 0.62 ± 6% perf-profile.calltrace.cycles-pp.sched_clock_cpu.update_rq_clock.__schedule.schedule.pipe_read
0.00 +0.6 0.63 ± 2% perf-profile.calltrace.cycles-pp.sched_clock_cpu.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.00 +0.6 0.63 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.finish_wait.pipe_read.vfs_read.ksys_read
0.00 +0.7 0.66 ± 10% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
0.00 +0.7 0.66 ± 4% perf-profile.calltrace.cycles-pp.__wrgsbase_inactive.read
0.00 +0.7 0.68 ± 8% perf-profile.calltrace.cycles-pp.___perf_sw_event.prepare_task_switch.__schedule.schedule.pipe_read
0.00 +0.7 0.69 ± 4% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.7 0.70 ± 6% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_task_fair.__schedule.schedule.pipe_read
0.00 +0.7 0.70 ± 8% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.vfs_read.ksys_read.do_syscall_64
0.65 ± 5% +0.7 1.37 ± 3% perf-profile.calltrace.cycles-pp.os_xsave.read
1.26 ± 4% +0.7 1.98 ± 2% perf-profile.calltrace.cycles-pp._copy_from_iter.copy_page_from_iter.pipe_write.vfs_write.ksys_write
0.00 +0.7 0.72 ± 10% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
0.00 +0.7 0.74 ± 5% perf-profile.calltrace.cycles-pp.copyin._copy_from_iter.copy_page_from_iter.pipe_write.vfs_write
0.00 +0.8 0.76 ± 3% perf-profile.calltrace.cycles-pp.update_load_avg.set_next_entity.pick_next_task_fair.__schedule.schedule
0.00 +0.8 0.76 ± 6% perf-profile.calltrace.cycles-pp.__calc_delta.update_curr.reweight_entity.dequeue_task_fair.__schedule
0.00 +0.8 0.80 ± 4% perf-profile.calltrace.cycles-pp.__calc_delta.update_curr.reweight_entity.enqueue_task_fair.ttwu_do_activate
0.00 +0.8 0.81 ± 4% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up
0.00 +0.8 0.82 ± 9% perf-profile.calltrace.cycles-pp.__switch_to.read
0.00 +0.8 0.82 ± 4% perf-profile.calltrace.cycles-pp.finish_wait.pipe_read.vfs_read.ksys_read.do_syscall_64
0.00 +0.9 0.87 ± 3% perf-profile.calltrace.cycles-pp.update_rq_clock.__schedule.schedule.pipe_read.vfs_read
0.00 +0.9 0.88 ± 4% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
0.00 +0.9 0.89 ± 3% perf-profile.calltrace.cycles-pp.main
0.00 +0.9 0.91 ± 5% perf-profile.calltrace.cycles-pp.update_cfs_group.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
0.00 +0.9 0.91 ± 5% perf-profile.calltrace.cycles-pp.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.00 +0.9 0.94 ± 2% perf-profile.calltrace.cycles-pp.__switch_to.__schedule.schedule.pipe_read.vfs_read
1.43 ± 3% +1.0 2.38 ± 2% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.vfs_write.ksys_write.do_syscall_64
0.00 +1.0 0.95 ± 2% perf-profile.calltrace.cycles-pp.update_cfs_group.dequeue_task_fair.__schedule.schedule.pipe_read
0.00 +1.0 1.01 ± 5% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule
0.00 +1.0 1.01 ± 4% perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function
0.91 ± 2% +1.0 1.93 ± 3% perf-profile.calltrace.cycles-pp.__switch_to_asm.read
0.00 +1.0 1.04 ± 4% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up
0.00 +1.2 1.18 ± 3% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_read
0.00 +1.3 1.27 ± 4% perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.00 +1.3 1.30 ± 4% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up
0.00 +1.4 1.38 ± 3% perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.54 ± 3% +1.4 1.94 ± 3% perf-profile.calltrace.cycles-pp.__entry_text_start.read
0.00 +1.6 1.58 ± 3% perf-profile.calltrace.cycles-pp.restore_fpregs_from_fpstate.switch_fpu_return.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
0.17 ±141% +1.7 1.84 ± 3% perf-profile.calltrace.cycles-pp.__entry_text_start.write
0.00 +1.7 1.67 ± 3% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_read
0.00 +1.7 1.68 ± 2% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
0.60 ± 5% +2.1 2.66 ± 2% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
0.00 +2.2 2.21 ± 2% perf-profile.calltrace.cycles-pp.switch_fpu_return.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.58 ± 6% +2.2 2.82 perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_read.vfs_read
0.00 +2.3 2.30 ± 3% perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
4.70 +2.4 7.11 perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_read.vfs_read
0.00 +3.1 3.15 ± 3% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
0.00 +5.1 5.05 ± 2% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.pipe_read.vfs_read
0.00 +7.3 7.32 perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
7.83 +7.3 15.18 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write.ksys_write
0.00 +7.5 7.52 perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
6.88 ± 2% +7.6 14.49 perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
8.24 +7.8 16.01 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.vfs_write.ksys_write.do_syscall_64
7.11 +7.8 14.94 perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write
11.86 ± 6% +8.6 20.50 ± 2% perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
11.86 ± 6% +8.6 20.51 ± 2% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
12.29 +9.5 21.75 perf-profile.calltrace.cycles-pp.pipe_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
13.40 +10.0 23.36 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
13.83 +10.4 24.21 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
14.17 +10.9 25.10 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
14.33 +11.1 25.46 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
9.16 +11.3 20.49 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.vfs_read.ksys_read
9.44 +11.9 21.39 perf-profile.calltrace.cycles-pp.schedule.pipe_read.vfs_read.ksys_read.do_syscall_64
15.53 +13.3 28.86 perf-profile.calltrace.cycles-pp.write
18.03 +14.7 32.74 perf-profile.calltrace.cycles-pp.pipe_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
19.37 +15.7 35.10 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
19.90 +16.3 36.21 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
20.88 +18.6 39.51 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
21.30 +19.5 40.84 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read
24.96 +24.5 49.46 perf-profile.calltrace.cycles-pp.read
57.34 -35.5 21.87 perf-profile.children.cycles-pp.secondary_startup_64_no_verify
57.34 -35.5 21.87 perf-profile.children.cycles-pp.cpu_startup_entry
57.17 -35.3 21.87 perf-profile.children.cycles-pp.do_idle
53.70 ± 2% -32.8 20.91 perf-profile.children.cycles-pp.start_secondary
21.62 -21.6 0.00 perf-profile.children.cycles-pp.poll_idle
41.05 -19.2 21.83 perf-profile.children.cycles-pp.cpuidle_idle_call
35.70 ± 2% -14.2 21.51 perf-profile.children.cycles-pp.cpuidle_enter
35.64 ± 2% -14.1 21.51 perf-profile.children.cycles-pp.cpuidle_enter_state
8.40 -8.4 0.00 perf-profile.children.cycles-pp.flush_smp_call_function_queue
6.02 -6.0 0.00 perf-profile.children.cycles-pp.sched_ttwu_pending
5.20 ± 2% -5.2 0.00 perf-profile.children.cycles-pp.schedule_idle
4.01 -3.8 0.25 ± 11% perf-profile.children.cycles-pp.menu_select
3.63 ± 24% -2.7 0.96 ± 31% perf-profile.children.cycles-pp.start_kernel
3.63 ± 24% -2.7 0.96 ± 31% perf-profile.children.cycles-pp.arch_call_rest_init
3.63 ± 24% -2.7 0.96 ± 31% perf-profile.children.cycles-pp.rest_init
2.59 ± 3% -2.3 0.32 ± 10% perf-profile.children.cycles-pp.ttwu_queue_wakelist
1.83 ± 2% -1.7 0.09 ± 20% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
4.18 ± 2% -1.4 2.81 perf-profile.children.cycles-pp.dequeue_entity
1.13 ± 2% -1.1 0.04 ± 72% perf-profile.children.cycles-pp.tick_nohz_next_event
1.92 ± 3% -0.8 1.11 ± 5% perf-profile.children.cycles-pp.select_task_rq_fair
2.12 ± 2% -0.8 1.34 ± 4% perf-profile.children.cycles-pp.select_task_rq
0.72 ± 3% -0.7 0.06 ± 47% perf-profile.children.cycles-pp.ktime_get
1.53 ± 2% -0.6 0.90 ± 4% perf-profile.children.cycles-pp._raw_spin_lock
1.86 -0.5 1.33 perf-profile.children.cycles-pp.sched_clock_cpu
1.63 ± 2% -0.4 1.20 perf-profile.children.cycles-pp.native_sched_clock
3.03 ± 3% -0.4 2.65 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
1.45 ± 3% -0.4 1.09 ± 8% perf-profile.children.cycles-pp.prepare_task_switch
2.58 -0.3 2.30 perf-profile.children.cycles-pp.mutex_lock
0.53 ± 4% -0.2 0.34 ± 6% perf-profile.children.cycles-pp.finish_task_switch
1.05 ± 4% -0.1 0.90 ± 5% perf-profile.children.cycles-pp.copyout
1.38 ± 5% -0.1 1.26 ± 3% perf-profile.children.cycles-pp.set_next_entity
0.17 ± 10% -0.1 0.09 ± 4% perf-profile.children.cycles-pp.rcu_note_context_switch
0.18 ± 9% -0.0 0.13 ± 16% perf-profile.children.cycles-pp.place_entity
0.10 ± 6% -0.0 0.05 ± 48% perf-profile.children.cycles-pp.rb_next
0.08 ± 11% -0.0 0.06 ± 11% perf-profile.children.cycles-pp.sched_clock
0.06 ± 11% +0.0 0.10 ± 13% perf-profile.children.cycles-pp.rw_verify_area
0.08 ± 11% +0.0 0.12 ± 11% perf-profile.children.cycles-pp.__x64_sys_read
0.30 ± 8% +0.0 0.34 ± 8% perf-profile.children.cycles-pp.perf_trace_buf_update
0.02 ±141% +0.0 0.06 ± 11% perf-profile.children.cycles-pp.clockevents_program_event
0.06 ± 45% +0.0 0.10 ± 13% perf-profile.children.cycles-pp.__softirqentry_text_start
0.04 ± 71% +0.0 0.08 ± 8% perf-profile.children.cycles-pp.perf_swevent_get_recursion_context
0.04 ± 71% +0.1 0.09 ± 14% perf-profile.children.cycles-pp.inode_needs_update_time
0.03 ±102% +0.1 0.08 ± 19% perf-profile.children.cycles-pp.save_fpregs_to_fpstate
0.22 ± 6% +0.1 0.28 ± 11% perf-profile.children.cycles-pp.put_prev_entity
0.01 ±223% +0.1 0.06 ± 11% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.08 ± 14% +0.1 0.13 ± 16% perf-profile.children.cycles-pp.__irq_exit_rcu
0.12 ± 6% +0.1 0.17 ± 4% perf-profile.children.cycles-pp.perf_trace_buf_alloc
0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.default_wake_function
0.31 ± 5% +0.1 0.38 ± 6% perf-profile.children.cycles-pp.__rdgsbase_inactive
0.09 ± 9% +0.1 0.16 ± 10% perf-profile.children.cycles-pp.__x64_sys_write
0.09 ± 10% +0.1 0.16 ± 8% perf-profile.children.cycles-pp.rcu_all_qs
0.04 ± 44% +0.1 0.12 ± 16% perf-profile.children.cycles-pp.page_copy_sane
0.04 ± 71% +0.1 0.11 ± 18% perf-profile.children.cycles-pp.scheduler_tick
0.30 ± 8% +0.1 0.38 ± 4% perf-profile.children.cycles-pp.pick_next_entity
0.11 ± 15% +0.1 0.20 ± 7% perf-profile.children.cycles-pp.syscall_enter_from_user_mode
0.09 ± 15% +0.1 0.18 ± 17% perf-profile.children.cycles-pp.update_process_times
0.05 ± 46% +0.1 0.14 ± 14% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64
0.07 ± 11% +0.1 0.15 ± 15% perf-profile.children.cycles-pp.perf_trace_run_bpf_submit
0.06 ± 11% +0.1 0.15 ± 12% perf-profile.children.cycles-pp.kill_fasync
0.10 ± 16% +0.1 0.20 ± 17% perf-profile.children.cycles-pp.tick_sched_handle
0.04 ± 72% +0.1 0.14 ± 34% perf-profile.children.cycles-pp.write@plt
0.12 ± 19% +0.1 0.23 ± 18% perf-profile.children.cycles-pp.tick_sched_timer
0.00 +0.1 0.12 ± 36% perf-profile.children.cycles-pp.exit_to_user_mode_loop
0.19 ± 9% +0.1 0.31 ± 9% perf-profile.children.cycles-pp.__get_task_ioprio
0.57 ± 6% +0.1 0.68 ± 4% perf-profile.children.cycles-pp.__wrgsbase_inactive
0.13 ± 15% +0.1 0.25 ± 7% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare
1.32 ± 3% +0.1 1.44 ± 2% perf-profile.children.cycles-pp._copy_to_iter
1.54 ± 2% +0.1 1.67 perf-profile.children.cycles-pp.__might_resched
0.00 +0.1 0.14 ± 11% perf-profile.children.cycles-pp.cgroup_rstat_updated
0.14 ± 9% +0.2 0.30 ± 6% perf-profile.children.cycles-pp.aa_file_perm
0.09 ± 20% +0.2 0.26 ± 5% perf-profile.children.cycles-pp.__cgroup_account_cputime
0.14 ± 13% +0.2 0.31 ± 12% perf-profile.children.cycles-pp.__list_del_entry_valid
0.14 ± 11% +0.2 0.31 ± 5% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
0.20 ± 14% +0.2 0.37 ± 12% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.21 ± 9% +0.2 0.38 ± 5% perf-profile.children.cycles-pp.__cond_resched
1.06 +0.2 1.23 ± 4% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
0.06 ± 19% +0.2 0.24 ± 8% perf-profile.children.cycles-pp.check_cfs_rq_runtime
0.22 ± 7% +0.2 0.41 ± 6% perf-profile.children.cycles-pp.atime_needs_update
0.00 +0.2 0.22 ± 6% perf-profile.children.cycles-pp.set_next_buddy
2.70 ± 2% +0.2 2.92 perf-profile.children.cycles-pp.prepare_to_wait_event
0.28 ± 11% +0.2 0.51 ± 9% perf-profile.children.cycles-pp.hrtimer_interrupt
0.88 ± 5% +0.2 1.12 ± 4% perf-profile.children.cycles-pp.__might_fault
0.22 ± 8% +0.2 0.46 ± 6% perf-profile.children.cycles-pp.current_time
0.28 ± 7% +0.2 0.52 ± 5% perf-profile.children.cycles-pp.touch_atime
0.30 ± 11% +0.2 0.55 ± 8% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.21 ± 9% +0.2 0.46 ± 8% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.22 ± 8% +0.2 0.47 ± 6% perf-profile.children.cycles-pp.file_update_time
0.62 ± 2% +0.3 0.88 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irq
2.96 ± 2% +0.3 3.23 ± 3% perf-profile.children.cycles-pp.enqueue_entity
1.44 ± 2% +0.3 1.72 perf-profile.children.cycles-pp.copy_page_to_iter
1.53 ± 3% +0.3 1.81 ± 3% perf-profile.children.cycles-pp.__switch_to
0.26 ± 6% +0.3 0.57 ± 5% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.21 ± 7% +0.3 0.53 ± 7% perf-profile.children.cycles-pp.update_min_vruntime
2.68 ± 5% +0.3 3.02 perf-profile.children.cycles-pp.pick_next_task_fair
0.12 ± 8% +0.3 0.46 ± 4% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
0.42 ± 8% +0.4 0.78 ± 9% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.73 ± 6% +0.4 1.11 ± 5% perf-profile.children.cycles-pp.mutex_unlock
0.48 ± 7% +0.4 0.86 ± 8% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.87 ± 2% +0.4 1.27 ± 4% perf-profile.children.cycles-pp.__update_load_avg_se
0.90 ± 3% +0.4 1.30 ± 4% perf-profile.children.cycles-pp.apparmor_file_permission
0.14 ± 15% +0.4 0.54 ± 2% perf-profile.children.cycles-pp.__list_add_valid
0.89 ± 4% +0.4 1.30 ± 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.37 ± 7% +0.4 0.78 ± 8% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.33 ± 4% +0.4 0.75 ± 5% perf-profile.children.cycles-pp.copyin
0.36 ± 4% +0.5 0.82 ± 4% perf-profile.children.cycles-pp.finish_wait
1.03 ± 3% +0.5 1.50 ± 4% perf-profile.children.cycles-pp.security_file_permission
0.18 ± 16% +0.5 0.68 ± 7% perf-profile.children.cycles-pp.cpuacct_charge
0.40 ± 6% +0.5 0.93 ± 3% perf-profile.children.cycles-pp.main
0.53 ± 2% +0.6 1.12 ± 7% perf-profile.children.cycles-pp.__fdget_pos
0.65 ± 5% +0.7 1.38 ± 3% perf-profile.children.cycles-pp.os_xsave
1.30 ± 4% +0.7 2.04 ± 2% perf-profile.children.cycles-pp._copy_from_iter
0.65 ± 3% +0.8 1.41 ± 2% perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
1.02 ± 2% +0.8 1.82 ± 2% perf-profile.children.cycles-pp.update_rq_clock
1.44 ± 3% +1.0 2.42 ± 2% perf-profile.children.cycles-pp.copy_page_from_iter
0.35 ± 5% +1.0 1.38 ± 3% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.00 +1.0 1.04 ± 5% perf-profile.children.cycles-pp.check_preempt_wakeup
0.21 ± 7% +1.1 1.28 ± 4% perf-profile.children.cycles-pp.check_preempt_curr
0.46 ± 5% +1.1 1.56 ± 2% perf-profile.children.cycles-pp.__calc_delta
0.83 ± 5% +1.1 1.96 ± 2% perf-profile.children.cycles-pp.update_cfs_group
2.68 ± 4% +1.3 3.98 ± 3% perf-profile.children.cycles-pp.update_load_avg
1.02 +1.4 2.47 ± 2% perf-profile.children.cycles-pp.__entry_text_start
0.00 +1.6 1.59 ± 3% perf-profile.children.cycles-pp.restore_fpregs_from_fpstate
0.36 ± 6% +1.9 2.23 ± 3% perf-profile.children.cycles-pp.switch_fpu_return
0.49 ± 4% +2.1 2.61 ± 4% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
4.72 +2.4 7.12 perf-profile.children.cycles-pp.dequeue_task_fair
0.74 ± 6% +2.4 3.15 ± 3% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.63 ± 4% +2.7 3.35 perf-profile.children.cycles-pp.reweight_entity
1.74 ± 3% +3.2 4.91 ± 2% perf-profile.children.cycles-pp.update_curr
3.34 ± 2% +4.0 7.34 perf-profile.children.cycles-pp.enqueue_task_fair
3.44 ± 2% +4.1 7.54 perf-profile.children.cycles-pp.ttwu_do_activate
0.26 ± 6% +4.9 5.12 ± 3% perf-profile.children.cycles-pp.switch_mm_irqs_off
14.17 ± 2% +6.6 20.77 perf-profile.children.cycles-pp.__schedule
7.84 +7.4 15.19 perf-profile.children.cycles-pp.__wake_up_common
6.90 +7.7 14.56 perf-profile.children.cycles-pp.try_to_wake_up
8.26 +7.8 16.03 perf-profile.children.cycles-pp.__wake_up_common_lock
7.12 +7.9 14.98 perf-profile.children.cycles-pp.autoremove_wake_function
11.94 ± 6% +8.6 20.59 ± 2% perf-profile.children.cycles-pp.mwait_idle_with_hints
11.95 ± 6% +8.6 20.59 ± 2% perf-profile.children.cycles-pp.intel_idle
12.32 +9.5 21.81 perf-profile.children.cycles-pp.pipe_write
13.43 +10.0 23.43 perf-profile.children.cycles-pp.vfs_write
13.85 +10.4 24.25 perf-profile.children.cycles-pp.ksys_write
9.45 +12.1 21.52 perf-profile.children.cycles-pp.schedule
15.82 +13.1 28.92 perf-profile.children.cycles-pp.write
18.16 +14.9 33.02 perf-profile.children.cycles-pp.pipe_read
19.41 +15.7 35.11 perf-profile.children.cycles-pp.vfs_read
19.92 +16.3 36.24 perf-profile.children.cycles-pp.ksys_read
25.25 +24.2 49.48 perf-profile.children.cycles-pp.read
35.17 +29.7 64.91 perf-profile.children.cycles-pp.do_syscall_64
35.72 +30.8 66.50 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
21.21 -21.2 0.00 perf-profile.self.cycles-pp.poll_idle
1.76 ± 2% -1.6 0.14 ± 11% perf-profile.self.cycles-pp.menu_select
1.52 ± 2% -0.6 0.90 ± 4% perf-profile.self.cycles-pp._raw_spin_lock
0.72 ± 5% -0.5 0.23 ± 11% perf-profile.self.cycles-pp.__wake_up_common
2.98 ± 3% -0.4 2.55 ± 2% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
1.58 ± 2% -0.4 1.16 perf-profile.self.cycles-pp.native_sched_clock
0.78 ± 2% -0.4 0.40 ± 14% perf-profile.self.cycles-pp.prepare_task_switch
0.32 ± 8% -0.2 0.11 ± 13% perf-profile.self.cycles-pp.cpuidle_enter_state
0.47 ± 5% -0.2 0.28 ± 8% perf-profile.self.cycles-pp.finish_task_switch
1.24 -0.2 1.06 ± 3% perf-profile.self.cycles-pp.mutex_lock
1.03 ± 3% -0.2 0.86 ± 2% perf-profile.self.cycles-pp.enqueue_entity
0.46 ± 13% -0.2 0.31 ± 12% perf-profile.self.cycles-pp.ttwu_queue_wakelist
0.20 ± 5% -0.1 0.10 ± 11% perf-profile.self.cycles-pp.sched_clock_cpu
0.16 ± 8% -0.1 0.08 ± 6% perf-profile.self.cycles-pp.rcu_note_context_switch
0.38 ± 9% -0.1 0.31 ± 10% perf-profile.self.cycles-pp.set_next_entity
0.35 ± 6% -0.1 0.29 ± 7% perf-profile.self.cycles-pp.perf_tp_event
0.16 ± 6% -0.0 0.11 ± 13% perf-profile.self.cycles-pp.put_prev_entity
0.16 ± 7% -0.0 0.12 ± 14% perf-profile.self.cycles-pp.place_entity
0.09 ± 7% -0.0 0.05 ± 47% perf-profile.self.cycles-pp.rb_next
0.07 ± 13% +0.0 0.09 ± 11% perf-profile.self.cycles-pp.perf_trace_buf_alloc
0.08 ± 12% +0.0 0.11 ± 13% perf-profile.self.cycles-pp.__x64_sys_read
0.14 ± 3% +0.0 0.17 ± 10% perf-profile.self.cycles-pp.__might_fault
0.20 ± 9% +0.0 0.24 ± 8% perf-profile.self.cycles-pp.select_task_rq
0.02 ±141% +0.0 0.06 ± 8% perf-profile.self.cycles-pp.perf_swevent_event
0.03 ± 70% +0.0 0.08 ± 14% perf-profile.self.cycles-pp.inode_needs_update_time
0.07 ± 11% +0.0 0.12 ± 14% perf-profile.self.cycles-pp.perf_trace_buf_update
0.06 ± 17% +0.1 0.12 ± 9% perf-profile.self.cycles-pp.rcu_all_qs
0.23 ± 7% +0.1 0.28 ± 3% perf-profile.self.cycles-pp.perf_trace_sched_wakeup_template
0.06 ± 7% +0.1 0.12 ± 12% perf-profile.self.cycles-pp.finish_wait
0.08 ± 11% +0.1 0.14 ± 14% perf-profile.self.cycles-pp.__x64_sys_write
0.11 ± 11% +0.1 0.18 ± 5% perf-profile.self.cycles-pp.atime_needs_update
0.31 ± 4% +0.1 0.38 ± 6% perf-profile.self.cycles-pp.__rdgsbase_inactive
0.05 ± 51% +0.1 0.12 ± 14% perf-profile.self.cycles-pp.__cgroup_account_cputime
0.01 ±223% +0.1 0.08 ± 8% perf-profile.self.cycles-pp.perf_swevent_get_recursion_context
0.05 ± 46% +0.1 0.12 ± 14% perf-profile.self.cycles-pp.touch_atime
0.01 ±223% +0.1 0.08 ± 17% perf-profile.self.cycles-pp.rw_verify_area
0.15 ± 8% +0.1 0.23 ± 6% perf-profile.self.cycles-pp.check_preempt_curr
0.13 ± 11% +0.1 0.20 ± 3% perf-profile.self.cycles-pp.security_file_permission
0.06 ± 11% +0.1 0.14 ± 13% perf-profile.self.cycles-pp.perf_trace_run_bpf_submit
0.04 ± 71% +0.1 0.12 ± 34% perf-profile.self.cycles-pp.write@plt
0.10 ± 6% +0.1 0.19 ± 13% perf-profile.self.cycles-pp.ttwu_do_activate
0.05 ± 46% +0.1 0.14 ± 18% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64
0.42 ± 7% +0.1 0.50 ± 7% perf-profile.self.cycles-pp.update_rq_clock
0.10 ± 11% +0.1 0.20 ± 7% perf-profile.self.cycles-pp.syscall_enter_from_user_mode
0.02 ± 99% +0.1 0.12 ± 15% perf-profile.self.cycles-pp.kill_fasync
0.11 ± 12% +0.1 0.20 ± 9% perf-profile.self.cycles-pp.__cond_resched
0.10 ± 3% +0.1 0.20 ± 9% perf-profile.self.cycles-pp.syscall_exit_to_user_mode
0.06 ± 11% +0.1 0.16 ± 14% perf-profile.self.cycles-pp.file_update_time
0.00 +0.1 0.10 ± 15% perf-profile.self.cycles-pp.page_copy_sane
0.18 ± 7% +0.1 0.28 ± 8% perf-profile.self.cycles-pp.__get_task_ioprio
0.11 ± 8% +0.1 0.22 ± 20% perf-profile.self.cycles-pp.__wake_up_common_lock
0.12 ± 17% +0.1 0.23 ± 5% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare
0.57 ± 6% +0.1 0.68 ± 4% perf-profile.self.cycles-pp.__wrgsbase_inactive
0.12 ± 6% +0.1 0.24 ± 9% perf-profile.self.cycles-pp.exit_to_user_mode_prepare
0.10 ± 8% +0.1 0.22 ± 4% perf-profile.self.cycles-pp.copy_page_to_iter
0.00 +0.1 0.13 ± 10% perf-profile.self.cycles-pp.cgroup_rstat_updated
0.12 ± 10% +0.1 0.25 ± 8% perf-profile.self.cycles-pp.ksys_write
0.17 ± 9% +0.1 0.30 ± 7% perf-profile.self.cycles-pp.autoremove_wake_function
1.54 ± 2% +0.1 1.67 perf-profile.self.cycles-pp.__might_resched
0.14 ± 14% +0.1 0.28 ± 13% perf-profile.self.cycles-pp.__list_del_entry_valid
0.13 ± 12% +0.1 0.27 ± 6% perf-profile.self.cycles-pp.aa_file_perm
0.16 ± 13% +0.1 0.30 ± 4% perf-profile.self.cycles-pp.current_time
0.14 ± 3% +0.2 0.30 ± 8% perf-profile.self.cycles-pp._copy_to_iter
0.14 ± 11% +0.2 0.31 ± 5% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
0.03 ±101% +0.2 0.21 ± 10% perf-profile.self.cycles-pp.check_cfs_rq_runtime
0.12 ± 10% +0.2 0.30 ± 7% perf-profile.self.cycles-pp.copy_page_from_iter
0.41 ± 4% +0.2 0.60 ± 5% perf-profile.self.cycles-pp.vfs_write
0.54 ± 4% +0.2 0.76 perf-profile.self.cycles-pp.dequeue_task_fair
0.00 +0.2 0.22 ± 6% perf-profile.self.cycles-pp.set_next_buddy
0.38 ± 9% +0.2 0.62 ± 5% perf-profile.self.cycles-pp.enqueue_task_fair
0.25 ± 5% +0.2 0.50 perf-profile.self.cycles-pp._copy_from_iter
0.21 ± 7% +0.3 0.46 ± 8% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.61 +0.3 0.87 ± 3% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.76 ± 5% +0.3 1.02 ± 4% perf-profile.self.cycles-pp.apparmor_file_permission
0.35 ± 6% +0.3 0.62 ± 4% perf-profile.self.cycles-pp.switch_fpu_return
1.48 ± 3% +0.3 1.76 ± 2% perf-profile.self.cycles-pp.__switch_to
0.18 ± 7% +0.3 0.46 ± 6% perf-profile.self.cycles-pp.update_min_vruntime
0.44 ± 7% +0.3 0.73 ± 6% perf-profile.self.cycles-pp.pick_next_task_fair
0.22 ± 7% +0.3 0.54 ± 5% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.28 ± 7% +0.3 0.61 ± 7% perf-profile.self.cycles-pp.ksys_read
0.12 ± 11% +0.3 0.46 ± 4% perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
0.73 ± 6% +0.4 1.09 ± 6% perf-profile.self.cycles-pp.mutex_unlock
0.36 ± 8% +0.4 0.74 ± 8% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.85 +0.4 1.24 ± 4% perf-profile.self.cycles-pp.__update_load_avg_se
0.35 ± 6% +0.4 0.74 ± 4% perf-profile.self.cycles-pp.try_to_wake_up
0.14 ± 16% +0.4 0.54 ± 2% perf-profile.self.cycles-pp.__list_add_valid
0.84 ± 4% +0.4 1.27 ± 3% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.38 ± 3% +0.5 0.84 ± 4% perf-profile.self.cycles-pp.do_syscall_64
0.39 ± 8% +0.5 0.86 ± 5% perf-profile.self.cycles-pp.read
0.40 ± 6% +0.5 0.88 ± 4% perf-profile.self.cycles-pp.write
0.94 ± 5% +0.5 1.42 ± 5% perf-profile.self.cycles-pp.update_load_avg
0.36 ± 6% +0.5 0.86 ± 4% perf-profile.self.cycles-pp.main
0.17 ± 16% +0.5 0.67 ± 7% perf-profile.self.cycles-pp.cpuacct_charge
0.22 ± 4% +0.5 0.76 ± 3% perf-profile.self.cycles-pp.schedule
0.52 ± 3% +0.6 1.08 ± 7% perf-profile.self.cycles-pp.__fdget_pos
0.58 ± 5% +0.6 1.16 ± 6% perf-profile.self.cycles-pp.vfs_read
0.63 ± 4% +0.6 1.26 ± 3% perf-profile.self.cycles-pp.reweight_entity
0.64 ± 6% +0.7 1.31 perf-profile.self.cycles-pp.pipe_write
0.46 ± 4% +0.7 1.15 ± 4% perf-profile.self.cycles-pp.__entry_text_start
0.65 ± 5% +0.7 1.37 ± 2% perf-profile.self.cycles-pp.os_xsave
0.62 ± 3% +0.7 1.35 ± 3% perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
0.35 ± 7% +0.7 1.09 ± 4% perf-profile.self.cycles-pp.select_task_rq_fair
0.77 ± 6% +0.8 1.52 ± 3% perf-profile.self.cycles-pp.update_curr
0.00 +0.9 0.90 ± 6% perf-profile.self.cycles-pp.check_preempt_wakeup
0.94 ± 5% +1.0 1.94 perf-profile.self.cycles-pp.pipe_read
0.55 ± 5% +1.1 1.60 ± 4% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.45 ± 5% +1.1 1.56 ± 2% perf-profile.self.cycles-pp.__calc_delta
0.81 ± 5% +1.1 1.93 ± 2% perf-profile.self.cycles-pp.update_cfs_group
0.00 +1.6 1.59 ± 3% perf-profile.self.cycles-pp.restore_fpregs_from_fpstate
0.25 ± 7% +4.8 5.07 ± 2% perf-profile.self.cycles-pp.switch_mm_irqs_off
11.94 ± 6% +8.6 20.59 ± 2% perf-profile.self.cycles-pp.mwait_idle_with_hints
***************************************************************************************************
lkp-cpl-4sp1: 144 threads 4 sockets Intel(R) Xeon(R) Gold 5318H CPU @ 2.50GHz (Cooper Lake) with 128G memory
=========================================================================================
cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase:
cs-localhost/gcc-11/performance/ipv4/x86_64-rhel-8.3/25%/debian-11.1-x86_64-20220510.cgz/300s/lkp-cpl-4sp1/TCP_RR/netperf
commit:
c36eae5a86 ("sched/psi: Fix possible missing or delayed pending event")
b6aabb01e3 ("sched: consider WF_SYNC to find idle siblings")
c36eae5a86d15a69 b6aabb01e3004b0285d953761e1
---------------- ---------------------------
%stddev %change %stddev
\ | \
3611595 +53.3% 5537882 netperf.Throughput_total_tps
100322 +53.3% 153830 netperf.Throughput_tps
130329 +8.7e+05% 1.128e+09 ± 19% netperf.time.involuntary_context_switches
16743 ± 2% -22.6% 12962 netperf.time.minor_page_faults
1202 +16.4% 1399 ± 2% netperf.time.percent_of_cpu_this_job_got
3432 +12.6% 3865 ± 2% netperf.time.system_time
188.48 +85.8% 350.16 ± 4% netperf.time.user_time
1.083e+09 -50.8% 5.334e+08 ± 39% netperf.time.voluntary_context_switches
1.083e+09 +53.3% 1.661e+09 netperf.workload
2.212e+09 -97.0% 65981036 cpuidle..usage
84551 ± 5% -18.5% 68943 ± 3% meminfo.Active
84433 ± 5% -18.5% 68833 ± 3% meminfo.Active(anon)
79312 ± 6% -21.0% 62677 ± 5% numa-meminfo.node3.Active
79280 ± 6% -21.0% 62645 ± 5% numa-meminfo.node3.Active(anon)
19806 ± 6% -20.8% 15687 ± 5% numa-vmstat.node3.nr_active_anon
19806 ± 6% -20.8% 15687 ± 5% numa-vmstat.node3.nr_zone_active_anon
8.60 -2.4 6.16 mpstat.cpu.all.soft%
18.97 -1.9 17.04 mpstat.cpu.all.sys%
1.30 +0.5 1.85 ± 3% mpstat.cpu.all.usr%
37.00 +47.7% 54.67 vmstat.procs.r
14204693 -23.0% 10930950 vmstat.system.cs
295062 +6.5% 314162 vmstat.system.in
21070 ± 5% -18.5% 17172 ± 3% proc-vmstat.nr_active_anon
106997 -2.5% 104321 proc-vmstat.nr_anon_pages
409349 ± 4% +10.0% 450452 ± 4% proc-vmstat.nr_mapped
83608 -1.5% 82352 proc-vmstat.nr_slab_unreclaimable
21070 ± 5% -18.5% 17172 ± 3% proc-vmstat.nr_zone_active_anon
531984 ± 9% -15.9% 447458 ± 6% proc-vmstat.numa_hint_faults_local
24368 ± 7% -29.6% 17161 ± 4% proc-vmstat.pgactivate
7844373 -3.4% 7576615 proc-vmstat.pgalloc_normal
1450970 ± 2% -18.0% 1189301 proc-vmstat.pgfree
1358 -36.3% 864.67 turbostat.Avg_MHz
41.28 -15.0 26.27 turbostat.Busy%
1.281e+09 -100.0% 15898 ± 95% turbostat.C1
12.60 -12.6 0.00 turbostat.C1%
58.66 +25.5% 73.65 turbostat.CPU%c1
0.21 +77.0% 0.37 turbostat.IPC
14.47 ± 21% -14.5 0.00 turbostat.PKG_%
8.861e+08 -100.0% 4493 ± 14% turbostat.POLL
6.47 -6.5 0.00 turbostat.POLL%
571.65 -8.5% 522.90 turbostat.PkgWatt
8.27 -4.0% 7.94 turbostat.RAMWatt
27233 ± 32% -95.4% 1255 ±136% sched_debug.cfs_rq:/.MIN_vruntime.avg
750764 ± 9% -76.0% 180343 ±136% sched_debug.cfs_rq:/.MIN_vruntime.max
133981 ± 19% -88.8% 14993 ±136% sched_debug.cfs_rq:/.MIN_vruntime.stddev
0.26 +23.0% 0.32 ± 6% sched_debug.cfs_rq:/.h_nr_running.avg
1.03 ± 6% +88.1% 1.93 ± 4% sched_debug.cfs_rq:/.h_nr_running.max
0.42 +46.9% 0.62 ± 5% sched_debug.cfs_rq:/.h_nr_running.stddev
9751 ± 10% -32.8% 6552 ± 13% sched_debug.cfs_rq:/.load.avg
27233 ± 32% -95.4% 1255 ±136% sched_debug.cfs_rq:/.max_vruntime.avg
750764 ± 9% -76.0% 180343 ±136% sched_debug.cfs_rq:/.max_vruntime.max
133981 ± 19% -88.8% 14993 ±136% sched_debug.cfs_rq:/.max_vruntime.stddev
724533 ± 8% +35.8% 983881 ± 8% sched_debug.cfs_rq:/.min_vruntime.avg
845404 ± 10% +54.1% 1302855 ± 8% sched_debug.cfs_rq:/.min_vruntime.max
27934 ± 13% +297.8% 111130 ± 11% sched_debug.cfs_rq:/.min_vruntime.stddev
0.26 -18.8% 0.21 ± 4% sched_debug.cfs_rq:/.nr_running.avg
197.05 +90.2% 374.73 ± 4% sched_debug.cfs_rq:/.runnable_avg.avg
849.80 ± 4% +77.3% 1506 ± 2% sched_debug.cfs_rq:/.runnable_avg.max
209.48 +161.4% 547.51 sched_debug.cfs_rq:/.runnable_avg.stddev
114133 ± 29% +172.2% 310722 ± 39% sched_debug.cfs_rq:/.spread0.max
-65385 +469.6% -372412 sched_debug.cfs_rq:/.spread0.min
27949 ± 13% +297.7% 111154 ± 11% sched_debug.cfs_rq:/.spread0.stddev
196.98 +35.1% 266.15 ± 3% sched_debug.cfs_rq:/.util_avg.avg
849.66 ± 4% +35.1% 1147 ± 3% sched_debug.cfs_rq:/.util_avg.max
209.42 +86.8% 391.19 sched_debug.cfs_rq:/.util_avg.stddev
93.07 ± 4% +76.1% 163.93 ± 4% sched_debug.cfs_rq:/.util_est_enqueued.avg
664.98 ± 8% +48.2% 985.29 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.max
162.79 +91.6% 311.99 sched_debug.cfs_rq:/.util_est_enqueued.stddev
250834 ± 9% +84.0% 461429 ± 5% sched_debug.cpu.avg_idle.avg
3157 ± 6% +939.5% 32825 ± 45% sched_debug.cpu.avg_idle.min
282325 ± 5% +16.5% 328926 sched_debug.cpu.avg_idle.stddev
7.58 ± 3% +34.3% 10.18 ± 2% sched_debug.cpu.clock.stddev
10236 ± 4% -13.5% 8854 ± 5% sched_debug.cpu.curr->pid.max
0.00 ± 13% +55.3% 0.00 ± 4% sched_debug.cpu.next_balance.stddev
0.23 ± 3% +41.6% 0.32 ± 5% sched_debug.cpu.nr_running.avg
1.03 ± 6% +81.6% 1.87 ± 5% sched_debug.cpu.nr_running.max
0.40 ± 2% +53.6% 0.62 ± 4% sched_debug.cpu.nr_running.stddev
13964454 ± 10% -31.5% 9562330 ± 9% sched_debug.cpu.nr_switches.avg
15141338 ± 10% -19.0% 12257576 ± 7% sched_debug.cpu.nr_switches.max
12639092 ± 9% -54.5% 5746871 ± 11% sched_debug.cpu.nr_switches.min
444327 ± 13% +150.9% 1114942 ± 10% sched_debug.cpu.nr_switches.stddev
950.00 -100.0% 0.00 sched_debug.rt_rq:/.rt_runtime.avg
950.00 -100.0% 0.00 sched_debug.rt_rq:/.rt_runtime.max
950.00 -100.0% 0.00 sched_debug.rt_rq:/.rt_runtime.min
12.33 -96.1% 0.48 ± 7% perf-stat.i.MPKI
2.876e+10 +11.9% 3.217e+10 perf-stat.i.branch-instructions
1.21 +0.1 1.26 perf-stat.i.branch-miss-rate%
3.406e+08 +17.8% 4.012e+08 perf-stat.i.branch-misses
1.90 ± 15% +34.8 36.70 ± 16% perf-stat.i.cache-miss-rate%
1.739e+09 -95.6% 76617839 ± 7% perf-stat.i.cache-references
14360622 -23.1% 11042127 perf-stat.i.context-switches
1.43 -46.7% 0.76 perf-stat.i.cpi
2.023e+11 -38.7% 1.239e+11 perf-stat.i.cpu-cycles
1322 +1109.9% 15995 ± 2% perf-stat.i.cpu-migrations
7928 ± 19% -35.8% 5089 ± 19% perf-stat.i.cycles-between-cache-misses
0.02 ± 6% -0.0 0.01 ± 4% perf-stat.i.dTLB-load-miss-rate%
6843508 ± 6% -32.1% 4644002 ± 3% perf-stat.i.dTLB-load-misses
4.213e+10 +15.6% 4.87e+10 perf-stat.i.dTLB-loads
0.00 ± 5% -0.0 0.00 ± 4% perf-stat.i.dTLB-store-miss-rate%
2.439e+10 +14.5% 2.793e+10 perf-stat.i.dTLB-stores
62.06 +2.8 64.90 perf-stat.i.iTLB-load-miss-rate%
1.596e+08 ± 2% +44.2% 2.302e+08 perf-stat.i.iTLB-load-misses
97319700 +27.8% 1.244e+08 perf-stat.i.iTLB-loads
1.429e+11 +14.0% 1.629e+11 perf-stat.i.instructions
918.99 ± 2% -19.1% 743.46 perf-stat.i.instructions-per-iTLB-miss
0.71 +85.9% 1.31 perf-stat.i.ipc
1.40 -38.7% 0.86 perf-stat.i.metric.GHz
873.16 +76.2% 1538 ± 3% perf-stat.i.metric.K/sec
673.72 +12.2% 755.60 perf-stat.i.metric.M/sec
82.06 ± 3% +9.3 91.32 perf-stat.i.node-store-miss-rate%
1049204 ± 16% +53.8% 1613542 ± 18% perf-stat.i.node-store-misses
443904 ± 6% -30.0% 310869 ± 6% perf-stat.i.node-stores
12.17 -96.1% 0.47 ± 7% perf-stat.overall.MPKI
1.18 +0.1 1.25 perf-stat.overall.branch-miss-rate%
1.75 ± 17% +36.3 38.07 ± 15% perf-stat.overall.cache-miss-rate%
1.42 -46.3% 0.76 perf-stat.overall.cpi
6820 ± 15% -35.7% 4383 ± 17% perf-stat.overall.cycles-between-cache-misses
0.02 ± 6% -0.0 0.01 ± 4% perf-stat.overall.dTLB-load-miss-rate%
0.00 ± 5% -0.0 0.00 ± 3% perf-stat.overall.dTLB-store-miss-rate%
62.11 +2.8 64.92 perf-stat.overall.iTLB-load-miss-rate%
895.64 ± 2% -21.0% 707.76 perf-stat.overall.instructions-per-iTLB-miss
0.71 +86.1% 1.31 perf-stat.overall.ipc
69.85 ± 6% +13.7 83.54 ± 2% perf-stat.overall.node-store-miss-rate%
39789 -25.6% 29606 perf-stat.overall.path-length
2.866e+10 +11.9% 3.206e+10 perf-stat.ps.branch-instructions
3.395e+08 +17.8% 3.999e+08 perf-stat.ps.branch-misses
1.733e+09 -95.6% 76361868 ± 7% perf-stat.ps.cache-references
14312085 -23.1% 11004648 perf-stat.ps.context-switches
2.016e+11 -38.7% 1.235e+11 perf-stat.ps.cpu-cycles
1317 +1109.8% 15942 ± 2% perf-stat.ps.cpu-migrations
6808921 ± 6% -32.1% 4621256 ± 3% perf-stat.ps.dTLB-load-misses
4.199e+10 +15.6% 4.854e+10 perf-stat.ps.dTLB-loads
2.431e+10 +14.5% 2.784e+10 perf-stat.ps.dTLB-stores
1.591e+08 ± 2% +44.2% 2.294e+08 perf-stat.ps.iTLB-load-misses
96991959 +27.8% 1.239e+08 perf-stat.ps.iTLB-loads
1.424e+11 +14.0% 1.624e+11 perf-stat.ps.instructions
0.01 ± 50% -72.0% 0.00 ±115% perf-stat.ps.major-faults
1046642 ± 16% +53.7% 1608395 ± 18% perf-stat.ps.node-store-misses
442450 ± 6% -29.9% 309961 ± 7% perf-stat.ps.node-stores
4.311e+13 +14.1% 4.919e+13 perf-stat.total.instructions
55.82 ± 5% -21.0 34.84 ± 14% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
55.50 ± 5% -20.9 34.61 ± 14% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
55.49 ± 5% -20.9 34.61 ± 14% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
55.40 ± 5% -20.8 34.61 ± 14% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
48.23 ± 7% -13.7 34.56 ± 14% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
46.09 ± 7% -11.7 34.40 ± 14% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
46.05 ± 7% -11.7 34.36 ± 14% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
10.29 ± 7% -10.3 0.00 perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
10.00 ± 6% -10.0 0.00 perf-profile.calltrace.cycles-pp.intel_idle_irq.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
9.36 ± 6% -9.4 0.00 perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle_irq.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
1.58 ± 7% -1.2 0.40 ± 72% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.schedule_timeout
6.53 ± 6% -1.1 5.42 ± 13% perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv.send_omni_inner
0.00 +0.8 0.82 ± 8% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.exit_to_user_mode_loop.exit_to_user_mode_prepare
0.00 +0.9 0.88 ± 42% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.recv_omni
0.00 +0.9 0.93 ± 9% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.schedule_timeout.wait_woken
0.00 +1.0 0.98 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send
0.00 +1.0 0.98 ± 7% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send
0.00 +1.0 0.98 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__send
0.00 +1.0 1.03 ± 22% perf-profile.calltrace.cycles-pp.skb_release_data.__kfree_skb.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established
0.00 +1.1 1.13 ± 9% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.recv
0.00 +1.1 1.13 ± 9% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv
0.00 +1.1 1.13 ± 9% perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv
0.00 +1.2 1.15 ± 8% perf-profile.calltrace.cycles-pp.__kfree_skb.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv
1.54 ± 6% +1.2 2.76 ± 8% perf-profile.calltrace.cycles-pp.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv
0.00 +1.3 1.32 ± 7% perf-profile.calltrace.cycles-pp.skb_do_copy_data_nocache.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
5.02 ± 7% +1.3 6.35 ± 8% perf-profile.calltrace.cycles-pp.wait_woken.sk_wait_data.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg
1.33 ± 7% +1.4 2.74 ± 7% perf-profile.calltrace.cycles-pp.__dev_queue_xmit.ip_finish_output2.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit
10.51 ± 5% +1.4 11.96 ± 8% perf-profile.calltrace.cycles-pp.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog
8.66 ± 7% +1.5 10.14 ± 8% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish
14.38 ± 6% +1.5 15.87 ± 7% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit
8.75 ± 7% +1.5 10.27 ± 8% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core
1.06 ± 22% +1.5 2.60 ± 8% perf-profile.calltrace.cycles-pp.tcp_stream_alloc_skb.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
14.55 ± 6% +1.5 16.09 ± 7% perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit.__tcp_transmit_skb
14.67 ± 6% +1.6 16.26 ± 7% perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit
13.28 ± 6% +1.7 14.93 ± 8% perf-profile.calltrace.cycles-pp.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv
0.00 +1.7 1.67 ± 8% perf-profile.calltrace.cycles-pp.loopback_xmit.dev_hard_start_xmit.__dev_queue_xmit.ip_finish_output2.__ip_queue_xmit
10.67 ± 5% +1.7 12.38 ± 7% perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog.__napi_poll
0.00 +1.7 1.75 ± 14% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.__wake_up_common.__wake_up_common_lock
2.39 ± 6% +1.8 4.16 ± 8% perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu
10.70 ± 5% +1.8 12.47 ± 7% perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action
0.00 +1.8 1.78 ± 8% perf-profile.calltrace.cycles-pp.dev_hard_start_xmit.__dev_queue_xmit.ip_finish_output2.__ip_queue_xmit.__tcp_transmit_skb
0.00 +1.8 1.79 ± 13% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable
0.00 +1.8 1.84 ± 8% perf-profile.calltrace.cycles-pp.__alloc_skb.tcp_stream_alloc_skb.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
3.91 ± 7% +1.9 5.78 ± 8% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg_locked
3.68 ± 7% +1.9 5.58 ± 8% perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.wait_woken.sk_wait_data
4.03 ± 7% +1.9 5.97 ± 8% perf-profile.calltrace.cycles-pp.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg_locked.tcp_recvmsg
0.00 +2.0 1.99 ± 8% perf-profile.calltrace.cycles-pp.__send
11.19 ± 6% +2.2 13.36 ± 7% perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start
0.00 +2.3 2.26 ± 21% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.send_omni_inner
0.00 +2.3 2.29 ± 9% perf-profile.calltrace.cycles-pp.recv
11.56 ± 6% +2.5 14.09 ± 7% perf-profile.calltrace.cycles-pp.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq
0.00 +2.6 2.59 ± 8% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode
11.58 ± 6% +2.6 14.18 ± 7% perf-profile.calltrace.cycles-pp.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip
0.00 +2.7 2.72 ± 8% perf-profile.calltrace.cycles-pp.schedule.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
0.00 +2.9 2.90 ± 8% perf-profile.calltrace.cycles-pp.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
16.38 ± 6% +3.4 19.76 ± 7% perf-profile.calltrace.cycles-pp.ip_finish_output2.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames
0.00 +3.9 3.86 ± 3% perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send
17.18 ± 6% +4.3 21.44 ± 6% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked
12.91 ± 6% +4.6 17.50 ± 7% perf-profile.calltrace.cycles-pp.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.recv_omni
18.67 ± 6% +5.3 24.00 ± 6% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg
13.02 ± 6% +5.5 18.56 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.recv_omni.process_requests
13.10 ± 6% +5.7 18.77 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__send.recv_omni.process_requests.spawn_child
13.24 ± 6% +5.8 19.06 ± 7% perf-profile.calltrace.cycles-pp.__send.recv_omni.process_requests.spawn_child.accept_connection
20.18 ± 6% +6.3 26.50 ± 6% perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
20.27 ± 6% +6.5 26.78 ± 6% perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
12.70 ± 6% +6.5 19.22 ± 9% perf-profile.calltrace.cycles-pp.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.send_omni_inner
21.45 ± 6% +8.3 29.75 ± 7% perf-profile.calltrace.cycles-pp.recv_omni.process_requests.spawn_child.accept_connection.accept_connections
12.80 ± 6% +8.8 21.60 ± 10% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.send_omni_inner.send_tcp_rr
19.60 ± 7% +8.9 28.49 ± 7% perf-profile.calltrace.cycles-pp.send_omni_inner.send_tcp_rr.main.__libc_start_main
24.83 ± 19% +8.9 33.77 ± 15% perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
24.83 ± 19% +8.9 33.77 ± 15% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
19.76 ± 7% +9.0 28.80 ± 7% perf-profile.calltrace.cycles-pp.send_tcp_rr.main.__libc_start_main
11.96 ± 7% +9.4 21.31 ± 9% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__send.send_omni_inner.send_tcp_rr.main
24.07 ± 6% +9.4 33.44 ± 7% perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto
12.09 ± 7% +9.4 21.47 ± 9% perf-profile.calltrace.cycles-pp.__send.send_omni_inner.send_tcp_rr.main.__libc_start_main
24.63 ± 6% +9.8 34.41 ± 7% perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64
25.07 ± 6% +10.3 35.41 ± 7% perf-profile.calltrace.cycles-pp.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
25.49 ± 6% +11.0 36.46 ± 7% perf-profile.calltrace.cycles-pp.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send
55.82 ± 5% -21.0 34.84 ± 14% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
55.82 ± 5% -21.0 34.84 ± 14% perf-profile.children.cycles-pp.cpu_startup_entry
55.50 ± 5% -20.9 34.61 ± 14% perf-profile.children.cycles-pp.start_secondary
55.74 ± 5% -20.9 34.84 ± 14% perf-profile.children.cycles-pp.do_idle
48.52 ± 7% -13.7 34.79 ± 14% perf-profile.children.cycles-pp.cpuidle_idle_call
46.36 ± 7% -11.7 34.63 ± 14% perf-profile.children.cycles-pp.cpuidle_enter
46.34 ± 7% -11.7 34.63 ± 14% perf-profile.children.cycles-pp.cpuidle_enter_state
10.35 ± 7% -10.4 0.00 perf-profile.children.cycles-pp.poll_idle
10.06 ± 6% -10.1 0.00 perf-profile.children.cycles-pp.intel_idle_irq
3.04 ± 7% -2.3 0.72 ± 10% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
1.70 ± 5% -1.6 0.15 ± 16% perf-profile.children.cycles-pp.menu_select
1.33 ± 6% -1.3 0.03 ± 70% perf-profile.children.cycles-pp.ttwu_queue_wakelist
1.68 ± 7% -0.9 0.75 ± 8% perf-profile.children.cycles-pp.dequeue_entity
0.92 ± 7% -0.6 0.29 ± 18% perf-profile.children.cycles-pp.select_task_rq
0.73 ± 4% -0.6 0.10 ± 18% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.90 ± 8% -0.6 0.28 ± 7% perf-profile.children.cycles-pp.skb_attempt_defer_free
0.79 ± 7% -0.6 0.24 ± 19% perf-profile.children.cycles-pp.select_task_rq_fair
0.66 ± 7% -0.5 0.19 ± 10% perf-profile.children.cycles-pp.sock_rfree
0.48 ± 3% -0.4 0.09 ± 20% perf-profile.children.cycles-pp.tick_nohz_next_event
1.11 ± 10% -0.4 0.74 ± 18% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.64 ± 5% -0.4 0.28 ± 11% perf-profile.children.cycles-pp.remove_wait_queue
0.96 ± 7% -0.3 0.62 ± 11% perf-profile.children.cycles-pp.__inet_lookup_established
0.81 ± 5% -0.3 0.47 ± 13% perf-profile.children.cycles-pp.prepare_task_switch
0.82 ± 7% -0.3 0.52 ± 7% perf-profile.children.cycles-pp._raw_spin_lock
0.57 ± 9% -0.2 0.33 ± 11% perf-profile.children.cycles-pp.tcp_check_space
1.10 ± 7% -0.2 0.88 ± 8% perf-profile.children.cycles-pp.enqueue_entity
0.56 ± 8% -0.2 0.35 ± 9% perf-profile.children.cycles-pp.copyout
0.55 ± 9% -0.2 0.34 ± 7% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
0.35 ± 8% -0.2 0.16 ± 8% perf-profile.children.cycles-pp.kfree_skbmem
0.58 ± 7% -0.2 0.41 ± 7% perf-profile.children.cycles-pp.copy_user_short_string
0.31 ± 8% -0.2 0.15 ± 11% perf-profile.children.cycles-pp.perf_tp_event
0.20 ± 8% -0.1 0.05 ± 7% perf-profile.children.cycles-pp.skb_release_head_state
0.58 ± 7% -0.1 0.47 ± 8% perf-profile.children.cycles-pp.tcp_rcv_space_adjust
0.09 ± 15% -0.0 0.04 ± 45% perf-profile.children.cycles-pp.rebalance_domains
0.07 ± 11% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.tracing_gen_ctx_irq_test
0.07 ± 7% +0.0 0.09 ± 7% perf-profile.children.cycles-pp.tcp_rate_skb_delivered
0.15 ± 7% +0.0 0.18 ± 9% perf-profile.children.cycles-pp.add_wait_queue
0.07 ± 7% +0.0 0.10 ± 10% perf-profile.children.cycles-pp.tcp_event_data_recv
0.04 ± 44% +0.0 0.08 ± 10% perf-profile.children.cycles-pp.place_entity
0.11 ± 11% +0.0 0.15 ± 15% perf-profile.children.cycles-pp.tcp_rearm_rto
0.06 ± 14% +0.0 0.11 ± 12% perf-profile.children.cycles-pp.ip_skb_dst_mtu
0.02 ±141% +0.0 0.06 ± 21% perf-profile.children.cycles-pp.timekeeping_max_deferment
0.05 ± 47% +0.0 0.09 ± 7% perf-profile.children.cycles-pp.nf_hook_slow
0.02 ±141% +0.0 0.06 ± 11% perf-profile.children.cycles-pp.__cgroup_account_cputime
0.22 ± 10% +0.1 0.27 ± 9% perf-profile.children.cycles-pp.__mod_timer
0.09 ± 14% +0.1 0.14 ± 19% perf-profile.children.cycles-pp.scheduler_tick
0.00 +0.1 0.05 ± 7% perf-profile.children.cycles-pp.minmax_running_min
0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.ip_local_deliver
0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.tcp_options_write
0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.inet_sendmsg
0.01 ±223% +0.1 0.06 ± 11% perf-profile.children.cycles-pp.skb_clone
0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.tcp_small_queue_check
0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.cubictcp_cong_avoid
0.00 +0.1 0.06 ± 18% perf-profile.children.cycles-pp.security_sock_rcv_skb
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.tcp_chrono_stop
0.00 +0.1 0.06 ± 15% perf-profile.children.cycles-pp.eth_type_trans
0.00 +0.1 0.06 ± 14% perf-profile.children.cycles-pp.kmalloc_slab
0.00 +0.1 0.06 ± 17% perf-profile.children.cycles-pp.__raise_softirq_irqoff
0.06 ± 6% +0.1 0.12 ± 7% perf-profile.children.cycles-pp.sk_filter_trim_cap
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.switch_ldt
0.00 +0.1 0.06 ± 14% perf-profile.children.cycles-pp.rb_insert_color
0.27 ± 9% +0.1 0.33 ± 7% perf-profile.children.cycles-pp.sk_reset_timer
0.10 ± 15% +0.1 0.17 ± 11% perf-profile.children.cycles-pp.perf_trace_sched_switch
0.10 ± 8% +0.1 0.16 ± 9% perf-profile.children.cycles-pp.__ip_finish_output
0.10 ± 14% +0.1 0.17 ± 11% perf-profile.children.cycles-pp.__tcp_cleanup_rbuf
0.08 ± 8% +0.1 0.15 ± 11% perf-profile.children.cycles-pp.update_min_vruntime
0.06 ± 9% +0.1 0.13 ± 9% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare
0.09 ± 6% +0.1 0.16 ± 9% perf-profile.children.cycles-pp.try_charge_memcg
0.00 +0.1 0.07 ± 10% perf-profile.children.cycles-pp.check_cfs_rq_runtime
0.00 +0.1 0.07 ± 10% perf-profile.children.cycles-pp.clear_buddies
0.04 ± 44% +0.1 0.11 ± 6% perf-profile.children.cycles-pp.tcp_established_options
0.00 +0.1 0.07 ± 14% perf-profile.children.cycles-pp.apparmor_socket_recvmsg
0.61 ± 7% +0.1 0.68 ± 6% perf-profile.children.cycles-pp.__switch_to
0.08 ± 6% +0.1 0.15 ± 9% perf-profile.children.cycles-pp.__copy_skb_header
0.00 +0.1 0.07 ± 20% perf-profile.children.cycles-pp.cubictcp_acked
0.06 ± 9% +0.1 0.13 ± 5% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
0.00 +0.1 0.07 ± 12% perf-profile.children.cycles-pp.memcg_slab_post_alloc_hook
0.06 ± 11% +0.1 0.14 ± 9% perf-profile.children.cycles-pp.tcp_rate_skb_sent
0.08 ± 13% +0.1 0.16 ± 8% perf-profile.children.cycles-pp.rcu_all_qs
0.12 ± 12% +0.1 0.20 ± 8% perf-profile.children.cycles-pp.ip_send_check
0.06 ± 9% +0.1 0.13 ± 7% perf-profile.children.cycles-pp.inet_ehashfn
0.03 ± 70% +0.1 0.11 ± 8% perf-profile.children.cycles-pp.sk_free
0.06 ± 11% +0.1 0.14 ± 8% perf-profile.children.cycles-pp.raw_v4_input
0.00 +0.1 0.08 ± 12% perf-profile.children.cycles-pp.tcp_inbound_md5_hash
0.00 +0.1 0.08 ± 10% perf-profile.children.cycles-pp.neigh_hh_output
0.19 ± 17% +0.1 0.28 ± 18% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.04 ± 45% +0.1 0.13 ± 8% perf-profile.children.cycles-pp.tcp_rate_gen
0.00 +0.1 0.08 ± 5% perf-profile.children.cycles-pp.syscall_enter_from_user_mode
0.00 +0.1 0.09 ± 10% perf-profile.children.cycles-pp.apparmor_socket_sendmsg
0.58 ± 8% +0.1 0.67 ± 8% perf-profile.children.cycles-pp.tcp_event_new_data_sent
0.00 +0.1 0.09 ± 6% perf-profile.children.cycles-pp.check_stack_object
0.00 +0.1 0.09 ± 6% perf-profile.children.cycles-pp.tcp_v4_fill_cb
0.00 +0.1 0.09 ± 11% perf-profile.children.cycles-pp.tcp_newly_delivered
0.07 ± 13% +0.1 0.16 ± 5% perf-profile.children.cycles-pp.sock_put
0.10 ± 8% +0.1 0.19 ± 7% perf-profile.children.cycles-pp.__ksize
0.05 ± 45% +0.1 0.14 ± 13% perf-profile.children.cycles-pp.netif_skb_features
0.10 ± 9% +0.1 0.20 ± 14% perf-profile.children.cycles-pp.sk_forced_mem_schedule
0.04 ± 71% +0.1 0.13 ± 5% perf-profile.children.cycles-pp.__usecs_to_jiffies
0.09 ± 14% +0.1 0.19 ± 14% perf-profile.children.cycles-pp.tcp_rtt_estimator
0.00 +0.1 0.10 ± 4% perf-profile.children.cycles-pp.tcp_push
0.10 ± 10% +0.1 0.21 ± 7% perf-profile.children.cycles-pp.__tcp_select_window
0.63 ± 6% +0.1 0.74 ± 7% perf-profile.children.cycles-pp.simple_copy_to_iter
0.00 +0.1 0.11 ± 11% perf-profile.children.cycles-pp.resched_curr
0.07 ± 12% +0.1 0.19 ± 22% perf-profile.children.cycles-pp.cgroup_rstat_updated
0.43 ± 6% +0.1 0.54 ± 9% perf-profile.children.cycles-pp.update_rq_clock
0.12 ± 9% +0.1 0.24 ± 7% perf-profile.children.cycles-pp.ip_output
0.18 ± 9% +0.1 0.29 ± 14% perf-profile.children.cycles-pp.__calc_delta
0.50 ± 6% +0.1 0.63 ± 9% perf-profile.children.cycles-pp.set_next_entity
0.09 ± 6% +0.1 0.21 ± 7% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.28 ± 7% +0.1 0.41 ± 8% perf-profile.children.cycles-pp.recv_data
0.00 +0.1 0.13 ± 8% perf-profile.children.cycles-pp.cubictcp_cwnd_event
0.04 ± 71% +0.1 0.17 ± 13% perf-profile.children.cycles-pp.skb_page_frag_refill
0.11 ± 8% +0.1 0.24 ± 10% perf-profile.children.cycles-pp.tcp_tso_segs
0.07 ± 14% +0.1 0.20 ± 10% perf-profile.children.cycles-pp.sk_page_frag_refill
0.11 ± 6% +0.1 0.25 ± 6% perf-profile.children.cycles-pp.__netif_receive_skb_core
0.30 ± 6% +0.1 0.44 ± 8% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.08 ± 18% +0.1 0.23 ± 19% perf-profile.children.cycles-pp.cpuacct_charge
0.29 ± 9% +0.1 0.44 ± 8% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.08 ± 8% +0.1 0.23 ± 9% perf-profile.children.cycles-pp.ip_rcv_core
0.16 ± 9% +0.2 0.32 ± 8% perf-profile.children.cycles-pp.__ip_local_out
0.29 ± 6% +0.2 0.46 ± 6% perf-profile.children.cycles-pp.__update_load_avg_se
0.12 ± 6% +0.2 0.28 ± 10% perf-profile.children.cycles-pp.__list_del_entry_valid
0.35 ± 4% +0.2 0.52 ± 8% perf-profile.children.cycles-pp.update_cfs_group
0.14 ± 5% +0.2 0.31 ± 7% perf-profile.children.cycles-pp.copyin
0.11 ± 6% +0.2 0.28 ± 8% perf-profile.children.cycles-pp.refill_stock
0.18 ± 7% +0.2 0.35 ± 8% perf-profile.children.cycles-pp.__cond_resched
0.14 ± 7% +0.2 0.32 ± 8% perf-profile.children.cycles-pp.tcp_update_pacing_rate
0.11 ± 9% +0.2 0.28 ± 8% perf-profile.children.cycles-pp.tcp_update_skb_after_send
0.15 ± 10% +0.2 0.32 ± 15% perf-profile.children.cycles-pp.mem_cgroup_uncharge_skmem
0.17 ± 6% +0.2 0.34 ± 7% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.12 ± 9% +0.2 0.30 ± 14% perf-profile.children.cycles-pp.validate_xmit_skb
0.15 ± 20% +0.2 0.33 ± 17% perf-profile.children.cycles-pp.find_busiest_group
0.07 ± 9% +0.2 0.26 ± 8% perf-profile.children.cycles-pp.__list_add_valid
0.18 ± 8% +0.2 0.36 ± 10% perf-profile.children.cycles-pp.ip_local_out
0.17 ± 6% +0.2 0.36 ± 17% perf-profile.children.cycles-pp.__mod_memcg_state
0.16 ± 4% +0.2 0.35 ± 9% perf-profile.children.cycles-pp.tcp_wfree
0.46 ± 6% +0.2 0.66 ± 7% perf-profile.children.cycles-pp.read_tsc
0.11 ± 9% +0.2 0.31 ± 8% perf-profile.children.cycles-pp.__might_fault
0.00 +0.2 0.20 ± 10% perf-profile.children.cycles-pp.set_next_buddy
0.12 ± 21% +0.2 0.33 ± 17% perf-profile.children.cycles-pp.update_sd_lb_stats
0.19 ± 17% +0.2 0.40 ± 17% perf-profile.children.cycles-pp.load_balance
0.06 ± 7% +0.2 0.27 ± 8% perf-profile.children.cycles-pp.__kmem_cache_free
0.10 ± 23% +0.2 0.32 ± 17% perf-profile.children.cycles-pp.update_sg_lb_stats
0.18 ± 8% +0.2 0.39 ± 6% perf-profile.children.cycles-pp.__might_sleep
0.23 ± 9% +0.2 0.45 ± 7% perf-profile.children.cycles-pp.send_data
0.35 ± 8% +0.2 0.58 ± 8% perf-profile.children.cycles-pp.__skb_clone
0.67 ± 6% +0.2 0.90 ± 9% perf-profile.children.cycles-pp.__switch_to_asm
0.07 ± 10% +0.2 0.31 ± 8% perf-profile.children.cycles-pp.kmem_cache_free
0.07 ± 12% +0.2 0.32 ± 6% perf-profile.children.cycles-pp.put_prev_entity
0.12 ± 10% +0.2 0.37 ± 8% perf-profile.children.cycles-pp.tcp_ack_update_rtt
0.24 ± 7% +0.3 0.50 ± 9% perf-profile.children.cycles-pp.mem_cgroup_charge_skmem
0.27 ± 12% +0.3 0.54 ± 12% perf-profile.children.cycles-pp.ip_rcv
0.48 ± 9% +0.3 0.75 ± 7% perf-profile.children.cycles-pp.release_sock
0.22 ± 8% +0.3 0.52 ± 6% perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
0.25 ± 7% +0.3 0.56 ± 8% perf-profile.children.cycles-pp.os_xsave
1.01 ± 6% +0.3 1.33 ± 8% perf-profile.children.cycles-pp.skb_release_data
0.56 ± 8% +0.3 0.87 ± 7% perf-profile.children.cycles-pp._raw_spin_lock_bh
0.24 ± 8% +0.3 0.56 ± 9% perf-profile.children.cycles-pp.__kmem_cache_alloc_node
0.14 ± 3% +0.3 0.47 ± 7% perf-profile.children.cycles-pp.__virt_addr_valid
0.20 ± 8% +0.3 0.54 ± 8% perf-profile.children.cycles-pp.tcp_schedule_loss_probe
0.83 ± 5% +0.3 1.17 ± 7% perf-profile.children.cycles-pp.__check_object_size
0.06 ± 9% +0.4 0.41 ± 9% perf-profile.children.cycles-pp.import_single_range
0.35 ± 7% +0.4 0.71 ± 7% perf-profile.children.cycles-pp.lock_sock_nested
0.13 ± 10% +0.4 0.50 ± 11% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.10 ± 13% +0.4 0.47 ± 11% perf-profile.children.cycles-pp.check_preempt_curr
0.27 ± 6% +0.4 0.66 ± 9% perf-profile.children.cycles-pp.irqtime_account_irq
0.20 ± 9% +0.4 0.59 ± 7% perf-profile.children.cycles-pp.kmem_cache_alloc_node
0.84 ± 2% +0.4 1.23 ± 11% perf-profile.children.cycles-pp.ktime_get
0.30 ± 5% +0.4 0.70 ± 8% perf-profile.children.cycles-pp.security_socket_sendmsg
0.28 ± 7% +0.4 0.68 ± 9% perf-profile.children.cycles-pp.__kmalloc_node_track_caller
0.19 ± 5% +0.4 0.60 ± 9% perf-profile.children.cycles-pp.enqueue_to_backlog
0.94 ± 6% +0.4 1.34 ± 7% perf-profile.children.cycles-pp.update_load_avg
0.00 +0.4 0.41 ± 12% perf-profile.children.cycles-pp.check_preempt_wakeup
0.21 ± 6% +0.4 0.63 ± 8% perf-profile.children.cycles-pp.netif_rx_internal
0.21 ± 5% +0.4 0.64 ± 9% perf-profile.children.cycles-pp.__netif_rx
0.30 ± 8% +0.4 0.74 ± 9% perf-profile.children.cycles-pp.kmalloc_reserve
0.34 ± 5% +0.4 0.78 ± 7% perf-profile.children.cycles-pp.tcp_mstamp_refresh
0.26 ± 8% +0.4 0.71 ± 10% perf-profile.children.cycles-pp.reweight_entity
0.27 ± 11% +0.5 0.75 ± 9% perf-profile.children.cycles-pp.__fdget
0.41 ± 5% +0.5 0.89 ± 8% perf-profile.children.cycles-pp.__sk_mem_reduce_allocated
0.00 +0.5 0.50 ± 10% perf-profile.children.cycles-pp.kfree
0.22 ± 7% +0.5 0.76 ± 8% perf-profile.children.cycles-pp.security_socket_recvmsg
0.29 ± 6% +0.5 0.84 ± 7% perf-profile.children.cycles-pp._copy_from_iter
0.77 ± 8% +0.5 1.31 ± 9% perf-profile.children.cycles-pp.update_curr
0.35 ± 6% +0.6 0.91 ± 7% perf-profile.children.cycles-pp.aa_sk_perm
0.24 ± 8% +0.6 0.80 ± 8% perf-profile.children.cycles-pp.sock_recvmsg
0.43 ± 7% +0.6 0.99 ± 7% perf-profile.children.cycles-pp.__entry_text_start
0.33 ± 10% +0.6 0.91 ± 9% perf-profile.children.cycles-pp.sockfd_lookup_light
1.23 ± 8% +0.6 1.82 ± 7% perf-profile.children.cycles-pp.pick_next_task_fair
0.54 ± 7% +0.6 1.18 ± 8% perf-profile.children.cycles-pp.__kfree_skb
1.28 ± 7% +0.8 2.04 ± 8% perf-profile.children.cycles-pp.enqueue_task_fair
1.32 ± 7% +0.8 2.09 ± 8% perf-profile.children.cycles-pp.ttwu_do_activate
0.54 ± 6% +0.8 1.36 ± 7% perf-profile.children.cycles-pp.skb_do_copy_data_nocache
0.00 +1.0 1.01 ± 8% perf-profile.children.cycles-pp.restore_fpregs_from_fpstate
0.63 ± 6% +1.1 1.70 ± 8% perf-profile.children.cycles-pp.loopback_xmit
0.82 ± 7% +1.1 1.91 ± 8% perf-profile.children.cycles-pp.__alloc_skb
5.25 ± 7% +1.1 6.39 ± 8% perf-profile.children.cycles-pp.wait_woken
0.68 ± 5% +1.1 1.82 ± 8% perf-profile.children.cycles-pp.dev_hard_start_xmit
0.15 ± 8% +1.2 1.33 ± 8% perf-profile.children.cycles-pp.switch_fpu_return
1.65 ± 6% +1.2 2.87 ± 8% perf-profile.children.cycles-pp.tcp_clean_rtx_queue
9.07 ± 7% +1.4 10.45 ± 8% perf-profile.children.cycles-pp.tcp_rcv_established
10.89 ± 6% +1.4 12.30 ± 8% perf-profile.children.cycles-pp.tcp_v4_rcv
1.40 ± 6% +1.4 2.82 ± 8% perf-profile.children.cycles-pp.__dev_queue_xmit
9.14 ± 6% +1.4 10.56 ± 8% perf-profile.children.cycles-pp.tcp_v4_do_rcv
1.20 ± 7% +1.5 2.68 ± 8% perf-profile.children.cycles-pp.tcp_stream_alloc_skb
13.56 ± 7% +1.6 15.16 ± 8% perf-profile.children.cycles-pp.__sys_recvfrom
11.05 ± 6% +1.7 12.71 ± 8% perf-profile.children.cycles-pp.ip_protocol_deliver_rcu
13.66 ± 7% +1.7 15.36 ± 8% perf-profile.children.cycles-pp.__x64_sys_recvfrom
11.08 ± 6% +1.7 12.80 ± 8% perf-profile.children.cycles-pp.ip_local_deliver_finish
2.51 ± 6% +1.8 4.28 ± 8% perf-profile.children.cycles-pp.tcp_ack
4.22 ± 7% +1.8 6.02 ± 7% perf-profile.children.cycles-pp.schedule_timeout
0.10 ± 9% +2.1 2.19 ± 8% perf-profile.children.cycles-pp.switch_mm_irqs_off
11.50 ± 6% +2.2 13.70 ± 7% perf-profile.children.cycles-pp.__netif_receive_skb_one_core
11.89 ± 6% +2.5 14.38 ± 7% perf-profile.children.cycles-pp.process_backlog
11.90 ± 6% +2.6 14.46 ± 7% perf-profile.children.cycles-pp.__napi_poll
6.04 ± 7% +2.6 8.61 ± 7% perf-profile.children.cycles-pp.__schedule
0.00 +3.2 3.24 ± 7% perf-profile.children.cycles-pp.exit_to_user_mode_loop
16.72 ± 6% +3.3 20.05 ± 7% perf-profile.children.cycles-pp.ip_finish_output2
15.81 ± 7% +3.7 19.46 ± 8% perf-profile.children.cycles-pp.recv
17.54 ± 6% +4.2 21.74 ± 7% perf-profile.children.cycles-pp.__ip_queue_xmit
0.20 ± 8% +4.7 4.87 ± 7% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
4.10 ± 7% +4.8 8.90 ± 7% perf-profile.children.cycles-pp.schedule
0.32 ± 6% +4.8 5.12 ± 7% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
18.90 ± 6% +5.5 24.36 ± 7% perf-profile.children.cycles-pp.__tcp_transmit_skb
20.44 ± 6% +6.5 26.93 ± 7% perf-profile.children.cycles-pp.tcp_write_xmit
20.51 ± 6% +6.6 27.14 ± 7% perf-profile.children.cycles-pp.__tcp_push_pending_frames
21.50 ± 6% +8.3 29.79 ± 6% perf-profile.children.cycles-pp.send_tcp_rr
21.57 ± 6% +8.5 30.03 ± 7% perf-profile.children.cycles-pp.send_omni_inner
21.58 ± 6% +8.5 30.06 ± 7% perf-profile.children.cycles-pp.recv_omni
21.59 ± 6% +8.5 30.08 ± 7% perf-profile.children.cycles-pp.accept_connections
21.59 ± 6% +8.5 30.08 ± 7% perf-profile.children.cycles-pp.accept_connection
21.59 ± 6% +8.5 30.08 ± 7% perf-profile.children.cycles-pp.spawn_child
21.59 ± 6% +8.5 30.08 ± 7% perf-profile.children.cycles-pp.process_requests
24.97 ± 19% +9.0 34.00 ± 15% perf-profile.children.cycles-pp.intel_idle
24.26 ± 6% +9.3 33.55 ± 7% perf-profile.children.cycles-pp.tcp_sendmsg_locked
24.84 ± 6% +9.7 34.54 ± 7% perf-profile.children.cycles-pp.tcp_sendmsg
25.26 ± 6% +10.2 35.49 ± 7% perf-profile.children.cycles-pp.sock_sendmsg
25.68 ± 6% +10.9 36.56 ± 7% perf-profile.children.cycles-pp.__sys_sendto
25.80 ± 6% +11.0 36.82 ± 7% perf-profile.children.cycles-pp.__x64_sys_sendto
26.86 ± 6% +17.5 44.40 ± 7% perf-profile.children.cycles-pp.__send
40.08 ± 6% +17.8 57.92 ± 7% perf-profile.children.cycles-pp.do_syscall_64
40.47 ± 6% +18.4 58.88 ± 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
10.22 ± 7% -10.2 0.00 perf-profile.self.cycles-pp.poll_idle
3.03 ± 7% -2.3 0.70 ± 10% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
1.03 ± 7% -0.7 0.37 ± 9% perf-profile.self.cycles-pp.wait_woken
0.66 ± 7% -0.5 0.19 ± 10% perf-profile.self.cycles-pp.sock_rfree
0.91 ± 7% -0.4 0.49 ± 12% perf-profile.self.cycles-pp.__inet_lookup_established
0.64 ± 8% -0.4 0.26 ± 8% perf-profile.self.cycles-pp.tcp_rcv_established
0.92 ± 6% -0.4 0.55 ± 6% perf-profile.self.cycles-pp.skb_release_data
0.40 ± 9% -0.3 0.06 ± 6% perf-profile.self.cycles-pp.skb_attempt_defer_free
0.40 ± 8% -0.3 0.08 ± 9% perf-profile.self.cycles-pp.tcp_rcv_space_adjust
0.82 ± 7% -0.3 0.50 ± 8% perf-profile.self.cycles-pp._raw_spin_lock
0.54 ± 5% -0.3 0.24 ± 9% perf-profile.self.cycles-pp.check_heap_object
0.43 ± 6% -0.3 0.16 ± 14% perf-profile.self.cycles-pp.prepare_task_switch
0.57 ± 6% -0.3 0.32 ± 11% perf-profile.self.cycles-pp.__skb_datagram_iter
0.56 ± 9% -0.2 0.32 ± 10% perf-profile.self.cycles-pp.tcp_check_space
0.39 ± 7% -0.2 0.19 ± 11% perf-profile.self.cycles-pp.enqueue_entity
0.35 ± 8% -0.2 0.16 ± 8% perf-profile.self.cycles-pp.kfree_skbmem
0.57 ± 7% -0.2 0.40 ± 8% perf-profile.self.cycles-pp.copy_user_short_string
0.26 ± 9% -0.2 0.09 ± 7% perf-profile.self.cycles-pp.__wake_up_common
0.19 ± 7% -0.2 0.03 ± 70% perf-profile.self.cycles-pp.skb_release_head_state
0.22 ± 8% -0.2 0.07 ± 20% perf-profile.self.cycles-pp.perf_tp_event
0.21 ± 4% -0.2 0.06 ± 19% perf-profile.self.cycles-pp.cpuidle_enter_state
0.67 ± 7% -0.2 0.52 ± 8% perf-profile.self.cycles-pp.tcp_recvmsg_locked
0.25 ± 9% -0.1 0.14 ± 12% perf-profile.self.cycles-pp.dequeue_entity
0.14 ± 9% -0.1 0.05 ± 46% perf-profile.self.cycles-pp.select_task_rq
0.34 ± 3% -0.1 0.27 ± 14% perf-profile.self.cycles-pp.___perf_sw_event
0.16 ± 12% -0.1 0.10 ± 16% perf-profile.self.cycles-pp.perf_trace_sched_wakeup_template
0.20 ± 7% -0.0 0.16 ± 7% perf-profile.self.cycles-pp.finish_task_switch
0.09 ± 11% -0.0 0.06 ± 9% perf-profile.self.cycles-pp.check_preempt_curr
0.06 ± 7% -0.0 0.03 ± 70% perf-profile.self.cycles-pp.tracing_gen_ctx_irq_test
0.06 ± 11% +0.0 0.09 ± 9% perf-profile.self.cycles-pp.tcp_event_data_recv
0.05 ± 45% +0.0 0.08 ± 7% perf-profile.self.cycles-pp.put_prev_entity
0.22 ± 7% +0.0 0.25 ± 7% perf-profile.self.cycles-pp.__wrgsbase_inactive
0.10 ± 10% +0.0 0.14 ± 15% perf-profile.self.cycles-pp.tcp_rearm_rto
0.22 ± 6% +0.0 0.26 ± 8% perf-profile.self.cycles-pp.dequeue_task_fair
0.13 ± 6% +0.0 0.17 ± 11% perf-profile.self.cycles-pp.update_rq_clock
0.04 ± 47% +0.0 0.09 ± 18% perf-profile.self.cycles-pp.tcp_v4_do_rcv
0.06 ± 11% +0.0 0.10 ± 13% perf-profile.self.cycles-pp.ip_skb_dst_mtu
0.02 ±142% +0.0 0.06 ± 21% perf-profile.self.cycles-pp.timekeeping_max_deferment
0.02 ±141% +0.0 0.06 ± 11% perf-profile.self.cycles-pp.place_entity
0.01 ±223% +0.0 0.06 ± 13% perf-profile.self.cycles-pp.skb_clone
0.00 +0.1 0.05 perf-profile.self.cycles-pp.minmax_running_min
0.06 ± 13% +0.1 0.11 ± 9% perf-profile.self.cycles-pp.rcu_all_qs
0.00 +0.1 0.05 ± 7% perf-profile.self.cycles-pp.__netif_receive_skb_one_core
0.00 +0.1 0.05 ± 7% perf-profile.self.cycles-pp.clear_buddies
0.00 +0.1 0.05 ± 7% perf-profile.self.cycles-pp.tcp_small_queue_check
0.18 ± 7% +0.1 0.23 ± 10% perf-profile.self.cycles-pp.do_softirq
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.ip_local_deliver
0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.__kmalloc_node_track_caller
0.00 +0.1 0.06 ± 13% perf-profile.self.cycles-pp.skb_copy_datagram_iter
0.00 +0.1 0.06 ± 13% perf-profile.self.cycles-pp.cubictcp_cong_avoid
0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.sk_reset_timer
0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.switch_ldt
0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.rb_insert_color
0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.remove_wait_queue
0.00 +0.1 0.06 ± 6% perf-profile.self.cycles-pp.__ip_finish_output
0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.nf_hook_slow
0.02 ±141% +0.1 0.08 ± 10% perf-profile.self.cycles-pp.tcp_send_mss
0.00 +0.1 0.06 ± 15% perf-profile.self.cycles-pp.eth_type_trans
0.13 ± 8% +0.1 0.19 ± 10% perf-profile.self.cycles-pp.schedule_timeout
0.00 +0.1 0.06 ± 13% perf-profile.self.cycles-pp.memcg_slab_post_alloc_hook
0.00 +0.1 0.06 ± 16% perf-profile.self.cycles-pp.kmalloc_reserve
0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.sk_filter_trim_cap
0.00 +0.1 0.06 ± 16% perf-profile.self.cycles-pp.kmalloc_slab
0.06 ± 13% +0.1 0.12 ± 8% perf-profile.self.cycles-pp.tcp_recvmsg
0.00 +0.1 0.06 ± 6% perf-profile.self.cycles-pp.tcp_inbound_md5_hash
0.00 +0.1 0.06 ± 17% perf-profile.self.cycles-pp.apparmor_socket_recvmsg
0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.__wake_up_common_lock
0.07 ± 12% +0.1 0.14 ± 14% perf-profile.self.cycles-pp.update_min_vruntime
0.06 ± 11% +0.1 0.12 ± 10% perf-profile.self.cycles-pp.tcp_rate_skb_sent
0.04 ± 45% +0.1 0.11 ± 9% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare
0.08 ± 6% +0.1 0.14 ± 6% perf-profile.self.cycles-pp.ip_output
0.04 ± 44% +0.1 0.11 ± 5% perf-profile.self.cycles-pp.tcp_established_options
0.01 ±223% +0.1 0.08 ± 9% perf-profile.self.cycles-pp.lock_sock_nested
0.10 ± 16% +0.1 0.17 ± 11% perf-profile.self.cycles-pp.perf_trace_sched_switch
0.08 ± 8% +0.1 0.16 ± 10% perf-profile.self.cycles-pp.try_charge_memcg
0.00 +0.1 0.07 ± 14% perf-profile.self.cycles-pp.tcp_stream_alloc_skb
0.00 +0.1 0.07 ± 20% perf-profile.self.cycles-pp.cubictcp_acked
0.59 ± 6% +0.1 0.66 ± 6% perf-profile.self.cycles-pp.__switch_to
0.07 ± 7% +0.1 0.14 ± 11% perf-profile.self.cycles-pp.irqtime_account_irq
0.07 ± 8% +0.1 0.14 ± 11% perf-profile.self.cycles-pp.__copy_skb_header
0.06 ± 9% +0.1 0.13 ± 5% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
0.00 +0.1 0.07 ± 10% perf-profile.self.cycles-pp.skb_do_copy_data_nocache
0.04 ± 44% +0.1 0.12 ± 10% perf-profile.self.cycles-pp.dev_hard_start_xmit
0.11 ± 6% +0.1 0.19 ± 11% perf-profile.self.cycles-pp._copy_to_iter
0.04 ± 71% +0.1 0.11 ± 8% perf-profile.self.cycles-pp.mem_cgroup_uncharge_skmem
0.03 ± 70% +0.1 0.11 ± 9% perf-profile.self.cycles-pp.syscall_exit_to_user_mode
0.06 ± 8% +0.1 0.14 ± 7% perf-profile.self.cycles-pp.raw_v4_input
0.03 ± 70% +0.1 0.11 ± 8% perf-profile.self.cycles-pp.sk_free
0.00 +0.1 0.08 ± 8% perf-profile.self.cycles-pp.neigh_hh_output
0.26 ± 7% +0.1 0.34 ± 9% perf-profile.self.cycles-pp.recv_data
0.12 ± 11% +0.1 0.20 ± 8% perf-profile.self.cycles-pp.ip_send_check
0.00 +0.1 0.08 ± 7% perf-profile.self.cycles-pp.syscall_enter_from_user_mode
0.06 ± 11% +0.1 0.15 ± 18% perf-profile.self.cycles-pp.validate_xmit_skb
0.00 +0.1 0.08 ± 5% perf-profile.self.cycles-pp.__napi_poll
0.00 +0.1 0.08 ± 5% perf-profile.self.cycles-pp.tcp_v4_fill_cb
0.00 +0.1 0.08 ± 11% perf-profile.self.cycles-pp.apparmor_socket_sendmsg
0.04 ± 45% +0.1 0.13 ± 8% perf-profile.self.cycles-pp.inet_ehashfn
0.04 ± 45% +0.1 0.13 ± 8% perf-profile.self.cycles-pp.tcp_rate_gen
0.02 ±141% +0.1 0.10 ± 8% perf-profile.self.cycles-pp.tcp_mstamp_refresh
0.00 +0.1 0.08 ± 14% perf-profile.self.cycles-pp.netif_skb_features
0.00 +0.1 0.09 ± 15% perf-profile.self.cycles-pp.__might_fault
0.00 +0.1 0.09 ± 7% perf-profile.self.cycles-pp.tcp_newly_delivered
0.10 ± 6% +0.1 0.19 ± 14% perf-profile.self.cycles-pp.sk_forced_mem_schedule
0.09 ± 7% +0.1 0.18 ± 9% perf-profile.self.cycles-pp.__ksize
0.00 +0.1 0.09 ± 6% perf-profile.self.cycles-pp.ip_local_deliver_finish
0.00 +0.1 0.09 ± 6% perf-profile.self.cycles-pp.check_stack_object
0.08 ± 14% +0.1 0.18 ± 16% perf-profile.self.cycles-pp.tcp_rtt_estimator
0.06 ± 11% +0.1 0.16 ± 5% perf-profile.self.cycles-pp.sock_put
0.10 ± 10% +0.1 0.20 ± 10% perf-profile.self.cycles-pp.__cond_resched
0.22 ± 8% +0.1 0.32 ± 9% perf-profile.self.cycles-pp.schedule
0.06 ± 9% +0.1 0.16 ± 9% perf-profile.self.cycles-pp.sockfd_lookup_light
0.00 +0.1 0.10 ± 12% perf-profile.self.cycles-pp.resched_curr
0.11 ± 6% +0.1 0.21 ± 20% perf-profile.self.cycles-pp.__mod_memcg_state
0.13 ± 9% +0.1 0.23 ± 19% perf-profile.self.cycles-pp.select_task_rq_fair
0.10 ± 10% +0.1 0.21 ± 7% perf-profile.self.cycles-pp.__tcp_select_window
0.07 ± 5% +0.1 0.17 ± 8% perf-profile.self.cycles-pp.enqueue_to_backlog
0.09 ± 5% +0.1 0.20 ± 12% perf-profile.self.cycles-pp.__x64_sys_recvfrom
0.06 ± 14% +0.1 0.17 ± 25% perf-profile.self.cycles-pp.cgroup_rstat_updated
0.00 +0.1 0.10 ± 4% perf-profile.self.cycles-pp.tcp_ack_update_rtt
0.00 +0.1 0.10 ± 4% perf-profile.self.cycles-pp.tcp_push
0.01 ±223% +0.1 0.11 ± 6% perf-profile.self.cycles-pp.__usecs_to_jiffies
0.35 ± 7% +0.1 0.45 ± 7% perf-profile.self.cycles-pp.update_load_avg
0.09 ± 7% +0.1 0.20 ± 8% perf-profile.self.cycles-pp.sock_sendmsg
0.00 +0.1 0.11 ± 9% perf-profile.self.cycles-pp.ip_rcv
0.12 ± 6% +0.1 0.23 ± 9% perf-profile.self.cycles-pp.security_socket_sendmsg
0.16 ± 10% +0.1 0.27 ± 8% perf-profile.self.cycles-pp.sk_wait_data
0.00 +0.1 0.11 ± 8% perf-profile.self.cycles-pp.__ip_local_out
0.09 ± 14% +0.1 0.20 ± 8% perf-profile.self.cycles-pp.mem_cgroup_charge_skmem
0.18 ± 10% +0.1 0.29 ± 14% perf-profile.self.cycles-pp.__calc_delta
0.09 ± 6% +0.1 0.21 ± 8% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.24 ± 6% +0.1 0.36 ± 11% perf-profile.self.cycles-pp.__local_bh_enable_ip
0.00 +0.1 0.13 ± 8% perf-profile.self.cycles-pp.cubictcp_cwnd_event
0.14 ± 7% +0.1 0.27 ± 5% perf-profile.self.cycles-pp.pick_next_task_fair
0.04 ± 71% +0.1 0.17 ± 11% perf-profile.self.cycles-pp.skb_page_frag_refill
0.15 ± 7% +0.1 0.28 ± 8% perf-profile.self.cycles-pp.switch_fpu_return
0.11 ± 6% +0.1 0.24 ± 9% perf-profile.self.cycles-pp.tcp_sendmsg
0.07 ± 14% +0.1 0.20 ± 8% perf-profile.self.cycles-pp.tcp_update_skb_after_send
0.15 ± 4% +0.1 0.29 ± 9% perf-profile.self.cycles-pp.__sk_mem_reduce_allocated
0.29 ± 6% +0.1 0.42 ± 8% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.29 ± 9% +0.1 0.42 ± 8% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.10 ± 7% +0.1 0.24 ± 9% perf-profile.self.cycles-pp.tcp_tso_segs
0.11 ± 6% +0.1 0.25 ± 7% perf-profile.self.cycles-pp.__netif_receive_skb_core
0.07 ± 12% +0.1 0.21 ± 5% perf-profile.self.cycles-pp.__tcp_push_pending_frames
0.34 ± 9% +0.1 0.48 ± 8% perf-profile.self.cycles-pp.update_curr
0.18 ± 9% +0.1 0.32 ± 7% perf-profile.self.cycles-pp.enqueue_task_fair
0.08 ± 19% +0.1 0.22 ± 18% perf-profile.self.cycles-pp.cpuacct_charge
0.08 ± 8% +0.1 0.23 ± 9% perf-profile.self.cycles-pp.ip_rcv_core
0.01 ±223% +0.1 0.16 ± 12% perf-profile.self.cycles-pp.inet_recvmsg
0.29 ± 5% +0.2 0.44 ± 6% perf-profile.self.cycles-pp.__update_load_avg_se
0.10 ± 5% +0.2 0.25 ± 10% perf-profile.self.cycles-pp.__list_del_entry_valid
0.10 ± 6% +0.2 0.26 ± 8% perf-profile.self.cycles-pp.process_backlog
0.12 ± 8% +0.2 0.27 ± 7% perf-profile.self.cycles-pp.__x64_sys_sendto
0.00 +0.2 0.16 ± 14% perf-profile.self.cycles-pp.check_preempt_wakeup
0.28 ± 8% +0.2 0.44 ± 6% perf-profile.self.cycles-pp.__skb_clone
0.12 ± 8% +0.2 0.28 ± 7% perf-profile.self.cycles-pp.tcp_schedule_loss_probe
0.34 ± 4% +0.2 0.51 ± 8% perf-profile.self.cycles-pp.update_cfs_group
0.16 ± 6% +0.2 0.33 ± 7% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.14 ± 7% +0.2 0.31 ± 8% perf-profile.self.cycles-pp.tcp_update_pacing_rate
0.11 ± 6% +0.2 0.28 ± 8% perf-profile.self.cycles-pp.refill_stock
0.09 ± 13% +0.2 0.26 ± 8% perf-profile.self.cycles-pp.ip_protocol_deliver_rcu
0.00 +0.2 0.17 ± 11% perf-profile.self.cycles-pp.security_socket_recvmsg
0.17 ± 9% +0.2 0.34 ± 8% perf-profile.self.cycles-pp.__alloc_skb
0.08 ± 25% +0.2 0.26 ± 17% perf-profile.self.cycles-pp.update_sg_lb_stats
0.27 ± 5% +0.2 0.46 ± 7% perf-profile.self.cycles-pp.send_omni_inner
0.06 ± 7% +0.2 0.25 ± 8% perf-profile.self.cycles-pp.__list_add_valid
0.22 ± 8% +0.2 0.40 ± 11% perf-profile.self.cycles-pp.recv
0.00 +0.2 0.19 ± 8% perf-profile.self.cycles-pp.exit_to_user_mode_loop
0.20 ± 8% +0.2 0.39 ± 8% perf-profile.self.cycles-pp.send_data
0.38 ± 8% +0.2 0.57 ± 23% perf-profile.self.cycles-pp.ktime_get
0.45 ± 7% +0.2 0.65 ± 7% perf-profile.self.cycles-pp.read_tsc
0.16 ± 4% +0.2 0.35 ± 9% perf-profile.self.cycles-pp.tcp_wfree
0.00 +0.2 0.20 ± 9% perf-profile.self.cycles-pp.set_next_buddy
0.16 ± 9% +0.2 0.36 ± 6% perf-profile.self.cycles-pp.__might_sleep
0.06 ± 7% +0.2 0.27 ± 8% perf-profile.self.cycles-pp.__kmem_cache_free
0.32 ± 5% +0.2 0.53 ± 9% perf-profile.self.cycles-pp.__sys_recvfrom
0.25 ± 14% +0.2 0.46 ± 9% perf-profile.self.cycles-pp.recv_omni
0.62 ± 7% +0.2 0.84 ± 8% perf-profile.self.cycles-pp.__schedule
0.16 ± 7% +0.2 0.38 ± 9% perf-profile.self.cycles-pp.__kmem_cache_alloc_node
0.19 ± 8% +0.2 0.42 ± 7% perf-profile.self.cycles-pp.__softirqentry_text_start
0.67 ± 6% +0.2 0.89 ± 8% perf-profile.self.cycles-pp.__switch_to_asm
0.09 ± 7% +0.2 0.32 ± 8% perf-profile.self.cycles-pp._copy_from_iter
0.18 ± 6% +0.2 0.42 ± 5% perf-profile.self.cycles-pp.__send
0.10 ± 11% +0.2 0.34 ± 10% perf-profile.self.cycles-pp.__check_object_size
0.07 ± 10% +0.2 0.31 ± 8% perf-profile.self.cycles-pp.kmem_cache_free
0.12 ± 7% +0.3 0.38 ± 8% perf-profile.self.cycles-pp.kmem_cache_alloc_node
0.26 ± 6% +0.3 0.52 ± 7% perf-profile.self.cycles-pp.do_syscall_64
0.21 ± 7% +0.3 0.48 ± 7% perf-profile.self.cycles-pp.__entry_text_start
0.22 ± 7% +0.3 0.50 ± 8% perf-profile.self.cycles-pp.__sys_sendto
0.22 ± 8% +0.3 0.50 ± 6% perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
0.53 ± 7% +0.3 0.84 ± 7% perf-profile.self.cycles-pp._raw_spin_lock_bh
0.25 ± 7% +0.3 0.56 ± 8% perf-profile.self.cycles-pp.os_xsave
0.13 ± 4% +0.3 0.46 ± 7% perf-profile.self.cycles-pp.__virt_addr_valid
0.16 ± 7% +0.3 0.49 ± 7% perf-profile.self.cycles-pp.loopback_xmit
0.05 ± 7% +0.3 0.40 ± 9% perf-profile.self.cycles-pp.import_single_range
0.32 ± 6% +0.4 0.67 ± 9% perf-profile.self.cycles-pp.ip_finish_output2
0.01 ±223% +0.4 0.36 ± 9% perf-profile.self.cycles-pp.exit_to_user_mode_prepare
0.21 ± 8% +0.4 0.59 ± 8% perf-profile.self.cycles-pp.aa_sk_perm
0.27 ± 6% +0.4 0.69 ± 6% perf-profile.self.cycles-pp.__ip_queue_xmit
0.29 ± 6% +0.4 0.73 ± 8% perf-profile.self.cycles-pp.tcp_write_xmit
0.27 ± 10% +0.5 0.74 ± 10% perf-profile.self.cycles-pp.__fdget
0.00 +0.5 0.49 ± 10% perf-profile.self.cycles-pp.kfree
0.66 ± 6% +0.5 1.16 ± 8% perf-profile.self.cycles-pp.__tcp_transmit_skb
0.38 ± 6% +0.6 0.96 ± 8% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.00 +1.0 1.01 ± 8% perf-profile.self.cycles-pp.restore_fpregs_from_fpstate
0.10 ± 9% +2.1 2.17 ± 7% perf-profile.self.cycles-pp.switch_mm_irqs_off
If you fix the issue, kindly add following tag
| Reported-by: kernel test robot <yujie.liu@...el.com>
| Link: https://lore.kernel.org/oe-lkp/202211221125.64e73ddf-yujie.liu@intel.com
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
sudo bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
sudo bin/lkp run generated-yaml-file
# if come across any failure that blocks the test,
# please remove ~/.lkp and /lkp dir to run from a clean state.
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
--
0-DAY CI Kernel Test Service
https://01.org/lkp
View attachment "config-6.1.0-rc2-00018-gb6aabb01e300" of type "text/plain" (166032 bytes)
View attachment "job-script" of type "text/plain" (8518 bytes)
View attachment "job.yaml" of type "text/plain" (5757 bytes)
View attachment "reproduce" of type "text/plain" (4769 bytes)
Powered by blists - more mailing lists