| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20211124083435.GB18309@xsang-OptiPlex-9020>
Date: Wed, 24 Nov 2021 16:34:35 +0800
From: kernel test robot <oliver.sang@...el.com>
To: Shakeel Butt <shakeelb@...gle.com>
Cc: Linus Torvalds <torvalds@...ux-foundation.org>,
Arnd Bergmann <arnd@...db.de>, Roman Gushchin <guro@...com>,
Michal Hocko <mhocko@...e.com>,
Muchun Song <songmuchun@...edance.com>,
Vasily Averin <vvs@...tuozzo.com>,
Johannes Weiner <hannes@...xchg.org>,
Andrew Morton <akpm@...ux-foundation.org>,
LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org,
lkp@...el.com, ying.huang@...el.com, feng.tang@...el.com,
zhengjun.xing@...ux.intel.com, fengwei.yin@...el.com
Subject: [memcg, kmem] 58056f7750: hackbench.throughput 10.3% improvement
Greeting,
FYI, we noticed a 10.3% improvement of hackbench.throughput due to commit:
commit: 58056f77502f3567b760c9a8fc8d2e9081515b2d ("memcg, kmem: further deprecate kmem.limit_in_bytes")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: hackbench
on test machine: 144 threads 4 sockets Intel(R) Xeon(R) Gold 5318H CPU @ 2.50GHz with 128G memory
with following parameters:
nr_threads: 100%
iterations: 4
mode: process
ipc: socket
cpufreq_governor: performance
ucode: 0x700001e
test-description: Hackbench is both a benchmark and a stress test for the Linux kernel scheduler.
test-url: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/sched/cfs-scheduler/hackbench.c
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
sudo bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
sudo bin/lkp run generated-yaml-file
# if come across any failure that blocks the test,
# please remove ~/.lkp and /lkp dir to run from a clean state.
=========================================================================================
compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-9/performance/socket/4/x86_64-rhel-8.3/process/100%/debian-10.4-x86_64-20200603.cgz/lkp-cpl-4sp1/hackbench/0x700001e
commit:
16f6bf266c ("mm/list_lru.c: prefer struct_size over open coded arithmetic")
58056f7750 ("memcg, kmem: further deprecate kmem.limit_in_bytes")
16f6bf266c94017c 58056f77502f3567b760c9a8fc8
---------------- ---------------------------
%stddev %change %stddev
\ | \
124164 +10.3% 137012 ± 2% hackbench.throughput
553.20 -9.9% 498.51 ± 2% hackbench.time.elapsed_time
553.20 -9.9% 498.51 ± 2% hackbench.time.elapsed_time.max
3.519e+08 ± 4% -56.3% 1.539e+08 ± 2% hackbench.time.involuntary_context_switches
76944 -10.0% 69266 ± 2% hackbench.time.system_time
1046 -15.9% 880.42 ± 2% hackbench.time.user_time
9.209e+08 ± 2% -41.2% 5.414e+08 hackbench.time.voluntary_context_switches
516011 ± 17% -33.5% 343117 ± 32% numa-numastat.node1.numa_hit
260286 ± 8% -16.7% 216841 ± 15% numa-vmstat.node3.nr_shmem
607.74 -8.8% 554.41 uptime.boot
164520 +2.7% 168980 proc-vmstat.nr_slab_unreclaimable
137719 -4.5% 131582 proc-vmstat.pgreuse
0.42 ± 6% +0.1 0.53 ± 6% turbostat.C1%
2.966e+08 -18.8% 2.408e+08 ± 2% turbostat.IRQ
2746 -12.3% 2408 vmstat.procs.r
2336697 ± 3% -38.4% 1439816 vmstat.system.cs
534308 -9.9% 481527 vmstat.system.in
5546 ± 5% -12.6% 4846 ± 6% slabinfo.kmalloc-cg-192.active_objs
5546 ± 5% -12.6% 4846 ± 6% slabinfo.kmalloc-cg-192.num_objs
157208 +9.5% 172170 slabinfo.kmalloc-cg-512.active_objs
3128 +11.3% 3480 slabinfo.kmalloc-cg-512.active_slabs
200236 +11.3% 222802 slabinfo.kmalloc-cg-512.num_objs
3128 +11.3% 3480 slabinfo.kmalloc-cg-512.num_slabs
157390 +9.5% 172289 slabinfo.skbuff_head_cache.active_objs
3134 +11.2% 3485 slabinfo.skbuff_head_cache.active_slabs
200626 +11.2% 223129 slabinfo.skbuff_head_cache.num_objs
3134 +11.2% 3485 slabinfo.skbuff_head_cache.num_slabs
1.669e+10 -5.2% 1.582e+10 perf-stat.i.branch-instructions
0.80 -0.2 0.64 ± 2% perf-stat.i.branch-miss-rate%
1.312e+08 -24.3% 99283197 ± 2% perf-stat.i.branch-misses
27.80 +1.1 28.86 perf-stat.i.cache-miss-rate%
8.619e+08 -1.5% 8.491e+08 perf-stat.i.cache-references
2349252 ± 3% -38.2% 1452325 perf-stat.i.context-switches
143032 ± 2% -14.7% 121977 ± 2% perf-stat.i.cpu-migrations
2001 -3.5% 1931 perf-stat.i.cycles-between-cache-misses
0.07 ± 5% -0.0 0.06 ± 5% perf-stat.i.dTLB-load-miss-rate%
16549027 ± 4% -12.9% 14414299 ± 3% perf-stat.i.dTLB-load-misses
2.392e+10 -1.9% 2.348e+10 perf-stat.i.dTLB-loads
0.03 ± 11% +0.0 0.04 ± 6% perf-stat.i.dTLB-store-miss-rate%
4025883 ± 11% +30.5% 5252098 ± 5% perf-stat.i.dTLB-store-misses
56000903 -35.7% 35989438 ± 8% perf-stat.i.iTLB-load-misses
23047129 ± 3% -35.8% 14787249 perf-stat.i.iTLB-loads
8.356e+10 -3.9% 8.027e+10 perf-stat.i.instructions
1548 +47.8% 2288 ± 7% perf-stat.i.instructions-per-iTLB-miss
0.18 -4.4% 0.17 perf-stat.i.ipc
869.08 +4.5% 908.26 perf-stat.i.metric.K/sec
384.93 -2.7% 374.53 perf-stat.i.metric.M/sec
7130 +3.7% 7398 ± 3% perf-stat.i.minor-faults
66.31 -6.3 60.05 ± 2% perf-stat.i.node-load-miss-rate%
39678105 -3.8% 38161927 perf-stat.i.node-load-misses
20549470 +25.2% 25719712 ± 5% perf-stat.i.node-loads
43.76 -6.0 37.72 ± 5% perf-stat.i.node-store-miss-rate%
15613166 ± 2% +10.0% 17180670 ± 4% perf-stat.i.node-store-misses
20301635 +40.5% 28518438 ± 4% perf-stat.i.node-stores
7172 +3.8% 7443 ± 3% perf-stat.i.page-faults
0.79 -0.2 0.62 ± 2% perf-stat.overall.branch-miss-rate%
28.13 +0.9 29.08 perf-stat.overall.cache-miss-rate%
5.65 +2.5% 5.79 perf-stat.overall.cpi
1942 -2.3% 1899 perf-stat.overall.cycles-between-cache-misses
0.07 ± 5% -0.0 0.06 ± 5% perf-stat.overall.dTLB-load-miss-rate%
0.03 ± 11% +0.0 0.04 ± 6% perf-stat.overall.dTLB-store-miss-rate%
1500 +49.7% 2246 ± 7% perf-stat.overall.instructions-per-iTLB-miss
0.18 -2.4% 0.17 perf-stat.overall.ipc
64.79 -5.9 58.87 perf-stat.overall.node-load-miss-rate%
40.92 -5.0 35.89 ± 3% perf-stat.overall.node-store-miss-rate%
1.641e+10 -3.5% 1.584e+10 perf-stat.ps.branch-instructions
1.288e+08 -23.6% 98487415 ± 2% perf-stat.ps.branch-misses
2337457 ± 3% -38.5% 1437257 perf-stat.ps.context-switches
130209 ± 2% -13.6% 112490 perf-stat.ps.cpu-migrations
16228057 ± 4% -11.2% 14403294 ± 3% perf-stat.ps.dTLB-load-misses
3947999 ± 10% +31.5% 5191308 ± 5% perf-stat.ps.dTLB-store-misses
54846930 -34.4% 35981347 ± 8% perf-stat.ps.iTLB-load-misses
22682918 ± 2% -35.8% 14572556 perf-stat.ps.iTLB-loads
8.23e+10 -2.4% 8.035e+10 perf-stat.ps.instructions
20.85 ± 3% +15.9% 24.16 ± 4% perf-stat.ps.major-faults
5283 +10.5% 5839 perf-stat.ps.minor-faults
39315654 -4.2% 37669399 perf-stat.ps.node-load-misses
21363970 +23.3% 26334557 ± 4% perf-stat.ps.node-loads
14602656 +11.5% 16280558 ± 2% perf-stat.ps.node-store-misses
21089334 +38.0% 29102191 ± 3% perf-stat.ps.node-stores
5304 +10.5% 5863 perf-stat.ps.page-faults
4.56e+13 -12.0% 4.011e+13 perf-stat.total.instructions
0.14 ±149% +6903.8% 10.10 ± 90% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
146.03 ± 20% -59.7% 58.91 ± 32% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
20.00 ± 16% -39.8% 12.05 ± 15% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
45.20 ± 21% -45.1% 24.80 ± 17% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
168.56 ± 19% -77.4% 38.03 ± 34% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
87.74 ± 39% -69.7% 26.57 ± 32% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
142.77 ± 20% -39.2% 86.82 ± 21% perf-sched.sch_delay.avg.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
22.06 ± 12% -51.5% 10.70 ± 19% perf-sched.sch_delay.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
6.08 ±146% +12273.2% 752.85 ± 96% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
22236 ± 14% -62.5% 8341 ± 29% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
26048 ± 9% -52.9% 12272 ± 28% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
24687 ± 8% -61.5% 9499 ± 27% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
24838 ± 8% -70.9% 7234 ± 34% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
21611 ± 13% -65.7% 7403 ± 44% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
26017 ± 10% -53.3% 12149 ± 32% perf-sched.sch_delay.max.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
26199 ± 9% -57.5% 11127 ± 28% perf-sched.sch_delay.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
26.44 ± 13% -36.5% 16.79 ± 16% perf-sched.total_sch_delay.average.ms
26690 ± 10% -52.4% 12702 ± 29% perf-sched.total_sch_delay.max.ms
92.56 ± 13% -34.8% 60.38 ± 16% perf-sched.total_wait_and_delay.average.ms
53236 ± 9% -51.9% 25601 ± 30% perf-sched.total_wait_and_delay.max.ms
66.12 ± 13% -34.1% 43.58 ± 17% perf-sched.total_wait_time.average.ms
26766 ± 8% -47.8% 13965 ± 29% perf-sched.total_wait_time.max.ms
549.67 ± 18% -64.8% 193.37 ± 28% perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
75.13 ± 14% -41.6% 43.84 ± 14% perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
167.49 ± 17% -45.5% 91.23 ± 16% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
594.39 ± 14% -76.9% 137.13 ± 29% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
317.87 ± 21% -65.9% 108.34 ± 28% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
1953 ±115% -90.1% 193.28 ±107% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.rmap_walk_anon.remove_migration_ptes.migrate_pages
154.92 ± 33% -82.5% 27.16 ±152% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr_locked
7364 ± 51% -83.3% 1228 ±110% perf-sched.wait_and_delay.avg.ms.rwsem_down_read_slowpath.rmap_walk_anon.remove_migration_ptes.migrate_pages
532.84 ± 22% -42.2% 308.22 ± 23% perf-sched.wait_and_delay.avg.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
73.20 ± 12% -48.1% 37.98 ± 19% perf-sched.wait_and_delay.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
131.17 ± 12% +54.1% 202.17 ± 14% perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
1020 ± 16% +30.8% 1335 ± 15% perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
12199 ± 15% +39.9% 17063 ± 19% perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
1423 ± 12% +44.5% 2056 ± 17% perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
6.00 ± 19% +88.9% 11.33 ± 15% perf-sched.wait_and_delay.count.schedule_timeout.kcompactd.kthread.ret_from_fork
48952 ± 19% +143.5% 119182 ± 12% perf-sched.wait_and_delay.count.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
240.00 ± 17% +52.6% 366.17 ± 18% perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
119.67 ± 28% +60.3% 191.83 ± 17% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
45862 ± 11% -63.5% 16731 ± 29% perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
52359 ± 9% -52.3% 24957 ± 30% perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
49522 ± 8% -61.6% 19021 ± 27% perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
49837 ± 8% -70.5% 14699 ± 33% perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
43808 ± 12% -66.0% 14910 ± 44% perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
36717 ± 23% -82.8% 6317 ±122% perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.rmap_walk_anon.remove_migration_ptes.migrate_pages
31347 ± 40% -75.5% 7689 ±112% perf-sched.wait_and_delay.max.ms.rwsem_down_read_slowpath.rmap_walk_anon.remove_migration_ptes.migrate_pages
52076 ± 10% -52.5% 24745 ± 34% perf-sched.wait_and_delay.max.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
52504 ± 9% -57.3% 22437 ± 28% perf-sched.wait_and_delay.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
1.82 ±163% +2252.9% 42.73 ±129% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
403.64 ± 24% -66.7% 134.46 ± 28% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
55.12 ± 14% -42.3% 31.79 ± 14% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.02 ±108% +534.1% 0.13 ± 85% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__alloc_pages.alloc_pages_vma.do_anonymous_page
122.29 ± 16% -45.7% 66.43 ± 16% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
425.82 ± 16% -76.7% 99.10 ± 28% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
465.09 ±223% -100.0% 0.04 ±117% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.change_p4d_range.change_protection.change_prot_numa
9.43 ±125% +219.2% 30.11 ±104% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
230.14 ± 17% -64.5% 81.77 ± 28% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
70.21 ± 31% -56.3% 30.65 ± 43% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.unix_stream_read_generic.unix_stream_recvmsg
1362 ±123% -88.1% 161.89 ± 95% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.rmap_walk_anon.remove_migration_ptes.migrate_pages
0.17 ±118% +1194.7% 2.26 ± 80% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.stop_one_cpu.sched_exec.bprm_execve
154.92 ± 33% -64.9% 54.45 ± 45% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr_locked
5277 ± 37% -81.9% 953.75 ±103% perf-sched.wait_time.avg.ms.rwsem_down_read_slowpath.rmap_walk_anon.remove_migration_ptes.migrate_pages
390.06 ± 23% -43.2% 221.40 ± 23% perf-sched.wait_time.avg.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
51.14 ± 12% -46.7% 27.27 ± 19% perf-sched.wait_time.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
42.41 ±143% +7008.4% 3014 ±126% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
17958 ± 24% -60.2% 7144 ± 33% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
24588 ± 7% -65.6% 8461 ± 28% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
26488 ± 8% -51.3% 12906 ± 31% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
20113 ± 16% -45.9% 10885 ± 38% perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
25666 ± 9% -57.7% 10860 ± 33% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
19730 ± 11% -59.6% 7972 ± 23% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_recvmsg.sock_recvmsg
25151 ± 8% -64.3% 8977 ± 31% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
1860 ±223% -100.0% 0.06 ±105% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.change_p4d_range.change_protection.change_prot_numa
4.30 ±194% -97.7% 0.10 ±116% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.down_write_killable.vm_mmap_pgoff.elf_map
320.22 ±175% +1509.4% 5153 ± 71% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
24087 ± 12% -61.8% 9191 ± 42% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
17911 ± 16% -62.7% 6687 ± 29% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.unix_stream_read_generic.unix_stream_recvmsg
20724 ± 17% -79.9% 4168 ± 82% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.rmap_walk_anon.remove_migration_ptes.migrate_pages
1.20 ±151% +2217.5% 27.84 ± 75% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.stop_one_cpu.sched_exec.bprm_execve
19696 ± 23% -77.1% 4500 ± 99% perf-sched.wait_time.max.ms.rwsem_down_read_slowpath.rmap_walk_anon.remove_migration_ptes.migrate_pages
26233 ± 10% -51.2% 12801 ± 36% perf-sched.wait_time.max.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
26305 ± 9% -56.7% 11393 ± 28% perf-sched.wait_time.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
24961 ± 11% -45.5% 13607 ± 28% perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
25452 ± 10% -47.3% 13401 ± 30% perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork
84420 ± 6% -11.5% 74716 ± 8% softirqs.CPU0.RCU
86481 ± 5% -11.8% 76240 ± 9% softirqs.CPU1.RCU
87185 ± 5% -13.1% 75796 ± 9% softirqs.CPU10.RCU
92504 ± 6% -14.8% 78845 ± 7% softirqs.CPU100.RCU
92105 ± 6% -15.2% 78097 ± 7% softirqs.CPU101.RCU
92008 ± 6% -14.9% 78278 ± 7% softirqs.CPU102.RCU
92262 ± 6% -15.3% 78190 ± 6% softirqs.CPU103.RCU
92609 ± 6% -14.8% 78921 ± 7% softirqs.CPU104.RCU
92568 ± 6% -15.0% 78674 ± 7% softirqs.CPU105.RCU
93113 ± 6% -15.2% 78921 ± 7% softirqs.CPU106.RCU
94050 ± 6% -15.6% 79354 ± 8% softirqs.CPU107.RCU
86336 ± 6% -12.2% 75835 ± 7% softirqs.CPU108.RCU
85713 ± 6% -12.9% 74646 ± 9% softirqs.CPU109.RCU
85964 ± 5% -12.0% 75619 ± 9% softirqs.CPU11.RCU
85393 ± 7% -12.6% 74646 ± 9% softirqs.CPU110.RCU
85002 ± 6% -12.3% 74549 ± 8% softirqs.CPU111.RCU
95689 ± 6% -14.9% 81465 ± 10% softirqs.CPU112.RCU
95426 ± 6% -13.5% 82531 ± 8% softirqs.CPU113.RCU
93843 ± 6% -13.2% 81424 ± 9% softirqs.CPU114.RCU
94782 ± 6% -13.7% 81765 ± 8% softirqs.CPU115.RCU
95329 ± 6% -13.2% 82770 ± 9% softirqs.CPU116.RCU
95098 ± 6% -13.3% 82488 ± 9% softirqs.CPU117.RCU
94061 ± 6% -13.4% 81501 ± 8% softirqs.CPU118.RCU
94978 ± 6% -13.7% 81931 ± 9% softirqs.CPU119.RCU
85245 ± 6% -10.5% 76276 ± 9% softirqs.CPU12.RCU
94735 ± 6% -13.3% 82148 ± 9% softirqs.CPU120.RCU
95239 ± 6% -13.3% 82558 ± 9% softirqs.CPU121.RCU
94890 ± 6% -13.5% 82113 ± 9% softirqs.CPU122.RCU
94265 ± 6% -13.5% 81563 ± 8% softirqs.CPU123.RCU
95082 ± 6% -14.6% 81160 ± 10% softirqs.CPU124.RCU
95351 ± 6% -13.6% 82390 ± 9% softirqs.CPU125.RCU
81884 ± 5% -11.7% 72288 ± 8% softirqs.CPU126.RCU
81461 ± 6% -12.1% 71593 ± 8% softirqs.CPU127.RCU
88358 ± 5% -13.9% 76053 ± 9% softirqs.CPU128.RCU
88086 ± 5% -13.4% 76318 ± 8% softirqs.CPU129.RCU
14400 +9.9% 15821 ± 3% softirqs.CPU13.SCHED
88026 ± 5% -13.4% 76206 ± 8% softirqs.CPU130.RCU
88351 ± 5% -13.7% 76205 ± 9% softirqs.CPU131.RCU
87490 ± 5% -13.3% 75876 ± 9% softirqs.CPU132.RCU
87529 ± 5% -13.6% 75668 ± 9% softirqs.CPU133.RCU
87798 ± 5% -12.5% 76813 ± 9% softirqs.CPU134.RCU
88136 ± 5% -13.5% 76276 ± 9% softirqs.CPU135.RCU
88294 ± 5% -13.6% 76251 ± 9% softirqs.CPU136.RCU
87208 ± 5% -13.5% 75465 ± 9% softirqs.CPU137.RCU
87695 ± 5% -13.3% 75994 ± 9% softirqs.CPU138.RCU
88320 ± 5% -13.6% 76309 ± 9% softirqs.CPU139.RCU
86555 ± 6% -12.3% 75887 ± 8% softirqs.CPU14.RCU
14298 +7.8% 15412 ± 4% softirqs.CPU14.SCHED
88184 ± 5% -13.7% 76139 ± 9% softirqs.CPU140.RCU
87603 ± 5% -13.1% 76136 ± 9% softirqs.CPU141.RCU
87625 ± 5% -13.3% 75972 ± 9% softirqs.CPU142.RCU
88256 ± 5% -13.3% 76557 ± 9% softirqs.CPU143.RCU
87050 ± 5% -12.2% 76430 ± 8% softirqs.CPU15.RCU
81130 ± 6% -12.1% 71332 ± 8% softirqs.CPU16.RCU
82146 ± 6% -12.5% 71850 ± 9% softirqs.CPU17.RCU
14082 +10.2% 15524 ± 3% softirqs.CPU17.SCHED
93814 ± 5% -15.8% 79008 ± 7% softirqs.CPU18.RCU
92946 ± 5% -15.5% 78562 ± 7% softirqs.CPU19.RCU
86506 ± 5% -9.5% 78315 ± 6% softirqs.CPU2.RCU
92407 ± 5% -15.2% 78322 ± 7% softirqs.CPU20.RCU
92212 ± 6% -15.7% 77774 ± 7% softirqs.CPU21.RCU
92309 ± 6% -15.1% 78328 ± 7% softirqs.CPU22.RCU
92929 ± 5% -15.4% 78577 ± 7% softirqs.CPU23.RCU
93052 ± 6% -15.7% 78470 ± 7% softirqs.CPU24.RCU
92080 ± 6% -15.3% 77996 ± 7% softirqs.CPU25.RCU
91828 ± 5% -15.3% 77740 ± 7% softirqs.CPU26.RCU
92903 ± 5% -15.6% 78432 ± 7% softirqs.CPU27.RCU
92628 ± 6% -15.3% 78450 ± 7% softirqs.CPU28.RCU
92249 ± 6% -16.2% 77285 ± 8% softirqs.CPU29.RCU
86877 ± 5% -12.0% 76462 ± 8% softirqs.CPU3.RCU
91350 ± 6% -14.7% 77958 ± 8% softirqs.CPU30.RCU
91443 ± 6% -16.0% 76829 ± 5% softirqs.CPU31.RCU
86014 ± 6% -14.2% 73818 ± 8% softirqs.CPU32.RCU
85380 ± 6% -13.7% 73689 ± 8% softirqs.CPU33.RCU
86155 ± 5% -14.3% 73854 ± 8% softirqs.CPU34.RCU
86462 ± 5% -14.1% 74234 ± 8% softirqs.CPU35.RCU
92387 ± 6% -13.1% 80287 ± 9% softirqs.CPU36.RCU
92117 ± 6% -12.9% 80218 ± 8% softirqs.CPU37.RCU
90761 ± 8% -11.9% 79957 ± 9% softirqs.CPU38.RCU
90705 ± 5% -11.5% 80235 ± 9% softirqs.CPU39.RCU
86170 ± 5% -11.2% 76515 ± 7% softirqs.CPU4.RCU
13627 ± 6% +14.0% 15539 ± 3% softirqs.CPU4.SCHED
92368 ± 6% -13.7% 79721 ± 9% softirqs.CPU40.RCU
92055 ± 6% -13.0% 80123 ± 8% softirqs.CPU41.RCU
91447 ± 6% -13.0% 79565 ± 9% softirqs.CPU42.RCU
91880 ± 6% -13.2% 79751 ± 8% softirqs.CPU43.RCU
92254 ± 6% -12.7% 80501 ± 8% softirqs.CPU44.RCU
91783 ± 6% -12.8% 80015 ± 9% softirqs.CPU45.RCU
91421 ± 6% -12.9% 79645 ± 9% softirqs.CPU46.RCU
91390 ± 6% -12.8% 79722 ± 8% softirqs.CPU47.RCU
88932 ± 6% -12.8% 77543 ± 8% softirqs.CPU48.RCU
88850 ± 6% -12.4% 77848 ± 9% softirqs.CPU49.RCU
86897 ± 4% -12.3% 76176 ± 8% softirqs.CPU5.RCU
88709 ± 6% -12.5% 77631 ± 8% softirqs.CPU50.RCU
88614 ± 6% -12.5% 77564 ± 8% softirqs.CPU51.RCU
88876 ± 6% -13.3% 77084 ± 9% softirqs.CPU52.RCU
88779 ± 6% -12.4% 77752 ± 8% softirqs.CPU53.RCU
89712 ± 5% -13.5% 77589 ± 8% softirqs.CPU54.RCU
88569 ± 5% -13.2% 76854 ± 8% softirqs.CPU55.RCU
88777 ± 5% -13.1% 77130 ± 8% softirqs.CPU56.RCU
89211 ± 5% -13.8% 76929 ± 8% softirqs.CPU57.RCU
88989 ± 5% -13.0% 77400 ± 8% softirqs.CPU58.RCU
88940 ± 5% -13.1% 77250 ± 8% softirqs.CPU59.RCU
86373 ± 5% -12.3% 75749 ± 8% softirqs.CPU6.RCU
88320 ± 5% -13.0% 76839 ± 9% softirqs.CPU60.RCU
88405 ± 5% -13.2% 76706 ± 8% softirqs.CPU61.RCU
88722 ± 5% -12.8% 77340 ± 8% softirqs.CPU62.RCU
88541 ± 5% -12.9% 77089 ± 9% softirqs.CPU63.RCU
88636 ± 5% -13.6% 76600 ± 8% softirqs.CPU64.RCU
88387 ± 5% -13.4% 76579 ± 8% softirqs.CPU65.RCU
88139 ± 5% -13.3% 76411 ± 8% softirqs.CPU66.RCU
88420 ± 5% -13.3% 76702 ± 8% softirqs.CPU67.RCU
88484 ± 5% -13.2% 76794 ± 8% softirqs.CPU68.RCU
88366 ± 5% -12.7% 77154 ± 8% softirqs.CPU69.RCU
86334 ± 5% -12.1% 75872 ± 7% softirqs.CPU7.RCU
88438 ± 5% -13.4% 76599 ± 8% softirqs.CPU70.RCU
88125 ± 5% -13.3% 76369 ± 8% softirqs.CPU71.RCU
87488 ± 6% -12.1% 76899 ± 8% softirqs.CPU72.RCU
13664 ± 6% +12.1% 15314 ± 3% softirqs.CPU73.SCHED
87669 ± 6% -11.6% 77534 ± 7% softirqs.CPU74.RCU
87675 ± 5% -11.5% 77573 ± 8% softirqs.CPU75.RCU
87948 ± 5% -12.0% 77402 ± 8% softirqs.CPU76.RCU
88247 ± 5% -12.2% 77474 ± 7% softirqs.CPU77.RCU
87612 ± 5% -11.7% 77372 ± 8% softirqs.CPU78.RCU
87923 ± 6% -12.4% 77023 ± 8% softirqs.CPU79.RCU
86391 ± 5% -11.4% 76543 ± 8% softirqs.CPU8.RCU
88597 ± 6% -12.0% 78003 ± 8% softirqs.CPU80.RCU
89101 ± 6% -12.5% 77968 ± 8% softirqs.CPU81.RCU
14183 ± 2% +8.3% 15363 ± 3% softirqs.CPU81.SCHED
88545 ± 6% -12.4% 77527 ± 7% softirqs.CPU82.RCU
87757 ± 7% -11.5% 77621 ± 8% softirqs.CPU83.RCU
88307 ± 6% -11.5% 78128 ± 8% softirqs.CPU84.RCU
14211 +9.8% 15605 ± 3% softirqs.CPU84.SCHED
89048 ± 6% -13.0% 77482 ± 8% softirqs.CPU85.RCU
89209 ± 5% -12.3% 78258 ± 7% softirqs.CPU86.RCU
88691 ± 5% -12.3% 77825 ± 8% softirqs.CPU87.RCU
88012 ± 5% -11.8% 77588 ± 8% softirqs.CPU88.RCU
88781 ± 5% -11.9% 78190 ± 8% softirqs.CPU89.RCU
86346 ± 6% -10.6% 77225 ± 8% softirqs.CPU9.RCU
90192 ± 5% -14.8% 76846 ± 7% softirqs.CPU90.RCU
89520 ± 6% -14.7% 76378 ± 7% softirqs.CPU91.RCU
89174 ± 5% -14.4% 76374 ± 7% softirqs.CPU92.RCU
88839 ± 6% -14.4% 76030 ± 7% softirqs.CPU93.RCU
89065 ± 5% -14.4% 76238 ± 7% softirqs.CPU94.RCU
89354 ± 6% -14.3% 76619 ± 7% softirqs.CPU95.RCU
92662 ± 6% -14.8% 78959 ± 7% softirqs.CPU96.RCU
91610 ± 6% -14.8% 78044 ± 7% softirqs.CPU97.RCU
92230 ± 6% -15.0% 78392 ± 7% softirqs.CPU98.RCU
92985 ± 6% -14.9% 79138 ± 7% softirqs.CPU99.RCU
12886523 ± 6% -13.3% 11171782 ± 8% softirqs.RCU
112751 ± 3% -10.3% 101130 ± 7% softirqs.TIMER
6866613 ± 7% -20.9% 5431401 ± 9% interrupts.CAL:Function_call_interrupts
1109865 -13.2% 963752 ± 7% interrupts.CPU0.LOC:Local_timer_interrupts
746403 ± 6% -26.6% 547987 ± 6% interrupts.CPU0.RES:Rescheduling_interrupts
1109844 -11.5% 982545 ± 4% interrupts.CPU1.LOC:Local_timer_interrupts
784948 ± 6% -26.8% 574218 ± 6% interrupts.CPU1.RES:Rescheduling_interrupts
1109908 -11.5% 982586 ± 4% interrupts.CPU10.LOC:Local_timer_interrupts
768936 ± 6% -25.0% 576330 ± 7% interrupts.CPU10.RES:Rescheduling_interrupts
1109673 -11.3% 983929 ± 3% interrupts.CPU100.LOC:Local_timer_interrupts
933365 ± 9% -33.5% 620758 ± 11% interrupts.CPU100.RES:Rescheduling_interrupts
1109648 -11.2% 985868 ± 3% interrupts.CPU101.LOC:Local_timer_interrupts
905179 ± 4% -32.6% 610073 ± 8% interrupts.CPU101.RES:Rescheduling_interrupts
55524 ± 32% -37.9% 34488 ± 22% interrupts.CPU102.CAL:Function_call_interrupts
1109644 -11.3% 983935 ± 3% interrupts.CPU102.LOC:Local_timer_interrupts
919161 ± 4% -33.6% 610117 ± 10% interrupts.CPU102.RES:Rescheduling_interrupts
1109645 -11.3% 983934 ± 3% interrupts.CPU103.LOC:Local_timer_interrupts
913475 ± 5% -33.8% 604948 ± 10% interrupts.CPU103.RES:Rescheduling_interrupts
1109644 -11.4% 983504 ± 3% interrupts.CPU104.LOC:Local_timer_interrupts
891411 ± 4% -30.6% 618904 ± 11% interrupts.CPU104.RES:Rescheduling_interrupts
1109685 -11.3% 983840 ± 3% interrupts.CPU105.LOC:Local_timer_interrupts
916161 ± 6% -33.4% 609904 ± 12% interrupts.CPU105.RES:Rescheduling_interrupts
1109651 -11.4% 983590 ± 3% interrupts.CPU106.LOC:Local_timer_interrupts
940033 ± 4% -36.6% 596222 ± 8% interrupts.CPU106.RES:Rescheduling_interrupts
45963 ± 35% -31.8% 31357 ± 20% interrupts.CPU107.CAL:Function_call_interrupts
1109713 -11.3% 983779 ± 3% interrupts.CPU107.LOC:Local_timer_interrupts
934019 ± 10% -35.0% 606683 ± 9% interrupts.CPU107.RES:Rescheduling_interrupts
1109681 -11.2% 985738 ± 3% interrupts.CPU108.LOC:Local_timer_interrupts
907098 ± 10% -26.3% 668850 ± 4% interrupts.CPU108.RES:Rescheduling_interrupts
1109685 -11.2% 985797 ± 3% interrupts.CPU109.LOC:Local_timer_interrupts
926079 ± 10% -27.0% 675677 ± 2% interrupts.CPU109.RES:Rescheduling_interrupts
1109842 -11.5% 982553 ± 3% interrupts.CPU11.LOC:Local_timer_interrupts
777408 ± 7% -24.9% 584102 ± 6% interrupts.CPU11.RES:Rescheduling_interrupts
1109698 -11.2% 985930 ± 3% interrupts.CPU110.LOC:Local_timer_interrupts
927794 ± 6% -26.1% 685774 ± 3% interrupts.CPU110.RES:Rescheduling_interrupts
1109662 -11.1% 986214 ± 3% interrupts.CPU111.LOC:Local_timer_interrupts
910204 ± 4% -26.3% 670493 ± 3% interrupts.CPU111.RES:Rescheduling_interrupts
1109686 -11.3% 984562 ± 3% interrupts.CPU112.LOC:Local_timer_interrupts
956972 ± 6% -29.5% 674228 ± 3% interrupts.CPU112.RES:Rescheduling_interrupts
1109660 -11.1% 986024 ± 3% interrupts.CPU113.LOC:Local_timer_interrupts
911937 ± 6% -25.3% 680865 ± 3% interrupts.CPU113.RES:Rescheduling_interrupts
1109640 -11.1% 986264 ± 3% interrupts.CPU114.LOC:Local_timer_interrupts
945548 ± 10% -27.3% 687639 ± 2% interrupts.CPU114.RES:Rescheduling_interrupts
1109707 -11.1% 986350 ± 3% interrupts.CPU115.LOC:Local_timer_interrupts
965953 ± 5% -28.4% 692092 ± 4% interrupts.CPU115.RES:Rescheduling_interrupts
1109635 -11.1% 985969 ± 3% interrupts.CPU116.LOC:Local_timer_interrupts
961063 ± 9% -30.4% 669281 ± 3% interrupts.CPU116.RES:Rescheduling_interrupts
1109648 -11.1% 986070 ± 3% interrupts.CPU117.LOC:Local_timer_interrupts
953449 ± 5% -29.3% 674189 ± 4% interrupts.CPU117.RES:Rescheduling_interrupts
1109688 -11.1% 986329 ± 3% interrupts.CPU118.LOC:Local_timer_interrupts
909710 ± 9% -22.2% 707837 ± 2% interrupts.CPU118.RES:Rescheduling_interrupts
1109682 -11.1% 986282 ± 3% interrupts.CPU119.LOC:Local_timer_interrupts
951262 ± 7% -27.6% 688722 ± 4% interrupts.CPU119.RES:Rescheduling_interrupts
1109883 -11.5% 982267 ± 4% interrupts.CPU12.LOC:Local_timer_interrupts
787531 ± 3% -27.2% 573213 ± 7% interrupts.CPU12.RES:Rescheduling_interrupts
1109698 -11.1% 986222 ± 3% interrupts.CPU120.LOC:Local_timer_interrupts
943666 ± 5% -28.5% 674336 ± 4% interrupts.CPU120.RES:Rescheduling_interrupts
1109696 -11.1% 986248 ± 3% interrupts.CPU121.LOC:Local_timer_interrupts
912017 ± 7% -24.5% 688310 interrupts.CPU121.RES:Rescheduling_interrupts
1109688 -11.1% 986359 ± 3% interrupts.CPU122.LOC:Local_timer_interrupts
962703 ± 8% -29.7% 676523 ± 2% interrupts.CPU122.RES:Rescheduling_interrupts
1109653 -11.1% 986405 ± 3% interrupts.CPU123.LOC:Local_timer_interrupts
949945 ± 6% -25.6% 706314 ± 4% interrupts.CPU123.RES:Rescheduling_interrupts
1109693 -11.1% 986142 ± 3% interrupts.CPU124.LOC:Local_timer_interrupts
926407 ± 8% -25.0% 695173 ± 3% interrupts.CPU124.RES:Rescheduling_interrupts
1109690 -11.1% 986479 ± 3% interrupts.CPU125.LOC:Local_timer_interrupts
897337 ± 8% -24.5% 677912 ± 4% interrupts.CPU125.RES:Rescheduling_interrupts
1109701 -11.2% 985053 ± 3% interrupts.CPU126.LOC:Local_timer_interrupts
870200 ± 5% -27.7% 629561 ± 12% interrupts.CPU126.RES:Rescheduling_interrupts
1109638 -11.2% 985218 ± 3% interrupts.CPU127.LOC:Local_timer_interrupts
903216 ± 5% -26.2% 667011 ± 11% interrupts.CPU127.RES:Rescheduling_interrupts
1109678 -11.2% 985554 ± 3% interrupts.CPU128.LOC:Local_timer_interrupts
898010 ± 3% -27.7% 649239 ± 10% interrupts.CPU128.RES:Rescheduling_interrupts
1109640 -11.2% 985393 ± 3% interrupts.CPU129.LOC:Local_timer_interrupts
914045 ± 5% -27.6% 661981 ± 13% interrupts.CPU129.RES:Rescheduling_interrupts
1109756 -11.5% 982011 ± 4% interrupts.CPU13.LOC:Local_timer_interrupts
770920 ± 8% -26.9% 563555 ± 7% interrupts.CPU13.RES:Rescheduling_interrupts
1109642 -11.2% 985483 ± 3% interrupts.CPU130.LOC:Local_timer_interrupts
902719 ± 3% -27.4% 655719 ± 12% interrupts.CPU130.RES:Rescheduling_interrupts
1109678 -11.2% 985411 ± 3% interrupts.CPU131.LOC:Local_timer_interrupts
927667 ± 2% -29.9% 650000 ± 12% interrupts.CPU131.RES:Rescheduling_interrupts
1109655 -11.2% 985585 ± 3% interrupts.CPU132.LOC:Local_timer_interrupts
895668 ± 4% -26.7% 656131 ± 9% interrupts.CPU132.RES:Rescheduling_interrupts
1109644 -11.2% 985607 ± 3% interrupts.CPU133.LOC:Local_timer_interrupts
932719 ± 8% -28.6% 665823 ± 11% interrupts.CPU133.RES:Rescheduling_interrupts
1109654 -11.2% 985756 ± 3% interrupts.CPU134.LOC:Local_timer_interrupts
919080 ± 4% -28.9% 653480 ± 10% interrupts.CPU134.RES:Rescheduling_interrupts
1109612 -11.2% 985559 ± 3% interrupts.CPU135.LOC:Local_timer_interrupts
919061 ± 6% -29.3% 649968 ± 10% interrupts.CPU135.RES:Rescheduling_interrupts
1109671 -11.2% 985352 ± 3% interrupts.CPU136.LOC:Local_timer_interrupts
942893 ± 4% -30.7% 653873 ± 10% interrupts.CPU136.RES:Rescheduling_interrupts
1109638 -11.2% 985586 ± 3% interrupts.CPU137.LOC:Local_timer_interrupts
909483 ± 4% -26.2% 671290 ± 12% interrupts.CPU137.RES:Rescheduling_interrupts
1109667 -11.2% 985336 ± 3% interrupts.CPU138.LOC:Local_timer_interrupts
905979 -24.9% 680152 ± 12% interrupts.CPU138.RES:Rescheduling_interrupts
1109666 -11.2% 985457 ± 3% interrupts.CPU139.LOC:Local_timer_interrupts
914012 ± 2% -30.0% 639753 ± 11% interrupts.CPU139.RES:Rescheduling_interrupts
1109769 -11.5% 982477 ± 4% interrupts.CPU14.LOC:Local_timer_interrupts
786901 ± 6% -26.1% 581705 ± 5% interrupts.CPU14.RES:Rescheduling_interrupts
1109661 -11.2% 985582 ± 3% interrupts.CPU140.LOC:Local_timer_interrupts
891457 ± 4% -25.1% 667637 ± 9% interrupts.CPU140.RES:Rescheduling_interrupts
1109567 -11.2% 985643 ± 3% interrupts.CPU141.LOC:Local_timer_interrupts
920671 ± 4% -29.1% 652448 ± 10% interrupts.CPU141.RES:Rescheduling_interrupts
1109583 -11.2% 985732 ± 3% interrupts.CPU142.LOC:Local_timer_interrupts
901948 ± 3% -25.7% 670496 ± 11% interrupts.CPU142.RES:Rescheduling_interrupts
1109693 -11.2% 985489 ± 3% interrupts.CPU143.LOC:Local_timer_interrupts
897848 ± 3% -29.8% 630227 ± 11% interrupts.CPU143.RES:Rescheduling_interrupts
1109764 -11.5% 982288 ± 4% interrupts.CPU15.LOC:Local_timer_interrupts
769032 ± 4% -24.6% 579800 ± 5% interrupts.CPU15.RES:Rescheduling_interrupts
1109749 -11.5% 981954 ± 4% interrupts.CPU16.LOC:Local_timer_interrupts
753284 ± 7% -24.5% 568931 ± 7% interrupts.CPU16.RES:Rescheduling_interrupts
1109740 -11.5% 982067 ± 4% interrupts.CPU17.LOC:Local_timer_interrupts
762912 ± 3% -24.3% 577864 ± 6% interrupts.CPU17.RES:Rescheduling_interrupts
1109715 -11.3% 983766 ± 3% interrupts.CPU18.LOC:Local_timer_interrupts
862584 ± 9% -31.5% 591294 ± 8% interrupts.CPU18.RES:Rescheduling_interrupts
1109709 -11.4% 983571 ± 3% interrupts.CPU19.LOC:Local_timer_interrupts
914512 ± 5% -33.8% 605096 ± 9% interrupts.CPU19.RES:Rescheduling_interrupts
1109849 -11.5% 982602 ± 4% interrupts.CPU2.LOC:Local_timer_interrupts
763288 ± 2% -24.2% 578849 ± 8% interrupts.CPU2.RES:Rescheduling_interrupts
1109697 -11.3% 983869 ± 3% interrupts.CPU20.LOC:Local_timer_interrupts
935470 ± 7% -33.8% 619417 ± 8% interrupts.CPU20.RES:Rescheduling_interrupts
1109689 -11.3% 983811 ± 3% interrupts.CPU21.LOC:Local_timer_interrupts
953999 ± 5% -35.9% 611854 ± 9% interrupts.CPU21.RES:Rescheduling_interrupts
1109713 -11.3% 983859 ± 3% interrupts.CPU22.LOC:Local_timer_interrupts
949063 ± 4% -34.9% 617867 ± 10% interrupts.CPU22.RES:Rescheduling_interrupts
1109677 -11.3% 983782 ± 3% interrupts.CPU23.LOC:Local_timer_interrupts
916942 ± 2% -31.5% 627663 ± 9% interrupts.CPU23.RES:Rescheduling_interrupts
1109676 -11.3% 983778 ± 3% interrupts.CPU24.LOC:Local_timer_interrupts
946944 ± 7% -34.4% 621109 ± 9% interrupts.CPU24.RES:Rescheduling_interrupts
1109671 -11.3% 983788 ± 3% interrupts.CPU25.LOC:Local_timer_interrupts
927026 ± 3% -32.5% 625454 ± 10% interrupts.CPU25.RES:Rescheduling_interrupts
1109686 -11.3% 983799 ± 3% interrupts.CPU26.LOC:Local_timer_interrupts
925930 ± 4% -31.8% 631366 ± 9% interrupts.CPU26.RES:Rescheduling_interrupts
1109675 -11.3% 983850 ± 3% interrupts.CPU27.LOC:Local_timer_interrupts
930402 ± 7% -32.6% 627189 ± 8% interrupts.CPU27.RES:Rescheduling_interrupts
1109729 -11.4% 983703 ± 3% interrupts.CPU28.LOC:Local_timer_interrupts
928439 ± 4% -32.0% 630948 ± 7% interrupts.CPU28.RES:Rescheduling_interrupts
1109731 -11.9% 977654 ± 4% interrupts.CPU29.LOC:Local_timer_interrupts
917894 ± 3% -31.6% 627431 ± 7% interrupts.CPU29.RES:Rescheduling_interrupts
1109813 -11.5% 982425 ± 4% interrupts.CPU3.LOC:Local_timer_interrupts
763728 ± 4% -25.8% 566752 ± 7% interrupts.CPU3.RES:Rescheduling_interrupts
1109702 -11.4% 983710 ± 3% interrupts.CPU30.LOC:Local_timer_interrupts
959814 ± 6% -34.4% 630064 ± 11% interrupts.CPU30.RES:Rescheduling_interrupts
1109691 -11.3% 983822 ± 3% interrupts.CPU31.LOC:Local_timer_interrupts
953721 ± 7% -34.2% 627837 ± 12% interrupts.CPU31.RES:Rescheduling_interrupts
1109716 -11.4% 983559 ± 3% interrupts.CPU32.LOC:Local_timer_interrupts
929251 ± 5% -32.4% 628141 ± 10% interrupts.CPU32.RES:Rescheduling_interrupts
1109720 -11.4% 983527 ± 3% interrupts.CPU33.LOC:Local_timer_interrupts
955082 ± 8% -34.0% 629974 ± 9% interrupts.CPU33.RES:Rescheduling_interrupts
1109706 -11.4% 983432 ± 3% interrupts.CPU34.LOC:Local_timer_interrupts
933432 ± 6% -33.6% 619604 ± 9% interrupts.CPU34.RES:Rescheduling_interrupts
1109724 -11.4% 983426 ± 3% interrupts.CPU35.LOC:Local_timer_interrupts
904660 ± 3% -31.4% 620967 ± 9% interrupts.CPU35.RES:Rescheduling_interrupts
1109716 -11.1% 986109 ± 3% interrupts.CPU36.LOC:Local_timer_interrupts
908916 ± 7% -25.5% 677428 ± 2% interrupts.CPU36.RES:Rescheduling_interrupts
1109635 -11.1% 986076 ± 3% interrupts.CPU37.LOC:Local_timer_interrupts
926313 ± 8% -24.1% 702863 ± 3% interrupts.CPU37.RES:Rescheduling_interrupts
1109685 -11.1% 986009 ± 3% interrupts.CPU38.LOC:Local_timer_interrupts
951056 ± 6% -27.3% 691529 ± 4% interrupts.CPU38.RES:Rescheduling_interrupts
1109743 -11.1% 986389 ± 3% interrupts.CPU39.LOC:Local_timer_interrupts
927813 ± 7% -24.0% 705228 ± 4% interrupts.CPU39.RES:Rescheduling_interrupts
1109737 -11.5% 982501 ± 3% interrupts.CPU4.LOC:Local_timer_interrupts
766534 ± 8% -24.7% 577507 ± 6% interrupts.CPU4.RES:Rescheduling_interrupts
1109783 -11.1% 986790 ± 3% interrupts.CPU40.LOC:Local_timer_interrupts
929300 ± 3% -24.5% 702028 ± 3% interrupts.CPU40.RES:Rescheduling_interrupts
1109714 -11.2% 985893 ± 3% interrupts.CPU41.LOC:Local_timer_interrupts
952762 ± 9% -26.6% 698904 ± 3% interrupts.CPU41.RES:Rescheduling_interrupts
1109708 -11.1% 986368 ± 3% interrupts.CPU42.LOC:Local_timer_interrupts
961704 ± 6% -26.4% 707713 ± 3% interrupts.CPU42.RES:Rescheduling_interrupts
1109757 -11.1% 986228 ± 3% interrupts.CPU43.LOC:Local_timer_interrupts
952246 ± 6% -26.2% 702405 ± 2% interrupts.CPU43.RES:Rescheduling_interrupts
1109751 -11.1% 986140 ± 3% interrupts.CPU44.LOC:Local_timer_interrupts
980627 ± 2% -29.8% 688585 ± 4% interrupts.CPU44.RES:Rescheduling_interrupts
1109689 -11.1% 985981 ± 3% interrupts.CPU45.LOC:Local_timer_interrupts
976440 ± 3% -29.2% 691802 ± 3% interrupts.CPU45.RES:Rescheduling_interrupts
1109645 -11.1% 986013 ± 3% interrupts.CPU46.LOC:Local_timer_interrupts
952352 ± 2% -26.7% 697811 ± 4% interrupts.CPU46.RES:Rescheduling_interrupts
1109682 -11.1% 986172 ± 3% interrupts.CPU47.LOC:Local_timer_interrupts
957266 ± 4% -28.5% 684262 interrupts.CPU47.RES:Rescheduling_interrupts
49803 ± 13% -18.0% 40844 ± 12% interrupts.CPU48.CAL:Function_call_interrupts
1109702 -11.1% 986471 ± 3% interrupts.CPU48.LOC:Local_timer_interrupts
977427 ± 6% -27.8% 705625 ± 3% interrupts.CPU48.RES:Rescheduling_interrupts
1109658 -11.1% 986172 ± 3% interrupts.CPU49.LOC:Local_timer_interrupts
937091 ± 7% -26.2% 691409 ± 3% interrupts.CPU49.RES:Rescheduling_interrupts
1109735 -11.5% 982317 ± 4% interrupts.CPU5.LOC:Local_timer_interrupts
741049 ± 6% -23.6% 566103 ± 6% interrupts.CPU5.RES:Rescheduling_interrupts
1109692 -11.1% 986149 ± 3% interrupts.CPU50.LOC:Local_timer_interrupts
942242 ± 7% -26.2% 695845 ± 3% interrupts.CPU50.RES:Rescheduling_interrupts
1109688 -11.1% 986153 ± 3% interrupts.CPU51.LOC:Local_timer_interrupts
978237 ± 5% -27.8% 706561 ± 2% interrupts.CPU51.RES:Rescheduling_interrupts
1109680 -11.1% 986101 ± 3% interrupts.CPU52.LOC:Local_timer_interrupts
942523 ± 6% -25.9% 698844 ± 2% interrupts.CPU52.RES:Rescheduling_interrupts
1109709 -11.1% 986142 ± 3% interrupts.CPU53.LOC:Local_timer_interrupts
968770 ± 8% -28.5% 692330 ± 4% interrupts.CPU53.RES:Rescheduling_interrupts
1109706 -11.2% 985372 ± 3% interrupts.CPU54.LOC:Local_timer_interrupts
892744 ± 3% -27.4% 648201 ± 10% interrupts.CPU54.RES:Rescheduling_interrupts
1109711 -11.2% 985410 ± 3% interrupts.CPU55.LOC:Local_timer_interrupts
905314 ± 4% -26.4% 666564 ± 10% interrupts.CPU55.RES:Rescheduling_interrupts
1109760 -11.2% 985583 ± 3% interrupts.CPU56.LOC:Local_timer_interrupts
923287 ± 5% -27.0% 673797 ± 10% interrupts.CPU56.RES:Rescheduling_interrupts
1109750 -11.2% 985363 ± 3% interrupts.CPU57.LOC:Local_timer_interrupts
941749 ± 6% -27.9% 678794 ± 9% interrupts.CPU57.RES:Rescheduling_interrupts
1109711 -11.2% 985560 ± 3% interrupts.CPU58.LOC:Local_timer_interrupts
911298 ± 3% -27.4% 661172 ± 10% interrupts.CPU58.RES:Rescheduling_interrupts
1109694 -11.2% 985480 ± 3% interrupts.CPU59.LOC:Local_timer_interrupts
934580 ± 3% -29.9% 654916 ± 9% interrupts.CPU59.RES:Rescheduling_interrupts
1109735 -11.5% 982399 ± 4% interrupts.CPU6.LOC:Local_timer_interrupts
740135 ± 6% -24.3% 560193 ± 5% interrupts.CPU6.RES:Rescheduling_interrupts
1109690 -11.2% 985677 ± 3% interrupts.CPU60.LOC:Local_timer_interrupts
927684 ± 3% -28.1% 667287 ± 11% interrupts.CPU60.RES:Rescheduling_interrupts
1109694 -11.2% 985524 ± 3% interrupts.CPU61.LOC:Local_timer_interrupts
958054 ± 4% -30.3% 667940 ± 11% interrupts.CPU61.RES:Rescheduling_interrupts
1109679 -11.2% 985364 ± 3% interrupts.CPU62.LOC:Local_timer_interrupts
946262 ± 4% -30.0% 662460 ± 11% interrupts.CPU62.RES:Rescheduling_interrupts
1109678 -11.2% 985301 ± 3% interrupts.CPU63.LOC:Local_timer_interrupts
931012 ± 4% -26.6% 683774 ± 11% interrupts.CPU63.RES:Rescheduling_interrupts
1109691 -11.2% 985194 ± 3% interrupts.CPU64.LOC:Local_timer_interrupts
929091 ± 6% -29.7% 653309 ± 11% interrupts.CPU64.RES:Rescheduling_interrupts
1109744 -11.2% 985320 ± 3% interrupts.CPU65.LOC:Local_timer_interrupts
933881 ± 3% -27.6% 676297 ± 9% interrupts.CPU65.RES:Rescheduling_interrupts
1109710 -11.2% 985673 ± 3% interrupts.CPU66.LOC:Local_timer_interrupts
956361 ± 5% -29.2% 676634 ± 9% interrupts.CPU66.RES:Rescheduling_interrupts
1109687 -11.2% 985315 ± 3% interrupts.CPU67.LOC:Local_timer_interrupts
909257 ± 3% -26.6% 667439 ± 12% interrupts.CPU67.RES:Rescheduling_interrupts
1109693 -11.2% 985287 ± 3% interrupts.CPU68.LOC:Local_timer_interrupts
925288 ± 2% -28.0% 665809 ± 9% interrupts.CPU68.RES:Rescheduling_interrupts
1109675 -11.2% 985486 ± 3% interrupts.CPU69.LOC:Local_timer_interrupts
958528 ± 4% -29.0% 680393 ± 9% interrupts.CPU69.RES:Rescheduling_interrupts
31192 ± 5% -17.8% 25645 ± 15% interrupts.CPU7.CAL:Function_call_interrupts
1109763 -11.4% 982851 ± 3% interrupts.CPU7.LOC:Local_timer_interrupts
766751 ± 7% -25.8% 569219 ± 4% interrupts.CPU7.RES:Rescheduling_interrupts
1109572 -11.2% 985353 ± 3% interrupts.CPU70.LOC:Local_timer_interrupts
934309 ± 2% -28.6% 666929 ± 11% interrupts.CPU70.RES:Rescheduling_interrupts
1109770 -11.2% 985376 ± 3% interrupts.CPU71.LOC:Local_timer_interrupts
923580 ± 5% -26.7% 676802 ± 9% interrupts.CPU71.RES:Rescheduling_interrupts
48586 ± 18% -31.1% 33495 ± 20% interrupts.CPU72.CAL:Function_call_interrupts
1109690 -11.4% 983437 ± 3% interrupts.CPU72.LOC:Local_timer_interrupts
732823 ± 6% -25.1% 548780 ± 6% interrupts.CPU72.RES:Rescheduling_interrupts
1109780 -11.5% 982255 ± 4% interrupts.CPU73.LOC:Local_timer_interrupts
748198 ± 6% -25.7% 555808 ± 2% interrupts.CPU73.RES:Rescheduling_interrupts
1109731 -11.5% 982361 ± 4% interrupts.CPU74.LOC:Local_timer_interrupts
732000 ± 5% -24.4% 553248 ± 8% interrupts.CPU74.RES:Rescheduling_interrupts
1109689 -11.5% 982406 ± 4% interrupts.CPU75.LOC:Local_timer_interrupts
732437 ± 4% -23.9% 557346 ± 5% interrupts.CPU75.RES:Rescheduling_interrupts
1109783 -11.5% 982253 ± 4% interrupts.CPU76.LOC:Local_timer_interrupts
727115 -22.8% 561035 ± 6% interrupts.CPU76.RES:Rescheduling_interrupts
1109749 -11.5% 981909 ± 4% interrupts.CPU77.LOC:Local_timer_interrupts
722320 ± 5% -23.9% 549545 ± 7% interrupts.CPU77.RES:Rescheduling_interrupts
1109776 -11.5% 982201 ± 4% interrupts.CPU78.LOC:Local_timer_interrupts
774003 ± 8% -27.3% 562990 ± 5% interrupts.CPU78.RES:Rescheduling_interrupts
1109771 -11.5% 982534 ± 3% interrupts.CPU79.LOC:Local_timer_interrupts
725034 ± 4% -21.5% 569192 ± 5% interrupts.CPU79.RES:Rescheduling_interrupts
1109745 -11.5% 982390 ± 4% interrupts.CPU8.LOC:Local_timer_interrupts
766391 ± 6% -26.4% 564224 ± 5% interrupts.CPU8.RES:Rescheduling_interrupts
1109765 -11.5% 982241 ± 4% interrupts.CPU80.LOC:Local_timer_interrupts
747959 ± 5% -24.2% 566670 ± 8% interrupts.CPU80.RES:Rescheduling_interrupts
1109814 -11.5% 981873 ± 4% interrupts.CPU81.LOC:Local_timer_interrupts
747605 ± 7% -24.3% 565728 ± 7% interrupts.CPU81.RES:Rescheduling_interrupts
1109714 -11.5% 982405 ± 4% interrupts.CPU82.LOC:Local_timer_interrupts
740632 ± 4% -25.4% 552433 ± 7% interrupts.CPU82.RES:Rescheduling_interrupts
1109762 -11.5% 982189 ± 4% interrupts.CPU83.LOC:Local_timer_interrupts
752433 ± 6% -23.9% 572430 ± 6% interrupts.CPU83.RES:Rescheduling_interrupts
1109801 -11.5% 982438 ± 4% interrupts.CPU84.LOC:Local_timer_interrupts
750543 ± 6% -25.7% 557447 ± 7% interrupts.CPU84.RES:Rescheduling_interrupts
1109757 -11.5% 982181 ± 4% interrupts.CPU85.LOC:Local_timer_interrupts
710008 ± 5% -20.3% 565565 ± 7% interrupts.CPU85.RES:Rescheduling_interrupts
1109819 -11.5% 982651 ± 3% interrupts.CPU86.LOC:Local_timer_interrupts
756402 ± 4% -26.1% 559174 ± 6% interrupts.CPU86.RES:Rescheduling_interrupts
1109763 -11.5% 982634 ± 3% interrupts.CPU87.LOC:Local_timer_interrupts
763567 ± 5% -27.5% 553678 ± 5% interrupts.CPU87.RES:Rescheduling_interrupts
1109795 -11.4% 982983 ± 3% interrupts.CPU88.LOC:Local_timer_interrupts
736222 ± 4% -24.9% 552717 ± 6% interrupts.CPU88.RES:Rescheduling_interrupts
1109788 -11.4% 982800 ± 3% interrupts.CPU89.LOC:Local_timer_interrupts
729712 ± 5% -24.6% 550308 ± 6% interrupts.CPU89.RES:Rescheduling_interrupts
1109951 -11.5% 982025 ± 4% interrupts.CPU9.LOC:Local_timer_interrupts
749632 ± 7% -25.7% 556858 ± 7% interrupts.CPU9.RES:Rescheduling_interrupts
1109721 -11.4% 983595 ± 3% interrupts.CPU90.LOC:Local_timer_interrupts
889032 ± 8% -34.4% 582776 ± 9% interrupts.CPU90.RES:Rescheduling_interrupts
1109668 -11.4% 983263 ± 3% interrupts.CPU91.LOC:Local_timer_interrupts
914855 ± 5% -35.1% 593653 ± 8% interrupts.CPU91.RES:Rescheduling_interrupts
1109647 -11.3% 983819 ± 3% interrupts.CPU92.LOC:Local_timer_interrupts
948214 ± 8% -35.0% 616661 ± 10% interrupts.CPU92.RES:Rescheduling_interrupts
1109694 -11.4% 983708 ± 3% interrupts.CPU93.LOC:Local_timer_interrupts
892302 ± 7% -32.9% 598886 ± 9% interrupts.CPU93.RES:Rescheduling_interrupts
1109699 -11.4% 983475 ± 3% interrupts.CPU94.LOC:Local_timer_interrupts
913830 ± 4% -33.8% 605237 ± 10% interrupts.CPU94.RES:Rescheduling_interrupts
1109687 -11.4% 983654 ± 3% interrupts.CPU95.LOC:Local_timer_interrupts
942816 ± 8% -35.3% 610115 ± 9% interrupts.CPU95.RES:Rescheduling_interrupts
1109662 -11.3% 983930 ± 3% interrupts.CPU96.LOC:Local_timer_interrupts
921056 ± 7% -33.8% 609373 ± 10% interrupts.CPU96.RES:Rescheduling_interrupts
1109663 -11.3% 983971 ± 3% interrupts.CPU97.LOC:Local_timer_interrupts
939132 ± 5% -35.0% 610299 ± 10% interrupts.CPU97.RES:Rescheduling_interrupts
1109659 -11.3% 983854 ± 3% interrupts.CPU98.LOC:Local_timer_interrupts
905810 ± 6% -32.7% 609848 ± 9% interrupts.CPU98.RES:Rescheduling_interrupts
1109697 -11.3% 983762 ± 3% interrupts.CPU99.LOC:Local_timer_interrupts
936646 ± 7% -34.8% 610801 ± 10% interrupts.CPU99.RES:Rescheduling_interrupts
1.598e+08 -11.3% 1.417e+08 ± 3% interrupts.LOC:Local_timer_interrupts
1.274e+08 ± 2% -28.5% 91108791 interrupts.RES:Rescheduling_interrupts
21.07 ± 13% -14.9 6.16 ± 35% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event
20.01 ± 13% -14.3 5.71 ± 38% perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow
19.91 ± 13% -14.2 5.68 ± 38% perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow
11.18 ± 11% -11.2 0.00 perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve
17.91 ± 12% -11.0 6.86 ± 22% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
17.29 ± 12% -10.9 6.42 ± 23% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg.sock_sendmsg
17.15 ± 12% -10.8 6.31 ± 23% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg
17.06 ± 12% -10.8 6.26 ± 23% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable
18.74 ± 11% -10.4 8.30 ± 17% perf-profile.calltrace.cycles-pp.sock_def_readable.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
12.96 ± 13% -9.5 3.45 ± 49% perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
15.35 ± 11% -9.5 5.88 ± 22% perf-profile.calltrace.cycles-pp.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
15.28 ± 11% -9.4 5.84 ± 22% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
15.16 ± 11% -9.4 5.77 ± 23% perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg
10.21 ± 12% -7.0 3.21 ± 32% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr
10.16 ± 12% -7.0 3.19 ± 32% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime
5.82 ± 26% -5.8 0.00 perf-profile.calltrace.cycles-pp.asm_exc_page_fault.__get_user_nocheck_8.perf_callchain_user.get_perf_callchain.perf_callchain
5.63 ± 11% -5.6 0.00 perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc_node.__alloc_skb
6.42 ± 13% -5.6 0.81 ±114% perf-profile.calltrace.cycles-pp.perf_callchain_user.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
6.28 ± 14% -4.8 1.48 ± 46% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule
6.27 ± 14% -4.8 1.47 ± 46% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule
5.06 ± 17% -4.8 0.28 ±223% perf-profile.calltrace.cycles-pp.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
6.24 ± 14% -4.8 1.46 ± 46% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch
6.92 ± 13% -4.5 2.40 ± 24% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
4.80 ± 13% -4.5 0.31 ±223% perf-profile.calltrace.cycles-pp.unwind_next_frame.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
6.87 ± 13% -4.5 2.38 ± 24% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
52.74 -4.5 48.28 perf-profile.calltrace.cycles-pp.__libc_write
52.06 -4.3 47.73 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write
6.76 ± 11% -4.3 2.44 ± 26% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.schedule_timeout.unix_stream_read_generic
51.93 -4.3 47.62 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
6.17 ± 13% -4.1 2.04 ± 27% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
6.08 ± 11% -4.0 2.11 ± 28% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.schedule_timeout
5.56 ± 13% -3.8 1.73 ± 30% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up
5.78 ± 11% -3.8 1.95 ± 30% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
5.51 ± 11% -3.7 1.83 ± 31% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule
5.25 ± 13% -3.7 1.59 ± 32% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate
5.45 ± 12% -3.6 1.83 ± 30% perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
5.36 ± 11% -3.6 1.77 ± 32% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair
5.08 ± 13% -3.6 1.52 ± 33% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.enqueue_entity.enqueue_task_fair
5.31 ± 12% -3.6 1.76 ± 31% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common
5.25 ± 11% -3.5 1.73 ± 31% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity
4.98 ± 13% -3.5 1.49 ± 33% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.enqueue_entity
5.10 ± 12% -3.4 1.68 ± 31% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function
5.09 ± 12% -3.4 1.67 ± 31% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up
5.07 ± 12% -3.4 1.66 ± 32% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template
4.63 ± 13% -3.2 1.45 ± 33% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.schedule_timeout.unix_stream_read_generic
4.45 ± 13% -3.1 1.38 ± 33% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.schedule_timeout
4.07 ± 12% -2.9 1.15 ± 27% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
4.01 ± 12% -2.9 1.11 ± 28% perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
3.78 ± 12% -2.8 0.99 ± 30% perf-profile.calltrace.cycles-pp.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.74 ± 12% -2.8 0.98 ± 30% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
2.40 ± 12% -1.5 0.90 ± 18% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
2.09 ± 13% -1.4 0.66 ± 46% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common
47.70 -1.3 46.38 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
47.45 -1.3 46.14 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
46.94 -1.3 45.63 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
46.52 -1.2 45.29 perf-profile.calltrace.cycles-pp.sock_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
46.20 -1.1 45.08 perf-profile.calltrace.cycles-pp.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write.ksys_write
45.87 -1.1 44.76 perf-profile.calltrace.cycles-pp.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write
1.33 ± 3% -0.6 0.78 ± 6% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.schedule_timeout.unix_stream_read_generic
0.62 ± 2% +0.2 0.78 perf-profile.calltrace.cycles-pp.__check_object_size.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
0.65 ± 2% +0.2 0.81 perf-profile.calltrace.cycles-pp.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic
0.75 ± 2% +0.2 0.92 ± 4% perf-profile.calltrace.cycles-pp.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
0.62 ± 3% +0.3 0.90 ± 3% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
0.65 ± 2% +0.3 0.94 ± 3% perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
0.81 ± 2% +0.3 1.12 ± 3% perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic
0.58 ± 7% +0.4 0.96 ± 4% perf-profile.calltrace.cycles-pp.__slab_free.skb_release_data.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
0.18 ±141% +0.4 0.57 ± 6% perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
1.06 ± 6% +0.4 1.47 ± 5% perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
0.73 ± 7% +0.4 1.18 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.skb_queue_tail.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
0.75 ± 7% +0.4 1.20 ± 7% perf-profile.calltrace.cycles-pp.skb_queue_tail.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
0.60 ± 12% +0.5 1.06 ± 8% perf-profile.calltrace.cycles-pp.skb_unlink.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
0.36 ± 71% +0.5 0.83 ± 9% perf-profile.calltrace.cycles-pp.get_mem_cgroup_from_objcg.obj_cgroup_charge_pages.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve
1.52 ± 2% +0.5 1.99 ± 2% perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
1.50 ± 2% +0.5 1.97 ± 2% perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg
0.37 ± 71% +0.5 0.84 ± 8% perf-profile.calltrace.cycles-pp.get_mem_cgroup_from_objcg.obj_cgroup_uncharge_pages.kfree.skb_release_data.consume_skb
1.53 +0.5 2.01 ± 2% perf-profile.calltrace.cycles-pp.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
0.09 ±223% +0.5 0.58 ± 6% perf-profile.calltrace.cycles-pp.mod_objcg_state.kfree.skb_release_data.consume_skb.unix_stream_read_generic
0.65 ± 5% +0.5 1.15 ± 6% perf-profile.calltrace.cycles-pp.skb_set_owner_w.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
0.08 ±223% +0.5 0.59 ± 5% perf-profile.calltrace.cycles-pp.mutex_lock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
0.41 ± 71% +0.6 1.00 ± 7% perf-profile.calltrace.cycles-pp.mutex_unlock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
0.41 ± 71% +0.6 1.02 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.skb_unlink.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.00 +0.7 0.67 ± 4% perf-profile.calltrace.cycles-pp.unix_write_space.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all
1.12 ± 7% +0.7 1.81 ± 5% perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
0.88 ± 10% +0.7 1.57 ± 5% perf-profile.calltrace.cycles-pp.__slab_free.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.96 ± 13% +0.7 1.69 ± 7% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.unix_stream_read_generic
1.16 ± 9% +0.8 1.99 ± 4% perf-profile.calltrace.cycles-pp.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb
1.20 ± 8% +0.8 2.03 ± 4% perf-profile.calltrace.cycles-pp.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic
1.22 ± 8% +0.8 2.06 ± 4% perf-profile.calltrace.cycles-pp.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
1.23 ± 8% +0.8 2.07 ± 4% perf-profile.calltrace.cycles-pp.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
1.97 ± 12% +1.4 3.41 ± 7% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_uncharge.obj_cgroup_uncharge_pages.kfree.skb_release_data
0.00 +1.7 1.72 ± 7% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc_node
3.75 ± 12% +1.8 5.55 ± 6% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.unix_stream_read_generic
6.27 ± 10% +2.2 8.45 ± 5% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
6.32 ± 10% +2.2 8.50 ± 5% perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
7.85 ± 8% +2.4 10.30 ± 4% perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
4.76 ± 12% +2.6 7.32 ± 5% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg
5.38 ± 11% +3.0 8.34 ± 5% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
7.80 ± 11% +3.3 11.14 ± 5% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kfree.skb_release_data
0.00 +3.4 3.40 ± 6% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_charge.obj_cgroup_charge_pages.obj_cgroup_charge.__kmalloc_node_track_caller
46.87 +3.5 50.41 perf-profile.calltrace.cycles-pp.__libc_read
45.87 +3.7 49.56 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_read
45.74 +3.7 49.46 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
44.83 +3.9 48.69 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
45.02 +3.9 48.89 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
43.59 +3.9 47.47 perf-profile.calltrace.cycles-pp.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read
43.68 +3.9 47.56 perf-profile.calltrace.cycles-pp.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read
44.11 +3.9 48.00 perf-profile.calltrace.cycles-pp.sock_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64
44.32 +3.9 48.22 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
7.54 ± 9% +4.1 11.64 ± 4% perf-profile.calltrace.cycles-pp.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
12.45 ± 10% +4.3 16.71 ± 5% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
12.50 ± 10% +4.3 16.77 ± 5% perf-profile.calltrace.cycles-pp.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
14.17 ± 9% +4.5 18.70 ± 4% perf-profile.calltrace.cycles-pp.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
14.23 ± 9% +4.5 18.75 ± 4% perf-profile.calltrace.cycles-pp.kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
9.88 ± 11% +4.8 14.72 ± 5% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kfree.skb_release_data.consume_skb
11.16 ± 11% +5.6 16.78 ± 4% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kfree.skb_release_data.consume_skb.unix_stream_read_generic
12.06 ± 9% +5.9 17.92 ± 4% perf-profile.calltrace.cycles-pp.kfree.skb_release_data.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
13.11 ± 9% +6.5 19.61 ± 4% perf-profile.calltrace.cycles-pp.skb_release_data.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
22.76 ± 8% +7.1 29.89 ± 4% perf-profile.calltrace.cycles-pp.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
22.70 ± 8% +7.1 29.82 ± 4% perf-profile.calltrace.cycles-pp.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
0.00 +7.4 7.38 ± 6% perf-profile.calltrace.cycles-pp.page_counter_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc_node.__alloc_skb
14.56 ± 9% +7.5 22.04 ± 4% perf-profile.calltrace.cycles-pp.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
23.80 ± 8% +7.9 31.75 ± 4% perf-profile.calltrace.cycles-pp.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
0.00 +14.6 14.60 ± 5% perf-profile.calltrace.cycles-pp.page_counter_charge.obj_cgroup_charge_pages.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve
17.18 ± 11% -16.6 0.59 ± 7% perf-profile.children.cycles-pp.page_counter_try_charge
23.30 ± 12% -15.5 7.83 ± 31% perf-profile.children.cycles-pp.perf_tp_event
22.67 ± 12% -15.1 7.58 ± 31% perf-profile.children.cycles-pp.perf_swevent_overflow
22.63 ± 12% -15.1 7.56 ± 31% perf-profile.children.cycles-pp.__perf_event_overflow
22.53 ± 12% -15.0 7.52 ± 31% perf-profile.children.cycles-pp.perf_event_output_forward
22.12 ± 12% -14.8 7.34 ± 31% perf-profile.children.cycles-pp.perf_prepare_sample
21.00 ± 12% -14.1 6.88 ± 32% perf-profile.children.cycles-pp.perf_callchain
20.89 ± 12% -14.0 6.85 ± 32% perf-profile.children.cycles-pp.get_perf_callchain
19.54 ± 10% -12.0 7.51 ± 22% perf-profile.children.cycles-pp.schedule
19.55 ± 10% -11.9 7.61 ± 21% perf-profile.children.cycles-pp.__schedule
18.11 ± 12% -10.8 7.26 ± 21% perf-profile.children.cycles-pp.__wake_up_common_lock
17.47 ± 12% -10.7 6.81 ± 21% perf-profile.children.cycles-pp.__wake_up_common
17.34 ± 12% -10.6 6.70 ± 22% perf-profile.children.cycles-pp.autoremove_wake_function
17.24 ± 12% -10.6 6.64 ± 22% perf-profile.children.cycles-pp.try_to_wake_up
18.74 ± 11% -10.4 8.30 ± 17% perf-profile.children.cycles-pp.sock_def_readable
13.76 ± 13% -9.3 4.50 ± 32% perf-profile.children.cycles-pp.perf_callchain_kernel
15.56 ± 10% -9.2 6.32 ± 21% perf-profile.children.cycles-pp.schedule_timeout
12.23 ± 12% -7.8 4.42 ± 28% perf-profile.children.cycles-pp.update_curr
11.26 ± 12% -7.3 3.96 ± 30% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
10.42 ± 13% -7.0 3.38 ± 32% perf-profile.children.cycles-pp.unwind_next_frame
6.96 ± 13% -4.8 2.12 ± 34% perf-profile.children.cycles-pp.perf_trace_sched_switch
6.72 ± 13% -4.5 2.22 ± 31% perf-profile.children.cycles-pp.perf_callchain_user
7.06 ± 13% -4.5 2.60 ± 24% perf-profile.children.cycles-pp.ttwu_do_activate
52.87 -4.4 48.42 perf-profile.children.cycles-pp.__libc_write
7.00 ± 13% -4.4 2.58 ± 24% perf-profile.children.cycles-pp.enqueue_task_fair
6.37 ± 12% -4.3 2.11 ± 31% perf-profile.children.cycles-pp.__get_user_nocheck_8
6.86 ± 11% -4.2 2.63 ± 24% perf-profile.children.cycles-pp.dequeue_task_fair
6.30 ± 13% -4.1 2.21 ± 26% perf-profile.children.cycles-pp.enqueue_entity
6.17 ± 11% -3.9 2.28 ± 27% perf-profile.children.cycles-pp.dequeue_entity
5.50 ± 12% -3.7 1.76 ± 32% perf-profile.children.cycles-pp.__unwind_start
5.52 ± 11% -3.6 1.94 ± 29% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
4.70 ± 11% -3.1 1.64 ± 19% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
4.61 ± 11% -3.0 1.56 ± 20% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
4.40 ± 13% -2.5 1.87 ± 28% perf-profile.children.cycles-pp.asm_exc_page_fault
2.99 ± 13% -2.0 1.00 ± 31% perf-profile.children.cycles-pp.__orc_find
3.90 ± 10% -1.5 2.38 ± 21% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
47.70 -1.3 46.38 perf-profile.children.cycles-pp.ksys_write
47.46 -1.3 46.14 perf-profile.children.cycles-pp.vfs_write
46.95 -1.3 45.64 perf-profile.children.cycles-pp.new_sync_write
46.55 -1.2 45.36 perf-profile.children.cycles-pp.sock_write_iter
1.67 ± 13% -1.1 0.55 ± 32% perf-profile.children.cycles-pp.orc_find
46.21 -1.1 45.09 perf-profile.children.cycles-pp.sock_sendmsg
45.90 -1.1 44.79 perf-profile.children.cycles-pp.unix_stream_sendmsg
1.66 ± 12% -1.1 0.57 ± 31% perf-profile.children.cycles-pp.unwind_get_return_address
5.94 ± 5% -1.1 4.86 ± 3% perf-profile.children.cycles-pp._raw_spin_lock
2.00 ± 3% -0.9 1.12 ± 6% perf-profile.children.cycles-pp.switch_mm_irqs_off
1.30 ± 12% -0.8 0.46 ± 30% perf-profile.children.cycles-pp.__kernel_text_address
1.20 ± 10% -0.8 0.44 ± 28% perf-profile.children.cycles-pp.native_irq_return_iret
1.09 ± 13% -0.8 0.34 ± 31% perf-profile.children.cycles-pp.kernelmode_fixup_or_oops
1.05 ± 14% -0.7 0.32 ± 34% perf-profile.children.cycles-pp.stack_access_ok
1.01 ± 13% -0.7 0.31 ± 31% perf-profile.children.cycles-pp.fixup_exception
1.00 ± 12% -0.7 0.34 ± 30% perf-profile.children.cycles-pp.kernel_text_address
0.91 ± 13% -0.6 0.28 ± 32% perf-profile.children.cycles-pp.search_exception_tables
0.88 ± 13% -0.6 0.27 ± 32% perf-profile.children.cycles-pp.search_extable
0.84 ± 14% -0.6 0.26 ± 31% perf-profile.children.cycles-pp.bsearch
1.44 ± 14% -0.5 0.90 ± 29% perf-profile.children.cycles-pp.exc_page_fault
97.98 -0.5 97.46 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
0.95 ± 14% -0.5 0.46 ± 13% perf-profile.children.cycles-pp.select_task_rq_fair
97.74 -0.5 97.26 perf-profile.children.cycles-pp.do_syscall_64
0.82 ± 11% -0.4 0.42 ± 8% perf-profile.children.cycles-pp.pick_next_task_fair
0.70 ± 14% -0.4 0.31 ± 15% perf-profile.children.cycles-pp.select_idle_sibling
0.65 ± 9% -0.4 0.27 ± 28% perf-profile.children.cycles-pp.__perf_event_header__init_id
0.87 ± 12% -0.4 0.51 ± 9% perf-profile.children.cycles-pp.update_load_avg
0.51 ± 14% -0.4 0.16 ± 31% perf-profile.children.cycles-pp.cmp_ex_search
0.46 ± 11% -0.3 0.21 ± 13% perf-profile.children.cycles-pp.reweight_entity
0.44 ± 9% -0.3 0.19 ± 10% perf-profile.children.cycles-pp.load_new_mm_cr3
0.41 ± 15% -0.2 0.18 ± 16% perf-profile.children.cycles-pp.select_idle_cpu
0.46 ± 8% -0.2 0.24 ± 7% perf-profile.children.cycles-pp.switch_fpu_return
0.37 ± 8% -0.2 0.15 ± 26% perf-profile.children.cycles-pp.perf_event_pid_type
0.45 ± 3% -0.2 0.25 ± 6% perf-profile.children.cycles-pp.__switch_to
0.31 ± 11% -0.2 0.11 ± 32% perf-profile.children.cycles-pp.ftrace_graph_ret_addr
0.31 ± 12% -0.2 0.12 ± 30% perf-profile.children.cycles-pp.perf_output_begin_forward
0.31 ± 9% -0.2 0.13 ± 17% perf-profile.children.cycles-pp.sched_clock_cpu
0.30 ± 8% -0.2 0.12 ± 28% perf-profile.children.cycles-pp.__task_pid_nr_ns
0.32 ± 16% -0.2 0.15 ± 31% perf-profile.children.cycles-pp.perf_trace_sched_migrate_task
0.37 ± 5% -0.2 0.21 ± 7% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.37 ± 16% -0.2 0.21 ± 24% perf-profile.children.cycles-pp.set_task_cpu
0.27 ± 10% -0.2 0.11 ± 18% perf-profile.children.cycles-pp.native_sched_clock
0.27 ± 17% -0.2 0.11 ± 22% perf-profile.children.cycles-pp.available_idle_cpu
0.30 ± 9% -0.2 0.15 ± 10% perf-profile.children.cycles-pp.set_next_entity
0.35 ± 5% -0.2 0.20 ± 7% perf-profile.children.cycles-pp.check_preempt_curr
0.23 ± 13% -0.2 0.08 ± 48% perf-profile.children.cycles-pp.bad_get_user
0.29 ± 10% -0.1 0.14 ± 9% perf-profile.children.cycles-pp.__switch_to_asm
0.32 ± 9% -0.1 0.18 ± 14% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.66 ± 5% -0.1 0.52 ± 6% perf-profile.children.cycles-pp.prepare_to_wait
0.30 ± 4% -0.1 0.16 ± 9% perf-profile.children.cycles-pp.check_preempt_wakeup
0.20 ± 11% -0.1 0.06 ± 47% perf-profile.children.cycles-pp.error_entry
0.26 ± 11% -0.1 0.12 ± 10% perf-profile.children.cycles-pp.update_rq_clock
0.21 ± 16% -0.1 0.08 ± 8% perf-profile.children.cycles-pp.__enqueue_entity
0.30 ± 15% -0.1 0.17 ± 22% perf-profile.children.cycles-pp.finish_task_switch
0.19 ± 14% -0.1 0.07 ± 48% perf-profile.children.cycles-pp.is_bpf_text_address
0.24 ± 11% -0.1 0.11 ± 13% perf-profile.children.cycles-pp.__restore_fpregs_from_fpstate
0.24 ± 12% -0.1 0.12 ± 11% perf-profile.children.cycles-pp.__update_load_avg_se
0.21 ± 16% -0.1 0.09 ± 20% perf-profile.children.cycles-pp.cpuacct_charge
0.22 ± 13% -0.1 0.10 ± 11% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.13 ± 15% -0.1 0.03 ±102% perf-profile.children.cycles-pp.perf_tp_event_match
0.16 ± 11% -0.1 0.05 ± 45% perf-profile.children.cycles-pp.perf_trace_buf_alloc
0.16 ± 16% -0.1 0.06 ± 11% perf-profile.children.cycles-pp.put_prev_entity
0.14 ± 14% -0.1 0.04 ± 75% perf-profile.children.cycles-pp.save_fpregs_to_fpstate
0.13 ± 11% -0.1 0.04 ± 73% perf-profile.children.cycles-pp.get_callchain_entry
0.15 ± 11% -0.1 0.06 ± 46% perf-profile.children.cycles-pp.perf_trace_buf_update
0.14 ± 15% -0.1 0.05 ± 51% perf-profile.children.cycles-pp.bpf_ksym_find
0.13 ± 11% -0.1 0.05 ± 47% perf-profile.children.cycles-pp.get_stack_info
0.16 ± 4% -0.1 0.08 ± 8% perf-profile.children.cycles-pp.resched_curr
0.11 ± 11% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.perf_instruction_pointer
0.11 ± 14% -0.1 0.04 ± 47% perf-profile.children.cycles-pp.___perf_sw_event
0.10 ± 14% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.tracing_gen_ctx_irq_test
0.13 ± 15% -0.1 0.07 ± 10% perf-profile.children.cycles-pp.update_min_vruntime
0.14 ± 14% -0.1 0.08 ± 11% perf-profile.children.cycles-pp.__list_del_entry_valid
0.14 ± 11% -0.0 0.09 ± 12% perf-profile.children.cycles-pp.preempt_schedule_common
0.10 ± 9% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.__list_add_valid
0.08 ± 6% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.iov_iter_init
0.10 ± 7% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.__calc_delta
0.17 ± 4% -0.0 0.14 ± 5% perf-profile.children.cycles-pp.asm_sysvec_reschedule_ipi
0.12 ± 6% -0.0 0.09 ± 6% perf-profile.children.cycles-pp.finish_wait
0.14 ± 3% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.rcu_read_unlock_strict
0.06 +0.0 0.07 perf-profile.children.cycles-pp.refill_obj_stock
0.06 +0.0 0.07 ± 5% perf-profile.children.cycles-pp.check_stack_object
0.07 +0.0 0.09 ± 7% perf-profile.children.cycles-pp.rcu_all_qs
0.15 ± 3% +0.0 0.17 ± 4% perf-profile.children.cycles-pp.__ksize
0.04 ± 45% +0.0 0.06 ± 7% perf-profile.children.cycles-pp.irq_exit_rcu
0.10 ± 4% +0.0 0.12 ± 3% perf-profile.children.cycles-pp.wait_for_unix_gc
0.13 ± 5% +0.0 0.16 ± 4% perf-profile.children.cycles-pp.__might_fault
0.19 ± 3% +0.0 0.22 ± 4% perf-profile.children.cycles-pp.__virt_addr_valid
0.18 +0.0 0.21 ± 4% perf-profile.children.cycles-pp.__might_sleep
0.19 ± 2% +0.0 0.22 ± 2% perf-profile.children.cycles-pp.security_socket_recvmsg
0.22 ± 2% +0.0 0.25 perf-profile.children.cycles-pp.sock_recvmsg
0.26 ± 3% +0.0 0.30 ± 2% perf-profile.children.cycles-pp.copyin
0.22 ± 2% +0.1 0.27 ± 3% perf-profile.children.cycles-pp.__check_heap_object
0.00 +0.1 0.05 ± 7% perf-profile.children.cycles-pp.load_balance
0.37 +0.1 0.42 ± 2% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.33 ± 2% +0.1 0.38 ± 2% perf-profile.children.cycles-pp.aa_sk_perm
0.04 ± 73% +0.1 0.09 ± 13% perf-profile.children.cycles-pp.generic_file_write_iter
0.04 ± 73% +0.1 0.09 ± 13% perf-profile.children.cycles-pp.__generic_file_write_iter
0.04 ± 73% +0.1 0.09 ± 13% perf-profile.children.cycles-pp.generic_perform_write
0.00 +0.1 0.06 ± 6% perf-profile.children.cycles-pp.__softirqentry_text_start
0.42 ± 2% +0.1 0.49 ± 4% perf-profile.children.cycles-pp._copy_from_iter
0.38 +0.1 0.45 ± 3% perf-profile.children.cycles-pp.__entry_text_start
0.00 +0.1 0.08 ± 17% perf-profile.children.cycles-pp.schedule_idle
0.26 ± 4% +0.1 0.35 ± 3% perf-profile.children.cycles-pp.__build_skb_around
0.01 ±223% +0.1 0.11 ± 41% perf-profile.children.cycles-pp.try_to_migrate_one
0.49 ± 3% +0.1 0.60 ± 5% perf-profile.children.cycles-pp.mutex_lock
0.90 ± 3% +0.1 1.01 ± 6% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.01 ±223% +0.1 0.14 ± 40% perf-profile.children.cycles-pp.try_to_migrate
0.65 +0.2 0.81 perf-profile.children.cycles-pp.simple_copy_to_iter
0.75 ± 2% +0.2 0.92 ± 4% perf-profile.children.cycles-pp.skb_copy_datagram_from_iter
0.02 ±223% +0.2 0.22 ± 48% perf-profile.children.cycles-pp.remove_migration_pte
0.76 ± 7% +0.2 0.99 ± 12% perf-profile.children.cycles-pp.__slab_alloc
0.76 ± 6% +0.2 0.98 ± 11% perf-profile.children.cycles-pp.___slab_alloc
0.92 +0.2 1.15 ± 2% perf-profile.children.cycles-pp.__check_object_size
0.29 ± 15% +0.2 0.53 ± 7% perf-profile.children.cycles-pp.drain_stock
0.49 ± 12% +0.2 0.74 ± 19% perf-profile.children.cycles-pp.__unfreeze_partials
0.03 ±155% +0.2 0.27 ± 47% perf-profile.children.cycles-pp.remove_migration_ptes
0.34 ± 13% +0.3 0.60 ± 6% perf-profile.children.cycles-pp.refill_stock
0.40 ± 9% +0.3 0.67 ± 4% perf-profile.children.cycles-pp.unix_write_space
0.34 ± 15% +0.3 0.62 ± 7% perf-profile.children.cycles-pp.try_charge_memcg
0.03 ±161% +0.3 0.32 ± 45% perf-profile.children.cycles-pp.page_vma_mapped_walk
0.65 ± 2% +0.3 0.94 ± 3% perf-profile.children.cycles-pp.copyout
0.82 ± 2% +0.3 1.12 ± 3% perf-profile.children.cycles-pp._copy_to_iter
0.17 ± 38% +0.3 0.49 ± 39% perf-profile.children.cycles-pp.do_user_addr_fault
0.88 ± 2% +0.3 1.20 ± 3% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.06 ±106% +0.4 0.41 ± 44% perf-profile.children.cycles-pp.rmap_walk_anon
1.28 ± 9% +0.4 1.63 ± 7% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
0.06 ±106% +0.4 0.42 ± 44% perf-profile.children.cycles-pp.migrate_misplaced_page
0.06 ±106% +0.4 0.42 ± 44% perf-profile.children.cycles-pp.migrate_pages
0.06 ±108% +0.4 0.42 ± 45% perf-profile.children.cycles-pp.do_numa_page
0.08 ± 81% +0.4 0.44 ± 43% perf-profile.children.cycles-pp.__handle_mm_fault
0.08 ± 84% +0.4 0.45 ± 42% perf-profile.children.cycles-pp.handle_mm_fault
1.79 ± 8% +0.4 2.22 ± 6% perf-profile.children.cycles-pp.mod_objcg_state
0.00 +0.4 0.43 ± 22% perf-profile.children.cycles-pp.intel_idle
0.58 ± 12% +0.4 1.01 ± 7% perf-profile.children.cycles-pp.mutex_unlock
0.76 ± 7% +0.4 1.21 ± 7% perf-profile.children.cycles-pp.skb_queue_tail
0.01 ±223% +0.5 0.46 ± 22% perf-profile.children.cycles-pp.cpuidle_enter_state
0.01 ±223% +0.5 0.46 ± 22% perf-profile.children.cycles-pp.cpuidle_enter
0.60 ± 12% +0.5 1.06 ± 7% perf-profile.children.cycles-pp.skb_unlink
1.52 +0.5 1.99 ± 2% perf-profile.children.cycles-pp.skb_copy_datagram_iter
1.50 ± 2% +0.5 1.98 ± 2% perf-profile.children.cycles-pp.__skb_datagram_iter
1.53 +0.5 2.01 ± 2% perf-profile.children.cycles-pp.unix_stream_read_actor
0.65 ± 5% +0.5 1.15 ± 6% perf-profile.children.cycles-pp.skb_set_owner_w
0.02 ±144% +0.6 0.58 ± 21% perf-profile.children.cycles-pp.start_secondary
0.02 ±144% +0.6 0.59 ± 21% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
0.02 ±144% +0.6 0.59 ± 21% perf-profile.children.cycles-pp.cpu_startup_entry
0.02 ±144% +0.6 0.59 ± 21% perf-profile.children.cycles-pp.do_idle
1.16 ± 9% +0.8 1.99 ± 4% perf-profile.children.cycles-pp.sock_wfree
1.21 ± 8% +0.8 2.04 ± 4% perf-profile.children.cycles-pp.unix_destruct_scm
1.22 ± 8% +0.8 2.06 ± 4% perf-profile.children.cycles-pp.skb_release_head_state
1.23 ± 8% +0.8 2.07 ± 4% perf-profile.children.cycles-pp.skb_release_all
1.58 ± 7% +1.0 2.53 ± 8% perf-profile.children.cycles-pp.get_mem_cgroup_from_objcg
1.47 ± 9% +1.1 2.54 ± 5% perf-profile.children.cycles-pp.__slab_free
2.99 ± 5% +1.1 4.13 ± 4% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
7.88 ± 8% +2.4 10.33 ± 4% perf-profile.children.cycles-pp.kmem_cache_alloc_node
47.00 +3.5 50.55 perf-profile.children.cycles-pp.__libc_read
7.43 ± 12% +3.7 11.08 ± 7% perf-profile.children.cycles-pp.propagate_protected_usage
44.84 +3.9 48.70 perf-profile.children.cycles-pp.vfs_read
45.04 +3.9 48.91 perf-profile.children.cycles-pp.ksys_read
43.69 +3.9 47.56 perf-profile.children.cycles-pp.unix_stream_recvmsg
43.61 +3.9 47.49 perf-profile.children.cycles-pp.unix_stream_read_generic
44.12 +3.9 48.00 perf-profile.children.cycles-pp.sock_read_iter
44.34 +3.9 48.24 perf-profile.children.cycles-pp.new_sync_read
7.55 ± 9% +4.1 11.66 ± 4% perf-profile.children.cycles-pp.kmem_cache_free
14.24 ± 9% +4.5 18.76 ± 4% perf-profile.children.cycles-pp.kmalloc_reserve
14.20 ± 9% +4.5 18.72 ± 4% perf-profile.children.cycles-pp.__kmalloc_node_track_caller
11.62 ± 11% +5.2 16.83 ± 6% perf-profile.children.cycles-pp.page_counter_cancel
12.06 ± 9% +5.9 17.92 ± 4% perf-profile.children.cycles-pp.kfree
18.74 ± 10% +6.4 25.17 ± 5% perf-profile.children.cycles-pp.obj_cgroup_charge_pages
18.83 ± 10% +6.5 25.28 ± 5% perf-profile.children.cycles-pp.obj_cgroup_charge
13.12 ± 9% +6.5 19.62 ± 4% perf-profile.children.cycles-pp.skb_release_data
22.71 ± 8% +7.1 29.84 ± 4% perf-profile.children.cycles-pp.__alloc_skb
22.76 ± 8% +7.1 29.89 ± 4% perf-profile.children.cycles-pp.alloc_skb_with_frags
14.56 ± 9% +7.5 22.04 ± 4% perf-profile.children.cycles-pp.consume_skb
14.92 ± 11% +7.6 22.56 ± 5% perf-profile.children.cycles-pp.page_counter_uncharge
23.80 ± 8% +8.0 31.76 ± 4% perf-profile.children.cycles-pp.sock_alloc_send_pskb
16.54 ± 11% +8.6 25.14 ± 5% perf-profile.children.cycles-pp.obj_cgroup_uncharge_pages
0.00 +22.0 22.04 ± 5% perf-profile.children.cycles-pp.page_counter_charge
12.79 ± 10% -12.6 0.17 ± 9% perf-profile.self.cycles-pp.page_counter_try_charge
4.56 ± 13% -3.1 1.44 ± 33% perf-profile.self.cycles-pp.unwind_next_frame
3.43 ± 14% -2.3 1.14 ± 32% perf-profile.self.cycles-pp.__get_user_nocheck_8
2.98 ± 13% -2.0 0.99 ± 32% perf-profile.self.cycles-pp.__orc_find
3.89 ± 10% -1.5 2.38 ± 21% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
1.61 ± 13% -1.1 0.53 ± 32% perf-profile.self.cycles-pp.orc_find
1.22 ± 12% -0.8 0.39 ± 34% perf-profile.self.cycles-pp.perf_callchain_kernel
1.20 ± 10% -0.8 0.44 ± 28% perf-profile.self.cycles-pp.native_irq_return_iret
0.92 ± 13% -0.6 0.29 ± 33% perf-profile.self.cycles-pp.stack_access_ok
1.54 ± 5% -0.6 0.92 ± 6% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.82 ± 9% -0.4 0.45 ± 9% perf-profile.self.cycles-pp.__schedule
0.50 ± 14% -0.3 0.16 ± 30% perf-profile.self.cycles-pp.cmp_ex_search
0.50 ± 12% -0.3 0.16 ± 30% perf-profile.self.cycles-pp.__unwind_start
0.48 ± 13% -0.3 0.16 ± 31% perf-profile.self.cycles-pp.kernel_text_address
0.49 ± 13% -0.3 0.22 ± 10% perf-profile.self.cycles-pp.update_curr
0.36 ± 12% -0.3 0.11 ± 31% perf-profile.self.cycles-pp.unwind_get_return_address
0.44 ± 10% -0.3 0.19 ± 10% perf-profile.self.cycles-pp.load_new_mm_cr3
0.34 ± 13% -0.2 0.10 ± 49% perf-profile.self.cycles-pp.bsearch
0.30 ± 17% -0.2 0.08 ± 47% perf-profile.self.cycles-pp.perf_callchain_user
0.45 ± 3% -0.2 0.25 ± 7% perf-profile.self.cycles-pp.__switch_to
0.27 ± 9% -0.2 0.08 ± 47% perf-profile.self.cycles-pp.get_perf_callchain
0.33 ± 10% -0.2 0.14 ± 21% perf-profile.self.cycles-pp.perf_tp_event
0.28 ± 12% -0.2 0.11 ± 30% perf-profile.self.cycles-pp.perf_output_begin_forward
0.28 ± 9% -0.2 0.11 ± 27% perf-profile.self.cycles-pp.__task_pid_nr_ns
0.26 ± 12% -0.2 0.09 ± 48% perf-profile.self.cycles-pp.ftrace_graph_ret_addr
0.26 ± 12% -0.2 0.10 ± 46% perf-profile.self.cycles-pp.__kernel_text_address
0.26 ± 17% -0.2 0.10 ± 21% perf-profile.self.cycles-pp.available_idle_cpu
0.27 ± 10% -0.2 0.11 ± 27% perf-profile.self.cycles-pp.perf_prepare_sample
0.26 ± 9% -0.2 0.10 ± 19% perf-profile.self.cycles-pp.native_sched_clock
0.22 ± 11% -0.2 0.07 ± 48% perf-profile.self.cycles-pp.asm_exc_page_fault
0.30 ± 6% -0.1 0.16 ± 3% perf-profile.self.cycles-pp.new_sync_write
0.28 ± 10% -0.1 0.14 ± 9% perf-profile.self.cycles-pp.__switch_to_asm
0.22 ± 12% -0.1 0.08 ± 49% perf-profile.self.cycles-pp.perf_trace_sched_switch
0.40 ± 12% -0.1 0.27 ± 9% perf-profile.self.cycles-pp.update_load_avg
0.28 ± 11% -0.1 0.14 ± 14% perf-profile.self.cycles-pp.try_to_wake_up
0.21 ± 16% -0.1 0.08 ± 8% perf-profile.self.cycles-pp.__enqueue_entity
0.24 ± 11% -0.1 0.11 ± 13% perf-profile.self.cycles-pp.__restore_fpregs_from_fpstate
0.24 ± 12% -0.1 0.12 ± 10% perf-profile.self.cycles-pp.__update_load_avg_se
0.17 ± 13% -0.1 0.05 ± 47% perf-profile.self.cycles-pp.error_entry
0.22 ± 14% -0.1 0.11 ± 16% perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
0.20 ± 16% -0.1 0.09 ± 16% perf-profile.self.cycles-pp.cpuacct_charge
0.21 ± 14% -0.1 0.10 ± 14% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.22 ± 6% -0.1 0.12 ± 8% perf-profile.self.cycles-pp.switch_fpu_return
0.13 ± 14% -0.1 0.03 ±102% perf-profile.self.cycles-pp.perf_tp_event_match
0.28 ± 6% -0.1 0.18 ± 5% perf-profile.self.cycles-pp.prepare_to_wait
0.17 ± 10% -0.1 0.07 ± 12% perf-profile.self.cycles-pp.reweight_entity
0.13 ± 8% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.get_callchain_entry
0.13 ± 14% -0.1 0.04 ± 75% perf-profile.self.cycles-pp.save_fpregs_to_fpstate
0.14 ± 15% -0.1 0.05 ± 49% perf-profile.self.cycles-pp.bpf_ksym_find
0.19 ± 11% -0.1 0.10 ± 5% perf-profile.self.cycles-pp.pick_next_task_fair
0.16 ± 14% -0.1 0.08 ± 13% perf-profile.self.cycles-pp.update_rq_clock
0.16 ± 11% -0.1 0.08 ± 5% perf-profile.self.cycles-pp.enqueue_entity
0.15 ± 13% -0.1 0.07 ± 12% perf-profile.self.cycles-pp.select_idle_sibling
0.34 ± 5% -0.1 0.26 ± 3% perf-profile.self.cycles-pp.sock_write_iter
0.15 ± 16% -0.1 0.07 ± 18% perf-profile.self.cycles-pp.select_idle_cpu
0.16 ± 4% -0.1 0.08 ± 8% perf-profile.self.cycles-pp.resched_curr
0.15 ± 10% -0.1 0.08 ± 4% perf-profile.self.cycles-pp.select_task_rq_fair
0.14 ± 14% -0.1 0.08 ± 16% perf-profile.self.cycles-pp.enqueue_task_fair
0.13 ± 8% -0.1 0.06 ± 11% perf-profile.self.cycles-pp.schedule
0.10 ± 14% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.tracing_gen_ctx_irq_test
0.14 ± 10% -0.1 0.07 ± 6% perf-profile.self.cycles-pp.dequeue_task_fair
0.12 ± 14% -0.1 0.06 ± 7% perf-profile.self.cycles-pp.update_min_vruntime
0.13 ± 7% -0.1 0.08 ± 7% perf-profile.self.cycles-pp.do_syscall_64
0.29 ± 6% -0.0 0.24 ± 2% perf-profile.self.cycles-pp.__libc_read
0.13 ± 12% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.__list_del_entry_valid
0.10 ± 9% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.__list_add_valid
0.11 ± 14% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.perf_trace_sched_wakeup_template
0.10 ± 7% -0.0 0.06 ± 6% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.08 ± 10% -0.0 0.03 ± 70% perf-profile.self.cycles-pp.iov_iter_init
0.10 ± 7% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.__calc_delta
0.09 ± 7% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.finish_task_switch
0.24 ± 3% -0.0 0.20 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.08 ± 10% -0.0 0.04 ± 45% perf-profile.self.cycles-pp.dequeue_entity
0.08 ± 8% -0.0 0.04 ± 45% perf-profile.self.cycles-pp.check_preempt_wakeup
0.08 ± 11% -0.0 0.05 ± 46% perf-profile.self.cycles-pp.__perf_event_header__init_id
0.07 ± 5% -0.0 0.04 ± 44% perf-profile.self.cycles-pp.sock_sendmsg
0.11 ± 6% -0.0 0.09 ± 5% perf-profile.self.cycles-pp.exit_to_user_mode_prepare
0.07 ± 10% -0.0 0.05 ± 7% perf-profile.self.cycles-pp.schedule_timeout
0.21 ± 2% -0.0 0.18 ± 4% perf-profile.self.cycles-pp.sock_read_iter
0.08 ± 4% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.rcu_read_unlock_strict
0.23 ± 3% -0.0 0.21 ± 3% perf-profile.self.cycles-pp.__libc_write
0.13 ± 6% -0.0 0.11 ± 6% perf-profile.self.cycles-pp.security_file_permission
0.06 ± 6% +0.0 0.07 perf-profile.self.cycles-pp.refill_obj_stock
0.05 ± 7% +0.0 0.06 ± 7% perf-profile.self.cycles-pp.check_stack_object
0.09 ± 7% +0.0 0.11 ± 6% perf-profile.self.cycles-pp.obj_cgroup_charge
0.05 ± 8% +0.0 0.07 perf-profile.self.cycles-pp.__cond_resched
0.05 ± 7% +0.0 0.07 ± 8% perf-profile.self.cycles-pp.rcu_all_qs
0.10 ± 4% +0.0 0.12 ± 6% perf-profile.self.cycles-pp._copy_from_iter
0.18 ± 2% +0.0 0.20 ± 4% perf-profile.self.cycles-pp.new_sync_read
0.14 ± 3% +0.0 0.16 ± 6% perf-profile.self.cycles-pp.__ksize
0.16 +0.0 0.19 ± 3% perf-profile.self.cycles-pp.__might_sleep
0.19 ± 4% +0.0 0.22 ± 3% perf-profile.self.cycles-pp.aa_sk_perm
0.18 ± 4% +0.0 0.21 ± 4% perf-profile.self.cycles-pp.__virt_addr_valid
0.17 ± 2% +0.0 0.21 ± 3% perf-profile.self.cycles-pp.__entry_text_start
0.27 ± 2% +0.0 0.31 ± 3% perf-profile.self.cycles-pp.__alloc_skb
0.34 +0.0 0.38 ± 4% perf-profile.self.cycles-pp.kmem_cache_alloc_node
0.04 ± 45% +0.0 0.09 ± 9% perf-profile.self.cycles-pp.skb_copy_datagram_from_iter
0.21 ± 2% +0.0 0.26 ± 4% perf-profile.self.cycles-pp.__check_heap_object
0.50 ± 2% +0.1 0.54 ± 2% perf-profile.self.cycles-pp.__kmalloc_node_track_caller
0.46 ± 4% +0.1 0.52 ± 5% perf-profile.self.cycles-pp.unix_stream_sendmsg
0.36 +0.1 0.42 ± 2% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.32 ± 2% +0.1 0.38 ± 5% perf-profile.self.cycles-pp.___slab_alloc
0.22 ± 5% +0.1 0.28 ± 6% perf-profile.self.cycles-pp.unix_write_space
0.00 +0.1 0.07 ± 7% perf-profile.self.cycles-pp.refill_stock
0.14 ± 16% +0.1 0.21 ± 3% perf-profile.self.cycles-pp.page_counter_uncharge
0.19 ± 6% +0.1 0.27 ± 5% perf-profile.self.cycles-pp.__build_skb_around
0.40 ± 2% +0.1 0.50 ± 4% perf-profile.self.cycles-pp.kfree
0.37 ± 4% +0.1 0.49 ± 6% perf-profile.self.cycles-pp.mutex_lock
0.45 ± 3% +0.1 0.59 ± 2% perf-profile.self.cycles-pp.__check_object_size
0.21 ± 8% +0.1 0.35 ± 9% perf-profile.self.cycles-pp.consume_skb
0.02 ±223% +0.1 0.16 ± 45% perf-profile.self.cycles-pp.page_vma_mapped_walk
0.20 ± 11% +0.2 0.36 ± 7% perf-profile.self.cycles-pp.skb_release_data
1.02 ± 3% +0.2 1.22 ± 4% perf-profile.self.cycles-pp.unix_stream_read_generic
0.58 ± 2% +0.2 0.78 ± 3% perf-profile.self.cycles-pp.kmem_cache_free
0.86 ± 2% +0.3 1.19 ± 3% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
1.25 ± 9% +0.4 1.60 ± 6% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state
0.00 +0.4 0.43 ± 22% perf-profile.self.cycles-pp.intel_idle
0.57 ± 12% +0.4 1.00 ± 7% perf-profile.self.cycles-pp.mutex_unlock
0.76 ± 8% +0.4 1.20 ± 8% perf-profile.self.cycles-pp.obj_cgroup_uncharge_pages
0.64 ± 4% +0.5 1.14 ± 6% perf-profile.self.cycles-pp.skb_set_owner_w
0.73 ± 8% +0.5 1.25 ± 9% perf-profile.self.cycles-pp.obj_cgroup_charge_pages
0.74 ± 9% +0.6 1.30 ± 7% perf-profile.self.cycles-pp.sock_wfree
0.81 ± 11% +0.6 1.42 ± 7% perf-profile.self.cycles-pp.sock_def_readable
2.22 ± 4% +0.7 2.88 ± 6% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
2.79 ± 4% +0.9 3.68 ± 5% perf-profile.self.cycles-pp._raw_spin_lock
1.56 ± 7% +0.9 2.50 ± 8% perf-profile.self.cycles-pp.get_mem_cgroup_from_objcg
1.45 ± 9% +1.1 2.51 ± 5% perf-profile.self.cycles-pp.__slab_free
7.36 ± 12% +3.6 10.97 ± 7% perf-profile.self.cycles-pp.propagate_protected_usage
11.50 ± 11% +5.2 16.66 ± 6% perf-profile.self.cycles-pp.page_counter_cancel
0.00 +16.7 16.74 ± 6% perf-profile.self.cycles-pp.page_counter_charge
hackbench.throughput
145000 +------------------------------------------------------------------+
| O |
140000 |-+ O O O |
| OO OO O O O O O O |
135000 |-O O O OO O O O O |
| O O O O O O O |
130000 |-+ O O O O |
| OO O ++ .+.+ |
125000 |.++.+.+ +.+.++.+.++.+.+.++ : + + +.+.+ .+ |
| + ++.+ : O : : +.+ +.+ |
120000 |-+ :O : : : : : |
| : : :: : |
115000 |-+ :: : + |
| + + |
110000 +------------------------------------------------------------------+
hackbench.time.voluntary_context_switches
1.4e+09 +-----------------------------------------------------------------+
| + |
1.3e+09 |-+ + :: |
1.2e+09 |-+ :+ : : |
| : + +.+. |
1.1e+09 |.++. .+ ++.++. .++.+.+ .+ + |
1e+09 |-+ + + + + + : |
| + + : .++. .+. |
9e+08 |-+ +.+. : ++.+ ++ + |
8e+08 |-+ + |
| |
7e+08 |-+ |
6e+08 |-+ |
| OO O OO O OO O OO OO O OO O OO O OO O OO O OO O O OO O OO O OO |
5e+08 +-----------------------------------------------------------------+
hackbench.time.involuntary_context_switches
6.5e+08 +-----------------------------------------------------------------+
6e+08 |-+ + |
| + : |
5.5e+08 |-+ :: : : |
5e+08 |-+ :: : +.+. |
|. : + ++. +. |
4.5e+08 |-++. .+ ++. .+ +.+ .+ .+ |
4e+08 |-+ + + + + + : |
3.5e+08 |-+ + : : .++. .+. |
| +.+.: ++.+ ++ + |
3e+08 |-+ + |
2.5e+08 |-+ |
| |
2e+08 |-+ O O O O OO |
1.5e+08 +-----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
---
0DAY/LKP+ Test Infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation
Thanks,
Oliver Sang
View attachment "config-5.15.0-00059-g58056f77502f" of type "text/plain" (172640 bytes)
View attachment "job-script" of type "text/plain" (8136 bytes)
View attachment "job.yaml" of type "text/plain" (5583 bytes)
View attachment "reproduce" of type "text/plain" (530 bytes)
Powered by blists - more mailing lists