| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20210726022421.GB21872@xsang-OptiPlex-9020>
Date: Mon, 26 Jul 2021 10:24:21 +0800
From: kernel test robot <oliver.sang@...el.com>
To: Shakeel Butt <shakeelb@...gle.com>
Cc: 0day robot <lkp@...el.com>, LKML <linux-kernel@...r.kernel.org>,
lkp@...ts.01.org, ying.huang@...el.com, feng.tang@...el.com,
zhengjun.xing@...ux.intel.com, Tejun Heo <tj@...nel.org>,
Johannes Weiner <hannes@...xchg.org>,
Muchun Song <songmuchun@...edance.com>,
Michal Hocko <mhocko@...nel.org>, Roman Gushchin <guro@...com>,
Michal Koutný <mkoutny@...e.com>,
Hillf Danton <hdanton@...a.com>,
Yang Yingliang <yangyingliang@...wei.com>,
Marek Szyprowski <m.szyprowski@...sung.com>,
Andrew Morton <akpm@...ux-foundation.org>,
cgroups@...r.kernel.org, linux-mm@...ck.org,
Shakeel Butt <shakeelb@...gle.com>
Subject: [memcg] cda0d20525: reaim.jobs_per_min -10.1% regression
Greeting,
FYI, we noticed a -10.1% regression of reaim.jobs_per_min due to commit:
commit: cda0d205259a7e7c1f8e296130ac091557134a66 ("[PATCH v5 2/2] memcg: infrastructure to flush memcg stats")
url: https://github.com/0day-ci/linux/commits/Shakeel-Butt/memcg-switch-lruvec-stats-to-rstat/20210718-103230
base: https://git.kernel.org/cgit/linux/kernel/git/tj/cgroup.git for-next
in testcase: reaim
on test machine: 16 threads 1 sockets Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory
with following parameters:
runtime: 300s
nr_task: 100%
test: new_dbase
cpufreq_governor: performance
ucode: 0xde
test-description: REAIM is an updated and improved version of AIM 7 benchmark.
test-url: https://sourceforge.net/projects/re-aim-7/
In addition to that, the commit also has significant impact on the following tests:
+------------------+-------------------------------------------------------------------------------------+
| testcase: change | reaim: reaim.jobs_per_min -9.5% regression |
| test machine | 16 threads 1 sockets Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory |
| test parameters | cpufreq_governor=performance |
| | nr_task=100% |
| | runtime=300s |
| | test=new_fserver |
| | ucode=0xde |
+------------------+-------------------------------------------------------------------------------------+
| testcase: change | fsmark: fsmark.files_per_sec -6.9% regression |
| test machine | 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| test parameters | cpufreq_governor=performance |
| | disk=1BRD_48G |
| | filesize=4M |
| | fs=btrfs |
| | iterations=1x |
| | nr_threads=1t |
| | sync_method=fsyncBeforeClose |
| | test_size=24G |
| | ucode=0x5003006 |
+------------------+-------------------------------------------------------------------------------------+
| testcase: change | reaim: reaim.jobs_per_min -10.7% regression |
| test machine | 16 threads 1 sockets Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory |
| test parameters | cpufreq_governor=performance |
| | nr_task=100% |
| | runtime=300s |
| | test=five_sec |
| | ucode=0xde |
+------------------+-------------------------------------------------------------------------------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang@...el.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
bin/lkp run generated-yaml-file
=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/300s/lkp-cfl-e1/new_dbase/reaim/0xde
commit:
0c7c5da68b ("memcg: switch lruvec stats to rstat")
cda0d20525 ("memcg: infrastructure to flush memcg stats")
0c7c5da68b4a09ff cda0d205259a7e7c1f8e296130a
---------------- ---------------------------
%stddev %change %stddev
\ | \
5.09 +5.8% 5.39 reaim.child_systime
5.44 -1.3% 5.37 reaim.child_utime
136122 -10.1% 122315 reaim.jobs_per_min
8507 -10.1% 7644 reaim.jobs_per_min_child
140593 -9.5% 127245 reaim.max_jobs_per_min
0.73 +11.3% 0.81 reaim.parent_time
1.68 +54.9% 2.61 ± 2% reaim.std_dev_percent
0.01 +82.8% 0.02 reaim.std_dev_time
62961 +1456.4% 979901 reaim.time.involuntary_context_switches
15091 -4.8% 14367 reaim.time.major_page_faults
30304357 -3.1% 29368128 reaim.time.minor_page_faults
557.91 +2.5% 571.83 reaim.time.system_time
589.92 -4.3% 564.56 reaim.time.user_time
194873 +2.7% 200206 reaim.time.voluntary_context_switches
173333 -3.1% 168000 reaim.workload
4225548 ± 3% +58.6% 6703765 ± 25% cpuidle..usage
12.09 +1.7 13.74 mpstat.cpu.all.sys%
2487 +700.9% 19922 vmstat.system.cs
21607 +28.8% 27835 ± 17% vmstat.system.in
36.54 +4.2% 38.08 boot-time.boot
11.56 +2.9% 11.90 ± 3% boot-time.dhcp
487.32 +4.7% 510.04 boot-time.idle
20013 +143.0% 48643 ± 3% meminfo.Active
19783 +145.2% 48504 ± 3% meminfo.Active(anon)
28739 +105.6% 59076 ± 2% meminfo.Shmem
1875 ± 6% -10.3% 1683 ± 3% slabinfo.kmalloc-rcl-256.active_objs
1916 ± 4% -9.5% 1734 ± 2% slabinfo.kmalloc-rcl-256.num_objs
1089 ± 4% -13.1% 946.14 ± 5% slabinfo.pid.active_objs
1089 ± 4% -13.1% 946.14 ± 5% slabinfo.pid.num_objs
1127 ± 5% -7.9% 1038 ± 2% slabinfo.task_delay_info.active_objs
1127 ± 5% -7.9% 1038 ± 2% slabinfo.task_delay_info.num_objs
32528 ± 74% +4337.7% 1443515 ± 17% turbostat.C1
0.02 ± 44% +1.0 1.02 ± 41% turbostat.C1%
358034 ± 38% +272.8% 1334615 ± 90% turbostat.C1E
0.06 ± 54% +694.8% 0.44 ± 98% turbostat.CPU%c3
35.84 +7.3% 38.46 turbostat.CorWatt
6582740 +28.6% 8467818 ± 17% turbostat.IRQ
11.71 ± 4% -11.2% 10.40 ± 8% turbostat.Pkg%pc2
36.27 +7.2% 38.90 turbostat.PkgWatt
4948 +145.1% 12127 ± 3% proc-vmstat.nr_active_anon
595570 +1.2% 602681 proc-vmstat.nr_file_pages
7890 +5.5% 8324 proc-vmstat.nr_mapped
7184 +105.7% 14775 ± 2% proc-vmstat.nr_shmem
4948 +145.1% 12127 ± 3% proc-vmstat.nr_zone_active_anon
29299018 -3.0% 28427445 proc-vmstat.numa_hit
29299018 -3.0% 28427445 proc-vmstat.numa_local
58688 +30.2% 76398 proc-vmstat.pgactivate
29590587 -3.0% 28698285 proc-vmstat.pgalloc_normal
30793164 -3.0% 29863963 proc-vmstat.pgfault
29571246 -3.1% 28659712 proc-vmstat.pgfree
1958161 -3.2% 1896449 proc-vmstat.pgreuse
34.22 -2.6% 33.34 perf-stat.i.MPKI
4.04e+09 +6.6% 4.306e+09 perf-stat.i.branch-instructions
1.34 -0.0 1.30 perf-stat.i.branch-miss-rate%
31862409 +8.4% 34539247 perf-stat.i.branch-misses
1.39 ± 4% -0.1 1.27 perf-stat.i.cache-miss-rate%
6.763e+08 +4.6% 7.073e+08 perf-stat.i.cache-references
2474 +707.5% 19982 perf-stat.i.context-switches
1.19 -2.4% 1.16 perf-stat.i.cpi
1.794e+10 +5.8% 1.897e+10 perf-stat.i.cpu-cycles
312.57 +85.3% 579.14 perf-stat.i.cpu-migrations
3386 ± 3% +8.3% 3665 perf-stat.i.cycles-between-cache-misses
0.07 -0.0 0.06 ± 2% perf-stat.i.dTLB-load-miss-rate%
3.903e+09 +9.1% 4.257e+09 perf-stat.i.dTLB-loads
0.02 -0.0 0.02 perf-stat.i.dTLB-store-miss-rate%
467914 -2.7% 455349 perf-stat.i.dTLB-store-misses
2.337e+09 +5.4% 2.463e+09 perf-stat.i.dTLB-stores
69.23 -3.8 65.41 ± 4% perf-stat.i.iTLB-load-miss-rate%
641802 +6.6% 683980 ± 4% perf-stat.i.iTLB-load-misses
344005 +26.1% 433937 ± 8% perf-stat.i.iTLB-loads
1.964e+10 +5.5% 2.073e+10 perf-stat.i.instructions
0.92 +1.3% 0.93 perf-stat.i.ipc
50.15 -4.7% 47.78 perf-stat.i.major-faults
1.12 +5.7% 1.19 perf-stat.i.metric.GHz
154.74 +2.7% 158.85 perf-stat.i.metric.K/sec
684.89 +7.1% 733.44 perf-stat.i.metric.M/sec
101097 -2.9% 98143 perf-stat.i.minor-faults
1266584 -1.7% 1245496 perf-stat.i.node-stores
101148 -2.9% 98191 perf-stat.i.page-faults
0.79 +0.0 0.80 perf-stat.overall.branch-miss-rate%
0.68 ± 2% -0.0 0.64 perf-stat.overall.cache-miss-rate%
3905 ± 2% +6.8% 4172 perf-stat.overall.cycles-between-cache-misses
0.02 -0.0 0.01 ± 4% perf-stat.overall.dTLB-load-miss-rate%
0.02 -0.0 0.02 perf-stat.overall.dTLB-store-miss-rate%
65.10 -3.9 61.22 perf-stat.overall.iTLB-load-miss-rate%
34168578 +8.8% 37185877 perf-stat.overall.path-length
4.028e+09 +6.6% 4.293e+09 perf-stat.ps.branch-instructions
31761900 +8.4% 34426923 perf-stat.ps.branch-misses
6.742e+08 +4.6% 7.05e+08 perf-stat.ps.cache-references
2467 +707.3% 19922 perf-stat.ps.context-switches
1.789e+10 +5.7% 1.891e+10 perf-stat.ps.cpu-cycles
311.76 +85.2% 577.51 perf-stat.ps.cpu-migrations
3.891e+09 +9.1% 4.244e+09 perf-stat.ps.dTLB-loads
466586 -2.7% 454015 perf-stat.ps.dTLB-store-misses
2.33e+09 +5.4% 2.456e+09 perf-stat.ps.dTLB-stores
639845 +6.6% 681843 ± 4% perf-stat.ps.iTLB-load-misses
343006 +26.1% 432660 ± 8% perf-stat.ps.iTLB-loads
1.958e+10 +5.5% 2.067e+10 perf-stat.ps.instructions
50.01 -4.7% 47.64 perf-stat.ps.major-faults
100816 -2.9% 97861 perf-stat.ps.minor-faults
1262741 -1.7% 1241553 perf-stat.ps.node-stores
100866 -2.9% 97908 perf-stat.ps.page-faults
5.923e+12 +5.5% 6.247e+12 perf-stat.total.instructions
20646 ± 5% +638.2% 152404 interrupts.CAL:Function_call_interrupts
1553 ± 65% +524.0% 9695 ± 8% interrupts.CPU0.CAL:Function_call_interrupts
1455 ± 2% +3654.2% 54636 ± 6% interrupts.CPU0.RES:Rescheduling_interrupts
4.00 ± 77% +5085.7% 207.43 ± 10% interrupts.CPU0.TLB:TLB_shootdowns
1429 ± 33% +535.0% 9077 ± 4% interrupts.CPU1.CAL:Function_call_interrupts
3201 ± 5% +23.3% 3948 ± 2% interrupts.CPU1.NMI:Non-maskable_interrupts
3201 ± 5% +23.3% 3948 ± 2% interrupts.CPU1.PMI:Performance_monitoring_interrupts
1427 ± 2% +3468.7% 50931 ± 6% interrupts.CPU1.RES:Rescheduling_interrupts
4.67 ± 52% +4488.8% 214.14 ± 10% interrupts.CPU1.TLB:TLB_shootdowns
843.83 ± 53% +1007.6% 9346 ± 9% interrupts.CPU10.CAL:Function_call_interrupts
1476 ± 3% +3642.0% 55231 ± 5% interrupts.CPU10.RES:Rescheduling_interrupts
4.00 ± 40% +5232.1% 213.29 ± 8% interrupts.CPU10.TLB:TLB_shootdowns
1094 ± 58% +744.2% 9242 ± 7% interrupts.CPU11.CAL:Function_call_interrupts
1452 ± 3% +3495.2% 52214 ± 3% interrupts.CPU11.RES:Rescheduling_interrupts
5.17 ± 47% +3978.3% 210.71 ± 9% interrupts.CPU11.TLB:TLB_shootdowns
1155 ± 68% +769.0% 10036 ± 11% interrupts.CPU12.CAL:Function_call_interrupts
1502 ± 4% +3506.4% 54174 ± 8% interrupts.CPU12.RES:Rescheduling_interrupts
4.00 ± 57% +5350.0% 218.00 ± 9% interrupts.CPU12.TLB:TLB_shootdowns
1530 ± 58% +544.1% 9857 ± 11% interrupts.CPU13.CAL:Function_call_interrupts
1484 ± 2% +3436.4% 52509 ± 2% interrupts.CPU13.RES:Rescheduling_interrupts
5.17 ± 26% +4069.6% 215.43 ± 8% interrupts.CPU13.TLB:TLB_shootdowns
1181 ± 46% +808.1% 10732 ± 10% interrupts.CPU14.CAL:Function_call_interrupts
1454 ± 3% +3479.2% 52065 ± 6% interrupts.CPU14.RES:Rescheduling_interrupts
4.17 ± 59% +5073.7% 215.57 ± 8% interrupts.CPU14.TLB:TLB_shootdowns
1553 ± 91% +507.4% 9435 ± 6% interrupts.CPU15.CAL:Function_call_interrupts
1462 ± 3% +3482.7% 52385 ± 5% interrupts.CPU15.RES:Rescheduling_interrupts
5.00 ± 25% +4048.6% 207.43 ± 5% interrupts.CPU15.TLB:TLB_shootdowns
907.33 ± 41% +891.7% 8997 ± 6% interrupts.CPU2.CAL:Function_call_interrupts
1453 ± 2% +3546.2% 53003 ± 8% interrupts.CPU2.RES:Rescheduling_interrupts
6.83 ± 39% +2847.7% 201.43 ± 7% interrupts.CPU2.TLB:TLB_shootdowns
835.50 ± 26% +1012.7% 9296 ± 8% interrupts.CPU3.CAL:Function_call_interrupts
2370 ± 35% +64.6% 3901 interrupts.CPU3.NMI:Non-maskable_interrupts
2370 ± 35% +64.6% 3901 interrupts.CPU3.PMI:Performance_monitoring_interrupts
1461 ± 3% +3520.2% 52921 ± 4% interrupts.CPU3.RES:Rescheduling_interrupts
5.00 ± 40% +3997.1% 204.86 ± 7% interrupts.CPU3.TLB:TLB_shootdowns
1083 ± 59% +770.8% 9437 ± 6% interrupts.CPU4.CAL:Function_call_interrupts
1438 ± 2% +3735.7% 55182 ± 4% interrupts.CPU4.RES:Rescheduling_interrupts
4.83 ± 46% +4191.6% 207.43 ± 4% interrupts.CPU4.TLB:TLB_shootdowns
1532 ±100% +510.2% 9350 ± 4% interrupts.CPU5.CAL:Function_call_interrupts
1455 ± 2% +3475.4% 52052 ± 7% interrupts.CPU5.RES:Rescheduling_interrupts
5.17 ± 36% +3809.7% 202.00 ± 6% interrupts.CPU5.TLB:TLB_shootdowns
822.17 ± 34% +1052.6% 9476 ± 3% interrupts.CPU6.CAL:Function_call_interrupts
1398 +3703.0% 53172 ± 3% interrupts.CPU6.RES:Rescheduling_interrupts
3.50 ± 48% +6242.9% 222.00 ± 8% interrupts.CPU6.TLB:TLB_shootdowns
1197 ± 14% +710.7% 9706 ± 4% interrupts.CPU7.CAL:Function_call_interrupts
1605 ± 3% +3303.7% 54657 ± 6% interrupts.CPU7.RES:Rescheduling_interrupts
5.17 ± 46% +4088.9% 216.43 ± 3% interrupts.CPU7.TLB:TLB_shootdowns
2116 ± 85% +354.4% 9617 ± 6% interrupts.CPU8.CAL:Function_call_interrupts
1453 ± 3% +3569.5% 53318 ± 8% interrupts.CPU8.RES:Rescheduling_interrupts
4.67 ± 44% +4396.9% 209.86 ± 9% interrupts.CPU8.TLB:TLB_shootdowns
1808 ± 70% +403.0% 9098 ± 4% interrupts.CPU9.CAL:Function_call_interrupts
1452 +3589.8% 53581 ± 7% interrupts.CPU9.RES:Rescheduling_interrupts
5.00 ± 69% +4020.0% 206.00 ± 3% interrupts.CPU9.TLB:TLB_shootdowns
46658 ± 7% +22.5% 57148 ± 2% interrupts.NMI:Non-maskable_interrupts
46658 ± 7% +22.5% 57148 ± 2% interrupts.PMI:Performance_monitoring_interrupts
23433 +3535.9% 852037 interrupts.RES:Rescheduling_interrupts
76.33 ± 10% +4317.5% 3372 interrupts.TLB:TLB_shootdowns
0.03 ± 5% -13.8% 0.03 ± 5% perf-sched.sch_delay.avg.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
0.08 ± 8% +65.6% 0.13 ± 13% perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.10 ± 9% +104.2% 0.20 ± 6% perf-sched.sch_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.03 ±144% -86.5% 0.00 ± 59% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
0.11 ± 68% -87.6% 0.01 ±147% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__dentry_kill.dput
0.20 ± 31% -45.1% 0.11 ± 19% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.filemap_map_pages.do_fault
0.18 ± 37% -87.1% 0.02 ± 34% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.zap_pte_range.unmap_page_range
0.00 ±223% +4738.1% 0.15 ± 76% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.wait_task_zombie.do_wait
0.01 ± 52% -88.3% 0.00 ±119% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.sched_fork.copy_process
0.01 ± 45% +1315.8% 0.20 ± 28% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_write_unlock_irq.release_task.wait_task_zombie
1.43 ± 15% -29.3% 1.01 ± 16% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk.schedule_tail.ret_from_fork.[unknown]
0.15 ±139% -94.7% 0.01 ± 81% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.fs_reclaim_acquire
0.25 ±110% -99.3% 0.00 ±198% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.kmem_cache_alloc
0.02 ± 46% -65.2% 0.01 ±121% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.memcg_slab_free_hook
0.03 ±139% -95.8% 0.00 ±148% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.kmem_cache_alloc
0.04 ±146% -96.5% 0.00 ± 68% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.get_obj_cgroup_from_current.kmem_cache_alloc
0.03 ± 98% -96.5% 0.00 ±106% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.__alloc_file
0.00 ±223% +4308.2% 0.05 ±131% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__alloc_file
0.01 ±187% +2343.6% 0.22 ±118% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__d_alloc
0.01 ±196% +900.3% 0.14 ± 95% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.vm_area_alloc
0.00 ±223% +1361.7% 0.05 ± 86% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.vm_area_dup
0.01 ± 29% -63.5% 0.00 ± 39% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
0.42 ± 73% -77.3% 0.10 ± 96% perf-sched.sch_delay.avg.ms.wait_for_partner.fifo_open.do_dentry_open.do_open.isra
0.02 ± 18% -23.9% 0.01 ± 3% perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
0.33 ±215% +1242.2% 4.44 ±109% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_read_unlock.wait_task_zombie.do_wait
0.37 ±203% +1154.6% 4.62 ± 79% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.do_wp_page.__handle_mm_fault
1.77 ± 72% +456.3% 9.84 ± 52% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.wp_page_copy.__handle_mm_fault
0.00 ±223% +74295.2% 2.23 ± 97% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.wait_task_zombie.do_wait
0.01 ± 45% +7769.2% 1.15 ± 83% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.delete_from_page_cache.truncate_inode_page
0.03 ± 76% -75.0% 0.01 ± 98% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.sched_fork.copy_process
0.01 ± 46% +54592.7% 7.84 ± 41% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_write_unlock_irq.release_task.wait_task_zombie
0.04 ± 13% +4067.1% 1.85 ± 70% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk.lru_add_drain.unmap_region.__do_munmap
0.00 ±142% +19789.1% 0.83 ±223% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__alloc_pages.alloc_pages_vma
0.01 ± 88% +22526.4% 1.47 ±165% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__handle_mm_fault.handle_mm_fault
3.84 ±115% +289.2% 14.94 ± 43% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.do_user_addr_fault.exc_page_fault
0.02 ±127% +12104.3% 2.01 ±133% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.exit_to_user_mode_prepare.syscall_exit_to_user_mode
0.29 ±184% +1499.9% 4.61 ± 27% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.handle_mm_fault.do_user_addr_fault
0.00 ±223% +4871.4% 0.06 ±123% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.get_mem_cgroup_from_mm
1.75 ±126% -88.3% 0.20 ±208% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.kmem_cache_alloc
0.00 ±141% +5568.6% 0.14 ±141% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release._copy_from_user
0.02 ±116% +7531.1% 1.74 ±116% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.rmqueue.get_page_from_freelist
0.01 ±223% +63806.9% 3.30 ±157% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.kmem_cache_free.___pmd_free_tlb.free_p4d_range
0.00 ±141% +1968.6% 0.05 ± 79% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.filemap_map_pages.do_fault
0.00 ±223% +1e+05% 1.22 ±129% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__alloc_file
0.01 ±187% +63923.9% 5.87 ±110% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__d_alloc
0.01 ±102% +3193.6% 0.26 ±165% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_fork
0.01 ±196% +24750.1% 3.52 ± 84% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.vm_area_alloc
0.00 ±223% +78405.3% 2.49 ± 99% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.vm_area_dup
0.42 ±217% +1802.0% 8.00 ± 67% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.memcg_slab_free_hook.kmem_cache_free
12.74 ± 72% -81.7% 2.34 ±140% perf-sched.sch_delay.max.ms.wait_for_partner.fifo_open.do_dentry_open.do_open.isra
2.94 +99.6% 5.88 ± 31% perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
0.09 ± 3% -60.2% 0.04 ± 4% perf-sched.total_sch_delay.average.ms
50.98 -85.8% 7.23 ± 3% perf-sched.total_wait_and_delay.average.ms
16846 +625.7% 122252 ± 3% perf-sched.total_wait_and_delay.count.ms
7245 -53.2% 3392 ± 28% perf-sched.total_wait_and_delay.max.ms
50.90 -85.9% 7.20 ± 3% perf-sched.total_wait_time.average.ms
7245 -53.2% 3392 ± 28% perf-sched.total_wait_time.max.ms
15.26 ± 7% -24.8% 11.47 ± 9% perf-sched.wait_and_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
11.02 ±110% -92.6% 0.82 ± 42% perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
681.03 ± 3% -73.2% 182.59 ± 11% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
231.38 -16.0% 194.27 perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
132.26 ± 2% -98.4% 2.08 ± 4% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
128.00 ± 12% +3429.0% 4517 ± 16% perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
139.00 ± 6% +848.6% 1318 ± 4% perf-sched.wait_and_delay.count.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.filemap_map_pages.do_fault
175.83 ± 7% +1452.4% 2729 ± 8% perf-sched.wait_and_delay.count.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.zap_pte_range.unmap_page_range
0.83 ±223% +15140.0% 127.00 ± 33% perf-sched.wait_and_delay.count.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.shmem_add_to_page_cache.shmem_getpage_gfp
26.33 ± 4% +139.8% 63.14 ± 5% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
1299 +21.2% 1573 ± 2% perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
1194 ± 2% +6800.0% 82420 ± 3% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
15.11 ± 45% +4686.8% 723.15 ± 61% perf-sched.wait_and_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.zap_pte_range.unmap_page_range
69.89 ±223% +2717.9% 1969 ± 32% perf-sched.wait_and_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.shmem_add_to_page_cache.shmem_getpage_gfp
7245 -80.2% 1432 ± 32% perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
15.16 ± 7% -25.7% 11.27 ± 9% perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
10.99 ±110% -92.6% 0.82 ± 42% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
3.02 ± 46% +216.9% 9.58 ± 79% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_read_unlock.wait_task_zombie.do_wait
0.01 ±112% +242.9% 0.02 ± 56% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__d_lookup.lookup_fast
0.00 ±223% +3292.9% 0.07 ±133% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__pmd_alloc.__handle_mm_fault
0.00 ±223% +8.2e+05% 24.46 ±238% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__pmd_alloc.copy_page_range
0.01 ±223% +15653.9% 1.16 ± 95% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.do_sigaction.__x64_sys_rt_sigaction
0.48 ±223% +1187.2% 6.13 ± 20% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.wait_task_zombie.do_wait
0.04 ± 58% +428.4% 0.20 ± 73% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.__wake_up_common_lock.pipe_write
0.00 ±223% +2.6e+05% 7.31 ±131% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.release_pages.__pagevec_release
2.11 ± 54% -81.6% 0.39 ±149% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.sched_fork.copy_process
0.54 ±223% +809.0% 4.87 ± 34% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.thread_group_cputime_adjusted.wait_task_zombie
0.00 ±223% +17141.8% 0.37 ±234% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.unix_stream_read_generic.unix_stream_recvmsg
2.45 ± 53% -94.2% 0.14 ± 66% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.kernel_init_free_pages.prep_new_page.get_page_from_freelist
2.71 ± 80% -97.4% 0.07 ± 88% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.perf_iterate_sb.perf_event_mmap.do_brk_flags
0.00 ±223% +13726.4% 0.30 ±207% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.shmem_write_end.generic_perform_write.__generic_file_write_iter
0.01 ± 92% +10275.6% 0.71 ±145% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__d_lookup_rcu.lookup_fast
0.00 ±223% +892.9% 0.02 ± 38% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__get_user_8.get_user_arg_ptr
0.00 ±223% +1422.3% 0.06 ±115% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__memcg_kmem_charge_page.__alloc_pages
0.00 ±144% +4191.6% 0.21 ± 81% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__might_sleep.__might_fault
0.00 ±223% +1490.5% 0.02 ± 64% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__rcu_read_unlock.memcg_slab_free_hook
0.00 ±223% +15559.3% 0.34 ±212% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.alloc_pages_vma.do_anonymous_page
0.00 ±223% +2156.0% 0.09 ± 96% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.call_rcu.task_work_run
0.00 ±223% +895.9% 0.03 ± 56% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.cgroup_throttle_swaprate.do_anonymous_page
0.00 ±223% +538.1% 0.02 ± 22% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.d_path.perf_event_mmap
0.01 ±223% +1628.2% 0.23 ± 83% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.debug_lockdep_rcu_enabled.___might_sleep
0.00 ±223% +9085.7% 0.18 ±139% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.down_read_trylock.do_user_addr_fault
0.03 ±199% +843.3% 0.28 ± 58% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.exit_to_user_mode_prepare
0.00 ±223% +836.7% 0.02 ± 24% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.fs_reclaim_release.__alloc_pages
0.01 ±223% +2452.4% 0.15 ±150% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_alloc.__d_alloc
0.02 ±223% +9193.2% 2.25 ±187% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_free.do_sys_openat2
0.00 ±150% +518.6% 0.02 ± 23% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_free.unlink_anon_vmas
0.00 ±223% +15682.1% 0.21 ±210% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.link_path_walk.part
0.00 ±223% +1210.9% 0.04 ± 79% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.load_elf_binary.exec_binprm
0.00 ±223% +1509.5% 0.02 ± 62% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.__memcg_kmem_charge_page
0.00 ±223% +19029.4% 0.54 ± 70% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.aa_file_perm
0.00 ±106% +2440.8% 0.06 ±121% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.memcg_slab_free_hook
0.00 ±223% +7689.3% 0.10 ±181% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.mntput_no_expire
0.04 ±223% +4078.3% 1.57 ±158% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.xas_start
0.00 ±223% +5391.4% 0.14 ±165% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.handle_mm_fault
0.00 ±223% +31278.0% 0.68 ± 90% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.refill_obj_stock.obj_cgroup_charge
0.04 ±148% +1296.7% 0.54 ± 42% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.set_root.nd_jump_root
0.00 ±223% +91500.0% 1.83 ±117% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.step_into.walk_component
0.00 ±223% +660.0% 0.02 ± 40% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.uncharge_page.mem_cgroup_uncharge_list
0.00 ±223% +1937.4% 0.04 ± 81% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.vm_area_dup.__split_vma
9.63 ±179% -97.1% 0.28 ± 34% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.__perf_sw_event.handle_mm_fault.do_user_addr_fault
3.30 ± 44% -96.7% 0.11 ± 98% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.__alloc_pages.alloc_pages_vma
3.10 ± 78% -93.5% 0.20 ± 93% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.get_mem_cgroup_from_mm.mem_cgroup_charge
0.00 ±223% +33366.7% 0.50 ±138% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.security_file_alloc
0.00 ±223% +602.9% 0.01 ± 45% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.mntput_no_expire.terminate_walk
0.01 ±146% +1014.3% 0.08 ±120% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.prepend_path.isra
0.00 ±223% +7828.6% 0.13 ±215% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.uncharge_page.mem_cgroup_uncharge_list
0.00 ±223% +1366.7% 0.04 ±142% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.unmap_vmas.unmap_region
0.00 ±223% +29440.8% 0.69 ±154% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.up_write.anon_vma_clone
0.00 ±223% +47480.0% 0.79 ±199% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.memcg_slab_free_hook.kmem_cache_free
681.02 ± 3% -73.2% 182.59 ± 11% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
231.35 -16.0% 194.25 perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
132.25 ± 2% -98.4% 2.07 ± 4% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
1.08 ±222% +633.8% 7.96 ± 66% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
0.21 ±191% +72062.8% 152.74 ±227% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.50 ± 59% +4625.6% 165.16 ±206% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_read_unlock.wait_task_zombie.do_wait
0.01 ±104% +619.5% 0.05 ± 98% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__d_lookup.lookup_fast
0.00 ±223% +17007.1% 0.68 ±166% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__pmd_alloc.__handle_mm_fault
0.00 ±223% +4.9e+06% 147.41 ±237% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__pmd_alloc.copy_page_range
16.77 ± 40% +5900.4% 1006 perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.do_anonymous_page.__handle_mm_fault
5.04 ± 66% +227.0% 16.48 ± 33% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.do_wp_page.__handle_mm_fault
0.08 ±223% +63355.5% 48.44 ±174% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.finish_fault.do_fault
0.84 ± 75% +68794.5% 577.34 ± 85% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.wp_page_copy.__handle_mm_fault
14.63 ± 47% +4838.6% 722.46 ± 61% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.zap_pte_range.unmap_page_range
0.01 ±223% +1.3e+05% 9.24 ± 62% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.do_sigaction.__x64_sys_rt_sigaction
74.80 ±206% +2533.1% 1969 ± 32% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.shmem_add_to_page_cache.shmem_getpage_gfp
0.48 ±223% +330.0% 2.04 ± 83% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.signal_setup_done.arch_do_signal_or_restart
1.09 ±223% +876.2% 10.59 ± 74% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.sigprocmask.__x64_sys_rt_sigprocmask
0.48 ±223% +4611.7% 22.44 ± 27% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.wait_task_zombie.do_wait
6.55 ± 55% +2355.2% 160.79 ±213% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.delete_from_page_cache.truncate_inode_page
1.40 ±223% +781.2% 12.31 ± 60% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.do_wait.kernel_wait4
2.33 ±197% +592.2% 16.16 ± 26% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.rcu_preempt_deferred_qs_irqrestore.__rcu_read_unlock
0.00 ±223% +1.1e+07% 297.91 ±149% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.release_pages.__pagevec_release
5.53 ± 50% +10419.9% 582.14 ± 83% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.release_pages.tlb_flush_mmu
0.54 ±223% +1856.2% 10.48 ± 51% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.thread_group_cputime_adjusted.wait_task_zombie
0.00 ±223% +51275.8% 1.11 ±235% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.unix_stream_read_generic.unix_stream_recvmsg
6.84 ± 45% +288.2% 26.57 ± 15% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_write_unlock_irq.release_task.wait_task_zombie
4.43 ±132% +236.2% 14.88 ± 37% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.cpu_stop_queue_work.stop_one_cpu.sched_exec
1.13 ±108% +1162.3% 14.32 ± 44% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.lru_add_drain.exit_mmap.mmput
4.43 ± 70% +21666.4% 964.76 ± 61% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.lru_cache_add.shmem_getpage_gfp.shmem_write_begin
0.01 ±143% +22411.0% 1.31 ± 84% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.schedule_tail.ret_from_fork.[unknown]
0.00 ±223% +51236.3% 1.11 ±224% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.shmem_write_end.generic_perform_write.__generic_file_write_iter
5.58 ± 74% +2113.2% 123.51 ±210% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.handle_mm_fault.do_user_addr_fault
0.00 ±223% +2078.6% 0.09 ± 71% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__get_user_8.get_user_arg_ptr
1.22 ±223% +12415.0% 152.35 ±228% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__handle_mm_fault.handle_mm_fault
0.11 ±175% +3.5e+05% 400.77 ±163% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__mem_cgroup_charge.mem_cgroup_charge
0.00 ±223% +10645.1% 0.45 ±160% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__memcg_kmem_charge_page.__alloc_pages
0.01 ±144% +64806.6% 4.44 ± 96% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__might_sleep.__might_fault
0.00 ±223% +2061.9% 0.06 ± 87% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__rcu_read_unlock.memcg_slab_free_hook
0.00 ±223% +31586.9% 1.32 ±217% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.alloc_pages_vma.do_anonymous_page
0.01 ±223% +12325.1% 1.01 ±111% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.call_rcu.task_work_run
0.00 ±223% +3996.4% 0.16 ±119% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.cgroup_throttle_swaprate.do_anonymous_page
0.00 ±223% +6485.7% 0.26 ±102% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.d_alloc_parallel.__lookup_slow
0.00 ±223% +2323.8% 0.07 ± 41% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.d_path.perf_event_mmap
0.02 ±223% +24760.7% 6.05 ± 84% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.debug_lockdep_rcu_enabled.___might_sleep
5.79 ± 45% +2650.3% 159.25 ±215% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.do_user_addr_fault.exc_page_fault
0.00 ±223% +38532.1% 1.55 ±163% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.down_read_trylock.do_user_addr_fault
0.08 ±214% +7364.1% 5.98 ± 65% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.exit_to_user_mode_prepare
4.99 ± 59% +2975.2% 153.54 ±226% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.exit_to_user_mode_prepare.irqentry_exit_to_user_mode
1.13 ±195% +1365.8% 16.53 ± 49% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.exit_to_user_mode_prepare.syscall_exit_to_user_mode
0.01 ±223% +1592.0% 0.14 ±125% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.fs_reclaim_acquire.kmem_cache_alloc
0.00 ±223% +5018.4% 0.12 ± 52% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.fs_reclaim_release.__alloc_pages
1.39 ±184% +1210.9% 18.17 ± 47% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.handle_mm_fault.do_user_addr_fault
0.00 ±223% +3631.1% 0.11 ±104% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_alloc.__anon_vma_prepare
0.02 ±223% +13668.3% 2.48 ±158% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_alloc.__d_alloc
0.01 ±186% +18694.4% 2.69 ±150% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_alloc.anon_vma_fork
0.07 ±223% +6497.8% 4.77 ± 93% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_free.do_sys_openat2
0.01 ±147% +3742.1% 0.26 ± 68% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_free.unlink_anon_vmas
2.35 ±121% +6566.9% 156.56 ±219% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.ktime_get_coarse_real_ts64.current_time
0.00 ±223% +28316.0% 1.18 ±225% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.link_path_walk.part
0.00 ±223% +4271.4% 0.12 ± 80% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.load_elf_binary.exec_binprm
0.00 ±223% +5105.5% 0.23 ±135% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.__memcg_kmem_charge_page
0.79 ±175% +15699.4% 124.95 ±209% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.__might_fault
0.00 ±223% +2.5e+05% 7.17 ± 71% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.aa_file_perm
0.17 ±223% +3438.4% 6.19 ± 48% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.down_read_trylock
1.37 ±170% +371.7% 6.46 ± 45% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.down_write
0.01 ±188% +27098.9% 3.54 ±121% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.get_mem_cgroup_from_objcg
1.22 ±220% +947.0% 12.73 ± 97% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.handle_mm_fault
0.01 ±117% +37573.5% 3.96 ±178% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.memcg_slab_free_hook
0.00 ±223% +55391.4% 1.39 ±206% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.mntput_no_expire
1.15 ±223% +892.5% 11.38 ± 78% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.pagecache_get_page
2.42 ±166% +452.4% 13.35 ± 60% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.thread_group_cputime
1.74 ±139% +863.3% 16.77 ± 46% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.___might_sleep
0.07 ±223% +25515.2% 19.17 ±154% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.xas_start
0.00 ±223% +1.4e+05% 3.51 ±203% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.handle_mm_fault
0.21 ±112% +3319.5% 7.11 ± 70% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.kmem_cache_alloc
0.05 ±223% +2412.6% 1.37 ±136% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.memcg_slab_free_hook
0.08 ±223% +3681.8% 3.08 ±136% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.mem_cgroup_from_task.handle_mm_fault
0.01 ±223% +1502.6% 0.15 ± 85% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.next_uptodate_page.filemap_map_pages
0.17 ±149% +3739.4% 6.53 ±153% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.refill_obj_stock.memcg_slab_free_hook
0.00 ±223% +1.9e+05% 7.75 ±101% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.refill_obj_stock.obj_cgroup_charge
0.08 ±182% +14050.5% 11.93 ± 46% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.set_root.nd_jump_root
0.00 ±223% +1.5e+05% 6.19 ±152% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.step_into.walk_component
2.32 ±120% +6531.1% 154.11 ±224% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.syscall_enter_from_user_mode.do_syscall_64
1.00 ±223% +365.8% 4.66 ± 93% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.uncharge_batch.mem_cgroup_uncharge_list
0.00 ±223% +2414.3% 0.06 ± 88% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.uncharge_page.mem_cgroup_uncharge_list
0.02 ±223% +3580.9% 0.76 ±144% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.unlink_anon_vmas.free_pgtables
0.00 ±223% +6717.6% 0.15 ±102% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.vm_area_dup.__split_vma
1.30 ±209% +559.4% 8.60 ± 73% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.vmacache_find.find_vma
0.01 ±162% +3435.1% 0.41 ±120% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.kfree.perf_event_mmap.do_brk_flags
0.00 ±223% +620.0% 0.03 ± 48% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.get_mem_cgroup_from_objcg.uncharge_page
0.01 ±141% +45293.8% 2.42 ±237% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.get_mem_cgroup_from_objcg.obj_cgroup_charge_pages
2.50 ±142% +219.9% 8.00 ± 71% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.handle_mm_fault.do_user_addr_fault
0.08 ±223% +2018.2% 1.68 ±135% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.__d_alloc
0.00 ±223% +84204.8% 2.53 ±126% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.security_file_alloc
0.00 ±223% +4519.0% 0.21 ±101% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.vm_area_alloc
0.00 ±223% +784.2% 0.03 ± 61% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.mntput_no_expire.terminate_walk
0.02 ±180% +2149.5% 0.34 ±134% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.prepend_path.isra
0.00 ±223% +29252.9% 0.98 ±232% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.uncharge_page.mem_cgroup_uncharge_list
0.00 ±223% +3247.6% 0.10 ±174% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.unmap_vmas.unmap_region
0.00 ±223% +1.3e+05% 2.98 ±163% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.up_write.anon_vma_clone
1.24 ±205% +12646.0% 157.82 ±219% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_clone
3.42 ± 93% +8491.7% 293.49 ±152% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_fork
1.01 ±222% +29628.1% 300.90 ±149% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.vm_area_dup
0.01 ±223% +3e+06% 153.73 ±225% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.memcg_slab_free_hook.kmem_cache_free
7245 -80.2% 1432 ± 32% perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
17.95 -4.6 13.35 perf-profile.calltrace.cycles-pp.sieve
6.02 ± 2% -1.6 4.38 ± 2% perf-profile.calltrace.cycles-pp.string_rtns_1
9.11 -1.2 7.86 perf-profile.calltrace.cycles-pp.page_test
8.75 -1.2 7.55 perf-profile.calltrace.cycles-pp.asm_exc_page_fault.page_test
8.58 -1.1 7.44 perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault.page_test
8.44 -1.1 7.34 perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.page_test
7.42 -0.9 6.55 perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.page_test
3.10 ± 3% -0.8 2.30 ± 2% perf-profile.calltrace.cycles-pp.add_int.add_int
6.73 ± 2% -0.8 5.93 perf-profile.calltrace.cycles-pp.brk
3.16 ± 4% -0.8 2.36 ± 2% perf-profile.calltrace.cycles-pp.__strncat_chk
3.05 ± 3% -0.8 2.25 ± 3% perf-profile.calltrace.cycles-pp.add_long.add_long
6.36 ± 2% -0.7 5.62 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.brk
6.34 ± 2% -0.7 5.60 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk
5.95 ± 2% -0.6 5.31 perf-profile.calltrace.cycles-pp.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk
2.23 ± 2% -0.6 1.61 ± 2% perf-profile.calltrace.cycles-pp.div_long
2.50 ± 7% -0.6 1.91 ± 9% perf-profile.calltrace.cycles-pp.write
2.49 ± 7% -0.6 1.90 ± 9% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
2.48 ± 7% -0.6 1.90 ± 9% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
2.39 ± 7% -0.6 1.83 ± 9% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
2.38 ± 8% -0.6 1.82 ± 9% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
2.08 ± 9% -0.5 1.59 ± 10% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.85 ± 3% -0.5 1.37 ± 3% perf-profile.calltrace.cycles-pp.__strcat_chk
1.80 ± 5% -0.5 1.32 ± 3% perf-profile.calltrace.cycles-pp.add_short.add_short
1.15 ± 14% -0.4 0.73 ± 13% perf-profile.calltrace.cycles-pp.drm_fb_helper_damage_work.process_one_work.worker_thread.kthread.ret_from_fork
1.12 ± 14% -0.4 0.72 ± 13% perf-profile.calltrace.cycles-pp.memcpy_toio.drm_fb_helper_damage_work.process_one_work.worker_thread.kthread
2.62 ± 9% -0.4 2.24 ± 10% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
2.72 ± 9% -0.4 2.35 ± 10% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
6.14 ± 2% -0.4 5.79 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.execve
6.14 ± 2% -0.4 5.79 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
6.14 ± 2% -0.3 5.79 perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
6.18 ± 2% -0.3 5.84 perf-profile.calltrace.cycles-pp.execve
6.08 ± 2% -0.3 5.74 perf-profile.calltrace.cycles-pp.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
1.42 ± 10% -0.3 1.10 ± 5% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
1.64 ± 5% -0.3 1.32 ± 3% perf-profile.calltrace.cycles-pp.do_brk_flags.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk
1.86 ± 10% -0.3 1.55 ± 9% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
1.77 ± 10% -0.3 1.46 ± 9% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
3.24 ± 2% -0.3 2.96 ± 9% perf-profile.calltrace.cycles-pp.lock_page_memcg.page_remove_rmap.zap_pte_range.unmap_page_range.unmap_vmas
0.74 ± 8% -0.3 0.48 ± 41% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
0.65 ± 7% -0.3 0.39 ± 63% perf-profile.calltrace.cycles-pp.shmem_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write
0.57 ± 5% -0.3 0.31 ± 86% perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.elf_map.load_elf_binary.exec_binprm.bprm_execve
0.65 ± 7% -0.3 0.39 ± 63% perf-profile.calltrace.cycles-pp.shmem_getpage_gfp.shmem_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
0.92 ± 10% -0.3 0.67 ± 8% perf-profile.calltrace.cycles-pp.div_int
1.00 ± 4% -0.2 0.77 ± 9% perf-profile.calltrace.cycles-pp.div_short
3.74 ± 2% -0.2 3.53 perf-profile.calltrace.cycles-pp.__do_munmap.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk
0.98 ± 6% -0.2 0.77 ± 5% perf-profile.calltrace.cycles-pp.generic_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
0.95 ± 4% -0.2 0.75 ± 5% perf-profile.calltrace.cycles-pp.get_mem_cgroup_from_mm.mem_cgroup_charge.do_anonymous_page.__handle_mm_fault.handle_mm_fault
0.87 ± 6% -0.2 0.68 ± 6% perf-profile.calltrace.cycles-pp.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write.ksys_write
1.50 ± 3% -0.2 1.33 ± 3% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.begin_new_exec
1.51 ± 3% -0.2 1.35 ± 3% perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.begin_new_exec.load_elf_binary
0.92 ± 6% -0.2 0.76 ± 4% perf-profile.calltrace.cycles-pp.__xstat64
0.88 ± 6% -0.2 0.73 ± 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__xstat64
0.88 ± 6% -0.2 0.73 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__xstat64
0.85 ± 6% -0.2 0.70 ± 3% perf-profile.calltrace.cycles-pp.__do_sys_newstat.do_syscall_64.entry_SYSCALL_64_after_hwframe.__xstat64
0.82 ± 5% -0.2 0.67 ± 5% perf-profile.calltrace.cycles-pp.perf_event_mmap.do_brk_flags.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.74 ± 7% -0.1 0.60 ± 5% perf-profile.calltrace.cycles-pp.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write
0.82 ± 7% -0.1 0.67 ± 4% perf-profile.calltrace.cycles-pp.vfs_statx.__do_sys_newstat.do_syscall_64.entry_SYSCALL_64_after_hwframe.__xstat64
1.52 ± 2% -0.1 1.37 ± 2% perf-profile.calltrace.cycles-pp.unmap_vmas.unmap_region.__do_munmap.__x64_sys_brk.do_syscall_64
1.49 ± 2% -0.1 1.35 ± 2% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.unmap_region.__do_munmap.__x64_sys_brk
1.46 ± 2% -0.1 1.34 ± 2% perf-profile.calltrace.cycles-pp.zap_pte_range.unmap_page_range.unmap_vmas.unmap_region.__do_munmap
1.20 ± 2% -0.1 1.12 ± 2% perf-profile.calltrace.cycles-pp.page_remove_rmap.zap_pte_range.unmap_page_range.unmap_vmas.unmap_region
0.63 ± 3% +0.1 0.70 ± 7% perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.94 ± 3% +0.1 2.07 ± 3% perf-profile.calltrace.cycles-pp.filemap_map_pages.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
2.55 ± 2% +0.1 2.69 perf-profile.calltrace.cycles-pp.dup_mmap.dup_mm.copy_process.kernel_clone.__do_sys_clone
2.10 ± 3% +0.1 2.24 ± 2% perf-profile.calltrace.cycles-pp.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
0.68 ± 2% +0.1 0.82 ± 6% perf-profile.calltrace.cycles-pp.vfs_statx.__do_sys_newstat.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.60 ± 2% +0.2 2.75 ± 2% perf-profile.calltrace.cycles-pp.dup_mm.copy_process.kernel_clone.__do_sys_clone.do_syscall_64
0.68 ± 2% +0.2 0.83 ± 6% perf-profile.calltrace.cycles-pp.__do_sys_newstat.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.15 +0.2 3.31 ± 2% perf-profile.calltrace.cycles-pp.copy_process.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.11 ± 2% +0.2 2.31 ± 6% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.do_exit
2.12 ± 2% +0.2 2.32 ± 6% perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.do_exit.do_group_exit
2.81 +0.4 3.24 ± 4% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group
2.81 +0.4 3.25 ± 4% perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
3.20 ± 2% +0.4 3.64 ± 4% perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.20 ± 2% +0.4 3.64 ± 4% perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.20 ± 2% +0.4 3.64 ± 4% perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.09 ± 4% +0.5 2.57 ± 3% perf-profile.calltrace.cycles-pp.mem_cgroup_charge.do_anonymous_page.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
0.00 +0.5 0.54 ± 5% perf-profile.calltrace.cycles-pp.page_add_new_anon_rmap.do_anonymous_page.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
0.00 +0.6 0.57 ± 8% perf-profile.calltrace.cycles-pp.lock_acquire._raw_spin_lock.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats
4.68 ± 2% +0.6 5.26 perf-profile.calltrace.cycles-pp.load_elf_binary.exec_binprm.bprm_execve.do_execveat_common.__x64_sys_execve
0.00 +0.6 0.59 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work
4.91 ± 2% +0.6 5.51 perf-profile.calltrace.cycles-pp.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.71 ± 2% +0.6 5.32 perf-profile.calltrace.cycles-pp.exec_binprm.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64
0.00 +0.7 0.66 ± 6% perf-profile.calltrace.cycles-pp.__schedule.schedule.worker_thread.kthread.ret_from_fork
0.00 +0.7 0.68 ± 6% perf-profile.calltrace.cycles-pp.schedule.worker_thread.kthread.ret_from_fork
0.00 +0.8 0.76 ± 6% perf-profile.calltrace.cycles-pp.lock_acquire.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work
3.15 ± 3% +0.9 4.01 perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.98 +1.0 7.96 ± 2% perf-profile.calltrace.cycles-pp.do_anonymous_page.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
0.00 +1.0 0.98 ± 3% perf-profile.calltrace.cycles-pp.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +1.0 1.00 ± 3% perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +1.1 1.14 ± 4% perf-profile.calltrace.cycles-pp.__do_munmap.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe
7.31 ± 4% +1.2 8.46 ± 7% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
7.36 ± 4% +1.2 8.53 ± 7% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
9.76 +1.2 10.93 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
8.08 ± 5% +1.5 9.59 ± 8% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
8.09 ± 5% +1.5 9.60 ± 8% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
8.09 ± 5% +1.5 9.60 ± 8% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
0.00 +1.6 1.57 ± 6% perf-profile.calltrace.cycles-pp.mem_cgroup_css_rstat_flush.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work
4.27 ± 2% +1.6 5.90 ± 4% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
8.61 ± 3% +1.7 10.27 ± 7% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
0.00 +1.7 1.69 ± 2% perf-profile.calltrace.cycles-pp.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.10 ± 3% +2.4 6.54 perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
4.55 ± 3% +2.7 7.26 ± 2% perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
4.60 ± 3% +2.8 7.35 ± 2% perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault
4.68 ± 3% +2.8 7.47 ± 2% perf-profile.calltrace.cycles-pp.asm_exc_page_fault
0.00 +4.1 4.07 ± 3% perf-profile.calltrace.cycles-pp.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work.worker_thread
0.00 +4.1 4.12 ± 3% perf-profile.calltrace.cycles-pp.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work.worker_thread.kthread
1.18 ± 14% +4.2 5.38 ± 2% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
0.00 +4.3 4.31 ± 3% perf-profile.calltrace.cycles-pp.mem_cgroup_flush_stats.process_one_work.worker_thread.kthread.ret_from_fork
9.40 +4.6 13.98 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
9.41 +4.6 13.99 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
1.19 ± 14% +5.0 6.15 ± 2% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
1.44 ± 11% +5.0 6.45 ± 2% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
1.45 ± 11% +5.0 6.46 ± 2% perf-profile.calltrace.cycles-pp.ret_from_fork
17.96 -4.6 13.36 perf-profile.children.cycles-pp.sieve
6.05 ± 2% -1.6 4.40 ± 2% perf-profile.children.cycles-pp.string_rtns_1
9.36 -1.3 8.06 perf-profile.children.cycles-pp.page_test
6.75 ± 2% -0.8 5.94 perf-profile.children.cycles-pp.brk
3.10 ± 3% -0.8 2.30 ± 2% perf-profile.children.cycles-pp.add_int
3.17 ± 4% -0.8 2.36 ± 2% perf-profile.children.cycles-pp.__strncat_chk
3.05 ± 3% -0.8 2.26 ± 3% perf-profile.children.cycles-pp.add_long
2.23 ± 2% -0.6 1.61 ± 2% perf-profile.children.cycles-pp.div_long
2.51 ± 7% -0.6 1.92 ± 9% perf-profile.children.cycles-pp.write
1.86 ± 3% -0.5 1.37 ± 2% perf-profile.children.cycles-pp.__strcat_chk
1.80 ± 5% -0.5 1.32 ± 3% perf-profile.children.cycles-pp.add_short
1.15 ± 14% -0.4 0.73 ± 13% perf-profile.children.cycles-pp.drm_fb_helper_damage_work
1.14 ± 14% -0.4 0.73 ± 13% perf-profile.children.cycles-pp.memcpy_toio
6.19 ± 2% -0.3 5.84 perf-profile.children.cycles-pp.execve
0.93 ± 10% -0.3 0.67 ± 8% perf-profile.children.cycles-pp.div_int
2.19 ± 5% -0.2 1.96 ± 4% perf-profile.children.cycles-pp.__hrtimer_run_queues
1.00 ± 4% -0.2 0.77 ± 9% perf-profile.children.cycles-pp.div_short
2.93 ± 6% -0.2 2.72 ± 5% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
2.78 ± 5% -0.2 2.58 ± 4% perf-profile.children.cycles-pp.hrtimer_interrupt
0.93 ± 5% -0.2 0.76 ± 4% perf-profile.children.cycles-pp.__xstat64
0.44 ± 9% -0.1 0.33 ± 8% perf-profile.children.cycles-pp.semop
0.57 ± 8% -0.1 0.46 ± 4% perf-profile.children.cycles-pp.mem_rtns_1
0.24 ± 5% -0.1 0.19 ± 11% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.18 ± 11% -0.0 0.13 ± 17% perf-profile.children.cycles-pp.strnlen_user
0.21 ± 4% -0.0 0.17 ± 8% perf-profile.children.cycles-pp.unlink
0.08 ± 14% -0.0 0.04 ± 64% perf-profile.children.cycles-pp.load_balance
0.30 ± 5% -0.0 0.26 ± 4% perf-profile.children.cycles-pp.count
0.06 ± 8% +0.0 0.07 ± 6% perf-profile.children.cycles-pp.__delete_from_page_cache
0.08 ± 5% +0.0 0.10 ± 9% perf-profile.children.cycles-pp.truncate_inode_page
0.08 ± 10% +0.0 0.10 ± 6% perf-profile.children.cycles-pp.delete_from_page_cache
0.07 ± 17% +0.0 0.09 ± 9% perf-profile.children.cycles-pp.wait_task_zombie
0.23 ± 5% +0.0 0.26 ± 7% perf-profile.children.cycles-pp.__get_user_pages_remote
0.06 ± 11% +0.0 0.09 ± 10% perf-profile.children.cycles-pp.__pagevec_release
0.06 ± 11% +0.0 0.09 ± 16% perf-profile.children.cycles-pp.__tlb_remove_page_size
0.23 ± 5% +0.0 0.26 ± 6% perf-profile.children.cycles-pp.__get_user_pages
0.08 ± 16% +0.0 0.11 ± 11% perf-profile.children.cycles-pp.alloc_bprm
0.20 ± 5% +0.0 0.24 ± 4% perf-profile.children.cycles-pp.shmem_truncate_range
0.23 ± 4% +0.0 0.27 ± 6% perf-profile.children.cycles-pp.get_arg_page
0.24 ± 6% +0.0 0.27 ± 5% perf-profile.children.cycles-pp.shmem_add_to_page_cache
0.21 ± 4% +0.0 0.24 ± 5% perf-profile.children.cycles-pp.shmem_evict_inode
0.10 ± 13% +0.0 0.14 ± 7% perf-profile.children.cycles-pp.free_p4d_range
0.21 ± 4% +0.0 0.25 ± 6% perf-profile.children.cycles-pp.evict
0.20 ± 4% +0.0 0.24 ± 5% perf-profile.children.cycles-pp.shmem_undo_range
0.11 ± 14% +0.0 0.15 ± 11% perf-profile.children.cycles-pp.update_rq_clock
0.04 ± 71% +0.0 0.07 ± 9% perf-profile.children.cycles-pp._raw_spin_trylock
0.10 ± 13% +0.0 0.14 ± 7% perf-profile.children.cycles-pp.free_pgd_range
0.08 ± 20% +0.0 0.12 ± 16% perf-profile.children.cycles-pp.lock_pin_lock
0.12 ± 18% +0.0 0.16 ± 5% perf-profile.children.cycles-pp.shift_arg_pages
0.12 ± 14% +0.0 0.16 ± 13% perf-profile.children.cycles-pp.uncharge_batch
0.24 ± 5% +0.0 0.28 ± 7% perf-profile.children.cycles-pp.smpboot_thread_fn
0.02 ±141% +0.0 0.06 ± 8% perf-profile.children.cycles-pp.unaccount_page_cache_page
0.14 ± 12% +0.0 0.19 ± 6% perf-profile.children.cycles-pp.setup_arg_pages
0.19 ± 8% +0.0 0.24 ± 3% perf-profile.children.cycles-pp.__put_anon_vma
0.20 ± 10% +0.0 0.24 ± 10% perf-profile.children.cycles-pp.preempt_count_add
0.30 ± 6% +0.0 0.35 ± 6% perf-profile.children.cycles-pp.debug_smp_processor_id
0.05 ± 45% +0.1 0.10 ± 11% perf-profile.children.cycles-pp.do_raw_spin_unlock
0.02 ±141% +0.1 0.07 ± 8% perf-profile.children.cycles-pp.newidle_balance
0.00 +0.1 0.06 ± 12% perf-profile.children.cycles-pp.select_idle_sibling
0.20 ± 12% +0.1 0.26 ± 7% perf-profile.children.cycles-pp.shmem_file_read_iter
0.25 ± 4% +0.1 0.31 ± 11% perf-profile.children.cycles-pp.create_elf_tables
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.check_preempt_curr
0.10 ± 14% +0.1 0.17 ± 7% perf-profile.children.cycles-pp.update_load_avg
0.58 ± 4% +0.1 0.66 ± 5% perf-profile.children.cycles-pp.__d_alloc
0.48 ± 5% +0.1 0.55 ± 7% perf-profile.children.cycles-pp.vm_area_alloc
0.00 +0.1 0.08 ± 8% perf-profile.children.cycles-pp.rcu_note_context_switch
0.00 +0.1 0.09 ± 21% perf-profile.children.cycles-pp.enqueue_entity
0.08 ± 9% +0.1 0.17 ± 7% perf-profile.children.cycles-pp.irqentry_exit
0.00 +0.1 0.09 ± 6% perf-profile.children.cycles-pp.preempt_schedule_irq
0.00 +0.1 0.09 ± 10% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.00 +0.1 0.09 ± 13% perf-profile.children.cycles-pp.perf_tp_event
0.00 +0.1 0.10 ± 14% perf-profile.children.cycles-pp.put_prev_entity
0.05 ± 47% +0.1 0.15 ± 12% perf-profile.children.cycles-pp.cpuacct_charge
0.00 +0.1 0.10 ± 13% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
0.71 +0.1 0.82 ± 5% perf-profile.children.cycles-pp.rcu_do_batch
0.79 ± 2% +0.1 0.89 ± 4% perf-profile.children.cycles-pp.rcu_core
1.49 ± 3% +0.1 1.59 ± 2% perf-profile.children.cycles-pp.fs_reclaim_acquire
1.45 ± 4% +0.1 1.56 ± 2% perf-profile.children.cycles-pp.mmap_region
0.82 ± 4% +0.1 0.94 ± 3% perf-profile.children.cycles-pp.mem_cgroup_uncharge_list
1.48 ± 4% +0.1 1.60 ± 2% perf-profile.children.cycles-pp.do_mmap
1.08 ± 7% +0.1 1.20 perf-profile.children.cycles-pp.anon_vma_fork
1.62 ± 4% +0.1 1.74 ± 3% perf-profile.children.cycles-pp.vm_mmap_pgoff
0.00 +0.1 0.12 ± 12% perf-profile.children.cycles-pp.preempt_schedule_common
0.00 +0.1 0.13 ± 11% perf-profile.children.cycles-pp.preempt_schedule_thunk
1.42 ± 2% +0.1 1.55 ± 4% perf-profile.children.cycles-pp.___perf_sw_event
2.14 ± 2% +0.1 2.27 perf-profile.children.cycles-pp.mark_lock
0.00 +0.1 0.13 ± 15% perf-profile.children.cycles-pp.enqueue_task_fair
0.00 +0.1 0.14 ± 11% perf-profile.children.cycles-pp._find_next_bit
2.55 ± 2% +0.1 2.70 ± 2% perf-profile.children.cycles-pp.dup_mmap
2.60 ± 2% +0.1 2.75 ± 2% perf-profile.children.cycles-pp.dup_mm
0.81 ± 7% +0.2 0.96 ± 4% perf-profile.children.cycles-pp.unlink_anon_vmas
2.23 +0.2 2.38 ± 2% perf-profile.children.cycles-pp.trace_hardirqs_on
0.00 +0.2 0.16 ± 12% perf-profile.children.cycles-pp.ttwu_do_activate
3.16 +0.2 3.32 ± 2% perf-profile.children.cycles-pp.copy_process
3.25 +0.2 3.41 ± 2% perf-profile.children.cycles-pp.__do_sys_clone
3.25 +0.2 3.41 ± 2% perf-profile.children.cycles-pp.kernel_clone
0.10 ± 14% +0.2 0.26 ± 9% perf-profile.children.cycles-pp._raw_spin_unlock_irq
0.05 ± 46% +0.2 0.22 ± 7% perf-profile.children.cycles-pp.mem_cgroup_charge_statistics
0.07 ± 17% +0.2 0.24 ± 9% perf-profile.children.cycles-pp.pick_next_task_fair
0.00 +0.2 0.17 ± 6% perf-profile.children.cycles-pp.asm_sysvec_reschedule_ipi
1.16 ± 3% +0.2 1.33 ± 3% perf-profile.children.cycles-pp.validate_chain
1.06 ± 4% +0.2 1.24 ± 4% perf-profile.children.cycles-pp.free_pgtables
0.58 ± 9% +0.2 0.76 ± 4% perf-profile.children.cycles-pp.remove_vma
0.01 ±223% +0.2 0.20 ± 8% perf-profile.children.cycles-pp.select_task_rq_fair
0.12 ± 30% +0.2 0.31 ± 4% perf-profile.children.cycles-pp.finish_task_switch
1.74 ± 3% +0.2 1.94 ± 3% perf-profile.children.cycles-pp.rcu_read_lock_held
1.09 ± 4% +0.2 1.29 ± 2% perf-profile.children.cycles-pp.__this_cpu_preempt_check
0.36 ± 7% +0.2 0.57 ± 6% perf-profile.children.cycles-pp.lru_add_drain_cpu
0.04 ± 72% +0.2 0.25 ± 9% perf-profile.children.cycles-pp.dequeue_entity
0.06 ± 17% +0.2 0.27 ± 7% perf-profile.children.cycles-pp.dequeue_task_fair
0.52 ± 5% +0.2 0.76 ± 6% perf-profile.children.cycles-pp.lru_add_drain
0.94 ± 5% +0.2 1.19 ± 3% perf-profile.children.cycles-pp.__mem_cgroup_charge
0.07 ± 12% +0.2 0.32 ± 5% perf-profile.children.cycles-pp.schedule_idle
0.18 ± 12% +0.3 0.47 ± 8% perf-profile.children.cycles-pp.update_curr
0.78 ± 5% +0.3 1.07 ± 4% perf-profile.children.cycles-pp.__pagevec_lru_add_fn
2.67 ± 3% +0.3 2.97 ± 3% perf-profile.children.cycles-pp.mem_cgroup_charge
0.56 ± 2% +0.3 0.89 ± 4% perf-profile.children.cycles-pp.page_add_new_anon_rmap
1.43 ± 3% +0.3 1.77 ± 2% perf-profile.children.cycles-pp.lru_cache_add
0.41 ± 12% +0.3 0.75 ± 5% perf-profile.children.cycles-pp.lock_page_lruvec_irqsave
2.69 ± 2% +0.4 3.06 ± 2% perf-profile.children.cycles-pp.rcu_is_watching
2.78 ± 2% +0.4 3.16 ± 2% perf-profile.children.cycles-pp.begin_new_exec
2.59 ± 2% +0.4 2.98 ± 3% perf-profile.children.cycles-pp.rcu_lockdep_current_cpu_online
1.44 ± 4% +0.4 1.83 ± 3% perf-profile.children.cycles-pp.match_held_lock
0.45 ± 7% +0.4 0.88 ± 5% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
1.54 ± 3% +0.4 1.97 ± 2% perf-profile.children.cycles-pp.memcg_slab_free_hook
3.23 ± 3% +0.4 3.67 ± 4% perf-profile.children.cycles-pp.__x64_sys_exit_group
3.23 ± 3% +0.4 3.67 ± 4% perf-profile.children.cycles-pp.do_group_exit
3.23 ± 3% +0.4 3.67 ± 4% perf-profile.children.cycles-pp.do_exit
1.01 ± 5% +0.4 1.46 ± 3% perf-profile.children.cycles-pp._raw_spin_unlock
1.90 ± 2% +0.4 2.35 ± 2% perf-profile.children.cycles-pp.kmem_cache_free
1.62 ± 3% +0.5 2.10 ± 2% perf-profile.children.cycles-pp.release_pages
1.79 ± 2% +0.5 2.28 ± 2% perf-profile.children.cycles-pp.tlb_finish_mmu
4.90 ± 2% +0.5 5.40 perf-profile.children.cycles-pp.kmem_cache_alloc
1.76 ± 2% +0.5 2.26 ± 2% perf-profile.children.cycles-pp.tlb_flush_mmu
4.05 +0.5 4.59 ± 2% perf-profile.children.cycles-pp.page_remove_rmap
1.20 ± 3% +0.6 1.75 ± 3% perf-profile.children.cycles-pp.__pagevec_lru_add
4.68 ± 2% +0.6 5.26 perf-profile.children.cycles-pp.load_elf_binary
4.91 ± 2% +0.6 5.51 perf-profile.children.cycles-pp.bprm_execve
4.72 ± 2% +0.6 5.32 perf-profile.children.cycles-pp.exec_binprm
5.13 +0.6 5.76 ± 3% perf-profile.children.cycles-pp.zap_pte_range
5.30 +0.6 5.93 ± 3% perf-profile.children.cycles-pp.unmap_vmas
5.24 +0.6 5.87 ± 3% perf-profile.children.cycles-pp.unmap_page_range
6.09 ± 2% +0.6 6.73 perf-profile.children.cycles-pp.do_execveat_common
6.15 ± 2% +0.6 6.80 perf-profile.children.cycles-pp.__x64_sys_execve
6.59 ± 2% +0.7 7.25 perf-profile.children.cycles-pp.check_preemption_disabled
0.17 ± 21% +0.7 0.86 ± 4% perf-profile.children.cycles-pp.schedule
0.09 ± 10% +0.7 0.82 ± 4% perf-profile.children.cycles-pp.try_to_wake_up
1.76 ± 3% +0.8 2.51 ± 2% perf-profile.children.cycles-pp._raw_spin_lock
5.65 ± 2% +0.8 6.41 ± 2% perf-profile.children.cycles-pp.rcu_read_lock_held_common
5.49 +0.8 6.27 ± 2% perf-profile.children.cycles-pp.exit_mmap
5.50 +0.8 6.28 ± 2% perf-profile.children.cycles-pp.mmput
1.00 ± 3% +0.8 1.81 ± 2% perf-profile.children.cycles-pp.__mod_lruvec_page_state
3.51 ± 2% +0.9 4.41 perf-profile.children.cycles-pp.unmap_region
0.29 ± 6% +1.0 1.25 ± 4% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
4.35 ± 3% +1.0 5.33 perf-profile.children.cycles-pp.__do_munmap
0.00 +1.0 0.98 ± 5% perf-profile.children.cycles-pp.__queue_work
7.13 +1.0 8.14 ± 2% perf-profile.children.cycles-pp.do_anonymous_page
0.00 +1.0 1.03 ± 5% perf-profile.children.cycles-pp.queue_work_on
5.95 ± 2% +1.0 7.00 perf-profile.children.cycles-pp.__x64_sys_brk
0.32 ± 9% +1.1 1.43 ± 2% perf-profile.children.cycles-pp.__schedule
0.32 ± 17% +1.1 1.43 ± 4% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
12.31 +1.2 13.53 perf-profile.children.cycles-pp.__handle_mm_fault
0.46 ± 10% +1.2 1.71 ± 3% perf-profile.children.cycles-pp.do_raw_spin_lock
0.04 ±100% +1.3 1.29 ± 4% perf-profile.children.cycles-pp.cgroup_rstat_updated
7.84 ± 3% +1.3 9.12 ± 5% perf-profile.children.cycles-pp.cpuidle_enter
7.84 ± 3% +1.3 9.12 ± 6% perf-profile.children.cycles-pp.cpuidle_enter_state
8.89 +1.4 10.25 perf-profile.children.cycles-pp.__lock_acquire
9.58 ± 2% +1.5 11.08 perf-profile.children.cycles-pp.lock_release
8.09 ± 5% +1.5 9.60 ± 8% perf-profile.children.cycles-pp.start_secondary
0.81 ± 7% +1.5 2.35 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.00 +1.6 1.58 ± 6% perf-profile.children.cycles-pp.mem_cgroup_css_rstat_flush
16.58 +1.6 18.18 perf-profile.children.cycles-pp.asm_exc_page_fault
15.80 +1.7 17.46 perf-profile.children.cycles-pp.do_user_addr_fault
8.61 ± 3% +1.7 10.27 ± 7% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
8.61 ± 3% +1.7 10.27 ± 7% perf-profile.children.cycles-pp.cpu_startup_entry
8.61 ± 3% +1.7 10.27 ± 7% perf-profile.children.cycles-pp.do_idle
14.31 +1.7 15.98 perf-profile.children.cycles-pp.handle_mm_fault
16.01 +1.7 17.68 perf-profile.children.cycles-pp.exc_page_fault
4.52 +1.7 6.22 ± 4% perf-profile.children.cycles-pp.intel_idle
9.80 +1.7 11.53 perf-profile.children.cycles-pp.lock_is_held_type
12.58 ± 2% +2.1 14.70 perf-profile.children.cycles-pp.rcu_read_lock_sched_held
30.34 +2.5 32.85 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
30.30 +2.5 32.82 perf-profile.children.cycles-pp.do_syscall_64
17.41 +2.9 20.27 perf-profile.children.cycles-pp.lock_acquire
0.00 +4.1 4.08 ± 3% perf-profile.children.cycles-pp.cgroup_rstat_flush_locked
0.00 +4.1 4.12 ± 3% perf-profile.children.cycles-pp.cgroup_rstat_flush_irqsafe
1.18 ± 14% +4.2 5.38 ± 2% perf-profile.children.cycles-pp.process_one_work
0.00 +4.3 4.31 ± 3% perf-profile.children.cycles-pp.mem_cgroup_flush_stats
1.19 ± 14% +5.0 6.15 ± 2% perf-profile.children.cycles-pp.worker_thread
1.44 ± 11% +5.0 6.45 ± 2% perf-profile.children.cycles-pp.kthread
1.49 ± 11% +5.0 6.51 ± 2% perf-profile.children.cycles-pp.ret_from_fork
17.68 -4.5 13.13 perf-profile.self.cycles-pp.sieve
5.91 ± 2% -1.6 4.31 perf-profile.self.cycles-pp.string_rtns_1
3.06 ± 3% -0.8 2.26 ± 2% perf-profile.self.cycles-pp.add_int
3.11 ± 4% -0.8 2.31 ± 2% perf-profile.self.cycles-pp.__strncat_chk
3.00 ± 3% -0.8 2.21 ± 3% perf-profile.self.cycles-pp.add_long
2.20 ± 2% -0.6 1.58 ± 2% perf-profile.self.cycles-pp.div_long
1.82 ± 3% -0.5 1.34 ± 2% perf-profile.self.cycles-pp.__strcat_chk
1.76 ± 4% -0.5 1.30 ± 3% perf-profile.self.cycles-pp.add_short
1.12 ± 14% -0.4 0.72 ± 13% perf-profile.self.cycles-pp.memcpy_toio
0.91 ± 10% -0.3 0.66 ± 8% perf-profile.self.cycles-pp.div_int
0.98 ± 4% -0.2 0.75 ± 10% perf-profile.self.cycles-pp.div_short
0.56 ± 9% -0.1 0.45 ± 5% perf-profile.self.cycles-pp.mem_rtns_1
0.32 ± 3% -0.1 0.25 ± 6% perf-profile.self.cycles-pp.page_test
0.19 ± 8% -0.0 0.15 ± 6% perf-profile.self.cycles-pp.malloc
0.06 ± 11% +0.0 0.08 ± 14% perf-profile.self.cycles-pp.__mod_lruvec_page_state
0.07 ± 18% +0.0 0.11 ± 13% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state
0.04 ± 71% +0.1 0.09 ± 14% perf-profile.self.cycles-pp.do_raw_spin_unlock
0.47 ± 6% +0.1 0.54 ± 5% perf-profile.self.cycles-pp.lockdep_hardirqs_on_prepare
0.00 +0.1 0.07 ± 12% perf-profile.self.cycles-pp.__schedule
0.04 ±100% +0.1 0.16 ± 12% perf-profile.self.cycles-pp.cgroup_rstat_updated
0.14 ± 9% +0.1 0.27 ± 5% perf-profile.self.cycles-pp.do_raw_spin_lock
2.06 ± 2% +0.1 2.19 ± 2% perf-profile.self.cycles-pp.mark_lock
0.00 +0.1 0.14 ± 9% perf-profile.self.cycles-pp._find_next_bit
1.08 ± 3% +0.1 1.22 ± 3% perf-profile.self.cycles-pp.rcu_read_lock_sched_held
1.12 ± 3% +0.2 1.30 ± 3% perf-profile.self.cycles-pp.validate_chain
1.61 ± 2% +0.2 1.83 ± 3% perf-profile.self.cycles-pp.rcu_is_watching
0.00 +0.3 0.27 ± 5% perf-profile.self.cycles-pp.cgroup_rstat_flush_locked
1.83 ± 3% +0.3 2.10 ± 4% perf-profile.self.cycles-pp.rcu_lockdep_current_cpu_online
1.16 ± 3% +0.3 1.46 ± 2% perf-profile.self.cycles-pp.match_held_lock
2.24 ± 2% +0.3 2.56 ± 2% perf-profile.self.cycles-pp.lock_acquire
2.32 ± 3% +0.5 2.86 ± 2% perf-profile.self.cycles-pp.lock_release
6.17 ± 2% +0.6 6.77 perf-profile.self.cycles-pp.check_preemption_disabled
6.33 +1.1 7.41 perf-profile.self.cycles-pp.__lock_acquire
0.32 ± 17% +1.1 1.43 ± 4% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
6.20 +1.2 7.39 perf-profile.self.cycles-pp.lock_is_held_type
0.00 +1.5 1.45 ± 7% perf-profile.self.cycles-pp.mem_cgroup_css_rstat_flush
4.52 +1.7 6.22 ± 4% perf-profile.self.cycles-pp.intel_idle
reaim.time.user_time
615 +---------------------------------------------------------------------+
610 |-+ + |
| : |
605 |-+ : : |
600 |-+ : : |
595 |-+ : : .+.+.+.+. .+ + + + |
590 |.+.: ++.+.+ + :.+. .+. + + :+ .+. .+. +. + +|
| + + +.+ +.+.+ + +.+ + +.+.+ +.+ |
585 |-+ |
580 |-+ |
575 |-O O O OO O O O O O O O O |
570 |-+ |
| O O O O |
565 |-+ O O O O O OO O O O |
560 +---------------------------------------------------------------------+
reaim.time.involuntary_context_switches
1e+06 +------------------------------------------------------------------+
900000 |-O OO O O O OO O O O O O |
| |
800000 |-+ |
700000 |-+ |
| |
600000 |-+ |
500000 |-+ |
400000 |-+ |
| |
300000 |-+ |
200000 |-+ |
| |
100000 |.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.|
0 +------------------------------------------------------------------+
reaim.jobs_per_min
142000 +------------------------------------------------------------------+
140000 |.+.+ .+.+. +.+. .+.+.+ |
| :.+ + + : |
138000 |-+ + : .|
136000 |-+ +.+.+.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+ |
134000 |-+ |
132000 |-+ |
| |
130000 |-+ OO O O O OO O O |
128000 |-O O O O |
126000 |-+ |
124000 |-+ |
| O O O OO O |
122000 |-+ O O O O O O O O |
120000 +------------------------------------------------------------------+
reaim.jobs_per_min_child
8800 +--------------------------------------------------------------------+
| + : + + .+.+.+.+ + |
8600 |-+ : : + : |
| + : .+. .+ .+. .+. .+. .+. .+ .+. .|
| + +.+ + +.+.+.+.++ + + + + + |
8400 |-+ |
| |
8200 |-+ |
| O |
8000 |-O O O O O O O O O OO O |
| |
| |
7800 |-+ |
| O O OO O O OO |
7600 +--------------------------------------------------------------------+
reaim.std_dev_time
0.022 +-------------------------------------------------------------------+
| |
0.02 |-+ O O O O O O |
| O O O O O O O O O O O O O O O O O O O |
| O O |
0.018 |-+ |
| |
0.016 |-+ |
| |
0.014 |-+ |
| |
| |
0.012 |-+.+.+ .+.+. +.+. .+. .+. |
|.+ +.+ +.+ +.+.+.++.+.+.+.+.++.+ +.+.++.+ +.+.++.+.+.|
0.01 +-------------------------------------------------------------------+
reaim.std_dev_percent
2.8 +---------------------------------------------------------------------+
| O O |
2.6 |-O O O O O O O OO O O O O O O O |
| O O O O O O O O |
| O |
2.4 |-+ |
| |
2.2 |-+ |
| |
2 |-+ |
| |
| .+. .+.+. .+.+.+. |
1.8 |.+ +.++ + +.+ .+. |
| +.+.+.+.+.+.+.+.+.++.+.+.+.+.+ +.++.+. .+.|
1.6 +---------------------------------------------------------------------+
reaim.max_jobs_per_min
150000 +------------------------------------------------------------------+
| |
| |
145000 |.+.+ +.+.+. .+.+. .+. |
| :+ ++ + + |
| + :.+.+. .++.+. .+. .+.+.+.+ .+.+.+. +.+.|
140000 |-+ + + + ++.+ + + |
| |
135000 |-+ |
| O OO O O O OO O O O O O |
| |
130000 |-+ |
| O O O OO O |
| O O O O O O |
125000 +------------------------------------------------------------------+
perf-sched.total_wait_time.average.ms
70 +----------------------------------------------------------------------+
| .+ + |
60 |-+ + .+ : +.+.+ : + |
|+ + + .+ :+ + : +. |
50 |-+ + + + +.+.+.+.++.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.|
| |
40 |-+ |
| |
30 |-+ |
| |
20 |-+ |
| |
10 |-O O O O O OO O O O O O O O O O O |
| O O O O O O O O O O |
0 +----------------------------------------------------------------------+
perf-sched.total_sch_delay.average.ms
0.1 +--------------------------------------------------------------------+
| + +. .+. + + + + + |
0.09 |-+ + + : +.+ ++ + .+. .++.+ +.+. .+. :+ |
|. .+.+ .+ +.+. .+. + : + + + + +.+.|
0.08 |-+ + + + : : |
| :: |
0.07 |-+ + |
| |
0.06 |-+ |
| |
0.05 |-+ |
| |
0.04 |-+ O |
| O O O O O O O O O OO O O O O OO O O O O O OO O |
0.03 +--------------------------------------------------------------------+
perf-sched.total_wait_and_delay.count.ms
140000 +------------------------------------------------------------------+
| O O O O O O |
120000 |-+ O O O O O O O OO O |
| O O OO O O O O O |
100000 |-+ O O |
| |
80000 |-+ |
| |
60000 |-+ |
| |
40000 |-+ |
| |
20000 |.+.++. .++. .+. .++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.|
| +.+.+ +.+ + |
0 +------------------------------------------------------------------+
perf-sched.total_wait_and_delay.average.ms
70 +----------------------------------------------------------------------+
| .+ + |
60 |-+ + .+ : +.+.+ : + |
|+ + + .+ :+ + : +. .+. |
50 |-+ + + + +.+.+.+.++.+.+.+ +.+.+.+.+.+.++.+.+.+.+.+.|
| |
40 |-+ |
| |
30 |-+ |
| |
20 |-+ |
| |
10 |-O O O O O OO O O O O O O O O O O |
| O O O O O O O O O O |
0 +----------------------------------------------------------------------+
perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
180 +---------------------------------------------------------------------+
| .+ .+. .+. |
160 |.+ : ++.+. .+.+ + + |
140 |-+ : + + : .+. .+. .+. .|
| + +.+.+.+.+.+ +.+.++.+.+ +.+.+.+.++.+ + |
120 |-+ |
100 |-+ |
| |
80 |-+ |
60 |-+ |
| |
40 |-+ |
20 |-+ |
| |
0 +---------------------------------------------------------------------+
perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
90000 +-------------------------------------------------------------------+
| O OO O O OO O O O |
80000 |-+ O O O O O |
70000 |-O O O O O OO O O O |
| O O |
60000 |-+ |
50000 |-+ |
| |
40000 |-+ |
30000 |-+ |
| |
20000 |-+ |
10000 |-+ |
| |
0 +-------------------------------------------------------------------+
perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
180 +---------------------------------------------------------------------+
| .+ .+. .+. |
160 |.+ : ++.+. .+.+ + + |
140 |-+ : + + : .+. .+. .+. .|
| + +.+.+.+.+.+ +.+.++.+.+ +.+.+.+.++.+ + |
120 |-+ |
100 |-+ |
| |
80 |-+ |
60 |-+ |
| |
40 |-+ |
20 |-+ |
| |
0 +---------------------------------------------------------------------+
6000 +--------------------------------------------------------------------+
| O O O |
5000 |-+ O O |
| O O O O OO O O |
| O O O O O |
4000 |-+ O |
| O O O O O O |
3000 |-+ O |
| O |
2000 |-+ |
| |
| |
1000 |-+ |
| |
0 +--------------------------------------------------------------------+
3500 +--------------------------------------------------------------------+
| O O O O O OO |
3000 |-+ O O O O O O O O O O |
| O O |
2500 |-+ O O O O |
| O O |
2000 |-+ |
| |
1500 |-+ |
| |
1000 |-+ |
| |
500 |-+ |
|.+.+.++.+.+.+.+.+. .++.+.+.+.+.+.++.+.+.+.+.+.++.+.+.+.+.+.+.++.+.+.|
0 +--------------------------------------------------------------------+
1800 +--------------------------------------------------------------------+
| O |
1600 |-+ O O O |
1400 |-O O O O O O O O O O O |
| O O O O OO O O O OO |
1200 |-+ O |
1000 |-+ |
| |
800 |-+ |
600 |-+ |
| |
400 |-+ |
200 |-+ |
|.+.+.++.+.+.+.+.+.+.++.+.+.+.+.+.++.+.+.+.+.+.++.+.+.+.+.+.+.++.+.+.|
0 +--------------------------------------------------------------------+
70 +----------------------------------------------------------------------+
65 |-+ O O O O |
| O O O O O O O O O O |
60 |-O O O O |
55 |-+ O O O O O O O O O |
| |
50 |-+ |
45 |-+ |
40 |-+ |
| |
35 |-+ |
30 |-+ .+. +. |
|.+. .+. +. .+. .+.+. +.+. .+.+.+.+. .+ +.+ +.+. .+. .|
25 |-+ + +.+.+ +.+.+.+.+.+ + + + + + + |
20 +----------------------------------------------------------------------+
1.4 +---------------------------------------------------------------------+
| |
1.2 |-+ |
| |
1 |-+ |
| |
0.8 |-+ |
| |
0.6 |-+ |
| |
0.4 |-+ |
| |
0.2 |-+ |
| O .+ O O O O |
0 +---------------------------------------------------------------------+
1.2 +---------------------------------------------------------------------+
| O |
1 |-+ |
| |
| |
0.8 |-+ O |
| |
0.6 |-+ |
| O O |
0.4 |-+ |
| |
| |
0.2 |-+ |
| O |
0 +---------------------------------------------------------------------+
0.14 +--------------------------------------------------------------------+
| |
0.12 |-+ |
| |
0.1 |-+ |
| |
0.08 |-+ :|
| :|
0.06 |-+ :|
| :|
0.04 |-O :|
| O O O O :|
0.02 |-+ O O O O O O O O O : |
| O O O O O O O O O O O : |
0 +--------------------------------------------------------------------+
1.2 +---------------------------------------------------------------------+
| O |
1 |-+ |
| |
| |
0.8 |-+ O |
| |
0.6 |-+ |
| |
0.4 |-+ |
| |
| |
0.2 |-+ |
| |
0 +---------------------------------------------------------------------+
3 +---------------------------------------------------------------------+
| O |
2.5 |-+ |
| |
| |
2 |-+ |
| |
1.5 |-+ |
| |
1 |-+ |
| |
| |
0.5 |-+ |
| |
0 +---------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-cfl-e1: 16 threads 1 sockets Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/300s/lkp-cfl-e1/new_fserver/reaim/0xde
commit:
0c7c5da68b ("memcg: switch lruvec stats to rstat")
cda0d20525 ("memcg: infrastructure to flush memcg stats")
0c7c5da68b4a09ff cda0d205259a7e7c1f8e296130a
---------------- ---------------------------
%stddev %change %stddev
\ | \
7.99 +1.2% 8.09 reaim.child_systime
3.26 -1.7% 3.20 reaim.child_utime
121091 -9.5% 109580 reaim.jobs_per_min
7568 -9.5% 6848 reaim.jobs_per_min_child
123789 -8.4% 113349 reaim.max_jobs_per_min
0.78 +10.6% 0.86 reaim.parent_time
1.04 +83.8% 1.91 reaim.std_dev_percent
0.01 +65.8% 0.02 reaim.std_dev_time
133153 +527.8% 835973 reaim.time.involuntary_context_switches
8602 -12.8% 7502 reaim.time.major_page_faults
16988615 -2.9% 16491812 reaim.time.minor_page_faults
398.33 -2.5% 388.29 reaim.time.percent_of_cpu_this_job_got
858.75 -1.7% 843.93 reaim.time.system_time
348.24 -4.5% 332.44 reaim.time.user_time
387134 +4.7% 405213 reaim.time.voluntary_context_switches
170666 -2.9% 165714 reaim.workload
4180544 ± 3% +75.6% 7340133 ± 24% cpuidle..usage
0.00 ± 42% +0.0 0.02 ±109% mpstat.cpu.all.iowait%
55207 ± 3% -7.3% 51204 ± 3% softirqs.CPU8.NET_RX
36.52 +3.6% 37.86 boot-time.boot
11.58 +1.5% 11.75 boot-time.dhcp
3985 +384.8% 19324 vmstat.system.cs
21914 ± 2% +33.7% 29299 ± 16% vmstat.system.in
28464 ± 2% +85.3% 52734 ± 7% meminfo.Active
28304 ± 2% +84.7% 52277 ± 8% meminfo.Active(anon)
37567 +67.6% 62971 ± 6% meminfo.Shmem
2909 ± 3% -14.5% 2489 ± 3% slabinfo.cred_jar.active_objs
2909 ± 3% -14.5% 2489 ± 3% slabinfo.cred_jar.num_objs
3454 ± 4% -10.0% 3110 ± 3% slabinfo.kmalloc-256.active_objs
3458 ± 4% -10.1% 3110 ± 3% slabinfo.kmalloc-256.num_objs
2242 ± 5% -17.4% 1853 ± 5% slabinfo.kmalloc-rcl-256.active_objs
2309 ± 5% -17.0% 1918 ± 5% slabinfo.kmalloc-rcl-256.num_objs
1264 ± 6% -18.5% 1030 ± 3% slabinfo.pid.active_objs
1264 ± 6% -18.5% 1030 ± 3% slabinfo.pid.num_objs
1323 ± 5% -18.6% 1077 ± 3% slabinfo.task_delay_info.active_objs
1323 ± 5% -18.6% 1077 ± 3% slabinfo.task_delay_info.num_objs
7105 ± 2% +84.0% 13071 ± 8% proc-vmstat.nr_active_anon
597622 +1.1% 604120 proc-vmstat.nr_file_pages
8026 +3.7% 8325 proc-vmstat.nr_mapped
9395 +67.6% 15749 ± 6% proc-vmstat.nr_shmem
7105 ± 2% +84.0% 13071 ± 8% proc-vmstat.nr_zone_active_anon
13394184 -2.6% 13042821 proc-vmstat.numa_hit
13394184 -2.6% 13042821 proc-vmstat.numa_local
63852 +24.3% 79392 ± 3% proc-vmstat.pgactivate
14095860 -2.9% 13688343 proc-vmstat.pgalloc_normal
17475839 -2.8% 16985166 proc-vmstat.pgfault
14070144 -3.0% 13646066 proc-vmstat.pgfree
1782101 -3.0% 1729290 proc-vmstat.pgreuse
77684 ± 98% +2574.2% 2077436 ± 48% turbostat.C1
0.05 ± 40% +1.7 1.75 ± 56% turbostat.C1%
341592 ± 41% +252.7% 1204679 ± 36% turbostat.C1E
0.58 ± 37% +1.2 1.75 ± 40% turbostat.C1E%
134873 ± 72% -74.8% 34048 ±131% turbostat.C8
2.65 ± 73% -2.0 0.64 ±131% turbostat.C8%
0.05 ± 20% +989.8% 0.51 ± 80% turbostat.CPU%c3
1.60 ± 80% -79.2% 0.33 ±150% turbostat.CPU%c7
37.60 +7.9% 40.56 ± 2% turbostat.CorWatt
6681937 +34.0% 8953217 ± 16% turbostat.IRQ
11.48 ± 4% -13.2% 9.96 ± 10% turbostat.Pkg%pc2
36.67 -11.0% 32.65 ± 11% turbostat.Pkg%pc3
38.03 +7.8% 41.00 ± 2% turbostat.PkgWatt
4.34e+09 +6.1% 4.604e+09 perf-stat.i.branch-instructions
26075142 +13.8% 29666013 perf-stat.i.branch-misses
2.63 ± 2% -0.4 2.24 perf-stat.i.cache-miss-rate%
1.968e+08 +24.3% 2.445e+08 perf-stat.i.cache-references
3986 +386.3% 19385 perf-stat.i.context-switches
1.18 -2.9% 1.15 perf-stat.i.cpi
1.917e+10 +4.2% 1.997e+10 perf-stat.i.cpu-cycles
533.29 +99.9% 1066 perf-stat.i.cpu-migrations
3085 ± 2% +5.1% 3243 perf-stat.i.cycles-between-cache-misses
0.06 ± 2% -0.0 0.06 perf-stat.i.dTLB-load-miss-rate%
5.264e+09 +6.1% 5.584e+09 perf-stat.i.dTLB-loads
2.789e+09 +4.0% 2.901e+09 perf-stat.i.dTLB-stores
72.39 -6.3 66.12 ± 3% perf-stat.i.iTLB-load-miss-rate%
378101 +30.7% 494297 ± 6% perf-stat.i.iTLB-loads
2.105e+10 +5.1% 2.212e+10 perf-stat.i.instructions
0.92 +2.1% 0.94 perf-stat.i.ipc
28.63 -12.8% 24.97 perf-stat.i.major-faults
1.20 +4.2% 1.25 perf-stat.i.metric.GHz
139.31 +6.4% 148.22 perf-stat.i.metric.K/sec
787.04 +5.2% 828.12 perf-stat.i.metric.M/sec
56273 -2.8% 54676 perf-stat.i.minor-faults
56302 -2.8% 54701 perf-stat.i.page-faults
9.35 +18.3% 11.06 perf-stat.overall.MPKI
0.60 +0.0 0.64 perf-stat.overall.branch-miss-rate%
2.79 -0.5 2.27 perf-stat.overall.cache-miss-rate%
3489 +3.2% 3600 perf-stat.overall.cycles-between-cache-misses
0.01 -0.0 0.01 ± 3% perf-stat.overall.dTLB-load-miss-rate%
0.01 -0.0 0.00 perf-stat.overall.dTLB-store-miss-rate%
70.11 -5.7 64.38 perf-stat.overall.iTLB-load-miss-rate%
23728 +4.6% 24813 ± 3% perf-stat.overall.instructions-per-iTLB-miss
37316899 +8.3% 40412939 perf-stat.overall.path-length
4.328e+09 +6.1% 4.591e+09 perf-stat.ps.branch-instructions
26003309 +13.8% 29583403 perf-stat.ps.branch-misses
1.962e+08 +24.3% 2.438e+08 perf-stat.ps.cache-references
3974 +386.3% 19331 perf-stat.ps.context-switches
1.911e+10 +4.2% 1.991e+10 perf-stat.ps.cpu-cycles
532.01 +99.9% 1063 perf-stat.ps.cpu-migrations
5.25e+09 +6.1% 5.568e+09 perf-stat.ps.dTLB-loads
2.781e+09 +4.0% 2.893e+09 perf-stat.ps.dTLB-stores
377084 +30.7% 492966 ± 6% perf-stat.ps.iTLB-loads
2.099e+10 +5.1% 2.206e+10 perf-stat.ps.instructions
28.55 -12.8% 24.91 perf-stat.ps.major-faults
56125 -2.8% 54533 perf-stat.ps.minor-faults
56153 -2.8% 54558 perf-stat.ps.page-faults
6.369e+12 +5.2% 6.697e+12 perf-stat.total.instructions
33001 ± 5% +333.4% 143034 ± 2% interrupts.CAL:Function_call_interrupts
2974 ± 54% +207.0% 9132 ± 7% interrupts.CPU0.CAL:Function_call_interrupts
2969 ± 2% +1363.9% 43462 ± 3% interrupts.CPU0.RES:Rescheduling_interrupts
21.83 ± 22% +1446.1% 337.57 ± 7% interrupts.CPU0.TLB:TLB_shootdowns
2239 ± 44% +300.7% 8974 ± 3% interrupts.CPU1.CAL:Function_call_interrupts
2965 ± 2% +1378.5% 43847 ± 4% interrupts.CPU1.RES:Rescheduling_interrupts
25.67 ± 14% +1200.2% 333.71 ± 5% interrupts.CPU1.TLB:TLB_shootdowns
2677 ± 53% +224.5% 8687 ± 13% interrupts.CPU10.CAL:Function_call_interrupts
2960 +1369.3% 43490 ± 3% interrupts.CPU10.RES:Rescheduling_interrupts
21.17 ± 21% +1474.6% 333.29 ± 3% interrupts.CPU10.TLB:TLB_shootdowns
2899 ± 41% +192.4% 8479 ± 9% interrupts.CPU11.CAL:Function_call_interrupts
2895 ± 29% +34.8% 3902 ± 4% interrupts.CPU11.NMI:Non-maskable_interrupts
2895 ± 29% +34.8% 3902 ± 4% interrupts.CPU11.PMI:Performance_monitoring_interrupts
2955 ± 2% +1382.3% 43803 ± 2% interrupts.CPU11.RES:Rescheduling_interrupts
23.50 ± 17% +1331.6% 336.43 ± 5% interrupts.CPU11.TLB:TLB_shootdowns
1597 ± 30% +456.0% 8882 ± 7% interrupts.CPU12.CAL:Function_call_interrupts
2949 +1365.1% 43212 ± 2% interrupts.CPU12.RES:Rescheduling_interrupts
23.33 ± 14% +1335.1% 334.86 ± 4% interrupts.CPU12.TLB:TLB_shootdowns
2284 ± 35% +316.9% 9522 ± 10% interrupts.CPU13.CAL:Function_call_interrupts
2865 ± 26% +34.1% 3841 ± 5% interrupts.CPU13.NMI:Non-maskable_interrupts
2865 ± 26% +34.1% 3841 ± 5% interrupts.CPU13.PMI:Performance_monitoring_interrupts
2947 +1376.6% 43523 ± 2% interrupts.CPU13.RES:Rescheduling_interrupts
24.50 ± 21% +1326.2% 349.43 ± 5% interrupts.CPU13.TLB:TLB_shootdowns
1620 ± 17% +433.1% 8639 ± 5% interrupts.CPU14.CAL:Function_call_interrupts
2946 ± 2% +1385.4% 43772 ± 2% interrupts.CPU14.RES:Rescheduling_interrupts
23.33 ± 18% +1385.3% 346.57 ± 7% interrupts.CPU14.TLB:TLB_shootdowns
1576 ± 20% +486.0% 9235 ± 9% interrupts.CPU15.CAL:Function_call_interrupts
2950 ± 2% +1360.2% 43077 ± 4% interrupts.CPU15.RES:Rescheduling_interrupts
22.83 ± 9% +1421.0% 347.29 ± 3% interrupts.CPU15.TLB:TLB_shootdowns
1797 ± 45% +392.3% 8846 ± 4% interrupts.CPU2.CAL:Function_call_interrupts
2909 ± 3% +1414.7% 44071 interrupts.CPU2.RES:Rescheduling_interrupts
25.17 ± 10% +1177.8% 321.57 ± 4% interrupts.CPU2.TLB:TLB_shootdowns
1569 ± 24% +473.5% 8998 ± 6% interrupts.CPU3.CAL:Function_call_interrupts
3174 ± 20% +23.1% 3908 ± 4% interrupts.CPU3.NMI:Non-maskable_interrupts
3174 ± 20% +23.1% 3908 ± 4% interrupts.CPU3.PMI:Performance_monitoring_interrupts
3013 +1343.3% 43492 interrupts.CPU3.RES:Rescheduling_interrupts
26.00 ± 14% +1180.8% 333.00 ± 6% interrupts.CPU3.TLB:TLB_shootdowns
1620 ± 13% +429.6% 8581 ± 5% interrupts.CPU4.CAL:Function_call_interrupts
2878 ± 27% +33.5% 3842 ± 4% interrupts.CPU4.NMI:Non-maskable_interrupts
2878 ± 27% +33.5% 3842 ± 4% interrupts.CPU4.PMI:Performance_monitoring_interrupts
2933 ± 2% +1398.8% 43966 ± 2% interrupts.CPU4.RES:Rescheduling_interrupts
20.67 ± 24% +1498.8% 330.43 ± 5% interrupts.CPU4.TLB:TLB_shootdowns
2286 ± 56% +280.6% 8703 ± 5% interrupts.CPU5.CAL:Function_call_interrupts
2975 ± 2% +1373.5% 43848 ± 3% interrupts.CPU5.RES:Rescheduling_interrupts
23.50 ± 8% +1334.7% 337.14 ± 8% interrupts.CPU5.TLB:TLB_shootdowns
1447 ± 15% +523.4% 9022 ± 4% interrupts.CPU6.CAL:Function_call_interrupts
2917 ± 3% +1401.5% 43802 ± 2% interrupts.CPU6.RES:Rescheduling_interrupts
22.17 ± 10% +1459.0% 345.57 ± 5% interrupts.CPU6.TLB:TLB_shootdowns
2405 ± 17% +291.3% 9412 ± 4% interrupts.CPU7.CAL:Function_call_interrupts
3094 ± 2% +1319.7% 43934 ± 2% interrupts.CPU7.RES:Rescheduling_interrupts
23.00 ± 17% +1322.4% 327.14 ± 3% interrupts.CPU7.TLB:TLB_shootdowns
1933 ± 39% +369.4% 9074 ± 7% interrupts.CPU8.CAL:Function_call_interrupts
3010 ± 2% +1374.6% 44387 ± 3% interrupts.CPU8.RES:Rescheduling_interrupts
24.17 ± 20% +1241.3% 324.14 ± 3% interrupts.CPU8.TLB:TLB_shootdowns
2072 ± 15% +326.6% 8840 ± 6% interrupts.CPU9.CAL:Function_call_interrupts
2955 +1383.1% 43828 ± 2% interrupts.CPU9.RES:Rescheduling_interrupts
25.33 ± 19% +1222.9% 335.14 ± 4% interrupts.CPU9.TLB:TLB_shootdowns
48547 ± 8% +17.3% 56951 ± 7% interrupts.NMI:Non-maskable_interrupts
48547 ± 8% +17.3% 56951 ± 7% interrupts.PMI:Performance_monitoring_interrupts
47453 +1374.1% 699520 interrupts.RES:Rescheduling_interrupts
376.17 ± 7% +1328.4% 5373 interrupts.TLB:TLB_shootdowns
5.54 ± 3% -0.9 4.59 ± 5% perf-profile.calltrace.cycles-pp.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.53 ± 3% -0.9 4.58 ± 5% perf-profile.calltrace.cycles-pp.iterate_dir.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.44 ± 3% -0.9 4.52 ± 5% perf-profile.calltrace.cycles-pp.dcache_readdir.iterate_dir.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.23 ± 3% -0.9 4.34 ± 5% perf-profile.calltrace.cycles-pp.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64.do_syscall_64
3.16 ± 10% -0.8 2.37 ± 11% perf-profile.calltrace.cycles-pp.write
3.13 ± 10% -0.8 2.35 ± 11% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
3.14 ± 10% -0.8 2.35 ± 11% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
3.04 ± 10% -0.8 2.27 ± 12% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
3.02 ± 10% -0.8 2.26 ± 12% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
2.88 ± 4% -0.6 2.30 ± 9% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.03 ± 10% -0.5 1.51 ± 6% perf-profile.calltrace.cycles-pp.div_long
2.11 ± 8% -0.5 1.63 ± 4% perf-profile.calltrace.cycles-pp.add_int.add_int
0.90 ± 13% -0.4 0.45 ± 67% perf-profile.calltrace.cycles-pp.devkmsg_write.cold.new_sync_write.vfs_write.ksys_write.do_syscall_64
0.90 ± 13% -0.4 0.45 ± 67% perf-profile.calltrace.cycles-pp.devkmsg_emit.devkmsg_write.cold.new_sync_write.vfs_write.ksys_write
0.75 ± 21% -0.4 0.31 ± 86% perf-profile.calltrace.cycles-pp.terminate_walk.path_openat.do_filp_open.do_sys_openat2.do_sys_open
0.90 ± 13% -0.4 0.45 ± 67% perf-profile.calltrace.cycles-pp.vprintk_emit.devkmsg_emit.devkmsg_write.cold.new_sync_write.vfs_write
1.77 ± 7% -0.4 1.34 ± 6% perf-profile.calltrace.cycles-pp.kill
1.76 ± 7% -0.4 1.33 ± 6% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill
1.76 ± 7% -0.4 1.33 ± 6% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.kill
5.86 ± 3% -0.4 5.47 ± 2% perf-profile.calltrace.cycles-pp.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap.mmput
9.51 ± 2% -0.4 9.15 ± 3% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64
9.52 ± 2% -0.4 9.16 ± 3% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.68 ± 12% -0.4 0.33 ± 88% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
1.59 ± 7% -0.3 1.25 ± 8% perf-profile.calltrace.cycles-pp.__strcat_chk
4.13 ± 2% -0.3 3.79 ± 2% perf-profile.calltrace.cycles-pp.lock_page_memcg.page_remove_rmap.zap_pte_range.unmap_page_range.unmap_vmas
2.10 ± 5% -0.3 1.78 ± 2% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.begin_new_exec
2.11 ± 5% -0.3 1.79 ± 2% perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.begin_new_exec.load_elf_binary
2.12 ± 3% -0.3 1.81 ± 6% perf-profile.calltrace.cycles-pp.dput.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64
1.19 ± 8% -0.3 0.89 ± 4% perf-profile.calltrace.cycles-pp.do_open.path_openat.do_filp_open.do_sys_openat2.do_sys_open
0.59 ± 7% -0.3 0.30 ± 86% perf-profile.calltrace.cycles-pp.__fput.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
1.03 ± 12% -0.3 0.75 ± 10% perf-profile.calltrace.cycles-pp.drm_fb_helper_damage_work.process_one_work.worker_thread.kthread.ret_from_fork
1.15 ± 7% -0.3 0.87 ± 5% perf-profile.calltrace.cycles-pp.__x64_sys_kill.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill
1.00 ± 12% -0.3 0.73 ± 10% perf-profile.calltrace.cycles-pp.memcpy_toio.drm_fb_helper_damage_work.process_one_work.worker_thread.kthread
4.40 ± 3% -0.3 4.15 ± 2% perf-profile.calltrace.cycles-pp.page_remove_rmap.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap
1.06 ± 7% -0.2 0.81 ± 4% perf-profile.calltrace.cycles-pp.kill_proc_info.__x64_sys_kill.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill
1.16 ± 9% -0.2 0.91 ± 3% perf-profile.calltrace.cycles-pp.add_short.add_short
1.20 ± 8% -0.2 0.95 ± 7% perf-profile.calltrace.cycles-pp.__close
1.16 ± 8% -0.2 0.92 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close
1.16 ± 8% -0.2 0.92 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__close
0.99 ± 7% -0.2 0.76 ± 4% perf-profile.calltrace.cycles-pp.kill_pid_info.kill_proc_info.__x64_sys_kill.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.84 ± 11% -0.2 0.62 ± 10% perf-profile.calltrace.cycles-pp.div_int
0.94 ± 8% -0.2 0.75 ± 7% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close
0.65 ± 5% -0.2 0.46 ± 41% perf-profile.calltrace.cycles-pp.lock_release._raw_spin_unlock.scan_positives.dcache_readdir.iterate_dir
1.21 ± 3% -0.2 1.03 ± 6% perf-profile.calltrace.cycles-pp.elf_map.load_elf_binary.exec_binprm.bprm_execve.do_execveat_common
1.18 ± 3% -0.2 1.01 ± 8% perf-profile.calltrace.cycles-pp.lockref_put_or_lock.dput.scan_positives.dcache_readdir.iterate_dir
1.18 ± 6% -0.2 1.01 ± 5% perf-profile.calltrace.cycles-pp.copy_strings.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.40 ± 2% -0.2 2.24 ± 2% perf-profile.calltrace.cycles-pp.lock_acquire.lock_page_memcg.page_remove_rmap.zap_pte_range.unmap_page_range
0.63 ± 5% -0.2 0.48 ± 41% perf-profile.calltrace.cycles-pp.lock_acquire._raw_spin_lock_nested.scan_positives.dcache_readdir.iterate_dir
0.80 ± 10% -0.1 0.65 ± 9% perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink
0.99 ± 4% -0.1 0.85 ± 6% perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.elf_map.load_elf_binary.exec_binprm.bprm_execve
0.78 ± 4% -0.1 0.66 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_unlock.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64
0.89 ± 5% -0.1 0.76 ± 6% perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.elf_map.load_elf_binary.exec_binprm
0.88 ± 5% -0.1 0.75 ± 6% perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.elf_map.load_elf_binary
2.04 ± 2% -0.1 1.93 ± 3% perf-profile.calltrace.cycles-pp.lock_page_memcg.page_add_file_rmap.do_set_pte.filemap_map_pages.do_fault
0.73 ± 4% -0.1 0.62 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock.lockref_put_or_lock.dput.scan_positives.dcache_readdir
0.66 ± 5% -0.1 0.56 ± 6% perf-profile.calltrace.cycles-pp.lock_acquire._raw_spin_lock.lockref_put_or_lock.dput.scan_positives
0.66 ± 5% -0.1 0.57 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock_nested.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64
0.69 ± 2% -0.1 0.61 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64
0.00 +0.5 0.53 ± 3% perf-profile.calltrace.cycles-pp.__schedule.schedule.worker_thread.kthread.ret_from_fork
0.00 +0.6 0.55 ± 3% perf-profile.calltrace.cycles-pp.schedule.worker_thread.kthread.ret_from_fork
0.00 +0.7 0.74 ± 4% perf-profile.calltrace.cycles-pp.lock_acquire.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work
0.00 +1.5 1.50 ± 3% perf-profile.calltrace.cycles-pp.mem_cgroup_css_rstat_flush.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work
6.91 +1.9 8.80 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
6.96 +1.9 8.86 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
4.22 ± 2% +2.0 6.18 ± 4% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
7.66 +2.4 10.02 ± 5% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
7.66 +2.4 10.03 ± 5% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
7.66 +2.4 10.03 ± 5% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
8.19 +2.5 10.65 ± 5% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
0.00 +3.8 3.80 perf-profile.calltrace.cycles-pp.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work.worker_thread
0.00 +3.8 3.85 perf-profile.calltrace.cycles-pp.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work.worker_thread.kthread
1.06 ± 12% +4.0 5.08 perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
0.00 +4.1 4.06 perf-profile.calltrace.cycles-pp.mem_cgroup_flush_stats.process_one_work.worker_thread.kthread.ret_from_fork
2.17 ± 6% +4.6 6.73 ± 2% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
2.18 ± 6% +4.6 6.75 ± 2% perf-profile.calltrace.cycles-pp.ret_from_fork
1.09 ± 12% +4.6 5.71 perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
57.01 ± 2% -2.6 54.44 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
56.96 ± 2% -2.6 54.40 perf-profile.children.cycles-pp.do_syscall_64
5.54 ± 3% -0.9 4.59 ± 5% perf-profile.children.cycles-pp.__x64_sys_getdents64
5.53 ± 3% -0.9 4.58 ± 5% perf-profile.children.cycles-pp.iterate_dir
5.44 ± 3% -0.9 4.52 ± 5% perf-profile.children.cycles-pp.dcache_readdir
5.24 ± 3% -0.9 4.34 ± 5% perf-profile.children.cycles-pp.scan_positives
3.16 ± 10% -0.8 2.37 ± 11% perf-profile.children.cycles-pp.write
2.03 ± 10% -0.5 1.51 ± 6% perf-profile.children.cycles-pp.div_long
11.54 -0.5 11.02 ± 2% perf-profile.children.cycles-pp.do_sys_open
11.53 -0.5 11.02 ± 2% perf-profile.children.cycles-pp.do_sys_openat2
8.26 ± 2% -0.5 7.77 perf-profile.children.cycles-pp.lock_page_memcg
2.12 ± 8% -0.5 1.64 ± 4% perf-profile.children.cycles-pp.add_int
4.96 ± 2% -0.4 4.52 ± 3% perf-profile.children.cycles-pp.dput
3.23 ± 3% -0.4 2.80 ± 7% perf-profile.children.cycles-pp.vfs_write
1.77 ± 7% -0.4 1.34 ± 6% perf-profile.children.cycles-pp.kill
3.24 ± 3% -0.4 2.81 ± 7% perf-profile.children.cycles-pp.ksys_write
10.26 ± 2% -0.4 9.84 ± 2% perf-profile.children.cycles-pp.path_openat
10.28 ± 2% -0.4 9.86 ± 2% perf-profile.children.cycles-pp.do_filp_open
2.96 ± 4% -0.4 2.54 ± 8% perf-profile.children.cycles-pp.new_sync_write
6.88 ± 2% -0.4 6.50 perf-profile.children.cycles-pp.filemap_map_pages
0.90 ± 13% -0.3 0.55 ± 33% perf-profile.children.cycles-pp.devkmsg_write.cold
0.90 ± 13% -0.3 0.55 ± 33% perf-profile.children.cycles-pp.devkmsg_emit
1.59 ± 7% -0.3 1.26 ± 8% perf-profile.children.cycles-pp.__strcat_chk
7.76 ± 2% -0.3 7.46 perf-profile.children.cycles-pp.do_fault
1.03 ± 12% -0.3 0.75 ± 10% perf-profile.children.cycles-pp.drm_fb_helper_damage_work
1.02 ± 12% -0.3 0.74 ± 10% perf-profile.children.cycles-pp.memcpy_toio
4.86 ± 2% -0.3 4.58 ± 2% perf-profile.children.cycles-pp.do_set_pte
4.00 ± 2% -0.3 3.75 ± 2% perf-profile.children.cycles-pp.page_add_file_rmap
1.20 ± 8% -0.2 0.95 ± 7% perf-profile.children.cycles-pp.__close
1.16 ± 9% -0.2 0.91 ± 3% perf-profile.children.cycles-pp.add_short
2.77 ± 4% -0.2 2.53 ± 2% perf-profile.children.cycles-pp.debug_lockdep_rcu_enabled
0.84 ± 11% -0.2 0.62 ± 10% perf-profile.children.cycles-pp.div_int
1.98 ± 2% -0.2 1.78 ± 5% perf-profile.children.cycles-pp.lockref_put_or_lock
1.50 ± 6% -0.2 1.32 ± 8% perf-profile.children.cycles-pp.do_open
1.20 ± 4% -0.2 1.02 ± 10% perf-profile.children.cycles-pp.__x64_sys_kill
1.11 ± 4% -0.2 0.94 ± 10% perf-profile.children.cycles-pp.kill_proc_info
2.28 ± 3% -0.2 2.13 ± 2% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
1.04 ± 4% -0.2 0.88 ± 10% perf-profile.children.cycles-pp.kill_pid_info
2.29 ± 3% -0.2 2.14 ± 3% perf-profile.children.cycles-pp.fs_reclaim_acquire
0.88 ± 4% -0.1 0.73 ± 4% perf-profile.children.cycles-pp.ip_finish_output2
0.96 ± 4% -0.1 0.81 ± 4% perf-profile.children.cycles-pp.ip_output
1.87 ± 5% -0.1 1.72 ± 3% perf-profile.children.cycles-pp.__might_fault
1.14 ± 5% -0.1 1.00 ± 3% perf-profile.children.cycles-pp.sock_write_iter
1.13 ± 5% -0.1 1.00 ± 4% perf-profile.children.cycles-pp.sock_sendmsg
3.18 ± 2% -0.1 3.06 ± 2% perf-profile.children.cycles-pp.mark_lock
1.63 ± 3% -0.1 1.52 ± 5% perf-profile.children.cycles-pp.step_into
0.72 ± 4% -0.1 0.62 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_nested
0.68 ± 2% -0.1 0.58 ± 3% perf-profile.children.cycles-pp.__local_bh_enable_ip
0.58 ± 3% -0.1 0.48 ± 8% perf-profile.children.cycles-pp.do_send_sig_info
0.46 ± 11% -0.1 0.36 ± 8% perf-profile.children.cycles-pp.__x64_sys_rt_sigreturn
1.22 ± 5% -0.1 1.12 ± 3% perf-profile.children.cycles-pp.trace_hardirqs_off
0.32 ± 6% -0.1 0.23 ± 3% perf-profile.children.cycles-pp.strnlen_user
0.94 ± 3% -0.1 0.85 ± 4% perf-profile.children.cycles-pp.lockdep_hardirqs_off
1.22 ± 4% -0.1 1.14 ± 3% perf-profile.children.cycles-pp.copy_strings
0.55 ± 2% -0.1 0.46 ± 6% perf-profile.children.cycles-pp.do_softirq
0.33 ± 9% -0.1 0.24 ± 10% perf-profile.children.cycles-pp.aa_get_task_label
0.50 ± 7% -0.1 0.41 ± 6% perf-profile.children.cycles-pp.net_rx_action
0.48 ± 6% -0.1 0.40 ± 6% perf-profile.children.cycles-pp.__napi_poll
0.61 ± 5% -0.1 0.53 ± 9% perf-profile.children.cycles-pp.do_truncate
0.46 ± 5% -0.1 0.39 ± 6% perf-profile.children.cycles-pp.process_backlog
1.44 ± 4% -0.1 1.37 ± 2% perf-profile.children.cycles-pp.terminate_walk
1.50 ± 3% -0.1 1.43 ± 3% perf-profile.children.cycles-pp.__open64_nocancel
0.80 ± 3% -0.1 0.73 ± 3% perf-profile.children.cycles-pp.get_user_arg_ptr
0.36 ± 9% -0.1 0.29 ± 6% perf-profile.children.cycles-pp.kernel_wait4
0.37 ± 9% -0.1 0.30 ± 8% perf-profile.children.cycles-pp.inet_recvmsg
0.18 ± 12% -0.1 0.11 ± 15% perf-profile.children.cycles-pp.__waitpid
0.54 ± 5% -0.1 0.47 ± 6% perf-profile.children.cycles-pp.count
0.38 ± 9% -0.1 0.31 ± 9% perf-profile.children.cycles-pp.sock_read_iter
0.36 ± 9% -0.1 0.29 ± 5% perf-profile.children.cycles-pp.__do_sys_wait4
0.34 ± 9% -0.1 0.28 ± 6% perf-profile.children.cycles-pp.do_wait
0.16 ± 13% -0.1 0.10 ± 16% perf-profile.children.cycles-pp.io_serial_out
0.49 ± 5% -0.1 0.43 ± 6% perf-profile.children.cycles-pp.arch_do_signal_or_restart
0.24 ± 12% -0.1 0.18 ± 13% perf-profile.children.cycles-pp.apparmor_task_getsecid
0.24 ± 14% -0.1 0.18 ± 15% perf-profile.children.cycles-pp.tcp_recvmsg
0.31 ± 8% -0.1 0.26 ± 6% perf-profile.children.cycles-pp.ip_local_deliver
0.31 ± 9% -0.1 0.26 ± 6% perf-profile.children.cycles-pp.ip_local_deliver_finish
0.21 ± 15% -0.1 0.15 ± 14% perf-profile.children.cycles-pp.tcp_recvmsg_locked
0.34 ± 6% -0.1 0.29 ± 6% perf-profile.children.cycles-pp.__netif_receive_skb_one_core
0.22 ± 14% -0.1 0.16 ± 9% perf-profile.children.cycles-pp.sighandler
0.46 ± 4% -0.1 0.41 perf-profile.children.cycles-pp.syscall_enter_from_user_mode
0.32 ± 7% -0.1 0.27 ± 5% perf-profile.children.cycles-pp.ip_rcv
0.24 ± 6% -0.0 0.20 ± 11% perf-profile.children.cycles-pp.send_signal
0.07 ± 10% -0.0 0.04 ± 65% perf-profile.children.cycles-pp.__libc_calloc
0.17 ± 10% -0.0 0.13 ± 7% perf-profile.children.cycles-pp.dentry_needs_remove_privs
0.12 ± 13% -0.0 0.08 ± 12% perf-profile.children.cycles-pp.compar1
0.13 ± 10% -0.0 0.10 ± 10% perf-profile.children.cycles-pp.simple_xattr_get
0.19 ± 8% -0.0 0.16 ± 11% perf-profile.children.cycles-pp.__setup_rt_frame
0.16 ± 10% -0.0 0.13 ± 7% perf-profile.children.cycles-pp.security_inode_need_killpriv
0.16 ± 8% -0.0 0.13 ± 6% perf-profile.children.cycles-pp.cap_inode_need_killpriv
0.16 ± 8% -0.0 0.13 ± 6% perf-profile.children.cycles-pp.__vfs_getxattr
0.12 ± 7% -0.0 0.10 ± 16% perf-profile.children.cycles-pp.security_task_kill
0.12 ± 8% -0.0 0.09 ± 15% perf-profile.children.cycles-pp.apparmor_task_kill
0.16 ± 11% -0.0 0.13 ± 8% perf-profile.children.cycles-pp.__alloc_skb
0.08 ± 10% -0.0 0.06 ± 41% perf-profile.children.cycles-pp.sort_rtns_1
0.08 ± 11% -0.0 0.06 ± 5% perf-profile.children.cycles-pp.kmalloc_reserve
0.08 ± 8% -0.0 0.06 ± 5% perf-profile.children.cycles-pp.__kmalloc_node_track_caller
0.06 ± 14% +0.0 0.09 ± 11% perf-profile.children.cycles-pp.prandom_u32
0.09 ± 12% +0.0 0.12 ± 4% perf-profile.children.cycles-pp.lock_pin_lock
0.24 ± 8% +0.0 0.27 ± 9% perf-profile.children.cycles-pp.shift_arg_pages
0.09 ± 19% +0.0 0.13 ± 12% perf-profile.children.cycles-pp.cpuacct_charge
0.01 ±223% +0.1 0.06 ± 12% perf-profile.children.cycles-pp.__pagevec_release
0.00 +0.1 0.06 ± 8% perf-profile.children.cycles-pp.select_idle_sibling
0.05 ± 46% +0.1 0.10 ± 19% perf-profile.children.cycles-pp.enqueue_entity
0.40 ± 3% +0.1 0.46 ± 4% perf-profile.children.cycles-pp.__pmd_alloc
0.35 ± 9% +0.1 0.41 ± 7% perf-profile.children.cycles-pp.__pagevec_lru_add
0.21 ± 6% +0.1 0.28 ± 5% perf-profile.children.cycles-pp.__pagevec_lru_add_fn
0.04 ± 71% +0.1 0.11 ± 10% perf-profile.children.cycles-pp.preempt_schedule_thunk
0.04 ± 71% +0.1 0.11 ± 10% perf-profile.children.cycles-pp.preempt_schedule_common
0.09 ± 10% +0.1 0.16 ± 11% perf-profile.children.cycles-pp.irqentry_exit
0.14 ± 7% +0.1 0.22 ± 10% perf-profile.children.cycles-pp.page_add_new_anon_rmap
0.11 ± 9% +0.1 0.19 ± 11% perf-profile.children.cycles-pp.update_load_avg
0.00 +0.1 0.07 ± 12% perf-profile.children.cycles-pp.rcu_note_context_switch
1.08 ± 5% +0.1 1.16 ± 4% perf-profile.children.cycles-pp.wp_page_copy
0.00 +0.1 0.08 ± 23% perf-profile.children.cycles-pp.put_prev_entity
0.00 +0.1 0.08 ± 17% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.06 ± 11% +0.1 0.14 ± 17% perf-profile.children.cycles-pp.enqueue_task_fair
0.83 ± 3% +0.1 0.92 ± 4% perf-profile.children.cycles-pp.tlb_finish_mmu
0.82 ± 3% +0.1 0.91 ± 4% perf-profile.children.cycles-pp.tlb_flush_mmu
0.00 +0.1 0.09 ± 14% perf-profile.children.cycles-pp.perf_tp_event
0.79 ± 3% +0.1 0.88 ± 4% perf-profile.children.cycles-pp.release_pages
0.00 +0.1 0.09 ± 15% perf-profile.children.cycles-pp.preempt_schedule_irq
0.38 ± 7% +0.1 0.48 ± 7% perf-profile.children.cycles-pp._raw_spin_unlock_irq
0.00 +0.1 0.11 ± 14% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
0.04 ± 71% +0.1 0.14 ± 17% perf-profile.children.cycles-pp.ttwu_do_activate
0.12 ± 12% +0.1 0.23 ± 12% perf-profile.children.cycles-pp.pick_next_task_fair
0.09 ± 9% +0.1 0.22 ± 8% perf-profile.children.cycles-pp.dequeue_entity
0.00 +0.1 0.14 ± 15% perf-profile.children.cycles-pp.asm_sysvec_reschedule_ipi
0.11 ± 11% +0.1 0.25 ± 7% perf-profile.children.cycles-pp.dequeue_task_fair
0.06 ± 48% +0.1 0.20 ± 8% perf-profile.children.cycles-pp.select_task_rq_fair
0.01 ±223% +0.1 0.15 ± 11% perf-profile.children.cycles-pp._find_next_bit
0.22 ± 7% +0.1 0.37 ± 5% perf-profile.children.cycles-pp.finish_task_switch
1.19 ± 7% +0.2 1.34 perf-profile.children.cycles-pp.unlink_anon_vmas
0.27 ± 6% +0.2 0.43 ± 9% perf-profile.children.cycles-pp.update_curr
0.99 ± 6% +0.2 1.15 ± 3% perf-profile.children.cycles-pp.remove_vma
0.12 ± 12% +0.2 0.35 ± 5% perf-profile.children.cycles-pp.schedule_idle
0.48 ± 21% +0.3 0.81 ± 6% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.49 ± 8% +0.4 0.84 ± 4% perf-profile.children.cycles-pp.__mod_lruvec_page_state
0.99 ± 9% +0.4 1.37 ± 3% perf-profile.children.cycles-pp.do_raw_spin_lock
2.95 ± 4% +0.4 3.33 ± 3% perf-profile.children.cycles-pp.memcg_slab_free_hook
0.50 ± 11% +0.4 0.88 ± 6% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.28 ± 7% +0.5 0.76 ± 3% perf-profile.children.cycles-pp.schedule
0.16 ± 10% +0.6 0.76 ± 2% perf-profile.children.cycles-pp.try_to_wake_up
0.00 +0.8 0.81 ± 4% perf-profile.children.cycles-pp.__queue_work
0.55 ± 4% +0.8 1.37 ± 3% perf-profile.children.cycles-pp.__schedule
0.25 ± 9% +0.8 1.10 ± 4% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
0.00 +0.8 0.85 ± 4% perf-profile.children.cycles-pp.queue_work_on
0.01 ±223% +0.9 0.90 ± 4% perf-profile.children.cycles-pp.cgroup_rstat_updated
0.57 ± 8% +0.9 1.51 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.00 +1.5 1.50 ± 3% perf-profile.children.cycles-pp.mem_cgroup_css_rstat_flush
4.51 +1.9 6.41 perf-profile.children.cycles-pp.intel_idle
7.44 +2.0 9.41 ± 3% perf-profile.children.cycles-pp.cpuidle_enter
7.44 +2.0 9.41 ± 3% perf-profile.children.cycles-pp.cpuidle_enter_state
7.66 +2.4 10.03 ± 5% perf-profile.children.cycles-pp.start_secondary
8.19 +2.5 10.65 ± 5% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
8.19 +2.5 10.65 ± 5% perf-profile.children.cycles-pp.cpu_startup_entry
8.19 +2.5 10.65 ± 5% perf-profile.children.cycles-pp.do_idle
0.00 +3.8 3.81 perf-profile.children.cycles-pp.cgroup_rstat_flush_locked
0.00 +3.8 3.85 perf-profile.children.cycles-pp.cgroup_rstat_flush_irqsafe
1.06 ± 12% +4.0 5.08 perf-profile.children.cycles-pp.process_one_work
0.00 +4.1 4.06 perf-profile.children.cycles-pp.mem_cgroup_flush_stats
2.17 ± 6% +4.6 6.73 ± 2% perf-profile.children.cycles-pp.kthread
2.27 ± 6% +4.6 6.85 ± 2% perf-profile.children.cycles-pp.ret_from_fork
1.09 ± 12% +4.6 5.71 perf-profile.children.cycles-pp.worker_thread
1.99 ± 10% -0.5 1.48 ± 6% perf-profile.self.cycles-pp.div_long
2.06 ± 8% -0.5 1.59 ± 4% perf-profile.self.cycles-pp.add_int
1.55 ± 7% -0.3 1.22 ± 8% perf-profile.self.cycles-pp.__strcat_chk
1.00 ± 12% -0.3 0.73 ± 10% perf-profile.self.cycles-pp.memcpy_toio
1.12 ± 8% -0.2 0.89 ± 3% perf-profile.self.cycles-pp.add_short
2.45 ± 4% -0.2 2.24 ± 3% perf-profile.self.cycles-pp.debug_lockdep_rcu_enabled
0.82 ± 11% -0.2 0.61 ± 10% perf-profile.self.cycles-pp.div_int
0.16 ± 13% -0.1 0.10 ± 16% perf-profile.self.cycles-pp.io_serial_out
0.11 ± 14% -0.0 0.08 ± 12% perf-profile.self.cycles-pp.compar1
0.08 ± 11% -0.0 0.06 ± 41% perf-profile.self.cycles-pp.sort_rtns_1
0.00 +0.1 0.06 ± 16% perf-profile.self.cycles-pp.__schedule
0.01 ±223% +0.1 0.11 ± 10% perf-profile.self.cycles-pp.cgroup_rstat_updated
0.00 +0.1 0.14 ± 9% perf-profile.self.cycles-pp._find_next_bit
0.00 +0.2 0.23 ± 8% perf-profile.self.cycles-pp.cgroup_rstat_flush_locked
3.62 ± 3% +0.2 3.86 ± 2% perf-profile.self.cycles-pp.lock_release
0.47 ± 22% +0.3 0.80 ± 5% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.00 +1.4 1.39 ± 3% perf-profile.self.cycles-pp.mem_cgroup_css_rstat_flush
4.51 +1.9 6.41 perf-profile.self.cycles-pp.intel_idle
0.05 ± 7% +112.1% 0.11 ± 8% perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.06 ± 7% +212.1% 0.18 ± 65% perf-sched.sch_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.02 ± 22% +198.9% 0.06 ± 14% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
0.03 ± 80% -62.9% 0.01 ± 39% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
0.11 ± 39% +105.5% 0.23 ± 28% perf-sched.sch_delay.avg.ms.io_schedule.__lock_page_killable.filemap_fault.__do_fault
0.05 ± 48% -65.6% 0.02 ± 74% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__dentry_kill.dput
0.05 ± 65% -79.6% 0.01 ±103% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.lockref_put_or_lock.dput
0.02 ±134% -92.5% 0.00 ± 67% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.pick_file.close_fd
0.10 ± 23% -64.9% 0.04 ± 26% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.zap_pte_range.unmap_page_range
0.00 ±152% +3058.7% 0.14 ±138% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.release_task.wait_task_zombie
0.00 ±151% +852.7% 0.04 ± 45% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.wait_task_zombie.do_wait
0.05 ±178% -90.8% 0.00 ± 83% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.do_send_sig_info.kill_pid_info
0.00 ±223% +11900.0% 0.20 ±183% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.thread_group_cputime_adjusted.wait_task_zombie
0.02 ± 10% +708.3% 0.13 ± 28% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_write_unlock_irq.release_task.wait_task_zombie
0.22 ±172% -98.8% 0.00 ± 80% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk.fd_install.do_sys_openat2.do_sys_open
0.14 ±178% -97.4% 0.00 ± 94% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk.perf_iterate_sb.perf_event_mmap.__install_special_mapping
0.05 ±171% -92.6% 0.00 ± 89% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.lock_acquire.mntput_no_expire
0.01 ± 49% -80.1% 0.00 ±158% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.d_alloc_parallel.__lookup_slow
0.06 ± 56% -98.0% 0.00 ± 98% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_free.remove_vma
0.09 ±131% -94.1% 0.01 ±108% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.ktime_get_coarse_real_ts64.current_time
0.01 ± 72% +1774.7% 0.12 ± 81% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.down_read_trylock
0.11 ±150% -92.3% 0.01 ±120% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.kmem_cache_alloc
0.18 ±137% -99.0% 0.00 ± 72% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.memcg_slab_free_hook
0.02 ±111% -95.3% 0.00 ± 75% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.obj_cgroup_charge_pages
0.11 ±116% -82.0% 0.02 ± 79% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.___might_sleep
0.02 ± 47% -86.1% 0.00 ± 81% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.refill_obj_stock.memcg_slab_free_hook
0.01 ± 48% -91.1% 0.00 ±115% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.kernel_clone.__do_sys_clone.do_syscall_64
0.14 ±203% -97.8% 0.00 ± 66% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.kmem_cache_alloc.__alloc_file
0.03 ±136% -93.7% 0.00 ± 67% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.kmem_cache_alloc.anon_vma_clone
0.34 ±213% -99.1% 0.00 ±106% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.mntput_no_expire.terminate_walk
0.21 ±136% -98.7% 0.00 ± 47% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release._copy_from_user.copy_strings
0.03 ±122% -89.4% 0.00 ±115% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.get_mem_cgroup_from_objcg.obj_cgroup_charge_pages
0.01 ± 37% -89.7% 0.00 ± 54% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.anon_vma_clone
0.05 ±184% -97.4% 0.00 ± 99% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.anon_vma_fork
0.02 ± 84% -87.8% 0.00 ± 79% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.getname_flags
0.03 ± 87% -90.9% 0.00 ± 95% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.mem_cgroup_uncharge_list.release_pages
0.14 ± 69% -93.8% 0.01 ±148% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.memcg_slab_free_hook.kmem_cache_free
0.01 ±201% +1003.1% 0.08 ± 91% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__anon_vma_prepare
0.01 ±119% +29990.8% 1.71 ±236% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_fork
0.01 ± 82% +615.4% 0.09 ±107% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.filename_create.do_linkat.__x64_sys_link
0.02 ± 23% +2589.1% 0.46 ±211% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2
11.95 ± 21% +48.1% 17.70 ± 18% perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
2.05 ± 44% +307.3% 8.34 ± 29% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
1.44 ± 91% +369.6% 6.79 ± 78% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
0.05 ± 94% +4699.5% 2.54 ±162% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.01 ±223% +23956.2% 1.28 ±213% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk.__local_bh_enable_ip.copy_fpstate_to_sigframe.__setup_rt_frame
0.57 ± 96% +933.8% 5.91 ± 41% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_read_unlock.wait_task_zombie.do_wait
5.55 ± 26% +138.7% 13.24 ± 22% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.filemap_map_pages.do_fault
0.02 ± 56% +13714.0% 2.39 ±154% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.get_signal.arch_do_signal_or_restart
0.00 ±152% +33881.0% 1.53 ±161% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.release_task.wait_task_zombie
0.00 ±100% +22738.6% 0.76 ±183% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.ret_from_fork.[unknown]
0.02 ± 22% +15927.9% 3.31 ± 98% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.shmem_add_to_page_cache.shmem_getpage_gfp
0.01 ±141% +7238.1% 0.44 ± 78% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.wait_task_zombie.do_wait
0.08 ±159% +2093.7% 1.84 ±186% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.delete_from_page_cache.truncate_inode_page
0.01 ±100% +13227.7% 1.09 ±223% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.release_pages.__pagevec_release
10.04 ± 34% +75.6% 17.63 ± 24% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.sched_move_task.do_exit
0.00 ±223% +96714.3% 1.61 ±215% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.thread_group_cputime_adjusted.wait_task_zombie
0.02 ± 14% +33245.7% 6.72 ± 69% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_write_unlock_irq.release_task.wait_task_zombie
0.00 ±223% +6237.1% 0.16 ±209% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk.lru_add_drain.__pagevec_release.shmem_undo_range
0.34 ±138% +781.6% 3.03 ± 81% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk.lru_add_drain.exit_mmap.mmput
0.26 ±182% -91.0% 0.02 ± 89% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk.perf_iterate_sb.perf_event_mmap.__install_special_mapping
0.35 ±209% +967.7% 3.74 ± 52% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk.try_to_wake_up.wake_up_q.rwsem_wake
0.04 ± 12% -14.6% 0.04 ± 9% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk.vprintk_emit.devkmsg_emit.constprop
0.33 ± 97% +1104.9% 3.94 ±146% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.memcg_slab_free_hook.kmem_cache_free
0.02 ± 98% +1101.6% 0.21 ±151% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__mem_cgroup_charge.mem_cgroup_charge
0.00 ±223% +47660.0% 0.40 ±233% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.count.isra
0.07 ±208% +4562.7% 3.28 ±130% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.find_vma.do_user_addr_fault
0.11 ±164% +3912.2% 4.51 ± 72% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.handle_mm_fault.do_user_addr_fault
0.20 ±102% -90.8% 0.02 ±101% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_free.remove_vma
0.46 ± 85% +857.7% 4.38 ± 91% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.__might_fault
0.01 ± 92% +35773.7% 4.42 ± 89% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.down_read_trylock
0.03 ± 94% +5361.6% 1.58 ±112% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.get_mem_cgroup_from_objcg
0.12 ±136% +638.2% 0.90 ± 83% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.mem_cgroup_from_task
0.00 ±142% +816.5% 0.02 ± 89% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.mem_cgroup_charge.wp_page_copy
0.00 ±141% +414.3% 0.02 ± 32% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.pagecache_get_page.shmem_getpage_gfp
0.19 ±209% +3391.3% 6.52 ± 79% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__alloc_file
0.01 ±187% +32718.1% 2.68 ±109% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__anon_vma_prepare
0.05 ±201% +2.7e+05% 144.31 ±242% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__d_alloc
0.00 ±141% +71174.5% 3.33 ±149% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_clone
0.01 ±108% +2.4e+06% 146.00 ±239% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_fork
0.02 ±141% +14953.1% 2.31 ±110% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.vm_area_alloc
0.01 ± 83% +27325.7% 2.74 ±103% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.vm_area_dup
0.02 ± 63% +17882.6% 4.44 ± 37% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.memcg_slab_free_hook.kmem_cache_free
0.15 ±183% +3099.4% 4.67 ±102% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.anon_vma_clone.anon_vma_fork.dup_mmap
0.34 ±117% +1079.3% 4.02 ± 99% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.filename_create.do_linkat.__x64_sys_link
2.37 ± 46% +6268.4% 151.16 ±229% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2
0.63 ±209% +1026.8% 7.12 ± 82% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.unlink_anon_vmas.free_pgtables.exit_mmap
3.11 ± 8% +1463.8% 48.56 ±217% perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
17.70 ± 20% +826.4% 163.98 ±209% perf-sched.total_sch_delay.max.ms
29.22 -73.2% 7.84 ± 8% perf-sched.total_wait_and_delay.average.ms
30290 +285.8% 116849 ± 9% perf-sched.total_wait_and_delay.count.ms
6631 -55.2% 2969 perf-sched.total_wait_and_delay.max.ms
29.17 -73.3% 7.78 ± 8% perf-sched.total_wait_time.average.ms
6631 -55.2% 2968 perf-sched.total_wait_time.max.ms
59.63 ± 13% -17.3% 49.31 ± 6% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk.cpu_stop_queue_work.affine_move_task.__set_cpus_allowed_ptr
310.64 ± 21% -33.9% 205.45 ± 9% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
71.02 ± 4% -96.6% 2.42 ± 9% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
134.33 ± 44% +1034.5% 1524 ± 12% perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
374.00 ± 3% +381.6% 1801 ± 12% perf-sched.wait_and_delay.count.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.filemap_map_pages.do_fault
382.67 ± 6% +469.6% 2179 ± 12% perf-sched.wait_and_delay.count.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.zap_pte_range.unmap_page_range
36.33 ± 5% +76.1% 64.00 ± 10% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
2283 ± 4% +3039.1% 71671 ± 10% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
14.07 ± 30% +5035.2% 722.29 ± 60% perf-sched.wait_and_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.zap_pte_range.unmap_page_range
0.38 ± 80% +165.3% 1.02 ± 28% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.93 ±193% +8638.6% 81.15 ±153% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.__mutex_lock.perf_poll.do_sys_poll
0.00 ±223% +3392.9% 0.05 ±108% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__pte_alloc.do_anonymous_page
0.00 ±223% +2573.2% 0.07 ±125% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__pte_alloc.finish_fault
2.90 ± 23% -88.9% 0.32 ± 67% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.copy_pte_range.copy_page_range
0.33 ±157% +6283.7% 21.35 ± 34% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.do_anonymous_page.__handle_mm_fault
0.08 ±223% +1416.0% 1.15 ± 56% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.do_exit.do_group_exit
0.00 ±141% +617.3% 0.02 ± 36% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.follow_page_pte.__get_user_pages
1.23 ± 35% -83.1% 0.21 ± 46% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.pick_file.close_fd
5.03 ± 30% -75.4% 1.24 ± 83% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.__x64_sys_rt_sigreturn.do_syscall_64
2.90 ± 43% -68.5% 0.91 ± 44% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.do_sigaction.__x64_sys_rt_sigaction
6.23 ± 29% +199.3% 18.65 ± 28% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.shmem_add_to_page_cache.shmem_getpage_gfp
1.24 ±141% +367.8% 5.79 ± 18% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.wait_task_zombie.do_wait
6.47 ± 13% -68.7% 2.02 ± 13% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.delete_from_page_cache.truncate_inode_page
0.51 ± 96% +384.9% 2.49 ± 40% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.rcu_preempt_deferred_qs_irqrestore.__rcu_read_unlock
2.08 ± 55% -50.8% 1.02 ± 87% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.sched_fork.copy_process
0.84 ± 16% +96.2% 1.65 ± 18% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.sched_move_task.do_exit
1.04 ±222% +455.7% 5.80 ± 18% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.thread_group_cputime_adjusted.wait_task_zombie
2.40 ± 35% -61.2% 0.93 ± 50% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.wake_up_new_task.kernel_clone
59.62 ± 13% -17.3% 49.31 ± 6% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.cpu_stop_queue_work.affine_move_task.__set_cpus_allowed_ptr
0.00 ±223% +20410.7% 0.82 ±158% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.d_alloc_parallel.lookup_open.path_openat
0.28 ± 86% +644.7% 2.07 ±132% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.lru_add_drain.exit_mmap.mmput
0.00 ±223% +1.8e+05% 3.69 ±216% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.touch_atime.pick_link.step_into
6.29 ± 21% -82.8% 1.08 ± 53% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.vfs_write.ksys_write.do_syscall_64
4.32 ±106% -99.9% 0.00 ±244% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.__local_bh_enable_ip.arch_do_signal_or_restart
19.54 ±122% -89.4% 2.08 ± 33% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.ktime_get_coarse_real_ts64.current_time
5.89 ±176% -89.6% 0.61 ± 45% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.lock_acquire.__might_fault
11.38 ±109% -89.2% 1.23 ± 33% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.lock_acquire.kmem_cache_alloc
2.46 ±105% -80.6% 0.48 ±244% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.lock_release.strncpy_from_user
0.00 ±223% +11214.3% 0.13 ±186% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.___might_sleep.kmem_cache_alloc
0.00 ±223% +2287.8% 0.03 ±151% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.___might_sleep.task_work_run
0.00 ±223% +1e+06% 22.50 ±239% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__alloc_pages.alloc_pages_vma
0.02 ±214% +557.3% 0.15 ± 91% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__mem_cgroup_charge.mem_cgroup_charge
0.06 ±167% +521.5% 0.40 ± 93% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__might_sleep.do_user_addr_fault
0.01 ±223% +1226.2% 0.08 ±104% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__might_sleep.kmem_cache_alloc
0.00 ±223% +7641.1% 0.21 ±199% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__pmd_alloc.copy_page_range
0.00 ±223% +1957.1% 0.04 ± 93% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__rcu_read_lock.get_obj_cgroup_from_current
0.00 ±150% +435.7% 0.02 ± 46% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__rcu_read_unlock.memcg_slab_free_hook
0.00 ±223% +4576.2% 0.07 ±133% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__vma_link_rb.vma_link
0.00 ±223% +2139.3% 0.03 ± 74% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi._copy_from_user.copy_strings
0.04 ±211% +2642.2% 1.19 ± 83% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.account_kernel_stack.dup_task_struct
0.00 ±223% +31985.7% 1.44 ± 85% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.acct_collect.do_exit
0.00 ±223% +3475.5% 0.04 ± 99% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.call_rcu.__dentry_kill
0.01 ±223% +9379.0% 0.54 ±111% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.check_stack_object.__check_object_size
0.00 ±158% +2526.9% 0.07 ±164% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.copy_strings.isra
0.08 ±223% +1204.5% 1.07 ± 94% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.copy_user_enhanced_fast_string.copyout
0.00 ±223% +1082.9% 0.02 ± 67% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.debug_lockdep_rcu_enabled.get_obj_cgroup_from_current
0.08 ±138% +744.4% 0.64 ± 55% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.debug_lockdep_rcu_enabled.rcu_read_lock_held_common
0.07 ± 67% +498.8% 0.45 ± 30% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.do_user_addr_fault.exc_page_fault
0.17 ±150% +399.5% 0.87 ± 36% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.exit_to_user_mode_prepare
0.11 ±140% +886.9% 1.08 ± 41% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.find_vma.do_user_addr_fault
0.00 ±223% +4042.9% 0.04 ±154% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.free_unref_page_list.release_pages
0.00 ±223% +45705.7% 1.15 ±118% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.generic_perform_write.__generic_file_write_iter
0.00 ±223% +3517.1% 0.03 ±113% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.get_mem_cgroup_from_objcg.obj_cgroup_charge_pages
0.10 ± 99% +270.0% 0.36 ± 33% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.handle_mm_fault.do_user_addr_fault
0.00 ±223% +14471.4% 0.36 ±122% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_free.do_sys_openat2
0.02 ±223% +3189.9% 0.56 ±161% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.link_path_walk.part
0.00 ±223% +4738.1% 0.07 ±180% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.__task_pid_nr_ns
0.00 ±223% +34475.0% 0.46 ± 70% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.aa_file_perm
0.00 ±223% +7828.6% 0.16 ±127% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.cgroup_throttle_swaprate
0.08 ± 97% +1132.5% 0.94 ± 19% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.handle_mm_fault
0.00 ±223% +17814.3% 0.42 ± 89% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.mnt_want_write
0.00 ±223% +7185.7% 0.08 ±101% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.mntput_no_expire
0.09 ±168% +8240.6% 7.37 ±225% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.pagecache_get_page
0.06 ±198% +656.0% 0.47 ±102% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.xas_start
0.00 ±223% +20115.7% 0.67 ± 84% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.aa_file_perm
0.00 ±223% +4428.6% 0.05 ± 99% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.get_user_arg_ptr
0.03 ±156% +893.6% 0.34 ± 87% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.handle_mm_fault
0.63 ±102% -90.6% 0.06 ± 73% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.memcg_slab_free_hook.kmem_cache_free
0.00 ±223% +66634.7% 0.78 ±156% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.path_init.path_lookupat
0.01 ±223% +5980.4% 0.49 ±110% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.rcu_read_lock_held_common.rcu_read_lock_held
0.01 ±104% +4944.9% 0.29 ± 76% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.step_into.walk_component
0.11 ±223% +760.9% 0.93 ± 99% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.strncpy_from_user.getname_flags
1.57 ±141% +269.0% 5.78 ± 19% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.thread_group_cputime.thread_group_cputime_adjusted
0.07 ±223% +915.4% 0.67 ± 67% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.unmap_page_range.unmap_vmas
0.01 ±223% +7943.2% 0.50 ±123% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.unmap_vmas.exit_mmap
2.91 ± 73% -87.7% 0.36 ± 95% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.kmem_cache_alloc.vm_area_dup
0.00 ±173% +1014.3% 0.04 ± 88% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.create_elf_tables.isra
1.94 ± 53% -83.8% 0.31 ± 39% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.get_obj_cgroup_from_current.kmem_cache_alloc
4.04 ± 46% -91.6% 0.34 ± 87% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.anon_vma_clone
2.23 ± 33% -80.9% 0.43 ± 61% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.anon_vma_fork
2.36 ± 42% -81.8% 0.43 ± 73% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.vm_area_dup
0.15 ±164% +417.2% 0.78 ± 64% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.up_read.do_user_addr_fault
0.41 ±166% +1891.6% 8.25 ± 60% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__alloc_file
0.04 ±203% +39096.9% 15.81 ± 44% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__d_alloc
1.17 ±222% +1833.9% 22.67 ± 43% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_clone
2.06 ±139% +618.3% 14.83 ± 92% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_fork
2.16 ± 34% -72.0% 0.60 ± 52% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_lockdep_current_cpu_online.rcu_read_lock_held_common.rcu_read_lock_held
310.63 ± 21% -33.9% 205.44 ± 9% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
71.01 ± 4% -96.6% 2.40 ± 9% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
1.50 ±134% +782.0% 13.23 ± 58% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.93 ±193% +32546.6% 303.18 ±147% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.__mutex_lock.perf_poll.do_sys_poll
0.57 ±222% +771.6% 4.94 ± 78% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.__mutex_lock.uprobe_clear_state.mmput
7.07 ± 48% +176.2% 19.53 ± 14% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_read_unlock.wait_task_zombie.do_wait
0.00 ±223% +7686.3% 0.32 ±155% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__pte_alloc.do_anonymous_page
0.00 ±223% +21741.1% 0.58 ±155% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__pte_alloc.finish_fault
2.40 ±124% +41821.9% 1004 perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.do_anonymous_page.__handle_mm_fault
0.08 ±223% +4129.1% 3.21 ± 41% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.do_exit.do_group_exit
0.02 ±158% +824.4% 0.18 ± 91% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.follow_page_pte.__get_user_pages
2.25 ±141% +139.2% 5.38 ± 68% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.shmem_mknod.lookup_open
3.11 ± 84% +23006.2% 718.99 ± 62% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.wp_page_copy.__handle_mm_fault
11.01 ± 26% +6440.1% 720.17 ± 61% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.zap_pte_range.unmap_page_range
1.51 ±150% +538.7% 9.66 ± 21% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.release_task.wait_task_zombie
9.42 ± 30% +13230.7% 1255 ± 21% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.shmem_add_to_page_cache.shmem_getpage_gfp
1.81 ±152% +830.7% 16.83 ± 35% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.wait_task_zombie.do_wait
8.13 ± 35% +140.9% 19.60 ± 39% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.delete_from_page_cache.truncate_inode_page
1.67 ±155% +419.5% 8.66 ± 42% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.do_wait.kernel_wait4
3.27 ±114% +325.8% 13.93 ± 33% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.rcu_preempt_deferred_qs_irqrestore.__rcu_read_unlock
3.40 ±100% +322.3% 14.35 ± 26% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.release_pages.__pagevec_release
2.40 ± 53% +232.2% 7.98 ± 51% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.release_pages.tlb_flush_mmu
12.67 ± 30% +53.0% 19.38 ± 15% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.sched_move_task.do_exit
1.04 ±222% +1216.2% 13.75 ± 30% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.thread_group_cputime_adjusted.wait_task_zombie
6.34 ± 23% +2406.8% 158.86 ±216% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_write_unlock_irq.release_task.wait_task_zombie
0.00 ±223% +48803.6% 1.96 ±155% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.d_alloc_parallel.lookup_open.path_openat
1.16 ±223% +979.8% 12.49 ± 46% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.lru_add_drain.__pagevec_release.shmem_undo_range
2.32 ± 75% +6569.2% 154.86 ±222% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.lru_add_drain.exit_mmap.mmput
7.51 ± 49% +13964.7% 1056 ± 76% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.lru_cache_add.shmem_getpage_gfp.shmem_write_begin
0.00 ±223% +1.9e+05% 3.86 ±205% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.touch_atime.pick_link.step_into
1.89 ±120% +217.2% 6.01 ± 34% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.try_to_wake_up.wake_up_q.rwsem_wake
5.55 ± 80% -100.0% 0.00 ±244% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.__local_bh_enable_ip.arch_do_signal_or_restart
0.00 ±223% +62691.8% 1.47 ±194% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.___might_sleep.kmem_cache_alloc
0.00 ±223% +7173.5% 0.08 ±209% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.___might_sleep.task_work_run
0.00 ±223% +6.5e+06% 269.83 ±239% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__alloc_pages.alloc_pages_vma
0.69 ±116% +1181.7% 8.87 ± 44% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__handle_mm_fault.handle_mm_fault
0.07 ±213% +6062.5% 4.12 ± 92% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__mem_cgroup_charge.mem_cgroup_charge
0.07 ±164% +5287.8% 3.52 ± 90% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__might_sleep.do_user_addr_fault
0.01 ±223% +7326.2% 0.45 ±108% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__might_sleep.kmem_cache_alloc
1.13 ±223% +548.7% 7.35 ±110% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__might_sleep.pagecache_get_page
0.00 ±223% +56926.8% 1.52 ±214% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__pmd_alloc.copy_page_range
0.00 ±223% +2255.3% 0.09 ± 73% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__rcu_read_lock.get_obj_cgroup_from_current
0.01 ±149% +673.6% 0.05 ± 73% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__rcu_read_unlock.memcg_slab_free_hook
0.01 ±223% +1334.2% 0.13 ± 78% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__virt_addr_valid.__check_object_size
0.00 ±223% +19252.4% 0.29 ±163% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__vma_link_rb.vma_link
0.00 ±223% +2439.3% 0.10 ±113% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi._copy_from_user.copy_strings
0.05 ±200% +10677.3% 4.89 ± 79% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.account_kernel_stack.dup_task_struct
0.00 ±223% +1.1e+05% 4.92 ±132% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.acct_collect.do_exit
0.00 ±223% +24379.4% 1.10 ±219% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.anon_vma_interval_tree_insert.anon_vma_clone
0.00 ±223% +14153.1% 0.33 ±122% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.call_rcu.__dentry_kill
0.02 ±223% +27738.5% 4.69 ±122% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.check_stack_object.__check_object_size
0.01 ±141% +15807.9% 1.30 ±197% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.copy_strings.isra
0.08 ±223% +5409.9% 4.52 ± 87% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.copy_user_enhanced_fast_string.copyout
0.33 ±134% +1612.6% 5.58 ± 54% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.debug_lockdep_rcu_enabled.___might_sleep
0.00 ±223% +5000.0% 0.08 ±126% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.debug_lockdep_rcu_enabled.get_obj_cgroup_from_current
0.15 ±138% +3202.9% 4.97 ± 56% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.debug_lockdep_rcu_enabled.rcu_read_lock_held_common
0.53 ± 42% +2440.7% 13.42 ± 36% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.do_user_addr_fault.exc_page_fault
0.52 ±109% +1741.8% 9.54 ± 19% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.exit_to_user_mode_prepare
1.47 ±180% +779.0% 12.90 ± 39% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.exit_to_user_mode_prepare.syscall_exit_to_user_mode
0.34 ±163% +3235.6% 11.44 ± 25% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.find_vma.do_user_addr_fault
0.00 ±223% +13706.9% 0.58 ±215% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.free_unref_page_list.release_pages
0.00 ±223% +1.6e+05% 3.95 ±123% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.generic_perform_write.__generic_file_write_iter
0.00 ±223% +15517.1% 0.13 ±185% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.get_mem_cgroup_from_objcg.obj_cgroup_charge_pages
0.11 ±161% +1.4e+05% 145.15 ±240% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.get_obj_cgroup_from_current.kmem_cache_alloc
0.00 ±223% +8128.6% 0.19 ±135% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.handle_mm_fault.__get_user_pages
0.31 ±105% +3112.1% 9.98 ± 16% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.handle_mm_fault.do_user_addr_fault
0.22 ±135% +2566.3% 5.80 ± 55% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_alloc.__alloc_file
0.01 ±109% +7095.6% 0.71 ±151% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_alloc.security_file_alloc
0.03 ±186% +5975.7% 1.64 ±212% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_alloc.vm_area_alloc
0.17 ±181% +3002.3% 5.41 ± 62% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_alloc.vm_area_dup
0.00 ±223% +1.2e+05% 3.07 ±122% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_free.do_sys_openat2
1.52 ±180% +28630.2% 437.90 ±112% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.ktime_get_coarse_real_ts64.current_time
0.07 ±223% +10748.6% 7.36 ±154% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.link_path_walk.part
0.02 ±179% +15642.6% 2.49 ±141% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.__memcg_kmem_charge_page
1.72 ±121% +740.8% 14.50 ± 80% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.__might_fault
0.00 ±223% +13528.6% 0.20 ±192% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.__task_pid_nr_ns
0.00 ±223% +1.6e+05% 6.69 ± 64% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.aa_file_perm
0.00 ±223% +59020.5% 2.27 ±131% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.cgroup_throttle_swaprate
0.09 ±207% +1812.0% 1.69 ± 94% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.down_read
1.48 ±156% +642.8% 11.02 ± 50% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.down_write
0.00 ±223% +20134.9% 0.91 ±232% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.filemap_get_read_batch
5.13 ± 61% +189.8% 14.88 ± 46% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.fs_reclaim_acquire
0.59 ±203% +1013.4% 6.55 ± 49% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.get_mem_cgroup_from_objcg
0.27 ± 89% +3684.7% 10.33 ± 26% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.handle_mm_fault
0.00 ±223% +3e+05% 7.02 ± 89% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.mnt_want_write
0.00 ±223% +68600.0% 1.60 ±127% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.mntput_no_expire
0.23 ±152% +63872.1% 147.14 ±236% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.pagecache_get_page
0.00 ±223% +17083.2% 0.49 ±172% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.perf_iterate_sb
5.92 ± 38% +104.5% 12.11 ± 29% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.___might_sleep
1.24 ±205% +369.3% 5.81 ± 42% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.mem_cgroup_from_task
2.56 ±108% +5832.7% 151.68 ±228% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.rcu_read_lock_held
0.08 ±204% +6275.1% 5.39 ± 97% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.xas_start
0.18 ±141% +1652.7% 3.18 ±127% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.__alloc_pages
0.00 ±223% +1.5e+05% 4.96 ± 71% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.aa_file_perm
0.13 ±223% +2434.5% 3.35 ± 96% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.get_mem_cgroup_from_mm
0.06 ±173% +4990.9% 2.84 ±128% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.get_mem_cgroup_from_objcg
0.00 ±223% +29529.7% 1.23 ±165% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.get_user_arg_ptr
0.09 ±167% +3688.7% 3.57 ± 89% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.handle_mm_fault
0.10 ±166% +1.4e+05% 144.52 ±241% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.up_read
0.30 ±140% +2334.1% 7.41 ±114% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.page_counter_try_charge.obj_cgroup_charge_pages
0.01 ±223% +18087.1% 2.15 ±148% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.path_openat.do_filp_open
0.01 ±223% +49341.1% 3.96 ±110% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.rcu_read_lock_held_common.rcu_read_lock_held
0.22 ±152% +3582.4% 8.22 ± 81% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.refill_obj_stock.obj_cgroup_charge
0.01 ±104% +94192.9% 5.66 ± 82% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.step_into.walk_component
0.11 ±223% +4235.2% 4.67 ± 86% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.strncpy_from_user.getname_flags
1.63 ±147% +693.9% 12.96 ± 31% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.syscall_enter_from_user_mode.do_syscall_64
0.49 ± 76% +1354.1% 7.06 ± 40% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.syscall_exit_to_user_mode_prepare.syscall_exit_to_user_mode
2.36 ±153% +532.7% 14.90 ± 45% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.thread_group_cputime.thread_group_cputime_adjusted
0.02 ±192% +7757.4% 1.62 ±191% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.try_charge.__mem_cgroup_charge
0.01 ±223% +5863.0% 0.51 ±216% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.uncharge_batch.mem_cgroup_uncharge_list
0.13 ±223% +2934.5% 4.02 ± 52% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.unmap_page_range.unmap_vmas
0.01 ±223% +39090.0% 2.42 ±134% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.unmap_vmas.exit_mmap
5.55 ± 36% +4888.5% 276.68 ±232% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.__perf_sw_event.handle_mm_fault.do_user_addr_fault
0.01 ±160% +4986.6% 0.54 ±201% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.kmem_cache_free.___pte_free_tlb.free_p4d_range
7.37 ± 9% +2047.6% 158.29 ±217% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.fs_reclaim_acquire.kmem_cache_alloc
1.90 ±142% +320.8% 8.00 ± 10% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.pagecache_get_page.shmem_getpage_gfp
0.00 ±150% +7446.6% 0.29 ±107% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.create_elf_tables.isra
0.02 ±120% +7077.1% 1.08 ±215% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.try_to_unlazy.path_openat
0.34 ±149% +2030.2% 7.21 ± 37% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.up_read.do_user_addr_fault
1.38 ±205% +62408.7% 862.20 ± 40% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__alloc_file
0.11 ±215% +8e+05% 861.59 ± 40% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__d_alloc
1.35 ±222% +63833.9% 863.32 ± 39% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_clone
2.09 ±136% +34318.0% 719.28 ± 62% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_fork
0.10 ±223% +1755.1% 1.94 ±162% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.vm_area_alloc
2.24 ±137% +6776.7% 153.76 ±225% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.vm_area_dup
1.02 ±139% +14761.3% 151.86 ±228% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.memcg_slab_free_hook.kmem_cache_free
0.21 ± 66% +703.0% 1.70 ±114% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.__vma_adjust.__split_vma.mprotect_fixup
8.58 ± 26% +3400.3% 300.47 ±147% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.anon_vma_clone.anon_vma_fork.dup_mmap
***************************************************************************************************
lkp-csl-2ap2: 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
=========================================================================================
compiler/cpufreq_governor/disk/filesize/fs/iterations/kconfig/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase/ucode:
gcc-9/performance/1BRD_48G/4M/btrfs/1x/x86_64-rhel-8.3/1t/debian-10.4-x86_64-20200603.cgz/fsyncBeforeClose/lkp-csl-2ap2/24G/fsmark/0x5003006
commit:
0c7c5da68b ("memcg: switch lruvec stats to rstat")
cda0d20525 ("memcg: infrastructure to flush memcg stats")
0c7c5da68b4a09ff cda0d205259a7e7c1f8e296130a
---------------- ---------------------------
%stddev %change %stddev
\ | \
55.18 -6.9% 51.40 fsmark.files_per_sec
120.73 +7.8% 130.20 fsmark.time.elapsed_time
120.73 +7.8% 130.20 fsmark.time.elapsed_time.max
116.29 +7.5% 125.06 fsmark.time.system_time
0.62 +0.3 0.91 mpstat.cpu.all.sys%
9117 ± 4% +154.2% 23180 ± 5% numa-meminfo.node3.Active(anon)
12157 ± 4% +118.1% 26513 ± 5% meminfo.Active(anon)
31896 +59.5% 50869 ± 2% meminfo.Shmem
2274 ± 4% +152.8% 5749 ± 5% numa-vmstat.node3.nr_active_anon
5404 ± 30% +58.1% 8546 ± 18% numa-vmstat.node3.nr_shmem
2274 ± 4% +152.8% 5749 ± 5% numa-vmstat.node3.nr_zone_active_anon
207967 -7.3% 192752 vmstat.io.bo
1.00 +100.0% 2.00 vmstat.procs.r
2604 +872.7% 25334 vmstat.system.cs
19899 ± 5% +12.2% 22332 ± 5% softirqs.CPU1.SCHED
17317 ± 5% +15.3% 19971 ± 5% softirqs.CPU100.SCHED
18092 ± 6% +14.0% 20631 ± 4% softirqs.CPU108.SCHED
32264 ± 14% -13.8% 27817 ± 17% softirqs.CPU109.RCU
18441 ± 3% +11.2% 20507 ± 4% softirqs.CPU118.SCHED
18586 ± 2% +13.6% 21117 ± 7% softirqs.CPU9.SCHED
9915 ± 57% +99.8% 19814 ± 4% softirqs.CPU96.SCHED
3039 ± 4% +118.1% 6628 ± 5% proc-vmstat.nr_active_anon
33813 -1.1% 33430 proc-vmstat.nr_active_file
9198 +9.8% 10100 proc-vmstat.nr_mapped
7974 +59.5% 12722 ± 2% proc-vmstat.nr_shmem
3039 ± 4% +118.1% 6628 ± 5% proc-vmstat.nr_zone_active_anon
33813 -1.1% 33430 proc-vmstat.nr_zone_active_file
4926 ± 5% -55.8% 2176 ± 31% proc-vmstat.numa_hint_faults
3990 ± 22% -63.0% 1475 ± 43% proc-vmstat.numa_hint_faults_local
27189 +74.6% 47479 proc-vmstat.pgactivate
666580 +4.9% 699366 proc-vmstat.pgfault
799.00 ± 21% +87.1% 1495 ± 12% slabinfo.btrfs_trans_handle.active_objs
799.00 ± 21% +87.1% 1495 ± 12% slabinfo.btrfs_trans_handle.num_objs
2044 ± 22% +118.4% 4466 ± 12% slabinfo.ext4_io_end_vec.active_objs
2044 ± 22% +118.4% 4466 ± 12% slabinfo.ext4_io_end_vec.num_objs
15776 ± 5% -7.0% 14666 ± 2% slabinfo.kmalloc-128.num_objs
3212 ± 20% +27.2% 4087 ± 6% slabinfo.posix_timers_cache.active_objs
3226 ± 19% +26.9% 4094 ± 6% slabinfo.posix_timers_cache.num_objs
1151 ± 5% -8.7% 1050 ± 3% slabinfo.task_group.active_objs
1151 ± 5% -8.7% 1050 ± 3% slabinfo.task_group.num_objs
2777 ± 6% +14.6% 3183 ± 5% slabinfo.xfs_efd_item.active_objs
2779 ± 6% +14.6% 3184 ± 5% slabinfo.xfs_efd_item.num_objs
3.698e+09 ± 5% +23.1% 4.551e+09 ± 6% perf-stat.i.branch-instructions
2464 +929.5% 25372 perf-stat.i.context-switches
1.64e+10 ± 4% +18.0% 1.936e+10 ± 8% perf-stat.i.cpu-cycles
5.176e+09 ± 5% +21.1% 6.266e+09 ± 5% perf-stat.i.dTLB-loads
1.736e+10 ± 5% +22.1% 2.121e+10 ± 6% perf-stat.i.instructions
5079 ± 10% +40.6% 7140 ± 11% perf-stat.i.instructions-per-iTLB-miss
3.83 ± 15% -15.3% 3.25 ± 6% perf-stat.i.major-faults
0.09 ± 4% +18.1% 0.10 ± 8% perf-stat.i.metric.GHz
61.30 ± 5% +21.0% 74.16 ± 5% perf-stat.i.metric.M/sec
4956 -2.5% 4835 perf-stat.i.minor-faults
4960 -2.5% 4838 perf-stat.i.page-faults
5096 ± 10% +40.5% 7158 ± 11% perf-stat.overall.instructions-per-iTLB-miss
3.666e+09 ± 5% +23.1% 4.515e+09 ± 6% perf-stat.ps.branch-instructions
2442 +930.3% 25167 perf-stat.ps.context-switches
1.626e+10 ± 4% +18.1% 1.921e+10 ± 8% perf-stat.ps.cpu-cycles
5.132e+09 ± 5% +21.1% 6.217e+09 ± 5% perf-stat.ps.dTLB-loads
1.721e+10 ± 5% +22.2% 2.104e+10 ± 6% perf-stat.ps.instructions
3.80 ± 15% -15.4% 3.22 ± 6% perf-stat.ps.major-faults
4909 -2.4% 4793 perf-stat.ps.minor-faults
4913 -2.4% 4796 perf-stat.ps.page-faults
2.086e+12 ± 4% +31.9% 2.751e+12 ± 5% perf-stat.total.instructions
525504 ± 4% +9.9% 577408 ± 2% interrupts.CAL:Function_call_interrupts
2667 ± 4% +25.3% 3341 ± 25% interrupts.CPU0.CAL:Function_call_interrupts
2533 ± 4% +128.4% 5785 ± 57% interrupts.CPU1.CAL:Function_call_interrupts
2204 ± 26% +111.5% 4663 ± 87% interrupts.CPU10.CAL:Function_call_interrupts
6.50 ± 90% +3190.1% 213.86 ±144% interrupts.CPU102.RES:Rescheduling_interrupts
215.33 ± 33% +117.8% 469.00 ± 42% interrupts.CPU108.NMI:Non-maskable_interrupts
215.33 ± 33% +117.8% 469.00 ± 42% interrupts.CPU108.PMI:Performance_monitoring_interrupts
202.00 ± 29% +56.4% 316.00 ± 32% interrupts.CPU117.NMI:Non-maskable_interrupts
202.00 ± 29% +56.4% 316.00 ± 32% interrupts.CPU117.PMI:Performance_monitoring_interrupts
222.17 ± 34% +114.0% 475.43 ± 40% interrupts.CPU12.NMI:Non-maskable_interrupts
222.17 ± 34% +114.0% 475.43 ± 40% interrupts.CPU12.PMI:Performance_monitoring_interrupts
2452 +65.9% 4069 ± 62% interrupts.CPU122.CAL:Function_call_interrupts
193.50 ± 32% +93.1% 373.57 ± 36% interrupts.CPU14.NMI:Non-maskable_interrupts
193.50 ± 32% +93.1% 373.57 ± 36% interrupts.CPU14.PMI:Performance_monitoring_interrupts
192.17 ± 38% +127.9% 438.00 ± 45% interrupts.CPU15.NMI:Non-maskable_interrupts
192.17 ± 38% +127.9% 438.00 ± 45% interrupts.CPU15.PMI:Performance_monitoring_interrupts
7.17 ±130% +6177.1% 449.86 ±129% interrupts.CPU17.RES:Rescheduling_interrupts
2472 ± 2% +56.4% 3868 ± 64% interrupts.CPU26.CAL:Function_call_interrupts
9.67 ±155% +2832.0% 283.43 ±138% interrupts.CPU3.RES:Rescheduling_interrupts
3.83 ± 96% +7223.0% 280.71 ±106% interrupts.CPU4.RES:Rescheduling_interrupts
7.50 ± 99% +4263.8% 327.29 ±105% interrupts.CPU5.RES:Rescheduling_interrupts
161.17 ± 20% +46.3% 235.86 ± 15% interrupts.CPU81.NMI:Non-maskable_interrupts
161.17 ± 20% +46.3% 235.86 ± 15% interrupts.CPU81.PMI:Performance_monitoring_interrupts
51316 ± 6% +20.1% 61616 ± 8% interrupts.NMI:Non-maskable_interrupts
51316 ± 6% +20.1% 61616 ± 8% interrupts.PMI:Performance_monitoring_interrupts
1298 ± 7% +1608.1% 22176 ± 9% interrupts.RES:Rescheduling_interrupts
0.01 ± 19% -72.9% 0.00 ± 26% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
0.02 ± 29% -68.9% 0.01 perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
0.01 ± 20% -31.4% 0.01 ± 6% perf-sched.total_sch_delay.average.ms
220.68 -89.1% 24.00 ± 2% perf-sched.total_wait_and_delay.average.ms
11823 +833.5% 110368 perf-sched.total_wait_and_delay.count.ms
9308 ± 6% -26.6% 6835 ± 15% perf-sched.total_wait_and_delay.max.ms
220.66 -89.1% 23.99 ± 2% perf-sched.total_wait_time.average.ms
9307 ± 6% -26.6% 6835 ± 15% perf-sched.total_wait_time.max.ms
0.07 ± 2% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.btrfs_start_ordered_extent.btrfs_wait_ordered_range.btrfs_sync_file.do_fsync
2.74 -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
6.82 ± 28% -36.0% 4.36 perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk.cpu_stop_queue_work.affine_move_task.__set_cpus_allowed_ptr
4.20 ± 9% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.rcu_gp_kthread.kthread.ret_from_fork
108.93 ±141% +176.8% 301.52 ± 36% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
622.38 ± 23% -49.8% 312.43 ± 2% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
0.01 ± 26% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.wait_for_partner.fifo_open.do_dentry_open.do_open.isra
300.44 ± 5% -98.4% 4.68 ± 9% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
541.33 -100.0% 0.00 perf-sched.wait_and_delay.count.btrfs_start_ordered_extent.btrfs_wait_ordered_range.btrfs_sync_file.do_fsync
306.00 -100.0% 0.00 perf-sched.wait_and_delay.count.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
397.67 ± 10% -100.0% 0.00 perf-sched.wait_and_delay.count.rcu_gp_kthread.kthread.ret_from_fork
21.67 ± 4% +167.7% 58.00 ± 3% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
72.00 -100.0% 0.00 perf-sched.wait_and_delay.count.wait_for_partner.fifo_open.do_dentry_open.do_open.isra
1328 +7408.2% 99708 perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
0.24 ± 9% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.btrfs_start_ordered_extent.btrfs_wait_ordered_range.btrfs_sync_file.do_fsync
41.50 -100.0% 0.00 perf-sched.wait_and_delay.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
5.03 -100.0% 0.00 perf-sched.wait_and_delay.max.ms.rcu_gp_kthread.kthread.ret_from_fork
0.09 ± 65% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.wait_for_partner.fifo_open.do_dentry_open.do_open.isra
8288 ± 10% -18.9% 6725 ± 15% perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork
0.06 ± 2% +18.6% 0.07 ± 2% perf-sched.wait_time.avg.ms.btrfs_start_ordered_extent.btrfs_wait_ordered_range.btrfs_sync_file.do_fsync
6.82 ± 28% -36.1% 4.36 perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.cpu_stop_queue_work.affine_move_task.__set_cpus_allowed_ptr
0.06 ± 41% -77.1% 0.01 ±158% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.cpu_stop_queue_work.stop_one_cpu.migrate_task_to
131.30 ±105% +129.6% 301.49 ± 36% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
622.37 ± 23% -49.8% 312.42 ± 2% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
300.41 ± 5% -98.4% 4.67 ± 9% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
0.23 ± 9% +119.2% 0.50 ± 36% perf-sched.wait_time.max.ms.btrfs_start_ordered_extent.btrfs_wait_ordered_range.btrfs_sync_file.do_fsync
0.03 ±100% +297.4% 0.11 ± 24% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.__set_page_dirty_nobuffers.btrfs_dirty_pages
0.06 ± 41% -77.1% 0.01 ±158% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.cpu_stop_queue_work.stop_one_cpu.migrate_task_to
14.70 ± 5% +16.6% 17.14 ± 4% perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
8288 ± 10% -18.9% 6725 ± 15% perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork
78.84 ± 2% -5.7 73.09 ± 3% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
78.84 ± 2% -5.7 73.09 ± 3% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
78.80 ± 2% -5.7 73.05 ± 3% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
79.28 ± 2% -5.5 73.80 ± 2% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
6.75 ± 6% -0.7 6.05 ± 4% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
0.86 ± 19% -0.3 0.52 ± 42% perf-profile.calltrace.cycles-pp.__lock_acquire.lock_acquire.ktime_get_update_offsets_now.hrtimer_interrupt.__sysvec_apic_timer_interrupt
0.66 ± 8% -0.2 0.41 ± 63% perf-profile.calltrace.cycles-pp.lock_release._raw_spin_unlock_irqrestore.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
0.74 ± 11% -0.2 0.52 ± 41% perf-profile.calltrace.cycles-pp.btrfs_set_extent_delalloc.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write
0.73 ± 10% -0.2 0.52 ± 41% perf-profile.calltrace.cycles-pp.set_extent_bit.btrfs_set_extent_delalloc.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter
1.92 ± 5% -0.2 1.73 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.23 ± 9% -0.2 1.05 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
1.00 ± 8% -0.2 0.83 ± 8% perf-profile.calltrace.cycles-pp.rcu_read_lock_sched_held.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
1.15 ± 8% -0.2 0.98 ± 9% perf-profile.calltrace.cycles-pp.lock_acquire._raw_spin_lock_irq.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
0.66 ± 10% -0.1 0.58 ± 7% perf-profile.calltrace.cycles-pp.lock_release.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt
0.00 +1.0 0.97 ± 11% perf-profile.calltrace.cycles-pp.rcu_read_lock_sched_held.lock_acquire._raw_spin_lock.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe
0.00 +1.0 1.01 ± 14% perf-profile.calltrace.cycles-pp.rcu_read_lock_sched_held.lock_release._raw_spin_unlock.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe
0.00 +1.6 1.60 ± 13% perf-profile.calltrace.cycles-pp.lock_release._raw_spin_unlock.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats
0.00 +1.8 1.85 ± 13% perf-profile.calltrace.cycles-pp._raw_spin_unlock.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work
0.00 +2.3 2.26 ± 10% perf-profile.calltrace.cycles-pp.__lock_acquire.lock_acquire._raw_spin_lock.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe
0.00 +3.7 3.74 ± 11% perf-profile.calltrace.cycles-pp.lock_acquire._raw_spin_lock.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats
0.00 +3.9 3.86 ± 11% perf-profile.calltrace.cycles-pp._raw_spin_lock.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work
0.00 +6.8 6.79 ± 10% perf-profile.calltrace.cycles-pp.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work.worker_thread
0.00 +6.8 6.83 ± 10% perf-profile.calltrace.cycles-pp.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work.worker_thread.kthread
0.00 +7.0 6.96 ± 10% perf-profile.calltrace.cycles-pp.mem_cgroup_flush_stats.process_one_work.worker_thread.kthread.ret_from_fork
0.00 +7.3 7.30 ± 10% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
0.00 +7.8 7.78 ± 10% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
0.00 +8.0 8.00 ± 10% perf-profile.calltrace.cycles-pp.ret_from_fork
0.00 +8.0 8.00 ± 10% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
78.84 ± 2% -5.7 73.09 ± 3% perf-profile.children.cycles-pp.start_secondary
79.28 ± 2% -5.5 73.80 ± 2% perf-profile.children.cycles-pp.do_idle
79.28 ± 2% -5.5 73.80 ± 2% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
79.28 ± 2% -5.5 73.80 ± 2% perf-profile.children.cycles-pp.cpu_startup_entry
1.16 ± 5% -0.2 0.98 ± 11% perf-profile.children.cycles-pp.lockdep_hardirqs_on_prepare
1.19 ± 8% -0.2 1.03 ± 6% perf-profile.children.cycles-pp.trace_hardirqs_on
0.74 ± 11% -0.2 0.58 ± 12% perf-profile.children.cycles-pp.btrfs_set_extent_delalloc
0.78 ± 11% -0.2 0.62 ± 16% perf-profile.children.cycles-pp.alloc_extent_state
1.41 ± 7% -0.2 1.26 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.76 ± 10% -0.2 0.61 ± 12% perf-profile.children.cycles-pp.set_extent_bit
0.66 ± 9% -0.1 0.52 ± 17% perf-profile.children.cycles-pp.lockdep_init_map_type
0.65 ± 9% -0.1 0.52 ± 17% perf-profile.children.cycles-pp.static_obj
0.60 ± 10% -0.1 0.47 ± 17% perf-profile.children.cycles-pp.__is_kernel_percpu_address
0.61 ± 11% -0.1 0.49 ± 18% perf-profile.children.cycles-pp.__init_waitqueue_head
0.34 ± 11% -0.1 0.27 ± 11% perf-profile.children.cycles-pp.hrtimer_update_next_event
0.24 ± 10% -0.1 0.17 ± 15% perf-profile.children.cycles-pp.__remove_hrtimer
0.19 ± 10% -0.1 0.13 ± 15% perf-profile.children.cycles-pp.timerqueue_del
0.46 ± 7% -0.1 0.39 ± 9% perf-profile.children.cycles-pp.trace_hardirqs_on_prepare
0.69 ± 3% -0.1 0.63 ± 4% perf-profile.children.cycles-pp.debug_smp_processor_id
0.13 ± 16% -0.0 0.09 ± 13% perf-profile.children.cycles-pp.menu_reflect
0.39 ± 5% -0.0 0.35 ± 6% perf-profile.children.cycles-pp.lockdep_hardirqs_on
0.16 ± 9% -0.0 0.12 ± 17% perf-profile.children.cycles-pp.btrfs_set_delalloc_extent
0.15 ± 7% -0.0 0.12 ± 11% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.10 ± 11% -0.0 0.08 ± 11% perf-profile.children.cycles-pp.rcu_nmi_enter
0.08 ± 14% +0.0 0.10 ± 11% perf-profile.children.cycles-pp.remove_vma
0.06 ± 13% +0.0 0.09 ± 16% perf-profile.children.cycles-pp.newidle_balance
0.09 ± 9% +0.1 0.14 ± 7% perf-profile.children.cycles-pp.___perf_sw_event
0.20 ± 9% +0.1 0.26 ± 8% perf-profile.children.cycles-pp.do_raw_spin_unlock
0.27 ± 12% +0.1 0.34 ± 6% perf-profile.children.cycles-pp.memcg_slab_free_hook
0.02 ±142% +0.1 0.10 ± 12% perf-profile.children.cycles-pp.select_task_rq_fair
0.00 +0.1 0.08 ± 18% perf-profile.children.cycles-pp.enqueue_task_fair
0.01 ±223% +0.1 0.10 ± 18% perf-profile.children.cycles-pp.pick_next_task_fair
0.00 +0.1 0.10 ± 22% perf-profile.children.cycles-pp.ttwu_do_activate
0.07 ± 50% +0.1 0.17 ± 14% perf-profile.children.cycles-pp.update_curr
0.01 ±223% +0.1 0.13 ± 18% perf-profile.children.cycles-pp.finish_task_switch
0.01 ±223% +0.1 0.13 ± 16% perf-profile.children.cycles-pp.dequeue_entity
0.01 ±223% +0.1 0.14 ± 16% perf-profile.children.cycles-pp.dequeue_task_fair
0.00 +0.2 0.22 ± 15% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
1.10 ± 2% +0.2 1.33 ± 4% perf-profile.children.cycles-pp.match_held_lock
0.06 ± 59% +0.2 0.29 ± 13% perf-profile.children.cycles-pp.schedule_idle
0.10 ± 15% +0.3 0.36 ± 13% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
0.00 +0.3 0.28 ± 17% perf-profile.children.cycles-pp.cgroup_rstat_updated
0.00 +0.4 0.35 ± 9% perf-profile.children.cycles-pp.mem_cgroup_css_rstat_flush
0.11 ± 19% +0.4 0.48 ± 15% perf-profile.children.cycles-pp.schedule
0.56 ± 9% +0.4 0.95 ± 7% perf-profile.children.cycles-pp.do_raw_spin_lock
0.07 ± 17% +0.4 0.49 ± 12% perf-profile.children.cycles-pp.try_to_wake_up
0.73 ± 11% +0.5 1.24 ± 14% perf-profile.children.cycles-pp.__mod_lruvec_page_state
0.00 +0.6 0.58 ± 13% perf-profile.children.cycles-pp.__queue_work
0.20 ± 14% +0.6 0.78 ± 13% perf-profile.children.cycles-pp.__schedule
0.00 +0.6 0.60 ± 14% perf-profile.children.cycles-pp.queue_work_on
2.35 ± 4% +1.6 3.96 ± 7% perf-profile.children.cycles-pp._raw_spin_unlock
3.97 ± 3% +3.5 7.48 ± 6% perf-profile.children.cycles-pp._raw_spin_lock
0.00 +6.8 6.81 ± 10% perf-profile.children.cycles-pp.cgroup_rstat_flush_locked
0.00 +6.8 6.83 ± 10% perf-profile.children.cycles-pp.cgroup_rstat_flush_irqsafe
0.00 +7.0 6.96 ± 10% perf-profile.children.cycles-pp.mem_cgroup_flush_stats
0.18 ± 10% +7.1 7.31 ± 10% perf-profile.children.cycles-pp.process_one_work
0.45 ± 6% +7.6 8.02 ± 10% perf-profile.children.cycles-pp.ret_from_fork
0.43 ± 8% +7.6 8.00 ± 10% perf-profile.children.cycles-pp.kthread
0.20 ± 10% +7.6 7.78 ± 10% perf-profile.children.cycles-pp.worker_thread
1.67 ± 9% -0.2 1.46 ± 2% perf-profile.self.cycles-pp.cpuidle_enter_state
0.47 ± 6% -0.1 0.40 ± 7% perf-profile.self.cycles-pp.lockdep_hardirqs_on_prepare
0.28 ± 10% -0.0 0.23 ± 7% perf-profile.self.cycles-pp.__hrtimer_next_event_base
0.25 ± 14% -0.0 0.21 ± 17% perf-profile.self.cycles-pp.__is_kernel_percpu_address
0.19 ± 5% -0.0 0.14 ± 21% perf-profile.self.cycles-pp.brd_do_bvec
0.07 ± 16% -0.0 0.03 ± 86% perf-profile.self.cycles-pp.timerqueue_del
0.16 ± 14% -0.0 0.12 ± 20% perf-profile.self.cycles-pp.sysvec_apic_timer_interrupt
0.08 ± 6% -0.0 0.05 ± 44% perf-profile.self.cycles-pp.rb_erase
0.18 ± 8% +0.0 0.23 ± 8% perf-profile.self.cycles-pp.do_raw_spin_unlock
0.06 ± 9% +0.1 0.12 ± 11% perf-profile.self.cycles-pp._raw_spin_lock
0.10 ± 9% +0.1 0.18 ± 14% perf-profile.self.cycles-pp._raw_spin_unlock
0.88 ± 3% +0.2 1.04 ± 3% perf-profile.self.cycles-pp.match_held_lock
0.54 ± 7% +0.2 0.72 ± 6% perf-profile.self.cycles-pp.do_raw_spin_lock
0.00 +0.2 0.22 ± 15% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.00 +0.2 0.22 ± 6% perf-profile.self.cycles-pp.cgroup_rstat_flush_locked
0.00 +0.3 0.33 ± 10% perf-profile.self.cycles-pp.mem_cgroup_css_rstat_flush
***************************************************************************************************
lkp-cfl-e1: 16 threads 1 sockets Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/300s/lkp-cfl-e1/five_sec/reaim/0xde
commit:
0c7c5da68b ("memcg: switch lruvec stats to rstat")
cda0d20525 ("memcg: infrastructure to flush memcg stats")
0c7c5da68b4a09ff cda0d205259a7e7c1f8e296130a
---------------- ---------------------------
%stddev %change %stddev
\ | \
16.84 +1.2% 17.05 reaim.child_systime
1.17 -1.7% 1.15 reaim.child_utime
77332 -10.7% 69064 reaim.jobs_per_min
4833 -10.7% 4316 reaim.jobs_per_min_child
79421 -10.3% 71230 reaim.max_jobs_per_min
1.23 +12.0% 1.38 reaim.parent_time
1.00 +61.7% 1.61 ± 2% reaim.std_dev_percent
0.01 +84.7% 0.02 ± 2% reaim.std_dev_time
186593 +620.2% 1343882 reaim.time.involuntary_context_switches
15277 -15.4% 12918 reaim.time.major_page_faults
45920952 -4.7% 43779707 reaim.time.minor_page_faults
549.17 -3.4% 530.57 reaim.time.percent_of_cpu_this_job_got
1554 -3.5% 1500 reaim.time.system_time
108.32 -6.3% 101.53 reaim.time.user_time
147200 -4.7% 140342 reaim.workload
4260000 ± 24% +85.2% 7890750 ± 16% cpuidle..usage
0.00 ± 94% +0.0 0.01 ±125% mpstat.cpu.all.iowait%
37619 +10.2% 41448 softirqs.CPU5.SCHED
4236 ± 3% -20.4% 3371 ± 4% slabinfo.cred_jar.active_objs
4236 ± 3% -20.4% 3371 ± 4% slabinfo.cred_jar.num_objs
5335 +425.1% 28014 vmstat.system.cs
25198 ± 13% +34.2% 33810 ± 10% vmstat.system.in
36327 +93.6% 70343 ± 3% meminfo.Active
36170 +93.4% 69943 ± 4% meminfo.Active(anon)
374203 +9.7% 410350 meminfo.Committed_AS
45925 +77.1% 81324 ± 4% meminfo.Shmem
60038 ± 58% +4479.1% 2749217 ± 35% turbostat.C1
0.06 ± 34% +1.9 1.96 ± 41% turbostat.C1%
374823 ± 34% +283.9% 1438762 ± 52% turbostat.C1E
0.66 ± 33% +1.4 2.10 ± 55% turbostat.C1E%
92340 ± 84% -99.3% 670.00 ±204% turbostat.C8
48.13 +7.4% 51.70 turbostat.CorWatt
7689266 ± 13% +33.9% 10296075 ± 9% turbostat.IRQ
11.56 ± 34% -29.5% 8.15 ± 9% turbostat.Pkg%pc2
48.58 +7.4% 52.15 turbostat.PkgWatt
9048 +92.7% 17436 ± 4% proc-vmstat.nr_active_anon
599695 +1.5% 608725 proc-vmstat.nr_file_pages
8070 +4.9% 8467 proc-vmstat.nr_mapped
11469 +77.0% 20298 ± 4% proc-vmstat.nr_shmem
9048 +92.7% 17436 ± 4% proc-vmstat.nr_zone_active_anon
42656780 -4.6% 40700689 proc-vmstat.numa_hit
42656780 -4.6% 40700689 proc-vmstat.numa_local
92407 +22.4% 113128 proc-vmstat.pgactivate
43569042 -4.7% 41530882 proc-vmstat.pgalloc_normal
46430133 -4.6% 44284838 proc-vmstat.pgfault
43538315 -4.7% 41477247 proc-vmstat.pgfree
1629898 -4.2% 1561322 proc-vmstat.pgreuse
5.801e+09 +6.4% 6.174e+09 perf-stat.i.branch-instructions
30752328 +16.4% 35783086 perf-stat.i.branch-misses
2.16 ± 2% -0.3 1.81 perf-stat.i.cache-miss-rate%
2.765e+08 +25.7% 3.475e+08 perf-stat.i.cache-references
5341 +426.1% 28100 perf-stat.i.context-switches
1.15 -2.8% 1.11 perf-stat.i.cpi
2.523e+10 +4.4% 2.633e+10 perf-stat.i.cpu-cycles
747.05 +84.5% 1378 perf-stat.i.cpu-migrations
3729 +7.5% 4008 perf-stat.i.cycles-between-cache-misses
7.878e+09 +5.2% 8.284e+09 perf-stat.i.dTLB-loads
0.01 ± 9% -0.0 0.01 perf-stat.i.dTLB-store-miss-rate%
560872 -4.9% 533422 perf-stat.i.dTLB-store-misses
4.383e+09 +2.4% 4.487e+09 perf-stat.i.dTLB-stores
68.79 ± 2% -5.0 63.79 ± 2% perf-stat.i.iTLB-load-miss-rate%
1070659 ± 3% +6.7% 1142823 ± 2% perf-stat.i.iTLB-load-misses
578652 +21.6% 703855 ± 3% perf-stat.i.iTLB-loads
2.76e+10 +5.4% 2.908e+10 perf-stat.i.instructions
0.94 +2.0% 0.96 perf-stat.i.ipc
50.65 -15.2% 42.96 perf-stat.i.major-faults
0.24 ± 3% +12.4% 0.27 ± 3% perf-stat.i.metric.G/sec
1.58 +4.4% 1.65 perf-stat.i.metric.GHz
199.04 +3.3% 205.70 perf-stat.i.metric.K/sec
904.24 +3.3% 933.92 perf-stat.i.metric.M/sec
150989 -4.4% 144377 perf-stat.i.minor-faults
151039 -4.4% 144420 perf-stat.i.page-faults
36548 ± 4% +356.1% 166686 interrupts.CAL:Function_call_interrupts
2202 ± 12% +376.0% 10483 ± 4% interrupts.CPU0.CAL:Function_call_interrupts
4568 +1471.3% 71781 ± 2% interrupts.CPU0.RES:Rescheduling_interrupts
25.83 ± 21% +1613.2% 442.57 ± 6% interrupts.CPU0.TLB:TLB_shootdowns
2257 ± 18% +353.1% 10231 ± 3% interrupts.CPU1.CAL:Function_call_interrupts
4591 +1491.3% 73058 ± 2% interrupts.CPU1.RES:Rescheduling_interrupts
30.33 ± 13% +1384.9% 450.43 ± 3% interrupts.CPU1.TLB:TLB_shootdowns
2310 ± 20% +330.9% 9957 ± 4% interrupts.CPU10.CAL:Function_call_interrupts
4562 ± 2% +1496.4% 72842 ± 2% interrupts.CPU10.RES:Rescheduling_interrupts
31.33 ± 19% +1291.0% 435.86 ± 4% interrupts.CPU10.TLB:TLB_shootdowns
2043 ± 19% +414.7% 10517 ± 8% interrupts.CPU11.CAL:Function_call_interrupts
4444 ± 2% +1532.5% 72561 interrupts.CPU11.RES:Rescheduling_interrupts
26.17 ± 17% +1588.6% 441.86 ± 4% interrupts.CPU11.TLB:TLB_shootdowns
2257 ± 24% +363.1% 10452 ± 6% interrupts.CPU12.CAL:Function_call_interrupts
4576 +1474.2% 72041 ± 2% interrupts.CPU12.RES:Rescheduling_interrupts
27.17 ± 15% +1606.4% 463.57 ± 5% interrupts.CPU12.TLB:TLB_shootdowns
1948 ± 18% +449.5% 10706 ± 7% interrupts.CPU13.CAL:Function_call_interrupts
4551 +1487.9% 72280 ± 3% interrupts.CPU13.RES:Rescheduling_interrupts
28.67 ± 17% +1446.8% 443.43 ± 4% interrupts.CPU13.TLB:TLB_shootdowns
2363 ± 23% +353.4% 10714 ± 7% interrupts.CPU14.CAL:Function_call_interrupts
4521 ± 2% +1497.2% 72218 ± 2% interrupts.CPU14.RES:Rescheduling_interrupts
27.00 ± 19% +1561.4% 448.57 ± 4% interrupts.CPU14.TLB:TLB_shootdowns
2111 ± 20% +381.6% 10167 ± 5% interrupts.CPU15.CAL:Function_call_interrupts
4594 +1460.4% 71685 interrupts.CPU15.RES:Rescheduling_interrupts
26.67 ± 21% +1553.2% 440.86 ± 5% interrupts.CPU15.TLB:TLB_shootdowns
2477 ± 23% +319.6% 10397 ± 4% interrupts.CPU2.CAL:Function_call_interrupts
4551 ± 2% +1496.0% 72641 ± 2% interrupts.CPU2.RES:Rescheduling_interrupts
29.50 ± 15% +1381.4% 437.00 ± 4% interrupts.CPU2.TLB:TLB_shootdowns
2220 ± 17% +348.6% 9962 ± 4% interrupts.CPU3.CAL:Function_call_interrupts
4577 +1483.3% 72469 ± 2% interrupts.CPU3.RES:Rescheduling_interrupts
27.83 ± 18% +1437.2% 427.86 ± 3% interrupts.CPU3.TLB:TLB_shootdowns
1926 ± 13% +457.0% 10731 ± 7% interrupts.CPU4.CAL:Function_call_interrupts
4532 +1490.4% 72080 ± 3% interrupts.CPU4.RES:Rescheduling_interrupts
29.00 ± 18% +1472.4% 456.00 ± 5% interrupts.CPU4.TLB:TLB_shootdowns
2441 ± 22% +313.7% 10101 ± 4% interrupts.CPU5.CAL:Function_call_interrupts
4613 +1479.5% 72866 interrupts.CPU5.RES:Rescheduling_interrupts
25.33 ± 25% +1684.8% 452.14 ± 5% interrupts.CPU5.TLB:TLB_shootdowns
2400 ± 16% +337.5% 10502 ± 6% interrupts.CPU6.CAL:Function_call_interrupts
4527 +1480.1% 71535 interrupts.CPU6.RES:Rescheduling_interrupts
26.50 ± 22% +1557.1% 439.14 ± 5% interrupts.CPU6.TLB:TLB_shootdowns
2799 ± 22% +294.2% 11034 ± 6% interrupts.CPU7.CAL:Function_call_interrupts
4623 ± 2% +1435.4% 70979 ± 2% interrupts.CPU7.RES:Rescheduling_interrupts
27.83 ± 21% +1465.4% 435.71 ± 8% interrupts.CPU7.TLB:TLB_shootdowns
2487 ± 12% +303.6% 10039 ± 5% interrupts.CPU8.CAL:Function_call_interrupts
4534 +1504.6% 72761 ± 3% interrupts.CPU8.RES:Rescheduling_interrupts
28.17 ± 21% +1515.4% 455.00 ± 5% interrupts.CPU8.TLB:TLB_shootdowns
2300 ± 13% +364.6% 10685 ± 7% interrupts.CPU9.CAL:Function_call_interrupts
4563 +1458.2% 71100 interrupts.CPU9.RES:Rescheduling_interrupts
25.00 ± 24% +1686.9% 446.71 ± 7% interrupts.CPU9.TLB:TLB_shootdowns
72933 +1483.5% 1154904 interrupts.RES:Rescheduling_interrupts
442.33 ± 4% +1508.9% 7116 interrupts.TLB:TLB_shootdowns
0.04 ± 4% +138.7% 0.11 ± 6% perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.05 ± 5% +158.0% 0.13 ± 4% perf-sched.sch_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.11 ± 39% +127.2% 0.26 ± 32% perf-sched.sch_delay.avg.ms.io_schedule.__lock_page_killable.filemap_fault.__do_fault
0.02 ± 27% -46.9% 0.01 ± 26% perf-sched.sch_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
0.03 ± 7% -15.7% 0.02 ± 15% perf-sched.sch_delay.avg.ms.pipe_write.new_sync_write.vfs_write.ksys_write
0.02 ± 31% +659.1% 0.13 ± 55% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_read_unlock.wait_task_zombie.do_wait
0.12 ± 78% -77.2% 0.03 ±121% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__dentry_kill.dput
0.20 ±142% -99.6% 0.00 ±115% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.begin_new_exec.load_elf_binary
0.02 ± 57% -78.0% 0.00 ±112% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.change_p4d_range.change_protection
0.02 ± 72% -79.4% 0.01 ± 91% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.do_sys_openat2.do_sys_open
0.04 ±150% -84.4% 0.01 ± 73% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.finish_fault.do_fault
0.07 ± 8% -62.4% 0.03 ± 33% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.zap_pte_range.unmap_page_range
0.00 ±223% +3094.3% 0.08 ±179% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.release_task.wait_task_zombie
0.12 ±108% -84.9% 0.02 ± 94% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.task_work_run.exit_to_user_mode_prepare
0.02 ± 46% +282.4% 0.07 ± 68% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.wait_task_zombie.do_wait
0.10 ±115% -88.0% 0.01 ±111% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.release_pages.tlb_flush_mmu
0.20 ± 70% +182.4% 0.55 ± 14% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.sched_move_task.do_exit
0.13 ±113% -93.8% 0.01 ± 98% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk.fd_install.do_sys_openat2.do_sys_open
0.21 ±215% -99.7% 0.00 ± 86% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk.flush_tlb_mm_range.tlb_flush_mmu.tlb_finish_mmu
0.12 ± 68% -76.3% 0.03 ± 40% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk.lru_add_drain.exit_mmap.mmput
0.03 ±143% -94.6% 0.00 ±105% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk.lru_add_drain.free_pages_and_swap_cache.tlb_flush_mmu
0.03 ±105% -94.8% 0.00 ± 66% perf-sched.sch_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk.lru_cache_add.shmem_getpage_gfp.shmem_write_begin
0.02 ± 30% -79.6% 0.00 ± 90% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.free_unref_page_list.release_pages
0.05 ±118% +288.1% 0.18 ± 55% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.handle_mm_fault.do_user_addr_fault
0.02 ± 47% -70.1% 0.01 ±130% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.kmem_cache_free.__put_anon_vma
0.03 ± 94% -80.9% 0.01 ± 92% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.lock_acquire.__lock_task_sighand
0.03 ± 40% -83.3% 0.01 ±155% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.lock_acquire.aa_file_perm
0.06 ± 60% +150.6% 0.15 ± 43% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.lock_is_held_type.___might_sleep
0.02 ±107% -93.1% 0.00 ±197% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.vm_area_dup.dup_mmap
0.04 ±118% -93.0% 0.00 ±116% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.vma_interval_tree_insert.__vma_adjust
0.04 ± 66% -89.1% 0.00 ±183% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.___might_sleep.unmap_page_range
0.01 ±106% -85.2% 0.00 ±101% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__might_sleep.dput
0.00 ±223% +610.2% 0.01 ± 47% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.acct_collect.do_exit
0.03 ±100% -85.6% 0.00 ±100% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.d_alloc_parallel.__lookup_slow
0.05 ± 89% -96.1% 0.00 ± 58% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.d_path.perf_event_mmap
0.11 ± 74% -86.4% 0.01 ± 80% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.__might_fault
0.05 ± 55% -78.8% 0.01 ± 23% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.fs_reclaim_acquire
0.11 ±118% -86.2% 0.02 ± 93% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.kmem_cache_alloc
0.09 ± 43% -86.4% 0.01 ±121% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.memcg_slab_free_hook
0.09 ± 56% -86.6% 0.01 ± 68% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.___might_sleep
0.17 ± 97% -96.3% 0.01 ± 86% perf-sched.sch_delay.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.kmem_cache_alloc
0.12 ±196% -97.3% 0.00 ±138% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.__mark_inode_dirty.generic_update_time.touch_atime
0.02 ± 26% -95.8% 0.00 ± 97% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.kmem_cache_free.unlink_anon_vmas.free_pgtables
0.02 ±138% -94.4% 0.00 ±118% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.kmem_cache_alloc.__anon_vma_prepare
0.04 ± 74% -95.4% 0.00 ± 86% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.kmem_cache_alloc.vm_area_dup
0.06 ± 40% -70.2% 0.02 ± 97% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.memcg_slab_free_hook.kmem_cache_free
0.01 ± 68% -86.9% 0.00 ±112% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.perf_iterate_sb.perf_event_mmap
0.05 ± 72% -71.1% 0.01 ±142% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.uncharge_page.mem_cgroup_uncharge_list
0.07 ±103% -88.2% 0.01 ± 99% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.get_obj_cgroup_from_current.kmem_cache_alloc
0.09 ± 79% -97.7% 0.00 ± 84% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.__anon_vma_prepare
0.02 ± 53% -85.6% 0.00 ±102% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.__pmd_alloc
0.03 ± 72% -79.2% 0.01 ±125% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc_trace.perf_event_mmap
0.03 ±142% -98.4% 0.00 ±169% perf-sched.sch_delay.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.mem_cgroup_charge.do_anonymous_page
0.01 ± 25% +243.9% 0.05 ± 67% perf-sched.sch_delay.avg.ms.rwsem_down_read_slowpath.down_read.walk_component.path_lookupat.isra
0.01 ± 13% +475.0% 0.09 ±132% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.anon_vma_clone.anon_vma_fork.dup_mmap
0.00 ±223% +4900.0% 0.05 ±140% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.anon_vma_fork.dup_mmap.dup_mm
0.01 ± 21% +307.1% 0.05 ± 58% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.01 ± 34% +397.9% 0.06 ± 61% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.filename_create.do_linkat.__x64_sys_link
12.48 ± 31% +76.0% 21.98 ± 9% perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.02 ± 27% +32984.9% 6.18 ± 74% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_read_unlock.wait_task_zombie.do_wait
0.48 ±179% -96.2% 0.02 ±116% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.begin_new_exec.load_elf_binary
0.99 ±103% +507.9% 6.01 ± 53% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.do_anonymous_page.__handle_mm_fault
0.06 ± 16% +4772.3% 2.90 ± 87% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.do_wp_page.__handle_mm_fault
5.12 ± 33% +186.5% 14.66 ± 22% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.filemap_map_pages.do_fault
1.66 ± 73% +491.1% 9.84 ± 29% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.wp_page_copy.__handle_mm_fault
5.83 ± 36% +131.2% 13.47 ± 30% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.zap_pte_range.unmap_page_range
0.06 ±127% +852.8% 0.53 ±146% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.get_signal.arch_do_signal_or_restart
0.00 ±223% +42871.4% 1.07 ±209% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.release_task.wait_task_zombie
0.02 ±128% +7349.0% 1.30 ±172% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.ret_from_fork.[unknown]
0.00 ±223% +3033.9% 0.08 ±113% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.sigprocmask.__x64_sys_rt_sigprocmask
0.02 ± 53% +9229.0% 2.02 ± 81% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.wait_task_zombie.do_wait
0.02 ± 8% +20160.8% 3.68 ±124% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.delete_from_page_cache.truncate_inode_page
0.01 ±101% +1939.1% 0.19 ±113% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.do_wait.kernel_wait4
6.45 ± 74% +198.2% 19.24 ± 21% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.sched_move_task.do_exit
0.01 ± 50% +9448.0% 1.42 ±143% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.thread_group_cputime_adjusted.wait_task_zombie
0.19 ±198% +2910.9% 5.70 ± 55% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_write_unlock_irq.release_task.wait_task_zombie
7.16 ± 66% +231.6% 23.74 ± 13% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk.cpu_stop_queue_work.stop_one_cpu.sched_exec
0.00 ±223% +1.1e+05% 3.91 ±157% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk.exit_signals.do_exit.do_group_exit
0.39 ±116% +1113.2% 4.70 ±139% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk.lru_add_drain.unmap_region.__do_munmap
8.76 ± 23% +77.5% 15.54 ± 22% perf-sched.sch_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk.schedule_tail.ret_from_fork.[unknown]
0.33 ±173% +1025.4% 3.72 ± 81% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.handle_mm_fault.do_user_addr_fault
1.95 ± 72% +325.3% 8.30 ± 52% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.lock_is_held_type.___might_sleep
0.47 ± 86% +1401.6% 6.99 ± 94% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.memcg_slab_free_hook.kmem_cache_free
0.07 ±134% -95.8% 0.00 ±115% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.vma_interval_tree_insert.__vma_adjust
0.03 ± 92% +10097.9% 2.72 ± 83% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__might_sleep.do_user_addr_fault
0.00 ±223% +1504.1% 0.02 ± 36% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.acct_collect.do_exit
2.34 ± 61% +468.7% 13.30 ± 33% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.do_user_addr_fault.exc_page_fault
0.00 ±223% +702.0% 0.02 ± 44% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.down_read_trylock.do_user_addr_fault
0.43 ±203% +1077.3% 5.11 ± 40% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.handle_mm_fault.do_user_addr_fault
0.02 ±109% +5174.6% 0.84 ±205% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.down_write_killable
0.06 ± 30% +4203.6% 2.75 ± 93% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.memcg_slab_free_hook.kmem_cache_free
0.05 ±181% +723.6% 0.42 ± 91% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.syscall_exit_to_user_mode_prepare.syscall_exit_to_user_mode
0.00 ±223% +8372.5% 0.18 ±101% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.up_read.do_user_addr_fault
0.00 ±223% +1.4e+05% 1.81 ±234% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.xas_load.xas_find
0.00 ±141% +3282.9% 0.08 ±194% perf-sched.sch_delay.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.xas_start.xas_load
0.13 ±175% -87.4% 0.02 ±123% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.__mark_inode_dirty.generic_update_time.touch_atime
1.66 ±113% +364.7% 7.71 ± 42% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.__perf_sw_event.do_user_addr_fault.exc_page_fault
0.05 ± 39% -77.3% 0.01 ± 68% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.kmem_cache_free.unlink_anon_vmas.free_pgtables
0.12 ±223% +1385.4% 1.78 ±105% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.thread_group_cputime.thread_group_cputime_adjusted
0.33 ± 90% -92.5% 0.02 ± 80% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.__anon_vma_prepare
0.12 ±174% -95.2% 0.01 ±160% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.mem_cgroup_charge.do_anonymous_page
0.02 ±141% +22936.6% 3.69 ±153% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__anon_vma_prepare
0.02 ±108% +13620.0% 2.74 ±129% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__d_alloc
0.00 ±146% +29728.6% 1.29 ±136% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_clone
0.05 ±223% +1812.4% 0.96 ± 99% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_fork
0.02 ±111% +9575.2% 1.84 ±111% perf-sched.sch_delay.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.vm_area_alloc
0.04 ± 52% +6245.5% 2.70 ±157% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.anon_vma_clone.anon_vma_fork.dup_mmap
0.00 ±223% +13314.3% 0.27 ±143% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.anon_vma_fork.dup_mmap.dup_mm
0.93 ± 33% +377.4% 4.42 ± 58% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.52 ± 6% +48.2% 5.22 ± 57% perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
0.05 ± 2% -20.3% 0.04 perf-sched.total_sch_delay.average.ms
24.34 ± 2% -77.8% 5.41 ± 10% perf-sched.total_wait_and_delay.average.ms
37615 +339.2% 165218 ± 5% perf-sched.total_wait_and_delay.count.ms
24.29 ± 2% -77.9% 5.37 ± 10% perf-sched.total_wait_time.average.ms
7.69 ± 6% -16.0% 6.46 ± 12% perf-sched.wait_and_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
6.52 ± 58% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
4.93 ± 44% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.scan_positives.dcache_readdir
294.26 ± 28% -50.5% 145.74 ± 16% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
87.51 ± 5% -16.8% 72.80 ± 11% perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
57.89 ± 3% -97.1% 1.70 ± 10% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
268.33 ± 5% -100.0% 0.00 perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
315.17 ± 4% -73.3% 84.00 ±158% perf-sched.wait_and_delay.count.io_schedule.__lock_page_killable.filemap_fault.__do_fault
491.17 ± 4% +502.5% 2959 ± 4% perf-sched.wait_and_delay.count.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.filemap_map_pages.do_fault
235.00 ± 5% -100.0% 0.00 perf-sched.wait_and_delay.count.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.scan_positives.dcache_readdir
547.67 ± 4% +584.9% 3751 ± 8% perf-sched.wait_and_delay.count.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.zap_pte_range.unmap_page_range
38.83 ± 9% +113.0% 82.71 ± 6% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
3532 +15.4% 4076 ± 5% perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
2873 ± 3% +3435.5% 101580 ± 5% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
509.76 ± 96% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
511.70 ± 95% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.scan_positives.dcache_readdir
177.28 ±207% +308.9% 724.85 ± 60% perf-sched.wait_and_delay.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.zap_pte_range.unmap_page_range
7.64 ± 6% -17.2% 6.33 ± 13% perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.04 ±130% +10932.2% 4.74 ±212% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
6.47 ± 58% -61.2% 2.51 ± 8% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
0.00 ±141% +2257.1% 0.06 ± 47% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_read_unlock.exec_binprm.bprm_execve
0.00 ±223% +1689.3% 0.02 ±139% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__pte_alloc.finish_fault
18.18 ±188% -97.1% 0.53 ± 41% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.copy_pte_range.copy_page_range
1.42 ±107% -86.1% 0.20 ±128% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.filemap_map_pages.do_fault
0.00 ±223% +31642.9% 0.16 ±192% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.pgd_alloc.mm_init
4.91 ± 44% -56.2% 2.15 ± 38% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.scan_positives.dcache_readdir
4.45 ± 16% -48.3% 2.30 ± 34% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.__x64_sys_rt_sigreturn.do_syscall_64
3.01 ± 49% +284.5% 11.58 ± 82% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.wait_task_zombie.do_wait
0.05 ± 45% +9984.3% 4.76 ±131% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.__wake_up_common_lock.pipe_write
6.77 ± 4% -85.3% 1.00 ± 27% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.delete_from_page_cache.truncate_inode_page
3.45 ± 10% -42.3% 1.99 ± 32% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.do_send_sig_info.kill_pid_info
0.00 ±223% +2182.1% 0.03 ±100% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.get_random_u32.arch_align_stack
2.99 ± 33% -53.1% 1.40 ± 31% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.sched_fork.copy_process
0.76 ± 11% +106.1% 1.57 ± 6% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.sched_move_task.do_exit
1.79 ± 23% -52.2% 0.86 ± 43% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.wake_up_new_task.kernel_clone
1.62 ± 35% -80.6% 0.32 ± 85% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.lru_add_drain.free_pages_and_swap_cache.tlb_flush_mmu
0.91 ± 33% -78.7% 0.19 ± 55% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.lru_add_drain.unmap_region.__do_munmap
0.00 ±223% +21276.2% 0.64 ±200% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.mnt_drop_write.do_unlinkat.do_syscall_64
2.76 ± 42% -90.9% 0.25 ± 64% perf-sched.wait_time.avg.ms.preempt_schedule_common.preempt_schedule_thunk.perf_iterate_sb.perf_event_mmap.do_brk_flags
2.72 ± 50% -74.3% 0.70 ±107% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.free_unref_page_list.release_pages
1.47 ± 57% -92.6% 0.11 ±183% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.kmem_cache_free.remove_vma
0.00 ±223% +1271.4% 0.01 ± 53% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.___might_sleep.copy_strings
0.04 ±156% +1105.5% 0.47 ± 40% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__handle_mm_fault.handle_mm_fault
0.03 ±161% +5970.7% 1.90 ± 76% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__local_bh_enable_ip.arch_do_signal_or_restart
0.00 ±223% +2400.0% 0.02 ±131% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__might_sleep.down_write_killable
0.00 ±223% +47861.0% 0.88 ±164% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.aa_get_task_label.apparmor_task_getsecid
0.00 ±223% +11202.9% 0.28 ± 84% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.common_file_perm.security_file_permission
0.00 ±223% +6298.9% 0.28 ±139% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.copy_user_enhanced_fast_string._copy_from_user
0.00 ±223% +12987.5% 0.35 ±106% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.copy_user_enhanced_fast_string.copyout
0.00 ±223% +65085.7% 0.22 ±224% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.do_dentry_open.do_open
0.01 ±223% +8642.9% 1.12 ± 67% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.42 ± 38% -47.4% 0.75 ± 26% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.exit_to_user_mode_prepare.syscall_exit_to_user_mode
0.00 ±146% +12813.0% 0.50 ± 96% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.find_vma.do_user_addr_fault
0.08 ±134% +184.1% 0.22 ± 59% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_alloc.__alloc_file
0.03 ±223% +20360.2% 6.41 ± 89% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.__fget_files
0.84 ±112% -89.9% 0.08 ± 73% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.get_mem_cgroup_from_mm
0.05 ± 75% +469.9% 0.31 ± 55% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.handle_mm_fault
0.00 ±223% +12399.2% 0.35 ±110% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.path_init
0.00 ±223% +26716.3% 0.31 ± 60% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.vfs_write
0.34 ± 25% -44.9% 0.19 ± 18% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.___might_sleep
0.00 ±223% +2968.6% 0.03 ± 68% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.__memcg_kmem_charge_page
0.00 ±223% +45528.6% 0.23 ± 70% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.xas_descend
0.04 ±223% +563.2% 0.27 ± 92% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.xas_start
0.04 ±203% +943.6% 0.46 ±110% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.aa_file_perm
0.01 ±165% +1435.2% 0.14 ±159% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.cgroup_throttle_swaprate
0.07 ±152% +838.6% 0.70 ± 65% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.handle_mm_fault
0.00 ±223% +23621.4% 0.16 ±152% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.page_counter_try_charge.obj_cgroup_charge_pages
0.01 ±154% +2000.5% 0.31 ± 71% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.refill_obj_stock.obj_cgroup_charge
0.04 ±188% +529.3% 0.27 ± 82% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.unmap_page_range.unmap_vmas
0.02 ±183% +3708.7% 0.63 ± 76% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.up_read.do_user_addr_fault
0.00 ±141% +1782.4% 0.08 ±180% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.vm_unmapped_area.arch_get_unmapped_area_topdown
0.00 ±223% +25295.9% 0.30 ± 64% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.vmacache_find.find_vma
0.00 ±223% +9764.3% 0.20 ±209% perf-sched.wait_time.avg.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.walk_component.link_path_walk
1.11 ± 25% -51.6% 0.54 ± 19% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.__perf_sw_event.handle_mm_fault.do_user_addr_fault
0.00 ±152% +7242.9% 0.15 ±224% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.kmem_cache_free.__put_anon_vma.unlink_anon_vmas
0.00 ± 76% +2532.7% 0.03 ±122% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.__might_fault.get_user_arg_ptr
2.73 ± 33% -48.9% 1.39 ± 32% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.dput.scan_positives
1.85 ± 42% -78.8% 0.39 ± 93% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.fs_reclaim_acquire.__alloc_pages
1.58 ± 12% -76.8% 0.37 ± 65% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.get_mem_cgroup_from_mm.mem_cgroup_charge
1.30 ± 39% -71.3% 0.37 ± 53% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.get_obj_cgroup_from_current.kmem_cache_alloc
0.01 ± 79% +318.4% 0.03 ± 68% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.prepend_path.isra
0.76 ±223% +600.6% 5.33 ± 11% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.thread_group_cputime.thread_group_cputime_adjusted
0.00 ±223% +1287.0% 0.03 ±121% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.__alloc_pages.__get_free_pages
105.94 ±219% -99.5% 0.53 ±108% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.__alloc_pages.alloc_pages_vma
20.60 ±200% -98.2% 0.37 ± 56% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.get_mem_cgroup_from_mm.mem_cgroup_charge
2.78 ± 41% -88.2% 0.33 ± 62% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.kmem_cache_alloc.vm_area_dup
1.85 ± 27% -88.8% 0.21 ± 88% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.mem_cgroup_charge.do_anonymous_page
1.55 ± 36% -78.4% 0.33 ±108% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.mem_cgroup_uncharge_list.release_pages
0.34 ± 32% -74.1% 0.09 ± 31% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.memcg_slab_free_hook.kmem_cache_free
2.16 ± 74% -92.9% 0.15 ±145% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.perf_event_mmap.do_brk_flags
0.58 ±223% +781.1% 5.15 ± 14% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.thread_group_cputime.thread_group_cputime_adjusted
0.01 ±141% +445.5% 0.03 ±111% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.wp_page_copy.__handle_mm_fault
2.04 ± 35% -78.3% 0.44 ±112% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.get_obj_cgroup_from_current.kmem_cache_alloc
0.04 ±223% +9636.3% 4.12 ±223% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__d_alloc
0.95 ±149% +917.3% 9.62 ± 79% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_clone
1.39 ± 37% -61.0% 0.54 ± 42% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.rcu_read_lock_held_common.rcu_read_lock_held
1.95 ± 60% -74.3% 0.50 ± 38% perf-sched.wait_time.avg.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_lockdep_current_cpu_online.rcu_read_lock_held_common.rcu_read_lock_held
0.04 ±141% +69206.9% 30.26 ±135% perf-sched.wait_time.avg.ms.rwsem_down_write_slowpath.anon_vma_fork.dup_mmap.dup_mm
294.22 ± 28% -50.5% 145.72 ± 16% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
87.48 ± 5% -16.8% 72.78 ± 11% perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
57.88 ± 3% -97.1% 1.69 ± 10% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
0.14 ±122% +1.1e+05% 152.64 ±227% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
6.21 ± 42% +143.8% 15.13 ± 35% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.02 ± 43% -68.6% 1.58 ±133% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_read_unlock.do_wait.kernel_wait4
0.01 ±142% +15034.7% 0.88 ± 28% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_read_unlock.exec_binprm.bprm_execve
6.53 ± 24% +7474.9% 494.92 ±113% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_read_unlock.wait_task_zombie.do_wait
2.34 ±141% +6328.5% 150.26 ±232% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.__pmd_alloc.copy_page_range
9.61 ± 13% +8868.5% 862.08 ± 39% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.do_anonymous_page.__handle_mm_fault
0.00 ±223% +42044.6% 1.12 ±186% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.pgd_alloc.mm_init
177.02 ±207% +308.8% 723.69 ± 60% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock.zap_pte_range.unmap_page_range
7.76 ± 15% +1879.9% 153.69 ±225% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.do_sigaction.__x64_sys_rt_sigaction
0.61 ±202% +511.4% 3.70 ± 80% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.futex_exit_release.exit_mm_release
8.75 ± 17% +92.0% 16.80 ± 39% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.get_signal.arch_do_signal_or_restart
1.84 ±223% +630.6% 13.43 ± 33% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.release_task.wait_task_zombie
3.27 ± 58% +9126.8% 302.10 ±146% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irq.wait_task_zombie.do_wait
3.17 ± 66% +13581.8% 433.83 ±112% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.__wake_up_common_lock.pipe_write
7.42 ± 8% +157.9% 19.12 ± 37% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.delete_from_page_cache.truncate_inode_page
3.27 ±113% +228.9% 10.75 ± 21% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.do_wait.kernel_wait4
3.72 ± 83% +283.0% 14.24 ± 8% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.rcu_preempt_deferred_qs_irqrestore.__rcu_read_unlock
6.55 ± 20% +10910.3% 721.27 ± 62% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.release_pages.tlb_flush_mmu
0.02 ±212% +2925.0% 0.60 ±113% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.rwsem_wake.isra
7.20 ± 8% +60.4% 11.54 ± 26% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.sched_fork.copy_process
10.13 ± 32% +118.2% 22.10 ± 14% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.sched_move_task.do_exit
4.88 ± 53% +180.5% 13.68 ± 29% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_spin_unlock_irqrestore.thread_group_cputime_adjusted.wait_task_zombie
4.92 ± 56% +8906.5% 442.97 ±109% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk._raw_write_unlock_irq.release_task.wait_task_zombie
3.40 ± 83% +294.7% 13.41 ± 24% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.cpu_stop_queue_work.stop_one_cpu.sched_exec
0.52 ±223% +1456.8% 8.11 ± 71% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.exit_signals.do_exit.do_group_exit
4.60 ± 42% +6321.5% 295.15 ±151% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.lru_add_drain.exit_mmap.mmput
0.00 ±223% +36990.3% 1.55 ±170% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.mnt_drop_write.do_unlinkat.do_syscall_64
0.00 ±223% +1495.2% 0.05 ±140% perf-sched.wait_time.max.ms.preempt_schedule_common.preempt_schedule_thunk.rwsem_down_write_slowpath.unlink_anon_vmas.free_pgtables
1.14 ±217% +347.5% 5.10 ± 75% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.kmem_cache_alloc.__d_alloc
7.78 ± 19% +59.5% 12.41 ± 25% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.kmem_cache_alloc.vm_area_dup
3.67 ± 50% -83.4% 0.61 ±197% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.kmem_cache_free.remove_vma
8.33 ± 10% +96.5% 16.37 ± 30% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_apic_timer_interrupt.lock_acquire.get_obj_cgroup_from_current
0.01 ±141% +11672.2% 0.69 ±172% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.___pmd_free_tlb.free_p4d_range
0.12 ±156% +8195.3% 9.69 ± 38% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__handle_mm_fault.handle_mm_fault
0.11 ±223% +2602.9% 2.85 ±111% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__local_bh_enable_ip.__fpu__restore_sig
0.03 ±161% +16380.9% 5.16 ± 66% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__local_bh_enable_ip.arch_do_signal_or_restart
2.82 ± 97% +5262.6% 151.09 ±230% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__mem_cgroup_charge.mem_cgroup_charge
0.60 ±206% +25004.9% 149.42 ±232% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__might_sleep.__might_fault
0.19 ±146% +1714.2% 3.48 ±102% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__might_sleep.down_write
0.01 ±223% +21307.5% 2.46 ±139% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.__rcu_read_unlock.dput
0.00 ±223% +5.8e+06% 144.37 ±241% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.aa_file_perm.common_file_perm
0.00 ±223% +71167.5% 2.61 ±151% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.aa_get_task_label.apparmor_task_getsecid
0.00 ±223% +1.6e+05% 3.98 ± 82% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.common_file_perm.security_file_permission
0.00 ±223% +11128.6% 0.30 ±121% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.copy_strings.isra
0.01 ±223% +29050.6% 3.74 ±150% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.copy_user_enhanced_fast_string._copy_from_user
0.00 ±223% +1.5e+05% 4.01 ± 86% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.copy_user_enhanced_fast_string.copyout
0.03 ±213% +4267.9% 1.40 ±186% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.d_lookup.lookup_dcache
0.00 ±223% +1.1e+05% 1.25 ±233% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.do_dentry_open.do_open
0.02 ±223% +22138.8% 4.74 ± 68% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.16 ± 64% +3667.2% 156.55 ±220% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.do_user_addr_fault.exc_page_fault
0.08 ±223% +6720.7% 5.66 ±112% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.down_read_trylock.do_user_addr_fault
1.52 ±145% +705.3% 12.27 ± 32% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.exit_to_user_mode_prepare
7.16 ± 11% +121.6% 15.86 ± 32% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.exit_to_user_mode_prepare.syscall_exit_to_user_mode
0.01 ±141% +84292.1% 7.60 ± 66% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.find_vma.do_user_addr_fault
0.04 ±205% +1176.4% 0.45 ±108% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.free_pages_and_swap_cache.tlb_flush_mmu
1.92 ±127% +8073.6% 157.00 ±219% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.handle_mm_fault.do_user_addr_fault
0.22 ±145% +3590.2% 8.30 ± 73% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_alloc.__alloc_file
0.10 ±223% +3801.0% 4.06 ±103% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_alloc.getname_flags
0.08 ±146% +2656.7% 2.34 ±122% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kmem_cache_alloc.vm_area_alloc
0.05 ±187% +1790.1% 0.99 ±120% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.kthread_blkcg.cgroup_throttle_swaprate
2.44 ±122% +449.2% 13.40 ± 32% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.ktime_get_coarse_real_ts64.current_time
0.00 ±223% +10871.4% 0.26 ±212% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.__d_lookup
0.08 ±223% +36479.6% 30.48 ± 82% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.__fget_files
5.50 ± 59% +97.5% 10.86 ± 31% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.__might_fault
2.10 ±113% +391.9% 10.35 ± 22% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.aa_file_perm
0.03 ±177% +3898.4% 1.29 ±211% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.cgroup_throttle_swaprate
1.30 ±176% +739.9% 10.96 ± 34% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.down_read_trylock
2.52 ±114% +344.4% 11.21 ± 46% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.down_write
3.22 ± 89% +161.8% 8.44 ± 27% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.get_obj_cgroup_from_current
0.29 ± 93% +2767.1% 8.21 ± 20% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.handle_mm_fault
0.14 ±223% +5523.7% 8.05 ± 43% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.pagecache_get_page
0.00 ±223% +1.2e+05% 3.34 ±110% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.path_init
0.00 ±223% +6.1e+05% 7.18 ± 50% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_acquire.vfs_write
6.77 ± 23% +239.8% 23.00 ± 53% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.___might_sleep
0.00 ±223% +8857.1% 0.15 ±104% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.__memcg_kmem_charge_page
0.19 ±137% +1638.9% 3.34 ± 95% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.cgroup_throttle_swaprate
0.06 ±175% +2742.1% 1.71 ±168% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.get_mem_cgroup_from_mm
2.16 ±124% +214.1% 6.78 ± 22% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.mem_cgroup_from_task
0.00 ±223% +2.9e+05% 5.25 ± 63% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.xas_descend
0.14 ±223% +3535.7% 4.96 ± 75% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_is_held_type.xas_start
1.06 ±223% +527.8% 6.68 ± 40% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release._copy_from_user
0.08 ±212% +6635.0% 5.47 ±102% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.aa_file_perm
0.01 ±177% +15285.1% 1.87 ±162% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.cgroup_throttle_swaprate
0.14 ±162% +6587.0% 9.46 ± 50% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.handle_mm_fault
1.77 ±125% +8487.3% 151.72 ±228% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.kmem_cache_alloc
0.05 ±223% +2843.3% 1.42 ±150% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lock_release.perf_event_mmap
0.00 ±223% +5081.8% 0.10 ±170% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.lockdep_init_map_type.__raw_spin_lock_init
1.17 ±187% +2440.6% 29.72 ±189% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.mem_cgroup_from_task.handle_mm_fault
1.26 ±105% +253.9% 4.46 ± 52% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.memcg_slab_free_hook.kmem_cache_free
1.74 ±132% +395.9% 8.64 ± 35% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.obj_cgroup_charge.kmem_cache_alloc
0.00 ±223% +2.4e+05% 3.15 ±165% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.page_counter_try_charge.obj_cgroup_charge_pages
0.00 ±223% +1987.8% 0.05 ±106% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.perf_lock_task_context.perf_pin_task_context
0.00 ±223% +8128.6% 0.18 ±113% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.pte_alloc_one.do_fault
0.00 ±223% +28185.7% 1.04 ±229% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.rcu_is_watching.dput
0.81 ± 86% +582.0% 5.49 ± 51% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.refill_obj_stock.memcg_slab_free_hook
0.04 ±168% +16709.0% 6.95 ± 44% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.refill_obj_stock.obj_cgroup_charge
0.22 ±146% +2102.2% 4.91 ±136% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.static_obj.lockdep_init_map_type
0.00 ±223% +50820.0% 1.27 ±185% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.strlen.perf_event_mmap
5.01 ± 52% +229.3% 16.51 ± 31% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.syscall_enter_from_user_mode.do_syscall_64
1.63 ±223% +951.0% 17.10 ± 16% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.thread_group_cputime.thread_group_cputime_adjusted
0.05 ±151% +5056.5% 2.71 ± 85% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.unmap_page_range.unmap_vmas
0.03 ±200% +19804.7% 6.00 ± 41% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.up_read.do_user_addr_fault
0.01 ±141% +2797.1% 0.24 ±192% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.vm_unmapped_area.arch_get_unmapped_area_topdown
0.00 ±223% +2e+05% 5.26 ± 46% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.vmacache_find.find_vma
0.00 ±223% +24977.0% 0.96 ±215% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.walk_component.link_path_walk
0.00 ±223% +70171.4% 1.41 ±238% perf-sched.wait_time.max.ms.preempt_schedule_irq.irqentry_exit.asm_sysvec_reschedule_ipi.xas_start.xas_load
8.36 ± 19% +2637.0% 228.71 ±158% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.__perf_sw_event.do_user_addr_fault.exc_page_fault
0.01 ±223% +19788.2% 1.16 ±214% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.kmem_cache_alloc.__alloc_file.alloc_empty_file
0.01 ±146% +8216.4% 0.57 ±231% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.kmem_cache_free.__put_anon_vma.unlink_anon_vmas
5.94 ± 42% +85.7% 11.02 ± 29% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.__might_fault._copy_from_user
0.01 ± 75% +11643.9% 1.60 ±186% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.__might_fault.get_user_arg_ptr
4.10 ± 75% +121.1% 9.07 ± 18% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.down_read_trylock.do_user_addr_fault
2.49 ±130% +402.6% 12.53 ± 55% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.handle_mm_fault.do_user_addr_fault
0.76 ±223% +1327.6% 10.87 ± 20% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_acquire.thread_group_cputime.thread_group_cputime_adjusted
0.00 ±223% +4832.5% 0.09 ±174% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.__alloc_pages.__get_free_pages
0.00 ±223% +71889.3% 1.92 ±156% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.copy_page_to_iter.shmem_file_read_iter
3.56 ± 10% +46.6% 5.22 ± 28% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.memcg_slab_free_hook.kmem_cache_free
0.58 ±223% +722.0% 4.78 ± 69% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.proc_invalidate_siblings_dcache.release_task
0.58 ±223% +1111.7% 7.08 ± 34% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.thread_group_cputime.thread_group_cputime_adjusted
0.01 ±168% +1055.9% 0.13 ±121% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.lock_release.wp_page_copy.__handle_mm_fault
1.13 ±215% +38575.8% 436.26 ±112% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__alloc_file
0.08 ±213% +3170.2% 2.58 ±203% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__anon_vma_prepare
0.08 ±223% +1.8e+05% 151.13 ±230% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.__d_alloc
2.20 ±142% +26189.6% 578.11 ± 84% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_clone
1.39 ±223% +31241.3% 436.58 ±112% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.kmem_cache_alloc.anon_vma_fork
6.93 ± 6% +82.5% 12.64 ± 58% perf-sched.wait_time.max.ms.preempt_schedule_notrace.preempt_schedule_notrace_thunk.rcu_is_watching.rcu_read_lock_held_common.rcu_read_lock_held
9.37 ± 14% +1609.7% 160.15 ±214% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.anon_vma_clone.anon_vma_fork.dup_mmap
0.05 ±141% +6.5e+05% 295.63 ±151% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.anon_vma_fork.dup_mmap.dup_mm
8.11 ± 19% +66.7% 13.52 ± 43% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.dup_mmap.dup_mm.copy_process
0.30 ± 61% +298.5% 1.20 ± 26% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.vma_link.mmap_region.do_mmap
12.00 -2.2 9.77 ± 4% perf-profile.calltrace.cycles-pp.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe
11.84 -2.2 9.62 ± 4% perf-profile.calltrace.cycles-pp.dcache_readdir.iterate_dir.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe
11.97 -2.2 9.75 ± 4% perf-profile.calltrace.cycles-pp.iterate_dir.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe
11.42 -2.2 9.24 ± 4% perf-profile.calltrace.cycles-pp.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64.do_syscall_64
11.56 ± 3% -1.2 10.36 ± 7% perf-profile.calltrace.cycles-pp.execve
11.44 ± 3% -1.2 10.26 ± 7% perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
11.44 ± 3% -1.2 10.26 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
11.44 ± 3% -1.2 10.26 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.execve
11.31 ± 3% -1.2 10.13 ± 7% perf-profile.calltrace.cycles-pp.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
12.64 ± 2% -1.1 11.57 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
2.84 ± 8% -1.1 1.78 ± 20% perf-profile.calltrace.cycles-pp.do_raw_spin_lock.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64
2.45 ± 9% -1.0 1.45 ± 24% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath.do_raw_spin_lock.scan_positives.dcache_readdir.iterate_dir
4.78 ± 3% -0.9 3.88 ± 17% perf-profile.calltrace.cycles-pp.kill
4.76 ± 3% -0.9 3.85 ± 17% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.kill
4.75 ± 3% -0.9 3.85 ± 17% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill
5.38 ± 4% -0.9 4.48 ± 3% perf-profile.calltrace.cycles-pp.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
5.01 ± 4% -0.8 4.21 ± 8% perf-profile.calltrace.cycles-pp.lock_page_memcg.page_remove_rmap.zap_pte_range.unmap_page_range.unmap_vmas
6.32 ± 4% -0.6 5.68 ± 3% perf-profile.calltrace.cycles-pp.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap.mmput
3.14 ± 3% -0.6 2.52 ± 16% perf-profile.calltrace.cycles-pp.__x64_sys_kill.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill
2.90 ± 4% -0.6 2.34 ± 16% perf-profile.calltrace.cycles-pp.kill_proc_info.__x64_sys_kill.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill
9.05 -0.5 8.56 ± 2% perf-profile.calltrace.cycles-pp.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.26 ± 5% -0.5 3.77 ± 3% perf-profile.calltrace.cycles-pp.filemap_map_pages.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
1.60 ± 15% -0.5 1.13 ± 3% perf-profile.calltrace.cycles-pp.path_lookupat.filename_lookup.vfs_statx.__do_sys_newstat.do_syscall_64
3.86 -0.5 3.39 perf-profile.calltrace.cycles-pp.dput.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64
8.61 -0.5 8.14 ± 3% perf-profile.calltrace.cycles-pp.load_elf_binary.exec_binprm.bprm_execve.do_execveat_common.__x64_sys_execve
4.76 ± 4% -0.5 4.29 ± 2% perf-profile.calltrace.cycles-pp.page_remove_rmap.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap
8.70 -0.5 8.23 ± 3% perf-profile.calltrace.cycles-pp.exec_binprm.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64
1.72 ± 12% -0.4 1.30 ± 19% perf-profile.calltrace.cycles-pp.filename_lookup.vfs_statx.__do_sys_newstat.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.98 ± 3% -0.4 4.59 ± 2% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.96 ± 3% -0.4 4.58 ± 2% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64
6.33 -0.3 5.99 ± 2% perf-profile.calltrace.cycles-pp.copy_process.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.54 ± 5% -0.3 1.23 ± 17% perf-profile.calltrace.cycles-pp.do_send_sig_info.kill_pid_info.kill_proc_info.__x64_sys_kill.do_syscall_64
2.61 ± 5% -0.3 2.30 ± 2% perf-profile.calltrace.cycles-pp.lock_acquire.lock_page_memcg.page_remove_rmap.zap_pte_range.unmap_page_range
2.74 ± 4% -0.3 2.43 ± 3% perf-profile.calltrace.cycles-pp.kill_pid_info.kill_proc_info.__x64_sys_kill.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.30 -0.3 5.01 ± 2% perf-profile.calltrace.cycles-pp.dup_mm.copy_process.kernel_clone.__do_sys_clone.do_syscall_64
5.19 -0.3 4.90 ± 2% perf-profile.calltrace.cycles-pp.dup_mmap.dup_mm.copy_process.kernel_clone.__do_sys_clone
2.89 ± 2% -0.3 2.61 ± 3% perf-profile.calltrace.cycles-pp.do_set_pte.filemap_map_pages.do_fault.__handle_mm_fault.handle_mm_fault
0.73 ± 6% -0.3 0.46 ± 63% perf-profile.calltrace.cycles-pp.get_mem_cgroup_from_mm.mem_cgroup_charge.do_anonymous_page.__handle_mm_fault.handle_mm_fault
2.01 ± 2% -0.3 1.74 ± 8% perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.begin_new_exec.load_elf_binary
1.99 ± 2% -0.3 1.73 ± 8% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.begin_new_exec
2.52 -0.2 2.27 ± 2% perf-profile.calltrace.cycles-pp.setlocale
5.87 -0.2 5.62 ± 3% perf-profile.calltrace.cycles-pp.do_anonymous_page.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
1.46 ± 5% -0.2 1.21 ± 6% perf-profile.calltrace.cycles-pp.elf_map.load_elf_binary.exec_binprm.bprm_execve.do_execveat_common
2.30 -0.2 2.06 ± 3% perf-profile.calltrace.cycles-pp.lock_page_memcg.page_add_file_rmap.do_set_pte.filemap_map_pages.do_fault
0.63 ± 7% -0.2 0.40 ± 63% perf-profile.calltrace.cycles-pp.perf_event_mmap.do_brk_flags.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.21 ± 5% -0.2 0.98 ± 15% perf-profile.calltrace.cycles-pp.unlink
1.20 ± 5% -0.2 0.97 ± 15% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.unlink
1.20 ± 6% -0.2 0.97 ± 15% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink
2.40 -0.2 2.17 ± 3% perf-profile.calltrace.cycles-pp.page_add_file_rmap.do_set_pte.filemap_map_pages.do_fault.__handle_mm_fault
1.26 ± 4% -0.2 1.04 ± 14% perf-profile.calltrace.cycles-pp.do_brk_flags.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk
2.10 -0.2 1.88 ± 2% perf-profile.calltrace.cycles-pp.lockref_put_or_lock.dput.scan_positives.dcache_readdir.iterate_dir
1.06 ± 24% -0.2 0.83 ± 3% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.vm_area_dup.dup_mmap.dup_mm.copy_process
1.19 ± 6% -0.2 0.98 ± 7% perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.elf_map.load_elf_binary.exec_binprm.bprm_execve
1.09 ± 6% -0.2 0.89 ± 15% perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink
0.80 ± 5% -0.2 0.60 ± 43% perf-profile.calltrace.cycles-pp.arch_do_signal_or_restart.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.96 ± 4% -0.2 0.77 ± 16% perf-profile.calltrace.cycles-pp.group_send_sig_info.kill_pid_info.kill_proc_info.__x64_sys_kill.do_syscall_64
1.08 ± 5% -0.2 0.88 ± 8% perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.elf_map.load_elf_binary.exec_binprm
1.06 ± 6% -0.2 0.87 ± 7% perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.elf_map.load_elf_binary
1.40 ± 2% -0.2 1.21 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_unlock.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64
1.29 ± 2% -0.2 1.11 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_lock_nested.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64
1.32 ± 3% -0.2 1.14 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64
1.24 ± 2% -0.2 1.06 ± 4% perf-profile.calltrace.cycles-pp.lock_acquire._raw_spin_lock_nested.scan_positives.dcache_readdir.iterate_dir
1.02 ± 3% -0.2 0.84 ± 15% perf-profile.calltrace.cycles-pp.alloc_pages_vma.do_anonymous_page.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
0.99 ± 3% -0.2 0.82 ± 15% perf-profile.calltrace.cycles-pp.__alloc_pages.alloc_pages_vma.do_anonymous_page.__handle_mm_fault.handle_mm_fault
2.02 -0.2 1.85 perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.96 ± 2% -0.2 1.80 perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64
0.92 ± 5% -0.2 0.76 ± 17% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill
1.22 ± 3% -0.2 1.06 ± 3% perf-profile.calltrace.cycles-pp.lock_acquire._raw_spin_lock.scan_positives.dcache_readdir.iterate_dir
1.13 ± 3% -0.2 0.97 ± 3% perf-profile.calltrace.cycles-pp.lock_release._raw_spin_unlock.scan_positives.dcache_readdir.iterate_dir
1.32 ± 2% -0.1 1.18 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.lockref_put_or_lock.dput.scan_positives.dcache_readdir
0.59 ± 6% -0.1 0.45 ± 40% perf-profile.calltrace.cycles-pp.rcu_read_lock_sched_held.lock_acquire.lock_page_memcg.page_remove_rmap.zap_pte_range
1.35 ± 2% -0.1 1.21 ± 3% perf-profile.calltrace.cycles-pp.lock_acquire.lock_page_memcg.page_add_file_rmap.do_set_pte.filemap_map_pages
2.85 ± 2% -0.1 2.72 perf-profile.calltrace.cycles-pp.__libc_fork
0.87 ± 3% -0.1 0.74 ± 3% perf-profile.calltrace.cycles-pp.sync
0.86 ± 3% -0.1 0.74 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.sync
0.86 ± 3% -0.1 0.74 ± 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.sync
0.97 ± 5% -0.1 0.84 ± 4% perf-profile.calltrace.cycles-pp.lock_acquire.dput.scan_positives.dcache_readdir.iterate_dir
0.86 ± 3% -0.1 0.74 ± 3% perf-profile.calltrace.cycles-pp.__x64_sys_sync.do_syscall_64.entry_SYSCALL_64_after_hwframe.sync
0.86 ± 3% -0.1 0.74 ± 3% perf-profile.calltrace.cycles-pp.ksys_sync.__x64_sys_sync.do_syscall_64.entry_SYSCALL_64_after_hwframe.sync
1.39 ± 2% -0.1 1.27 ± 3% perf-profile.calltrace.cycles-pp.vfs_statx.__do_sys_newstat.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.40 ± 2% -0.1 1.28 ± 4% perf-profile.calltrace.cycles-pp.__do_sys_newstat.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.44 ± 2% -0.1 2.32 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_fork
2.44 ± 2% -0.1 2.32 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
0.80 ± 3% -0.1 0.69 ± 4% perf-profile.calltrace.cycles-pp.iterate_supers.ksys_sync.__x64_sys_sync.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.94 ± 4% -0.1 0.82 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__mmap
0.94 ± 4% -0.1 0.82 ± 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__mmap
2.44 ± 2% -0.1 2.32 ± 2% perf-profile.calltrace.cycles-pp.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
2.44 ± 2% -0.1 2.32 ± 2% perf-profile.calltrace.cycles-pp.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
0.94 ± 4% -0.1 0.83 ± 3% perf-profile.calltrace.cycles-pp.__mmap
1.19 ± 2% -0.1 1.08 ± 4% perf-profile.calltrace.cycles-pp.lock_acquire._raw_spin_lock.lockref_put_or_lock.dput.scan_positives
0.82 ± 3% -0.1 0.71 ± 6% perf-profile.calltrace.cycles-pp.__xstat64
0.68 ± 3% -0.1 0.57 ± 4% perf-profile.calltrace.cycles-pp.__lock_acquire.lock_acquire._raw_spin_lock_nested.scan_positives.dcache_readdir
0.91 ± 4% -0.1 0.80 ± 3% perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.__mmap
0.94 ± 4% -0.1 0.84 ± 8% perf-profile.calltrace.cycles-pp.__clear_user.load_elf_binary.exec_binprm.bprm_execve.do_execveat_common
0.73 ± 3% -0.1 0.63 ± 6% perf-profile.calltrace.cycles-pp.__do_sys_newstat.do_syscall_64.entry_SYSCALL_64_after_hwframe.__xstat64
0.88 ± 4% -0.1 0.78 ± 3% perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.__mmap
0.75 ± 3% -0.1 0.65 ± 6% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__xstat64
0.75 ± 2% -0.1 0.65 ± 6% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__xstat64
2.33 -0.1 2.23 ± 2% perf-profile.calltrace.cycles-pp.anon_vma_fork.dup_mmap.dup_mm.copy_process.kernel_clone
0.70 ± 3% -0.1 0.61 ± 6% perf-profile.calltrace.cycles-pp.vfs_statx.__do_sys_newstat.do_syscall_64.entry_SYSCALL_64_after_hwframe.__xstat64
1.14 ± 3% -0.1 1.05 ± 3% perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel
1.18 ± 3% -0.1 1.09 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__open64_nocancel.setlocale
1.18 ± 3% -0.1 1.09 ± 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel.setlocale
1.19 ± 3% -0.1 1.11 ± 3% perf-profile.calltrace.cycles-pp.__open64_nocancel.setlocale
1.14 ± 3% -0.1 1.06 ± 3% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel.setlocale
0.69 ± 4% -0.1 0.61 ± 4% perf-profile.calltrace.cycles-pp.__lock_acquire.lock_acquire._raw_spin_lock.scan_positives.dcache_readdir
0.69 ± 4% -0.1 0.61 ± 2% perf-profile.calltrace.cycles-pp.asm_exc_page_fault.setlocale
0.67 ± 5% -0.1 0.59 ± 5% perf-profile.calltrace.cycles-pp.rcu_read_lock_sched_held.lock_release._raw_spin_unlock.scan_positives.dcache_readdir
0.67 ± 4% -0.1 0.59 ± 2% perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.setlocale
0.68 ± 5% -0.1 0.60 ± 2% perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault.setlocale
0.79 ± 4% -0.1 0.72 ± 4% perf-profile.calltrace.cycles-pp.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.79 ± 4% -0.1 0.72 ± 4% perf-profile.calltrace.cycles-pp.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.70 ± 3% -0.1 0.63 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_unlock.lockref_put_or_lock.dput.scan_positives.dcache_readdir
0.66 ± 3% -0.1 0.60 ± 5% perf-profile.calltrace.cycles-pp.__lock_acquire.lock_acquire._raw_spin_lock.lockref_put_or_lock.dput
0.70 ± 5% -0.1 0.63 ± 4% perf-profile.calltrace.cycles-pp.__lock_acquire.lock_acquire.lock_page_memcg.page_add_file_rmap.do_set_pte
0.74 ± 3% -0.1 0.68 ± 4% perf-profile.calltrace.cycles-pp.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.58 ± 5% -0.1 0.52 ± 2% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.setlocale
0.00 +0.6 0.56 ± 5% perf-profile.calltrace.cycles-pp.__schedule.schedule.worker_thread.kthread.ret_from_fork
0.00 +0.6 0.58 ± 4% perf-profile.calltrace.cycles-pp.schedule.worker_thread.kthread.ret_from_fork
0.00 +0.9 0.92 ± 2% perf-profile.calltrace.cycles-pp.lock_acquire.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work
3.56 ± 8% +1.6 5.19 ± 3% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
5.41 ± 8% +1.8 7.24 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
5.46 ± 8% +1.8 7.30 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.00 +1.9 1.95 ± 2% perf-profile.calltrace.cycles-pp.mem_cgroup_css_rstat_flush.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work
6.01 ± 9% +2.3 8.32 ± 3% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
6.01 ± 9% +2.3 8.32 ± 3% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
6.00 ± 9% +2.3 8.31 ± 3% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
6.40 ± 9% +2.4 8.83 ± 4% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
0.00 +4.7 4.70 ± 2% perf-profile.calltrace.cycles-pp.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work.worker_thread
0.00 +4.7 4.75 ± 2% perf-profile.calltrace.cycles-pp.cgroup_rstat_flush_irqsafe.mem_cgroup_flush_stats.process_one_work.worker_thread.kthread
0.00 +5.0 4.97 ± 2% perf-profile.calltrace.cycles-pp.mem_cgroup_flush_stats.process_one_work.worker_thread.kthread.ret_from_fork
0.51 ± 44% +5.3 5.83 perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
1.45 ± 3% +5.8 7.26 perf-profile.calltrace.cycles-pp.ret_from_fork
1.43 ± 3% +5.8 7.24 perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
0.62 ± 7% +5.9 6.49 perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
68.58 -6.3 62.23 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
68.53 -6.3 62.18 perf-profile.children.cycles-pp.do_syscall_64
12.00 -2.2 9.77 ± 4% perf-profile.children.cycles-pp.__x64_sys_getdents64
11.84 -2.2 9.62 ± 4% perf-profile.children.cycles-pp.dcache_readdir
11.98 -2.2 9.75 ± 4% perf-profile.children.cycles-pp.iterate_dir
11.43 -2.2 9.26 ± 4% perf-profile.children.cycles-pp.scan_positives
21.24 -1.5 19.78 perf-profile.children.cycles-pp.asm_exc_page_fault
20.66 -1.3 19.32 perf-profile.children.cycles-pp.exc_page_fault
20.44 -1.3 19.12 perf-profile.children.cycles-pp.do_user_addr_fault
33.66 -1.2 32.41 perf-profile.children.cycles-pp.lock_acquire
16.79 -1.2 15.57 perf-profile.children.cycles-pp.__handle_mm_fault
11.56 ± 3% -1.2 10.37 ± 7% perf-profile.children.cycles-pp.execve
19.02 -1.1 17.89 perf-profile.children.cycles-pp.handle_mm_fault
9.69 ± 2% -1.1 8.61 perf-profile.children.cycles-pp.lock_page_memcg
11.75 -0.9 10.80 perf-profile.children.cycles-pp.check_preemption_disabled
23.40 -0.9 22.48 perf-profile.children.cycles-pp.rcu_read_lock_sched_held
8.81 -0.9 7.90 perf-profile.children.cycles-pp.do_fault
4.80 ± 3% -0.9 3.88 ± 17% perf-profile.children.cycles-pp.kill
7.80 -0.8 6.96 ± 2% perf-profile.children.cycles-pp.filemap_map_pages
11.79 -0.8 11.00 perf-profile.children.cycles-pp.__x64_sys_execve
11.65 -0.8 10.86 perf-profile.children.cycles-pp.do_execveat_common
6.16 -0.7 5.46 ± 2% perf-profile.children.cycles-pp.dput
18.41 -0.7 17.71 perf-profile.children.cycles-pp.lock_release
10.40 -0.6 9.78 perf-profile.children.cycles-pp.rcu_read_lock_held_common
5.50 ± 2% -0.6 4.88 ± 2% perf-profile.children.cycles-pp.do_set_pte
8.02 ± 3% -0.6 7.42 perf-profile.children.cycles-pp.unmap_vmas
7.92 ± 3% -0.6 7.33 perf-profile.children.cycles-pp.unmap_page_range
10.56 ± 2% -0.6 9.99 perf-profile.children.cycles-pp.mmput
17.03 -0.6 16.46 perf-profile.children.cycles-pp.__lock_acquire
10.54 ± 2% -0.6 9.97 perf-profile.children.cycles-pp.exit_mmap
7.72 ± 3% -0.6 7.16 perf-profile.children.cycles-pp.zap_pte_range
9.20 -0.5 8.66 perf-profile.children.cycles-pp.bprm_execve
17.96 -0.5 17.42 perf-profile.children.cycles-pp.lock_is_held_type
3.54 ± 2% -0.5 3.01 ± 2% perf-profile.children.cycles-pp.debug_lockdep_rcu_enabled
8.75 -0.5 8.23 ± 2% perf-profile.children.cycles-pp.load_elf_binary
8.85 -0.5 8.33 ± 2% perf-profile.children.cycles-pp.exec_binprm
4.53 -0.5 4.02 ± 2% perf-profile.children.cycles-pp.page_add_file_rmap
6.34 ± 2% -0.5 5.88 perf-profile.children.cycles-pp.do_sys_open
6.34 ± 2% -0.5 5.87 perf-profile.children.cycles-pp.do_sys_openat2
10.06 -0.4 9.63 perf-profile.children.cycles-pp.kmem_cache_alloc
3.29 -0.4 2.87 ± 2% perf-profile.children.cycles-pp.__x64_sys_kill
5.76 ± 3% -0.4 5.36 perf-profile.children.cycles-pp.do_filp_open
5.75 ± 3% -0.4 5.35 perf-profile.children.cycles-pp.path_openat
6.01 ± 3% -0.4 5.63 perf-profile.children.cycles-pp.page_remove_rmap
3.04 -0.4 2.66 ± 2% perf-profile.children.cycles-pp.kill_proc_info
2.86 -0.4 2.49 ± 2% perf-profile.children.cycles-pp.kill_pid_info
3.93 -0.4 3.56 perf-profile.children.cycles-pp.trace_hardirqs_on
3.40 -0.4 3.03 ± 2% perf-profile.children.cycles-pp.vm_mmap_pgoff
6.52 -0.4 6.17 ± 2% perf-profile.children.cycles-pp.__do_sys_clone
6.52 -0.4 6.17 ± 2% perf-profile.children.cycles-pp.kernel_clone
3.56 ± 2% -0.4 3.20 ± 2% perf-profile.children.cycles-pp.lockdep_hardirqs_on_prepare
6.34 -0.3 5.99 ± 2% perf-profile.children.cycles-pp.copy_process
3.13 -0.3 2.79 ± 2% perf-profile.children.cycles-pp.do_mmap
3.05 -0.3 2.72 ± 2% perf-profile.children.cycles-pp.mmap_region
3.31 ± 2% -0.3 2.98 ± 3% perf-profile.children.cycles-pp.___might_sleep
3.99 -0.3 3.67 perf-profile.children.cycles-pp.mark_lock
2.22 -0.3 1.91 perf-profile.children.cycles-pp.perf_event_mmap
2.51 -0.3 2.20 ± 2% perf-profile.children.cycles-pp.fs_reclaim_acquire
4.88 -0.3 4.57 perf-profile.children.cycles-pp.rcu_is_watching
2.86 ± 2% -0.3 2.55 perf-profile.children.cycles-pp.__alloc_pages
2.75 ± 3% -0.3 2.45 ± 2% perf-profile.children.cycles-pp.lockref_put_or_lock
5.31 -0.3 5.02 ± 2% perf-profile.children.cycles-pp.dup_mm
5.20 -0.3 4.91 ± 2% perf-profile.children.cycles-pp.dup_mmap
2.65 ± 3% -0.3 2.37 ± 2% perf-profile.children.cycles-pp.get_obj_cgroup_from_current
2.45 ± 2% -0.3 2.18 ± 4% perf-profile.children.cycles-pp.__might_fault
3.55 ± 2% -0.3 3.28 perf-profile.children.cycles-pp.walk_component
4.79 -0.3 4.54 perf-profile.children.cycles-pp.rcu_lockdep_current_cpu_online
2.52 -0.2 2.27 ± 2% perf-profile.children.cycles-pp.setlocale
3.08 ± 3% -0.2 2.83 perf-profile.children.cycles-pp.link_path_walk
2.58 ± 2% -0.2 2.33 perf-profile.children.cycles-pp.syscall_exit_to_user_mode
1.21 ± 5% -0.2 0.98 ± 15% perf-profile.children.cycles-pp.unlink
6.20 -0.2 5.97 ± 2% perf-profile.children.cycles-pp.do_anonymous_page
1.61 ± 2% -0.2 1.39 ± 3% perf-profile.children.cycles-pp.do_send_sig_info
2.16 -0.2 1.95 ± 2% perf-profile.children.cycles-pp.vfs_statx
2.12 -0.2 1.91 ± 2% perf-profile.children.cycles-pp.__do_sys_newstat
2.88 ± 3% -0.2 2.67 ± 3% perf-profile.children.cycles-pp.rcu_read_lock_held
1.50 ± 4% -0.2 1.29 ± 3% perf-profile.children.cycles-pp.elf_map
2.14 ± 2% -0.2 1.94 ± 3% perf-profile.children.cycles-pp.filename_lookup
2.11 ± 2% -0.2 1.91 perf-profile.children.cycles-pp.mark_held_locks
2.24 -0.2 2.04 perf-profile.children.cycles-pp.ksys_mmap_pgoff
2.09 ± 2% -0.2 1.90 ± 3% perf-profile.children.cycles-pp.path_lookupat
1.35 ± 2% -0.2 1.16 ± 5% perf-profile.children.cycles-pp._raw_spin_lock_nested
1.58 -0.2 1.41 perf-profile.children.cycles-pp.alloc_pages_vma
2.00 ± 2% -0.2 1.84 perf-profile.children.cycles-pp.exit_to_user_mode_prepare
1.62 -0.2 1.45 ± 3% perf-profile.children.cycles-pp.__perf_sw_event
1.48 ± 5% -0.2 1.32 ± 4% perf-profile.children.cycles-pp.trace_hardirqs_off
1.17 ± 4% -0.2 1.01 ± 3% perf-profile.children.cycles-pp.lockdep_hardirqs_off
1.41 -0.2 1.25 ± 2% perf-profile.children.cycles-pp.do_brk_flags
1.16 ± 5% -0.1 1.02 ± 6% perf-profile.children.cycles-pp.do_unlinkat
4.74 -0.1 4.60 perf-profile.children.cycles-pp.__x64_sys_brk
1.20 ± 3% -0.1 1.06 ± 2% perf-profile.children.cycles-pp.down_write
1.64 ± 2% -0.1 1.51 perf-profile.children.cycles-pp.__open64_nocancel
2.86 ± 2% -0.1 2.73 ± 2% perf-profile.children.cycles-pp.__libc_fork
1.01 ± 3% -0.1 0.88 ± 5% perf-profile.children.cycles-pp.group_send_sig_info
1.22 ± 5% -0.1 1.09 ± 3% perf-profile.children.cycles-pp.step_into
0.89 ± 3% -0.1 0.76 ± 2% perf-profile.children.cycles-pp.get_user_arg_ptr
1.33 ± 4% -0.1 1.21 ± 2% perf-profile.children.cycles-pp.copy_strings
0.87 ± 3% -0.1 0.74 ± 3% perf-profile.children.cycles-pp.sync
1.16 ± 2% -0.1 1.04 ± 4% perf-profile.children.cycles-pp.get_mem_cgroup_from_mm
0.86 ± 3% -0.1 0.74 ± 3% perf-profile.children.cycles-pp.__x64_sys_sync
0.86 ± 3% -0.1 0.74 ± 3% perf-profile.children.cycles-pp.ksys_sync
1.26 ± 4% -0.1 1.14 ± 3% perf-profile.children.cycles-pp.arch_do_signal_or_restart
2.50 ± 2% -0.1 2.38 ± 3% perf-profile.children.cycles-pp.mem_cgroup_charge
0.77 ± 4% -0.1 0.65 ± 4% perf-profile.children.cycles-pp.d_path
0.81 ± 3% -0.1 0.69 ± 4% perf-profile.children.cycles-pp.iterate_supers
0.59 ± 4% -0.1 0.47 ± 5% perf-profile.children.cycles-pp.count
0.95 ± 5% -0.1 0.83 ± 3% perf-profile.children.cycles-pp.__mmap
1.10 ± 4% -0.1 0.99 ± 4% perf-profile.children.cycles-pp.getname_flags
0.87 -0.1 0.76 ± 3% perf-profile.children.cycles-pp.terminate_walk
2.02 ± 2% -0.1 1.91 perf-profile.children.cycles-pp.vm_area_dup
0.94 ± 3% -0.1 0.83 ± 4% perf-profile.children.cycles-pp.next_uptodate_page
0.82 ± 3% -0.1 0.71 ± 6% perf-profile.children.cycles-pp.__xstat64
1.79 ± 3% -0.1 1.68 ± 2% perf-profile.children.cycles-pp.d_alloc_parallel
0.93 ± 3% -0.1 0.82 ± 4% perf-profile.children.cycles-pp.perf_iterate_sb
0.66 ± 5% -0.1 0.56 ± 4% perf-profile.children.cycles-pp.__task_pid_nr_ns
1.12 ± 3% -0.1 1.02 ± 5% perf-profile.children.cycles-pp.__x64_sys_rt_sigreturn
1.91 ± 3% -0.1 1.80 ± 2% perf-profile.children.cycles-pp.__lookup_slow
0.66 ± 4% -0.1 0.55 ± 4% perf-profile.children.cycles-pp.syscall_enter_from_user_mode
0.84 ± 3% -0.1 0.74 ± 2% perf-profile.children.cycles-pp.obj_cgroup_charge
0.56 ± 4% -0.1 0.46 ± 8% perf-profile.children.cycles-pp.prepend_path
0.56 ± 6% -0.1 0.46 ± 18% perf-profile.children.cycles-pp.sighandler
2.33 -0.1 2.24 ± 2% perf-profile.children.cycles-pp.anon_vma_fork
0.88 ± 5% -0.1 0.79 ± 3% perf-profile.children.cycles-pp.__memcg_kmem_charge_page
0.68 ± 5% -0.1 0.59 ± 5% perf-profile.children.cycles-pp.send_signal
0.31 ± 13% -0.1 0.22 ± 12% perf-profile.children.cycles-pp.strnlen_user
0.98 ± 4% -0.1 0.90 ± 3% perf-profile.children.cycles-pp.__clear_user
0.85 ± 4% -0.1 0.77 perf-profile.children.cycles-pp.get_mem_cgroup_from_objcg
0.80 ± 5% -0.1 0.72 ± 4% perf-profile.children.cycles-pp.obj_cgroup_charge_pages
0.64 ± 4% -0.1 0.56 ± 3% perf-profile.children.cycles-pp.up_write
0.66 ± 3% -0.1 0.58 ± 5% perf-profile.children.cycles-pp.restore_sigcontext
0.85 ± 3% -0.1 0.77 ± 2% perf-profile.children.cycles-pp.lockdep_hardirqs_on
0.73 ± 4% -0.1 0.65 ± 4% perf-profile.children.cycles-pp.mem_cgroup_from_task
0.79 ± 4% -0.1 0.72 ± 4% perf-profile.children.cycles-pp.__x64_sys_mprotect
0.79 ± 4% -0.1 0.72 ± 4% perf-profile.children.cycles-pp.do_mprotect_pkey
0.54 ± 5% -0.1 0.47 ± 3% perf-profile.children.cycles-pp.lookup_fast
0.80 ± 4% -0.1 0.73 ± 7% perf-profile.children.cycles-pp.__x64_sys_link
0.54 ± 4% -0.1 0.47 ± 6% perf-profile.children.cycles-pp.try_to_unlazy
0.50 ± 6% -0.1 0.43 ± 7% perf-profile.children.cycles-pp.check_kill_permission
1.08 ± 2% -0.1 1.01 ± 3% perf-profile.children.cycles-pp.__split_vma
0.64 ± 5% -0.1 0.57 ± 4% perf-profile.children.cycles-pp.cgroup_throttle_swaprate
0.50 ± 4% -0.1 0.43 ± 3% perf-profile.children.cycles-pp.vma_merge
0.65 ± 2% -0.1 0.59 ± 6% perf-profile.children.cycles-pp.get_page_from_freelist
0.43 ± 5% -0.1 0.37 ± 6% perf-profile.children.cycles-pp.__legitimize_path
0.36 ± 2% -0.1 0.29 ± 3% perf-profile.children.cycles-pp._dl_addr
0.55 ± 4% -0.1 0.49 ± 5% perf-profile.children.cycles-pp.time
0.74 ± 3% -0.1 0.68 ± 4% perf-profile.children.cycles-pp.mprotect_fixup
0.46 ± 3% -0.1 0.40 ± 3% perf-profile.children.cycles-pp.__lock_task_sighand
0.45 ± 5% -0.1 0.39 ± 7% perf-profile.children.cycles-pp.copy_fpstate_to_sigframe
0.25 ± 6% -0.1 0.19 ± 8% perf-profile.children.cycles-pp.irqentry_exit_to_user_mode
0.38 ± 5% -0.1 0.32 ± 7% perf-profile.children.cycles-pp.do_wait
0.43 ± 2% -0.1 0.37 ± 4% perf-profile.children.cycles-pp.down_read
0.39 ± 5% -0.1 0.34 ± 7% perf-profile.children.cycles-pp.__do_sys_wait4
0.39 ± 5% -0.1 0.34 ± 7% perf-profile.children.cycles-pp.kernel_wait4
0.41 ± 6% -0.1 0.35 ± 6% perf-profile.children.cycles-pp.lockref_get_not_dead
0.46 ± 4% -0.1 0.40 ± 7% perf-profile.children.cycles-pp.trace_hardirqs_off_finish
0.08 ± 10% -0.1 0.03 ± 87% perf-profile.children.cycles-pp.load_balance
0.36 ± 4% -0.0 0.32 ± 4% perf-profile.children.cycles-pp.free_unref_page_list
0.33 ± 5% -0.0 0.29 ± 5% perf-profile.children.cycles-pp.__rcu_read_lock
0.50 ± 3% -0.0 0.45 ± 5% perf-profile.children.cycles-pp.preempt_count_add
0.61 ± 4% -0.0 0.57 ± 4% perf-profile.children.cycles-pp.lookup_open
0.40 ± 5% -0.0 0.35 ± 2% perf-profile.children.cycles-pp.__get_free_pages
0.25 ± 5% -0.0 0.21 ± 11% perf-profile.children.cycles-pp.audit_signal_info_syscall
0.30 ± 4% -0.0 0.27 ± 3% perf-profile.children.cycles-pp.tracer_hardirqs_on
0.26 ± 4% -0.0 0.23 ± 5% perf-profile.children.cycles-pp.kmem_cache_alloc_trace
0.24 ± 5% -0.0 0.21 ± 8% perf-profile.children.cycles-pp.wait4
0.27 ± 4% -0.0 0.24 ± 5% perf-profile.children.cycles-pp.__pud_alloc
0.09 ± 17% -0.0 0.06 ± 12% perf-profile.children.cycles-pp.do_task_dead
0.25 ± 4% -0.0 0.22 ± 9% perf-profile.children.cycles-pp.__cxa_atexit
0.17 ± 6% -0.0 0.14 ± 10% perf-profile.children.cycles-pp.__might_sleep
0.09 ± 11% -0.0 0.07 ± 13% perf-profile.children.cycles-pp.__x64_sys_unlink
0.06 ± 7% +0.0 0.09 ± 12% perf-profile.children.cycles-pp.delete_from_page_cache
0.07 ± 13% +0.0 0.10 ± 14% perf-profile.children.cycles-pp.truncate_inode_page
0.08 ± 12% +0.0 0.11 ± 22% perf-profile.children.cycles-pp.run_rebalance_domains
0.04 ± 71% +0.0 0.07 ± 15% perf-profile.children.cycles-pp.prandom_u32
0.08 ± 12% +0.1 0.13 ± 12% perf-profile.children.cycles-pp.update_rq_clock
0.06 ± 13% +0.1 0.11 ± 7% perf-profile.children.cycles-pp.enqueue_entity
0.05 ± 48% +0.1 0.10 ± 14% perf-profile.children.cycles-pp.lock_pin_lock
0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.select_idle_sibling
0.00 +0.1 0.06 ± 8% perf-profile.children.cycles-pp.unaccount_page_cache_page
0.02 ±141% +0.1 0.07 ± 15% perf-profile.children.cycles-pp.__delete_from_page_cache
0.00 +0.1 0.06 ± 12% perf-profile.children.cycles-pp.check_preempt_curr
0.08 ± 13% +0.1 0.14 ± 10% perf-profile.children.cycles-pp.cpuacct_charge
0.12 ± 7% +0.1 0.18 ± 13% perf-profile.children.cycles-pp.update_load_avg
0.07 ± 14% +0.1 0.15 ± 6% perf-profile.children.cycles-pp.enqueue_task_fair
0.00 +0.1 0.08 ± 12% perf-profile.children.cycles-pp.rcu_note_context_switch
0.06 ± 13% +0.1 0.14 ± 10% perf-profile.children.cycles-pp.preempt_schedule_common
0.00 +0.1 0.09 ± 8% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.06 ± 11% +0.1 0.15 ± 10% perf-profile.children.cycles-pp.preempt_schedule_thunk
0.00 +0.1 0.09 ± 12% perf-profile.children.cycles-pp.put_prev_entity
0.08 ± 14% +0.1 0.17 ± 6% perf-profile.children.cycles-pp.irqentry_exit
0.35 ± 7% +0.1 0.44 ± 6% perf-profile.children.cycles-pp.lock_page_lruvec_irqsave
0.58 ± 4% +0.1 0.67 ± 7% perf-profile.children.cycles-pp._raw_spin_unlock_irq
0.67 ± 2% +0.1 0.77 ± 3% perf-profile.children.cycles-pp.lru_add_drain
0.00 +0.1 0.11 ± 11% perf-profile.children.cycles-pp.perf_tp_event
0.00 +0.1 0.11 ± 10% perf-profile.children.cycles-pp.preempt_schedule_irq
0.22 ± 26% +0.1 0.33 ± 21% perf-profile.children.cycles-pp.tick_nohz_next_event
0.00 +0.1 0.11 ± 10% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
0.13 ± 10% +0.1 0.24 ± 9% perf-profile.children.cycles-pp.pick_next_task_fair
0.44 ± 4% +0.1 0.55 ± 3% perf-profile.children.cycles-pp.lru_add_drain_cpu
0.28 ± 26% +0.1 0.41 ± 21% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.73 ± 3% +0.1 0.86 perf-profile.children.cycles-pp.__pagevec_lru_add_fn
0.26 ± 10% +0.1 0.40 ± 10% perf-profile.children.cycles-pp.finish_task_switch
0.02 ±142% +0.1 0.16 ± 6% perf-profile.children.cycles-pp.ttwu_do_activate
0.09 ± 18% +0.1 0.23 ± 6% perf-profile.children.cycles-pp.select_task_rq_fair
0.05 ± 45% +0.1 0.19 ± 8% perf-profile.children.cycles-pp.mem_cgroup_charge_statistics
0.00 +0.1 0.15 ± 14% perf-profile.children.cycles-pp.asm_sysvec_reschedule_ipi
0.09 ± 15% +0.1 0.24 ± 8% perf-profile.children.cycles-pp.dequeue_entity
0.52 ± 4% +0.2 0.68 ± 3% perf-profile.children.cycles-pp.page_add_new_anon_rmap
0.10 ± 13% +0.2 0.27 ± 8% perf-profile.children.cycles-pp.dequeue_task_fair
0.37 ± 27% +0.2 0.54 ± 21% perf-profile.children.cycles-pp.menu_select
0.00 +0.2 0.18 ± 4% perf-profile.children.cycles-pp._find_next_bit
0.26 ± 7% +0.2 0.45 ± 5% perf-profile.children.cycles-pp.update_curr
1.12 ± 2% +0.2 1.32 perf-profile.children.cycles-pp.__pagevec_lru_add
0.14 ± 16% +0.2 0.34 ± 12% perf-profile.children.cycles-pp.schedule_idle
0.64 ± 4% +0.5 1.09 ± 4% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.28 ± 7% +0.5 0.78 ± 3% perf-profile.children.cycles-pp.schedule
1.06 ± 2% +0.6 1.61 ± 3% perf-profile.children.cycles-pp.__mod_lruvec_page_state
0.16 ± 4% +0.7 0.83 ± 3% perf-profile.children.cycles-pp.try_to_wake_up
0.59 ± 8% +0.9 1.45 ± 3% perf-profile.children.cycles-pp.__schedule
0.00 +0.9 0.90 ± 2% perf-profile.children.cycles-pp.__queue_work
0.00 +1.0 0.97 ± 2% perf-profile.children.cycles-pp.queue_work_on
0.37 ± 4% +1.1 1.46 ± 3% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
0.07 ± 18% +1.3 1.37 ± 4% perf-profile.children.cycles-pp.cgroup_rstat_updated
0.92 ± 2% +1.4 2.30 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
3.78 ± 8% +1.7 5.50 ± 3% perf-profile.children.cycles-pp.intel_idle
5.82 ± 9% +1.9 7.75 ± 4% perf-profile.children.cycles-pp.cpuidle_enter
5.81 ± 9% +1.9 7.75 ± 4% perf-profile.children.cycles-pp.cpuidle_enter_state
0.00 +2.0 1.96 ± 2% perf-profile.children.cycles-pp.mem_cgroup_css_rstat_flush
6.01 ± 9% +2.3 8.32 ± 3% perf-profile.children.cycles-pp.start_secondary
6.40 ± 9% +2.4 8.83 ± 4% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
6.40 ± 9% +2.4 8.83 ± 4% perf-profile.children.cycles-pp.cpu_startup_entry
6.40 ± 9% +2.4 8.83 ± 4% perf-profile.children.cycles-pp.do_idle
0.00 +4.7 4.71 ± 2% perf-profile.children.cycles-pp.cgroup_rstat_flush_locked
0.00 +4.7 4.75 ± 2% perf-profile.children.cycles-pp.cgroup_rstat_flush_irqsafe
0.00 +5.0 4.97 ± 2% perf-profile.children.cycles-pp.mem_cgroup_flush_stats
0.59 ± 8% +5.2 5.83 perf-profile.children.cycles-pp.process_one_work
1.54 ± 3% +5.8 7.35 perf-profile.children.cycles-pp.ret_from_fork
1.43 ± 3% +5.8 7.24 perf-profile.children.cycles-pp.kthread
0.62 ± 7% +5.9 6.49 perf-profile.children.cycles-pp.worker_thread
10.94 -0.9 10.05 perf-profile.self.cycles-pp.check_preemption_disabled
3.13 ± 2% -0.5 2.67 ± 2% perf-profile.self.cycles-pp.debug_lockdep_rcu_enabled
3.82 -0.3 3.51 perf-profile.self.cycles-pp.mark_lock
11.35 -0.2 11.11 perf-profile.self.cycles-pp.lock_is_held_type
4.30 -0.2 4.13 ± 2% perf-profile.self.cycles-pp.lock_acquire
2.89 -0.1 2.75 perf-profile.self.cycles-pp.rcu_is_watching
2.00 ± 2% -0.1 1.90 ± 3% perf-profile.self.cycles-pp.rcu_read_lock_sched_held
0.90 -0.1 0.80 ± 3% perf-profile.self.cycles-pp.lockdep_hardirqs_on_prepare
0.60 ± 6% -0.1 0.53 ± 4% perf-profile.self.cycles-pp.lockdep_hardirqs_off
0.93 ± 2% -0.1 0.86 ± 3% perf-profile.self.cycles-pp.mark_held_locks
0.58 ± 5% -0.1 0.52 ± 4% perf-profile.self.cycles-pp.lockdep_hardirqs_on
0.64 -0.1 0.58 ± 4% perf-profile.self.cycles-pp.___might_sleep
0.33 ± 3% -0.1 0.27 ± 4% perf-profile.self.cycles-pp._dl_addr
0.30 ± 5% -0.0 0.25 ± 4% perf-profile.self.cycles-pp.__rcu_read_lock
0.30 ± 7% -0.0 0.26 ± 8% perf-profile.self.cycles-pp.next_uptodate_page
0.07 ± 5% -0.0 0.04 ± 63% perf-profile.self.cycles-pp.down_write
0.23 ± 7% -0.0 0.19 ± 8% perf-profile.self.cycles-pp.lock_page_memcg
0.25 ± 4% -0.0 0.22 ± 3% perf-profile.self.cycles-pp.tracer_hardirqs_on
0.16 ± 7% -0.0 0.13 ± 12% perf-profile.self.cycles-pp.page_remove_rmap
0.10 ± 10% -0.0 0.08 ± 15% perf-profile.self.cycles-pp.trace_hardirqs_on_prepare
0.06 ± 7% -0.0 0.05 ± 9% perf-profile.self.cycles-pp.__rb_insert_augmented
0.10 ± 11% +0.0 0.12 ± 6% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state
0.00 +0.1 0.07 ± 16% perf-profile.self.cycles-pp.__schedule
0.07 ± 18% +0.1 0.18 ± 15% perf-profile.self.cycles-pp.cgroup_rstat_updated
0.00 +0.2 0.17 ± 4% perf-profile.self.cycles-pp._find_next_bit
4.45 +0.2 4.67 perf-profile.self.cycles-pp.lock_release
0.00 +0.3 0.30 ± 6% perf-profile.self.cycles-pp.cgroup_rstat_flush_locked
3.78 ± 8% +1.7 5.50 ± 3% perf-profile.self.cycles-pp.intel_idle
0.00 +1.8 1.80 ± 2% perf-profile.self.cycles-pp.mem_cgroup_css_rstat_flush
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
---
0DAY/LKP+ Test Infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation
Thanks,
Oliver Sang
View attachment "config-5.13.0-rc2-00028-gcda0d205259a" of type "text/plain" (177861 bytes)
View attachment "job-script" of type "text/plain" (7904 bytes)
View attachment "job.yaml" of type "text/plain" (5242 bytes)
View attachment "reproduce" of type "text/plain" (11053 bytes)
Powered by blists - more mailing lists