| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <202307060958.3594f22f-oliver.sang@intel.com>
Date: Thu, 6 Jul 2023 15:29:31 +0800
From: kernel test robot <oliver.sang@...el.com>
To: Jens Axboe <axboe@...nel.dk>
CC: <oe-lkp@...ts.linux.dev>, <lkp@...el.com>,
<linux-kernel@...r.kernel.org>, <io-uring@...r.kernel.org>,
<ying.huang@...el.com>, <feng.tang@...el.com>,
<fengwei.yin@...el.com>, <oliver.sang@...el.com>
Subject: [linus:master] [io_uring] caec5ebe77:
stress-ng.io-uring.ops_per_sec -33.1% regression
Hello,
kernel test robot noticed a -33.1% regression of stress-ng.io-uring.ops_per_sec on:
commit: caec5ebe77f97d948dcf46f07d622bda7f1f6dfd ("io_uring: rely solely on FMODE_NOWAIT")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
NOTE:
one thing we want to mention is we reported
"[linux-next:master] [io_uring] caec5ebe77: stress-ng.io-uring.ops_per_sec 32.5% improvement"
on
https://lore.kernel.org/all/202306061247.510feb23-oliver.sang@intel.com/
however, by further checking, we realized the test machine ran in abnormal
status at that time due to BIOS issue, which we finally fixed recently.
please just ignore that report upon linus-next/master.
testcase: stress-ng
test machine: 64 threads 2 sockets Intel(R) Xeon(R) Gold 6346 CPU @ 3.10GHz (Ice Lake) with 256G memory
parameters:
nr_threads: 100%
testtime: 60s
class: pts
test: io-uring
cpufreq_governor: performance
In addition to that, the commit also has significant impact on the following tests:
+------------------+-------------------------------------------------------------------------------------------------+
| testcase: change | stress-ng: stress-ng.io-uring.ops_per_sec -1.3% regression |
| test machine | 36 threads 1 sockets Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz (Cascade Lake) with 128G memory |
| test parameters | class=pts |
| | cpufreq_governor=performance |
| | nr_threads=100% |
| | test=io-uring |
| | testtime=60s |
+------------------+-------------------------------------------------------------------------------------------------+
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <oliver.sang@...el.com>
| Closes: https://lore.kernel.org/oe-lkp/202307060958.3594f22f-oliver.sang@intel.com
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
sudo bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
sudo bin/lkp run generated-yaml-file
# if come across any failure that blocks the test,
# please remove ~/.lkp and /lkp dir to run from a clean state.
=========================================================================================
class/compiler/cpufreq_governor/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime:
pts/gcc-12/performance/x86_64-rhel-8.3/100%/debian-11.1-x86_64-20220510.cgz/lkp-icl-2sp7/io-uring/stress-ng/60s
commit:
e9833d8701 ("block: mark bdev files as FMODE_NOWAIT if underlying device supports it")
caec5ebe77 ("io_uring: rely solely on FMODE_NOWAIT")
e9833d8701028b5d caec5ebe77f97d948dcf46f07d6
---------------- ---------------------------
%stddev %change %stddev
\ | \
2.594e+08 ± 7% +108.5% 5.408e+08 ± 5% cpuidle..time
265280 ± 6% +5210.7% 14088284 ± 4% cpuidle..usage
5.66 ± 8% +7.2 12.84 ± 5% mpstat.cpu.all.idle%
0.04 ± 7% +0.1 0.15 ± 2% mpstat.cpu.all.soft%
2.04 ± 3% -0.2 1.84 ± 3% mpstat.cpu.all.usr%
7.67 ± 6% +93.5% 14.83 ± 6% vmstat.cpu.id
61.83 +29.1% 79.83 ± 2% vmstat.procs.r
622749 +44.5% 899809 vmstat.system.cs
108826 +59.8% 173876 ± 2% vmstat.system.in
9355 ± 6% +84.7% 17276 ± 2% numa-meminfo.node0.KernelStack
1579458 ± 27% -67.5% 513874 ± 17% numa-meminfo.node1.Inactive
1579339 ± 27% -67.5% 513754 ± 17% numa-meminfo.node1.Inactive(anon)
9295 ± 6% +67.5% 15572 ± 3% numa-meminfo.node1.KernelStack
824102 ± 15% -46.4% 441722 ± 25% numa-meminfo.node1.Mapped
69099 ± 4% -71.9% 19445 ± 13% perf-c2c.DRAM.local
5392 ± 32% +570.9% 36177 ± 9% perf-c2c.DRAM.remote
98.17 ± 13% +11039.9% 10935 ± 2% perf-c2c.HITM.local
61.33 ± 42% +1277.2% 844.67 ± 19% perf-c2c.HITM.remote
159.50 ± 14% +7285.8% 11780 ± 3% perf-c2c.HITM.total
3690643 ± 5% -26.7% 2705345 ± 12% numa-numastat.node0.local_node
3745826 ± 4% -27.3% 2723971 ± 12% numa-numastat.node0.numa_hit
55147 ± 13% -66.2% 18626 ± 71% numa-numastat.node0.other_node
4234422 ± 3% -28.1% 3043703 ± 8% numa-numastat.node1.local_node
4245483 ± 3% -27.2% 3091278 ± 9% numa-numastat.node1.numa_hit
11061 ± 67% +330.1% 47574 ± 27% numa-numastat.node1.other_node
37095722 -33.1% 24820858 stress-ng.io-uring.ops
617812 -33.1% 413235 stress-ng.io-uring.ops_per_sec
228755 ± 5% +3421.6% 8055857 stress-ng.time.involuntary_context_switches
5935 -9.0% 5403 stress-ng.time.percent_of_cpu_this_job_got
3679 -9.1% 3342 stress-ng.time.system_time
10.99 ± 4% +55.0% 17.04 ± 7% stress-ng.time.user_time
40467435 -10.9% 36037982 stress-ng.time.voluntary_context_switches
460384 ± 42% +308.0% 1878137 meminfo.Active
460256 ± 42% +308.0% 1878009 meminfo.Active(anon)
416447 ± 2% -12.8% 363303 meminfo.AnonPages
2233362 ± 8% -60.2% 888272 meminfo.Inactive
2233179 ± 8% -60.2% 888092 meminfo.Inactive(anon)
18675 +75.7% 32806 meminfo.KernelStack
1109597 ± 13% -32.7% 747212 ± 7% meminfo.Mapped
11174 ± 3% -11.4% 9898 ± 2% meminfo.PageTables
2310 ± 18% +98373.0% 2275219 ± 9% turbostat.C1
20795 ± 6% +55407.8% 11543024 ± 4% turbostat.C1E
0.12 ± 5% +5.6 5.69 ± 2% turbostat.C1E%
5.87 ± 6% +102.3% 11.87 ± 5% turbostat.CPU%c1
0.05 -20.0% 0.04 turbostat.IPC
7116993 +60.9% 11448631 ± 2% turbostat.IRQ
85.17 ± 58% +19991.6% 17111 ± 8% turbostat.POLL
245.45 -2.8% 238.59 turbostat.PkgWatt
118.03 -8.3% 108.24 turbostat.RAMWatt
9358 ± 6% +84.6% 17273 ± 2% numa-vmstat.node0.nr_kernel_stack
3746119 ± 4% -27.3% 2723944 ± 12% numa-vmstat.node0.numa_hit
3690936 ± 5% -26.7% 2705318 ± 12% numa-vmstat.node0.numa_local
55147 ± 13% -66.2% 18626 ± 71% numa-vmstat.node0.numa_other
395951 ± 27% -67.6% 128400 ± 17% numa-vmstat.node1.nr_inactive_anon
9297 ± 6% +67.5% 15570 ± 3% numa-vmstat.node1.nr_kernel_stack
207192 ± 15% -46.8% 110289 ± 24% numa-vmstat.node1.nr_mapped
395950 ± 27% -67.6% 128399 ± 17% numa-vmstat.node1.nr_zone_inactive_anon
4245753 ± 3% -27.2% 3091083 ± 9% numa-vmstat.node1.numa_hit
4234692 ± 3% -28.1% 3043508 ± 8% numa-vmstat.node1.numa_local
11061 ± 67% +330.1% 47574 ± 27% numa-vmstat.node1.numa_other
115285 ± 42% +307.6% 469935 proc-vmstat.nr_active_anon
104064 ± 2% -12.7% 90796 proc-vmstat.nr_anon_pages
1264863 +2.5% 1296456 proc-vmstat.nr_file_pages
558025 ± 8% -60.3% 221737 proc-vmstat.nr_inactive_anon
18670 +75.7% 32807 proc-vmstat.nr_kernel_stack
277364 ± 13% -32.8% 186281 ± 7% proc-vmstat.nr_mapped
2789 ± 3% -11.3% 2473 ± 2% proc-vmstat.nr_page_table_pages
569317 +5.5% 600873 proc-vmstat.nr_shmem
61861 +5.3% 65163 proc-vmstat.nr_slab_unreclaimable
115285 ± 42% +307.6% 469935 proc-vmstat.nr_zone_active_anon
558025 ± 8% -60.3% 221737 proc-vmstat.nr_zone_inactive_anon
7995646 -27.2% 5817349 proc-vmstat.numa_hit
7929402 -27.5% 5751148 proc-vmstat.numa_local
3517 ±101% +224.0% 11397 ± 41% proc-vmstat.numa_pages_migrated
28296925 -31.3% 19428277 proc-vmstat.pgalloc_normal
27448011 -32.1% 18626826 proc-vmstat.pgfree
3517 ±101% +224.0% 11397 ± 41% proc-vmstat.pgmigrate_success
14631 ± 24% +44.3% 21116 ± 22% proc-vmstat.pgreuse
1177088 ± 2% +5.3% 1238912 proc-vmstat.unevictable_pgs_scanned
0.64 ± 2% +29.0% 0.82 ± 6% sched_debug.cfs_rq:/.h_nr_running.avg
1.50 +66.7% 2.50 ± 20% sched_debug.cfs_rq:/.h_nr_running.max
0.50 -100.0% 0.00 sched_debug.cfs_rq:/.h_nr_running.min
0.31 ± 8% +98.8% 0.62 ± 6% sched_debug.cfs_rq:/.h_nr_running.stddev
7171 ± 13% -100.0% 0.00 sched_debug.cfs_rq:/.load.min
7981 ± 16% +288.4% 30998 ± 92% sched_debug.cfs_rq:/.load.stddev
6.67 ± 14% -53.8% 3.08 ± 27% sched_debug.cfs_rq:/.load_avg.min
1885498 -16.5% 1575192 sched_debug.cfs_rq:/.min_vruntime.avg
1954909 -10.1% 1757298 sched_debug.cfs_rq:/.min_vruntime.max
1727273 -21.1% 1363179 ± 3% sched_debug.cfs_rq:/.min_vruntime.min
41645 ± 8% +91.7% 79846 ± 17% sched_debug.cfs_rq:/.min_vruntime.stddev
0.50 -100.0% 0.00 sched_debug.cfs_rq:/.nr_running.min
0.21 ± 4% +67.2% 0.35 ± 6% sched_debug.cfs_rq:/.nr_running.stddev
751.58 ± 3% +18.6% 891.11 ± 5% sched_debug.cfs_rq:/.runnable_avg.avg
1665 ± 9% +42.5% 2373 ± 15% sched_debug.cfs_rq:/.runnable_avg.max
513.17 -68.7% 160.75 ± 20% sched_debug.cfs_rq:/.runnable_avg.min
248.44 ± 12% +96.2% 487.49 ± 11% sched_debug.cfs_rq:/.runnable_avg.stddev
41543 ± 8% +92.3% 79899 ± 17% sched_debug.cfs_rq:/.spread0.stddev
1298 ± 10% +45.8% 1892 ± 14% sched_debug.cfs_rq:/.util_avg.max
453.00 ± 15% -70.4% 134.00 ± 16% sched_debug.cfs_rq:/.util_avg.min
182.62 ± 12% +118.3% 398.65 ± 11% sched_debug.cfs_rq:/.util_avg.stddev
280.79 ± 2% -46.8% 149.49 ± 7% sched_debug.cfs_rq:/.util_est_enqueued.avg
165.17 ± 30% -100.0% 0.00 sched_debug.cfs_rq:/.util_est_enqueued.min
105.37 ± 16% +51.5% 159.68 ± 6% sched_debug.cfs_rq:/.util_est_enqueued.stddev
530405 ± 8% -26.0% 392728 ± 5% sched_debug.cpu.avg_idle.avg
19942 ± 41% -66.1% 6758 ± 5% sched_debug.cpu.avg_idle.min
272109 ± 11% -44.4% 151324 ± 17% sched_debug.cpu.avg_idle.stddev
8.57 ± 13% -38.8% 5.24 ± 6% sched_debug.cpu.clock.stddev
2936 ± 3% +116.4% 6353 ± 2% sched_debug.cpu.curr->pid.avg
5578 +57.2% 8768 sched_debug.cpu.curr->pid.max
1578 ± 30% -100.0% 0.00 sched_debug.cpu.curr->pid.min
1176 ± 6% +125.1% 2649 ± 6% sched_debug.cpu.curr->pid.stddev
0.64 ± 2% +26.6% 0.81 ± 5% sched_debug.cpu.nr_running.avg
1.50 +66.7% 2.50 ± 11% sched_debug.cpu.nr_running.max
0.50 -100.0% 0.00 sched_debug.cpu.nr_running.min
0.32 ± 8% +94.7% 0.62 ± 8% sched_debug.cpu.nr_running.stddev
309449 +44.0% 445760 sched_debug.cpu.nr_switches.avg
502157 ± 2% +27.3% 639259 sched_debug.cpu.nr_switches.max
210908 ± 12% +89.5% 399733 ± 3% sched_debug.cpu.nr_switches.min
42371 ± 8% -28.7% 30230 ± 2% sched_debug.cpu.nr_switches.stddev
54.82 -1.3% 54.13 perf-stat.i.MPKI
5.309e+09 -16.8% 4.418e+09 perf-stat.i.branch-instructions
0.39 ± 2% +0.3 0.70 ± 2% perf-stat.i.branch-miss-rate%
18374324 ± 2% +58.4% 29111083 ± 2% perf-stat.i.branch-misses
57.60 -10.6 47.02 perf-stat.i.cache-miss-rate%
8.211e+08 -34.1% 5.408e+08 perf-stat.i.cache-misses
1.401e+09 -18.8% 1.138e+09 perf-stat.i.cache-references
654272 +45.0% 948647 perf-stat.i.context-switches
7.35 +14.9% 8.45 perf-stat.i.cpi
1.879e+11 -5.5% 1.775e+11 perf-stat.i.cpu-cycles
229.56 ± 3% +66076.8% 151915 perf-stat.i.cpu-migrations
282.44 +31.7% 371.93 ± 2% perf-stat.i.cycles-between-cache-misses
1.08 ± 4% +0.2 1.28 ± 2% perf-stat.i.dTLB-load-miss-rate%
5.363e+09 -11.7% 4.736e+09 perf-stat.i.dTLB-loads
0.14 ± 5% +0.0 0.17 perf-stat.i.dTLB-store-miss-rate%
2.903e+09 -12.1% 2.551e+09 perf-stat.i.dTLB-stores
2.573e+10 -16.4% 2.152e+10 perf-stat.i.instructions
0.17 -8.9% 0.15 perf-stat.i.ipc
2.93 -5.5% 2.77 perf-stat.i.metric.GHz
764.19 ± 20% -82.7% 132.26 ± 12% perf-stat.i.metric.K/sec
245.84 -15.1% 208.78 perf-stat.i.metric.M/sec
7.28 ± 18% +50.4 57.70 ± 6% perf-stat.i.node-load-miss-rate%
22932644 ± 21% +577.3% 1.553e+08 ± 5% perf-stat.i.node-load-misses
3.872e+08 ± 2% -70.8% 1.131e+08 ± 10% perf-stat.i.node-loads
6.12 ± 23% +49.4 55.48 ± 6% perf-stat.i.node-store-miss-rate%
20676280 ± 24% +600.9% 1.449e+08 ± 5% perf-stat.i.node-store-misses
3.782e+08 ± 2% -69.6% 1.15e+08 ± 10% perf-stat.i.node-stores
54.67 -2.7% 53.17 perf-stat.overall.MPKI
0.34 ± 2% +0.3 0.65 ± 2% perf-stat.overall.branch-miss-rate%
58.57 -11.1 47.50 perf-stat.overall.cache-miss-rate%
7.33 +13.2% 8.29 perf-stat.overall.cpi
228.79 +43.6% 328.44 perf-stat.overall.cycles-between-cache-misses
1.08 ± 4% +0.2 1.26 ± 2% perf-stat.overall.dTLB-load-miss-rate%
0.14 ± 5% +0.0 0.17 perf-stat.overall.dTLB-store-miss-rate%
0.14 -11.7% 0.12 perf-stat.overall.ipc
5.60 ± 22% +52.4 58.04 ± 6% perf-stat.overall.node-load-miss-rate%
5.20 ± 25% +50.8 55.97 ± 6% perf-stat.overall.node-store-miss-rate%
5.212e+09 -16.9% 4.331e+09 perf-stat.ps.branch-instructions
17804128 ± 2% +59.3% 28355867 ± 2% perf-stat.ps.branch-misses
8.089e+08 -34.1% 5.328e+08 perf-stat.ps.cache-misses
1.381e+09 -18.8% 1.122e+09 perf-stat.ps.cache-references
644681 +45.1% 935594 perf-stat.ps.context-switches
1.851e+11 -5.5% 1.749e+11 perf-stat.ps.cpu-cycles
223.81 ± 3% +66855.8% 149855 perf-stat.ps.cpu-migrations
5.265e+09 -11.8% 4.642e+09 perf-stat.ps.dTLB-loads
2.852e+09 -12.3% 2.502e+09 perf-stat.ps.dTLB-stores
2.526e+10 -16.5% 2.11e+10 perf-stat.ps.instructions
22590325 ± 21% +579.0% 1.534e+08 ± 5% perf-stat.ps.node-load-misses
3.815e+08 ± 2% -70.9% 1.111e+08 ± 10% perf-stat.ps.node-loads
20384897 ± 24% +602.4% 1.432e+08 ± 5% perf-stat.ps.node-store-misses
3.726e+08 ± 2% -69.7% 1.129e+08 ± 10% perf-stat.ps.node-stores
1.593e+12 -16.2% 1.335e+12 perf-stat.total.instructions
0.02 ±188% +490.1% 0.14 ± 28% perf-sched.sch_delay.avg.ms.__cond_resched.__alloc_pages.__kmalloc_large_node.__kmalloc.iovec_from_user
0.00 ± 71% +368.2% 0.02 ± 4% perf-sched.sch_delay.avg.ms.__cond_resched.__filemap_get_folio.pagecache_get_page.simple_write_begin.generic_perform_write
0.00 ±103% +5050.0% 0.07 ± 5% perf-sched.sch_delay.avg.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc.io_alloc_async_data.io_setup_async_rw
0.00 ±223% +1833.3% 0.03 ± 85% perf-sched.sch_delay.avg.ms.__cond_resched.__wait_for_common.stop_two_cpus.migrate_swap.task_numa_migrate
0.00 ±223% +22500.0% 0.04 ± 71% perf-sched.sch_delay.avg.ms.__cond_resched.change_pmd_range.change_p4d_range.change_protection_range.change_prot_numa
0.00 ±223% +7000.0% 0.06 ± 78% perf-sched.sch_delay.avg.ms.__cond_resched.down_write.generic_file_write_iter.io_write.io_issue_sqe
0.00 ± 72% +240.7% 0.02 ± 3% perf-sched.sch_delay.avg.ms.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.io_write
0.00 ±223% +12316.7% 0.12 ± 32% perf-sched.sch_delay.avg.ms.__cond_resched.io_write.io_issue_sqe.io_wq_submit_work.io_worker_handle_work
0.03 ± 71% +248.4% 0.09 ± 6% perf-sched.sch_delay.avg.ms.__cond_resched.mutex_lock.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 ±141% +1782.4% 0.05 ± 55% perf-sched.sch_delay.avg.ms.__cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate.isra
0.01 ± 76% +354.7% 0.04 ± 32% perf-sched.sch_delay.avg.ms.__cond_resched.task_work_run.io_run_task_work.io_cqring_wait.__do_sys_io_uring_enter
0.00 ± 83% +459.3% 0.03 ± 25% perf-sched.sch_delay.avg.ms.__cond_resched.task_work_run.io_run_task_work.io_wq_worker.ret_from_fork
0.00 ±166% +422.2% 0.01 ± 73% perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt
0.00 ±180% +2452.9% 0.07 ± 44% perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single
0.00 ±157% +2845.0% 0.10 ± 22% perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi
0.01 ± 71% +137.3% 0.02 ± 2% perf-sched.sch_delay.avg.ms.io_cqring_wait.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.01 ± 72% +660.6% 0.04 ± 2% perf-sched.sch_delay.avg.ms.schedule_timeout.io_wq_worker.ret_from_fork.[unknown]
0.01 ± 71% +108.3% 0.02 ± 15% perf-sched.sch_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
0.15 ±207% +1495.9% 2.33 ± 29% perf-sched.sch_delay.max.ms.__cond_resched.__alloc_pages.__kmalloc_large_node.__kmalloc.iovec_from_user
0.10 ±128% +10698.5% 10.74 ± 41% perf-sched.sch_delay.max.ms.__cond_resched.__filemap_get_folio.pagecache_get_page.simple_write_begin.generic_perform_write
0.12 ±184% +3432.9% 4.20 ± 36% perf-sched.sch_delay.max.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc.io_alloc_async_data.io_setup_async_rw
0.00 ±223% +21533.3% 0.32 ±123% perf-sched.sch_delay.max.ms.__cond_resched.__wait_for_common.stop_two_cpus.migrate_swap.task_numa_migrate
0.01 ±103% +4800.0% 0.32 ± 85% perf-sched.sch_delay.max.ms.__cond_resched.change_pmd_range.change_p4d_range.change_protection_range.change_prot_numa
0.00 ±223% +1e+05% 0.84 ± 85% perf-sched.sch_delay.max.ms.__cond_resched.down_write.generic_file_write_iter.io_write.io_issue_sqe
0.29 ±108% +1816.6% 5.62 ± 39% perf-sched.sch_delay.max.ms.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.io_write
1.17 ±168% +519.0% 7.27 ± 69% perf-sched.sch_delay.max.ms.__cond_resched.io_assign_current_work.io_worker_handle_work.io_wq_worker.ret_from_fork
0.00 ±223% +2.4e+05% 2.39 ± 42% perf-sched.sch_delay.max.ms.__cond_resched.io_write.io_issue_sqe.io_wq_submit_work.io_worker_handle_work
0.98 ± 90% +443.2% 5.32 ± 34% perf-sched.sch_delay.max.ms.__cond_resched.mutex_lock.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 ±141% +23558.8% 0.67 ± 56% perf-sched.sch_delay.max.ms.__cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate.isra
0.21 ±129% +292.1% 0.83 ± 36% perf-sched.sch_delay.max.ms.__cond_resched.task_work_run.io_run_task_work.io_cqring_wait.__do_sys_io_uring_enter
0.01 ± 80% +37525.8% 1.94 ± 45% perf-sched.sch_delay.max.ms.__cond_resched.task_work_run.io_run_task_work.io_wq_worker.ret_from_fork
0.11 ±191% +1132.7% 1.33 ± 32% perf-sched.sch_delay.max.ms.__cond_resched.tctx_task_work.task_work_run.io_run_task_work.io_cqring_wait
0.09 ±196% +596.7% 0.63 ± 94% perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt
0.01 ±189% +37034.5% 3.59 ± 77% perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single
0.37 ± 97% +971.5% 3.99 ± 10% perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi
0.01 ±164% +3227.9% 0.24 ±106% perf-sched.sch_delay.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write_killable.do_madvise
0.04 ± 71% +142.1% 0.10 ± 10% perf-sched.sch_delay.max.ms.schedule_timeout.kcompactd.kthread.ret_from_fork
0.01 ± 71% +329.8% 0.03 ± 2% perf-sched.total_sch_delay.average.ms
0.24 ± 70% +300.7% 0.97 perf-sched.total_wait_and_delay.average.ms
2487 ± 71% +72.6% 4293 ± 5% perf-sched.total_wait_and_delay.max.ms
0.23 ± 70% +300.0% 0.94 perf-sched.total_wait_time.average.ms
2487 ± 71% +72.6% 4293 ± 5% perf-sched.total_wait_time.max.ms
0.08 ± 70% +196.5% 0.24 perf-sched.wait_and_delay.avg.ms.io_cqring_wait.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
53.10 ± 73% +146.8% 131.03 ± 25% perf-sched.wait_and_delay.avg.ms.pipe_read.vfs_read.ksys_read.do_syscall_64
0.09 ± 70% +1232.8% 1.15 perf-sched.wait_and_delay.avg.ms.schedule_timeout.io_wq_worker.ret_from_fork.[unknown]
4.87 ± 70% +88.7% 9.20 ± 2% perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
405.33 ±104% +6124.0% 25227 ± 3% perf-sched.wait_and_delay.count.__cond_resched.__filemap_get_folio.pagecache_get_page.simple_write_begin.generic_perform_write
673.83 ± 73% +4455.5% 30696 ± 3% perf-sched.wait_and_delay.count.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.io_write
1.67 ±157% +1e+05% 1673 ± 4% perf-sched.wait_and_delay.count.__cond_resched.io_assign_current_work.io_worker_handle_work.io_wq_worker.ret_from_fork
64.67 ± 70% +68.6% 109.00 perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
89.00 ± 87% +655.8% 672.67 ± 6% perf-sched.wait_and_delay.count.do_task_dead.do_exit.io_wq_worker.ret_from_fork
45.33 ± 70% +69.9% 77.00 perf-sched.wait_and_delay.count.schedule_timeout.__wait_for_common.wait_for_completion_state.kernel_clone
1015874 ± 70% +75.8% 1786051 perf-sched.wait_and_delay.count.schedule_timeout.io_wq_worker.ret_from_fork.[unknown]
366.17 ± 71% +77.1% 648.50 ± 2% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
0.84 ±100% +4269.4% 36.90 ± 15% perf-sched.wait_and_delay.max.ms.__cond_resched.__filemap_get_folio.pagecache_get_page.simple_write_begin.generic_perform_write
1.17 ±168% +3446.2% 41.54 ± 22% perf-sched.wait_and_delay.max.ms.__cond_resched.io_assign_current_work.io_worker_handle_work.io_wq_worker.ret_from_fork
55.54 ± 72% +141.0% 133.87 ± 19% perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
2367 ± 71% +78.3% 4221 ± 5% perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
0.06 ± 94% +894.2% 0.63 ± 10% perf-sched.wait_time.avg.ms.__cond_resched.__alloc_pages.__kmalloc_large_node.__kmalloc.iovec_from_user
0.03 ± 74% +1090.6% 0.38 perf-sched.wait_time.avg.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc.io_alloc_async_data.io_setup_async_rw
0.00 ±223% +3964.3% 0.09 ±166% perf-sched.wait_time.avg.ms.__cond_resched.__vmalloc_area_node.__vmalloc_node_range.alloc_thread_stack_node.dup_task_struct
0.00 ±141% +5678.3% 0.22 ± 27% perf-sched.wait_time.avg.ms.__cond_resched.__wait_for_common.stop_two_cpus.migrate_swap.task_numa_migrate
0.00 ±223% +1318.8% 0.04 ±128% perf-sched.wait_time.avg.ms.__cond_resched.copy_strings.isra.0.do_execveat_common
0.00 ±223% +1160.9% 0.05 ± 96% perf-sched.wait_time.avg.ms.__cond_resched.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.[unknown]
0.00 ±223% +20680.0% 0.17 ±183% perf-sched.wait_time.avg.ms.__cond_resched.down_write.free_pgtables.exit_mmap.__mmput
0.00 ±223% +29607.1% 0.69 ± 17% perf-sched.wait_time.avg.ms.__cond_resched.down_write.generic_file_write_iter.io_write.io_issue_sqe
0.00 ±223% +1200.0% 0.04 ±130% perf-sched.wait_time.avg.ms.__cond_resched.dput.step_into.link_path_walk.part
0.09 ±114% +1394.8% 1.40 ± 8% perf-sched.wait_time.avg.ms.__cond_resched.io_assign_current_work.io_worker_handle_work.io_wq_worker.ret_from_fork
0.00 ±223% +19322.2% 0.87 ± 35% perf-sched.wait_time.avg.ms.__cond_resched.io_write.io_issue_sqe.io_wq_submit_work.io_worker_handle_work
0.03 ±182% +5037.2% 1.77 ±114% perf-sched.wait_time.avg.ms.__cond_resched.mutex_lock.perf_event_ctx_lock_nested.constprop.0
0.00 ±223% +512.5% 0.02 ± 73% perf-sched.wait_time.avg.ms.__cond_resched.mutex_lock_killable.pcpu_alloc.__percpu_counter_init.mm_init
0.00 ±141% +5309.5% 0.19 ± 58% perf-sched.wait_time.avg.ms.__cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate.isra
0.00 ±223% +9055.6% 0.14 ± 77% perf-sched.wait_time.avg.ms.__cond_resched.stop_one_cpu.sched_exec.bprm_execve.part
0.07 ± 75% +228.6% 0.22 ± 22% perf-sched.wait_time.avg.ms.__cond_resched.task_work_run.io_run_task_work.io_cqring_wait.__do_sys_io_uring_enter
0.07 ±113% +311.7% 0.28 ± 14% perf-sched.wait_time.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single
0.10 ± 73% +368.8% 0.47 ± 12% perf-sched.wait_time.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi
0.07 ± 70% +204.5% 0.21 perf-sched.wait_time.avg.ms.io_cqring_wait.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
52.84 ± 73% +148.0% 131.02 ± 25% perf-sched.wait_time.avg.ms.pipe_read.vfs_read.ksys_read.do_syscall_64
0.01 ±153% +9910.9% 0.92 ±183% perf-sched.wait_time.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write_killable.do_madvise
0.08 ± 70% +1268.9% 1.11 perf-sched.wait_time.avg.ms.schedule_timeout.io_wq_worker.ret_from_fork.[unknown]
4.84 ± 70% +89.1% 9.16 ± 2% perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
0.01 ±144% +5547.8% 0.43 ±138% perf-sched.wait_time.max.ms.__cond_resched.__alloc_pages.__folio_alloc.vma_alloc_folio.do_anonymous_page
0.34 ±101% +855.3% 3.24 ± 19% perf-sched.wait_time.max.ms.__cond_resched.__alloc_pages.__kmalloc_large_node.__kmalloc.iovec_from_user
1.06 ± 71% +3383.1% 36.89 ± 15% perf-sched.wait_time.max.ms.__cond_resched.__filemap_get_folio.pagecache_get_page.simple_write_begin.generic_perform_write
0.56 ± 77% +987.5% 6.06 ± 30% perf-sched.wait_time.max.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc.io_alloc_async_data.io_setup_async_rw
0.00 ±223% +5500.0% 0.17 ±186% perf-sched.wait_time.max.ms.__cond_resched.__vmalloc_area_node.__vmalloc_node_range.alloc_thread_stack_node.dup_task_struct
0.00 ±141% +45243.5% 1.74 ± 47% perf-sched.wait_time.max.ms.__cond_resched.__wait_for_common.stop_two_cpus.migrate_swap.task_numa_migrate
0.00 ±223% +5621.1% 0.18 ±177% perf-sched.wait_time.max.ms.__cond_resched.copy_strings.isra.0.do_execveat_common
0.00 ±223% +4717.4% 0.18 ±121% perf-sched.wait_time.max.ms.__cond_resched.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.[unknown]
0.00 ±223% +25600.0% 0.21 ±146% perf-sched.wait_time.max.ms.__cond_resched.down_write.free_pgtables.exit_mmap.__mmput
0.00 ±223% +2.3e+05% 5.41 ± 40% perf-sched.wait_time.max.ms.__cond_resched.down_write.generic_file_write_iter.io_write.io_issue_sqe
0.00 ±223% +3295.0% 0.11 ±168% perf-sched.wait_time.max.ms.__cond_resched.dput.step_into.link_path_walk.part
0.32 ±102% +13069.0% 41.50 ± 22% perf-sched.wait_time.max.ms.__cond_resched.io_assign_current_work.io_worker_handle_work.io_wq_worker.ret_from_fork
0.00 ±223% +2.8e+05% 12.76 ± 80% perf-sched.wait_time.max.ms.__cond_resched.io_write.io_issue_sqe.io_wq_submit_work.io_worker_handle_work
0.06 ±198% +7022.2% 4.23 ± 93% perf-sched.wait_time.max.ms.__cond_resched.mutex_lock.perf_event_ctx_lock_nested.constprop.0
0.00 ±223% +2779.2% 0.12 ±117% perf-sched.wait_time.max.ms.__cond_resched.mutex_lock_killable.pcpu_alloc.__percpu_counter_init.mm_init
0.00 ±141% +34166.7% 1.20 ± 53% perf-sched.wait_time.max.ms.__cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate.isra
0.10 ±214% +1111.3% 1.23 ± 68% perf-sched.wait_time.max.ms.__cond_resched.stop_one_cpu.sched_exec.bprm_execve.part
0.62 ± 74% +146.4% 1.54 ± 24% perf-sched.wait_time.max.ms.__cond_resched.task_work_run.io_run_task_work.io_cqring_wait.__do_sys_io_uring_enter
0.11 ±204% +8137.5% 8.81 ± 52% perf-sched.wait_time.max.ms.__cond_resched.task_work_run.io_run_task_work.io_wq_worker.ret_from_fork
0.75 ±101% +347.2% 3.36 ± 17% perf-sched.wait_time.max.ms.__cond_resched.tctx_task_work.task_work_run.io_run_task_work.io_cqring_wait
0.01 ±153% +1.2e+05% 11.01 ±178% perf-sched.wait_time.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write_killable.do_madvise
55.51 ± 72% +141.1% 133.84 ± 19% perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
2367 ± 71% +78.3% 4221 ± 5% perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
47.63 -47.6 0.00 perf-profile.calltrace.cycles-pp.io_issue_sqe.io_submit_sqes.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
47.25 -47.3 0.00 perf-profile.calltrace.cycles-pp.io_read.io_issue_sqe.io_submit_sqes.__do_sys_io_uring_enter.do_syscall_64
46.60 -46.6 0.00 perf-profile.calltrace.cycles-pp.filemap_read.io_read.io_issue_sqe.io_submit_sqes.__do_sys_io_uring_enter
48.31 -46.6 1.74 perf-profile.calltrace.cycles-pp.io_submit_sqes.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe.syscall
49.01 -45.6 3.42 perf-profile.calltrace.cycles-pp.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe.syscall
49.08 -45.5 3.55 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.syscall
49.12 -45.5 3.61 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.syscall
50.96 -44.5 6.42 perf-profile.calltrace.cycles-pp.syscall
42.90 -42.9 0.00 perf-profile.calltrace.cycles-pp.copy_page_to_iter.filemap_read.io_read.io_issue_sqe.io_submit_sqes
34.63 -4.3 30.37 perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.copy_page_to_iter.filemap_read.io_read
40.60 -4.2 36.42 perf-profile.calltrace.cycles-pp._copy_to_iter.copy_page_to_iter.filemap_read.io_read.io_issue_sqe
25.86 -3.3 22.60 perf-profile.calltrace.cycles-pp.copy_page_from_iter_atomic.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.io_write
45.60 -3.1 42.47 perf-profile.calltrace.cycles-pp.__generic_file_write_iter.generic_file_write_iter.io_write.io_issue_sqe.io_wq_submit_work
45.63 -3.1 42.50 perf-profile.calltrace.cycles-pp.generic_file_write_iter.io_write.io_issue_sqe.io_wq_submit_work.io_worker_handle_work
45.53 -3.1 42.41 perf-profile.calltrace.cycles-pp.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.io_write.io_issue_sqe
45.69 -3.1 42.58 perf-profile.calltrace.cycles-pp.io_write.io_issue_sqe.io_wq_submit_work.io_worker_handle_work.io_wq_worker
25.02 -3.0 21.98 perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter_atomic.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
16.79 -0.9 15.93 perf-profile.calltrace.cycles-pp.fault_in_readable.fault_in_iov_iter_readable.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
1.10 -0.1 1.03 perf-profile.calltrace.cycles-pp.simple_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.io_write
1.06 -0.1 1.00 perf-profile.calltrace.cycles-pp.pagecache_get_page.simple_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
1.01 -0.0 0.96 perf-profile.calltrace.cycles-pp.__filemap_get_folio.pagecache_get_page.simple_write_begin.generic_perform_write.__generic_file_write_iter
0.72 +0.1 0.85 perf-profile.calltrace.cycles-pp.copy_iovec_from_user.syscall
0.64 +0.2 0.81 ± 2% perf-profile.calltrace.cycles-pp.filemap_get_read_batch.filemap_get_pages.filemap_read.io_read.io_issue_sqe
0.74 ± 2% +0.2 0.95 ± 2% perf-profile.calltrace.cycles-pp.folio_unlock.simple_write_end.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
0.94 ± 2% +0.3 1.19 ± 2% perf-profile.calltrace.cycles-pp.simple_write_end.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.io_write
0.00 +0.5 0.52 ± 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.io_wq_activate_free_worker.io_wq_enqueue.io_queue_iowq.io_queue_async
0.00 +0.5 0.53 ± 2% perf-profile.calltrace.cycles-pp.io_queue_iowq.io_submit_sqes.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.6 0.55 ± 2% perf-profile.calltrace.cycles-pp.io_wq_activate_free_worker.io_wq_enqueue.io_queue_iowq.io_queue_async.io_submit_sqes
0.00 +0.6 0.58 ± 2% perf-profile.calltrace.cycles-pp.io_wq_enqueue.io_queue_iowq.io_queue_async.io_submit_sqes.__do_sys_io_uring_enter
0.00 +0.6 0.64 perf-profile.calltrace.cycles-pp.io_queue_iowq.io_queue_async.io_submit_sqes.__do_sys_io_uring_enter.do_syscall_64
0.00 +0.7 0.65 ± 2% perf-profile.calltrace.cycles-pp.io_queue_async.io_submit_sqes.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.7 0.66 ± 4% perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.00 +0.8 0.76 ± 10% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.schedule_timeout.io_wq_worker
1.00 ± 2% +0.8 1.77 perf-profile.calltrace.cycles-pp.__entry_text_start.syscall
0.00 +0.8 0.83 ± 2% perf-profile.calltrace.cycles-pp.filemap_get_pages.filemap_read.io_read.io_issue_sqe.io_wq_submit_work
0.00 +0.9 0.92 ± 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.io_issue_sqe
0.58 +0.9 1.51 ± 2% perf-profile.calltrace.cycles-pp.io_cqring_wait.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe.syscall
0.00 +0.9 0.94 perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.io_issue_sqe.io_wq_submit_work
0.00 +0.9 0.94 ± 5% perf-profile.calltrace.cycles-pp.__schedule.schedule.io_cqring_wait.__do_sys_io_uring_enter.do_syscall_64
0.00 +1.0 0.96 ± 5% perf-profile.calltrace.cycles-pp.schedule.io_cqring_wait.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +1.0 0.99 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.io_issue_sqe.io_wq_submit_work.io_worker_handle_work
0.00 +1.0 1.02 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.io_issue_sqe.io_wq_submit_work.io_worker_handle_work.io_wq_worker
0.60 +1.5 2.06 ± 4% perf-profile.calltrace.cycles-pp.schedule_timeout.io_wq_worker.ret_from_fork
0.00 +1.5 1.51 ± 4% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
0.00 +1.7 1.66 ± 5% perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.io_wq_worker.ret_from_fork
0.00 +1.7 1.71 ± 5% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.io_wq_worker.ret_from_fork
0.00 +2.0 1.96 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
0.00 +2.0 1.98 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
0.00 +2.2 2.21 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.00 +3.4 3.43 ± 4% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.00 +3.4 3.44 ± 4% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.00 +3.4 3.44 ± 4% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
0.00 +3.5 3.50 ± 4% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
0.00 +38.3 38.32 perf-profile.calltrace.cycles-pp.copy_page_to_iter.filemap_read.io_read.io_issue_sqe.io_wq_submit_work
46.29 +39.8 86.07 perf-profile.calltrace.cycles-pp.io_issue_sqe.io_wq_submit_work.io_worker_handle_work.io_wq_worker.ret_from_fork
46.30 +39.8 86.11 perf-profile.calltrace.cycles-pp.io_wq_submit_work.io_worker_handle_work.io_wq_worker.ret_from_fork
46.44 +39.9 86.38 perf-profile.calltrace.cycles-pp.io_worker_handle_work.io_wq_worker.ret_from_fork
47.13 +41.5 88.61 perf-profile.calltrace.cycles-pp.io_wq_worker.ret_from_fork
47.17 +41.5 88.66 perf-profile.calltrace.cycles-pp.ret_from_fork
0.00 +41.8 41.76 perf-profile.calltrace.cycles-pp.filemap_read.io_read.io_issue_sqe.io_wq_submit_work.io_worker_handle_work
0.00 +42.2 42.15 perf-profile.calltrace.cycles-pp.io_read.io_issue_sqe.io_wq_submit_work.io_worker_handle_work.io_wq_worker
48.31 -46.6 1.74 perf-profile.children.cycles-pp.io_submit_sqes
49.02 -45.6 3.42 perf-profile.children.cycles-pp.__do_sys_io_uring_enter
49.21 -45.5 3.73 perf-profile.children.cycles-pp.do_syscall_64
49.25 -45.5 3.79 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
51.00 -44.5 6.47 perf-profile.children.cycles-pp.syscall
93.92 -7.4 86.49 perf-profile.children.cycles-pp.io_issue_sqe
47.26 -4.9 42.35 perf-profile.children.cycles-pp.io_read
46.95 -4.9 42.07 perf-profile.children.cycles-pp.filemap_read
43.23 -4.6 38.60 perf-profile.children.cycles-pp.copy_page_to_iter
34.88 -4.3 30.61 perf-profile.children.cycles-pp.copyout
40.64 -4.2 36.46 perf-profile.children.cycles-pp._copy_to_iter
25.91 -3.3 22.64 perf-profile.children.cycles-pp.copy_page_from_iter_atomic
46.01 -3.2 42.76 perf-profile.children.cycles-pp.io_write
45.59 -3.1 42.45 perf-profile.children.cycles-pp.generic_perform_write
45.66 -3.1 42.52 perf-profile.children.cycles-pp.generic_file_write_iter
45.63 -3.1 42.49 perf-profile.children.cycles-pp.__generic_file_write_iter
25.33 -3.1 22.21 perf-profile.children.cycles-pp.copyin
16.98 -0.9 16.06 perf-profile.children.cycles-pp.fault_in_readable
0.88 ± 23% -0.8 0.10 ± 18% perf-profile.children.cycles-pp.ordered_events__queue
0.88 ± 23% -0.8 0.10 ± 18% perf-profile.children.cycles-pp.queue_event
0.94 ± 21% -0.8 0.18 ± 35% perf-profile.children.cycles-pp.process_simple
0.50 -0.2 0.33 ± 3% perf-profile.children.cycles-pp.__io_import_iovec
0.48 -0.2 0.32 ± 2% perf-profile.children.cycles-pp.__import_iovec
1.12 -0.1 1.04 perf-profile.children.cycles-pp.simple_write_begin
1.07 -0.1 1.01 perf-profile.children.cycles-pp.pagecache_get_page
1.03 -0.1 0.98 perf-profile.children.cycles-pp.__filemap_get_folio
0.25 ± 2% -0.0 0.21 ± 3% perf-profile.children.cycles-pp.xas_load
0.14 ± 2% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.iovec_from_user
0.16 ± 3% -0.0 0.13 ± 2% perf-profile.children.cycles-pp.xas_descend
0.08 ± 4% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.io_madvise
0.08 ± 11% +0.0 0.10 ± 3% perf-profile.children.cycles-pp.__intel_pmu_enable_all
0.08 ± 4% +0.0 0.11 perf-profile.children.cycles-pp.__mod_timer
0.04 ± 44% +0.0 0.07 ± 5% perf-profile.children.cycles-pp.__timer_delete_sync
0.05 +0.0 0.09 ± 8% perf-profile.children.cycles-pp.io_wq_free_work
0.02 ± 99% +0.0 0.06 ± 6% perf-profile.children.cycles-pp.update_rq_clock
0.05 +0.0 0.09 ± 4% perf-profile.children.cycles-pp.lock_timer_base
0.06 +0.0 0.10 ± 4% perf-profile.children.cycles-pp.set_next_entity
0.02 ±141% +0.0 0.06 ± 7% perf-profile.children.cycles-pp.__try_to_del_timer_sync
0.00 +0.1 0.05 perf-profile.children.cycles-pp.update_min_vruntime
0.07 ± 5% +0.1 0.12 ± 4% perf-profile.children.cycles-pp.io_clean_op
0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.call_function_single_prep_ipi
0.01 ±223% +0.1 0.06 ± 7% perf-profile.children.cycles-pp.__io_commit_cqring_flush
0.00 +0.1 0.06 ± 8% perf-profile.children.cycles-pp.asm_sysvec_reschedule_ipi
0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.unmap_vmas
0.00 +0.1 0.06 ± 6% perf-profile.children.cycles-pp._raw_spin_trylock
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.x86_64_start_kernel
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.x86_64_start_reservations
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.start_kernel
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.arch_call_rest_init
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.rest_init
0.00 +0.1 0.06 ± 6% perf-profile.children.cycles-pp.fput
0.00 +0.1 0.06 perf-profile.children.cycles-pp.io_cq_unlock_post
0.00 +0.1 0.06 ± 6% perf-profile.children.cycles-pp.native_sched_clock
0.00 +0.1 0.06 ± 6% perf-profile.children.cycles-pp.sched_clock_cpu
0.01 ±223% +0.1 0.07 perf-profile.children.cycles-pp.io_prep_async_link
0.07 +0.1 0.13 ± 5% perf-profile.children.cycles-pp.__update_load_avg_se
0.09 ± 4% +0.1 0.15 ± 3% perf-profile.children.cycles-pp.__cond_resched
0.00 +0.1 0.07 ± 13% perf-profile.children.cycles-pp.do_exit
0.00 +0.1 0.07 perf-profile.children.cycles-pp.switch_fpu_return
0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.migrate_task_rq_fair
0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.15 ± 8% +0.1 0.23 ± 8% perf-profile.children.cycles-pp.stress_io_uring
0.00 +0.1 0.08 ± 9% perf-profile.children.cycles-pp.__io_req_task_work_add
0.00 +0.1 0.08 ± 8% perf-profile.children.cycles-pp.raw_spin_rq_lock_nested
0.00 +0.1 0.08 ± 4% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.00 +0.1 0.08 ± 5% perf-profile.children.cycles-pp.llist_reverse_order
0.06 ± 7% +0.1 0.15 ± 3% perf-profile.children.cycles-pp.__switch_to
0.00 +0.1 0.08 ± 5% perf-profile.children.cycles-pp.rebalance_domains
0.19 ± 3% +0.1 0.28 ± 2% perf-profile.children.cycles-pp.update_curr
0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.00 +0.1 0.09 ± 5% perf-profile.children.cycles-pp.set_task_cpu
0.11 ± 6% +0.1 0.20 ± 4% perf-profile.children.cycles-pp.io_free_req_tw
0.00 +0.1 0.10 ± 9% perf-profile.children.cycles-pp.select_idle_core
0.00 +0.1 0.10 ± 4% perf-profile.children.cycles-pp.update_blocked_averages
0.00 +0.1 0.10 ± 3% perf-profile.children.cycles-pp.sched_mm_cid_migrate_to
0.00 +0.1 0.10 ± 13% perf-profile.children.cycles-pp.__sysvec_call_function_single
0.06 ± 7% +0.1 0.17 ± 4% perf-profile.children.cycles-pp.__switch_to_asm
0.80 +0.1 0.90 perf-profile.children.cycles-pp.copy_iovec_from_user
0.00 +0.1 0.11 ± 11% perf-profile.children.cycles-pp.detach_tasks
0.00 +0.1 0.11 ± 4% perf-profile.children.cycles-pp.__flush_smp_call_function_queue
0.00 +0.1 0.12 ± 6% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.00 +0.1 0.12 ± 4% perf-profile.children.cycles-pp.__io_req_complete_post
0.00 +0.1 0.12 ± 11% perf-profile.children.cycles-pp.sysvec_call_function_single
0.05 +0.1 0.17 ± 6% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.00 +0.1 0.12 ± 4% perf-profile.children.cycles-pp.task_h_load
0.10 ± 5% +0.1 0.22 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.04 ± 45% +0.1 0.18 ± 2% perf-profile.children.cycles-pp.__irq_exit_rcu
0.00 +0.1 0.14 ± 2% perf-profile.children.cycles-pp.update_rq_clock_task
0.00 +0.1 0.14 ± 4% perf-profile.children.cycles-pp.llist_add_batch
0.00 +0.1 0.14 ± 2% perf-profile.children.cycles-pp.io_wq_worker_running
0.00 +0.1 0.15 ± 12% perf-profile.children.cycles-pp.asm_sysvec_call_function_single
0.00 +0.2 0.16 ± 4% perf-profile.children.cycles-pp.__smp_call_single_queue
0.00 +0.2 0.16 ± 3% perf-profile.children.cycles-pp.__do_softirq
0.17 ± 2% +0.2 0.33 ± 3% perf-profile.children.cycles-pp.tctx_task_work
0.65 ± 2% +0.2 0.82 ± 2% perf-profile.children.cycles-pp.filemap_get_read_batch
0.66 +0.2 0.83 ± 2% perf-profile.children.cycles-pp.filemap_get_pages
0.00 +0.2 0.17 ± 3% perf-profile.children.cycles-pp.switch_mm_irqs_off
0.47 ± 12% +0.2 0.64 ± 3% perf-profile.children.cycles-pp.scheduler_tick
0.04 ± 44% +0.2 0.22 ± 6% perf-profile.children.cycles-pp.___perf_sw_event
0.00 +0.2 0.18 ± 4% perf-profile.children.cycles-pp.menu_select
0.08 ± 6% +0.2 0.26 ± 2% perf-profile.children.cycles-pp.prepare_task_switch
0.04 ± 45% +0.2 0.23 ± 8% perf-profile.children.cycles-pp.task_tick_fair
0.22 ± 2% +0.2 0.42 ± 4% perf-profile.children.cycles-pp.task_work_run
0.52 ± 11% +0.2 0.72 ± 3% perf-profile.children.cycles-pp.update_process_times
0.00 +0.2 0.20 ± 4% perf-profile.children.cycles-pp.select_idle_cpu
0.52 ± 11% +0.2 0.73 ± 3% perf-profile.children.cycles-pp.tick_sched_handle
0.00 +0.2 0.21 ± 4% perf-profile.children.cycles-pp.wake_affine
0.74 ± 2% +0.2 0.95 ± 2% perf-profile.children.cycles-pp.folio_unlock
0.54 ± 11% +0.2 0.75 ± 3% perf-profile.children.cycles-pp.tick_sched_timer
0.00 +0.2 0.22 ± 5% perf-profile.children.cycles-pp.finish_task_switch
0.60 ± 10% +0.2 0.82 ± 2% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.00 +0.2 0.22 ± 3% perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.66 ± 10% +0.2 0.89 ± 2% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.66 ± 10% +0.2 0.89 ± 2% perf-profile.children.cycles-pp.hrtimer_interrupt
0.24 +0.2 0.48 ± 4% perf-profile.children.cycles-pp.io_run_task_work
0.00 +0.3 0.26 ± 4% perf-profile.children.cycles-pp.available_idle_cpu
0.95 +0.3 1.23 ± 2% perf-profile.children.cycles-pp.simple_write_end
0.05 ± 13% +0.3 0.35 ± 3% perf-profile.children.cycles-pp.select_idle_sibling
0.20 ± 3% +0.3 0.52 ± 4% perf-profile.children.cycles-pp.enqueue_entity
0.00 +0.3 0.32 ± 7% perf-profile.children.cycles-pp.intel_idle_irq
0.11 ± 3% +0.3 0.46 ± 2% perf-profile.children.cycles-pp.dequeue_entity
0.73 ± 10% +0.4 1.09 ± 2% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.28 ± 3% +0.4 0.65 ± 2% perf-profile.children.cycles-pp.io_queue_async
0.01 ±223% +0.4 0.39 ± 4% perf-profile.children.cycles-pp.update_cfs_group
0.00 +0.4 0.39 ± 19% perf-profile.children.cycles-pp.update_sg_lb_stats
0.81 ± 9% +0.4 1.22 ± 2% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.00 +0.4 0.44 ± 18% perf-profile.children.cycles-pp.update_sd_lb_stats
0.00 +0.4 0.45 ± 18% perf-profile.children.cycles-pp.find_busiest_group
0.22 ± 2% +0.5 0.68 ± 2% perf-profile.children.cycles-pp.dequeue_task_fair
0.15 ± 3% +0.5 0.61 ± 3% perf-profile.children.cycles-pp._raw_spin_lock
0.00 +0.5 0.48 ± 3% perf-profile.children.cycles-pp.schedule_idle
0.10 ± 11% +0.5 0.61 ± 3% perf-profile.children.cycles-pp.select_task_rq
0.09 ± 12% +0.5 0.61 ± 2% perf-profile.children.cycles-pp.select_task_rq_fair
0.36 +0.5 0.88 ± 3% perf-profile.children.cycles-pp.enqueue_task_fair
0.18 ± 3% +0.5 0.71 ± 4% perf-profile.children.cycles-pp.update_load_avg
0.39 ± 2% +0.5 0.94 perf-profile.children.cycles-pp.autoremove_wake_function
0.44 +0.6 1.02 ± 3% perf-profile.children.cycles-pp.ttwu_do_activate
0.00 +0.6 0.58 ± 4% perf-profile.children.cycles-pp.sched_ttwu_pending
0.40 ± 2% +0.6 0.99 perf-profile.children.cycles-pp.__wake_up_common
0.42 ± 2% +0.6 1.02 perf-profile.children.cycles-pp.__wake_up_common_lock
0.42 ± 2% +0.6 1.03 perf-profile.children.cycles-pp.io_wq_activate_free_worker
0.37 +0.6 1.00 ± 3% perf-profile.children.cycles-pp.activate_task
0.45 +0.6 1.09 perf-profile.children.cycles-pp.io_wq_enqueue
0.50 ± 2% +0.7 1.16 perf-profile.children.cycles-pp.io_queue_iowq
0.00 +0.7 0.67 ± 4% perf-profile.children.cycles-pp.flush_smp_call_function_queue
0.00 +0.7 0.73 ± 15% perf-profile.children.cycles-pp.load_balance
1.01 ± 2% +0.8 1.78 perf-profile.children.cycles-pp.__entry_text_start
0.00 +0.9 0.88 ± 12% perf-profile.children.cycles-pp.newidle_balance
0.59 +0.9 1.52 ± 2% perf-profile.children.cycles-pp.io_cqring_wait
0.17 ± 2% +1.0 1.18 ± 9% perf-profile.children.cycles-pp.pick_next_task_fair
0.79 ± 2% +1.1 1.90 perf-profile.children.cycles-pp.try_to_wake_up
0.61 ± 2% +1.5 2.06 ± 4% perf-profile.children.cycles-pp.schedule_timeout
0.00 +1.5 1.54 ± 4% perf-profile.children.cycles-pp.intel_idle
0.74 +1.9 2.67 ± 5% perf-profile.children.cycles-pp.schedule
0.00 +2.0 2.00 ± 4% perf-profile.children.cycles-pp.cpuidle_enter_state
0.00 +2.0 2.01 ± 4% perf-profile.children.cycles-pp.cpuidle_enter
0.00 +2.2 2.25 ± 4% perf-profile.children.cycles-pp.cpuidle_idle_call
0.71 +2.5 3.17 ± 4% perf-profile.children.cycles-pp.__schedule
0.00 +3.4 3.44 ± 4% perf-profile.children.cycles-pp.start_secondary
0.00 +3.5 3.50 ± 4% perf-profile.children.cycles-pp.do_idle
0.00 +3.5 3.50 ± 4% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
0.00 +3.5 3.50 ± 4% perf-profile.children.cycles-pp.cpu_startup_entry
46.30 +39.8 86.11 perf-profile.children.cycles-pp.io_wq_submit_work
46.45 +39.9 86.38 perf-profile.children.cycles-pp.io_worker_handle_work
47.14 +41.5 88.61 perf-profile.children.cycles-pp.io_wq_worker
47.17 +41.5 88.66 perf-profile.children.cycles-pp.ret_from_fork
34.58 -4.3 30.26 perf-profile.self.cycles-pp.copyout
25.11 -3.2 21.95 perf-profile.self.cycles-pp.copyin
16.80 -1.0 15.84 perf-profile.self.cycles-pp.fault_in_readable
0.86 ± 23% -0.8 0.10 ± 20% perf-profile.self.cycles-pp.queue_event
3.31 -0.5 2.82 perf-profile.self.cycles-pp.filemap_read
2.55 -0.5 2.08 perf-profile.self.cycles-pp.copy_page_to_iter
0.60 -0.2 0.44 ± 2% perf-profile.self.cycles-pp.copy_page_from_iter_atomic
0.34 -0.1 0.21 ± 2% perf-profile.self.cycles-pp.__import_iovec
0.32 ± 13% -0.1 0.26 ± 4% perf-profile.self.cycles-pp.perf_adjust_freq_unthr_context
0.20 -0.1 0.14 ± 3% perf-profile.self.cycles-pp.__filemap_get_folio
0.18 ± 2% -0.0 0.14 ± 2% perf-profile.self.cycles-pp.generic_perform_write
0.06 -0.0 0.03 ± 70% perf-profile.self.cycles-pp.xas_load
0.08 ± 11% +0.0 0.10 ± 3% perf-profile.self.cycles-pp.__intel_pmu_enable_all
0.58 +0.0 0.60 perf-profile.self.cycles-pp.filemap_get_entry
0.05 +0.0 0.08 ± 8% perf-profile.self.cycles-pp.pick_next_task_fair
0.09 ± 4% +0.0 0.12 ± 3% perf-profile.self.cycles-pp.update_curr
0.14 ± 8% +0.0 0.18 ± 9% perf-profile.self.cycles-pp.stress_io_uring
0.06 ± 9% +0.0 0.10 ± 4% perf-profile.self.cycles-pp.io_cqring_wait
0.02 ±141% +0.0 0.06 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.02 ± 99% +0.0 0.07 ± 6% perf-profile.self.cycles-pp.enqueue_entity
0.00 +0.1 0.05 perf-profile.self.cycles-pp.io_wq_activate_free_worker
0.00 +0.1 0.05 perf-profile.self.cycles-pp.__wake_up_common
0.00 +0.1 0.05 ± 7% perf-profile.self.cycles-pp.select_idle_sibling
0.00 +0.1 0.05 ± 7% perf-profile.self.cycles-pp.__fget_light
0.01 ±223% +0.1 0.06 ± 6% perf-profile.self.cycles-pp.__io_commit_cqring_flush
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.call_function_single_prep_ipi
0.00 +0.1 0.06 ± 8% perf-profile.self.cycles-pp.fput
0.00 +0.1 0.06 ± 6% perf-profile.self.cycles-pp._raw_spin_trylock
0.00 +0.1 0.06 ± 6% perf-profile.self.cycles-pp.native_sched_clock
0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.flush_smp_call_function_queue
0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.dequeue_entity
0.00 +0.1 0.06 perf-profile.self.cycles-pp.ttwu_queue_wakelist
0.07 ± 7% +0.1 0.13 ± 6% perf-profile.self.cycles-pp.__update_load_avg_se
0.02 ±141% +0.1 0.08 ± 4% perf-profile.self.cycles-pp.dequeue_task_fair
0.00 +0.1 0.07 ± 5% perf-profile.self.cycles-pp.__io_req_complete_post
0.00 +0.1 0.07 perf-profile.self.cycles-pp.finish_task_switch
0.00 +0.1 0.07 perf-profile.self.cycles-pp.menu_select
0.14 ± 3% +0.1 0.22 ± 5% perf-profile.self.cycles-pp.simple_write_end
0.00 +0.1 0.08 ± 6% perf-profile.self.cycles-pp.io_wq_worker
0.00 +0.1 0.08 ± 8% perf-profile.self.cycles-pp.newidle_balance
0.00 +0.1 0.08 ± 5% perf-profile.self.cycles-pp.llist_reverse_order
0.00 +0.1 0.08 ± 8% perf-profile.self.cycles-pp.prepare_task_switch
0.06 ± 6% +0.1 0.15 ± 4% perf-profile.self.cycles-pp.__switch_to
0.05 +0.1 0.14 ± 2% perf-profile.self.cycles-pp.try_to_wake_up
0.00 +0.1 0.09 ± 6% perf-profile.self.cycles-pp.tctx_task_work
0.00 +0.1 0.10 ± 14% perf-profile.self.cycles-pp.syscall
0.00 +0.1 0.10 ± 3% perf-profile.self.cycles-pp.sched_mm_cid_migrate_to
5.43 +0.1 5.53 perf-profile.self.cycles-pp._copy_to_iter
0.06 ± 7% +0.1 0.17 ± 4% perf-profile.self.cycles-pp.__switch_to_asm
0.79 +0.1 0.89 perf-profile.self.cycles-pp.copy_iovec_from_user
0.00 +0.1 0.11 ± 4% perf-profile.self.cycles-pp.update_rq_clock_task
0.00 +0.1 0.12 ± 6% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.00 +0.1 0.12 ± 4% perf-profile.self.cycles-pp.task_h_load
0.03 ± 70% +0.1 0.16 ± 5% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.08 ± 5% +0.1 0.22 ± 3% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.00 +0.1 0.14 ± 3% perf-profile.self.cycles-pp.llist_add_batch
0.00 +0.1 0.14 perf-profile.self.cycles-pp.io_wq_worker_running
0.06 +0.2 0.21 ± 4% perf-profile.self.cycles-pp.enqueue_task_fair
0.60 +0.2 0.76 ± 2% perf-profile.self.cycles-pp.filemap_get_read_batch
0.00 +0.2 0.17 ± 4% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.01 ±223% +0.2 0.20 ± 7% perf-profile.self.cycles-pp.___perf_sw_event
0.73 ± 2% +0.2 0.94 ± 2% perf-profile.self.cycles-pp.folio_unlock
0.11 ± 4% +0.2 0.34 perf-profile.self.cycles-pp.__schedule
0.00 +0.3 0.26 ± 4% perf-profile.self.cycles-pp.available_idle_cpu
0.00 +0.3 0.28 ± 7% perf-profile.self.cycles-pp.intel_idle_irq
0.06 ± 6% +0.3 0.39 ± 4% perf-profile.self.cycles-pp.update_load_avg
0.00 +0.3 0.34 ± 19% perf-profile.self.cycles-pp.update_sg_lb_stats
0.14 ± 2% +0.4 0.49 ± 3% perf-profile.self.cycles-pp._raw_spin_lock
0.00 +0.4 0.38 ± 4% perf-profile.self.cycles-pp.update_cfs_group
0.99 +0.8 1.77 perf-profile.self.cycles-pp.__entry_text_start
0.43 ± 2% +0.9 1.29 ± 3% perf-profile.self.cycles-pp.fault_in_iov_iter_readable
0.00 +1.5 1.54 ± 4% perf-profile.self.cycles-pp.intel_idle
***************************************************************************************************
lkp-csl-d02: 36 threads 1 sockets Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz (Cascade Lake) with 128G memory
=========================================================================================
class/compiler/cpufreq_governor/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime:
pts/gcc-12/performance/x86_64-rhel-8.3/100%/debian-11.1-x86_64-20220510.cgz/lkp-csl-d02/io-uring/stress-ng/60s
commit:
e9833d8701 ("block: mark bdev files as FMODE_NOWAIT if underlying device supports it")
caec5ebe77 ("io_uring: rely solely on FMODE_NOWAIT")
e9833d8701028b5d caec5ebe77f97d948dcf46f07d6
---------------- ---------------------------
%stddev %change %stddev
\ | \
5308533 ± 5% +47.1% 7808651 cpuidle..usage
0.07 ± 2% +0.0 0.08 ± 4% mpstat.cpu.all.soft%
2.32 +0.4 2.67 ± 2% mpstat.cpu.all.usr%
2803 ± 8% +20.5% 3378 ± 9% perf-c2c.DRAM.local
296.00 ± 13% +37.4% 406.67 ± 12% perf-c2c.HITM.local
36.33 +30.3% 47.33 vmstat.procs.r
330079 +39.5% 460335 vmstat.system.cs
109577 -5.4% 103641 vmstat.system.in
10701593 -1.3% 10560464 stress-ng.io-uring.ops
178275 -1.3% 175926 stress-ng.io-uring.ops_per_sec
1960002 +151.4% 4926804 stress-ng.time.involuntary_context_switches
3091 +1.3% 3130 stress-ng.time.percent_of_cpu_this_job_got
11661888 +31.4% 15325792 stress-ng.time.voluntary_context_switches
4077643 ± 5% +68.5% 6871201 turbostat.C1
605983 ± 2% -65.1% 211732 turbostat.C1E
1.06 -0.7 0.35 turbostat.C1E%
9.73 ± 5% -11.7% 8.59 turbostat.CPU%c1
491190 ± 7% +23.2% 604908 ± 4% turbostat.POLL
1914052 +10.3% 2111343 meminfo.Committed_AS
930287 +23.8% 1151328 ± 2% meminfo.Inactive
930123 +23.8% 1151164 ± 2% meminfo.Inactive(anon)
17477 +20.4% 21036 meminfo.KernelStack
489503 ± 2% +27.9% 626186 meminfo.Mapped
1070375 +17.6% 1258628 meminfo.Shmem
28375 +12.5% 31930 meminfo.VmallocUsed
81701 +2.8% 83976 proc-vmstat.nr_anon_pages
958569 +4.9% 1005343 proc-vmstat.nr_file_pages
231219 +24.5% 287829 ± 2% proc-vmstat.nr_inactive_anon
17489 +20.3% 21036 proc-vmstat.nr_kernel_stack
120756 +29.7% 156655 proc-vmstat.nr_mapped
267914 +17.5% 314689 proc-vmstat.nr_shmem
37175 +2.2% 37996 proc-vmstat.nr_slab_unreclaimable
231219 +24.5% 287829 ± 2% proc-vmstat.nr_zone_inactive_anon
2580741 +1.9% 2629260 proc-vmstat.numa_hit
2578639 +2.0% 2629404 proc-vmstat.numa_local
324060 ± 5% +20.7% 391137 proc-vmstat.pgactivate
305237 +1.8% 310644 proc-vmstat.pgfault
8075240 -0.8% 8009939 proc-vmstat.pgfree
0.76 ± 7% +30.5% 0.99 ± 8% sched_debug.cfs_rq:/.h_nr_running.avg
0.51 ± 17% +41.8% 0.72 ± 12% sched_debug.cfs_rq:/.h_nr_running.stddev
112.29 ± 5% -19.2% 90.79 sched_debug.cfs_rq:/.load_avg.avg
6.83 ± 15% -26.8% 5.00 ± 8% sched_debug.cfs_rq:/.load_avg.min
190.88 ± 4% -11.8% 168.40 sched_debug.cfs_rq:/.load_avg.stddev
94373 ± 2% -27.5% 68398 ± 3% sched_debug.cfs_rq:/.min_vruntime.stddev
52.03 ± 12% -45.9% 28.16 sched_debug.cfs_rq:/.removed.load_avg.avg
154.10 ± 5% -24.7% 116.10 sched_debug.cfs_rq:/.removed.load_avg.stddev
16.01 ± 19% -62.0% 6.09 ± 63% sched_debug.cfs_rq:/.removed.runnable_avg.avg
232.67 ± 15% -37.1% 146.33 ± 62% sched_debug.cfs_rq:/.removed.runnable_avg.max
53.79 ± 17% -50.6% 26.59 ± 62% sched_debug.cfs_rq:/.removed.runnable_avg.stddev
16.01 ± 19% -62.0% 6.09 ± 63% sched_debug.cfs_rq:/.removed.util_avg.avg
232.67 ± 15% -37.2% 146.17 ± 62% sched_debug.cfs_rq:/.removed.util_avg.max
53.78 ± 17% -50.6% 26.57 ± 62% sched_debug.cfs_rq:/.removed.util_avg.stddev
-139143 -66.4% -46818 sched_debug.cfs_rq:/.spread0.avg
15031 ± 61% +589.4% 103628 ± 21% sched_debug.cfs_rq:/.spread0.max
-276846 -19.9% -221798 sched_debug.cfs_rq:/.spread0.min
94452 ± 2% -27.5% 68458 ± 3% sched_debug.cfs_rq:/.spread0.stddev
172.17 ± 23% -30.0% 120.50 ± 7% sched_debug.cfs_rq:/.util_avg.min
544.17 ± 7% +31.2% 714.00 ± 15% sched_debug.cfs_rq:/.util_est_enqueued.max
237021 ± 4% -10.9% 211258 sched_debug.cpu.avg_idle.avg
2.78 ± 5% +29.6% 3.61 ± 10% sched_debug.cpu.clock.stddev
4017 ± 3% +29.7% 5209 ± 2% sched_debug.cpu.curr->pid.avg
6035 +13.2% 6830 sched_debug.cpu.curr->pid.max
0.77 ± 5% +32.9% 1.03 ± 11% sched_debug.cpu.nr_running.avg
0.53 ± 9% +37.2% 0.73 ± 8% sched_debug.cpu.nr_running.stddev
288481 +38.4% 399165 sched_debug.cpu.nr_switches.avg
506142 ± 3% +10.8% 560658 sched_debug.cpu.nr_switches.max
145805 ± 7% +73.9% 253621 sched_debug.cpu.nr_switches.min
95935 ± 4% +10.7% 106211 ± 2% sched_debug.cpu.nr_switches.stddev
56.54 -3.0% 54.84 perf-stat.i.MPKI
2.414e+09 +5.0% 2.534e+09 perf-stat.i.branch-instructions
1.04 +0.1 1.10 perf-stat.i.branch-miss-rate%
26162111 +10.3% 28862610 perf-stat.i.branch-misses
344022 +38.8% 477562 perf-stat.i.context-switches
60948 ± 3% +46.8% 89481 perf-stat.i.cpu-migrations
0.48 -0.0 0.45 ± 3% perf-stat.i.dTLB-load-miss-rate%
20160713 -3.8% 19400778 ± 3% perf-stat.i.dTLB-load-misses
4.188e+09 +2.3% 4.284e+09 perf-stat.i.dTLB-loads
0.51 -0.0 0.49 perf-stat.i.dTLB-store-miss-rate%
3.15e+09 +2.1% 3.215e+09 perf-stat.i.dTLB-stores
64.08 -10.0 54.11 perf-stat.i.iTLB-load-miss-rate%
2679077 +9.0% 2919839 perf-stat.i.iTLB-load-misses
1395424 ± 3% +55.8% 2174067 perf-stat.i.iTLB-loads
1.535e+10 +3.2% 1.583e+10 perf-stat.i.instructions
6665 -3.4% 6438 perf-stat.i.instructions-per-iTLB-miss
0.16 +2.5% 0.17 perf-stat.i.ipc
539.48 +3.0% 555.91 perf-stat.i.metric.K/sec
299.70 +2.5% 307.28 perf-stat.i.metric.M/sec
2813 +4.2% 2931 perf-stat.i.minor-faults
2813 +4.2% 2932 perf-stat.i.page-faults
55.86 -3.2% 54.07 perf-stat.overall.MPKI
1.09 +0.0 1.13 perf-stat.overall.branch-miss-rate%
8.09 -1.7% 7.95 perf-stat.overall.cpi
237.89 +1.1% 240.44 perf-stat.overall.cycles-between-cache-misses
0.48 -0.0 0.45 ± 3% perf-stat.overall.dTLB-load-miss-rate%
0.52 -0.0 0.50 perf-stat.overall.dTLB-store-miss-rate%
65.75 -8.5 57.25 perf-stat.overall.iTLB-load-miss-rate%
5739 -5.3% 5437 perf-stat.overall.instructions-per-iTLB-miss
0.12 +1.7% 0.13 perf-stat.overall.ipc
2.377e+09 +4.6% 2.486e+09 perf-stat.ps.branch-instructions
25873412 +8.4% 28037995 perf-stat.ps.branch-misses
338522 +38.8% 470012 perf-stat.ps.context-switches
1.223e+11 +1.1% 1.236e+11 perf-stat.ps.cpu-cycles
59954 ± 3% +46.8% 87990 perf-stat.ps.cpu-migrations
19842577 -3.6% 19121495 ± 3% perf-stat.ps.dTLB-load-misses
4.123e+09 +2.1% 4.209e+09 perf-stat.ps.dTLB-loads
3.101e+09 +2.0% 3.163e+09 perf-stat.ps.dTLB-stores
2634058 ± 2% +8.5% 2858568 perf-stat.ps.iTLB-load-misses
1372598 ± 3% +55.5% 2134390 perf-stat.ps.iTLB-loads
1.511e+10 +2.8% 1.554e+10 perf-stat.ps.instructions
2758 +3.5% 2855 perf-stat.ps.minor-faults
2758 +3.5% 2856 perf-stat.ps.page-faults
9.489e+11 +2.3% 9.704e+11 perf-stat.total.instructions
0.02 ± 34% -36.7% 0.01 ± 4% perf-sched.sch_delay.avg.ms.__cond_resched.__alloc_pages.__kmalloc_large_node.__kmalloc.iovec_from_user
0.00 ±141% +1200.0% 0.01 ± 66% perf-sched.sch_delay.avg.ms.__cond_resched.__alloc_pages_bulk.__vmalloc_area_node.__vmalloc_node_range.alloc_thread_stack_node
0.02 ± 3% -47.8% 0.01 perf-sched.sch_delay.avg.ms.__cond_resched.__filemap_get_folio.pagecache_get_page.simple_write_begin.generic_perform_write
0.00 ± 10% +50.0% 0.01 perf-sched.sch_delay.avg.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc.security_task_alloc.copy_process
0.00 ± 40% -75.0% 0.00 ±141% perf-sched.sch_delay.avg.ms.__cond_resched.alloc_vmap_area.__get_vm_area_node.__vmalloc_node_range.alloc_thread_stack_node
0.00 ± 12% +45.5% 0.01 ± 8% perf-sched.sch_delay.avg.ms.__cond_resched.cgroup_css_set_fork.cgroup_can_fork.copy_process.create_io_thread
0.08 ± 7% +111.1% 0.18 ± 17% perf-sched.sch_delay.avg.ms.__cond_resched.down_write.generic_file_write_iter.io_write.io_issue_sqe
0.02 ± 87% +710.0% 0.16 ± 53% perf-sched.sch_delay.avg.ms.__cond_resched.down_write_killable.exec_mmap.begin_new_exec.load_elf_binary
0.01 ± 93% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.dput.step_into.open_last_lookups.path_openat
0.01 ± 7% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.filemap_read.io_read.io_issue_sqe.io_submit_sqes
0.02 ± 2% -53.7% 0.01 ± 4% perf-sched.sch_delay.avg.ms.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.io_write
0.02 ± 15% +910.6% 0.16 ± 9% perf-sched.sch_delay.avg.ms.__cond_resched.io_assign_current_work.io_worker_handle_work.io_wq_worker.ret_from_fork
0.08 ± 13% +197.3% 0.22 ± 15% perf-sched.sch_delay.avg.ms.__cond_resched.io_write.io_issue_sqe.io_wq_submit_work.io_worker_handle_work
0.03 ± 4% -47.3% 0.02 ± 25% perf-sched.sch_delay.avg.ms.__cond_resched.mutex_lock.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.01 ±130% +861.5% 0.12 ±100% perf-sched.sch_delay.avg.ms.__cond_resched.stop_one_cpu.sched_exec.bprm_execve.part
0.04 ± 32% -69.2% 0.01 ± 32% perf-sched.sch_delay.avg.ms.__cond_resched.task_work_run.io_run_task_work.io_cqring_wait.__do_sys_io_uring_enter
0.02 ± 29% -63.1% 0.01 ± 71% perf-sched.sch_delay.avg.ms.__cond_resched.tctx_task_work.task_work_run.get_signal.arch_do_signal_or_restart
0.03 ± 8% -47.1% 0.02 ± 22% perf-sched.sch_delay.avg.ms.__cond_resched.tctx_task_work.task_work_run.io_run_task_work.io_cqring_wait
0.26 ± 51% -71.4% 0.08 ±112% perf-sched.sch_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
0.02 +138.6% 0.05 ± 2% perf-sched.sch_delay.avg.ms.do_nanosleep.hrtimer_nanosleep.common_nsleep.__x64_sys_clock_nanosleep
0.03 ± 27% +66.2% 0.04 ± 9% perf-sched.sch_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.02 ± 7% -46.3% 0.01 ± 12% perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
0.47 ± 87% -95.7% 0.02 ±141% perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.ret_from_fork
0.02 -25.9% 0.01 ± 3% perf-sched.sch_delay.avg.ms.io_cqring_wait.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.02 ± 2% +145.8% 0.05 perf-sched.sch_delay.avg.ms.schedule_timeout.io_wq_worker.ret_from_fork.[unknown]
0.02 ± 2% +8.2% 0.02 ± 2% perf-sched.sch_delay.avg.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
0.02 +13.3% 0.02 perf-sched.sch_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
0.18 ± 75% -82.7% 0.03 ± 27% perf-sched.sch_delay.avg.ms.syslog_print.do_syslog.kmsg_read.vfs_read
0.61 ± 53% -91.4% 0.05 ± 13% perf-sched.sch_delay.max.ms.__cond_resched.__alloc_pages.__kmalloc_large_node.__kmalloc.iovec_from_user
0.00 ±141% +600.0% 0.01 ± 61% perf-sched.sch_delay.max.ms.__cond_resched.__alloc_pages_bulk.__vmalloc_area_node.__vmalloc_node_range.alloc_thread_stack_node
2.52 +42.4% 3.59 ± 13% perf-sched.sch_delay.max.ms.__cond_resched.__filemap_get_folio.pagecache_get_page.simple_write_begin.generic_perform_write
0.01 ± 22% -82.4% 0.00 ±141% perf-sched.sch_delay.max.ms.__cond_resched.alloc_vmap_area.__get_vm_area_node.__vmalloc_node_range.alloc_thread_stack_node
1.09 ± 9% +57.3% 1.71 ± 16% perf-sched.sch_delay.max.ms.__cond_resched.down_write.generic_file_write_iter.io_write.io_issue_sqe
0.03 ± 38% +1632.3% 0.57 ±115% perf-sched.sch_delay.max.ms.__cond_resched.down_write_killable.do_madvise.io_madvise.io_issue_sqe
0.12 ±109% +914.7% 1.17 ± 77% perf-sched.sch_delay.max.ms.__cond_resched.down_write_killable.exec_mmap.begin_new_exec.load_elf_binary
0.02 ± 99% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.dput.step_into.open_last_lookups.path_openat
1.59 -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.filemap_read.io_read.io_issue_sqe.io_submit_sqes
2.17 ± 47% +101.3% 4.36 ± 13% perf-sched.sch_delay.max.ms.__cond_resched.io_assign_current_work.io_worker_handle_work.io_wq_worker.ret_from_fork
1.60 ± 20% +33.6% 2.14 ± 13% perf-sched.sch_delay.max.ms.__cond_resched.io_write.io_issue_sqe.io_wq_submit_work.io_worker_handle_work
0.01 ±121% +5431.7% 0.76 ±122% perf-sched.sch_delay.max.ms.__cond_resched.stop_one_cpu.sched_exec.bprm_execve.part
1.08 ± 23% -84.6% 0.17 ± 98% perf-sched.sch_delay.max.ms.__cond_resched.task_work_run.io_run_task_work.io_cqring_wait.__do_sys_io_uring_enter
0.59 ± 65% +224.4% 1.91 perf-sched.sch_delay.max.ms.__cond_resched.task_work_run.io_run_task_work.io_wq_worker.ret_from_fork
0.47 ± 53% -71.3% 0.14 ±117% perf-sched.sch_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
0.34 ± 66% +164.2% 0.89 ± 45% perf-sched.sch_delay.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
3.51 ± 39% -38.6% 2.16 ± 8% perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi
2.25 ± 23% -49.3% 1.14 ± 32% perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
0.72 ± 70% -97.2% 0.02 ±141% perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.ret_from_fork
4.80 ± 10% +267.2% 17.64 ± 47% perf-sched.sch_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
0.33 ± 81% -86.6% 0.04 ± 22% perf-sched.sch_delay.max.ms.syslog_print.do_syslog.kmsg_read.vfs_read
0.02 ± 2% +76.4% 0.03 perf-sched.total_sch_delay.average.ms
1308716 +33.8% 1750883 perf-sched.total_wait_and_delay.count.ms
4.99 ± 27% -67.9% 1.60 ± 3% perf-sched.wait_and_delay.avg.ms.__cond_resched.__alloc_pages.__folio_alloc.vma_alloc_folio.shmem_alloc_folio
0.39 ± 5% -62.2% 0.15 ± 3% perf-sched.wait_and_delay.avg.ms.__cond_resched.__filemap_get_folio.pagecache_get_page.simple_write_begin.generic_perform_write
0.05 ± 4% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.__cond_resched.filemap_read.io_read.io_issue_sqe.io_submit_sqes
0.37 -64.3% 0.13 ± 3% perf-sched.wait_and_delay.avg.ms.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.io_write
3.55 ± 14% -60.6% 1.40 ± 14% perf-sched.wait_and_delay.avg.ms.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.vfs_write
0.13 ± 16% +67.2% 0.22 ± 5% perf-sched.wait_and_delay.avg.ms.do_nanosleep.hrtimer_nanosleep.common_nsleep.__x64_sys_clock_nanosleep
0.78 ± 71% +243.6% 2.69 ± 42% perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
1.01 +13.5% 1.14 perf-sched.wait_and_delay.avg.ms.schedule_timeout.io_wq_worker.ret_from_fork.[unknown]
1.89 ± 2% +81.3% 3.43 ± 5% perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
91.67 ± 7% +124.4% 205.67 ± 5% perf-sched.wait_and_delay.count.__cond_resched.__alloc_pages.__folio_alloc.vma_alloc_folio.shmem_alloc_folio
34911 +15.9% 40453 ± 2% perf-sched.wait_and_delay.count.__cond_resched.__filemap_get_folio.pagecache_get_page.simple_write_begin.generic_perform_write
32972 -100.0% 0.00 perf-sched.wait_and_delay.count.__cond_resched.filemap_read.io_read.io_issue_sqe.io_submit_sqes
41032 +9.7% 45024 ± 3% perf-sched.wait_and_delay.count.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.io_write
283.67 ± 7% +158.3% 732.67 ± 4% perf-sched.wait_and_delay.count.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.vfs_write
181.00 ± 10% +37.8% 249.33 ± 2% perf-sched.wait_and_delay.count.__cond_resched.mutex_lock.perf_poll.do_poll.constprop
57633 -24.2% 43664 perf-sched.wait_and_delay.count.do_nanosleep.hrtimer_nanosleep.common_nsleep.__x64_sys_clock_nanosleep
463.33 +17.0% 542.00 ± 3% perf-sched.wait_and_delay.count.do_task_dead.do_exit.io_wq_worker.ret_from_fork
3110 ± 9% -74.2% 803.00 ± 70% perf-sched.wait_and_delay.count.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi
476298 +30.4% 621321 perf-sched.wait_and_delay.count.io_cqring_wait.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
543784 +64.9% 896929 perf-sched.wait_and_delay.count.schedule_timeout.io_wq_worker.ret_from_fork.[unknown]
107418 -42.9% 61300 ± 4% perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
4.37 ± 13% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.__cond_resched.filemap_read.io_read.io_issue_sqe.io_submit_sqes
13.48 ± 16% +44.0% 19.41 ± 23% perf-sched.wait_and_delay.max.ms.io_cqring_wait.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
248.34 ± 18% +22.5% 304.33 ± 7% perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
4.99 ± 27% -67.9% 1.60 ± 3% perf-sched.wait_time.avg.ms.__cond_resched.__alloc_pages.__folio_alloc.vma_alloc_folio.shmem_alloc_folio
0.00 ±141% +13030.0% 0.44 ±134% perf-sched.wait_time.avg.ms.__cond_resched.__alloc_pages.__get_free_pages.__pollwait.perf_poll
0.05 ± 16% +748.4% 0.43 ± 7% perf-sched.wait_time.avg.ms.__cond_resched.__alloc_pages.__kmalloc_large_node.__kmalloc.iovec_from_user
0.01 ± 36% -83.3% 0.00 ±141% perf-sched.wait_time.avg.ms.__cond_resched.__alloc_pages.allocate_slab.___slab_alloc.constprop
0.37 ± 6% -63.2% 0.14 ± 4% perf-sched.wait_time.avg.ms.__cond_resched.__filemap_get_folio.pagecache_get_page.simple_write_begin.generic_perform_write
0.04 ± 9% +518.0% 0.25 perf-sched.wait_time.avg.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc.io_alloc_async_data.io_setup_async_rw
0.01 ± 35% +63.9% 0.02 ± 34% perf-sched.wait_time.avg.ms.__cond_resched.__kmem_cache_alloc_node.kmalloc_trace.perf_event_mmap_event.perf_event_mmap
0.01 ± 14% -60.0% 0.00 ± 86% perf-sched.wait_time.avg.ms.__cond_resched.alloc_vmap_area.__get_vm_area_node.__vmalloc_node_range.alloc_thread_stack_node
0.00 ± 70% +9200.0% 0.31 ±136% perf-sched.wait_time.avg.ms.__cond_resched.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.rep_movs_alternative
0.50 ± 8% +42.8% 0.72 ± 25% perf-sched.wait_time.avg.ms.__cond_resched.down_write.generic_file_write_iter.io_write.io_issue_sqe
0.01 ± 36% +2067.9% 0.20 ± 77% perf-sched.wait_time.avg.ms.__cond_resched.down_write.mmap_region.do_mmap.vm_mmap_pgoff
0.07 ±107% +667.1% 0.57 ± 60% perf-sched.wait_time.avg.ms.__cond_resched.down_write_killable.do_madvise.io_madvise.io_issue_sqe
0.05 ± 4% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.filemap_read.io_read.io_issue_sqe.io_submit_sqes
0.35 -64.9% 0.12 ± 3% perf-sched.wait_time.avg.ms.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.io_write
3.55 ± 14% -60.6% 1.40 ± 14% perf-sched.wait_time.avg.ms.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.vfs_write
0.71 ± 13% +85.8% 1.32 ± 4% perf-sched.wait_time.avg.ms.__cond_resched.io_assign_current_work.io_worker_handle_work.io_wq_worker.ret_from_fork
0.61 ± 6% -59.2% 0.25 ± 17% perf-sched.wait_time.avg.ms.__cond_resched.task_work_run.io_run_task_work.io_cqring_wait.__do_sys_io_uring_enter
0.32 ± 20% -51.1% 0.15 ± 20% perf-sched.wait_time.avg.ms.__cond_resched.task_work_run.io_run_task_work.io_wq_worker.ret_from_fork
0.80 ± 2% -35.3% 0.52 ± 5% perf-sched.wait_time.avg.ms.__cond_resched.tctx_task_work.task_work_run.io_run_task_work.io_cqring_wait
0.02 ± 50% -55.6% 0.01 ± 20% perf-sched.wait_time.avg.ms.__cond_resched.unmap_vmas.unmap_region.do_vmi_align_munmap.do_vmi_munmap
0.12 ± 18% +54.6% 0.18 ± 6% perf-sched.wait_time.avg.ms.do_nanosleep.hrtimer_nanosleep.common_nsleep.__x64_sys_clock_nanosleep
1.60 ± 90% -69.3% 0.49 ± 4% perf-sched.wait_time.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt
1.06 ± 16% +153.8% 2.68 ± 42% perf-sched.wait_time.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
0.99 +11.0% 1.10 perf-sched.wait_time.avg.ms.schedule_timeout.io_wq_worker.ret_from_fork.[unknown]
1.88 ± 2% +81.8% 3.41 ± 5% perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
0.00 ±141% +25270.0% 0.85 ±137% perf-sched.wait_time.max.ms.__cond_resched.__alloc_pages.__get_free_pages.__pollwait.perf_poll
0.01 ± 46% -85.7% 0.00 ±141% perf-sched.wait_time.max.ms.__cond_resched.__alloc_pages.allocate_slab.___slab_alloc.constprop
1.75 ± 14% -47.4% 0.92 ± 70% perf-sched.wait_time.max.ms.__cond_resched.__kmem_cache_alloc_node.kmalloc_trace.create_io_worker.create_worker_cb
0.02 ± 29% -70.6% 0.01 ±141% perf-sched.wait_time.max.ms.__cond_resched.__kmem_cache_alloc_node.kmalloc_trace.single_open.do_dentry_open
0.01 ± 37% -72.2% 0.00 ± 86% perf-sched.wait_time.max.ms.__cond_resched.alloc_vmap_area.__get_vm_area_node.__vmalloc_node_range.alloc_thread_stack_node
1.70 ± 4% +154.4% 4.34 ± 20% perf-sched.wait_time.max.ms.__cond_resched.down_write.generic_file_write_iter.io_write.io_issue_sqe
1.62 ±131% +378.9% 7.78 ± 55% perf-sched.wait_time.max.ms.__cond_resched.down_write_killable.do_madvise.io_madvise.io_issue_sqe
0.01 ± 35% +240.0% 0.03 ± 40% perf-sched.wait_time.max.ms.__cond_resched.down_write_killable.exec_mmap.begin_new_exec.load_elf_binary
3.65 ± 4% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.filemap_read.io_read.io_issue_sqe.io_submit_sqes
3.11 +16.4% 3.62 ± 10% perf-sched.wait_time.max.ms.__cond_resched.mutex_lock.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.93 ± 20% -29.1% 2.08 ± 24% perf-sched.wait_time.max.ms.__cond_resched.task_work_run.io_run_task_work.io_cqring_wait.__do_sys_io_uring_enter
1.70 ± 45% +262.6% 6.16 ± 42% perf-sched.wait_time.max.ms.__cond_resched.task_work_run.io_run_task_work.io_wq_worker.ret_from_fork
336.28 ±139% -99.5% 1.66 ± 6% perf-sched.wait_time.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt
9.76 ± 14% +42.5% 13.91 ± 9% perf-sched.wait_time.max.ms.io_cqring_wait.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
248.32 ± 18% +22.6% 304.32 ± 7% perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
22.65 -22.6 0.00 perf-profile.calltrace.cycles-pp.io_issue_sqe.io_submit_sqes.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
22.38 -22.4 0.00 perf-profile.calltrace.cycles-pp.io_read.io_issue_sqe.io_submit_sqes.__do_sys_io_uring_enter.do_syscall_64
22.10 -22.1 0.00 perf-profile.calltrace.cycles-pp.filemap_read.io_read.io_issue_sqe.io_submit_sqes.__do_sys_io_uring_enter
23.44 -21.9 1.49 perf-profile.calltrace.cycles-pp.io_submit_sqes.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe.syscall
24.49 -21.9 2.55 perf-profile.calltrace.cycles-pp.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe.syscall
24.59 -21.9 2.66 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.syscall
24.57 -21.9 2.64 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.syscall
25.54 -21.9 3.67 perf-profile.calltrace.cycles-pp.syscall
21.27 -21.3 0.00 perf-profile.calltrace.cycles-pp.copy_page_to_iter.filemap_read.io_read.io_issue_sqe.io_submit_sqes
5.56 -0.8 4.74 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
5.56 -0.8 4.74 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
5.73 -0.8 4.93 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
1.47 ± 2% -0.8 0.71 ± 34% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.intel_idle_irq.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
6.21 -0.7 5.53 ± 2% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
6.21 -0.7 5.54 ± 2% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
6.21 -0.7 5.54 ± 2% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
6.23 -0.6 5.59 ± 2% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
0.64 -0.1 0.58 ± 4% perf-profile.calltrace.cycles-pp.filemap_get_entry.__filemap_get_folio.pagecache_get_page.simple_write_begin.generic_perform_write
0.80 ± 2% -0.1 0.75 ± 3% perf-profile.calltrace.cycles-pp.simple_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.io_write
0.76 ± 2% -0.0 0.72 ± 4% perf-profile.calltrace.cycles-pp.__filemap_get_folio.pagecache_get_page.simple_write_begin.generic_perform_write.__generic_file_write_iter
0.78 -0.0 0.73 ± 3% perf-profile.calltrace.cycles-pp.pagecache_get_page.simple_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
0.58 ± 5% -0.0 0.54 ± 5% perf-profile.calltrace.cycles-pp.filemap_get_read_batch.filemap_get_pages.filemap_read.io_read.io_issue_sqe
3.68 +0.2 3.84 ± 2% perf-profile.calltrace.cycles-pp.intel_idle_irq.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
0.75 ± 4% +0.3 1.06 perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.io_wq_worker.ret_from_fork
0.78 ± 3% +0.3 1.10 perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.io_wq_worker.ret_from_fork
0.97 ± 4% +0.4 1.41 perf-profile.calltrace.cycles-pp.schedule_timeout.io_wq_worker.ret_from_fork
0.00 +0.5 0.52 perf-profile.calltrace.cycles-pp.io_queue_iowq.io_queue_async.io_submit_sqes.__do_sys_io_uring_enter.do_syscall_64
0.00 +0.5 0.53 perf-profile.calltrace.cycles-pp.io_queue_async.io_submit_sqes.__do_sys_io_uring_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.6 0.56 ± 5% perf-profile.calltrace.cycles-pp.filemap_get_pages.filemap_read.io_read.io_issue_sqe.io_wq_submit_work
0.00 +0.7 0.69 perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.io_issue_sqe
0.00 +0.7 0.70 perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.io_issue_sqe.io_wq_submit_work
0.00 +0.7 0.73 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.io_issue_sqe.io_wq_submit_work.io_worker_handle_work
0.00 +0.7 0.74 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.io_issue_sqe.io_wq_submit_work.io_worker_handle_work.io_wq_worker
0.00 +21.2 21.19 perf-profile.calltrace.cycles-pp.copy_page_to_iter.filemap_read.io_read.io_issue_sqe.io_wq_submit_work
0.00 +22.0 21.98 perf-profile.calltrace.cycles-pp.filemap_read.io_read.io_issue_sqe.io_wq_submit_work.io_worker_handle_work
63.41 +22.0 85.44 perf-profile.calltrace.cycles-pp.io_issue_sqe.io_wq_submit_work.io_worker_handle_work.io_wq_worker.ret_from_fork
63.43 +22.0 85.46 perf-profile.calltrace.cycles-pp.io_wq_submit_work.io_worker_handle_work.io_wq_worker.ret_from_fork
0.00 +22.1 22.06 perf-profile.calltrace.cycles-pp.io_read.io_issue_sqe.io_wq_submit_work.io_worker_handle_work.io_wq_worker
63.60 +22.1 85.71 perf-profile.calltrace.cycles-pp.io_worker_handle_work.io_wq_worker.ret_from_fork
64.83 +22.5 87.33 perf-profile.calltrace.cycles-pp.ret_from_fork
64.64 +22.6 87.22 perf-profile.calltrace.cycles-pp.io_wq_worker.ret_from_fork
24.80 -22.0 2.84 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
24.78 -22.0 2.82 perf-profile.children.cycles-pp.do_syscall_64
23.44 -21.9 1.49 perf-profile.children.cycles-pp.io_submit_sqes
24.49 -21.9 2.55 perf-profile.children.cycles-pp.__do_sys_io_uring_enter
25.59 -21.9 3.70 perf-profile.children.cycles-pp.syscall
5.57 -0.8 4.78 ± 2% perf-profile.children.cycles-pp.cpuidle_enter
5.57 -0.8 4.78 ± 2% perf-profile.children.cycles-pp.cpuidle_enter_state
5.74 -0.8 4.97 ± 2% perf-profile.children.cycles-pp.cpuidle_idle_call
0.88 ± 5% -0.7 0.15 perf-profile.children.cycles-pp.intel_idle
6.21 -0.7 5.54 ± 2% perf-profile.children.cycles-pp.start_secondary
6.22 -0.6 5.58 ± 2% perf-profile.children.cycles-pp.do_idle
6.23 -0.6 5.59 ± 2% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
6.23 -0.6 5.59 ± 2% perf-profile.children.cycles-pp.cpu_startup_entry
48.82 -0.4 48.39 perf-profile.children.cycles-pp.rep_movs_alternative
0.74 ± 2% -0.3 0.49 perf-profile.children.cycles-pp.newidle_balance
1.65 ± 2% -0.2 1.40 perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.66 ± 2% -0.2 0.43 perf-profile.children.cycles-pp.load_balance
0.46 ± 4% -0.2 0.23 ± 2% perf-profile.children.cycles-pp.find_busiest_group
0.44 ± 3% -0.2 0.23 perf-profile.children.cycles-pp.update_sd_lb_stats
0.37 ± 4% -0.2 0.19 perf-profile.children.cycles-pp.update_sg_lb_stats
0.83 -0.1 0.69 ± 2% perf-profile.children.cycles-pp.pick_next_task_fair
0.18 ± 5% -0.1 0.11 ± 4% perf-profile.children.cycles-pp.kthread
0.17 ± 2% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.smpboot_thread_fn
0.64 -0.1 0.59 ± 4% perf-profile.children.cycles-pp.filemap_get_entry
0.80 ± 2% -0.1 0.75 ± 3% perf-profile.children.cycles-pp.simple_write_begin
0.59 ± 4% -0.0 0.54 ± 5% perf-profile.children.cycles-pp.filemap_get_read_batch
0.78 -0.0 0.73 ± 3% perf-profile.children.cycles-pp.pagecache_get_page
0.77 -0.0 0.72 ± 3% perf-profile.children.cycles-pp.__filemap_get_folio
0.60 ± 4% -0.0 0.56 ± 5% perf-profile.children.cycles-pp.filemap_get_pages
0.06 ± 7% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.cpu_stop_queue_work
0.11 -0.0 0.08 perf-profile.children.cycles-pp.clock_nanosleep
0.08 -0.0 0.05 perf-profile.children.cycles-pp.do_nanosleep
0.09 ± 5% -0.0 0.06 perf-profile.children.cycles-pp.__x64_sys_clock_nanosleep
0.08 ± 5% -0.0 0.06 ± 8% perf-profile.children.cycles-pp.hrtimer_nanosleep
0.14 ± 6% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.__kmalloc
0.08 ± 5% -0.0 0.06 perf-profile.children.cycles-pp.common_nsleep
0.07 -0.0 0.06 ± 8% perf-profile.children.cycles-pp.update_blocked_averages
0.07 ± 6% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.io_prep_async_link
0.05 +0.0 0.07 ± 14% perf-profile.children.cycles-pp.switch_fpu_return
0.13 +0.0 0.15 ± 3% perf-profile.children.cycles-pp.menu_select
0.07 +0.0 0.09 ± 10% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.06 ± 7% +0.0 0.08 perf-profile.children.cycles-pp.check_preempt_curr
0.05 +0.0 0.07 perf-profile.children.cycles-pp.__smp_call_single_queue
0.07 ± 7% +0.0 0.09 ± 10% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.05 ± 8% +0.0 0.07 ± 6% perf-profile.children.cycles-pp.update_cfs_group
0.12 ± 8% +0.0 0.14 ± 3% perf-profile.children.cycles-pp.__cond_resched
0.05 ± 8% +0.0 0.08 ± 6% perf-profile.children.cycles-pp.__update_load_avg_se
0.06 ± 8% +0.0 0.08 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.05 +0.0 0.07 ± 6% perf-profile.children.cycles-pp.__timer_delete_sync
0.05 +0.0 0.08 ± 6% perf-profile.children.cycles-pp.detach_tasks
0.03 ± 70% +0.0 0.06 perf-profile.children.cycles-pp.update_rq_clock_task
0.07 ± 7% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.__switch_to
0.07 ± 7% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.lock_timer_base
0.07 +0.0 0.10 perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.13 ± 3% +0.0 0.16 ± 5% perf-profile.children.cycles-pp.___perf_sw_event
0.03 ± 70% +0.0 0.07 ± 7% perf-profile.children.cycles-pp.llist_add_batch
0.03 ± 70% +0.0 0.07 perf-profile.children.cycles-pp.__try_to_del_timer_sync
0.03 ± 70% +0.0 0.07 perf-profile.children.cycles-pp.set_next_entity
0.10 ± 4% +0.0 0.14 perf-profile.children.cycles-pp.select_idle_cpu
0.10 +0.0 0.14 perf-profile.children.cycles-pp.switch_mm_irqs_off
0.02 ±141% +0.0 0.06 ± 8% perf-profile.children.cycles-pp.io_wq_free_work
0.04 ± 71% +0.0 0.08 ± 6% perf-profile.children.cycles-pp.sched_mm_cid_migrate_to
0.02 ±141% +0.0 0.06 perf-profile.children.cycles-pp.reweight_entity
0.10 ± 4% +0.0 0.14 ± 3% perf-profile.children.cycles-pp.__mod_timer
0.05 +0.0 0.10 ± 4% perf-profile.children.cycles-pp.task_h_load
0.08 ± 10% +0.0 0.13 ± 3% perf-profile.children.cycles-pp.available_idle_cpu
0.02 ±141% +0.1 0.07 ± 7% perf-profile.children.cycles-pp.enqueue_timer
0.00 +0.1 0.05 perf-profile.children.cycles-pp.put_prev_entity
0.09 ± 5% +0.1 0.14 ± 3% perf-profile.children.cycles-pp.prepare_task_switch
0.18 ± 4% +0.1 0.23 ± 2% perf-profile.children.cycles-pp.dequeue_entity
0.10 ± 9% +0.1 0.15 ± 5% perf-profile.children.cycles-pp.__switch_to_asm
0.08 ± 6% +0.1 0.13 ± 3% perf-profile.children.cycles-pp.wake_affine
0.14 ± 6% +0.1 0.20 ± 2% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
0.16 ± 2% +0.1 0.23 ± 2% perf-profile.children.cycles-pp.select_idle_sibling
0.00 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.io_wq_worker_running
0.14 ± 5% +0.1 0.21 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.18 ± 2% +0.1 0.26 ± 6% perf-profile.children.cycles-pp.update_curr
0.19 +0.1 0.26 ± 6% perf-profile.children.cycles-pp.sched_ttwu_pending
0.10 ± 4% +0.1 0.17 ± 2% perf-profile.children.cycles-pp.io_clean_op
0.14 ± 3% +0.1 0.22 ± 2% perf-profile.children.cycles-pp.io_free_req_tw
0.26 +0.1 0.35 ± 2% perf-profile.children.cycles-pp._raw_spin_lock
0.25 +0.1 0.33 ± 2% perf-profile.children.cycles-pp.update_load_avg
0.26 +0.1 0.35 ± 2% perf-profile.children.cycles-pp.dequeue_task_fair
0.20 +0.1 0.29 perf-profile.children.cycles-pp.enqueue_entity
1.56 ± 2% +0.1 1.65 perf-profile.children.cycles-pp.schedule
0.20 ± 2% +0.1 0.30 ± 5% perf-profile.children.cycles-pp.flush_smp_call_function_queue
1.83 +0.1 1.93 perf-profile.children.cycles-pp.__schedule
0.21 ± 2% +0.1 0.32 ± 2% perf-profile.children.cycles-pp.tctx_task_work
0.28 +0.1 0.40 perf-profile.children.cycles-pp.io_run_task_work
0.26 +0.1 0.38 perf-profile.children.cycles-pp.task_work_run
0.31 +0.1 0.44 ± 2% perf-profile.children.cycles-pp.enqueue_task_fair
0.28 ± 2% +0.2 0.43 perf-profile.children.cycles-pp.select_task_rq_fair
0.37 +0.2 0.53 ± 3% perf-profile.children.cycles-pp.activate_task
0.32 ± 3% +0.2 0.48 perf-profile.children.cycles-pp.select_task_rq
0.40 +0.2 0.58 ± 2% perf-profile.children.cycles-pp.ttwu_do_activate
0.34 ± 3% +0.2 0.53 perf-profile.children.cycles-pp.io_queue_async
0.44 ± 3% +0.3 0.70 perf-profile.children.cycles-pp.autoremove_wake_function
0.46 ± 3% +0.3 0.73 perf-profile.children.cycles-pp.__wake_up_common
0.48 +0.3 0.74 perf-profile.children.cycles-pp.__wake_up_common_lock
0.55 ± 3% +0.3 0.85 perf-profile.children.cycles-pp.io_wq_enqueue
0.50 ± 3% +0.3 0.82 perf-profile.children.cycles-pp.io_wq_activate_free_worker
0.62 ± 3% +0.3 0.94 perf-profile.children.cycles-pp.io_queue_iowq
0.98 ± 4% +0.4 1.42 perf-profile.children.cycles-pp.schedule_timeout
1.00 ± 2% +0.5 1.53 perf-profile.children.cycles-pp.try_to_wake_up
63.43 +22.0 85.46 perf-profile.children.cycles-pp.io_wq_submit_work
63.60 +22.1 85.71 perf-profile.children.cycles-pp.io_worker_handle_work
64.83 +22.5 87.33 perf-profile.children.cycles-pp.ret_from_fork
64.64 +22.6 87.22 perf-profile.children.cycles-pp.io_wq_worker
0.88 ± 5% -0.7 0.15 perf-profile.self.cycles-pp.intel_idle
48.22 -0.4 47.81 perf-profile.self.cycles-pp.rep_movs_alternative
0.31 ± 6% -0.2 0.15 ± 3% perf-profile.self.cycles-pp.update_sg_lb_stats
0.46 ± 3% -0.1 0.40 ± 7% perf-profile.self.cycles-pp.filemap_get_entry
0.52 ± 5% -0.0 0.48 ± 6% perf-profile.self.cycles-pp.filemap_get_read_batch
0.51 -0.0 0.49 perf-profile.self.cycles-pp.copyout
0.25 ± 3% -0.0 0.23 ± 3% perf-profile.self.cycles-pp.__ordered_events__flush
0.07 ± 7% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.ordered_events__queue
0.06 +0.0 0.07 ± 6% perf-profile.self.cycles-pp.io_prep_async_link
0.05 ± 8% +0.0 0.07 ± 11% perf-profile.self.cycles-pp.enqueue_task_fair
0.10 ± 4% +0.0 0.12 ± 4% perf-profile.self.cycles-pp.update_curr
0.05 +0.0 0.07 perf-profile.self.cycles-pp.update_cfs_group
0.07 ± 7% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.__switch_to
0.06 ± 8% +0.0 0.08 ± 6% perf-profile.self.cycles-pp.tctx_task_work
0.06 ± 8% +0.0 0.08 ± 6% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.05 +0.0 0.07 ± 6% perf-profile.self.cycles-pp.__update_load_avg_se
0.03 ± 70% +0.0 0.06 perf-profile.self.cycles-pp.finish_task_switch
0.11 ± 4% +0.0 0.14 ± 6% perf-profile.self.cycles-pp.___perf_sw_event
0.03 ± 70% +0.0 0.07 ± 7% perf-profile.self.cycles-pp.llist_add_batch
0.03 ± 70% +0.0 0.07 ± 7% perf-profile.self.cycles-pp.select_task_rq_fair
0.10 +0.0 0.14 perf-profile.self.cycles-pp.switch_mm_irqs_off
0.04 ± 71% +0.0 0.08 ± 6% perf-profile.self.cycles-pp.sched_mm_cid_migrate_to
0.07 ± 14% +0.0 0.11 perf-profile.self.cycles-pp.perf_trace_sched_wakeup_template
0.13 +0.0 0.18 ± 2% perf-profile.self.cycles-pp.update_load_avg
0.08 ± 6% +0.0 0.12 ± 3% perf-profile.self.cycles-pp.available_idle_cpu
0.05 +0.0 0.10 ± 4% perf-profile.self.cycles-pp.task_h_load
0.02 ±141% +0.1 0.07 ± 7% perf-profile.self.cycles-pp.enqueue_timer
0.00 +0.1 0.05 perf-profile.self.cycles-pp.pick_next_task_fair
0.10 ± 9% +0.1 0.15 ± 5% perf-profile.self.cycles-pp.__switch_to_asm
0.21 ± 2% +0.1 0.26 ± 3% perf-profile.self.cycles-pp._raw_spin_lock
0.00 +0.1 0.06 ± 8% perf-profile.self.cycles-pp.prepare_task_switch
0.08 ± 10% +0.1 0.14 ± 3% perf-profile.self.cycles-pp.try_to_wake_up
0.14 ± 3% +0.1 0.21 ± 6% perf-profile.self.cycles-pp.__schedule
0.00 +0.1 0.07 ± 7% perf-profile.self.cycles-pp.io_wq_worker_running
0.14 ± 5% +0.1 0.21 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp.enqueue_entity
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
View attachment "config-6.4.0-rc2-00003-gcaec5ebe77f9" of type "text/plain" (158740 bytes)
View attachment "job-script" of type "text/plain" (9147 bytes)
View attachment "job.yaml" of type "text/plain" (6561 bytes)
View attachment "reproduce" of type "text/plain" (341 bytes)
Powered by blists - more mailing lists