| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Wed, 14 Apr 2021 22:08:05 +0800
From: kernel test robot <oliver.sang@...el.com>
To: Christoph Hellwig <hch@....de>
Cc: Jens Axboe <axboe@...nel.dk>, LKML <linux-kernel@...r.kernel.org>,
Linux Memory Management List <linux-mm@...ck.org>,
lkp@...ts.01.org, lkp@...el.com, ying.huang@...el.com,
feng.tang@...el.com, zhengjun.xing@...el.com
Subject: [block] c76f48eb5c: stress-ng.loop.ops_per_sec -99.9% regression
Greeting,
FYI, we noticed a -99.9% regression of stress-ng.loop.ops_per_sec due to commit:
commit: c76f48eb5c084b1e15c931ae8cc1826cd771d70d ("block: take bd_mutex around delete_partitions in del_gendisk")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
in testcase: stress-ng
on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 192G memory
with following parameters:
nr_threads: 10%
disk: 1HDD
testtime: 60s
fs: ext4
class: os
test: loop
cpufreq_governor: performance
ucode: 0x5003006
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang@...el.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml
bin/lkp run compatible-job.yaml
=========================================================================================
class/compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode:
os/gcc-9/performance/1HDD/ext4/x86_64-rhel-8.3/10%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp5/loop/stress-ng/60s/0x5003006
commit:
d3c4a43d92 ("block: refactor blk_drop_partitions")
c76f48eb5c ("block: take bd_mutex around delete_partitions in del_gendisk")
d3c4a43d9291279c c76f48eb5c084b1e15c931ae8cc
---------------- ---------------------------
%stddev %change %stddev
\ | \
1464 -99.2% 12.00 ± 33% stress-ng.loop.ops
24.35 -99.9% 0.03 ± 41% stress-ng.loop.ops_per_sec
62.28 +789.4% 553.94 ± 13% stress-ng.time.elapsed_time
62.28 +789.4% 553.94 ± 13% stress-ng.time.elapsed_time.max
3671 -99.1% 32.20 ± 27% stress-ng.time.involuntary_context_switches
9792 -1.6% 9636 stress-ng.time.minor_page_faults
36.17 -100.0% 0.00 stress-ng.time.percent_of_cpu_this_job_got
22.65 ± 2% -98.9% 0.26 ± 35% stress-ng.time.system_time
59875 ± 14% -98.6% 851.60 ±110% stress-ng.time.voluntary_context_switches
2189 ± 2% -2.9% 2125 boot-time.idle
102.01 +482.0% 593.67 ± 12% uptime.boot
9211 +501.5% 55405 ± 11% uptime.idle
0.60 ± 3% -0.6 0.02 ± 12% mpstat.cpu.all.iowait%
1.27 +0.6 1.86 ± 27% mpstat.cpu.all.irq%
0.59 -0.6 0.03 ± 2% mpstat.cpu.all.sys%
0.07 ± 2% -0.1 0.01 ± 2% mpstat.cpu.all.usr%
179096 ± 16% +272.4% 666980 ± 17% numa-numastat.node0.local_node
224078 ± 5% +220.9% 718975 ± 19% numa-numastat.node0.numa_hit
155998 ± 18% +281.6% 595329 ± 33% numa-numastat.node1.local_node
199503 ± 4% +215.8% 630069 ± 31% numa-numastat.node1.numa_hit
1598 ± 3% -99.9% 1.20 ±122% vmstat.io.bi
5044 -66.3% 1700 ± 2% vmstat.memory.buff
1.00 -100.0% 0.00 vmstat.procs.r
8439 ± 4% -83.8% 1367 ± 6% vmstat.system.cs
5636242 -49.3% 2860359 ± 6% cpuidle.C1.time
60648 ± 2% -25.4% 45246 ± 7% cpuidle.C1.usage
5.204e+09 ± 22% +607.9% 3.684e+10 ± 41% cpuidle.C1E.time
11235694 ± 14% +684.5% 88145411 ± 16% cpuidle.C1E.usage
41714 ± 6% +162.2% 109376 ± 11% cpuidle.POLL.usage
9288 +58.8% 14748 ± 4% meminfo.Active
3197 +296.2% 12670 ± 5% meminfo.Active(anon)
6089 -65.9% 2076 ± 5% meminfo.Active(file)
86058 ± 5% +107.8% 178815 meminfo.AnonHugePages
4981 -65.9% 1698 ± 2% meminfo.Buffers
825613 -12.4% 723628 ± 3% meminfo.Committed_AS
16237013 ± 3% -11.0% 14454104 ± 8% meminfo.DirectMap2M
18587 -12.6% 16252 meminfo.KernelStack
2850592 -17.0% 2366313 meminfo.Memused
7.00 +26105.7% 1834 ± 24% meminfo.Mlocked
5522 -14.7% 4711 meminfo.PageTables
23075 +42.7% 32929 ± 2% meminfo.Shmem
915.33 ± 33% -73.1% 245.80 ± 23% numa-vmstat.node0.nr_active_file
40463 ± 35% -39.9% 24330 ± 69% numa-vmstat.node0.nr_anon_pages
42808 ± 31% -36.5% 27171 ± 57% numa-vmstat.node0.nr_inactive_anon
0.00 +2.5e+104% 247.20 ± 26% numa-vmstat.node0.nr_mlock
797.33 ± 28% -31.1% 549.60 ± 24% numa-vmstat.node0.nr_page_table_pages
12722 ± 3% -13.8% 10971 ± 9% numa-vmstat.node0.nr_slab_reclaimable
27196 ± 8% -13.5% 23535 ± 8% numa-vmstat.node0.nr_slab_unreclaimable
915.33 ± 33% -73.1% 245.80 ± 23% numa-vmstat.node0.nr_zone_active_file
42808 ± 31% -36.5% 27171 ± 57% numa-vmstat.node0.nr_zone_inactive_anon
849837 ± 15% +26.7% 1076564 ± 9% numa-vmstat.node0.numa_local
457.67 ± 20% +399.8% 2287 ± 17% numa-vmstat.node1.nr_active_anon
8770 ± 8% -15.2% 7437 ± 3% numa-vmstat.node1.nr_kernel_stack
0.00 +2.1e+104% 210.40 ± 29% numa-vmstat.node1.nr_mlock
457.67 ± 20% +399.8% 2287 ± 17% numa-vmstat.node1.nr_zone_active_anon
3665 ± 33% -73.1% 985.00 ± 23% numa-meminfo.node0.Active(file)
161847 ± 35% -39.9% 97309 ± 69% numa-meminfo.node0.AnonPages
171672 ± 31% -36.4% 109247 ± 57% numa-meminfo.node0.Inactive
171227 ± 31% -36.5% 108673 ± 57% numa-meminfo.node0.Inactive(anon)
50893 ± 3% -13.8% 43888 ± 9% numa-meminfo.node0.KReclaimable
1526012 ± 4% -26.1% 1127392 ± 12% numa-meminfo.node0.MemUsed
3.00 +32893.3% 989.80 ± 26% numa-meminfo.node0.Mlocked
3195 ± 28% -31.4% 2193 ± 24% numa-meminfo.node0.PageTables
50893 ± 3% -13.8% 43888 ± 9% numa-meminfo.node0.SReclaimable
108773 ± 8% -13.5% 94141 ± 8% numa-meminfo.node0.SUnreclaim
159668 ± 6% -13.6% 138030 ± 6% numa-meminfo.node0.Slab
4255 ± 29% +140.8% 10246 ± 14% numa-meminfo.node1.Active
1829 ± 20% +400.4% 9154 ± 17% numa-meminfo.node1.Active(anon)
8775 ± 8% -15.2% 7439 ± 3% numa-meminfo.node1.KernelStack
3.00 +28053.3% 844.60 ± 29% numa-meminfo.node1.Mlocked
799.67 +296.1% 3167 ± 5% proc-vmstat.nr_active_anon
1522 -65.9% 518.80 ± 5% proc-vmstat.nr_active_file
63795 -8.3% 58521 proc-vmstat.nr_anon_pages
87.00 ± 28% +78.9% 155.60 ± 28% proc-vmstat.nr_dirtied
68723 -7.6% 63478 proc-vmstat.nr_inactive_anon
18586 -12.6% 16247 proc-vmstat.nr_kernel_stack
10029 -4.4% 9587 proc-vmstat.nr_mapped
1.00 +45720.0% 458.20 ± 24% proc-vmstat.nr_mlock
1380 -14.9% 1174 proc-vmstat.nr_page_table_pages
5765 +42.8% 8231 ± 2% proc-vmstat.nr_shmem
24003 -5.8% 22600 proc-vmstat.nr_slab_reclaimable
49519 -7.8% 45650 proc-vmstat.nr_slab_unreclaimable
77.50 ± 31% +95.1% 151.20 ± 29% proc-vmstat.nr_written
799.67 +296.1% 3167 ± 5% proc-vmstat.nr_zone_active_anon
1522 -65.9% 518.80 ± 5% proc-vmstat.nr_zone_active_file
68723 -7.6% 63478 proc-vmstat.nr_zone_inactive_anon
971.50 ± 30% -58.3% 405.00 ± 51% proc-vmstat.numa_hint_faults
457086 +202.7% 1383638 ± 10% proc-vmstat.numa_hit
368542 +251.9% 1296853 ± 11% proc-vmstat.numa_local
88543 -2.0% 86784 proc-vmstat.numa_other
9561 ± 2% -60.7% 3757 ± 5% proc-vmstat.pgactivate
553335 +163.0% 1455536 ± 10% proc-vmstat.pgalloc_normal
220882 +623.5% 1598121 ± 12% proc-vmstat.pgfault
415665 +247.1% 1442871 ± 10% proc-vmstat.pgfree
104075 ± 3% -99.1% 916.80 ± 69% proc-vmstat.pgpgin
451476 ± 7% +734.6% 3767973 ± 15% proc-vmstat.pgpgout
14210 +649.7% 106533 ± 12% proc-vmstat.pgreuse
8304 ± 82% +301.2% 33317 ± 9% sched_debug.cfs_rq:/.load.avg
46081 ±116% +188.8% 133105 ± 7% sched_debug.cfs_rq:/.load.stddev
178.19 ± 31% -73.7% 46.80 ± 10% sched_debug.cfs_rq:/.load_avg.avg
6121 ± 14% -76.4% 1442 ± 11% sched_debug.cfs_rq:/.load_avg.max
751.44 ± 17% -72.0% 210.71 ± 9% sched_debug.cfs_rq:/.load_avg.stddev
0.17 ± 17% -58.6% 0.07 ± 8% sched_debug.cfs_rq:/.nr_running.avg
0.37 ± 7% -36.2% 0.24 ± 3% sched_debug.cfs_rq:/.nr_running.stddev
393.19 ± 8% -83.9% 63.44 ± 6% sched_debug.cfs_rq:/.runnable_avg.avg
1333 ± 17% -51.9% 641.22 ± 3% sched_debug.cfs_rq:/.runnable_avg.max
317.55 ± 10% -61.0% 123.72 ± 4% sched_debug.cfs_rq:/.runnable_avg.stddev
392.44 ± 8% -83.9% 63.32 ± 6% sched_debug.cfs_rq:/.util_avg.avg
1330 ± 17% -51.9% 640.31 ± 3% sched_debug.cfs_rq:/.util_avg.max
316.96 ± 10% -61.0% 123.61 ± 4% sched_debug.cfs_rq:/.util_avg.stddev
37.95 ± 33% -86.6% 5.08 ± 34% sched_debug.cfs_rq:/.util_est_enqueued.avg
518.00 ± 5% -79.3% 107.10 ± 25% sched_debug.cfs_rq:/.util_est_enqueued.max
121.83 ± 18% -83.3% 20.33 ± 23% sched_debug.cfs_rq:/.util_est_enqueued.stddev
740981 ± 2% +26.7% 938799 sched_debug.cpu.avg_idle.avg
1996 ± 23% +6329.5% 128386 ± 56% sched_debug.cpu.avg_idle.min
306945 ± 5% -53.4% 143028 ± 13% sched_debug.cpu.avg_idle.stddev
38618 +637.1% 284649 ± 15% sched_debug.cpu.clock.avg
38623 +637.0% 284653 ± 15% sched_debug.cpu.clock.max
38613 +637.2% 284645 ± 15% sched_debug.cpu.clock.min
38490 +626.0% 279434 ± 14% sched_debug.cpu.clock_task.avg
38609 +624.6% 279743 ± 14% sched_debug.cpu.clock_task.max
33619 +716.4% 274485 ± 15% sched_debug.cpu.clock_task.min
286.92 ± 19% -51.0% 140.52 ± 7% sched_debug.cpu.curr->pid.avg
2432 +254.8% 8630 ± 12% sched_debug.cpu.curr->pid.max
755.23 ± 8% +28.1% 967.57 ± 10% sched_debug.cpu.curr->pid.stddev
897805 ± 31% -40.4% 535075 sched_debug.cpu.max_idle_balance_cost.max
55398 ± 60% -89.2% 5981 ± 39% sched_debug.cpu.max_idle_balance_cost.stddev
0.00 ± 13% -48.9% 0.00 ± 8% sched_debug.cpu.next_balance.stddev
0.13 ± 18% -73.1% 0.03 ± 11% sched_debug.cpu.nr_running.avg
0.33 ± 7% -49.8% 0.17 ± 3% sched_debug.cpu.nr_running.stddev
1577 +225.4% 5133 ± 10% sched_debug.cpu.nr_switches.avg
10235 ± 13% +526.1% 64083 ± 22% sched_debug.cpu.nr_switches.max
471.33 ± 17% +200.5% 1416 ± 16% sched_debug.cpu.nr_switches.min
1438 ± 6% +476.3% 8287 ± 14% sched_debug.cpu.nr_switches.stddev
0.01 ± 31% +854.0% 0.12 ± 4% sched_debug.cpu.nr_uninterruptible.avg
38615 +637.1% 284646 ± 15% sched_debug.cpu_clk
38121 +645.4% 284151 ± 15% sched_debug.ktime
38979 +631.2% 285008 ± 15% sched_debug.sched_clk
11.12 ± 2% +175.9% 30.68 ±104% perf-stat.i.MPKI
3.418e+08 -49.6% 1.723e+08 ± 7% perf-stat.i.branch-instructions
1.34 ± 4% +2.2 3.49 ±107% perf-stat.i.branch-miss-rate%
8485 ± 4% -84.2% 1344 ± 6% perf-stat.i.context-switches
3.44 ± 3% +52.6% 5.25 ± 28% perf-stat.i.cpi
160.47 -39.2% 97.62 perf-stat.i.cpu-migrations
4.201e+08 -46.4% 2.254e+08 ± 3% perf-stat.i.dTLB-loads
2.097e+08 -41.9% 1.217e+08 ± 2% perf-stat.i.dTLB-stores
2039329 -16.7% 1698838 ± 21% perf-stat.i.iTLB-loads
1.625e+09 -49.1% 8.277e+08 ± 7% perf-stat.i.instructions
1029 ± 4% -41.2% 605.47 ± 22% perf-stat.i.instructions-per-iTLB-miss
0.30 ± 3% -31.8% 0.21 ± 22% perf-stat.i.ipc
11.18 -85.5% 1.62 ± 20% perf-stat.i.major-faults
1.64 ± 9% -40.8% 0.97 ± 17% perf-stat.i.metric.K/sec
10.33 -45.0% 5.69 ± 5% perf-stat.i.metric.M/sec
3162 -11.6% 2796 perf-stat.i.minor-faults
500094 ± 3% -59.2% 203915 ± 34% perf-stat.i.node-load-misses
86458 ± 5% -65.8% 29541 ± 55% perf-stat.i.node-loads
273470 -86.8% 36116 ± 14% perf-stat.i.node-store-misses
60952 ± 9% -86.2% 8425 ±101% perf-stat.i.node-stores
3205 -12.7% 2797 perf-stat.i.page-faults
10.58 ± 2% +182.6% 29.91 ±104% perf-stat.overall.MPKI
1.37 ± 3% +2.1 3.48 ±104% perf-stat.overall.branch-miss-rate%
3.29 ± 3% +55.9% 5.13 ± 28% perf-stat.overall.cpi
1034 ± 4% -41.2% 608.25 ± 22% perf-stat.overall.instructions-per-iTLB-miss
0.30 ± 3% -31.6% 0.21 ± 22% perf-stat.overall.ipc
3.363e+08 -48.9% 1.72e+08 ± 7% perf-stat.ps.branch-instructions
8351 ± 4% -83.9% 1342 ± 6% perf-stat.ps.context-switches
94471 +1.4% 95822 perf-stat.ps.cpu-clock
157.89 -38.3% 97.44 perf-stat.ps.cpu-migrations
4.134e+08 -45.6% 2.25e+08 ± 3% perf-stat.ps.dTLB-loads
2.063e+08 -41.1% 1.215e+08 ± 2% perf-stat.ps.dTLB-stores
2006862 -15.5% 1695650 ± 21% perf-stat.ps.iTLB-loads
1.6e+09 -48.3% 8.263e+08 ± 7% perf-stat.ps.instructions
11.01 -85.2% 1.63 ± 20% perf-stat.ps.major-faults
3112 -10.3% 2791 perf-stat.ps.minor-faults
492165 ± 3% -58.6% 203545 ± 33% perf-stat.ps.node-load-misses
85077 ± 5% -65.3% 29492 ± 55% perf-stat.ps.node-loads
269158 -86.6% 36060 ± 14% perf-stat.ps.node-store-misses
59987 ± 9% -86.0% 8420 ±101% perf-stat.ps.node-stores
3154 -11.5% 2793 perf-stat.ps.page-faults
94471 +1.4% 95822 perf-stat.ps.task-clock
1.016e+11 +350.5% 4.579e+11 ± 13% perf-stat.total.instructions
3935 ± 4% -57.2% 1684 ± 6% slabinfo.Acpi-Parse.active_objs
3935 ± 4% -57.2% 1684 ± 6% slabinfo.Acpi-Parse.num_objs
1575 -84.5% 244.20 ± 6% slabinfo.bdev_cache.active_objs
1575 -84.5% 244.20 ± 6% slabinfo.bdev_cache.num_objs
323.00 ± 11% -33.7% 214.00 ± 22% slabinfo.biovec-128.num_objs
2514 ± 4% -42.9% 1435 ± 4% slabinfo.buffer_head.active_objs
2535 ± 4% -43.4% 1435 ± 4% slabinfo.buffer_head.num_objs
4061 ± 5% -20.4% 3232 ± 7% slabinfo.dmaengine-unmap-16.active_objs
4072 ± 5% -20.6% 3233 ± 7% slabinfo.dmaengine-unmap-16.num_objs
2479 -90.3% 240.20 ± 28% slabinfo.dquot.active_objs
2479 -90.3% 240.20 ± 28% slabinfo.dquot.num_objs
5213 ± 13% -68.9% 1620 ± 23% slabinfo.ext4_extent_status.active_objs
5213 ± 13% -68.9% 1620 ± 23% slabinfo.ext4_extent_status.num_objs
4.67 ±127% +4400.0% 210.00 ± 36% slabinfo.ext4_pending_reservation.active_objs
4.67 ±127% +4400.0% 210.00 ± 36% slabinfo.ext4_pending_reservation.num_objs
2360 ± 2% -49.0% 1204 ± 5% slabinfo.file_lock_cache.active_objs
2360 ± 2% -49.0% 1204 ± 5% slabinfo.file_lock_cache.num_objs
7031 ± 5% -47.0% 3729 ± 14% slabinfo.fsnotify_mark_connector.active_objs
7031 ± 5% -47.0% 3729 ± 14% slabinfo.fsnotify_mark_connector.num_objs
1417 -9.7% 1280 slabinfo.inode_cache.active_slabs
76560 -9.7% 69145 slabinfo.inode_cache.num_objs
1417 -9.7% 1280 slabinfo.inode_cache.num_slabs
99699 -14.6% 85176 slabinfo.kernfs_node_cache.active_objs
3119 -14.7% 2661 slabinfo.kernfs_node_cache.active_slabs
99815 -14.7% 85176 slabinfo.kernfs_node_cache.num_objs
3119 -14.7% 2661 slabinfo.kernfs_node_cache.num_slabs
44972 -18.9% 36462 slabinfo.kmalloc-16.num_objs
7059 ± 2% -18.4% 5763 ± 2% slabinfo.kmalloc-192.num_objs
6610 ± 2% -12.7% 5771 ± 2% slabinfo.kmalloc-1k.active_objs
7524 ± 2% -21.1% 5937 slabinfo.kmalloc-1k.num_objs
9426 ± 2% -8.0% 8676 ± 3% slabinfo.kmalloc-256.active_objs
9582 ± 2% -8.8% 8737 ± 3% slabinfo.kmalloc-256.num_objs
7902 ± 2% -9.3% 7170 ± 3% slabinfo.kmalloc-2k.active_objs
8225 ± 2% -11.7% 7265 ± 3% slabinfo.kmalloc-2k.num_objs
2221 -12.4% 1945 slabinfo.kmalloc-4k.num_objs
6092 +29.2% 7873 ± 3% slabinfo.kmalloc-96.active_objs
7021 ± 2% +13.9% 7994 ± 3% slabinfo.kmalloc-96.num_objs
3282 -84.4% 513.20 ± 6% slabinfo.kmalloc-rcl-192.active_objs
3282 -84.4% 513.20 ± 6% slabinfo.kmalloc-rcl-192.num_objs
5514 ± 2% -11.6% 4875 slabinfo.kmalloc-rcl-512.active_objs
5518 ± 2% -11.6% 4876 slabinfo.kmalloc-rcl-512.num_objs
6565 ± 3% -21.8% 5131 ± 8% slabinfo.kmalloc-rcl-64.active_objs
6565 ± 3% -21.8% 5131 ± 8% slabinfo.kmalloc-rcl-64.num_objs
2880 ± 6% -19.0% 2332 ± 11% slabinfo.kmalloc-rcl-96.active_objs
2880 ± 6% -19.0% 2332 ± 11% slabinfo.kmalloc-rcl-96.num_objs
6217 -77.1% 1424 ± 21% slabinfo.mbcache.active_objs
6217 -77.1% 1424 ± 21% slabinfo.mbcache.num_objs
13404 +13.2% 15170 slabinfo.proc_inode_cache.active_objs
13413 +13.3% 15197 slabinfo.proc_inode_cache.num_objs
1246 -88.5% 142.80 ± 13% slabinfo.request_queue.active_objs
1246 -88.5% 142.80 ± 13% slabinfo.request_queue.num_objs
528.17 ± 7% -33.5% 351.00 ± 13% slabinfo.skbuff_fclone_cache.active_objs
528.17 ± 7% -33.5% 351.00 ± 13% slabinfo.skbuff_fclone_cache.num_objs
1399 -9.4% 1268 slabinfo.task_struct.active_objs
1403 -9.2% 1274 slabinfo.task_struct.active_slabs
1403 -9.2% 1274 slabinfo.task_struct.num_objs
1403 -9.2% 1274 slabinfo.task_struct.num_slabs
5015 ± 3% -23.2% 3852 ± 3% slabinfo.trace_event_file.active_objs
5015 ± 3% -23.2% 3852 ± 3% slabinfo.trace_event_file.num_objs
11008 +54.0% 16950 ± 8% slabinfo.vmap_area.active_objs
11022 +55.3% 17115 ± 8% slabinfo.vmap_area.num_objs
0.00 ±223% +1820.0% 0.01 ± 34% perf-sched.sch_delay.avg.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
0.00 ± 56% +232.3% 0.01 ± 26% perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.00 ±152% +428.0% 0.01 ± 30% perf-sched.sch_delay.avg.ms.schedule_timeout.ext4_lazyinit_thread.part.0.kthread
0.00 ±141% +644.0% 0.01 ± 31% perf-sched.sch_delay.avg.ms.schedule_timeout.io_schedule_timeout.wait_for_completion_io.submit_bio_wait
0.00 ±223% +753.3% 0.01 ± 32% perf-sched.sch_delay.avg.ms.schedule_timeout.kcompactd.kthread.ret_from_fork
0.00 ± 22% +268.3% 0.02 ± 98% perf-sched.sch_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
0.00 ±223% +2930.0% 0.02 ± 28% perf-sched.sch_delay.max.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
0.01 ± 63% +550.0% 0.04 ± 61% perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.00 ±152% +756.0% 0.02 ± 35% perf-sched.sch_delay.max.ms.schedule_timeout.ext4_lazyinit_thread.part.0.kthread
0.00 ±141% +920.0% 0.02 ± 22% perf-sched.sch_delay.max.ms.schedule_timeout.io_schedule_timeout.wait_for_completion_io.submit_bio_wait
0.00 ±223% +1126.7% 0.02 ± 25% perf-sched.sch_delay.max.ms.schedule_timeout.kcompactd.kthread.ret_from_fork
0.01 ± 30% +25140.0% 2.02 ±197% perf-sched.sch_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
0.18 ± 25% +1.2e+05% 219.77 ± 10% perf-sched.total_wait_and_delay.average.ms
332.83 ± 18% +1933.6% 6768 ± 10% perf-sched.total_wait_and_delay.count.ms
14.17 ± 33% +70409.9% 9992 perf-sched.total_wait_and_delay.max.ms
0.16 ± 31% +1.3e+05% 219.43 ± 10% perf-sched.total_wait_time.average.ms
13.18 ± 43% +75740.1% 9992 perf-sched.total_wait_time.max.ms
0.00 ±223% +1.2e+08% 818.24 perf-sched.wait_and_delay.avg.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
0.25 ±123% +1.1e+05% 272.84 ± 5% perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.02 ±146% +3853.7% 0.75 ± 2% perf-sched.wait_and_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.04 ± 43% +2.2e+05% 76.70 ± 5% perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
0.00 ±141% +1.7e+07% 289.31 ± 12% perf-sched.wait_and_delay.avg.ms.schedule_timeout.io_schedule_timeout.wait_for_completion_io.submit_bio_wait
0.00 ±223% +3.2e+07% 478.42 perf-sched.wait_and_delay.avg.ms.schedule_timeout.kcompactd.kthread.ret_from_fork
2.71 ± 35% +583.0% 18.48 ± 61% perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
0.06 ±132% +1.3e+06% 723.00 ± 3% perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
1.48 ± 92% +30586.5% 453.90 ± 7% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
0.50 ±100% +2100.0% 11.00 perf-sched.wait_and_delay.count.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
5.67 ± 13% +4223.5% 245.00 perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.83 ±107% +29108.0% 243.40 perf-sched.wait_and_delay.count.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
104.83 ± 58% +1907.8% 2104 ± 5% perf-sched.wait_and_delay.count.pipe_read.new_sync_read.vfs_read.ksys_read
96.33 +956.3% 1017 ± 4% perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
0.33 ±141% +9920.0% 33.40 ± 14% perf-sched.wait_and_delay.count.schedule_timeout.io_schedule_timeout.wait_for_completion_io.submit_bio_wait
0.17 ±223% +23540.0% 39.40 ± 2% perf-sched.wait_and_delay.count.schedule_timeout.kcompactd.kthread.ret_from_fork
4.33 ± 31% +20036.9% 872.60 ± 79% perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
97.00 +1149.9% 1212 ± 3% perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
5.50 ± 20% +10583.6% 587.60 ± 2% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
0.00 ±223% +1.5e+08% 1000 perf-sched.wait_and_delay.max.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
0.97 ±137% +3.9e+05% 3771 ± 65% perf-sched.wait_and_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.03 ±148% +40301.4% 13.40 perf-sched.wait_and_delay.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.44 ±101% +2.3e+05% 1006 perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
0.00 ±223% +2.1e+08% 9992 perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
0.00 ±141% +1.8e+07% 301.75 ± 12% perf-sched.wait_and_delay.max.ms.schedule_timeout.io_schedule_timeout.wait_for_completion_io.submit_bio_wait
0.00 ±223% +3.4e+07% 505.03 perf-sched.wait_and_delay.max.ms.schedule_timeout.kcompactd.kthread.ret_from_fork
4.46 ± 25% +6607.2% 299.40 ± 48% perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
4.80 ±147% +31635.7% 1522 ± 67% perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
7.15 ±103% +1.3e+05% 8972 perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork
0.25 ±124% +1.1e+05% 272.84 ± 5% perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.02 ±149% +4656.6% 0.75 ± 2% perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.03 ± 51% +2.6e+05% 76.70 ± 5% perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
2.08 ± 21% +2272.6% 49.41 ±143% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
2.70 ± 35% +583.6% 18.46 ± 61% perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
0.03 ±222% +2.4e+06% 722.99 ± 3% perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
0.93 ±150% +48396.0% 450.37 ± 6% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
0.97 ±137% +3.9e+05% 3771 ± 65% perf-sched.wait_time.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.03 ±149% +42879.3% 13.40 perf-sched.wait_time.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.43 ±106% +2.3e+05% 1006 perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
0.00 ±223% +2.1e+08% 9992 perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
9.79 ± 29% +4253.2% 426.40 ± 19% perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
2.42 ±141% +281.7% 9.22 ± 2% perf-sched.wait_time.max.ms.schedule_timeout.ext4_lazyinit_thread.part.0.kthread
4.46 ± 25% +6613.9% 299.37 ± 48% perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
2.96 ±222% +51425.8% 1522 ± 67% perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
5.46 ±154% +1.6e+05% 8972 perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork
128.00 +768.1% 1111 ± 12% interrupts.9:IO-APIC.9-fasteoi.acpi
64630 +21.1% 78254 ± 3% interrupts.CAL:Function_call_interrupts
127590 +770.8% 1111103 ± 13% interrupts.CPU0.LOC:Local_timer_interrupts
128.00 +768.1% 1111 ± 12% interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi
127666 +770.3% 1111040 ± 13% interrupts.CPU1.LOC:Local_timer_interrupts
127702 +770.1% 1111139 ± 13% interrupts.CPU10.LOC:Local_timer_interrupts
127701 +770.1% 1111121 ± 13% interrupts.CPU11.LOC:Local_timer_interrupts
127505 +771.5% 1111167 ± 13% interrupts.CPU12.LOC:Local_timer_interrupts
127697 +770.1% 1111136 ± 13% interrupts.CPU13.LOC:Local_timer_interrupts
127617 +770.7% 1111119 ± 13% interrupts.CPU14.LOC:Local_timer_interrupts
127415 +772.2% 1111307 ± 13% interrupts.CPU15.LOC:Local_timer_interrupts
127685 +770.3% 1111238 ± 13% interrupts.CPU16.LOC:Local_timer_interrupts
127875 +768.9% 1111078 ± 13% interrupts.CPU17.LOC:Local_timer_interrupts
127440 +771.9% 1111101 ± 13% interrupts.CPU18.LOC:Local_timer_interrupts
127667 +770.4% 1111256 ± 13% interrupts.CPU19.LOC:Local_timer_interrupts
837.00 ± 18% -26.5% 615.20 ± 2% interrupts.CPU2.CAL:Function_call_interrupts
127702 +770.1% 1111115 ± 13% interrupts.CPU2.LOC:Local_timer_interrupts
263.00 ± 43% -53.1% 123.40 ± 40% interrupts.CPU2.NMI:Non-maskable_interrupts
263.00 ± 43% -53.1% 123.40 ± 40% interrupts.CPU2.PMI:Performance_monitoring_interrupts
127644 +770.4% 1111076 ± 13% interrupts.CPU20.LOC:Local_timer_interrupts
127463 +771.7% 1111091 ± 13% interrupts.CPU21.LOC:Local_timer_interrupts
291.67 ± 15% -60.8% 114.20 ± 16% interrupts.CPU21.NMI:Non-maskable_interrupts
291.67 ± 15% -60.8% 114.20 ± 16% interrupts.CPU21.PMI:Performance_monitoring_interrupts
127839 +769.1% 1111087 ± 13% interrupts.CPU22.LOC:Local_timer_interrupts
127883 +769.1% 1111402 ± 13% interrupts.CPU23.LOC:Local_timer_interrupts
665.50 ± 7% -9.8% 600.40 interrupts.CPU24.CAL:Function_call_interrupts
127511 +771.4% 1111102 ± 13% interrupts.CPU24.LOC:Local_timer_interrupts
127638 +770.5% 1111094 ± 13% interrupts.CPU25.LOC:Local_timer_interrupts
127979 +768.3% 1111283 ± 13% interrupts.CPU26.LOC:Local_timer_interrupts
127715 +770.1% 1111187 ± 13% interrupts.CPU27.LOC:Local_timer_interrupts
242.00 ± 37% -44.0% 135.60 ± 27% interrupts.CPU27.NMI:Non-maskable_interrupts
242.00 ± 37% -44.0% 135.60 ± 27% interrupts.CPU27.PMI:Performance_monitoring_interrupts
127644 +770.5% 1111182 ± 13% interrupts.CPU28.LOC:Local_timer_interrupts
127860 +769.0% 1111086 ± 13% interrupts.CPU29.LOC:Local_timer_interrupts
127959 +768.4% 1111218 ± 13% interrupts.CPU3.LOC:Local_timer_interrupts
232.50 ± 19% -46.3% 124.80 ± 38% interrupts.CPU3.NMI:Non-maskable_interrupts
232.50 ± 19% -46.3% 124.80 ± 38% interrupts.CPU3.PMI:Performance_monitoring_interrupts
127875 +768.8% 1111027 ± 13% interrupts.CPU30.LOC:Local_timer_interrupts
248.83 ± 33% -44.7% 137.60 ± 29% interrupts.CPU30.NMI:Non-maskable_interrupts
248.83 ± 33% -44.7% 137.60 ± 29% interrupts.CPU30.PMI:Performance_monitoring_interrupts
128053 +767.6% 1111029 ± 13% interrupts.CPU31.LOC:Local_timer_interrupts
206.50 ± 21% -44.3% 115.00 ± 16% interrupts.CPU31.NMI:Non-maskable_interrupts
206.50 ± 21% -44.3% 115.00 ± 16% interrupts.CPU31.PMI:Performance_monitoring_interrupts
127400 +772.1% 1111026 ± 13% interrupts.CPU32.LOC:Local_timer_interrupts
127540 +771.2% 1111145 ± 13% interrupts.CPU33.LOC:Local_timer_interrupts
127782 +769.6% 1111188 ± 13% interrupts.CPU34.LOC:Local_timer_interrupts
127628 +770.5% 1111058 ± 13% interrupts.CPU35.LOC:Local_timer_interrupts
187.83 ± 30% -37.3% 117.80 ± 14% interrupts.CPU35.NMI:Non-maskable_interrupts
187.83 ± 30% -37.3% 117.80 ± 14% interrupts.CPU35.PMI:Performance_monitoring_interrupts
127515 +771.4% 1111140 ± 13% interrupts.CPU36.LOC:Local_timer_interrupts
127613 +770.7% 1111074 ± 13% interrupts.CPU37.LOC:Local_timer_interrupts
127622 +770.6% 1111045 ± 13% interrupts.CPU38.LOC:Local_timer_interrupts
127671 +770.2% 1111029 ± 13% interrupts.CPU39.LOC:Local_timer_interrupts
127590 +770.8% 1111091 ± 13% interrupts.CPU4.LOC:Local_timer_interrupts
280.67 ± 37% -50.0% 140.40 ± 24% interrupts.CPU4.NMI:Non-maskable_interrupts
280.67 ± 37% -50.0% 140.40 ± 24% interrupts.CPU4.PMI:Performance_monitoring_interrupts
127627 +770.5% 1111031 ± 13% interrupts.CPU40.LOC:Local_timer_interrupts
127492 +771.5% 1111110 ± 13% interrupts.CPU41.LOC:Local_timer_interrupts
127525 +771.2% 1110964 ± 13% interrupts.CPU42.LOC:Local_timer_interrupts
127450 +771.7% 1111040 ± 13% interrupts.CPU43.LOC:Local_timer_interrupts
127883 +768.8% 1111113 ± 13% interrupts.CPU44.LOC:Local_timer_interrupts
127463 +771.6% 1111027 ± 13% interrupts.CPU45.LOC:Local_timer_interrupts
127432 +771.9% 1111029 ± 13% interrupts.CPU46.LOC:Local_timer_interrupts
127545 +771.1% 1111016 ± 13% interrupts.CPU47.LOC:Local_timer_interrupts
128036 +767.8% 1111089 ± 13% interrupts.CPU48.LOC:Local_timer_interrupts
203.33 ± 15% -47.4% 107.00 ± 31% interrupts.CPU48.NMI:Non-maskable_interrupts
203.33 ± 15% -47.4% 107.00 ± 31% interrupts.CPU48.PMI:Performance_monitoring_interrupts
127859 +769.0% 1111058 ± 13% interrupts.CPU49.LOC:Local_timer_interrupts
246.33 ± 18% -59.0% 101.00 ± 31% interrupts.CPU49.NMI:Non-maskable_interrupts
246.33 ± 18% -59.0% 101.00 ± 31% interrupts.CPU49.PMI:Performance_monitoring_interrupts
127782 +769.6% 1111137 ± 13% interrupts.CPU5.LOC:Local_timer_interrupts
216.00 ± 24% -34.6% 141.20 ± 29% interrupts.CPU5.NMI:Non-maskable_interrupts
216.00 ± 24% -34.6% 141.20 ± 29% interrupts.CPU5.PMI:Performance_monitoring_interrupts
687.17 ± 7% -12.0% 604.60 interrupts.CPU50.CAL:Function_call_interrupts
127612 +770.7% 1111107 ± 13% interrupts.CPU50.LOC:Local_timer_interrupts
260.00 ± 46% -59.4% 105.60 ± 44% interrupts.CPU50.NMI:Non-maskable_interrupts
260.00 ± 46% -59.4% 105.60 ± 44% interrupts.CPU50.PMI:Performance_monitoring_interrupts
127753 +769.7% 1111107 ± 13% interrupts.CPU51.LOC:Local_timer_interrupts
127741 +769.8% 1111151 ± 13% interrupts.CPU52.LOC:Local_timer_interrupts
220.50 ± 25% -42.9% 125.80 ± 37% interrupts.CPU52.NMI:Non-maskable_interrupts
220.50 ± 25% -42.9% 125.80 ± 37% interrupts.CPU52.PMI:Performance_monitoring_interrupts
127472 +771.8% 1111284 ± 13% interrupts.CPU53.LOC:Local_timer_interrupts
127618 +770.6% 1111088 ± 13% interrupts.CPU54.LOC:Local_timer_interrupts
127806 +769.5% 1111287 ± 13% interrupts.CPU55.LOC:Local_timer_interrupts
127518 +771.4% 1111184 ± 13% interrupts.CPU56.LOC:Local_timer_interrupts
127480 +771.8% 1111368 ± 13% interrupts.CPU57.LOC:Local_timer_interrupts
127654 +770.4% 1111081 ± 13% interrupts.CPU58.LOC:Local_timer_interrupts
756.00 ± 18% -22.7% 584.40 ± 5% interrupts.CPU59.CAL:Function_call_interrupts
127593 +770.9% 1111192 ± 13% interrupts.CPU59.LOC:Local_timer_interrupts
127625 +770.6% 1111090 ± 13% interrupts.CPU6.LOC:Local_timer_interrupts
127575 +770.9% 1111088 ± 13% interrupts.CPU60.LOC:Local_timer_interrupts
127992 +768.1% 1111073 ± 13% interrupts.CPU61.LOC:Local_timer_interrupts
127785 +769.5% 1111137 ± 13% interrupts.CPU62.LOC:Local_timer_interrupts
188.33 ± 27% -53.2% 88.20 ± 25% interrupts.CPU62.NMI:Non-maskable_interrupts
188.33 ± 27% -53.2% 88.20 ± 25% interrupts.CPU62.PMI:Performance_monitoring_interrupts
127774 +769.6% 1111109 ± 13% interrupts.CPU63.LOC:Local_timer_interrupts
127792 +769.5% 1111184 ± 13% interrupts.CPU64.LOC:Local_timer_interrupts
217.67 ± 42% -52.0% 104.40 ± 31% interrupts.CPU64.NMI:Non-maskable_interrupts
217.67 ± 42% -52.0% 104.40 ± 31% interrupts.CPU64.PMI:Performance_monitoring_interrupts
127921 +768.6% 1111130 ± 13% interrupts.CPU65.LOC:Local_timer_interrupts
211.50 ± 32% -44.8% 116.80 ± 38% interrupts.CPU65.NMI:Non-maskable_interrupts
211.50 ± 32% -44.8% 116.80 ± 38% interrupts.CPU65.PMI:Performance_monitoring_interrupts
127828 +769.3% 1111230 ± 13% interrupts.CPU66.LOC:Local_timer_interrupts
201.33 ± 22% -46.1% 108.60 ± 30% interrupts.CPU66.NMI:Non-maskable_interrupts
201.33 ± 22% -46.1% 108.60 ± 30% interrupts.CPU66.PMI:Performance_monitoring_interrupts
127662 +770.4% 1111223 ± 13% interrupts.CPU67.LOC:Local_timer_interrupts
280.00 ± 29% -62.6% 104.60 ± 26% interrupts.CPU67.NMI:Non-maskable_interrupts
280.00 ± 29% -62.6% 104.60 ± 26% interrupts.CPU67.PMI:Performance_monitoring_interrupts
127887 +769.1% 1111510 ± 13% interrupts.CPU68.LOC:Local_timer_interrupts
231.17 ± 24% -52.2% 110.60 ± 33% interrupts.CPU68.NMI:Non-maskable_interrupts
231.17 ± 24% -52.2% 110.60 ± 33% interrupts.CPU68.PMI:Performance_monitoring_interrupts
127704 +770.1% 1111152 ± 13% interrupts.CPU69.LOC:Local_timer_interrupts
241.83 ± 29% -49.6% 121.80 ± 39% interrupts.CPU69.NMI:Non-maskable_interrupts
241.83 ± 29% -49.6% 121.80 ± 39% interrupts.CPU69.PMI:Performance_monitoring_interrupts
127479 +771.6% 1111091 ± 13% interrupts.CPU7.LOC:Local_timer_interrupts
127637 +770.5% 1111111 ± 13% interrupts.CPU70.LOC:Local_timer_interrupts
127568 +771.0% 1111167 ± 13% interrupts.CPU71.LOC:Local_timer_interrupts
232.00 ± 24% -49.2% 117.80 ± 43% interrupts.CPU71.NMI:Non-maskable_interrupts
232.00 ± 24% -49.2% 117.80 ± 43% interrupts.CPU71.PMI:Performance_monitoring_interrupts
127584 +770.8% 1111048 ± 13% interrupts.CPU72.LOC:Local_timer_interrupts
127752 +769.9% 1111272 ± 13% interrupts.CPU73.LOC:Local_timer_interrupts
127705 +770.1% 1111229 ± 13% interrupts.CPU74.LOC:Local_timer_interrupts
187.17 ± 30% -34.1% 123.40 ± 43% interrupts.CPU74.NMI:Non-maskable_interrupts
187.17 ± 30% -34.1% 123.40 ± 43% interrupts.CPU74.PMI:Performance_monitoring_interrupts
127802 +769.4% 1111144 ± 13% interrupts.CPU75.LOC:Local_timer_interrupts
253.83 ± 24% -54.3% 116.00 ± 22% interrupts.CPU75.NMI:Non-maskable_interrupts
253.83 ± 24% -54.3% 116.00 ± 22% interrupts.CPU75.PMI:Performance_monitoring_interrupts
127785 +769.6% 1111238 ± 13% interrupts.CPU76.LOC:Local_timer_interrupts
202.00 ± 37% -43.3% 114.60 ± 17% interrupts.CPU76.NMI:Non-maskable_interrupts
202.00 ± 37% -43.3% 114.60 ± 17% interrupts.CPU76.PMI:Performance_monitoring_interrupts
127789 +769.6% 1111269 ± 13% interrupts.CPU77.LOC:Local_timer_interrupts
203.50 ± 35% -42.4% 117.20 ± 17% interrupts.CPU77.NMI:Non-maskable_interrupts
203.50 ± 35% -42.4% 117.20 ± 17% interrupts.CPU77.PMI:Performance_monitoring_interrupts
127542 +771.2% 1111187 ± 13% interrupts.CPU78.LOC:Local_timer_interrupts
127678 +770.2% 1111113 ± 13% interrupts.CPU79.LOC:Local_timer_interrupts
200.17 ± 19% -48.1% 103.80 ± 31% interrupts.CPU79.NMI:Non-maskable_interrupts
200.17 ± 19% -48.1% 103.80 ± 31% interrupts.CPU79.PMI:Performance_monitoring_interrupts
127650 +770.6% 1111283 ± 13% interrupts.CPU8.LOC:Local_timer_interrupts
127841 +769.2% 1111160 ± 13% interrupts.CPU80.LOC:Local_timer_interrupts
127489 +771.6% 1111176 ± 13% interrupts.CPU81.LOC:Local_timer_interrupts
127668 +770.5% 1111340 ± 13% interrupts.CPU82.LOC:Local_timer_interrupts
127892 +768.8% 1111175 ± 13% interrupts.CPU83.LOC:Local_timer_interrupts
127640 +770.5% 1111123 ± 13% interrupts.CPU84.LOC:Local_timer_interrupts
127693 +770.2% 1111192 ± 13% interrupts.CPU85.LOC:Local_timer_interrupts
127458 +771.7% 1111105 ± 13% interrupts.CPU86.LOC:Local_timer_interrupts
127492 +771.5% 1111079 ± 13% interrupts.CPU87.LOC:Local_timer_interrupts
127699 +770.1% 1111103 ± 13% interrupts.CPU88.LOC:Local_timer_interrupts
127963 +768.3% 1111112 ± 13% interrupts.CPU89.LOC:Local_timer_interrupts
165.17 ± 22% -33.6% 109.60 ± 27% interrupts.CPU89.NMI:Non-maskable_interrupts
165.17 ± 22% -33.6% 109.60 ± 27% interrupts.CPU89.PMI:Performance_monitoring_interrupts
127532 +771.2% 1111094 ± 13% interrupts.CPU9.LOC:Local_timer_interrupts
127736 +769.9% 1111139 ± 13% interrupts.CPU90.LOC:Local_timer_interrupts
127585 +770.9% 1111141 ± 13% interrupts.CPU91.LOC:Local_timer_interrupts
127793 +769.6% 1111251 ± 13% interrupts.CPU92.LOC:Local_timer_interrupts
127737 +770.0% 1111275 ± 13% interrupts.CPU93.LOC:Local_timer_interrupts
207.67 ± 41% -38.3% 128.20 ± 43% interrupts.CPU93.NMI:Non-maskable_interrupts
207.67 ± 41% -38.3% 128.20 ± 43% interrupts.CPU93.PMI:Performance_monitoring_interrupts
127478 +771.7% 1111196 ± 13% interrupts.CPU94.LOC:Local_timer_interrupts
127809 +769.5% 1111269 ± 13% interrupts.CPU95.LOC:Local_timer_interrupts
12257088 +770.3% 1.067e+08 ± 13% interrupts.LOC:Local_timer_interrupts
0.00 +1.2e+104% 115.20 ± 33% interrupts.MCP:Machine_check_polls
18319 ± 2% -36.9% 11562 ± 20% interrupts.NMI:Non-maskable_interrupts
18319 ± 2% -36.9% 11562 ± 20% interrupts.PMI:Performance_monitoring_interrupts
19041 ± 2% -66.9% 6302 ± 8% softirqs.BLOCK
12376 ± 16% +188.6% 35722 ± 27% softirqs.CPU0.RCU
11554 ± 8% +416.4% 59667 ± 32% softirqs.CPU0.SCHED
9451 ± 4% +250.7% 33142 ± 37% softirqs.CPU1.RCU
9894 ± 15% +601.5% 69406 ± 14% softirqs.CPU1.SCHED
8818 ± 3% +274.3% 33010 ± 38% softirqs.CPU10.RCU
7845 ± 17% +767.1% 68029 ± 16% softirqs.CPU10.SCHED
8758 ± 2% +269.8% 32390 ± 35% softirqs.CPU11.RCU
8268 ± 12% +564.8% 54965 ± 19% softirqs.CPU11.SCHED
8777 ± 3% +283.0% 33620 ± 37% softirqs.CPU12.RCU
8562 ± 6% +635.3% 62958 ± 27% softirqs.CPU12.SCHED
8618 ± 2% +291.8% 33766 ± 39% softirqs.CPU13.RCU
8896 ± 5% +671.5% 68637 ± 15% softirqs.CPU13.SCHED
8822 ± 3% +274.8% 33071 ± 39% softirqs.CPU14.RCU
8765 ± 7% +617.6% 62905 ± 27% softirqs.CPU14.SCHED
8892 ± 5% +266.9% 32623 ± 38% softirqs.CPU15.RCU
8770 ± 13% +690.8% 69353 ± 15% softirqs.CPU15.SCHED
8633 ± 5% +261.4% 31202 ± 35% softirqs.CPU16.RCU
8236 ± 12% +728.8% 68262 ± 15% softirqs.CPU16.SCHED
8631 ± 3% +286.5% 33363 ± 35% softirqs.CPU17.RCU
8590 ± 7% +702.7% 68958 ± 15% softirqs.CPU17.SCHED
8570 ± 4% +292.6% 33648 ± 38% softirqs.CPU18.RCU
9000 ± 3% +665.7% 68922 ± 16% softirqs.CPU18.SCHED
8654 ± 4% +276.0% 32543 ± 35% softirqs.CPU19.RCU
8259 ± 7% +663.8% 63080 ± 27% softirqs.CPU19.SCHED
9003 ± 3% +278.0% 34036 ± 38% softirqs.CPU2.RCU
8969 ± 9% +664.7% 68583 ± 15% softirqs.CPU2.SCHED
8553 ± 2% +285.0% 32935 ± 37% softirqs.CPU20.RCU
8481 ± 14% +707.0% 68448 ± 16% softirqs.CPU20.SCHED
8347 ± 2% +294.6% 32937 ± 36% softirqs.CPU21.RCU
8053 ± 16% +685.5% 63253 ± 27% softirqs.CPU21.SCHED
8486 ± 2% +292.4% 33301 ± 36% softirqs.CPU22.RCU
8747 ± 7% +624.9% 63411 ± 27% softirqs.CPU22.SCHED
8466 ± 3% +288.6% 32904 ± 36% softirqs.CPU23.RCU
8409 ± 12% +715.9% 68618 ± 15% softirqs.CPU23.SCHED
8609 ± 2% +273.7% 32175 ± 35% softirqs.CPU24.RCU
7874 ± 10% +644.0% 58587 ± 36% softirqs.CPU24.SCHED
8457 ± 3% +251.9% 29760 ± 35% softirqs.CPU25.RCU
7892 ± 11% +770.0% 68665 ± 17% softirqs.CPU25.SCHED
8221 ± 3% +270.5% 30462 ± 36% softirqs.CPU26.RCU
8702 ± 5% +688.0% 68574 ± 17% softirqs.CPU26.SCHED
8603 ± 6% +269.9% 31822 ± 37% softirqs.CPU27.RCU
8671 ± 6% +692.2% 68696 ± 17% softirqs.CPU27.SCHED
8324 ± 3% +278.8% 31527 ± 35% softirqs.CPU28.RCU
8693 ± 9% +688.6% 68556 ± 17% softirqs.CPU28.SCHED
8367 ± 2% +267.4% 30742 ± 38% softirqs.CPU29.RCU
8451 ± 5% +711.8% 68609 ± 17% softirqs.CPU29.SCHED
9104 ± 7% +268.9% 33581 ± 38% softirqs.CPU3.RCU
8759 ± 4% +683.0% 68583 ± 16% softirqs.CPU3.SCHED
8364 ± 2% +276.4% 31485 ± 38% softirqs.CPU30.RCU
8018 ± 11% +646.0% 59812 ± 9% softirqs.CPU30.SCHED
8229 ± 3% +278.4% 31143 ± 35% softirqs.CPU31.RCU
8340 ± 12% +627.0% 60629 ± 30% softirqs.CPU31.SCHED
8608 +287.8% 33384 ± 36% softirqs.CPU32.RCU
8322 ± 8% +720.5% 68286 ± 17% softirqs.CPU32.SCHED
9482 ± 17% +244.0% 32616 ± 35% softirqs.CPU33.RCU
8092 ± 10% +744.1% 68312 ± 16% softirqs.CPU33.SCHED
8629 ± 3% +286.6% 33363 ± 35% softirqs.CPU34.RCU
8400 ± 11% +715.7% 68515 ± 17% softirqs.CPU34.SCHED
8834 ± 2% +279.4% 33515 ± 35% softirqs.CPU35.RCU
7691 ± 18% +683.2% 60233 ± 7% softirqs.CPU35.SCHED
9121 ± 13% +254.2% 32306 ± 35% softirqs.CPU36.RCU
8076 ± 17% +747.2% 68419 ± 17% softirqs.CPU36.SCHED
8649 +278.8% 32764 ± 35% softirqs.CPU37.RCU
8334 ± 9% +719.0% 68259 ± 17% softirqs.CPU37.SCHED
9588 ± 17% +242.1% 32807 ± 35% softirqs.CPU38.RCU
7864 ± 19% +780.2% 69217 ± 16% softirqs.CPU38.SCHED
8701 ± 8% +279.0% 32980 ± 35% softirqs.CPU39.RCU
8315 ± 15% +720.1% 68199 ± 17% softirqs.CPU39.SCHED
9298 ± 3% +260.1% 33479 ± 38% softirqs.CPU4.RCU
7896 ± 16% +768.0% 68540 ± 16% softirqs.CPU4.SCHED
8717 ± 2% +284.6% 33526 ± 36% softirqs.CPU40.RCU
8084 ± 13% +747.4% 68504 ± 16% softirqs.CPU40.SCHED
9221 ± 11% +265.7% 33718 ± 35% softirqs.CPU41.RCU
8288 ± 17% +663.6% 63284 ± 29% softirqs.CPU41.SCHED
8549 ± 5% +264.7% 31178 ± 36% softirqs.CPU42.RCU
8585 ± 15% +693.0% 68084 ± 17% softirqs.CPU42.SCHED
8447 ± 5% +289.6% 32916 ± 35% softirqs.CPU43.RCU
8799 ± 4% +674.1% 68118 ± 17% softirqs.CPU43.SCHED
9261 ± 18% +259.0% 33248 ± 36% softirqs.CPU44.RCU
8813 ± 5% +674.3% 68242 ± 17% softirqs.CPU44.SCHED
8566 ± 2% +284.7% 32959 ± 36% softirqs.CPU45.RCU
8649 ± 5% +688.2% 68177 ± 17% softirqs.CPU45.SCHED
8503 ± 2% +284.8% 32717 ± 35% softirqs.CPU46.RCU
8827 ± 4% +677.6% 68649 ± 16% softirqs.CPU46.SCHED
8764 ± 2% +288.3% 34030 ± 35% softirqs.CPU47.RCU
7976 ± 11% +465.8% 45132 ± 32% softirqs.CPU47.SCHED
8597 ± 2% +254.6% 30487 ± 34% softirqs.CPU48.RCU
6897 ± 28% +705.5% 55560 ± 42% softirqs.CPU48.SCHED
8707 ± 2% +262.2% 31540 ± 36% softirqs.CPU49.RCU
7951 ± 15% +771.6% 69308 ± 14% softirqs.CPU49.SCHED
8812 ± 4% +257.0% 31464 ± 48% softirqs.CPU5.RCU
8683 ± 7% +693.4% 68897 ± 15% softirqs.CPU5.SCHED
8667 ± 3% +284.5% 33326 ± 38% softirqs.CPU50.RCU
8629 ± 5% +639.1% 63781 ± 26% softirqs.CPU50.SCHED
8612 ± 2% +285.5% 33205 ± 39% softirqs.CPU51.RCU
8038 ± 14% +769.7% 69915 ± 15% softirqs.CPU51.SCHED
9342 ± 18% +258.4% 33480 ± 37% softirqs.CPU52.RCU
8941 ± 8% +686.8% 70350 ± 14% softirqs.CPU52.SCHED
8657 ± 4% +268.4% 31894 ± 36% softirqs.CPU53.RCU
9130 ± 5% +670.6% 70360 ± 14% softirqs.CPU53.SCHED
8567 ± 3% +294.4% 33793 ± 40% softirqs.CPU54.RCU
8566 ± 6% +720.0% 70248 ± 15% softirqs.CPU54.SCHED
8452 ± 3% +240.4% 28775 ± 35% softirqs.CPU55.RCU
8596 ± 8% +703.8% 69098 ± 15% softirqs.CPU55.SCHED
8433 ± 5% +295.7% 33366 ± 38% softirqs.CPU56.RCU
8981 ± 10% +673.9% 69506 ± 15% softirqs.CPU56.SCHED
8646 ± 4% +287.2% 33478 ± 38% softirqs.CPU57.RCU
7896 ± 14% +788.8% 70183 ± 14% softirqs.CPU57.SCHED
8628 ± 5% +281.9% 32951 ± 37% softirqs.CPU58.RCU
8711 ± 6% +701.7% 69841 ± 14% softirqs.CPU58.SCHED
9119 ± 6% +271.0% 33834 ± 40% softirqs.CPU59.RCU
7599 ± 22% +831.7% 70802 ± 14% softirqs.CPU59.SCHED
8794 ± 3% +257.0% 31390 ± 50% softirqs.CPU6.RCU
7587 ± 15% +806.2% 68752 ± 15% softirqs.CPU6.SCHED
8794 ± 7% +280.6% 33472 ± 38% softirqs.CPU60.RCU
8915 ± 12% +683.9% 69888 ± 14% softirqs.CPU60.SCHED
8736 ± 6% +272.0% 32497 ± 39% softirqs.CPU61.RCU
7830 ± 20% +789.0% 69619 ± 14% softirqs.CPU61.SCHED
8438 ± 2% +289.9% 32903 ± 38% softirqs.CPU62.RCU
8923 ± 3% +682.3% 69806 ± 14% softirqs.CPU62.SCHED
8514 ± 4% +236.4% 28638 ± 37% softirqs.CPU63.RCU
8927 ± 4% +683.3% 69927 ± 14% softirqs.CPU63.SCHED
8684 ± 5% +260.8% 31333 ± 34% softirqs.CPU64.RCU
8740 ± 9% +700.0% 69919 ± 14% softirqs.CPU64.SCHED
8657 ± 9% +285.0% 33334 ± 35% softirqs.CPU65.RCU
8775 ± 7% +643.6% 65250 ± 25% softirqs.CPU65.SCHED
8601 ± 7% +284.4% 33061 ± 36% softirqs.CPU66.RCU
7824 ± 25% +796.5% 70146 ± 14% softirqs.CPU66.SCHED
8926 ± 6% +261.6% 32278 ± 35% softirqs.CPU67.RCU
7495 ± 17% +817.6% 68774 ± 12% softirqs.CPU67.SCHED
8306 ± 4% +298.9% 33131 ± 37% softirqs.CPU68.RCU
9084 ± 4% +674.4% 70346 ± 14% softirqs.CPU68.SCHED
8712 ± 4% +276.1% 32765 ± 36% softirqs.CPU69.RCU
7531 ± 19% +829.7% 70023 ± 15% softirqs.CPU69.SCHED
8949 ± 6% +273.5% 33428 ± 37% softirqs.CPU7.RCU
8804 ± 15% +676.2% 68346 ± 16% softirqs.CPU7.SCHED
8292 ± 4% +301.4% 33290 ± 37% softirqs.CPU70.RCU
8676 ± 9% +710.8% 70353 ± 14% softirqs.CPU70.SCHED
8563 +282.7% 32769 ± 35% softirqs.CPU71.RCU
8678 ± 8% +707.5% 70081 ± 14% softirqs.CPU71.SCHED
8123 ± 3% +279.7% 30844 ± 37% softirqs.CPU72.RCU
8626 ± 10% +717.3% 70498 ± 16% softirqs.CPU72.SCHED
7931 ± 2% +277.6% 29949 ± 38% softirqs.CPU73.RCU
8313 ± 9% +668.6% 63900 ± 27% softirqs.CPU73.SCHED
7884 ± 6% +282.8% 30180 ± 36% softirqs.CPU74.RCU
8014 ± 16% +778.3% 70389 ± 16% softirqs.CPU74.SCHED
8094 ± 5% +282.6% 30968 ± 36% softirqs.CPU75.RCU
8563 ± 9% +721.0% 70309 ± 16% softirqs.CPU75.SCHED
7935 ± 2% +292.7% 31159 ± 35% softirqs.CPU76.RCU
8914 ± 5% +694.1% 70791 ± 15% softirqs.CPU76.SCHED
8066 ± 3% +296.0% 31941 ± 41% softirqs.CPU77.RCU
8786 ± 6% +665.6% 67270 ± 26% softirqs.CPU77.SCHED
8060 ± 2% +283.3% 30894 ± 37% softirqs.CPU78.RCU
8524 ± 9% +728.1% 70589 ± 16% softirqs.CPU78.SCHED
7895 ± 3% +288.4% 30666 ± 36% softirqs.CPU79.RCU
9026 ± 2% +677.7% 70198 ± 16% softirqs.CPU79.SCHED
8299 ± 9% +305.4% 33649 ± 38% softirqs.CPU8.RCU
8638 ± 8% +628.8% 62951 ± 27% softirqs.CPU8.SCHED
8113 ± 3% +295.0% 32048 ± 36% softirqs.CPU80.RCU
8343 ± 8% +744.1% 70426 ± 16% softirqs.CPU80.SCHED
8513 ± 5% +267.0% 31241 ± 36% softirqs.CPU81.RCU
8074 ± 14% +767.3% 70027 ± 16% softirqs.CPU81.SCHED
8183 ± 3% +291.7% 32057 ± 37% softirqs.CPU82.RCU
8472 ± 11% +733.9% 70651 ± 16% softirqs.CPU82.SCHED
8534 ± 3% +249.6% 29836 ± 44% softirqs.CPU83.RCU
8262 ± 7% +750.6% 70283 ± 16% softirqs.CPU83.SCHED
8470 ± 7% +267.7% 31146 ± 35% softirqs.CPU84.RCU
7999 ± 16% +775.9% 70062 ± 16% softirqs.CPU84.SCHED
8495 ± 5% +272.5% 31646 ± 36% softirqs.CPU85.RCU
7740 ± 16% +805.9% 70119 ± 16% softirqs.CPU85.SCHED
8248 ± 4% +269.8% 30507 ± 33% softirqs.CPU86.RCU
9214 ± 4% +662.6% 70274 ± 16% softirqs.CPU86.SCHED
8240 ± 2% +278.6% 31196 ± 36% softirqs.CPU87.RCU
8462 ± 10% +727.6% 70035 ± 16% softirqs.CPU87.SCHED
8201 ± 4% +295.1% 32406 ± 37% softirqs.CPU88.RCU
8727 ± 11% +708.8% 70592 ± 16% softirqs.CPU88.SCHED
8665 ± 3% +275.1% 32503 ± 36% softirqs.CPU89.RCU
8501 ± 5% +732.1% 70743 ± 16% softirqs.CPU89.SCHED
8602 ± 7% +287.1% 33299 ± 38% softirqs.CPU9.RCU
8684 ± 8% +688.1% 68440 ± 16% softirqs.CPU9.SCHED
8018 ± 3% +268.1% 29515 ± 34% softirqs.CPU90.RCU
8549 ± 8% +710.1% 69266 ± 17% softirqs.CPU90.SCHED
8327 ± 3% +280.5% 31684 ± 37% softirqs.CPU91.RCU
7884 ± 18% +787.3% 69959 ± 16% softirqs.CPU91.SCHED
8110 +291.6% 31763 ± 36% softirqs.CPU92.RCU
9117 ± 3% +667.6% 69986 ± 16% softirqs.CPU92.SCHED
8112 ± 2% +295.0% 32044 ± 37% softirqs.CPU93.RCU
8179 ± 12% +759.1% 70263 ± 16% softirqs.CPU93.SCHED
8804 ± 9% +267.0% 32313 ± 36% softirqs.CPU94.RCU
7897 ± 14% +793.6% 70567 ± 16% softirqs.CPU94.SCHED
8547 ± 5% +284.9% 32900 ± 36% softirqs.CPU95.RCU
7890 ± 16% +633.5% 57881 ± 24% softirqs.CPU95.SCHED
5318 ±154% +353.4% 24111 ± 60% softirqs.NET_RX
827812 +274.8% 3102713 ± 36% softirqs.RCU
811580 +699.7% 6489945 ± 16% softirqs.SCHED
17742 ± 4% +117.8% 38644 ± 17% softirqs.TIMER
18.86 ± 2% -18.9 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
16.79 ± 3% -16.8 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
13.83 ± 5% -13.8 0.00 perf-profile.calltrace.cycles-pp.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
12.17 ± 3% -12.1 0.11 ±200% perf-profile.calltrace.cycles-pp.ret_from_fork
12.15 ± 3% -12.0 0.11 ±200% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
11.77 ± 3% -11.8 0.00 perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
11.69 ± 3% -11.7 0.00 perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
11.10 ± 3% -11.1 0.00 perf-profile.calltrace.cycles-pp.drm_fb_helper_damage_work.process_one_work.worker_thread.kthread.ret_from_fork
10.71 ± 3% -10.7 0.00 perf-profile.calltrace.cycles-pp.memcpy_toio.drm_fb_helper_damage_work.process_one_work.worker_thread.kthread
10.30 ± 8% -10.3 0.00 perf-profile.calltrace.cycles-pp.block_ioctl.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
10.30 ± 8% -10.3 0.00 perf-profile.calltrace.cycles-pp.blkdev_ioctl.block_ioctl.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
10.29 ± 8% -10.3 0.00 perf-profile.calltrace.cycles-pp.lo_ioctl.blkdev_ioctl.block_ioctl.__x64_sys_ioctl.do_syscall_64
8.22 ± 8% -8.2 0.00 perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.printk.set_capacity_and_notify.cold.loop_set_size
7.64 ± 8% -7.6 0.00 perf-profile.calltrace.cycles-pp.serial8250_console_write.console_unlock.vprintk_emit.printk.set_capacity_and_notify.cold
7.29 ± 8% -7.0 0.33 ±124% perf-profile.calltrace.cycles-pp.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit.printk
6.73 ± 8% -6.4 0.31 ±123% perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock
6.74 ± 8% -6.4 0.31 ±123% perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit
5.37 ± 5% -5.4 0.00 perf-profile.calltrace.cycles-pp.loop_configure.lo_ioctl.blkdev_ioctl.block_ioctl.__x64_sys_ioctl
5.32 ± 5% -5.3 0.00 perf-profile.calltrace.cycles-pp.printk.set_capacity_and_notify.cold.loop_set_size.loop_configure.lo_ioctl
5.32 ± 5% -5.3 0.00 perf-profile.calltrace.cycles-pp.vprintk_emit.printk.set_capacity_and_notify.cold.loop_set_size.loop_configure
5.32 ± 5% -5.3 0.00 perf-profile.calltrace.cycles-pp.loop_set_size.loop_configure.lo_ioctl.blkdev_ioctl.block_ioctl
5.32 ± 5% -5.3 0.00 perf-profile.calltrace.cycles-pp.set_capacity_and_notify.cold.loop_set_size.loop_configure.lo_ioctl.blkdev_ioctl
5.57 ± 10% -5.3 0.26 ±123% perf-profile.calltrace.cycles-pp.io_serial_in.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write
0.66 ± 7% +0.5 1.19 ± 15% perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
0.57 ± 4% +0.5 1.11 ± 17% perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.72 ± 18% +0.6 1.30 ± 13% perf-profile.calltrace.cycles-pp.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.86 ± 7% +0.6 1.45 ± 22% perf-profile.calltrace.cycles-pp.lapic_next_deadline.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
0.00 +0.7 0.72 ± 11% perf-profile.calltrace.cycles-pp.load_balance.rebalance_domains.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt
0.75 ± 5% +0.8 1.55 ± 9% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.96 ± 14% +0.8 1.78 ± 13% perf-profile.calltrace.cycles-pp.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
1.02 ± 13% +0.8 1.87 ± 14% perf-profile.calltrace.cycles-pp.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
1.17 ± 7% +0.9 2.03 ± 11% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
1.71 ± 5% +1.1 2.84 ± 15% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
2.19 ± 5% +1.6 3.75 ± 16% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
2.16 ± 5% +1.7 3.82 ± 16% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
2.22 ± 5% +1.8 4.07 ± 19% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
2.08 ± 7% +2.0 4.04 ± 26% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary
2.56 ± 5% +2.2 4.74 ± 15% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
3.80 ± 3% +3.4 7.16 ± 18% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
7.75 ± 2% +6.2 13.98 ± 10% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
7.89 ± 2% +6.4 14.24 ± 10% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
12.29 ± 2% +9.7 22.01 ± 10% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
5.35 ± 15% +10.1 15.48 ± 30% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
57.16 +16.8 73.99 ± 8% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
17.95 ± 4% +18.3 36.22 ± 18% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
59.09 +20.6 79.73 ± 5% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
65.46 +31.4 96.82 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
65.49 +31.4 96.89 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
65.49 +31.4 96.89 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
66.20 +31.9 98.14 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
20.79 ± 2% -19.8 1.00 ± 21% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
18.43 ± 3% -17.5 0.97 ± 21% perf-profile.children.cycles-pp.do_syscall_64
13.86 ± 5% -13.9 0.00 perf-profile.children.cycles-pp.__x64_sys_ioctl
12.17 ± 3% -11.8 0.40 ± 31% perf-profile.children.cycles-pp.ret_from_fork
12.15 ± 3% -11.8 0.40 ± 32% perf-profile.children.cycles-pp.kthread
11.77 ± 3% -11.5 0.22 ± 40% perf-profile.children.cycles-pp.worker_thread
11.69 ± 3% -11.5 0.21 ± 44% perf-profile.children.cycles-pp.process_one_work
11.10 ± 3% -10.9 0.19 ± 51% perf-profile.children.cycles-pp.drm_fb_helper_damage_work
11.08 ± 3% -10.9 0.19 ± 51% perf-profile.children.cycles-pp.memcpy_toio
10.33 ± 8% -10.3 0.00 perf-profile.children.cycles-pp.block_ioctl
10.33 ± 8% -10.3 0.00 perf-profile.children.cycles-pp.blkdev_ioctl
10.33 ± 8% -10.3 0.00 perf-profile.children.cycles-pp.lo_ioctl
9.80 ± 9% -9.8 0.00 perf-profile.children.cycles-pp.loop_set_size
9.80 ± 9% -9.8 0.00 perf-profile.children.cycles-pp.set_capacity_and_notify.cold
10.12 ± 9% -9.3 0.77 ± 47% perf-profile.children.cycles-pp.printk
10.12 ± 9% -9.3 0.77 ± 47% perf-profile.children.cycles-pp.vprintk_emit
8.50 ± 8% -7.7 0.77 ± 47% perf-profile.children.cycles-pp.console_unlock
7.91 ± 8% -7.2 0.74 ± 47% perf-profile.children.cycles-pp.serial8250_console_write
7.55 ± 8% -6.8 0.71 ± 47% perf-profile.children.cycles-pp.uart_console_write
7.33 ± 8% -6.6 0.69 ± 47% perf-profile.children.cycles-pp.wait_for_xmitr
6.52 ± 12% -6.5 0.00 perf-profile.children.cycles-pp.__mutex_lock
6.99 ± 8% -6.3 0.67 ± 47% perf-profile.children.cycles-pp.serial8250_console_putchar
6.03 ± 9% -5.4 0.59 ± 43% perf-profile.children.cycles-pp.io_serial_in
5.37 ± 5% -5.4 0.00 perf-profile.children.cycles-pp.loop_configure
3.14 ± 16% -3.0 0.13 ± 31% perf-profile.children.cycles-pp.do_sys_open
3.13 ± 16% -3.0 0.13 ± 31% perf-profile.children.cycles-pp.do_sys_openat2
2.99 ± 16% -2.9 0.11 ± 27% perf-profile.children.cycles-pp.do_filp_open
2.98 ± 16% -2.9 0.11 ± 27% perf-profile.children.cycles-pp.path_openat
1.29 ± 16% -1.2 0.10 ± 76% perf-profile.children.cycles-pp.delay_tsc
0.58 ± 16% -0.5 0.04 ± 85% perf-profile.children.cycles-pp.io_serial_out
0.47 ± 16% -0.3 0.16 ± 45% perf-profile.children.cycles-pp.__schedule
0.37 ± 17% -0.3 0.11 ± 52% perf-profile.children.cycles-pp.schedule
0.30 ± 16% -0.2 0.06 ± 52% perf-profile.children.cycles-pp.walk_component
0.26 ± 15% -0.2 0.07 ± 50% perf-profile.children.cycles-pp.link_path_walk
0.23 ± 14% -0.2 0.07 ± 77% perf-profile.children.cycles-pp.newidle_balance
0.30 ± 13% -0.2 0.15 ± 30% perf-profile.children.cycles-pp.ksys_read
0.30 ± 15% -0.2 0.14 ± 29% perf-profile.children.cycles-pp.vfs_read
0.20 ± 12% -0.1 0.07 ± 55% perf-profile.children.cycles-pp.new_sync_read
0.01 ±223% +0.1 0.06 ± 18% perf-profile.children.cycles-pp.unmap_page_range
0.03 ±102% +0.1 0.09 ± 27% perf-profile.children.cycles-pp.__do_sys_clone
0.01 ±223% +0.1 0.07 ± 11% perf-profile.children.cycles-pp.unmap_vmas
0.05 ± 75% +0.1 0.11 ± 22% perf-profile.children.cycles-pp.__libc_fork
0.02 ±141% +0.1 0.08 ± 23% perf-profile.children.cycles-pp.rcu_needs_cpu
0.14 ± 19% +0.1 0.21 ± 14% perf-profile.children.cycles-pp.hrtimer_get_next_event
0.04 ± 77% +0.1 0.11 ± 29% perf-profile.children.cycles-pp.do_fault
0.08 ± 22% +0.1 0.15 ± 23% perf-profile.children.cycles-pp.rcu_dynticks_eqs_enter
0.02 ±144% +0.1 0.09 ± 38% perf-profile.children.cycles-pp.begin_new_exec
0.02 ±143% +0.1 0.10 ± 27% perf-profile.children.cycles-pp.filemap_map_pages
0.07 ± 47% +0.1 0.14 ± 28% perf-profile.children.cycles-pp.mmput
0.07 ± 47% +0.1 0.14 ± 28% perf-profile.children.cycles-pp.exit_mmap
0.09 ± 17% +0.1 0.17 ± 34% perf-profile.children.cycles-pp.trigger_load_balance
0.11 ± 23% +0.1 0.20 ± 21% perf-profile.children.cycles-pp.__handle_mm_fault
0.12 ± 23% +0.1 0.21 ± 19% perf-profile.children.cycles-pp.handle_mm_fault
0.13 ± 21% +0.1 0.22 ± 19% perf-profile.children.cycles-pp.exc_page_fault
0.13 ± 21% +0.1 0.22 ± 19% perf-profile.children.cycles-pp.do_user_addr_fault
0.08 ± 23% +0.1 0.17 ± 37% perf-profile.children.cycles-pp.exec_binprm
0.07 ± 26% +0.1 0.17 ± 37% perf-profile.children.cycles-pp.load_elf_binary
0.14 ± 21% +0.1 0.25 ± 23% perf-profile.children.cycles-pp.asm_exc_page_fault
0.13 ± 12% +0.1 0.24 ± 22% perf-profile.children.cycles-pp.rcu_eqs_enter
0.10 ± 14% +0.1 0.21 ± 61% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.09 ± 25% +0.1 0.21 ± 28% perf-profile.children.cycles-pp.bprm_execve
0.13 ± 22% +0.1 0.25 ± 48% perf-profile.children.cycles-pp.__hrtimer_get_next_event
0.12 ± 18% +0.1 0.24 ± 27% perf-profile.children.cycles-pp.execve
0.12 ± 18% +0.1 0.24 ± 27% perf-profile.children.cycles-pp.__x64_sys_execve
0.12 ± 18% +0.1 0.24 ± 27% perf-profile.children.cycles-pp.do_execveat_common
0.14 ± 9% +0.2 0.31 ± 40% perf-profile.children.cycles-pp.timerqueue_add
0.13 ± 6% +0.2 0.30 ± 65% perf-profile.children.cycles-pp.timerqueue_del
0.00 +0.2 0.19 ± 91% perf-profile.children.cycles-pp.run_posix_cpu_timers
0.61 ± 7% +0.2 0.80 ± 16% perf-profile.children.cycles-pp.load_balance
0.20 ± 12% +0.2 0.39 ± 28% perf-profile.children.cycles-pp.update_irq_load_avg
0.17 ± 12% +0.2 0.36 ± 33% perf-profile.children.cycles-pp.enqueue_hrtimer
0.17 ± 12% +0.2 0.37 ± 64% perf-profile.children.cycles-pp.__remove_hrtimer
0.30 ± 13% +0.2 0.51 ± 13% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.19 ± 22% +0.2 0.42 ± 23% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
0.19 ± 21% +0.2 0.43 ± 25% perf-profile.children.cycles-pp.arch_cpu_idle_enter
0.24 ± 18% +0.2 0.48 ± 34% perf-profile.children.cycles-pp.__hrtimer_next_event_base
0.23 ± 10% +0.2 0.48 ± 40% perf-profile.children.cycles-pp.hrtimer_update_next_event
0.34 ± 4% +0.2 0.58 ± 19% perf-profile.children.cycles-pp.update_rq_clock
0.32 ± 14% +0.3 0.57 ± 9% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.52 ± 6% +0.4 0.91 ± 14% perf-profile.children.cycles-pp.native_sched_clock
0.54 ± 5% +0.4 0.96 ± 17% perf-profile.children.cycles-pp.sched_clock
0.32 ± 29% +0.4 0.77 ± 47% perf-profile.children.cycles-pp.irq_work_single
0.61 ± 7% +0.4 1.06 ± 11% perf-profile.children.cycles-pp.read_tsc
0.32 ± 31% +0.5 0.77 ± 47% perf-profile.children.cycles-pp.asm_sysvec_irq_work
0.32 ± 31% +0.5 0.77 ± 47% perf-profile.children.cycles-pp.sysvec_irq_work
0.32 ± 31% +0.5 0.77 ± 47% perf-profile.children.cycles-pp.__sysvec_irq_work
0.32 ± 30% +0.5 0.77 ± 47% perf-profile.children.cycles-pp.irq_work_run
0.34 ± 28% +0.5 0.81 ± 46% perf-profile.children.cycles-pp.irq_work_run_list
0.61 ± 9% +0.5 1.12 ± 16% perf-profile.children.cycles-pp.sched_clock_cpu
0.60 ± 4% +0.5 1.13 ± 17% perf-profile.children.cycles-pp.ktime_get_update_offsets_now
0.69 ± 8% +0.5 1.23 ± 14% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.72 ± 8% +0.6 1.34 ± 24% perf-profile.children.cycles-pp.irqtime_account_irq
0.89 ± 7% +0.7 1.54 ± 23% perf-profile.children.cycles-pp.lapic_next_deadline
0.77 ± 4% +0.8 1.58 ± 8% perf-profile.children.cycles-pp.rebalance_domains
0.98 ± 14% +0.9 1.84 ± 14% perf-profile.children.cycles-pp.tick_irq_enter
1.23 ± 6% +0.9 2.09 ± 11% perf-profile.children.cycles-pp.scheduler_tick
1.02 ± 13% +0.9 1.91 ± 14% perf-profile.children.cycles-pp.irq_enter_rcu
1.81 ± 3% +1.1 2.90 ± 15% perf-profile.children.cycles-pp.__softirqentry_text_start
2.24 ± 4% +1.6 3.82 ± 16% perf-profile.children.cycles-pp.irq_exit_rcu
2.25 ± 4% +1.6 3.89 ± 16% perf-profile.children.cycles-pp.update_process_times
2.29 ± 4% +1.8 4.11 ± 19% perf-profile.children.cycles-pp.tick_sched_handle
2.12 ± 7% +2.0 4.10 ± 25% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
2.65 ± 5% +2.2 4.82 ± 15% perf-profile.children.cycles-pp.tick_sched_timer
3.94 ± 3% +3.3 7.26 ± 18% perf-profile.children.cycles-pp.__hrtimer_run_queues
7.97 ± 2% +6.2 14.18 ± 10% perf-profile.children.cycles-pp.hrtimer_interrupt
8.09 ± 2% +6.3 14.43 ± 10% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
12.58 ± 2% +9.7 22.32 ± 10% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
5.43 ± 16% +10.3 15.69 ± 29% perf-profile.children.cycles-pp.menu_select
15.95 ± 3% +13.7 29.69 ± 13% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
59.71 +21.0 80.76 ± 5% perf-profile.children.cycles-pp.cpuidle_enter_state
59.73 +21.1 80.81 ± 5% perf-profile.children.cycles-pp.cpuidle_enter
65.49 +31.4 96.89 perf-profile.children.cycles-pp.start_secondary
66.20 +31.9 98.14 perf-profile.children.cycles-pp.secondary_startup_64_no_verify
66.20 +31.9 98.14 perf-profile.children.cycles-pp.cpu_startup_entry
66.20 +31.9 98.14 perf-profile.children.cycles-pp.do_idle
10.98 ± 3% -10.8 0.19 ± 51% perf-profile.self.cycles-pp.memcpy_toio
6.03 ± 9% -5.4 0.59 ± 43% perf-profile.self.cycles-pp.io_serial_in
1.29 ± 16% -1.2 0.10 ± 76% perf-profile.self.cycles-pp.delay_tsc
0.58 ± 16% -0.5 0.04 ± 85% perf-profile.self.cycles-pp.io_serial_out
0.06 ± 52% +0.1 0.11 ± 29% perf-profile.self.cycles-pp.tick_sched_timer
0.02 ±141% +0.1 0.08 ± 23% perf-profile.self.cycles-pp.rcu_needs_cpu
0.08 ± 12% +0.1 0.15 ± 21% perf-profile.self.cycles-pp.load_balance
0.08 ± 22% +0.1 0.15 ± 23% perf-profile.self.cycles-pp.rcu_dynticks_eqs_enter
0.08 ± 13% +0.1 0.15 ± 32% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.00 +0.1 0.09 ± 23% perf-profile.self.cycles-pp.sched_clock
0.07 ± 14% +0.1 0.16 ± 13% perf-profile.self.cycles-pp.tick_irq_enter
0.07 ± 23% +0.1 0.17 ± 63% perf-profile.self.cycles-pp.sysvec_apic_timer_interrupt
0.16 ± 19% +0.1 0.26 ± 18% perf-profile.self.cycles-pp.asm_sysvec_apic_timer_interrupt
0.08 ± 14% +0.1 0.19 ± 33% perf-profile.self.cycles-pp.timerqueue_add
0.07 ± 59% +0.1 0.19 ± 49% perf-profile.self.cycles-pp.scheduler_tick
0.14 ± 20% +0.1 0.27 ± 9% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler
0.15 ± 12% +0.1 0.29 ± 39% perf-profile.self.cycles-pp.__hrtimer_run_queues
0.17 ± 16% +0.2 0.33 ± 14% perf-profile.self.cycles-pp.hrtimer_interrupt
0.09 ± 23% +0.2 0.25 ± 20% perf-profile.self.cycles-pp.rebalance_domains
0.00 +0.2 0.19 ± 91% perf-profile.self.cycles-pp.run_posix_cpu_timers
0.20 ± 12% +0.2 0.39 ± 29% perf-profile.self.cycles-pp.update_irq_load_avg
0.19 ± 22% +0.2 0.40 ± 27% perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
0.32 ± 13% +0.2 0.53 ± 8% perf-profile.self.cycles-pp.do_idle
0.19 ± 20% +0.2 0.42 ± 33% perf-profile.self.cycles-pp.__hrtimer_next_event_base
0.42 ± 10% +0.3 0.71 ± 29% perf-profile.self.cycles-pp.tick_nohz_next_event
0.50 ± 5% +0.3 0.85 ± 16% perf-profile.self.cycles-pp.native_sched_clock
0.44 ± 12% +0.4 0.82 ± 26% perf-profile.self.cycles-pp.irqtime_account_irq
0.42 ± 13% +0.4 0.81 ± 13% perf-profile.self.cycles-pp.update_process_times
0.60 ± 7% +0.4 1.02 ± 12% perf-profile.self.cycles-pp.read_tsc
0.50 ± 7% +0.5 0.96 ± 20% perf-profile.self.cycles-pp.ktime_get_update_offsets_now
0.89 ± 7% +0.6 1.54 ± 23% perf-profile.self.cycles-pp.lapic_next_deadline
3.09 ± 27% +8.1 11.17 ± 35% perf-profile.self.cycles-pp.menu_select
perf-sched.total_wait_time.max.ms
10000 +-------------------------------------------------------------------+
9000 |-+ |
| |
8000 |-+ |
7000 |-+ |
| |
6000 |-+ |
5000 |-+ |
4000 |-+ |
| |
3000 |-+ |
2000 |-+ |
| |
1000 |-+ |
0 +-------------------------------------------------------------------+
perf-sched.total_wait_time.average.ms
300 +---------------------------------------------------------------------+
| |
250 |-+ O O O O OOOO |
| O OO O OO OO OO O O OO OO O O |
|OO O O O |
200 |-+ O O O |
| |
150 |-+ |
| |
100 |-+ |
| |
| |
50 |-+ |
| |
0 +---------------------------------------------------------------------+
perf-sched.total_wait_and_delay.count.ms
8000 +--------------------------------------------------------------------+
| O |
7000 |OO O O OOO |
6000 |-+OOOOO OO OO OO OO OO O OO O |
| O O O O O |
5000 |-+ |
| |
4000 |-+ |
| |
3000 |-+ |
2000 |-+ |
| |
1000 |-+ |
|+ +++ ++.+++ +++ ++++++.+++ +++ +++ +++ .++++ ++++++++ +.++++++++|
0 +--------------------------------------------------------------------+
perf-sched.total_wait_and_delay.max.ms
10000 +-------------------------------------------------------------------+
9000 |-+ |
| |
8000 |-+ |
7000 |-+ |
| |
6000 |-+ |
5000 |-+ |
4000 |-+ |
| |
3000 |-+ |
2000 |-+ |
| |
1000 |-+ |
0 +-------------------------------------------------------------------+
perf-sched.total_wait_and_delay.average.ms
300 +---------------------------------------------------------------------+
| |
250 |-+ O O O O O O OOOO |
| O OO O O O OO O O OO OO O O |
|OO O O O |
200 |-+ O O O |
| |
150 |-+ |
| |
100 |-+ |
| |
| |
50 |-+ |
| |
0 +---------------------------------------------------------------------+
perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
1200 +--------------------------------------------------------------------+
| |
1000 |OOOOOO OOOOOOOOOOOOO OOOOOOOOOOOO OOO |
| |
| |
800 |-+ |
| |
600 |-+ |
| |
400 |-+ |
| |
| |
200 |-+ |
| |
0 +--------------------------------------------------------------------+
perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
90 +----------------------------------------------------------------------+
| O OOOOOO O O O O O O |
80 |-+ O OO O O O O O |
70 |-+ OO O O O O O |
| |
60 |-+ |
50 |-+ |
| |
40 |-+ |
30 |-+ |
| |
20 |-+ |
10 |-+ |
| |
0 +----------------------------------------------------------------------+
perf-sched.wait_and_delay.count.pipe_read.new_sync_read.vfs_read.ksys_read
2500 +--------------------------------------------------------------------+
| O O O |
| O O OO OO OO OOOO |
2000 |-O O OOOOO O O O O |
|O OO O O O OO |
| |
1500 |-+ |
| |
1000 |-+ |
| |
| |
500 |-+ |
| + + +++ + ++ |
| +++ ++.+++ + ++ ::+.++ +++ ++ +++ .+ :+++ ++ + :+ : + +++|
0 +--------------------------------------------------------------------+
perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
1200 +--------------------------------------------------------------------+
| |
1000 |OOOOOOOO OOOOOOOOOOOOOOOO OOOOOOOOOO |
| |
| |
800 |-+ |
| |
600 |-+ |
| |
400 |-+ |
| |
| |
200 |-+ |
| |
0 +--------------------------------------------------------------------+
perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
90 +----------------------------------------------------------------------+
| O OOOOOO O O O O O O |
80 |-+ O OO O O OO O |
70 |-+ O O O O O O O |
| |
60 |-+ |
50 |-+ |
| |
40 |-+ |
30 |-+ |
| |
20 |-+ |
10 |-+ |
| |
0 +----------------------------------------------------------------------+
perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
6000 +--------------------------------------------------------------------+
| O |
5000 |-+ O |
| |
| |
4000 |-+ |
| O |
3000 |-+ |
| |
2000 |-+ |
| |
| |
1000 |OO OOO OOOOOOOOOOOOO OOOOOOO OOO OOO |
| |
0 +--------------------------------------------------------------------+
perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
800 +---------------------------------------------------------------------+
|OOOOO OOOOOOOO O OOOOOOOOOOO O OO OO |
700 |-+ O O O O |
600 |-+ |
| |
500 |-+ |
| |
400 |-+ |
| |
300 |-+ |
200 |-+ |
| |
100 |-+ |
| |
0 +---------------------------------------------------------------------+
perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
1400 +--------------------------------------------------------------------+
|OOOO OO O OO OOOOO O O OOOOO O |
1200 |-+ OO O O OO OO OO OO |
| |
1000 |-+ |
| |
800 |-+ |
| |
600 |-+ |
| |
400 |-+ |
| |
200 |-+ + |
|++++++++.++++++++++++++++.++++++++++++++++.+++ ++++++++++++.++++++++|
0 +--------------------------------------------------------------------+
perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
6000 +--------------------------------------------------------------------+
| O |
5000 |-+ O |
| |
| |
4000 |-+ |
| O |
3000 |-+ |
| |
2000 |-+ |
| |
| |
1000 |OO OOOOO OOOOOOOOOOOOOOOO OO OOO OOO |
| |
0 +--------------------------------------------------------------------+
perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
800 +---------------------------------------------------------------------+
|OOOOOO OOOOOOO OOOOO OOOOOOOO OO O O |
700 |-+ O O O O |
600 |-+ |
| |
500 |-+ |
| |
400 |-+ |
| |
300 |-+ |
200 |-+ |
| |
100 |-+ |
| |
0 +---------------------------------------------------------------------+
10000 +-------------------------------------------------------------------+
9000 |-+ |
| |
8000 |-+ |
7000 |-+ |
| |
6000 |-+ |
5000 |-+ |
4000 |-+ |
| |
3000 |-+ |
2000 |-+ |
| |
1000 |-+ |
0 +-------------------------------------------------------------------+
20 +----------------------------------------------------------------------+
18 |O+OOO O O OOO OOO OOO OOO OOO OOO OO OOO OOO O |
| |
16 |-+ |
14 |-+ |
| |
12 |-+ |
10 |-+ |
8 |-+ |
| |
6 |-+ |
4 |-+ |
| |
2 |-+ |
0 +----------------------------------------------------------------------+
1100 +--------------------------------------------------------------------+
1000 |OOOO OO OOOOOOOOOO O O OOOOO O |
| OO OO OO OO OO |
900 |-+ |
800 |-+ |
700 |-+ |
600 |-+ |
| |
500 |-+ |
400 |-+ |
300 |-+ |
200 |-+ |
| |
100 |++++++++.++++++++++++++++.++++++++++++++++.++++++++++++++++.++++++++|
0 +--------------------------------------------------------------------+
10000 +-------------------------------------------------------------------+
9000 |-+ |
| |
8000 |-+ |
7000 |-+ |
| |
6000 |-+ |
5000 |-+ |
4000 |-+ |
| |
3000 |-+ |
2000 |-+ |
| |
1000 |-+ |
0 +-------------------------------------------------------------------+
20 +----------------------------------------------------------------------+
18 |O+OOO O O OOO OOO OOO OOO OOO OOO OO OOO OOO O |
| |
16 |-+ |
14 |-+ |
| |
12 |-+ |
10 |-+ |
8 |-+ |
| |
6 |-+ |
4 |-+ |
| |
2 |-+ |
0 +----------------------------------------------------------------------+
perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork
10000 +-------------------------------------------------------------------+
9000 |O+ OOO O O O |
| OO O O O OO OOOOOOOOOO OOO O OO |
8000 |-O O O O |
7000 |-+ |
| |
6000 |-+ |
5000 |-+ |
4000 |-+ |
| |
3000 |-+ |
2000 |-+ |
| |
1000 |-+ |
0 +-------------------------------------------------------------------+
perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
600 +---------------------------------------------------------------------+
| |
500 |-+ O O O |
| O O O O O O OO O O O O |
|O O O O OO OO O O OO OOO O O |
400 |-+ O O |
| |
300 |-+ |
| |
200 |-+ |
| |
| |
100 |-+ |
| |
0 +---------------------------------------------------------------------+
perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
700 +---------------------------------------------------------------------+
| O O |
600 |OOOOOO OO OOOOOOOO OO OOOOOOO O O |
| OO O O O |
500 |-+ |
| |
400 |-+ |
| |
300 |-+ |
| |
200 |-+ |
| |
100 |-+ |
| |
0 +---------------------------------------------------------------------+
perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork
10000 +-------------------------------------------------------------------+
9000 |O+ OOO O O O |
| OO O O O OO OOOOOOOOOO OOO O OO |
8000 |-O O O O |
7000 |-+ |
| |
6000 |-+ |
5000 |-+ |
4000 |-+ |
| |
3000 |-+ |
2000 |-+ |
| |
1000 |-+ |
0 +-------------------------------------------------------------------+
perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
600 +---------------------------------------------------------------------+
| |
500 |-+ O O O O |
| O O O O O O OO O O O |
|O O O O OO OO O O OO OOO O O |
400 |-+ O O |
| |
300 |-+ |
| |
200 |-+ |
| |
| |
100 |-+ |
| |
0 +---------------------------------------------------------------------+
2500 +--------------------------------------------------------------------+
| |
| O |
2000 |-+ |
| |
| |
1500 |-+ |
| O O |
1000 |-+ O |
|O |
| O O O O |
500 |-+ O OO |
| OO O OO OO OOOOOOO OOOOOOO O |
| |
0 +--------------------------------------------------------------------+
300 +---------------------------------------------------------------------+
|O O O O OOOO O O OOOO O OO |
250 |-O OOO O OO OO OO OO OO O O |
| |
| |
200 |-+ |
| |
150 |-+ |
| |
100 |-+ |
| |
| |
50 |-+ |
| |
0 +---------------------------------------------------------------------+
250 +---------------------------------------------------------------------+
| |
| |
200 |-+ |
| |
| |
150 |-+ |
| |
100 |-+ |
| |
| |
50 |-+ |
| |
| + + + + + + + + |
0 +---------------------------------------------------------------------+
300 +---------------------------------------------------------------------+
|O O O O OOOO O O OOOO O OO |
250 |-O OOO O OO OO OO OO OO O O |
| |
| |
200 |-+ |
| |
150 |-+ |
| |
100 |-+ |
| |
| |
50 |-+ |
| |
0 +---------------------------------------------------------------------+
perf-sched.wait_time.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
16 +----------------------------------------------------------------------+
| O O |
14 |OOOOO O OO OOOO OOOOO OO O O OOO O |
12 |-+ OO O O O O O |
| |
10 |-+ |
| |
8 |-+ |
| |
6 |-+ |
4 |-+ |
| |
2 |-+ |
| |
0 +----------------------------------------------------------------------+
perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.9 +---------------------------------------------------------------------+
| O O OO OO OO O O O OOO |
0.8 |O+O O O O O OO OO OO O |
0.7 |-+ OO O O O O O |
| |
0.6 |-+ |
0.5 |-+ |
| |
0.4 |-+ |
0.3 |-+ |
| |
0.2 |-+ |
0.1 |-+ |
| + + + + + |
0 +---------------------------------------------------------------------+
perf-sched.wait_and_delay.count.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_6
250 +---------------------------------------------------------------------+
|O O OOOOOO O O O |
| O O O O O O O |
200 |-+ |
| |
| |
150 |-+ |
| |
100 |-+ |
| |
| |
50 |-+ |
| |
| |
0 +---------------------------------------------------------------------+
16 +----------------------------------------------------------------------+
| O O |
14 |OOOOO O OO OOOO OOOOO OO O O OOO O |
12 |-+ OO O O O O O |
| |
10 |-+ |
| |
8 |-+ |
| |
6 |-+ |
4 |-+ |
| |
2 |-+ |
| |
0 +----------------------------------------------------------------------+
0.9 +---------------------------------------------------------------------+
|OOO O OO OO O O O |
0.8 |-+ O O O OO OO OO O |
0.7 |-+ OO O O O O O |
| |
0.6 |-+ |
0.5 |-+ |
| |
0.4 |-+ |
0.3 |-+ |
| |
0.2 |-+ |
0.1 |-+ |
| + + + + + + |
0 +---------------------------------------------------------------------+
perf-sched.sch_delay.max.ms.schedule_timeout.kcompactd.kthread.ret_from_fork
0.05 +-------------------------------------------------------------------+
0.045 |-+ O O |
| O |
0.04 |-+ O O |
0.035 |-+ O O |
| |
0.03 |-+ |
0.025 |OOOO OO OO O O O |
0.02 |-+ O O O |
| O O |
0.015 |-+ OO O O OO OO O OO |
0.01 |-+ |
| |
0.005 |-+ :|
0 +-------------------------------------------------------------------+
40 +----------------------------------------------------------------------+
| O O O O O O |
35 |-+ |
30 |-+ |
| |
25 |-+ |
| |
20 |-+ |
| |
15 |-+ |
10 |-+ |
| |
5 |-+ |
| |
0 +----------------------------------------------------------------------+
perf-sched.wait_and_delay.max.ms.schedule_timeout.kcompactd.kthread.ret_from_for
600 +---------------------------------------------------------------------+
| |
500 |OOOOO OOOOOOOOOO OOOOOOOOOOO OOOOOOOO |
| |
| |
400 |-+ |
| |
300 |-+ |
| |
200 |-+ |
| |
| |
100 |-+ |
| |
0 +---------------------------------------------------------------------+
perf-sched.wait_and_delay.avg.ms.schedule_timeout.kcompactd.kthread.ret_from_for
500 +---------------------------------------------------------------------+
450 |OOOOO OOOOOOOOOO OOOOOOOOOOO OOOOOOOO |
| |
400 |-+ |
350 |-+ |
| |
300 |-+ |
250 |-+ |
200 |-+ |
| |
150 |-+ |
100 |-+ |
| |
50 |-+ |
0 +---------------------------------------------------------------------+
50 +----------------------------------------------------------------------+
45 |-+ O O O O O O O |
| O |
40 |-+ OO O O O |
35 |-+ O O O O O |
| OO OO O O O OO |
30 |O+ O O O O |
25 |-+ |
20 |-O O |
| |
15 |-+ |
10 |-+ |
| |
5 |-+ |
0 +----------------------------------------------------------------------+
450 +---------------------------------------------------------------------+
| O |
400 |-+ |
350 |O+ O |
| O OO O O O OOOO |
300 |-+ O O O O O O |
250 |-+ O O O O O |
| O O O O O O |
200 |-+ O O O |
150 |-+ |
| |
100 |-+ |
50 |-+ |
| |
0 +---------------------------------------------------------------------+
450 +---------------------------------------------------------------------+
| O |
400 |-+ O |
350 |-+ |
|O O O O OO |
300 |-+OO OOO OO O OO |
250 |-+ O O O O OO O |
| O O O |
200 |-+ O O O O O O |
150 |-+ |
| |
100 |-+ |
50 |-+ |
| |
0 +---------------------------------------------------------------------+
20 +----------------------------------------------------------------------+
| |
| |
15 |-+ |
| |
| |
| |
10 |-+ |
| |
| |
5 |-+ |
| |
| |
| .+.+ |
0 +----------------------------------------------------------------------+
1000 +--------------------------------------------------------------------+
900 |-+ |
| |
800 |-+ |
700 |-+ |
| |
600 |-+ |
500 |-+ |
400 |-+ |
| |
300 |-+ |
200 |-+ |
| |
100 |-+ |
0 +--------------------------------------------------------------------+
900 +---------------------------------------------------------------------+
| |
800 |-+ |
700 |-+ |
| |
600 |-+ |
500 |-+ |
| |
400 |-+ |
300 |-+ |
| |
200 |-+ |
100 |-+ |
| |
0 +---------------------------------------------------------------------+
14 +----------------------------------------------------------------------+
|O |
12 |-+ O O OO |
| OO O O OOOOOO O OOOOOOOOOOO OOOOOOO |
10 |-+ |
| |
8 |-+ |
| |
6 |-+ |
| |
4 |-+ |
| |
2 |-+ |
| + +. + .+ + + + +.+ ++ ++ |
0 +----------------------------------------------------------------------+
1200 +--------------------------------------------------------------------+
| |
1000 |OOOO OOOOOOOOO OOOOOOOOO OOOOOOOO OOOO |
| |
| |
800 |-+ |
| |
600 |-+ |
| |
400 |-+ |
| |
| |
200 |-+ |
| |
0 +--------------------------------------------------------------------+
900 +---------------------------------------------------------------------+
| OO OO OOOO OO OOOO OOOOOOOO OOOOOOO |
800 |-+ O O OO |
700 |O+ |
| |
600 |-+ |
500 |-+ |
| |
400 |-+ |
300 |-+ |
| |
200 |-+ |
100 |-+ |
| |
0 +---------------------------------------------------------------------+
stress-ng.time.system_time
30 +----------------------------------------------------------------------+
| |
25 |-++ + + ++++ + + + + + ++++ + |
| ::+.+ ++ +++++ + + : ::: ::++ : :+ ++ +++ + + :++++++ .++ +|
|++ + + + + + ++.+ + +++ + + + + ++ |
20 |-+ |
| |
15 |-+ |
| |
10 |-+ |
| |
| |
5 |-+ |
| |
0 +----------------------------------------------------------------------+
stress-ng.time.elapsed_time
900 +---------------------------------------------------------------------+
| |
800 |-+ |
700 |-+O O OOOOOO O O O |
| |
600 |-+ |
500 |-O O O O O O OO OO |
| |
400 |-+ O O |
300 |O+ O O |
| |
200 |-+ |
100 |-+ |
|++++++.+++++++++++++.+++++++++++++.+++++++++++++.+++++++++++++.++++++|
0 +---------------------------------------------------------------------+
stress-ng.time.elapsed_time.max
900 +---------------------------------------------------------------------+
| |
800 |-+ |
700 |-+O O OOOOOO O O O |
| |
600 |-+ |
500 |-O O O O O O OO OO |
| |
400 |-+ O O |
300 |O+ O O |
| |
200 |-+ |
100 |-+ |
|++++++.+++++++++++++.+++++++++++++.+++++++++++++.+++++++++++++.++++++|
0 +---------------------------------------------------------------------+
stress-ng.time.voluntary_context_switches
90000 +-------------------------------------------------------------------+
| + |
80000 |-+ :+ + |
70000 |+++ .+ + : + + + + + :: + + + +|
|: :++ + : + : : : : :++ : :++ : + :: + + +: + :: :|
60000 |-:: + : : :+ : ++: +: :: +. : :: + + : + :: ::+ :+: :++ |
50000 |-: ++: ++ + : + + : ++ : + + +: +: + + : |
| + + + + + + + + |
40000 |-+ |
30000 |-+ |
| |
20000 |-+ |
10000 |-+ |
| O O O O |
0 +-------------------------------------------------------------------+
stress-ng.time.involuntary_context_switches
4500 +--------------------------------------------------------------------+
| + + +++ + ++ + + + + .+ ++++ + |
4000 |-+:: + +. +++ :+ :+ :+ : :: + : :: + : : :+ :++ +. + +|
3500 |++ ++ ++ + + + +++ ++ +++ ++ +++ + + + ++++ ++ |
| |
3000 |-+ |
2500 |-+ |
| |
2000 |-+ |
1500 |-+ |
| |
1000 |-+ |
500 |-+ |
| |
0 +--------------------------------------------------------------------+
stress-ng.loop.ops
1800 +--------------------------------------------------------------------+
| + + ++ + + + + + .+ + + |
1600 |-+:: + : + + :+ :+++ + + :: + + :: + : ++++ :+ :+ +.+ + |
1400 |++ ++ +.++ ++ + + + ++ ++ ++ ++ ++ + + ++ +++++ +|
| |
1200 |-+ |
1000 |-+ |
| |
800 |-+ |
600 |-+ |
| |
400 |-+ |
200 |-+ |
| |
0 +--------------------------------------------------------------------+
stress-ng.loop.ops_per_sec
30 +----------------------------------------------------------------------+
| + + ++ + + + + + + + + |
25 |-+:: .+ : + + + :+++ :: :: + : :+ + : ++++ + + :+ ++ + |
|++ ++ +++ ++ +++ + ++.+ ++ +++ ++ ++ + + ++ ++.+++ +|
| |
20 |-+ |
| |
15 |-+ |
| |
10 |-+ |
| |
| |
5 |-+ |
| |
0 +----------------------------------------------------------------------+
stress-ng.time.percent_of_cpu_this_job_got
42 +----------------------------------------------------------------------+
| : |
41 |-++ + : + + + + + |
40 |-+: : : : : ++ + : : : : + ++ + |
| : : : : :::: : : : :: : :::: : |
39 |-+: :: ++ : :+ : : : :: :: :: :+ : +: + + |
| :: : : : : : : : :: :: :: : : : : :: : : |
38 |-: : + : : : : : : : : + : : : + : : : :: :: : |
| : : : : : : : : : : : : : : : : : : : :: ::: : |
37 |-: : : : ++: : : : : : :: : : :: :+: : : ::+: : + +|
36 |-+ ++ ++ : + ++: : : + ++ : + ++ + + :: : + ++ +: + |
| : : : : : + : : :: : :+ :: |
35 |++ + + + ++ ++ : : + + |
| : : |
34 +----------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
---
0DAY/LKP+ Test Infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation
Thanks,
Oliver Sang
View attachment "config-5.12.0-rc4-00024-gc76f48eb5c08" of type "text/plain" (172837 bytes)
View attachment "job-script" of type "text/plain" (8116 bytes)
View attachment "job.yaml" of type "text/plain" (5668 bytes)
View attachment "reproduce" of type "text/plain" (532 bytes)
Powered by blists - more mailing lists