[<prev] [next>] [day] [month] [year] [list]
Message-ID: <20190916010021.GX15734@shao2-debian>
Date: Mon, 16 Sep 2019 09:00:22 +0800
From: kernel test robot <rong.a.chen@...el.com>
To: "Darrick J. Wong" <darrick.wong@...cle.com>
Cc: LKML <linux-kernel@...r.kernel.org>,
"Darrick J. Wong" <darrick.wong@...cle.com>, lkp@...org
Subject: [xfs] 73a16a2894: aim7.jobs-per-min -31.6% regression
Greeting,
FYI, we noticed a -31.6% regression of aim7.jobs-per-min due to commit:
commit: 73a16a2894549c9e544ef61aad3c30a60019a2cb ("xfs: parallelize inode inactivation")
https://kernel.googlesource.com/pub/scm/linux/kernel/git/djwong/xfs-linux.git rough-fixes
in testcase: aim7
on test machine: 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz with 384G memory
with following parameters:
disk: 4BRD_12G
md: RAID0
fs: xfs
test: disk_src
load: 3000
cpufreq_governor: performance
test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system.
test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen@...el.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase:
gcc-7/performance/4BRD_12G/xfs/x86_64-rhel-7.6/3000/RAID0/debian-x86_64-2019-05-14.cgz/lkp-ivb-ep01/disk_src/aim7
commit:
6298ae61d2 ("xfs: force inactivation before fallocate when space is low")
73a16a2894 ("xfs: parallelize inode inactivation")
6298ae61d2ecd22d 73a16a2894549c9e544ef61aad3
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
3:4 -75% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
:4 50% 2:4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
%stddev %change %stddev
\ | \
28682 -31.6% 19623 aim7.jobs-per-min
628.00 +46.1% 917.77 aim7.time.elapsed_time
628.00 +46.1% 917.77 aim7.time.elapsed_time.max
32590 ± 6% -35.9% 20896 ± 6% aim7.time.file_system_inputs
21342 +51.1% 32253 aim7.time.involuntary_context_switches
3491 +58.6% 5538 aim7.time.system_time
122.28 +8.1% 132.16 aim7.time.user_time
7211143 +64.6% 11870679 ± 2% aim7.time.voluntary_context_switches
1285 -1.2% 1269 boot-time.idle
6590 ± 70% +98.0% 13046 numa-numastat.node1.other_node
1.549e+10 ± 10% +43.6% 2.224e+10 ± 9% cpuidle.C6.time
26014805 ± 17% +61.0% 41883750 ± 19% cpuidle.C6.usage
365519 +66.2% 607360 cpuidle.POLL.time
23959 ± 3% +60.6% 38475 cpuidle.POLL.usage
84.05 -3.7% 80.90 iostat.cpu.idle
15.38 +21.3% 18.66 iostat.cpu.system
1296 -24.8% 975.49 iostat.md0.w/s
28242 -22.7% 21842 iostat.md0.wkB/s
2265 ± 41% +92.7% 4364 ± 28% numa-vmstat.node0.nr_shmem
479835 -94.1% 28115 ± 2% numa-vmstat.node0.nr_slab_reclaimable
323938 ± 2% -92.3% 24986 ± 32% numa-vmstat.node0.nr_slab_unreclaimable
488494 -95.4% 22470 ± 3% numa-vmstat.node1.nr_slab_reclaimable
315962 ± 2% -89.7% 32671 ± 24% numa-vmstat.node1.nr_slab_unreclaimable
255.50 +8.4% 277.00 ± 2% turbostat.Avg_MHz
20.01 +3.0 23.04 ± 2% turbostat.Busy%
26010933 ± 17% +61.0% 41879491 ± 19% turbostat.C6
51358007 +44.5% 74196457 turbostat.IRQ
1.18 ± 18% -36.8% 0.74 ± 11% turbostat.Pkg%pc2
50370 +46.0% 73540 turbostat.SMI
83.50 -3.9% 80.25 vmstat.cpu.id
27381 -22.8% 21144 vmstat.io.bo
4984658 -73.7% 1313263 vmstat.memory.cache
6.00 +16.7% 7.00 vmstat.procs.r
38725 +5.5% 40838 vmstat.system.cs
81494 -1.0% 80660 vmstat.system.in
3872245 -94.8% 202334 meminfo.KReclaimable
13158326 -61.0% 5130076 meminfo.Memused
101553 ± 5% +14.5% 116292 ± 8% meminfo.PageTables
3872245 -94.8% 202334 meminfo.SReclaimable
2558865 -91.0% 230631 meminfo.SUnreclaim
31509 +11.1% 35010 meminfo.Shmem
6431111 -93.3% 432966 meminfo.Slab
338337 -16.4% 282859 meminfo.VmallocUsed
37848 -78.1% 8278 meminfo.max_used_kB
1918514 -94.1% 112464 ± 2% numa-meminfo.node0.KReclaimable
6329865 ± 6% -58.8% 2610379 ± 6% numa-meminfo.node0.MemUsed
1918514 -94.1% 112464 ± 2% numa-meminfo.node0.SReclaimable
1295259 ± 2% -92.3% 99946 ± 32% numa-meminfo.node0.SUnreclaim
9059 ± 41% +92.7% 17459 ± 28% numa-meminfo.node0.Shmem
3213774 -93.4% 212411 ± 15% numa-meminfo.node0.Slab
1953128 -95.4% 89882 ± 3% numa-meminfo.node1.KReclaimable
6827591 ± 5% -63.1% 2519244 ± 5% numa-meminfo.node1.MemUsed
1953128 -95.4% 89882 ± 3% numa-meminfo.node1.SReclaimable
1263297 ± 2% -89.7% 130693 ± 24% numa-meminfo.node1.SUnreclaim
3216425 -93.1% 220576 ± 14% numa-meminfo.node1.Slab
9520889 +2.1% 9721313 proc-vmstat.nr_dirty_background_threshold
19065060 +2.1% 19466397 proc-vmstat.nr_dirty_threshold
95762281 +2.1% 97769457 proc-vmstat.nr_free_pages
5062 +2.5% 5191 proc-vmstat.nr_inactive_anon
25386 ± 5% +14.5% 29072 ± 8% proc-vmstat.nr_page_table_pages
7877 +11.1% 8752 proc-vmstat.nr_shmem
968099 -94.8% 50584 proc-vmstat.nr_slab_reclaimable
639746 -91.0% 57656 proc-vmstat.nr_slab_unreclaimable
5062 +2.5% 5191 proc-vmstat.nr_zone_inactive_anon
5050595 -3.7% 4861550 proc-vmstat.numa_hit
5037460 -3.8% 4848364 proc-vmstat.numa_local
1324 ± 11% +32.0% 1748 ± 9% proc-vmstat.pgactivate
7847951 -22.1% 6117074 proc-vmstat.pgalloc_normal
1764564 +39.9% 2469500 proc-vmstat.pgfault
2388604 +99.2% 4758851 proc-vmstat.pgfree
16475 ± 5% -35.5% 10624 ± 8% proc-vmstat.pgpgin
17259303 +12.8% 19464058 proc-vmstat.pgpgout
1879 +1.5% 1907 proc-vmstat.unevictable_pgs_culled
48666 +76.8% 86051 sched_debug.cfs_rq:/.exec_clock.avg
135393 -33.2% 90412 sched_debug.cfs_rq:/.exec_clock.max
26618 +215.6% 84021 sched_debug.cfs_rq:/.exec_clock.min
14312 -91.0% 1289 ± 9% sched_debug.cfs_rq:/.exec_clock.stddev
393911 ± 4% +96.7% 774700 ± 4% sched_debug.cfs_rq:/.min_vruntime.avg
579123 ± 2% +57.0% 909216 ± 2% sched_debug.cfs_rq:/.min_vruntime.max
159226 ± 10% +355.1% 724696 ± 4% sched_debug.cfs_rq:/.min_vruntime.min
65888 ± 12% -40.6% 39155 ± 30% sched_debug.cfs_rq:/.min_vruntime.stddev
1.12 ± 12% +49.9% 1.67 ± 32% sched_debug.cfs_rq:/.nr_spread_over.avg
0.45 +75.3% 0.80 ± 12% sched_debug.cfs_rq:/.nr_spread_over.min
18.27 ± 15% +72.7% 31.55 ± 8% sched_debug.cfs_rq:/.runnable_load_avg.avg
241590 ± 29% -43.5% 136534 ± 14% sched_debug.cfs_rq:/.spread0.max
-178307 -73.1% -48028 sched_debug.cfs_rq:/.spread0.min
65886 ± 12% -40.6% 39156 ± 30% sched_debug.cfs_rq:/.spread0.stddev
157.45 +17.3% 184.66 ± 3% sched_debug.cfs_rq:/.util_avg.stddev
23.29 ± 9% +71.3% 39.90 ± 10% sched_debug.cfs_rq:/.util_est_enqueued.avg
78.72 ± 7% +46.0% 114.96 ± 11% sched_debug.cfs_rq:/.util_est_enqueued.stddev
93948 ± 9% +146.0% 231107 ± 6% sched_debug.cpu.avg_idle.min
343542 +44.8% 497596 sched_debug.cpu.clock.avg
343569 +44.8% 497627 sched_debug.cpu.clock.max
343513 +44.8% 497564 sched_debug.cpu.clock.min
343542 +44.8% 497596 sched_debug.cpu.clock_task.avg
343569 +44.8% 497627 sched_debug.cpu.clock_task.max
343513 +44.8% 497564 sched_debug.cpu.clock_task.min
11973 +31.8% 15786 sched_debug.cpu.curr->pid.max
2293 +20.8% 2771 ± 2% sched_debug.cpu.curr->pid.stddev
291435 +59.6% 465252 sched_debug.cpu.nr_switches.avg
3322828 -84.8% 506717 sched_debug.cpu.nr_switches.max
123943 ± 6% +243.8% 426081 ± 2% sched_debug.cpu.nr_switches.min
485747 -96.4% 17504 ± 4% sched_debug.cpu.nr_switches.stddev
154.41 ± 6% +25.5% 193.75 ± 9% sched_debug.cpu.nr_uninterruptible.max
45.66 ± 15% +44.5% 65.97 ± 9% sched_debug.cpu.nr_uninterruptible.stddev
289211 +60.1% 462992 sched_debug.cpu.sched_count.avg
3319749 -84.8% 503177 sched_debug.cpu.sched_count.max
121121 ± 7% +249.9% 423747 ± 2% sched_debug.cpu.sched_count.min
485550 -96.5% 17224 ± 4% sched_debug.cpu.sched_count.stddev
129527 +68.6% 218367 ± 2% sched_debug.cpu.sched_goidle.avg
1495010 -84.2% 236513 ± 2% sched_debug.cpu.sched_goidle.max
56307 ± 7% +256.6% 200766 ± 2% sched_debug.cpu.sched_goidle.min
218759 -96.4% 7832 ± 5% sched_debug.cpu.sched_goidle.stddev
146549 +59.7% 233968 sched_debug.cpu.ttwu_count.avg
1376890 -81.6% 253020 ± 2% sched_debug.cpu.ttwu_count.max
66441 ± 5% +221.1% 213322 sched_debug.cpu.ttwu_count.min
197195 -95.6% 8696 ± 3% sched_debug.cpu.ttwu_count.stddev
14892 +23.4% 18370 sched_debug.cpu.ttwu_local.avg
3035 ± 15% +355.4% 13824 ± 2% sched_debug.cpu.ttwu_local.min
4226 ± 3% -39.2% 2569 ± 6% sched_debug.cpu.ttwu_local.stddev
343511 +44.8% 497561 sched_debug.cpu_clk
339915 +45.3% 493967 sched_debug.ktime
343869 +44.8% 497922 sched_debug.sched_clk
3715816 -98.4% 61244 slabinfo.dmaengine-unmap-16.active_objs
88703 -97.7% 2027 slabinfo.dmaengine-unmap-16.active_slabs
3725580 -97.7% 85200 slabinfo.dmaengine-unmap-16.num_objs
88703 -97.7% 2027 slabinfo.dmaengine-unmap-16.num_slabs
933.75 ± 6% -8.5% 854.75 ± 7% slabinfo.file_lock_cache.active_objs
933.75 ± 6% -8.5% 854.75 ± 7% slabinfo.file_lock_cache.num_objs
11739 -10.7% 10482 slabinfo.filp.num_objs
3550 ± 3% +187.1% 10192 slabinfo.kmalloc-128.active_objs
110.25 ± 3% +189.8% 319.50 slabinfo.kmalloc-128.active_slabs
3550 ± 3% +188.6% 10244 slabinfo.kmalloc-128.num_objs
110.25 ± 3% +189.8% 319.50 slabinfo.kmalloc-128.num_slabs
3699063 -97.6% 89992 slabinfo.kmalloc-16.active_objs
14449 -97.3% 384.75 slabinfo.kmalloc-16.active_slabs
3699207 -97.3% 98606 slabinfo.kmalloc-16.num_objs
14449 -97.3% 384.75 slabinfo.kmalloc-16.num_slabs
4338 ± 2% +33.2% 5779 ± 2% slabinfo.kmalloc-1k.active_objs
4357 ± 2% +32.7% 5780 ± 2% slabinfo.kmalloc-1k.num_objs
129558 -85.0% 19424 ± 6% slabinfo.kmalloc-32.active_objs
1011 -84.9% 152.75 ± 6% slabinfo.kmalloc-32.active_slabs
129579 -84.8% 19638 ± 6% slabinfo.kmalloc-32.num_objs
1011 -84.9% 152.75 ± 6% slabinfo.kmalloc-32.num_slabs
3138133 -98.7% 41306 slabinfo.kmalloc-512.active_objs
98102 -98.1% 1906 slabinfo.kmalloc-512.active_slabs
3139296 -98.1% 61014 slabinfo.kmalloc-512.num_objs
98102 -98.1% 1906 slabinfo.kmalloc-512.num_slabs
288.50 ± 9% +46.0% 421.25 slabinfo.nfs_read_data.active_objs
288.50 ± 9% +46.0% 421.25 slabinfo.nfs_read_data.num_objs
83301 -70.8% 24296 slabinfo.radix_tree_node.active_objs
2981 -70.5% 878.50 slabinfo.radix_tree_node.active_slabs
83509 -70.5% 24607 slabinfo.radix_tree_node.num_objs
2981 -70.5% 878.50 slabinfo.radix_tree_node.num_slabs
15967 +19.7% 19115 slabinfo.vmap_area.active_objs
16696 +17.5% 19610 slabinfo.vmap_area.num_objs
324.00 ± 19% +49.7% 485.00 ± 3% slabinfo.xfrm_state.active_objs
324.00 ± 19% +49.7% 485.00 ± 3% slabinfo.xfrm_state.num_objs
116536 -93.7% 7313 ± 3% slabinfo.xfs_buf.active_objs
2774 -93.5% 180.25 ± 3% slabinfo.xfs_buf.active_slabs
116547 -93.5% 7597 ± 3% slabinfo.xfs_buf.num_objs
2774 -93.5% 180.25 ± 3% slabinfo.xfs_buf.num_slabs
9169 -35.4% 5923 slabinfo.xfs_buf_item.active_objs
310.25 -35.4% 200.50 slabinfo.xfs_buf_item.active_slabs
9315 -35.3% 6028 slabinfo.xfs_buf_item.num_objs
310.25 -35.4% 200.50 slabinfo.xfs_buf_item.num_slabs
1966 ± 4% +226.1% 6411 ± 5% slabinfo.xfs_efd_item.active_objs
52.75 ± 4% +230.3% 174.25 ± 5% slabinfo.xfs_efd_item.active_slabs
1966 ± 4% +228.8% 6464 ± 5% slabinfo.xfs_efd_item.num_objs
52.75 ± 4% +230.3% 174.25 ± 5% slabinfo.xfs_efd_item.num_slabs
3715508 -98.4% 60982 slabinfo.xfs_inode.active_objs
116415 -97.7% 2626 slabinfo.xfs_inode.active_slabs
3725291 -97.7% 84066 slabinfo.xfs_inode.num_objs
116415 -97.7% 2626 slabinfo.xfs_inode.num_slabs
16.65 ± 2% -36.6% 10.55 ± 5% perf-stat.i.MPKI
1.507e+09 +28.4% 1.935e+09 perf-stat.i.branch-instructions
3.14 -0.6 2.54 perf-stat.i.branch-miss-rate%
15.46 ± 2% -2.1 13.36 ± 4% perf-stat.i.cache-miss-rate%
17898559 ± 3% -34.4% 11746315 ± 7% perf-stat.i.cache-misses
1.131e+08 ± 2% -21.2% 89131641 ± 5% perf-stat.i.cache-references
38840 +5.3% 40916 perf-stat.i.context-switches
1.60 ± 2% -20.5% 1.27 ± 3% perf-stat.i.cpi
561.37 +65.4% 928.27 ± 7% perf-stat.i.cpu-migrations
669.68 ± 2% +43.5% 961.21 ± 5% perf-stat.i.cycles-between-cache-misses
0.62 ± 5% -0.3 0.28 ± 3% perf-stat.i.dTLB-load-miss-rate%
11040116 ± 5% -40.3% 6594857 ± 4% perf-stat.i.dTLB-load-misses
1.823e+09 +42.1% 2.59e+09 perf-stat.i.dTLB-loads
0.15 ± 4% -0.1 0.07 ± 7% perf-stat.i.dTLB-store-miss-rate%
1176632 ± 3% -34.2% 773684 ± 9% perf-stat.i.dTLB-store-misses
7.961e+08 ± 2% +48.9% 1.185e+09 ± 5% perf-stat.i.dTLB-stores
41.74 ± 2% +1.8 43.58 perf-stat.i.iTLB-load-miss-rate%
1680621 ± 2% -9.0% 1529900 ± 3% perf-stat.i.iTLB-load-misses
2421937 ± 4% -14.1% 2079312 ± 2% perf-stat.i.iTLB-loads
7.145e+09 +33.7% 9.555e+09 perf-stat.i.instructions
4232 ± 2% +46.3% 6193 ± 3% perf-stat.i.instructions-per-iTLB-miss
0.65 +27.9% 0.83 ± 3% perf-stat.i.ipc
2724 -3.7% 2622 perf-stat.i.minor-faults
42.83 +1.5 44.34 perf-stat.i.node-load-miss-rate%
6361451 -47.5% 3336831 perf-stat.i.node-load-misses
9312635 -55.1% 4185756 ± 2% perf-stat.i.node-loads
2772972 -25.5% 2064781 perf-stat.i.node-store-misses
4138412 -27.1% 3017312 perf-stat.i.node-stores
2724 -3.7% 2622 perf-stat.i.page-faults
15.83 ± 2% -41.1% 9.33 ± 4% perf-stat.overall.MPKI
2.96 -0.6 2.33 perf-stat.overall.branch-miss-rate%
15.82 ± 2% -2.7 13.17 ± 4% perf-stat.overall.cache-miss-rate%
1.52 -22.3% 1.18 ± 2% perf-stat.overall.cpi
608.08 ± 2% +58.9% 966.12 ± 5% perf-stat.overall.cycles-between-cache-misses
0.60 ± 6% -0.3 0.25 ± 4% perf-stat.overall.dTLB-load-miss-rate%
0.15 ± 4% -0.1 0.07 ± 7% perf-stat.overall.dTLB-store-miss-rate%
4253 ± 2% +47.0% 6251 ± 3% perf-stat.overall.instructions-per-iTLB-miss
0.66 +28.7% 0.85 ± 2% perf-stat.overall.ipc
40.59 +3.8 44.36 perf-stat.overall.node-load-miss-rate%
40.12 +0.5 40.63 perf-stat.overall.node-store-miss-rate%
1.505e+09 +28.5% 1.933e+09 perf-stat.ps.branch-instructions
17869714 ± 3% -34.3% 11733432 ± 7% perf-stat.ps.cache-misses
1.129e+08 ± 2% -21.1% 89033383 ± 5% perf-stat.ps.cache-references
38777 +5.4% 40870 perf-stat.ps.context-switches
560.48 +65.4% 927.25 ± 7% perf-stat.ps.cpu-migrations
11022157 ± 5% -40.2% 6587600 ± 4% perf-stat.ps.dTLB-load-misses
1.82e+09 +42.1% 2.587e+09 perf-stat.ps.dTLB-loads
1174725 ± 3% -34.2% 772840 ± 9% perf-stat.ps.dTLB-store-misses
7.948e+08 ± 2% +48.9% 1.184e+09 ± 5% perf-stat.ps.dTLB-stores
1677919 ± 2% -8.9% 1528226 ± 3% perf-stat.ps.iTLB-load-misses
2418014 ± 4% -14.1% 2077000 ± 2% perf-stat.ps.iTLB-loads
7.134e+09 +33.8% 9.544e+09 perf-stat.ps.instructions
2720 -3.7% 2620 perf-stat.ps.minor-faults
6351137 -47.5% 3333134 perf-stat.ps.node-load-misses
9297533 -55.0% 4181124 ± 2% perf-stat.ps.node-loads
2768483 -25.5% 2062498 perf-stat.ps.node-store-misses
4131727 -27.1% 3013998 perf-stat.ps.node-stores
2720 -3.7% 2620 perf-stat.ps.page-faults
4.486e+12 +95.4% 8.766e+12 perf-stat.total.instructions
1494 +15.6% 1727 ± 2% interrupts.10:IR-IO-APIC.10-edge.ipmi_si
792.50 ± 13% +36.3% 1080 ± 14% interrupts.39:IR-PCI-MSI.524289-edge.eth0-TxRx-0
366.50 ± 3% +4427.7% 16594 ±166% interrupts.40:IR-PCI-MSI.524290-edge.eth0-TxRx-1
523.25 ± 26% +3155.0% 17032 ± 57% interrupts.45:IR-PCI-MSI.524295-edge.eth0-TxRx-6
204268 +43.9% 293848 interrupts.CAL:Function_call_interrupts
4733 ± 2% +46.0% 6909 interrupts.CPU0.CAL:Function_call_interrupts
1258857 +46.1% 1838720 interrupts.CPU0.LOC:Local_timer_interrupts
5356 ± 6% +38.4% 7416 ± 4% interrupts.CPU1.CAL:Function_call_interrupts
1258156 +46.1% 1838261 interrupts.CPU1.LOC:Local_timer_interrupts
4926 ± 8% +60.7% 7918 interrupts.CPU10.CAL:Function_call_interrupts
1258344 +46.1% 1838592 interrupts.CPU10.LOC:Local_timer_interrupts
164.50 ±151% +376.6% 784.00 ± 2% interrupts.CPU10.NMI:Non-maskable_interrupts
164.50 ±151% +376.6% 784.00 ± 2% interrupts.CPU10.PMI:Performance_monitoring_interrupts
4305 ± 32% +103.6% 8764 ± 4% interrupts.CPU10.RES:Rescheduling_interrupts
4914 ± 6% +53.7% 7556 ± 7% interrupts.CPU11.CAL:Function_call_interrupts
1258282 +46.1% 1838248 interrupts.CPU11.LOC:Local_timer_interrupts
5243 ± 8% +45.5% 7629 ± 7% interrupts.CPU12.CAL:Function_call_interrupts
1258363 +46.1% 1838546 interrupts.CPU12.LOC:Local_timer_interrupts
4884 ± 6% +56.6% 7646 ± 6% interrupts.CPU13.CAL:Function_call_interrupts
1258209 +46.1% 1838551 interrupts.CPU13.LOC:Local_timer_interrupts
4984 ± 7% +42.6% 7109 ± 8% interrupts.CPU14.CAL:Function_call_interrupts
1258516 +46.1% 1838593 interrupts.CPU14.LOC:Local_timer_interrupts
5286 ± 5% +33.5% 7057 ± 5% interrupts.CPU15.CAL:Function_call_interrupts
1258314 +46.1% 1838531 interrupts.CPU15.LOC:Local_timer_interrupts
4737 ± 3% +56.5% 7413 ± 7% interrupts.CPU16.CAL:Function_call_interrupts
1258679 +46.1% 1838447 interrupts.CPU16.LOC:Local_timer_interrupts
4237 ± 36% +73.6% 7356 ± 23% interrupts.CPU16.RES:Rescheduling_interrupts
4806 +57.5% 7572 ± 5% interrupts.CPU17.CAL:Function_call_interrupts
1257486 +46.2% 1838428 interrupts.CPU17.LOC:Local_timer_interrupts
4968 ± 7% +57.9% 7847 ± 3% interrupts.CPU18.CAL:Function_call_interrupts
1258427 +46.1% 1838038 interrupts.CPU18.LOC:Local_timer_interrupts
4864 ± 39% +87.5% 9119 ± 8% interrupts.CPU18.RES:Rescheduling_interrupts
4956 ± 4% +43.7% 7120 ± 4% interrupts.CPU19.CAL:Function_call_interrupts
1257616 +46.2% 1838487 interrupts.CPU19.LOC:Local_timer_interrupts
4973 ± 6% +57.3% 7820 ± 2% interrupts.CPU2.CAL:Function_call_interrupts
1258621 +46.1% 1838386 interrupts.CPU2.LOC:Local_timer_interrupts
165.75 ±138% +487.3% 973.50 ± 32% interrupts.CPU2.NMI:Non-maskable_interrupts
165.75 ±138% +487.3% 973.50 ± 32% interrupts.CPU2.PMI:Performance_monitoring_interrupts
4318 ± 35% +96.7% 8494 ± 2% interrupts.CPU2.RES:Rescheduling_interrupts
5505 +44.5% 7952 interrupts.CPU20.CAL:Function_call_interrupts
1258218 +46.1% 1838462 interrupts.CPU20.LOC:Local_timer_interrupts
4940 ± 5% +42.3% 7028 ± 8% interrupts.CPU21.CAL:Function_call_interrupts
1257717 +46.2% 1838448 interrupts.CPU21.LOC:Local_timer_interrupts
5320 ± 7% +28.8% 6853 interrupts.CPU22.CAL:Function_call_interrupts
1257738 +46.2% 1838357 interrupts.CPU22.LOC:Local_timer_interrupts
792.75 ± 32% -88.4% 91.75 ±106% interrupts.CPU22.NMI:Non-maskable_interrupts
792.75 ± 32% -88.4% 91.75 ±106% interrupts.CPU22.PMI:Performance_monitoring_interrupts
5453 ± 3% +33.8% 7297 ± 5% interrupts.CPU23.CAL:Function_call_interrupts
1257399 +46.2% 1838017 interrupts.CPU23.LOC:Local_timer_interrupts
5133 ± 7% +43.9% 7386 ± 7% interrupts.CPU24.CAL:Function_call_interrupts
1257560 +46.2% 1838381 interrupts.CPU24.LOC:Local_timer_interrupts
4539 ± 33% +50.1% 6813 ± 18% interrupts.CPU24.RES:Rescheduling_interrupts
5308 ± 10% +35.9% 7214 ± 9% interrupts.CPU25.CAL:Function_call_interrupts
1258418 +46.1% 1838405 interrupts.CPU25.LOC:Local_timer_interrupts
5124 ± 7% +40.3% 7188 ± 6% interrupts.CPU26.CAL:Function_call_interrupts
1258415 +46.1% 1838409 interrupts.CPU26.LOC:Local_timer_interrupts
5248 ± 8% +43.1% 7512 ± 6% interrupts.CPU27.CAL:Function_call_interrupts
1258222 +46.1% 1838381 interrupts.CPU27.LOC:Local_timer_interrupts
5114 ± 8% +42.8% 7305 ± 5% interrupts.CPU28.CAL:Function_call_interrupts
1258628 +46.0% 1838215 interrupts.CPU28.LOC:Local_timer_interrupts
4114 ± 47% +64.9% 6786 ± 22% interrupts.CPU28.RES:Rescheduling_interrupts
4910 ± 7% +52.7% 7501 ± 5% interrupts.CPU29.CAL:Function_call_interrupts
1258603 +46.1% 1838647 interrupts.CPU29.LOC:Local_timer_interrupts
4737 ± 3% +53.9% 7289 ± 7% interrupts.CPU3.CAL:Function_call_interrupts
1258257 +46.1% 1838143 interrupts.CPU3.LOC:Local_timer_interrupts
792.50 ± 13% +36.3% 1080 ± 14% interrupts.CPU30.39:IR-PCI-MSI.524289-edge.eth0-TxRx-0
5271 ± 5% +40.0% 7381 ± 6% interrupts.CPU30.CAL:Function_call_interrupts
1258798 +46.1% 1838480 interrupts.CPU30.LOC:Local_timer_interrupts
5050 ± 37% +38.1% 6973 ± 25% interrupts.CPU30.RES:Rescheduling_interrupts
5266 ± 6% +33.7% 7041 ± 7% interrupts.CPU31.CAL:Function_call_interrupts
1258115 +46.1% 1838307 interrupts.CPU31.LOC:Local_timer_interrupts
366.50 ± 3% +4427.7% 16594 ±166% interrupts.CPU32.40:IR-PCI-MSI.524290-edge.eth0-TxRx-1
4706 ± 3% +50.6% 7088 ± 5% interrupts.CPU32.CAL:Function_call_interrupts
1258290 +46.0% 1837663 interrupts.CPU32.LOC:Local_timer_interrupts
3612 ± 57% +79.6% 6488 ± 20% interrupts.CPU32.RES:Rescheduling_interrupts
5301 ± 7% +31.2% 6957 ± 7% interrupts.CPU33.CAL:Function_call_interrupts
1258353 +46.1% 1838459 interrupts.CPU33.LOC:Local_timer_interrupts
5273 ± 5% +46.0% 7697 ± 5% interrupts.CPU34.CAL:Function_call_interrupts
1258565 +46.0% 1838076 interrupts.CPU34.LOC:Local_timer_interrupts
5048 ± 5% +49.0% 7520 ± 7% interrupts.CPU35.CAL:Function_call_interrupts
1258233 +46.1% 1838319 interrupts.CPU35.LOC:Local_timer_interrupts
5447 ± 3% +36.0% 7409 ± 6% interrupts.CPU36.CAL:Function_call_interrupts
1258484 +46.0% 1837980 interrupts.CPU36.LOC:Local_timer_interrupts
5440 ± 3% +29.5% 7047 ± 8% interrupts.CPU37.CAL:Function_call_interrupts
1258540 +46.1% 1838267 interrupts.CPU37.LOC:Local_timer_interrupts
5299 ± 5% +30.1% 6896 interrupts.CPU38.CAL:Function_call_interrupts
1258420 +46.1% 1837968 interrupts.CPU38.LOC:Local_timer_interrupts
5023 ± 9% +44.8% 7273 ± 10% interrupts.CPU39.CAL:Function_call_interrupts
1258243 +46.1% 1838310 interrupts.CPU39.LOC:Local_timer_interrupts
143058 ±163% -95.5% 6406 ± 16% interrupts.CPU39.RES:Rescheduling_interrupts
523.25 ± 26% +3155.0% 17032 ± 57% interrupts.CPU4.45:IR-PCI-MSI.524295-edge.eth0-TxRx-6
5110 ± 8% +43.8% 7346 ± 7% interrupts.CPU4.CAL:Function_call_interrupts
1258092 +46.1% 1838428 interrupts.CPU4.LOC:Local_timer_interrupts
5054 ± 7% +45.3% 7341 ± 6% interrupts.CPU5.CAL:Function_call_interrupts
1258423 +46.1% 1838142 interrupts.CPU5.LOC:Local_timer_interrupts
5153 ± 7% +47.1% 7580 ± 8% interrupts.CPU6.CAL:Function_call_interrupts
1258463 +46.1% 1838103 interrupts.CPU6.LOC:Local_timer_interrupts
4679 ± 44% +65.2% 7732 ± 21% interrupts.CPU6.RES:Rescheduling_interrupts
4876 ± 5% +47.2% 7178 ± 6% interrupts.CPU7.CAL:Function_call_interrupts
1258508 +46.1% 1838410 interrupts.CPU7.LOC:Local_timer_interrupts
5121 ± 5% +48.9% 7623 ± 8% interrupts.CPU8.CAL:Function_call_interrupts
1258614 +46.1% 1838880 interrupts.CPU8.LOC:Local_timer_interrupts
5307 ± 5% +30.4% 6922 ± 8% interrupts.CPU9.CAL:Function_call_interrupts
1258499 +46.1% 1838174 interrupts.CPU9.LOC:Local_timer_interrupts
50331699 +46.1% 73533674 interrupts.LOC:Local_timer_interrupts
768133 -62.9% 285162 ± 4% interrupts.RES:Rescheduling_interrupts
1532 ± 6% +45.9% 2234 ± 10% interrupts.TLB:TLB_shootdowns
141788 ± 6% +41.9% 201238 ± 4% softirqs.CPU0.RCU
89538 ± 7% +50.5% 134779 softirqs.CPU0.SCHED
237443 ± 6% +37.1% 325641 ± 11% softirqs.CPU0.TIMER
146251 ± 3% +41.1% 206410 ± 5% softirqs.CPU1.RCU
92026 ± 3% +45.0% 133455 ± 3% softirqs.CPU1.SCHED
224949 ± 10% +38.3% 310998 ± 8% softirqs.CPU1.TIMER
154233 +33.6% 206060 ± 3% softirqs.CPU10.RCU
90504 ± 2% +46.3% 132452 ± 2% softirqs.CPU10.SCHED
242657 ± 6% +33.8% 324585 ± 11% softirqs.CPU10.TIMER
148976 ± 4% +39.3% 207517 ± 4% softirqs.CPU11.RCU
90739 +48.1% 134427 softirqs.CPU11.SCHED
249720 ± 4% +26.7% 316341 ± 10% softirqs.CPU11.TIMER
93965 ± 3% +41.6% 133035 softirqs.CPU12.SCHED
248423 ± 6% +32.4% 328906 ± 11% softirqs.CPU12.TIMER
149906 ± 4% +40.2% 210201 ± 6% softirqs.CPU13.RCU
90989 +49.6% 136105 softirqs.CPU13.SCHED
251792 ± 4% +42.1% 357900 ± 19% softirqs.CPU13.TIMER
152226 ± 2% +35.9% 206915 ± 3% softirqs.CPU14.RCU
92046 +44.6% 133104 softirqs.CPU14.SCHED
244799 ± 7% +34.3% 328652 ± 11% softirqs.CPU14.TIMER
153485 ± 2% +37.2% 210539 ± 4% softirqs.CPU15.RCU
91285 +47.8% 134965 softirqs.CPU15.SCHED
252238 ± 4% +32.8% 335030 ± 12% softirqs.CPU15.TIMER
147988 ± 2% +40.2% 207451 ± 5% softirqs.CPU16.RCU
93079 ± 2% +43.5% 133600 softirqs.CPU16.SCHED
146479 ± 4% +42.1% 208082 ± 7% softirqs.CPU17.RCU
91037 +48.3% 134972 softirqs.CPU17.SCHED
254044 ± 4% +33.6% 339310 ± 12% softirqs.CPU17.TIMER
150731 ± 2% +35.3% 203892 ± 4% softirqs.CPU18.RCU
92163 +45.1% 133721 ± 2% softirqs.CPU18.SCHED
139984 ± 9% +48.0% 207167 ± 6% softirqs.CPU19.RCU
87051 ± 8% +55.4% 135288 softirqs.CPU19.SCHED
256404 ± 3% +31.0% 335908 ± 12% softirqs.CPU19.TIMER
153581 +33.5% 205065 ± 3% softirqs.CPU2.RCU
91389 ± 4% +44.5% 132091 ± 2% softirqs.CPU2.SCHED
252327 ± 16% +28.5% 324205 ± 11% softirqs.CPU2.TIMER
91204 ± 5% +44.1% 131392 ± 2% softirqs.CPU20.SCHED
233549 ± 5% +38.4% 323275 ± 11% softirqs.CPU20.TIMER
147747 ± 2% +42.2% 210110 ± 6% softirqs.CPU21.RCU
91321 ± 3% +45.1% 132530 ± 3% softirqs.CPU21.SCHED
261236 ± 13% +19.5% 312199 ± 9% softirqs.CPU21.TIMER
152947 +36.4% 208566 ± 4% softirqs.CPU22.RCU
90046 ± 4% +46.4% 131871 softirqs.CPU22.SCHED
217967 ± 9% +50.4% 327721 ± 11% softirqs.CPU22.TIMER
151811 ± 3% +39.3% 211522 ± 6% softirqs.CPU23.RCU
91204 +48.4% 135364 ± 3% softirqs.CPU23.SCHED
250031 ± 3% +19.2% 297969 ± 9% softirqs.CPU23.TIMER
158314 +33.2% 210939 ± 3% softirqs.CPU24.RCU
89110 ± 2% +48.1% 131999 ± 2% softirqs.CPU24.SCHED
228849 ± 3% +42.3% 325682 ± 11% softirqs.CPU24.TIMER
157157 +36.5% 214446 ± 5% softirqs.CPU25.RCU
90115 +47.5% 132882 ± 3% softirqs.CPU25.SCHED
246845 ± 4% +27.7% 315332 ± 10% softirqs.CPU25.TIMER
157705 +33.4% 210455 ± 3% softirqs.CPU26.RCU
89991 ± 2% +46.7% 132047 ± 2% softirqs.CPU26.SCHED
229330 ± 5% +42.3% 326286 ± 11% softirqs.CPU26.TIMER
154583 ± 2% +33.7% 206619 ± 2% softirqs.CPU27.RCU
90397 +47.6% 133442 ± 3% softirqs.CPU27.SCHED
247821 ± 4% +27.8% 316809 ± 10% softirqs.CPU27.TIMER
150252 ± 9% +39.8% 209980 ± 2% softirqs.CPU28.RCU
88703 ± 8% +48.2% 131470 softirqs.CPU28.SCHED
229069 ± 12% +43.5% 328754 ± 11% softirqs.CPU28.TIMER
155740 +37.8% 214536 ± 5% softirqs.CPU29.RCU
90612 +48.1% 134168 ± 3% softirqs.CPU29.SCHED
248694 ± 4% +41.4% 351652 ± 20% softirqs.CPU29.TIMER
149921 ± 3% +40.5% 210592 ± 4% softirqs.CPU3.RCU
91270 ± 2% +48.0% 135072 ± 3% softirqs.CPU3.SCHED
247492 ± 4% +40.2% 347080 ± 21% softirqs.CPU3.TIMER
153901 +35.8% 209011 ± 3% softirqs.CPU30.RCU
90334 ± 2% +46.3% 132136 ± 2% softirqs.CPU30.SCHED
243300 ± 6% +34.8% 327959 ± 11% softirqs.CPU30.TIMER
149985 ± 3% +41.2% 211733 ± 6% softirqs.CPU31.RCU
90682 +47.7% 133945 softirqs.CPU31.SCHED
249462 ± 4% +27.5% 318038 ± 9% softirqs.CPU31.TIMER
215.00 ± 4% +4426.2% 9731 ±167% softirqs.CPU32.NET_RX
130036 ± 4% +36.9% 177992 ± 3% softirqs.CPU32.RCU
87420 ± 9% +52.8% 133544 ± 3% softirqs.CPU32.SCHED
248459 ± 7% +34.4% 333827 ± 9% softirqs.CPU32.TIMER
131167 ± 2% +36.7% 179367 ± 5% softirqs.CPU33.RCU
90830 +49.0% 135292 softirqs.CPU33.SCHED
253786 ± 5% +22.5% 310907 ± 13% softirqs.CPU33.TIMER
134361 +31.8% 177135 softirqs.CPU34.RCU
92004 +43.8% 132280 softirqs.CPU34.SCHED
244369 ± 7% +33.7% 326841 ± 11% softirqs.CPU34.TIMER
137605 ± 2% +30.7% 179816 ± 3% softirqs.CPU35.RCU
91052 +47.2% 134057 softirqs.CPU35.SCHED
252092 ± 4% +32.2% 333240 ± 12% softirqs.CPU35.TIMER
137204 +31.4% 180221 ± 2% softirqs.CPU36.RCU
93058 ± 2% +42.6% 132667 softirqs.CPU36.SCHED
236948 ± 13% +39.1% 329695 ± 12% softirqs.CPU36.TIMER
138811 ± 2% +31.6% 182739 ± 4% softirqs.CPU37.RCU
90951 +46.7% 133448 ± 2% softirqs.CPU37.SCHED
253893 ± 4% +33.7% 339581 ± 13% softirqs.CPU37.TIMER
138263 +28.9% 178202 softirqs.CPU38.RCU
92003 +43.7% 132213 ± 2% softirqs.CPU38.SCHED
257946 ± 3% +28.0% 330201 ± 11% softirqs.CPU38.TIMER
93241 ± 3% +43.2% 133556 softirqs.CPU39.SCHED
254720 ± 4% +32.1% 336502 ± 13% softirqs.CPU39.TIMER
292.50 ± 24% +3105.5% 9376 ± 56% softirqs.CPU4.NET_RX
156349 +34.9% 210934 ± 3% softirqs.CPU4.RCU
89535 ± 2% +48.3% 132816 ± 2% softirqs.CPU4.SCHED
229314 ± 3% +43.5% 329036 ± 10% softirqs.CPU4.TIMER
155921 ± 2% +35.7% 211534 ± 4% softirqs.CPU5.RCU
90416 +47.4% 133232 ± 2% softirqs.CPU5.SCHED
247319 ± 4% +27.7% 315787 ± 10% softirqs.CPU5.TIMER
156332 +32.8% 207536 ± 3% softirqs.CPU6.RCU
90409 ± 2% +46.2% 132144 ± 2% softirqs.CPU6.SCHED
229623 ± 4% +41.8% 325539 ± 10% softirqs.CPU6.TIMER
154113 ± 2% +34.4% 207084 ± 2% softirqs.CPU7.RCU
90560 +47.7% 133733 ± 2% softirqs.CPU7.SCHED
248302 ± 4% +47.4% 365894 ± 16% softirqs.CPU7.TIMER
94614 ± 3% +39.0% 131538 softirqs.CPU8.SCHED
261763 ± 13% +24.2% 325131 ± 11% softirqs.CPU8.TIMER
154317 +34.8% 208035 ± 4% softirqs.CPU9.RCU
90493 +48.3% 134160 ± 2% softirqs.CPU9.SCHED
248684 ± 4% +22.7% 305256 ± 11% softirqs.CPU9.TIMER
6073358 +33.6% 8111475 ± 4% softirqs.RCU
3637411 +46.7% 5335018 ± 2% softirqs.SCHED
9847530 ± 3% +33.2% 13112728 ± 10% softirqs.TIMER
61.94 ± 2% -6.5 55.44 perf-profile.calltrace.cycles-pp.secondary_startup_64
6.08 ± 32% -6.1 0.00 perf-profile.calltrace.cycles-pp.xfs_inactive_inodes.xfs_inactive_worker.process_one_work.worker_thread.kthread
6.08 ± 32% -6.1 0.00 perf-profile.calltrace.cycles-pp.xfs_ici_walk_fns.xfs_inactive_inodes.xfs_inactive_worker.process_one_work.worker_thread
6.08 ± 32% -6.1 0.00 perf-profile.calltrace.cycles-pp.xfs_ici_walk_ag.xfs_ici_walk_fns.xfs_inactive_inodes.xfs_inactive_worker.process_one_work
6.06 ± 32% -6.1 0.00 perf-profile.calltrace.cycles-pp.xfs_inactive_inode.xfs_ici_walk_ag.xfs_ici_walk_fns.xfs_inactive_inodes.xfs_inactive_worker
6.02 ± 33% -6.0 0.00 perf-profile.calltrace.cycles-pp.xfs_inactive.xfs_inactive_inode.xfs_ici_walk_ag.xfs_ici_walk_fns.xfs_inactive_inodes
6.00 ± 33% -6.0 0.00 perf-profile.calltrace.cycles-pp.xfs_inactive_ifree.xfs_inactive.xfs_inactive_inode.xfs_ici_walk_ag.xfs_ici_walk_fns
60.89 ± 2% -5.5 55.40 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
60.89 ± 2% -5.5 55.40 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
60.84 ± 2% -5.5 55.37 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
55.86 ± 2% -4.3 51.52 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
55.48 ± 2% -4.2 51.25 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
7.39 ± 4% -2.8 4.58 ± 9% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
6.97 ± 4% -2.7 4.29 ± 10% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
3.90 ± 9% -1.6 2.30 ± 9% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
2.49 ± 6% -1.3 1.20 ± 8% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
2.60 ± 11% -1.1 1.51 ± 7% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state
2.75 ± 4% -0.7 2.02 ± 13% perf-profile.calltrace.cycles-pp.xfs_dir2_block_removename.xfs_dir_removename.xfs_remove.xfs_vn_unlink.vfs_unlink
2.21 ± 12% -0.7 1.51 ± 19% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
2.39 ± 6% -0.6 1.79 ± 4% perf-profile.calltrace.cycles-pp.xfs_dir_createname.xfs_create.xfs_generic_create.path_openat.do_filp_open
1.41 ± 13% -0.5 0.87 ± 11% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
1.59 ± 5% -0.5 1.07 ± 9% perf-profile.calltrace.cycles-pp.xfs_dir2_block_addname.xfs_dir_createname.xfs_create.xfs_generic_create.path_openat
0.97 ± 12% -0.5 0.47 ± 59% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
1.77 ± 15% -0.5 1.28 ± 21% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state
7.51 ± 4% -0.5 7.04 ± 2% perf-profile.calltrace.cycles-pp.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink
1.13 ± 12% -0.5 0.68 ± 14% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt
1.23 ± 4% -0.4 0.82 ± 8% perf-profile.calltrace.cycles-pp.xfs_dir3_data_check.xfs_dir2_block_addname.xfs_dir_createname.xfs_create.xfs_generic_create
1.21 ± 4% -0.4 0.80 ± 8% perf-profile.calltrace.cycles-pp.__xfs_dir3_data_check.xfs_dir3_data_check.xfs_dir2_block_addname.xfs_dir_createname.xfs_create
4.24 ± 3% -0.4 3.87 ± 4% perf-profile.calltrace.cycles-pp.xfs_dir_removename.xfs_remove.xfs_vn_unlink.vfs_unlink.do_unlinkat
1.38 ± 2% -0.4 1.03 ± 16% perf-profile.calltrace.cycles-pp.xfs_dir2_block_lookup_int.xfs_dir2_block_removename.xfs_dir_removename.xfs_remove.xfs_vn_unlink
1.20 ± 8% -0.3 0.85 ± 10% perf-profile.calltrace.cycles-pp.xfs_dir3_data_check.xfs_dir2_block_removename.xfs_dir_removename.xfs_remove.xfs_vn_unlink
1.18 ± 9% -0.3 0.84 ± 10% perf-profile.calltrace.cycles-pp.__xfs_dir3_data_check.xfs_dir3_data_check.xfs_dir2_block_removename.xfs_dir_removename.xfs_remove
1.20 ± 2% -0.3 0.90 ± 17% perf-profile.calltrace.cycles-pp.__xfs_dir3_data_check.xfs_dir3_data_check.xfs_dir2_block_lookup_int.xfs_dir2_block_removename.xfs_dir_removename
1.21 ± 2% -0.3 0.92 ± 16% perf-profile.calltrace.cycles-pp.xfs_dir3_data_check.xfs_dir2_block_lookup_int.xfs_dir2_block_removename.xfs_dir_removename.xfs_remove
2.18 ± 6% -0.3 1.90 ± 3% perf-profile.calltrace.cycles-pp.__xfs_trans_commit.xfs_create.xfs_generic_create.path_openat.do_filp_open
2.09 ± 5% -0.3 1.81 ± 3% perf-profile.calltrace.cycles-pp.xfs_log_commit_cil.__xfs_trans_commit.xfs_create.xfs_generic_create.path_openat
0.87 ± 2% -0.1 0.76 ± 2% perf-profile.calltrace.cycles-pp.__xstat64
0.77 ± 4% -0.1 0.67 ± 5% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__xstat64
0.77 ± 4% -0.1 0.67 ± 5% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__xstat64
0.67 ± 3% -0.1 0.58 ± 4% perf-profile.calltrace.cycles-pp.__do_sys_newstat.do_syscall_64.entry_SYSCALL_64_after_hwframe.__xstat64
0.59 ± 9% +0.3 0.88 ± 8% perf-profile.calltrace.cycles-pp.__account_scheduler_latency.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.26 ±100% +0.3 0.59 ± 9% perf-profile.calltrace.cycles-pp.__sched_text_start.schedule_idle.do_idle.cpu_startup_entry.start_secondary
0.27 ±100% +0.3 0.60 ± 10% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.46 ± 58% +0.3 0.80 ± 5% perf-profile.calltrace.cycles-pp.xfs_da3_node_lookup_int.xfs_dir2_node_removename.xfs_dir_removename.xfs_remove.xfs_vn_unlink
0.74 ± 8% +0.3 1.08 ± 9% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.sched_ttwu_pending
0.78 ± 8% +0.3 1.12 ± 10% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.sched_ttwu_pending.do_idle.cpu_startup_entry
0.78 ± 8% +0.3 1.12 ± 10% perf-profile.calltrace.cycles-pp.ttwu_do_activate.sched_ttwu_pending.do_idle.cpu_startup_entry.start_secondary
0.77 ± 8% +0.3 1.11 ± 10% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.sched_ttwu_pending.do_idle
0.40 ± 58% +0.3 0.75 ± 10% perf-profile.calltrace.cycles-pp.arch_stack_walk.stack_trace_save_tsk.__account_scheduler_latency.enqueue_entity.enqueue_task_fair
0.90 ± 8% +0.4 1.25 ± 10% perf-profile.calltrace.cycles-pp.sched_ttwu_pending.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.42 ± 58% +0.4 0.78 ± 9% perf-profile.calltrace.cycles-pp.stack_trace_save_tsk.__account_scheduler_latency.enqueue_entity.enqueue_task_fair.activate_task
1.92 ± 6% +0.4 2.36 ± 6% perf-profile.calltrace.cycles-pp.rwsem_spin_on_owner.rwsem_optimistic_spin.rwsem_down_write_slowpath.path_openat.do_filp_open
1.02 ± 7% +0.4 1.47 ± 6% perf-profile.calltrace.cycles-pp.xfs_dir2_node_removename.xfs_dir_removename.xfs_remove.xfs_vn_unlink.vfs_unlink
0.12 ±173% +0.5 0.64 ± 4% perf-profile.calltrace.cycles-pp.xfs_dir2_leafn_lookup_for_entry.xfs_da3_node_lookup_int.xfs_dir2_node_removename.xfs_dir_removename.xfs_remove
0.95 ± 32% +0.6 1.57 ± 18% perf-profile.calltrace.cycles-pp.xfs_btree_check_sblock.xfs_btree_get_rec.xfs_inobt_get_rec.xfs_check_agi_freecount.xfs_difree_inobt
1.08 ± 35% +0.6 1.72 ± 14% perf-profile.calltrace.cycles-pp.xfs_log_commit_cil.__xfs_trans_commit.xfs_inactive_ifree.xfs_inactive.xfs_inactive_inode
0.59 ± 5% +0.7 1.27 ± 3% perf-profile.calltrace.cycles-pp.xfs_btree_check_sblock.xfs_btree_get_rec.xfs_inobt_get_rec.xfs_check_agi_freecount.xfs_dialloc_ag
0.99 ± 36% +0.7 1.67 ± 17% perf-profile.calltrace.cycles-pp.xfs_btree_check_sblock.xfs_btree_increment.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree
12.23 ± 2% +0.7 12.95 ± 3% perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink
0.61 ± 6% +0.7 1.33 ± 3% perf-profile.calltrace.cycles-pp.xfs_btree_check_sblock.xfs_btree_increment.xfs_check_agi_freecount.xfs_dialloc_ag.xfs_dialloc
0.46 ±104% +0.8 1.21 ± 14% perf-profile.calltrace.cycles-pp.__xfs_btree_check_sblock.xfs_btree_check_sblock.xfs_btree_increment.xfs_check_agi_freecount.xfs_difree_inobt
1.01 ± 27% +0.8 1.81 ± 4% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.path_openat.do_filp_open
1.32 ± 29% +0.8 2.15 ± 17% perf-profile.calltrace.cycles-pp.xfs_btree_get_rec.xfs_inobt_get_rec.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree
1.33 ± 35% +0.9 2.26 ± 16% perf-profile.calltrace.cycles-pp.xfs_btree_increment.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree.xfs_ifree
0.00 +0.9 0.92 ± 2% perf-profile.calltrace.cycles-pp.__xfs_btree_check_sblock.xfs_btree_check_sblock.xfs_btree_increment.xfs_check_agi_freecount.xfs_dialloc_ag
0.81 ± 5% +1.0 1.78 ± 3% perf-profile.calltrace.cycles-pp.xfs_btree_get_rec.xfs_inobt_get_rec.xfs_check_agi_freecount.xfs_dialloc_ag.xfs_dialloc
0.86 ± 6% +1.0 1.87 ± 2% perf-profile.calltrace.cycles-pp.xfs_btree_increment.xfs_check_agi_freecount.xfs_dialloc_ag.xfs_dialloc.xfs_ialloc
0.76 ± 8% +1.0 1.79 ± 6% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.do_unlinkat.do_syscall_64
2.95 ± 12% +1.2 4.19 ± 5% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_open
3.28 ± 2% +1.2 4.52 ± 6% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.18 ± 12% +1.3 4.45 ± 4% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_open.do_syscall_64
3.54 +1.3 4.83 ± 5% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink
10.09 ± 3% +1.4 11.45 perf-profile.calltrace.cycles-pp.xfs_generic_create.path_openat.do_filp_open.do_sys_open.do_syscall_64
9.80 ± 3% +1.5 11.25 perf-profile.calltrace.cycles-pp.xfs_create.xfs_generic_create.path_openat.do_filp_open.do_sys_open
0.55 ± 64% +1.5 2.01 ± 9% perf-profile.calltrace.cycles-pp.__xfs_btree_check_sblock.xfs_btree_check_sblock.xfs_btree_get_rec.xfs_inobt_get_rec.xfs_check_agi_freecount
1.32 ± 3% +1.6 2.92 perf-profile.calltrace.cycles-pp.xfs_inobt_get_rec.xfs_check_agi_freecount.xfs_dialloc_ag.xfs_dialloc.xfs_ialloc
4.81 ± 3% +2.4 7.18 perf-profile.calltrace.cycles-pp.xfs_dir_ialloc.xfs_create.xfs_generic_create.path_openat.do_filp_open
4.76 ± 3% +2.4 7.16 perf-profile.calltrace.cycles-pp.xfs_ialloc.xfs_dir_ialloc.xfs_create.xfs_generic_create.path_openat
3.62 ± 4% +2.4 6.03 perf-profile.calltrace.cycles-pp.xfs_dialloc_ag.xfs_dialloc.xfs_ialloc.xfs_dir_ialloc.xfs_create
4.14 ± 3% +2.5 6.65 perf-profile.calltrace.cycles-pp.xfs_dialloc.xfs_ialloc.xfs_dir_ialloc.xfs_create.xfs_generic_create
3.67 ± 35% +2.5 6.19 ± 15% perf-profile.calltrace.cycles-pp.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree.xfs_ifree.xfs_inactive_ifree
15.05 ± 4% +2.6 17.61 ± 2% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat
15.15 ± 4% +2.6 17.71 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.creat
15.14 ± 4% +2.6 17.70 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat
3.91 ± 34% +2.6 6.48 ± 15% perf-profile.calltrace.cycles-pp.xfs_difree_inobt.xfs_difree.xfs_ifree.xfs_inactive_ifree.xfs_inactive
14.85 ± 5% +2.6 17.43 ± 2% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat
14.82 ± 5% +2.6 17.41 perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
15.41 ± 4% +2.6 18.03 ± 2% perf-profile.calltrace.cycles-pp.creat
4.11 ± 34% +2.7 6.83 ± 16% perf-profile.calltrace.cycles-pp.xfs_difree.xfs_ifree.xfs_inactive_ifree.xfs_inactive.xfs_inactive_inode
2.67 ± 4% +2.8 5.46 perf-profile.calltrace.cycles-pp.xfs_check_agi_freecount.xfs_dialloc_ag.xfs_dialloc.xfs_ialloc.xfs_dir_ialloc
4.75 ± 32% +3.0 7.71 ± 16% perf-profile.calltrace.cycles-pp.xfs_ifree.xfs_inactive_ifree.xfs_inactive.xfs_inactive_inode.xfs_ici_walk_ag
6.08 ± 32% +3.7 9.80 ± 16% perf-profile.calltrace.cycles-pp.xfs_inactive_worker.process_one_work.worker_thread.kthread.ret_from_fork
6.85 ± 29% +3.8 10.61 ± 15% perf-profile.calltrace.cycles-pp.ret_from_fork
6.85 ± 29% +3.8 10.61 ± 15% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
6.58 ± 29% +3.8 10.34 ± 15% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
6.56 ± 29% +3.8 10.32 ± 15% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
0.00 +9.7 9.67 ± 16% perf-profile.calltrace.cycles-pp.xfs_inactive_ifree.xfs_inactive.xfs_inactive_inode.xfs_ici_walk_ag.xfs_inactive_worker
0.00 +9.7 9.70 ± 16% perf-profile.calltrace.cycles-pp.xfs_inactive.xfs_inactive_inode.xfs_ici_walk_ag.xfs_inactive_worker.process_one_work
0.00 +9.8 9.77 ± 16% perf-profile.calltrace.cycles-pp.xfs_inactive_inode.xfs_ici_walk_ag.xfs_inactive_worker.process_one_work.worker_thread
0.00 +9.8 9.80 ± 16% perf-profile.calltrace.cycles-pp.xfs_ici_walk_ag.xfs_inactive_worker.process_one_work.worker_thread.kthread
61.96 ± 2% -6.5 55.45 perf-profile.children.cycles-pp.do_idle
61.94 ± 2% -6.5 55.44 perf-profile.children.cycles-pp.secondary_startup_64
61.94 ± 2% -6.5 55.44 perf-profile.children.cycles-pp.cpu_startup_entry
6.08 ± 32% -6.1 0.00 perf-profile.children.cycles-pp.xfs_inactive_inodes
6.08 ± 32% -6.1 0.00 perf-profile.children.cycles-pp.xfs_ici_walk_fns
60.89 ± 2% -5.5 55.40 perf-profile.children.cycles-pp.start_secondary
56.85 ± 2% -5.3 51.56 ± 2% perf-profile.children.cycles-pp.cpuidle_enter
56.81 ± 2% -5.3 51.55 ± 2% perf-profile.children.cycles-pp.cpuidle_enter_state
8.15 ± 4% -2.7 5.42 ± 11% perf-profile.children.cycles-pp.apic_timer_interrupt
7.68 ± 4% -2.6 5.07 ± 11% perf-profile.children.cycles-pp.smp_apic_timer_interrupt
4.43 ± 8% -1.5 2.89 ± 10% perf-profile.children.cycles-pp.hrtimer_interrupt
2.56 ± 4% -1.3 1.22 ± 7% perf-profile.children.cycles-pp.menu_select
3.00 ± 9% -1.0 1.97 ± 9% perf-profile.children.cycles-pp.__hrtimer_run_queues
4.76 ± 3% -0.8 3.97 ± 4% perf-profile.children.cycles-pp.__xfs_dir3_data_check
4.77 ± 3% -0.8 3.99 ± 4% perf-profile.children.cycles-pp.xfs_dir3_data_check
2.75 ± 4% -0.7 2.02 ± 13% perf-profile.children.cycles-pp.xfs_dir2_block_removename
2.35 ± 12% -0.7 1.68 ± 19% perf-profile.children.cycles-pp.irq_exit
2.39 ± 6% -0.6 1.79 ± 4% perf-profile.children.cycles-pp.xfs_dir_createname
0.88 ± 5% -0.6 0.29 ± 17% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
1.59 ± 5% -0.5 1.07 ± 9% perf-profile.children.cycles-pp.xfs_dir2_block_addname
7.51 ± 4% -0.5 7.05 ± 2% perf-profile.children.cycles-pp.vfs_unlink
1.65 ± 11% -0.5 1.19 ± 15% perf-profile.children.cycles-pp.tick_sched_timer
0.66 ± 8% -0.5 0.21 ± 21% perf-profile.children.cycles-pp.tick_nohz_next_event
1.89 ± 16% -0.4 1.46 ± 20% perf-profile.children.cycles-pp.__softirqentry_text_start
1.35 ± 10% -0.4 0.97 ± 18% perf-profile.children.cycles-pp.tick_sched_handle
4.24 ± 3% -0.4 3.87 ± 4% perf-profile.children.cycles-pp.xfs_dir_removename
1.58 ± 3% -0.4 1.23 ± 22% perf-profile.children.cycles-pp.xfs_dir2_block_lookup_int
0.67 ± 8% -0.3 0.35 ± 9% perf-profile.children.cycles-pp.irq_enter
1.20 ± 9% -0.3 0.89 ± 19% perf-profile.children.cycles-pp.update_process_times
0.36 ± 11% -0.3 0.06 ± 65% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.40 ± 7% -0.3 0.14 ± 6% perf-profile.children.cycles-pp.xfs_dialloc_ag_update_inobt
0.48 ± 8% -0.2 0.24 ± 13% perf-profile.children.cycles-pp.tick_irq_enter
0.24 ± 2% -0.2 0.03 ±102% perf-profile.children.cycles-pp.__next_timer_interrupt
0.48 ± 11% -0.2 0.29 ± 3% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.37 ± 12% -0.2 0.19 ± 24% perf-profile.children.cycles-pp.cpuidle_governor_latency_req
0.54 ± 19% -0.2 0.37 ± 22% perf-profile.children.cycles-pp.run_rebalance_domains
0.30 ± 11% -0.2 0.14 ± 5% perf-profile.children.cycles-pp.native_irq_return_iret
0.24 ± 8% -0.2 0.08 ± 16% perf-profile.children.cycles-pp.__slab_alloc
0.38 ± 7% -0.1 0.23 ± 12% perf-profile.children.cycles-pp.read_tsc
0.32 ± 7% -0.1 0.17 ± 22% perf-profile.children.cycles-pp.__remove_hrtimer
0.40 ± 10% -0.1 0.26 ± 21% perf-profile.children.cycles-pp.native_write_msr
0.22 ± 3% -0.1 0.08 ± 14% perf-profile.children.cycles-pp.___slab_alloc
0.41 ± 11% -0.1 0.27 ± 19% perf-profile.children.cycles-pp.kmem_zone_alloc
0.21 ± 17% -0.1 0.07 ± 19% perf-profile.children.cycles-pp.__hrtimer_next_event_base
0.62 ± 13% -0.1 0.49 ± 8% perf-profile.children.cycles-pp.console_unlock
0.26 ± 7% -0.1 0.13 ± 27% perf-profile.children.cycles-pp.timerqueue_del
0.31 ± 4% -0.1 0.18 ± 33% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.49 ± 11% -0.1 0.36 ± 11% perf-profile.children.cycles-pp.kmem_cache_alloc
0.32 ± 20% -0.1 0.20 ± 13% perf-profile.children.cycles-pp.exit_to_usermode_loop
0.88 ± 3% -0.1 0.76 ± 2% perf-profile.children.cycles-pp.__xstat64
0.50 ± 2% -0.1 0.38 ± 11% perf-profile.children.cycles-pp.close
0.33 ± 16% -0.1 0.22 ± 18% perf-profile.children.cycles-pp.lapic_next_deadline
0.32 ± 6% -0.1 0.21 ± 19% perf-profile.children.cycles-pp.__lock_text_start
0.18 ± 7% -0.1 0.07 ± 12% perf-profile.children.cycles-pp.find_next_bit
0.71 ± 7% -0.1 0.60 ± 11% perf-profile.children.cycles-pp.xfs_da_read_buf
0.82 ± 3% -0.1 0.72 ± 3% perf-profile.children.cycles-pp.xfs_dir_ino_validate
0.32 ± 18% -0.1 0.22 ± 17% perf-profile.children.cycles-pp.xfs_dir3_block_read
0.23 ± 2% -0.1 0.14 ± 19% perf-profile.children.cycles-pp.enqueue_hrtimer
0.65 ± 5% -0.1 0.56 ± 9% perf-profile.children.cycles-pp.link_path_walk
0.34 ± 15% -0.1 0.25 ± 7% perf-profile.children.cycles-pp.destroy_inode
0.19 ± 13% -0.1 0.10 ± 9% perf-profile.children.cycles-pp.tick_nohz_irq_exit
0.59 -0.1 0.49 ± 5% perf-profile.children.cycles-pp.xfs_verify_ino
0.67 ± 3% -0.1 0.58 ± 4% perf-profile.children.cycles-pp.__do_sys_newstat
0.30 ± 7% -0.1 0.22 ± 19% perf-profile.children.cycles-pp.xfs_dabuf_map
0.11 ± 20% -0.1 0.03 ±100% perf-profile.children.cycles-pp.pm_qos_request
0.28 ± 8% -0.1 0.20 ± 7% perf-profile.children.cycles-pp.__kmalloc
0.25 ± 4% -0.1 0.17 ± 11% perf-profile.children.cycles-pp.run_timer_softirq
0.14 ± 16% -0.1 0.06 ± 63% perf-profile.children.cycles-pp.__hrtimer_get_next_event
0.19 ± 9% -0.1 0.11 ± 20% perf-profile.children.cycles-pp.timerqueue_add
0.23 ± 16% -0.1 0.15 ± 12% perf-profile.children.cycles-pp.delay_tsc
0.17 ± 4% -0.1 0.10 ± 18% perf-profile.children.cycles-pp.xfs_inode_alloc
0.10 ± 11% -0.1 0.03 ±100% perf-profile.children.cycles-pp.xfs_btree_del_cursor
0.19 ± 14% -0.1 0.12 ± 12% perf-profile.children.cycles-pp.kmem_alloc_large
0.17 ± 14% -0.1 0.10 ± 15% perf-profile.children.cycles-pp.note_gp_changes
0.30 ± 7% -0.1 0.23 ± 6% perf-profile.children.cycles-pp.kmem_alloc
0.28 ± 9% -0.1 0.21 ± 5% perf-profile.children.cycles-pp.sched_clock_cpu
0.10 ± 4% -0.1 0.03 ±102% perf-profile.children.cycles-pp.get_cpu_device
0.18 ± 20% -0.1 0.11 ± 6% perf-profile.children.cycles-pp.ktime_get_update_offsets_now
0.22 ± 11% -0.1 0.16 ± 17% perf-profile.children.cycles-pp.xfs_bmapi_read
0.18 ± 17% -0.1 0.12 ± 12% perf-profile.children.cycles-pp.xfs_default_hashname
0.15 ± 13% -0.1 0.10 ± 22% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.24 ± 14% -0.1 0.18 ± 7% perf-profile.children.cycles-pp.sched_clock
0.23 ± 12% -0.1 0.18 ± 8% perf-profile.children.cycles-pp.native_sched_clock
0.11 ± 25% -0.1 0.05 ± 8% perf-profile.children.cycles-pp.rb_next
0.10 ± 14% -0.1 0.05 perf-profile.children.cycles-pp.__x64_sys_close
0.08 ± 10% -0.1 0.03 ±100% perf-profile.children.cycles-pp.xfs_trans_brelse
0.35 ± 9% -0.1 0.30 ± 2% perf-profile.children.cycles-pp.xfs_dir3_data_get_ftype
0.09 ± 8% -0.0 0.04 ± 63% perf-profile.children.cycles-pp.perf_event_task_tick
0.10 ± 14% -0.0 0.05 ± 59% perf-profile.children.cycles-pp.tick_nohz_tick_stopped
0.14 ± 13% -0.0 0.09 ± 20% perf-profile.children.cycles-pp.__destroy_inode
0.14 ± 25% -0.0 0.09 ± 23% perf-profile.children.cycles-pp.rcu_eqs_enter
0.11 ± 24% -0.0 0.06 ± 11% perf-profile.children.cycles-pp.account_process_tick
0.12 ± 4% -0.0 0.08 ± 15% perf-profile.children.cycles-pp.rb_erase
0.09 ± 17% -0.0 0.04 ± 59% perf-profile.children.cycles-pp.pm_qos_read_value
0.10 ± 15% -0.0 0.06 ± 20% perf-profile.children.cycles-pp.rcu_irq_enter
0.07 ± 7% -0.0 0.03 ±100% perf-profile.children.cycles-pp._xfs_buf_ioapply
0.08 ± 5% -0.0 0.04 ± 58% perf-profile.children.cycles-pp.xfs_buf_item_put
0.11 ± 10% -0.0 0.07 ± 26% perf-profile.children.cycles-pp.d_instantiate
0.10 ± 4% -0.0 0.08 ± 6% perf-profile.children.cycles-pp.brd_make_request
0.11 ± 7% -0.0 0.09 ± 12% perf-profile.children.cycles-pp.nr_iowait_cpu
0.10 ± 8% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.dput
0.08 ± 5% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.brd_do_bvec
0.07 ± 12% +0.0 0.10 ± 11% perf-profile.children.cycles-pp.___perf_sw_event
0.20 ± 9% +0.0 0.24 ± 6% perf-profile.children.cycles-pp.xfs_inobt_update
0.11 ± 17% +0.0 0.15 ± 12% perf-profile.children.cycles-pp.xfs_btree_check_block
0.03 ±100% +0.0 0.07 ± 12% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.11 ± 17% +0.0 0.16 ± 11% perf-profile.children.cycles-pp.xfs_log_done
0.11 ± 10% +0.1 0.16 ± 15% perf-profile.children.cycles-pp.xfs_iextents_copy
0.07 ± 14% +0.1 0.13 ± 12% perf-profile.children.cycles-pp.xfs_btree_ptr_to_daddr
0.00 +0.1 0.06 ± 15% perf-profile.children.cycles-pp.check_preempt_curr
0.03 ±100% +0.1 0.09 ± 26% perf-profile.children.cycles-pp.tick_nohz_idle_exit
0.07 ± 17% +0.1 0.14 ± 35% perf-profile.children.cycles-pp.xfs_iunlink_map_prev
0.03 ±100% +0.1 0.09 ± 44% perf-profile.children.cycles-pp.xfs_iunlink_map_ino
0.15 ± 19% +0.1 0.21 ± 12% perf-profile.children.cycles-pp.xfs_dir3_leaf_check_int
0.14 ± 22% +0.1 0.21 ± 22% perf-profile.children.cycles-pp.select_task_rq_fair
0.08 ± 60% +0.1 0.15 ± 14% perf-profile.children.cycles-pp.stack_trace_consume_entry_nosched
0.11 ± 30% +0.1 0.19 ± 20% perf-profile.children.cycles-pp.kernel_text_address
0.27 ± 7% +0.1 0.35 ± 10% perf-profile.children.cycles-pp.dequeue_task_fair
0.08 ± 27% +0.1 0.16 ± 16% perf-profile.children.cycles-pp.xfs_inobt_rec_freecount
0.04 ±100% +0.1 0.12 ± 22% perf-profile.children.cycles-pp.__module_text_address
0.04 ±100% +0.1 0.13 ± 23% perf-profile.children.cycles-pp.is_module_text_address
0.40 ± 11% +0.1 0.49 ± 7% perf-profile.children.cycles-pp.wake_up_q
0.13 ± 23% +0.1 0.22 ± 19% perf-profile.children.cycles-pp.unwind_get_return_address
0.04 ±100% +0.1 0.13 ± 17% perf-profile.children.cycles-pp.xfs_inobt_key_diff
0.23 ± 6% +0.1 0.32 ± 18% perf-profile.children.cycles-pp.xfs_imap_to_bp
0.11 ± 33% +0.1 0.21 ± 20% perf-profile.children.cycles-pp.__kernel_text_address
0.23 ± 7% +0.1 0.33 ± 11% perf-profile.children.cycles-pp.__orc_find
0.20 ± 9% +0.1 0.30 ± 10% perf-profile.children.cycles-pp.xfs_btree_rec_offset
0.07 ± 14% +0.1 0.18 ± 17% perf-profile.children.cycles-pp.xfs_btree_readahead
0.33 ± 12% +0.1 0.43 ± 12% perf-profile.children.cycles-pp.pick_next_task_fair
0.29 ± 12% +0.1 0.39 ± 6% perf-profile.children.cycles-pp.orc_find
0.49 ± 10% +0.1 0.60 ± 10% perf-profile.children.cycles-pp.schedule_idle
0.38 ± 7% +0.1 0.50 ± 5% perf-profile.children.cycles-pp.xfs_ialloc_read_agi
0.16 ± 19% +0.1 0.29 ± 7% perf-profile.children.cycles-pp.xfs_btree_get_block
0.18 ± 25% +0.1 0.32 ± 23% perf-profile.children.cycles-pp.xfs_difree_finobt
0.65 ± 21% +0.2 0.84 ± 8% perf-profile.children.cycles-pp.xfs_buf_unlock
0.74 ± 20% +0.2 0.94 ± 7% perf-profile.children.cycles-pp.xfs_buf_item_release
0.63 ± 7% +0.2 0.84 ± 4% perf-profile.children.cycles-pp.unwind_next_frame
0.44 ± 13% +0.2 0.65 ± 3% perf-profile.children.cycles-pp.xfs_dir2_leafn_lookup_for_entry
0.55 ± 23% +0.2 0.77 ± 9% perf-profile.children.cycles-pp.up
0.28 ± 31% +0.2 0.51 ± 7% perf-profile.children.cycles-pp.xfs_inobt_btrec_to_irec
0.79 ± 11% +0.3 1.04 ± 7% perf-profile.children.cycles-pp.xfs_da3_node_lookup_int
1.25 ± 12% +0.3 1.54 ± 10% perf-profile.children.cycles-pp.__sched_text_start
0.97 ± 16% +0.3 1.27 ± 6% perf-profile.children.cycles-pp.try_to_wake_up
0.93 ± 7% +0.4 1.29 ± 9% perf-profile.children.cycles-pp.sched_ttwu_pending
0.92 ± 11% +0.4 1.29 ± 6% perf-profile.children.cycles-pp.arch_stack_walk
0.96 ± 11% +0.4 1.36 ± 6% perf-profile.children.cycles-pp.stack_trace_save_tsk
1.07 ± 10% +0.4 1.51 ± 5% perf-profile.children.cycles-pp.__account_scheduler_latency
1.02 ± 7% +0.4 1.47 ± 6% perf-profile.children.cycles-pp.xfs_dir2_node_removename
1.36 ± 9% +0.5 1.86 ± 7% perf-profile.children.cycles-pp.enqueue_entity
1.41 ± 10% +0.5 1.92 ± 7% perf-profile.children.cycles-pp.enqueue_task_fair
1.43 ± 10% +0.5 1.94 ± 7% perf-profile.children.cycles-pp.ttwu_do_activate
1.43 ± 10% +0.5 1.94 ± 7% perf-profile.children.cycles-pp.activate_task
0.62 ± 41% +0.6 1.17 ± 9% perf-profile.children.cycles-pp.xfs_verify_agino
0.80 ± 15% +0.6 1.36 ± 10% perf-profile.children.cycles-pp.xfs_errortag_test
4.44 ± 3% +0.7 5.09 ± 6% perf-profile.children.cycles-pp.rwsem_spin_on_owner
12.23 ± 2% +0.7 12.95 ± 3% perf-profile.children.cycles-pp.do_unlinkat
1.14 ± 18% +0.9 2.05 ± 12% perf-profile.children.cycles-pp.xfs_inobt_get_maxrecs
10.09 ± 3% +1.4 11.45 perf-profile.children.cycles-pp.xfs_generic_create
9.80 ± 3% +1.4 11.25 perf-profile.children.cycles-pp.xfs_create
2.21 ± 16% +1.8 4.05 ± 9% perf-profile.children.cycles-pp.xfs_btree_get_rec
1.79 ± 16% +1.9 3.65 ± 5% perf-profile.children.cycles-pp.osq_lock
2.27 ± 20% +2.0 4.24 ± 8% perf-profile.children.cycles-pp.xfs_btree_increment
2.40 ± 19% +2.0 4.43 ± 8% perf-profile.children.cycles-pp.__xfs_btree_check_sblock
4.81 ± 3% +2.4 7.18 perf-profile.children.cycles-pp.xfs_dir_ialloc
4.76 ± 3% +2.4 7.16 perf-profile.children.cycles-pp.xfs_ialloc
3.62 ± 3% +2.4 6.03 perf-profile.children.cycles-pp.xfs_dialloc_ag
4.14 ± 3% +2.5 6.65 perf-profile.children.cycles-pp.xfs_dialloc
6.28 ± 6% +2.5 8.79 ± 5% perf-profile.children.cycles-pp.rwsem_optimistic_spin
15.09 ± 4% +2.6 17.64 ± 2% perf-profile.children.cycles-pp.do_sys_open
6.72 ± 6% +2.6 9.28 ± 4% perf-profile.children.cycles-pp.rwsem_down_write_slowpath
3.91 ± 34% +2.6 6.48 ± 15% perf-profile.children.cycles-pp.xfs_difree_inobt
14.88 ± 5% +2.6 17.45 perf-profile.children.cycles-pp.do_filp_open
14.85 ± 5% +2.6 17.43 perf-profile.children.cycles-pp.path_openat
15.43 ± 4% +2.6 18.05 ± 2% perf-profile.children.cycles-pp.creat
4.11 ± 34% +2.7 6.83 ± 16% perf-profile.children.cycles-pp.xfs_difree
3.28 ± 19% +2.8 6.09 ± 9% perf-profile.children.cycles-pp.xfs_btree_check_sblock
29.22 ± 3% +2.9 32.14 ± 2% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
29.19 ± 3% +2.9 32.12 ± 2% perf-profile.children.cycles-pp.do_syscall_64
4.75 ± 32% +3.0 7.71 ± 16% perf-profile.children.cycles-pp.xfs_ifree
3.55 ± 22% +3.0 6.56 ± 8% perf-profile.children.cycles-pp.xfs_inobt_get_rec
6.00 ± 33% +3.7 9.67 ± 16% perf-profile.children.cycles-pp.xfs_inactive_ifree
6.02 ± 33% +3.7 9.70 ± 16% perf-profile.children.cycles-pp.xfs_inactive
6.06 ± 32% +3.7 9.77 ± 16% perf-profile.children.cycles-pp.xfs_inactive_inode
6.08 ± 32% +3.7 9.80 ± 16% perf-profile.children.cycles-pp.xfs_inactive_worker
6.08 ± 32% +3.7 9.80 ± 16% perf-profile.children.cycles-pp.xfs_ici_walk_ag
6.86 ± 29% +3.8 10.61 ± 15% perf-profile.children.cycles-pp.ret_from_fork
6.85 ± 29% +3.8 10.61 ± 15% perf-profile.children.cycles-pp.kthread
6.58 ± 29% +3.8 10.34 ± 15% perf-profile.children.cycles-pp.worker_thread
6.56 ± 29% +3.8 10.32 ± 15% perf-profile.children.cycles-pp.process_one_work
6.42 ± 20% +5.4 11.80 ± 8% perf-profile.children.cycles-pp.xfs_check_agi_freecount
1.24 ± 4% -0.6 0.69 ± 8% perf-profile.self.cycles-pp.menu_select
2.15 ± 4% -0.4 1.70 ± 6% perf-profile.self.cycles-pp.__xfs_dir3_data_check
0.71 ± 8% -0.3 0.43 ± 17% perf-profile.self.cycles-pp.cpuidle_enter_state
0.30 ± 11% -0.2 0.14 ± 5% perf-profile.self.cycles-pp.native_irq_return_iret
0.36 ± 6% -0.1 0.21 ± 15% perf-profile.self.cycles-pp.read_tsc
0.40 ± 9% -0.1 0.26 ± 21% perf-profile.self.cycles-pp.native_write_msr
0.19 ± 15% -0.1 0.07 ± 17% perf-profile.self.cycles-pp.__hrtimer_next_event_base
0.16 ± 9% -0.1 0.06 ± 20% perf-profile.self.cycles-pp.find_next_bit
0.22 ± 14% -0.1 0.12 ± 13% perf-profile.self.cycles-pp.__hrtimer_run_queues
0.20 ± 21% -0.1 0.10 ± 25% perf-profile.self.cycles-pp.hrtimer_interrupt
0.12 ± 20% -0.1 0.03 ±100% perf-profile.self.cycles-pp.clockevents_program_event
0.29 ± 13% -0.1 0.20 ± 7% perf-profile.self.cycles-pp.do_idle
0.11 ± 20% -0.1 0.03 ±100% perf-profile.self.cycles-pp.pm_qos_request
0.15 ± 17% -0.1 0.08 ± 16% perf-profile.self.cycles-pp.tick_nohz_next_event
0.36 ± 17% -0.1 0.29 ± 18% perf-profile.self.cycles-pp.io_serial_in
0.14 ± 15% -0.1 0.07 ± 61% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.10 ± 13% -0.1 0.03 ±100% perf-profile.self.cycles-pp.irq_exit
0.10 ± 5% -0.1 0.03 ±102% perf-profile.self.cycles-pp.get_cpu_device
0.25 ± 10% -0.1 0.19 ± 8% perf-profile.self.cycles-pp.xfs_verify_ino
0.49 ± 5% -0.1 0.43 ± 5% perf-profile.self.cycles-pp.xfs_log_commit_cil
0.16 ± 15% -0.1 0.10 ± 12% perf-profile.self.cycles-pp.xfs_default_hashname
0.22 ± 15% -0.1 0.16 ± 13% perf-profile.self.cycles-pp.native_sched_clock
0.19 ± 7% -0.1 0.13 ± 8% perf-profile.self.cycles-pp.run_timer_softirq
0.33 ± 8% -0.1 0.28 ± 2% perf-profile.self.cycles-pp.xfs_dir3_data_get_ftype
0.09 ± 8% -0.0 0.04 ± 63% perf-profile.self.cycles-pp.perf_event_task_tick
0.12 ± 4% -0.0 0.08 ± 19% perf-profile.self.cycles-pp.rb_erase
0.11 ± 24% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.account_process_tick
0.20 ± 9% -0.0 0.15 ± 12% perf-profile.self.cycles-pp.delay_tsc
0.13 ± 23% -0.0 0.09 ± 23% perf-profile.self.cycles-pp.rcu_eqs_enter
0.12 ± 12% -0.0 0.08 ± 13% perf-profile.self.cycles-pp.timerqueue_add
0.08 ± 5% -0.0 0.05 ± 61% perf-profile.self.cycles-pp.native_apic_mem_write
0.10 ± 7% -0.0 0.07 ± 6% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler
0.07 ± 10% -0.0 0.04 ± 58% perf-profile.self.cycles-pp.tick_nohz_tick_stopped
0.09 ± 23% -0.0 0.06 ± 17% perf-profile.self.cycles-pp.rcu_core
0.07 ± 7% -0.0 0.04 ± 58% perf-profile.self.cycles-pp.xfs_buf_item_put
0.11 ± 4% -0.0 0.09 ± 12% perf-profile.self.cycles-pp.nr_iowait_cpu
0.08 ± 19% +0.0 0.11 ± 17% perf-profile.self.cycles-pp.select_task_rq_fair
0.13 ± 12% +0.0 0.17 ± 5% perf-profile.self.cycles-pp.update_load_avg
0.04 ± 59% +0.0 0.09 ± 15% perf-profile.self.cycles-pp.xfs_btree_check_block
0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.xfs_btree_ptr_to_daddr
0.14 ± 18% +0.1 0.20 ± 10% perf-profile.self.cycles-pp.xfs_dir3_leaf_check_int
0.03 ±105% +0.1 0.11 ± 20% perf-profile.self.cycles-pp.xfs_inobt_key_diff
0.05 ± 62% +0.1 0.14 ± 19% perf-profile.self.cycles-pp.xfs_inobt_rec_freecount
0.14 ± 8% +0.1 0.22 ± 8% perf-profile.self.cycles-pp.xfs_btree_get_block
0.07 ± 20% +0.1 0.16 ± 18% perf-profile.self.cycles-pp.xfs_btree_readahead
0.23 ± 7% +0.1 0.33 ± 10% perf-profile.self.cycles-pp.__orc_find
0.15 ± 19% +0.1 0.25 ± 7% perf-profile.self.cycles-pp.xfs_check_agi_freecount
0.19 ± 8% +0.1 0.29 ± 13% perf-profile.self.cycles-pp.xfs_btree_rec_offset
0.15 ± 28% +0.1 0.27 ± 13% perf-profile.self.cycles-pp.xfs_ag_block_count
0.26 ± 29% +0.2 0.46 ± 6% perf-profile.self.cycles-pp.xfs_inobt_btrec_to_irec
0.21 ± 29% +0.2 0.42 ± 8% perf-profile.self.cycles-pp.xfs_verify_agino
0.50 ± 17% +0.2 0.74 ± 7% perf-profile.self.cycles-pp.xfs_agino_range
0.26 ± 18% +0.3 0.55 ± 13% perf-profile.self.cycles-pp.xfs_btree_check_sblock
0.34 ± 32% +0.3 0.67 ± 9% perf-profile.self.cycles-pp.xfs_btree_get_rec
0.40 ± 17% +0.3 0.74 ± 11% perf-profile.self.cycles-pp.xfs_inobt_get_rec
0.42 ± 13% +0.4 0.80 ± 9% perf-profile.self.cycles-pp.xfs_btree_increment
0.73 ± 19% +0.5 1.25 ± 11% perf-profile.self.cycles-pp.xfs_errortag_test
4.36 ± 3% +0.6 4.97 ± 5% perf-profile.self.cycles-pp.rwsem_spin_on_owner
0.95 ± 14% +0.8 1.77 ± 13% perf-profile.self.cycles-pp.xfs_inobt_get_maxrecs
1.20 ± 22% +1.0 2.20 ± 7% perf-profile.self.cycles-pp.__xfs_btree_check_sblock
1.75 ± 16% +1.8 3.59 ± 5% perf-profile.self.cycles-pp.osq_lock
aim7.jobs-per-min
32000 +-+-----------------------------------------------------------------+
|.+. +.+.+. .++.+.+.++.+.+.+.++.+. .+. .+. .+. +.+ |
30000 +-+ + + + ++ + ++.+.+.+ : |
| : .+.++.+.|
28000 +-+ + |
| |
26000 +-+ |
| |
24000 +-+ |
| |
22000 +-+ |
| |
20000 +-+ O OO O O O O |
O O OO O O OO O O |
18000 +-+-----------------------------------------------------------------+
aim7.time.system_time
6000 +-+------------------------------------------------------------------+
O |
5500 +-O OO O O O O OO O O OO O |
| O O |
| |
5000 +-+ |
| |
4500 +-+ |
| |
4000 +-+ |
| |
| +. |
3500 +-+.++.+.+.+.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.+.+.++.+.+.+.+.+ +.|
| |
3000 +-+------------------------------------------------------------------+
aim7.time.elapsed_time
950 +-+-------------------------------------------------------------------+
O O O OO O O O O OO O O O O OO |
900 +-+ |
850 +-+ |
| |
800 +-+ |
| |
750 +-+ |
| |
700 +-+ |
650 +-+ |
| +.++.+.+.|
600 +-+ .+. .+. .+.+ .+. .+.+.++. + |
|.+.+.++.+ +.+.++.+.+.+.+.++.+.+ + + + +.+.+ |
550 +-+-------------------------------------------------------------------+
aim7.time.elapsed_time.max
950 +-+-------------------------------------------------------------------+
O O O OO O O O O OO O O O O OO |
900 +-+ |
850 +-+ |
| |
800 +-+ |
| |
750 +-+ |
| |
700 +-+ |
650 +-+ |
| +.++.+.+.|
600 +-+ .+. .+. .+.+ .+. .+.+.++. + |
|.+.+.++.+ +.+.++.+.+.+.+.++.+.+ + + + +.+.+ |
550 +-+-------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
View attachment "config-5.3.0-rc7-00096-g73a16a2894549" of type "text/plain" (199463 bytes)
View attachment "job-script" of type "text/plain" (8042 bytes)
View attachment "job.yaml" of type "text/plain" (5739 bytes)
View attachment "reproduce" of type "text/plain" (1046 bytes)
Powered by blists - more mailing lists