| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20210227120804.GB22871@xsang-OptiPlex-9020>
Date: Sat, 27 Feb 2021 20:08:04 +0800
From: kernel test robot <oliver.sang@...el.com>
To: Jan Kara <jack@...e.cz>
Cc: Theodore Ts'o <tytso@....edu>, LKML <linux-kernel@...r.kernel.org>,
lkp@...ts.01.org, lkp@...el.com, ying.huang@...el.com,
feng.tang@...el.com, zhengjun.xing@...el.com
Subject: [ext4] 05c2c00f37: aim7.jobs-per-min -11.8% regression
Greeting,
FYI, we noticed a -11.8% regression of aim7.jobs-per-min due to commit:
commit: 05c2c00f3769abb9e323fcaca70d2de0b48af7ba ("ext4: protect superblock modifications with a buffer lock")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: aim7
on test machine: 88 threads Intel(R) Xeon(R) Gold 6238M CPU @ 2.10GHz with 128G memory
with following parameters:
disk: 4BRD_12G
md: RAID1
fs: ext4
test: creat-clo
load: 1000
cpufreq_governor: performance
ucode: 0x5003006
test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system.
test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/
In addition to that, the commit also has significant impact on the following tests:
+------------------+---------------------------------------------------------------------------+
| testcase: change | aim9: aim9.udp_test.ops_per_sec 2.1% improvement |
| test machine | 256 threads Intel(R) Genuine Intel(R) CPU 0000 @ 1.30GHz with 112G memory |
| test parameters | cpufreq_governor=performance |
| | test=all |
| | testtime=5s |
| | ucode=0xffff0190 |
+------------------+---------------------------------------------------------------------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang@...el.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml
bin/lkp run compatible-job.yaml
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase/ucode:
gcc-9/performance/4BRD_12G/ext4/x86_64-rhel-8.3/1000/RAID1/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp9/creat-clo/aim7/0x5003006
commit:
4392fbc4ba ("ext4: drop sync argument of ext4_commit_super()")
05c2c00f37 ("ext4: protect superblock modifications with a buffer lock")
4392fbc4bab57db3 05c2c00f3769abb9e323fcaca70
---------------- ---------------------------
%stddev %change %stddev
\ | \
13342 -11.8% 11771 ± 2% aim7.jobs-per-min
449.81 +13.4% 510.10 ± 2% aim7.time.elapsed_time
449.81 +13.4% 510.10 ± 2% aim7.time.elapsed_time.max
1246834 ± 2% +13.8% 1418574 ± 3% aim7.time.involuntary_context_switches
204562 ± 4% +12.2% 229478 ± 3% aim7.time.minor_page_faults
35777 +12.9% 40390 ± 2% aim7.time.system_time
6270530 ± 3% +34.5% 8435526 ± 2% aim7.time.voluntary_context_switches
8686688 ± 12% +24.1% 10784294 ± 4% cpuidle.C1E.usage
8.74 +4.5% 9.14 iostat.cpu.idle
0.02 ± 12% -0.0 0.02 ± 9% mpstat.cpu.all.iowait%
6988 ± 11% +43.1% 9999 ± 9% numa-meminfo.node0.Active(file)
1440589 ± 7% +7.8% 1552890 ± 6% numa-numastat.node1.numa_hit
1747 ± 11% +43.0% 2499 ± 9% numa-vmstat.node0.nr_active_file
1747 ± 11% +43.0% 2499 ± 9% numa-vmstat.node0.nr_zone_active_file
481.09 +12.6% 541.71 ± 2% uptime.boot
5541 +10.8% 6140 uptime.idle
42743 -10.6% 38207 slabinfo.filp.active_objs
1430 -10.0% 1287 slabinfo.filp.active_slabs
45785 -10.0% 41214 slabinfo.filp.num_objs
1430 -10.0% 1287 slabinfo.filp.num_slabs
91.00 -1.1% 90.00 vmstat.cpu.sy
2850 ± 2% +14.5% 3263 vmstat.io.bo
13739 ± 3% +29.2% 17754 ± 2% vmstat.memory.buff
25822 ± 2% +14.8% 29645 vmstat.system.cs
452981 +11.7% 505788 ± 3% meminfo.Active
439264 +11.1% 488079 ± 3% meminfo.Active(anon)
13715 ± 3% +29.1% 17708 ± 2% meminfo.Active(file)
13713 ± 3% +29.2% 17723 ± 2% meminfo.Buffers
459294 +10.6% 507955 ± 3% meminfo.Shmem
109825 +11.1% 121996 ± 3% proc-vmstat.nr_active_anon
3428 ± 3% +29.1% 4426 ± 2% proc-vmstat.nr_active_file
1427 +13.3% 1618 ± 2% proc-vmstat.nr_dirtied
376669 +3.5% 389792 proc-vmstat.nr_file_pages
114820 +10.6% 126987 ± 3% proc-vmstat.nr_shmem
1372 +15.1% 1579 ± 4% proc-vmstat.nr_written
109825 +11.1% 121996 ± 3% proc-vmstat.nr_zone_active_anon
3428 ± 3% +29.1% 4426 ± 2% proc-vmstat.nr_zone_active_file
151153 ± 5% +16.9% 176679 ± 3% proc-vmstat.numa_hint_faults
106816 ± 7% +17.4% 125437 ± 4% proc-vmstat.numa_hint_faults_local
2915186 +5.4% 3071445 proc-vmstat.numa_hit
2835807 +5.4% 2988976 proc-vmstat.numa_local
129566 ± 2% +7.4% 139212 ± 4% proc-vmstat.pgactivate
4233617 +2.8% 4353951 proc-vmstat.pgalloc_normal
1622353 +12.0% 1816326 proc-vmstat.pgfault
3863258 +2.8% 3970690 proc-vmstat.pgfree
1290723 ± 3% +29.8% 1675288 ± 2% proc-vmstat.pgpgout
86298 +10.3% 95227 ± 3% proc-vmstat.pgreuse
189.67 ± 5% -9.6% 171.45 ± 5% sched_debug.cfs_rq:/.load_avg.stddev
15090675 +13.9% 17181333 sched_debug.cfs_rq:/.min_vruntime.avg
15410230 +14.0% 17563602 sched_debug.cfs_rq:/.min_vruntime.max
14179241 +13.8% 16142220 sched_debug.cfs_rq:/.min_vruntime.min
909405 ± 5% +13.7% 1034081 ± 6% sched_debug.cfs_rq:/.spread0.avg
1229377 ± 4% +15.2% 1416676 ± 3% sched_debug.cfs_rq:/.spread0.max
240213 +12.5% 270308 sched_debug.cpu.clock.avg
240225 +12.5% 270320 sched_debug.cpu.clock.max
240200 +12.5% 270295 sched_debug.cpu.clock.min
237580 +12.5% 267317 sched_debug.cpu.clock_task.avg
237797 +12.5% 267523 sched_debug.cpu.clock_task.max
232767 +12.8% 262508 sched_debug.cpu.clock_task.min
64022 ± 2% +31.4% 84141 sched_debug.cpu.nr_switches.avg
77073 +28.8% 99292 ± 3% sched_debug.cpu.nr_switches.max
60980 ± 2% +32.5% 80775 sched_debug.cpu.nr_switches.min
1399 ± 8% +23.7% 1731 ± 4% sched_debug.cpu.nr_uninterruptible.max
248.99 ± 6% +21.4% 302.15 ± 9% sched_debug.cpu.nr_uninterruptible.stddev
240200 +12.5% 270295 sched_debug.cpu_clk
239703 +12.6% 269797 sched_debug.ktime
240543 +12.5% 270629 sched_debug.sched_clk
1.221e+10 -1.1% 1.208e+10 perf-stat.i.branch-instructions
0.22 ± 23% -0.0 0.17 perf-stat.i.branch-miss-rate%
14793770 -6.0% 13912167 perf-stat.i.branch-misses
25953 ± 2% +14.8% 29803 perf-stat.i.context-switches
4869 +13.6% 5530 perf-stat.i.cpu-migrations
1.64e+10 -1.3% 1.62e+10 perf-stat.i.dTLB-loads
1.075e+09 -9.9% 9.682e+08 ± 2% perf-stat.i.dTLB-stores
430893 ± 2% +11.6% 480826 perf-stat.i.iTLB-loads
6.072e+10 -1.1% 6.005e+10 perf-stat.i.instructions
338.69 -1.5% 333.53 perf-stat.i.metric.M/sec
6039682 -7.8% 5565765 ± 2% perf-stat.i.node-store-misses
96902 ± 2% -4.0% 93045 ± 2% perf-stat.i.node-stores
0.12 -0.0 0.12 perf-stat.overall.branch-miss-rate%
1.219e+10 -1.0% 1.206e+10 perf-stat.ps.branch-instructions
14740466 -5.9% 13872099 perf-stat.ps.branch-misses
25873 ± 2% +14.9% 29722 perf-stat.ps.context-switches
4856 +13.6% 5517 perf-stat.ps.cpu-migrations
1.637e+10 -1.2% 1.617e+10 perf-stat.ps.dTLB-loads
1.072e+09 -9.9% 9.665e+08 ± 2% perf-stat.ps.dTLB-stores
428747 ± 2% +11.6% 478487 perf-stat.ps.iTLB-loads
6.061e+10 -1.1% 5.996e+10 perf-stat.ps.instructions
6028729 -7.8% 5557348 ± 2% perf-stat.ps.node-store-misses
96841 -4.1% 92846 ± 3% perf-stat.ps.node-stores
2.734e+13 +12.2% 3.066e+13 ± 2% perf-stat.total.instructions
1.54 ± 72% -86.8% 0.20 ±221% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
0.17 ± 19% -47.8% 0.09 ± 22% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.__ext4_journal_get_write_access.ext4_orphan_add.ext4_truncate
1.57 ± 11% +18.6% 1.86 ± 5% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.__getblk_gfp.__ext4_get_inode_loc.ext4_get_inode_loc
1.45 ± 37% +126.7% 3.29 ± 34% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.jbd2__journal_start.__ext4_journal_start_sb
0.59 ± 34% +58.4% 0.93 ± 24% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate
3.16 ± 68% -80.8% 0.61 ±223% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
8.52 ± 28% +80.5% 15.38 ± 25% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.__alloc_file.alloc_empty_file
162500 ± 9% +25.1% 203298 ± 10% perf-sched.total_wait_and_delay.count.ms
116.08 ± 9% -21.7% 90.91 ± 13% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.__ext4_handle_dirty_metadata.ext4_do_update_inode.ext4_mark_iloc_dirty
134.71 ± 9% -24.4% 101.80 ± 7% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.__getblk_gfp.__ext4_get_inode_loc.ext4_get_inode_loc
145.16 ± 21% -42.0% 84.13 ± 46% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.__mutex_lock.isra.0
153.37 ± 30% -68.2% 48.78 ±103% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.ext4_journal_check_start.__ext4_journal_start_sb.ext4_truncate
10.19 ± 5% -84.0% 1.62 ±223% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity
76.08 ± 11% -23.6% 58.16 ± 12% perf-sched.wait_and_delay.avg.ms.wait_transaction_locked.add_transaction_credits.start_this_handle.jbd2__journal_start
69.33 ± 14% +36.5% 94.67 ± 13% perf-sched.wait_and_delay.count.jbd2_journal_commit_transaction.kjournald2.kthread.ret_from_fork
890.17 ± 5% -30.8% 615.83 ± 4% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.__ext4_handle_dirty_metadata.ext4_do_update_inode.ext4_mark_iloc_dirty
3176 ± 15% +43.3% 4551 ± 17% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.__ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty
349.17 ± 5% -17.3% 288.83 ± 5% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.__getblk_gfp.__ext4_get_inode_loc.ext4_get_inode_loc
654.17 ± 73% +108.3% 1362 ± 18% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.kmem_cache_alloc.start_this_handle.jbd2__journal_start
953.33 ± 3% -84.6% 146.67 ±223% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity
35615 ± 10% +26.7% 45108 ± 10% perf-sched.wait_and_delay.count.schedule_preempt_disabled.__mutex_lock.isra.0.ext4_orphan_add
54444 ± 15% +44.7% 78784 ± 17% perf-sched.wait_and_delay.count.wait_transaction_locked.add_transaction_credits.start_this_handle.jbd2__journal_start
749.81 ± 5% -18.6% 610.38 ± 3% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.__ext4_handle_dirty_metadata.ext4_do_update_inode.ext4_mark_iloc_dirty
751.99 ± 7% -21.6% 589.37 ± 5% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.__getblk_gfp.__ext4_get_inode_loc.ext4_get_inode_loc
545.93 ± 17% -31.5% 374.21 ± 24% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.__kmalloc.ext4_ext_remove_space.ext4_ext_truncate
619.18 ± 17% -45.6% 337.10 ± 49% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.__mutex_lock.isra.0
573.86 ± 20% -64.0% 206.68 ±104% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.ext4_journal_check_start.__ext4_journal_start_sb.ext4_truncate
1067 ± 12% -84.4% 166.83 ±223% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity
650.19 ± 8% -13.1% 564.84 ± 3% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate
676.68 ± 9% -17.8% 556.38 ± 3% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.wait_for_completion.stop_two_cpus.migrate_swap
745.90 ± 4% -18.2% 609.91 ± 2% perf-sched.wait_and_delay.max.ms.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2
775.20 ± 6% -18.9% 628.79 perf-sched.wait_and_delay.max.ms.schedule_preempt_disabled.__mutex_lock.isra.0.ext4_orphan_add
770.37 ± 6% -18.9% 624.81 perf-sched.wait_and_delay.max.ms.schedule_preempt_disabled.__mutex_lock.isra.0.ext4_orphan_del
114.49 ± 9% -22.1% 89.15 ± 13% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.__ext4_handle_dirty_metadata.ext4_do_update_inode.ext4_mark_iloc_dirty
133.14 ± 9% -24.9% 99.94 ± 7% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.__getblk_gfp.__ext4_get_inode_loc.ext4_get_inode_loc
142.91 ± 22% -37.9% 88.74 ± 26% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.__mutex_lock.isra.0
120.05 ± 39% -61.7% 46.00 ± 96% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_write.ext4_truncate.ext4_setattr
151.19 ± 30% -57.1% 64.88 ± 53% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.ext4_journal_check_start.__ext4_journal_start_sb.ext4_truncate
4.55 ± 4% +12.8% 5.13 ± 3% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.jbd2_journal_commit_transaction.kjournald2.kthread
74.36 ± 11% -24.3% 56.28 ± 13% perf-sched.wait_time.avg.ms.wait_transaction_locked.add_transaction_credits.start_this_handle.jbd2__journal_start
749.64 ± 5% -18.6% 610.37 ± 3% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.__ext4_handle_dirty_metadata.ext4_do_update_inode.ext4_mark_iloc_dirty
751.98 ± 7% -21.6% 589.37 ± 5% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.__getblk_gfp.__ext4_get_inode_loc.ext4_get_inode_loc
545.92 ± 17% -32.0% 371.40 ± 24% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.__kmalloc.ext4_ext_remove_space.ext4_ext_truncate
619.17 ± 17% -38.7% 379.32 ± 23% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.__mutex_lock.isra.0
571.69 ± 20% -52.1% 273.60 ± 56% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.ext4_journal_check_start.__ext4_journal_start_sb.ext4_truncate
650.17 ± 8% -13.1% 564.83 ± 3% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate
676.67 ± 9% -17.8% 556.37 ± 3% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.wait_for_completion.stop_two_cpus.migrate_swap
745.40 ± 4% -18.3% 609.29 ± 2% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2
769.37 ± 6% -19.0% 622.87 ± 2% perf-sched.wait_time.max.ms.schedule_preempt_disabled.__mutex_lock.isra.0.ext4_orphan_add
769.35 ± 6% -19.1% 622.61 ± 2% perf-sched.wait_time.max.ms.schedule_preempt_disabled.__mutex_lock.isra.0.ext4_orphan_del
0.59 ± 4% -0.2 0.35 ± 70% perf-profile.calltrace.cycles-pp.ext4_ext_truncate.ext4_truncate.ext4_setattr.notify_change.do_truncate
0.81 ± 3% -0.1 0.70 ± 4% perf-profile.children.cycles-pp.ext4_reserve_inode_write
0.89 ± 3% -0.1 0.78 ± 3% perf-profile.children.cycles-pp.__ext4_mark_inode_dirty
1.00 ± 2% -0.1 0.89 ± 3% perf-profile.children.cycles-pp.ext4_mark_iloc_dirty
0.59 ± 4% -0.1 0.52 ± 4% perf-profile.children.cycles-pp.ext4_ext_truncate
0.65 ± 2% -0.1 0.59 ± 4% perf-profile.children.cycles-pp.ext4_do_update_inode
0.32 ± 2% -0.0 0.28 ± 2% perf-profile.children.cycles-pp.__mutex_unlock_slowpath
0.35 ± 4% -0.0 0.30 ± 4% perf-profile.children.cycles-pp.ext4_ext_remove_space
0.21 ± 3% -0.0 0.17 ± 4% perf-profile.children.cycles-pp.__getblk_gfp
0.15 ± 13% -0.0 0.11 ± 15% perf-profile.children.cycles-pp.run_ksoftirqd
0.18 ± 3% -0.0 0.14 ± 5% perf-profile.children.cycles-pp.crc32c_pcl_intel_update
0.15 ± 11% -0.0 0.12 ± 11% perf-profile.children.cycles-pp.smpboot_thread_fn
0.18 ± 2% -0.0 0.15 ± 4% perf-profile.children.cycles-pp.__find_get_block
0.12 ± 4% -0.0 0.09 ± 5% perf-profile.children.cycles-pp.mutex_unlock
0.15 ± 3% -0.0 0.12 ± 5% perf-profile.children.cycles-pp.ext4_inode_csum_set
0.24 ± 3% -0.0 0.21 ± 3% perf-profile.children.cycles-pp.__close
0.10 ± 9% -0.0 0.08 ± 12% perf-profile.children.cycles-pp.osq_unlock
0.20 ± 3% -0.0 0.18 ± 3% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.14 ± 3% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.wake_up_q
0.10 ± 4% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.cap_inode_need_killpriv
0.12 ± 4% -0.0 0.11 ± 6% perf-profile.children.cycles-pp.mutex_lock
0.08 -0.0 0.06 ± 7% perf-profile.children.cycles-pp.ext4_xattr_get
0.11 -0.0 0.09 ± 5% perf-profile.children.cycles-pp.dentry_needs_remove_privs
0.10 -0.0 0.08 ± 5% perf-profile.children.cycles-pp.__vfs_getxattr
0.11 ± 3% -0.0 0.10 ± 3% perf-profile.children.cycles-pp.alloc_empty_file
0.11 -0.0 0.10 ± 4% perf-profile.children.cycles-pp.__alloc_file
0.18 ± 6% +0.0 0.20 ± 4% perf-profile.children.cycles-pp.do_softirq_own_stack
0.19 ± 6% +0.0 0.21 ± 4% perf-profile.children.cycles-pp.irq_exit_rcu
0.03 ±100% +0.0 0.07 ± 7% perf-profile.children.cycles-pp.ktime_get_update_offsets_now
0.06 ± 45% +0.0 0.10 ± 11% perf-profile.children.cycles-pp.ktime_get
0.69 +0.1 0.75 perf-profile.children.cycles-pp.mutex_spin_on_owner
0.35 ± 6% +0.1 0.42 ± 6% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.30 ± 5% +0.1 0.38 ± 8% perf-profile.children.cycles-pp.tick_sched_timer
0.47 ± 7% +0.1 0.57 ± 7% perf-profile.children.cycles-pp.hrtimer_interrupt
0.47 ± 7% +0.1 0.58 ± 7% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.68 ± 6% +0.1 0.80 ± 4% perf-profile.children.cycles-pp.asm_call_sysvec_on_stack
0.68 ± 6% +0.1 0.81 ± 4% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.72 ± 7% +0.1 0.85 ± 4% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.24 -0.0 0.19 ± 3% perf-profile.self.cycles-pp.ext4_reserve_inode_write
0.29 ± 4% -0.0 0.24 ± 2% perf-profile.self.cycles-pp.ext4_mark_iloc_dirty
0.32 ± 3% -0.0 0.27 ± 4% perf-profile.self.cycles-pp.__mutex_lock
0.07 ± 7% -0.0 0.03 ±100% perf-profile.self.cycles-pp.ext4_inode_csum_set
0.12 ± 4% -0.0 0.09 ± 5% perf-profile.self.cycles-pp.mutex_unlock
0.12 ± 5% -0.0 0.10 ± 7% perf-profile.self.cycles-pp.mutex_lock
0.10 ± 6% -0.0 0.08 ± 12% perf-profile.self.cycles-pp.osq_unlock
0.12 -0.0 0.10 perf-profile.self.cycles-pp._raw_spin_lock
0.09 ± 7% -0.0 0.07 ± 6% perf-profile.self.cycles-pp.__find_get_block
0.07 ± 5% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.ext4_xattr_get
0.09 -0.0 0.08 ± 6% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.05 ± 45% +0.0 0.09 ± 12% perf-profile.self.cycles-pp.ktime_get
0.02 ±141% +0.0 0.06 perf-profile.self.cycles-pp.ktime_get_update_offsets_now
0.68 +0.1 0.74 ± 2% perf-profile.self.cycles-pp.mutex_spin_on_owner
0.02 ±141% +0.1 0.08 ± 8% perf-profile.self.cycles-pp.ext4_orphan_add
20823 ± 2% +14.2% 23775 ± 3% softirqs.CPU0.SCHED
17539 ± 2% +22.4% 21460 ± 5% softirqs.CPU1.SCHED
17122 +19.9% 20527 softirqs.CPU10.SCHED
17055 ± 2% +19.8% 20433 softirqs.CPU11.SCHED
16919 +21.4% 20538 softirqs.CPU12.SCHED
17013 ± 2% +20.6% 20510 ± 2% softirqs.CPU13.SCHED
17119 +19.8% 20501 softirqs.CPU14.SCHED
17158 +19.1% 20431 ± 2% softirqs.CPU15.SCHED
17223 +18.7% 20443 softirqs.CPU16.SCHED
16969 ± 2% +19.7% 20305 softirqs.CPU17.SCHED
16997 +20.3% 20441 softirqs.CPU18.SCHED
17126 +19.1% 20399 softirqs.CPU19.SCHED
17544 ± 3% +18.5% 20796 ± 5% softirqs.CPU2.SCHED
17207 +19.1% 20489 softirqs.CPU20.SCHED
17215 +17.4% 20219 softirqs.CPU21.SCHED
16574 +19.2% 19758 ± 3% softirqs.CPU22.SCHED
16901 ± 2% +17.7% 19893 ± 2% softirqs.CPU23.SCHED
16901 ± 3% +19.4% 20180 ± 2% softirqs.CPU24.SCHED
16818 +19.5% 20099 ± 2% softirqs.CPU25.SCHED
16892 +18.6% 20039 softirqs.CPU26.SCHED
16839 +20.0% 20205 softirqs.CPU27.SCHED
16926 +18.7% 20091 softirqs.CPU28.SCHED
16762 ± 2% +20.1% 20125 softirqs.CPU29.SCHED
17050 ± 2% +19.7% 20410 ± 2% softirqs.CPU3.SCHED
16961 +19.7% 20304 ± 2% softirqs.CPU30.SCHED
17013 +19.0% 20238 ± 2% softirqs.CPU31.SCHED
16836 +22.4% 20599 ± 3% softirqs.CPU32.SCHED
17160 +17.5% 20164 softirqs.CPU33.SCHED
16658 ± 2% +21.7% 20268 ± 2% softirqs.CPU34.SCHED
16980 ± 2% +18.2% 20069 ± 3% softirqs.CPU35.SCHED
16720 +20.0% 20067 ± 2% softirqs.CPU36.SCHED
17043 +19.3% 20338 softirqs.CPU37.SCHED
16935 ± 2% +20.0% 20321 softirqs.CPU38.SCHED
17031 +19.5% 20349 softirqs.CPU39.SCHED
17457 ± 3% +17.1% 20438 softirqs.CPU4.SCHED
16977 +20.1% 20393 ± 2% softirqs.CPU40.SCHED
16873 +20.7% 20369 softirqs.CPU41.SCHED
17216 ± 2% +17.6% 20238 ± 2% softirqs.CPU42.SCHED
17045 ± 2% +19.3% 20342 ± 2% softirqs.CPU43.SCHED
16175 ± 3% +20.4% 19473 ± 2% softirqs.CPU44.SCHED
16913 ± 2% +18.6% 20051 softirqs.CPU45.SCHED
16989 ± 2% +17.6% 19980 softirqs.CPU46.SCHED
16774 ± 2% +18.9% 19949 softirqs.CPU47.SCHED
16922 ± 2% +19.1% 20162 softirqs.CPU48.SCHED
16933 ± 3% +19.8% 20286 softirqs.CPU49.SCHED
17135 ± 3% +20.1% 20585 softirqs.CPU5.SCHED
16998 ± 2% +16.8% 19847 softirqs.CPU50.SCHED
16815 ± 2% +19.4% 20077 softirqs.CPU51.SCHED
16962 +18.6% 20111 softirqs.CPU52.SCHED
16853 ± 2% +20.0% 20224 softirqs.CPU53.SCHED
17002 +19.5% 20320 softirqs.CPU54.SCHED
17025 ± 2% +18.8% 20228 ± 2% softirqs.CPU55.SCHED
16851 +19.8% 20190 softirqs.CPU56.SCHED
17018 +18.3% 20132 softirqs.CPU57.SCHED
16949 ± 2% +19.8% 20308 ± 2% softirqs.CPU58.SCHED
17189 +18.0% 20287 softirqs.CPU59.SCHED
16891 +19.2% 20135 softirqs.CPU6.SCHED
16989 +20.0% 20387 softirqs.CPU60.SCHED
17075 +17.7% 20102 ± 2% softirqs.CPU61.SCHED
16938 ± 2% +19.8% 20285 softirqs.CPU62.SCHED
16854 ± 2% +19.2% 20094 ± 2% softirqs.CPU63.SCHED
17063 +18.5% 20216 softirqs.CPU64.SCHED
17048 +17.2% 19984 softirqs.CPU65.SCHED
16870 +19.4% 20138 softirqs.CPU66.SCHED
16817 +20.6% 20274 ± 2% softirqs.CPU67.SCHED
16868 ± 2% +20.1% 20256 softirqs.CPU68.SCHED
16890 +19.7% 20212 ± 2% softirqs.CPU69.SCHED
16842 +20.2% 20252 softirqs.CPU7.SCHED
17011 +19.8% 20375 softirqs.CPU70.SCHED
17048 +20.1% 20478 ± 2% softirqs.CPU71.SCHED
16881 ± 2% +19.6% 20194 softirqs.CPU72.SCHED
16931 +19.9% 20294 softirqs.CPU73.SCHED
17102 +18.7% 20308 softirqs.CPU74.SCHED
16926 ± 2% +19.4% 20206 softirqs.CPU75.SCHED
16757 ± 2% +21.6% 20380 ± 2% softirqs.CPU76.SCHED
17052 +19.8% 20420 softirqs.CPU77.SCHED
16998 +19.8% 20368 softirqs.CPU78.SCHED
17209 ± 2% +19.3% 20533 softirqs.CPU79.SCHED
17084 ± 2% +18.6% 20260 softirqs.CPU8.SCHED
16973 ± 2% +20.9% 20525 ± 2% softirqs.CPU80.SCHED
17198 +18.9% 20457 ± 2% softirqs.CPU81.SCHED
16907 ± 2% +20.8% 20419 ± 2% softirqs.CPU82.SCHED
17023 +19.3% 20316 ± 2% softirqs.CPU83.SCHED
16935 +18.8% 20116 ± 2% softirqs.CPU84.SCHED
17084 +18.8% 20297 softirqs.CPU85.SCHED
17086 +19.2% 20363 softirqs.CPU86.SCHED
16995 ± 3% +19.7% 20350 ± 2% softirqs.CPU87.SCHED
17100 ± 2% +19.4% 20420 softirqs.CPU9.SCHED
1498602 +19.3% 1788245 softirqs.SCHED
80372 +12.5% 90439 ± 2% softirqs.TIMER
902.67 +13.6% 1025 ± 2% interrupts.9:IO-APIC.9-fasteoi.acpi
1556804 ± 4% +35.0% 2102423 ± 3% interrupts.CAL:Function_call_interrupts
17978 ± 4% +27.2% 22867 ± 8% interrupts.CPU0.CAL:Function_call_interrupts
902210 +13.5% 1024347 ± 2% interrupts.CPU0.LOC:Local_timer_interrupts
3865 ± 3% +22.0% 4714 ± 3% interrupts.CPU0.RES:Rescheduling_interrupts
902.67 +13.6% 1025 ± 2% interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi
17763 ± 7% +27.9% 22718 ± 8% interrupts.CPU1.CAL:Function_call_interrupts
902064 +13.5% 1024088 ± 2% interrupts.CPU1.LOC:Local_timer_interrupts
3156 ± 3% +22.8% 3874 ± 5% interrupts.CPU1.RES:Rescheduling_interrupts
16929 ± 5% +27.5% 21592 ± 6% interrupts.CPU10.CAL:Function_call_interrupts
902163 +13.5% 1024115 ± 2% interrupts.CPU10.LOC:Local_timer_interrupts
3174 ± 4% +20.5% 3825 ± 3% interrupts.CPU10.RES:Rescheduling_interrupts
17099 ± 4% +25.8% 21503 ± 7% interrupts.CPU11.CAL:Function_call_interrupts
902105 +13.5% 1024101 ± 2% interrupts.CPU11.LOC:Local_timer_interrupts
3166 ± 3% +22.1% 3865 ± 3% interrupts.CPU11.RES:Rescheduling_interrupts
16935 ± 5% +27.2% 21542 ± 7% interrupts.CPU12.CAL:Function_call_interrupts
902122 +13.5% 1024089 ± 2% interrupts.CPU12.LOC:Local_timer_interrupts
3134 ± 4% +22.4% 3837 ± 5% interrupts.CPU12.RES:Rescheduling_interrupts
17107 ± 5% +25.2% 21414 ± 6% interrupts.CPU13.CAL:Function_call_interrupts
902138 +13.5% 1024071 ± 2% interrupts.CPU13.LOC:Local_timer_interrupts
3154 ± 4% +21.3% 3825 ± 5% interrupts.CPU13.RES:Rescheduling_interrupts
16949 ± 4% +28.0% 21692 ± 8% interrupts.CPU14.CAL:Function_call_interrupts
902142 +13.5% 1024014 ± 2% interrupts.CPU14.LOC:Local_timer_interrupts
3154 ± 2% +20.8% 3809 ± 5% interrupts.CPU14.RES:Rescheduling_interrupts
16971 ± 6% +26.5% 21460 ± 7% interrupts.CPU15.CAL:Function_call_interrupts
902125 +13.5% 1024008 ± 2% interrupts.CPU15.LOC:Local_timer_interrupts
3159 ± 3% +21.7% 3847 ± 5% interrupts.CPU15.RES:Rescheduling_interrupts
16914 ± 4% +26.9% 21469 ± 6% interrupts.CPU16.CAL:Function_call_interrupts
902263 +13.5% 1024072 ± 2% interrupts.CPU16.LOC:Local_timer_interrupts
3176 ± 4% +20.6% 3831 ± 3% interrupts.CPU16.RES:Rescheduling_interrupts
16959 ± 5% +27.7% 21659 ± 7% interrupts.CPU17.CAL:Function_call_interrupts
902131 +13.5% 1024076 ± 2% interrupts.CPU17.LOC:Local_timer_interrupts
3164 ± 3% +21.1% 3834 ± 5% interrupts.CPU17.RES:Rescheduling_interrupts
16921 ± 5% +27.9% 21647 ± 7% interrupts.CPU18.CAL:Function_call_interrupts
901992 +13.5% 1024059 ± 2% interrupts.CPU18.LOC:Local_timer_interrupts
3114 ± 3% +23.7% 3851 ± 3% interrupts.CPU18.RES:Rescheduling_interrupts
16953 ± 5% +27.8% 21669 ± 7% interrupts.CPU19.CAL:Function_call_interrupts
902113 +13.5% 1024021 ± 2% interrupts.CPU19.LOC:Local_timer_interrupts
3186 ± 2% +20.8% 3850 ± 4% interrupts.CPU19.RES:Rescheduling_interrupts
17479 ± 5% +32.7% 23186 ± 3% interrupts.CPU2.CAL:Function_call_interrupts
902148 +13.5% 1024034 ± 2% interrupts.CPU2.LOC:Local_timer_interrupts
3163 ± 3% +21.2% 3835 ± 3% interrupts.CPU2.RES:Rescheduling_interrupts
16875 ± 5% +27.8% 21575 ± 7% interrupts.CPU20.CAL:Function_call_interrupts
902126 +13.5% 1024077 ± 2% interrupts.CPU20.LOC:Local_timer_interrupts
3125 ± 3% +23.9% 3874 ± 3% interrupts.CPU20.RES:Rescheduling_interrupts
16824 ± 4% +27.8% 21502 ± 7% interrupts.CPU21.CAL:Function_call_interrupts
902123 +13.5% 1024032 ± 2% interrupts.CPU21.LOC:Local_timer_interrupts
3119 ± 2% +23.8% 3861 ± 4% interrupts.CPU21.RES:Rescheduling_interrupts
18853 ± 10% +42.1% 26784 ± 7% interrupts.CPU22.CAL:Function_call_interrupts
902129 +13.5% 1024053 ± 2% interrupts.CPU22.LOC:Local_timer_interrupts
3076 ± 2% +15.9% 3564 ± 3% interrupts.CPU22.RES:Rescheduling_interrupts
18904 ± 10% +41.5% 26758 ± 7% interrupts.CPU23.CAL:Function_call_interrupts
902141 +13.5% 1024100 ± 2% interrupts.CPU23.LOC:Local_timer_interrupts
3055 ± 2% +13.8% 3478 ± 2% interrupts.CPU23.RES:Rescheduling_interrupts
18697 ± 10% +42.0% 26548 ± 7% interrupts.CPU24.CAL:Function_call_interrupts
902132 +13.5% 1024051 ± 2% interrupts.CPU24.LOC:Local_timer_interrupts
2996 ± 3% +15.3% 3454 ± 2% interrupts.CPU24.RES:Rescheduling_interrupts
18610 ± 10% +42.4% 26498 ± 7% interrupts.CPU25.CAL:Function_call_interrupts
902099 +13.5% 1024078 ± 2% interrupts.CPU25.LOC:Local_timer_interrupts
3061 ± 3% +15.4% 3531 ± 2% interrupts.CPU25.RES:Rescheduling_interrupts
18506 ± 10% +42.8% 26417 ± 7% interrupts.CPU26.CAL:Function_call_interrupts
902116 +13.5% 1024074 ± 2% interrupts.CPU26.LOC:Local_timer_interrupts
3016 ± 3% +15.6% 3488 ± 2% interrupts.CPU26.RES:Rescheduling_interrupts
18712 ± 11% +40.1% 26212 ± 7% interrupts.CPU27.CAL:Function_call_interrupts
902131 +13.5% 1023984 ± 2% interrupts.CPU27.LOC:Local_timer_interrupts
3032 ± 3% +16.4% 3528 ± 2% interrupts.CPU27.RES:Rescheduling_interrupts
18533 ± 10% +41.4% 26214 ± 7% interrupts.CPU28.CAL:Function_call_interrupts
902108 +13.5% 1024107 ± 2% interrupts.CPU28.LOC:Local_timer_interrupts
3022 ± 2% +17.1% 3539 ± 2% interrupts.CPU28.RES:Rescheduling_interrupts
18483 ± 10% +41.9% 26234 ± 7% interrupts.CPU29.CAL:Function_call_interrupts
902262 +13.5% 1024052 ± 2% interrupts.CPU29.LOC:Local_timer_interrupts
3034 +16.4% 3533 ± 2% interrupts.CPU29.RES:Rescheduling_interrupts
17359 ± 5% +29.3% 22454 ± 7% interrupts.CPU3.CAL:Function_call_interrupts
902230 +13.5% 1024080 ± 2% interrupts.CPU3.LOC:Local_timer_interrupts
3160 ± 4% +22.2% 3863 ± 3% interrupts.CPU3.RES:Rescheduling_interrupts
18526 ± 10% +42.2% 26340 ± 7% interrupts.CPU30.CAL:Function_call_interrupts
902146 +13.5% 1024088 ± 2% interrupts.CPU30.LOC:Local_timer_interrupts
3003 +16.6% 3503 ± 2% interrupts.CPU30.RES:Rescheduling_interrupts
18417 ± 10% +42.1% 26162 ± 8% interrupts.CPU31.CAL:Function_call_interrupts
902108 +13.5% 1024056 ± 2% interrupts.CPU31.LOC:Local_timer_interrupts
2992 ± 3% +17.2% 3506 ± 2% interrupts.CPU31.RES:Rescheduling_interrupts
18324 ± 10% +41.6% 25944 ± 7% interrupts.CPU32.CAL:Function_call_interrupts
902092 +13.5% 1024078 ± 2% interrupts.CPU32.LOC:Local_timer_interrupts
3037 ± 3% +15.2% 3498 ± 2% interrupts.CPU32.RES:Rescheduling_interrupts
18353 ± 11% +42.0% 26057 ± 7% interrupts.CPU33.CAL:Function_call_interrupts
902085 +13.5% 1024101 ± 2% interrupts.CPU33.LOC:Local_timer_interrupts
3055 ± 4% +14.6% 3502 ± 2% interrupts.CPU33.RES:Rescheduling_interrupts
18376 ± 10% +41.2% 25953 ± 7% interrupts.CPU34.CAL:Function_call_interrupts
902095 +13.5% 1024078 ± 2% interrupts.CPU34.LOC:Local_timer_interrupts
3015 ± 2% +14.4% 3448 ± 3% interrupts.CPU34.RES:Rescheduling_interrupts
18232 ± 10% +42.5% 25986 ± 7% interrupts.CPU35.CAL:Function_call_interrupts
902197 +13.5% 1024042 ± 2% interrupts.CPU35.LOC:Local_timer_interrupts
2990 ± 3% +16.6% 3487 ± 2% interrupts.CPU35.RES:Rescheduling_interrupts
18314 ± 10% +41.9% 25991 ± 7% interrupts.CPU36.CAL:Function_call_interrupts
902046 +13.5% 1024078 ± 2% interrupts.CPU36.LOC:Local_timer_interrupts
3002 ± 2% +15.9% 3479 interrupts.CPU36.RES:Rescheduling_interrupts
18369 ± 10% +40.6% 25829 ± 7% interrupts.CPU37.CAL:Function_call_interrupts
902117 +13.5% 1024026 ± 2% interrupts.CPU37.LOC:Local_timer_interrupts
2999 ± 2% +16.4% 3490 ± 2% interrupts.CPU37.RES:Rescheduling_interrupts
18331 ± 10% +41.5% 25932 ± 7% interrupts.CPU38.CAL:Function_call_interrupts
902118 +13.5% 1024085 ± 2% interrupts.CPU38.LOC:Local_timer_interrupts
2995 ± 3% +16.0% 3474 ± 4% interrupts.CPU38.RES:Rescheduling_interrupts
18313 ± 10% +41.6% 25935 ± 7% interrupts.CPU39.CAL:Function_call_interrupts
902082 +13.5% 1024103 ± 2% interrupts.CPU39.LOC:Local_timer_interrupts
2973 ± 2% +18.1% 3510 interrupts.CPU39.RES:Rescheduling_interrupts
18089 ± 10% +20.9% 21864 ± 6% interrupts.CPU4.CAL:Function_call_interrupts
902120 +13.5% 1024041 ± 2% interrupts.CPU4.LOC:Local_timer_interrupts
3126 ± 2% +22.9% 3842 ± 3% interrupts.CPU4.RES:Rescheduling_interrupts
18303 ± 10% +42.3% 26045 ± 7% interrupts.CPU40.CAL:Function_call_interrupts
902120 +13.5% 1024097 ± 2% interrupts.CPU40.LOC:Local_timer_interrupts
3019 ± 2% +14.7% 3462 ± 3% interrupts.CPU40.RES:Rescheduling_interrupts
18288 ± 10% +42.8% 26107 ± 7% interrupts.CPU41.CAL:Function_call_interrupts
902064 +13.5% 1024075 ± 2% interrupts.CPU41.LOC:Local_timer_interrupts
2992 +16.6% 3489 ± 2% interrupts.CPU41.RES:Rescheduling_interrupts
18169 ± 10% +43.3% 26042 ± 7% interrupts.CPU42.CAL:Function_call_interrupts
902151 +13.5% 1024091 ± 2% interrupts.CPU42.LOC:Local_timer_interrupts
3010 ± 2% +16.7% 3512 ± 2% interrupts.CPU42.RES:Rescheduling_interrupts
18161 ± 11% +42.4% 25871 ± 7% interrupts.CPU43.CAL:Function_call_interrupts
902074 +13.5% 1024076 ± 2% interrupts.CPU43.LOC:Local_timer_interrupts
2946 ± 2% +18.1% 3478 ± 2% interrupts.CPU43.RES:Rescheduling_interrupts
16873 ± 4% +26.9% 21415 ± 6% interrupts.CPU44.CAL:Function_call_interrupts
902175 +13.5% 1024065 ± 2% interrupts.CPU44.LOC:Local_timer_interrupts
3067 ± 4% +21.2% 3719 ± 5% interrupts.CPU44.RES:Rescheduling_interrupts
16915 ± 5% +28.3% 21704 ± 6% interrupts.CPU45.CAL:Function_call_interrupts
902101 +13.5% 1024072 ± 2% interrupts.CPU45.LOC:Local_timer_interrupts
3056 ± 3% +22.2% 3734 ± 3% interrupts.CPU45.RES:Rescheduling_interrupts
16894 ± 4% +27.6% 21553 ± 7% interrupts.CPU46.CAL:Function_call_interrupts
902132 +13.5% 1024049 ± 2% interrupts.CPU46.LOC:Local_timer_interrupts
3069 ± 2% +22.9% 3773 ± 4% interrupts.CPU46.RES:Rescheduling_interrupts
16781 ± 5% +27.6% 21408 ± 7% interrupts.CPU47.CAL:Function_call_interrupts
902147 +13.5% 1024088 ± 2% interrupts.CPU47.LOC:Local_timer_interrupts
3086 ± 3% +20.9% 3732 ± 4% interrupts.CPU47.RES:Rescheduling_interrupts
16892 ± 5% +27.3% 21496 ± 7% interrupts.CPU48.CAL:Function_call_interrupts
902150 +13.5% 1024084 ± 2% interrupts.CPU48.LOC:Local_timer_interrupts
3123 ± 4% +19.1% 3721 ± 4% interrupts.CPU48.RES:Rescheduling_interrupts
16686 ± 4% +27.9% 21340 ± 7% interrupts.CPU49.CAL:Function_call_interrupts
902113 +13.5% 1024070 ± 2% interrupts.CPU49.LOC:Local_timer_interrupts
3039 ± 4% +22.8% 3733 ± 4% interrupts.CPU49.RES:Rescheduling_interrupts
17142 ± 5% +29.2% 22150 ± 7% interrupts.CPU5.CAL:Function_call_interrupts
902296 +13.5% 1024251 ± 2% interrupts.CPU5.LOC:Local_timer_interrupts
3178 ± 3% +21.7% 3866 ± 3% interrupts.CPU5.RES:Rescheduling_interrupts
16894 ± 4% +27.4% 21529 ± 6% interrupts.CPU50.CAL:Function_call_interrupts
902077 +13.5% 1024082 ± 2% interrupts.CPU50.LOC:Local_timer_interrupts
3106 ± 4% +21.0% 3759 ± 3% interrupts.CPU50.RES:Rescheduling_interrupts
16842 ± 4% +28.0% 21565 ± 6% interrupts.CPU51.CAL:Function_call_interrupts
902143 +13.5% 1024096 ± 2% interrupts.CPU51.LOC:Local_timer_interrupts
3106 ± 3% +20.1% 3730 ± 3% interrupts.CPU51.RES:Rescheduling_interrupts
16754 ± 5% +28.8% 21571 ± 7% interrupts.CPU52.CAL:Function_call_interrupts
902098 +13.5% 1024069 ± 2% interrupts.CPU52.LOC:Local_timer_interrupts
3050 ± 2% +21.8% 3715 ± 4% interrupts.CPU52.RES:Rescheduling_interrupts
16868 ± 5% +27.3% 21481 ± 6% interrupts.CPU53.CAL:Function_call_interrupts
902150 +13.5% 1024062 ± 2% interrupts.CPU53.LOC:Local_timer_interrupts
3054 ± 3% +22.1% 3729 ± 4% interrupts.CPU53.RES:Rescheduling_interrupts
16759 ± 5% +27.2% 21311 ± 7% interrupts.CPU54.CAL:Function_call_interrupts
902146 +13.5% 1024081 ± 2% interrupts.CPU54.LOC:Local_timer_interrupts
3061 ± 4% +23.5% 3780 ± 5% interrupts.CPU54.RES:Rescheduling_interrupts
16893 ± 4% +26.4% 21358 ± 7% interrupts.CPU55.CAL:Function_call_interrupts
902113 +13.5% 1024081 ± 2% interrupts.CPU55.LOC:Local_timer_interrupts
3073 ± 3% +21.8% 3742 ± 5% interrupts.CPU55.RES:Rescheduling_interrupts
16820 ± 4% +26.6% 21288 ± 7% interrupts.CPU56.CAL:Function_call_interrupts
902136 +13.5% 1024085 ± 2% interrupts.CPU56.LOC:Local_timer_interrupts
3052 ± 4% +22.3% 3734 ± 5% interrupts.CPU56.RES:Rescheduling_interrupts
16837 ± 4% +26.6% 21324 ± 7% interrupts.CPU57.CAL:Function_call_interrupts
902135 +13.5% 1024226 ± 2% interrupts.CPU57.LOC:Local_timer_interrupts
3075 ± 3% +21.1% 3723 ± 3% interrupts.CPU57.RES:Rescheduling_interrupts
16851 ± 5% +26.7% 21359 ± 7% interrupts.CPU58.CAL:Function_call_interrupts
902145 +13.5% 1024084 ± 2% interrupts.CPU58.LOC:Local_timer_interrupts
3067 ± 3% +22.0% 3742 ± 4% interrupts.CPU58.RES:Rescheduling_interrupts
16813 ± 4% +27.8% 21480 ± 7% interrupts.CPU59.CAL:Function_call_interrupts
902135 +13.5% 1024105 ± 2% interrupts.CPU59.LOC:Local_timer_interrupts
3075 ± 3% +23.1% 3784 ± 5% interrupts.CPU59.RES:Rescheduling_interrupts
17318 ± 4% +25.8% 21778 ± 6% interrupts.CPU6.CAL:Function_call_interrupts
902058 +13.5% 1024091 ± 2% interrupts.CPU6.LOC:Local_timer_interrupts
3167 ± 4% +22.1% 3866 ± 4% interrupts.CPU6.RES:Rescheduling_interrupts
16910 ± 4% +26.7% 21430 ± 7% interrupts.CPU60.CAL:Function_call_interrupts
902154 +13.5% 1024028 ± 2% interrupts.CPU60.LOC:Local_timer_interrupts
3097 ± 3% +21.4% 3760 ± 4% interrupts.CPU60.RES:Rescheduling_interrupts
16900 ± 4% +27.5% 21555 ± 7% interrupts.CPU61.CAL:Function_call_interrupts
902139 +13.5% 1024089 ± 2% interrupts.CPU61.LOC:Local_timer_interrupts
3063 ± 4% +22.0% 3738 ± 3% interrupts.CPU61.RES:Rescheduling_interrupts
16991 ± 4% +26.0% 21414 ± 7% interrupts.CPU62.CAL:Function_call_interrupts
902132 +13.5% 1024089 ± 2% interrupts.CPU62.LOC:Local_timer_interrupts
3125 ± 3% +19.7% 3742 ± 3% interrupts.CPU62.RES:Rescheduling_interrupts
16967 ± 5% +26.9% 21537 ± 6% interrupts.CPU63.CAL:Function_call_interrupts
902111 +13.5% 1024071 ± 2% interrupts.CPU63.LOC:Local_timer_interrupts
3110 ± 3% +22.1% 3799 ± 4% interrupts.CPU63.RES:Rescheduling_interrupts
16832 ± 4% +28.1% 21557 ± 7% interrupts.CPU64.CAL:Function_call_interrupts
902165 +13.5% 1024037 ± 2% interrupts.CPU64.LOC:Local_timer_interrupts
3095 ± 3% +22.5% 3791 ± 3% interrupts.CPU64.RES:Rescheduling_interrupts
16887 ± 5% +27.7% 21563 ± 7% interrupts.CPU65.CAL:Function_call_interrupts
902159 +13.5% 1024083 ± 2% interrupts.CPU65.LOC:Local_timer_interrupts
3079 ± 4% +23.5% 3802 ± 4% interrupts.CPU65.RES:Rescheduling_interrupts
18236 ± 10% +43.5% 26177 ± 8% interrupts.CPU66.CAL:Function_call_interrupts
902097 +13.5% 1023991 ± 2% interrupts.CPU66.LOC:Local_timer_interrupts
2952 ± 3% +16.0% 3423 ± 3% interrupts.CPU66.RES:Rescheduling_interrupts
18440 ± 11% +41.6% 26117 ± 7% interrupts.CPU67.CAL:Function_call_interrupts
902050 +13.5% 1024090 ± 2% interrupts.CPU67.LOC:Local_timer_interrupts
2948 ± 2% +16.8% 3443 ± 3% interrupts.CPU67.RES:Rescheduling_interrupts
18284 ± 10% +43.0% 26148 ± 7% interrupts.CPU68.CAL:Function_call_interrupts
902119 +13.5% 1024096 ± 2% interrupts.CPU68.LOC:Local_timer_interrupts
2960 ± 2% +17.7% 3485 ± 3% interrupts.CPU68.RES:Rescheduling_interrupts
18357 ± 10% +42.0% 26070 ± 7% interrupts.CPU69.CAL:Function_call_interrupts
902096 +13.5% 1024089 ± 2% interrupts.CPU69.LOC:Local_timer_interrupts
2977 +16.1% 3455 ± 5% interrupts.CPU69.RES:Rescheduling_interrupts
16984 ± 5% +28.6% 21843 ± 6% interrupts.CPU7.CAL:Function_call_interrupts
902119 +13.5% 1024116 ± 2% interrupts.CPU7.LOC:Local_timer_interrupts
3122 ± 2% +22.8% 3835 ± 4% interrupts.CPU7.RES:Rescheduling_interrupts
18176 ± 10% +43.4% 26056 ± 7% interrupts.CPU70.CAL:Function_call_interrupts
902106 +13.5% 1024063 ± 2% interrupts.CPU70.LOC:Local_timer_interrupts
2937 ± 2% +18.5% 3479 ± 2% interrupts.CPU70.RES:Rescheduling_interrupts
18320 ± 10% +42.2% 26047 ± 8% interrupts.CPU71.CAL:Function_call_interrupts
902142 +13.5% 1024123 ± 2% interrupts.CPU71.LOC:Local_timer_interrupts
2999 ± 3% +16.3% 3487 ± 3% interrupts.CPU71.RES:Rescheduling_interrupts
18410 ± 10% +42.0% 26143 ± 7% interrupts.CPU72.CAL:Function_call_interrupts
902129 +13.5% 1024094 ± 2% interrupts.CPU72.LOC:Local_timer_interrupts
2965 ± 3% +17.6% 3488 ± 4% interrupts.CPU72.RES:Rescheduling_interrupts
18305 ± 10% +42.5% 26085 ± 8% interrupts.CPU73.CAL:Function_call_interrupts
902133 +13.5% 1024041 ± 2% interrupts.CPU73.LOC:Local_timer_interrupts
2977 ± 3% +16.6% 3471 ± 3% interrupts.CPU73.RES:Rescheduling_interrupts
18344 ± 10% +42.6% 26152 ± 7% interrupts.CPU74.CAL:Function_call_interrupts
902128 +13.5% 1024065 ± 2% interrupts.CPU74.LOC:Local_timer_interrupts
3002 ± 2% +17.0% 3511 ± 3% interrupts.CPU74.RES:Rescheduling_interrupts
18310 ± 10% +41.9% 25976 ± 7% interrupts.CPU75.CAL:Function_call_interrupts
902143 +13.5% 1024072 ± 2% interrupts.CPU75.LOC:Local_timer_interrupts
2922 ± 3% +18.5% 3463 ± 3% interrupts.CPU75.RES:Rescheduling_interrupts
18275 ± 10% +41.6% 25883 ± 7% interrupts.CPU76.CAL:Function_call_interrupts
902099 +13.5% 1024098 ± 2% interrupts.CPU76.LOC:Local_timer_interrupts
2921 ± 4% +18.3% 3457 ± 2% interrupts.CPU76.RES:Rescheduling_interrupts
18330 ± 10% +42.1% 26054 ± 7% interrupts.CPU77.CAL:Function_call_interrupts
902158 +13.5% 1024103 ± 2% interrupts.CPU77.LOC:Local_timer_interrupts
2960 ± 3% +16.4% 3447 ± 3% interrupts.CPU77.RES:Rescheduling_interrupts
18199 ± 10% +43.9% 26183 ± 8% interrupts.CPU78.CAL:Function_call_interrupts
902127 +13.5% 1024113 ± 2% interrupts.CPU78.LOC:Local_timer_interrupts
2936 ± 3% +17.8% 3460 ± 2% interrupts.CPU78.RES:Rescheduling_interrupts
18302 ± 9% +42.2% 26018 ± 7% interrupts.CPU79.CAL:Function_call_interrupts
902128 +13.5% 1024093 ± 2% interrupts.CPU79.LOC:Local_timer_interrupts
2933 ± 3% +16.1% 3406 ± 2% interrupts.CPU79.RES:Rescheduling_interrupts
17257 ± 4% +26.1% 21766 ± 6% interrupts.CPU8.CAL:Function_call_interrupts
902064 +13.5% 1024249 ± 2% interrupts.CPU8.LOC:Local_timer_interrupts
3148 ± 4% +22.8% 3867 ± 4% interrupts.CPU8.RES:Rescheduling_interrupts
18351 ± 10% +41.6% 25983 ± 8% interrupts.CPU80.CAL:Function_call_interrupts
902119 +13.5% 1024104 ± 2% interrupts.CPU80.LOC:Local_timer_interrupts
2950 ± 2% +15.0% 3393 ± 2% interrupts.CPU80.RES:Rescheduling_interrupts
18351 ± 10% +42.5% 26147 ± 8% interrupts.CPU81.CAL:Function_call_interrupts
902138 +13.5% 1024087 ± 2% interrupts.CPU81.LOC:Local_timer_interrupts
2952 ± 2% +16.0% 3424 ± 2% interrupts.CPU81.RES:Rescheduling_interrupts
18259 ± 10% +42.4% 26004 ± 7% interrupts.CPU82.CAL:Function_call_interrupts
902149 +13.5% 1024272 ± 2% interrupts.CPU82.LOC:Local_timer_interrupts
2931 ± 2% +17.0% 3429 ± 2% interrupts.CPU82.RES:Rescheduling_interrupts
18210 ± 10% +42.8% 26012 ± 8% interrupts.CPU83.CAL:Function_call_interrupts
902111 +13.5% 1024099 ± 2% interrupts.CPU83.LOC:Local_timer_interrupts
2933 ± 2% +16.6% 3419 ± 2% interrupts.CPU83.RES:Rescheduling_interrupts
18085 ± 10% +44.2% 26086 ± 8% interrupts.CPU84.CAL:Function_call_interrupts
902118 +13.5% 1024109 ± 2% interrupts.CPU84.LOC:Local_timer_interrupts
2981 ± 2% +13.9% 3396 ± 2% interrupts.CPU84.RES:Rescheduling_interrupts
18294 ± 10% +43.0% 26167 ± 7% interrupts.CPU85.CAL:Function_call_interrupts
902135 +13.5% 1024098 ± 2% interrupts.CPU85.LOC:Local_timer_interrupts
2937 ± 2% +16.8% 3430 ± 3% interrupts.CPU85.RES:Rescheduling_interrupts
18253 ± 10% +42.7% 26056 ± 8% interrupts.CPU86.CAL:Function_call_interrupts
902302 +13.5% 1024102 ± 2% interrupts.CPU86.LOC:Local_timer_interrupts
2999 ± 2% +13.5% 3403 ± 3% interrupts.CPU86.RES:Rescheduling_interrupts
18248 ± 10% +41.5% 25817 ± 8% interrupts.CPU87.CAL:Function_call_interrupts
902127 +13.5% 1024064 ± 2% interrupts.CPU87.LOC:Local_timer_interrupts
2954 ± 2% +14.3% 3377 ± 2% interrupts.CPU87.RES:Rescheduling_interrupts
16991 ± 4% +26.9% 21557 ± 6% interrupts.CPU9.CAL:Function_call_interrupts
902162 +13.5% 1024110 ± 2% interrupts.CPU9.LOC:Local_timer_interrupts
3164 ± 3% +22.5% 3876 ± 5% interrupts.CPU9.RES:Rescheduling_interrupts
79387471 +13.5% 90119618 ± 2% interrupts.LOC:Local_timer_interrupts
269188 +19.2% 320766 ± 2% interrupts.RES:Rescheduling_interrupts
aim7.jobs-per-min
14000 +-------------------------------------------------------------------+
| + |
13500 |-+ + + ++ + +.+ + + + .+ |
| .+ + + : :: : : :: : :.+. + + : .++ + +.|
|. +.+ + : : :: : :: : + ++.+. + +. + + :: .+ + |
13000 |-+ + + :: + + + + :: + |
| : + : : |
12500 |-+ + :: |
| :: |
12000 |-+ O O O O O O O + |
| O O O O O O |
| O O O O O O O O |
11500 |-+O O O O O |
| O O |
11000 +-------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-knl-f1: 256 threads Intel(R) Genuine Intel(R) CPU 0000 @ 1.30GHz with 112G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/tbox_group/test/testcase/testtime/ucode:
gcc-9/performance/x86_64-rhel-8.3/debian-10.4-x86_64-20200603.cgz/lkp-knl-f1/all/aim9/5s/0xffff0190
commit:
4392fbc4ba ("ext4: drop sync argument of ext4_commit_super()")
05c2c00f37 ("ext4: protect superblock modifications with a buffer lock")
4392fbc4bab57db3 05c2c00f3769abb9e323fcaca70
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:5 20% 1:6 dmesg.WARNING:missing_R10_value_at__fsnotify_parent/0x
0:5 4% 0:6 perf-profile.children.cycles-pp.error_return
2:5 8% 2:6 perf-profile.self.cycles-pp.error_entry
%stddev %change %stddev
\ | \
74384 +3.4% 76883 aim9.dgram_pipe.ops_per_sec
121865 +2.6% 125020 aim9.disk_cp.ops_per_sec
198997 +1.9% 202688 aim9.disk_wrt.ops_per_sec
52029 -3.9% 49990 aim9.link_test.ops_per_sec
95645 +7.5% 102865 aim9.signal_test.ops_per_sec
80028 +5.0% 84023 aim9.stream_pipe.ops_per_sec
101888 +3.9% 105872 aim9.sync_disk_rw.ops_per_sec
17794 +1.5% 18069 aim9.tcp_test.ops_per_sec
35329 +2.1% 36070 aim9.udp_test.ops_per_sec
84505 +1.3% 85562 proc-vmstat.pgactivate
1.037e+08 -6.0% 97479544 ± 6% perf-stat.i.cache-references
1.034e+08 -6.0% 97161961 ± 6% perf-stat.ps.cache-references
412.50 ± 3% +25.8% 519.03 ± 9% sched_debug.cfs_rq:/.util_est_enqueued.max
45.99 ± 10% +31.6% 60.51 ± 11% sched_debug.cfs_rq:/.util_est_enqueued.stddev
20873 ± 13% -16.2% 17493 ± 6% softirqs.CPU134.RCU
10054 ± 56% -81.3% 1880 ±126% softirqs.CPU14.NET_RX
0.05 ± 6% +45.0% 0.08 ± 28% perf-sched.sch_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
0.04 ± 4% +22.3% 0.05 ± 10% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
0.01 ±123% +153.5% 0.04 ± 24% perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.__flush_work.lru_add_drain_all
0.09 ± 18% +82.9% 0.17 ± 52% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
0.01 ±123% +175.8% 0.04 ± 12% perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.__flush_work.lru_add_drain_all
141.15 ± 21% -41.1% 83.12 ± 41% perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
0.18 ± 40% +70.5% 0.31 ± 41% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.dput.path_put.set_fs_pwd
0.23 ± 33% +59.8% 0.37 ± 26% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.getname_flags.__x64_sys_unlink
141.12 ± 21% -41.1% 83.09 ± 41% perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
305.60 ± 29% -43.3% 173.17 ± 21% interrupts.CPU114.NMI:Non-maskable_interrupts
305.60 ± 29% -43.3% 173.17 ± 21% interrupts.CPU114.PMI:Performance_monitoring_interrupts
335.00 ± 20% -36.6% 212.33 ± 32% interrupts.CPU239.NMI:Non-maskable_interrupts
335.00 ± 20% -36.6% 212.33 ± 32% interrupts.CPU239.PMI:Performance_monitoring_interrupts
314.80 ± 32% -37.0% 198.33 ± 46% interrupts.CPU251.NMI:Non-maskable_interrupts
314.80 ± 32% -37.0% 198.33 ± 46% interrupts.CPU251.PMI:Performance_monitoring_interrupts
23.20 ± 44% +117.7% 50.50 ± 57% interrupts.CPU53.RES:Rescheduling_interrupts
1417 ± 10% +24.6% 1766 ± 13% interrupts.CPU6.CAL:Function_call_interrupts
30.40 ± 59% +198.8% 90.83 ± 23% interrupts.CPU6.RES:Rescheduling_interrupts
1260 ± 5% +9.3% 1377 ± 7% interrupts.CPU7.CAL:Function_call_interrupts
253.40 ± 29% +46.7% 371.83 ± 27% interrupts.CPU90.NMI:Non-maskable_interrupts
253.40 ± 29% +46.7% 371.83 ± 27% interrupts.CPU90.PMI:Performance_monitoring_interrupts
1.07 ± 2% -0.2 0.86 ± 24% perf-profile.calltrace.cycles-pp.irqtime_account_irq.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
0.75 ± 4% +0.3 1.03 ± 34% perf-profile.calltrace.cycles-pp.timerqueue_add.enqueue_hrtimer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
0.90 ± 5% +0.3 1.21 ± 33% perf-profile.calltrace.cycles-pp.enqueue_hrtimer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack
2.40 ± 6% +0.4 2.76 ± 9% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt
1.42 ± 7% +0.4 1.84 ± 14% perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack
0.33 ± 7% -0.1 0.25 ± 11% perf-profile.children.cycles-pp.irq_work_needs_cpu
0.09 ± 13% +0.0 0.11 ± 8% perf-profile.children.cycles-pp.hrtimer_run_queues
0.09 ± 9% +0.0 0.12 ± 19% perf-profile.children.cycles-pp.__set_cpus_allowed_ptr
0.06 ± 12% +0.0 0.09 ± 13% perf-profile.children.cycles-pp.irq_work_single
0.06 ± 12% +0.0 0.09 ± 13% perf-profile.children.cycles-pp.asm_sysvec_irq_work
0.06 ± 12% +0.0 0.09 ± 13% perf-profile.children.cycles-pp.sysvec_irq_work
0.06 ± 12% +0.0 0.09 ± 13% perf-profile.children.cycles-pp.__sysvec_irq_work
0.06 ± 12% +0.0 0.09 ± 13% perf-profile.children.cycles-pp.irq_work_run
0.11 ± 8% +0.0 0.15 ± 22% perf-profile.children.cycles-pp.__x64_sys_sched_setaffinity
0.14 ± 5% +0.1 0.19 ± 17% perf-profile.children.cycles-pp.irq_work_run_list
0.29 ± 6% +0.1 0.39 ± 28% perf-profile.children.cycles-pp.rb_insert_color
0.77 ± 4% +0.3 1.07 ± 34% perf-profile.children.cycles-pp.timerqueue_add
0.92 ± 5% +0.3 1.23 ± 32% perf-profile.children.cycles-pp.enqueue_hrtimer
2.45 ± 6% +0.4 2.82 ± 9% perf-profile.children.cycles-pp.clockevents_program_event
4.23 ± 2% +0.4 4.61 ± 7% perf-profile.children.cycles-pp.scheduler_tick
0.97 ± 2% -0.1 0.83 ± 15% perf-profile.self.cycles-pp.irqtime_account_irq
0.75 ± 4% -0.1 0.67 ± 6% perf-profile.self.cycles-pp.__sysvec_apic_timer_interrupt
0.30 ± 8% -0.1 0.23 ± 15% perf-profile.self.cycles-pp.irq_work_needs_cpu
0.19 ± 3% +0.0 0.21 ± 6% perf-profile.self.cycles-pp.trigger_load_balance
0.09 ± 13% +0.0 0.11 ± 8% perf-profile.self.cycles-pp.hrtimer_run_queues
0.02 ±122% +0.0 0.06 ± 14% perf-profile.self.cycles-pp.io_serial_in
0.14 ± 11% +0.0 0.19 ± 22% perf-profile.self.cycles-pp.tick_sched_handle
0.27 ± 4% +0.1 0.37 ± 27% perf-profile.self.cycles-pp.rb_insert_color
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Oliver Sang
View attachment "config-5.10.0-rc5-00032-g05c2c00f3769" of type "text/plain" (171050 bytes)
View attachment "job-script" of type "text/plain" (8055 bytes)
View attachment "job.yaml" of type "text/plain" (5554 bytes)
View attachment "reproduce" of type "text/plain" (1011 bytes)
Powered by blists - more mailing lists