lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <20200622012238.GN5535@shao2-debian>
Date:   Mon, 22 Jun 2020 09:22:39 +0800
From:   kernel test robot <rong.a.chen@...el.com>
To:     zhangyi <yi.zhang@...wei.com>
Cc:     Theodore Ts'o <tytso@....edu>, Jan Kara <jack@...e.cz>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org
Subject: [ext4, jbd2] 7b97d868b7: aim7.jobs-per-min 21.9% improvement

Greeting,

FYI, we noticed a 21.9% improvement of aim7.jobs-per-min due to commit:


commit: 7b97d868b7ab2448859668de9222b8af43f76e78 ("ext4, jbd2: ensure panic by fix a race between jbd2 abort and ext4 error handlers")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

in testcase: aim7
on test machine: 72 threads Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz with 128G memory
with following parameters:

	disk: 4BRD_12G
	md: RAID0
	fs: ext4
	test: disk_cp
	load: 3000
	cpufreq_governor: performance
	ucode: 0x2006906

test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system.
test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/





Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase/ucode:
  gcc-9/performance/4BRD_12G/ext4/x86_64-rhel-7.6/3000/RAID0/debian-x86_64-20191114.cgz/lkp-skl-2sp7/disk_cp/aim7/0x2006906

commit: 
  88ee9d571b ("ext4: support xattr gnu.* namespace for the Hurd")
  7b97d868b7 ("ext4, jbd2: ensure panic by fix a race between jbd2 abort and ext4 error handlers")

88ee9d571b6d8ed3 7b97d868b7ab2448859668de922 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    109637 ±  4%     +21.9%     133664 ±  3%  aim7.jobs-per-min
    164.62 ±  4%     -18.0%     135.04 ±  3%  aim7.time.elapsed_time
    164.62 ±  4%     -18.0%     135.04 ±  3%  aim7.time.elapsed_time.max
   1088548 ±  9%     -26.3%     802133        aim7.time.involuntary_context_switches
     10428 ±  4%     -19.1%       8433 ±  4%  aim7.time.system_time
    627.87 ± 12%     -21.9%     490.20        aim7.time.user_time
    125578 ±  2%     -12.7%     109582 ±  4%  aim7.time.voluntary_context_switches
      7.20 ±  5%      +1.5        8.71 ±  2%  mpstat.cpu.all.idle%
   2260125 ±  7%     -20.5%    1797236 ± 18%  numa-meminfo.node1.MemUsed
     61121 ± 53%     -43.9%      34289 ± 88%  numa-vmstat.node1.nr_written
    134480 ±  4%     -13.8%     115956 ±  3%  meminfo.AnonHugePages
     26272 ±  3%     +20.4%      31621 ±  4%  meminfo.max_used_kB
      7.25 ±  5%     +27.6%       9.25 ±  4%  vmstat.cpu.id
      8701 ±  3%      -6.6%       8123        vmstat.system.cs
      8.13 ±  6%     +22.7%       9.98 ±  2%  iostat.cpu.idle
     86.55            -1.8%      84.96        iostat.cpu.system
    115.61 ±  5%      +9.7%     126.79 ±  5%  iostat.md0.w/s
      2667 ±  2%      -9.5%       2413 ±  3%  slabinfo.fsnotify_mark_connector.active_objs
      2667 ±  2%      -9.5%       2413 ±  3%  slabinfo.fsnotify_mark_connector.num_objs
      3560 ±  8%     -10.1%       3200 ±  7%  slabinfo.skbuff_head_cache.active_objs
      3560 ±  8%     -10.1%       3200 ±  7%  slabinfo.skbuff_head_cache.num_objs
      3861 ± 15%     -25.7%       2867 ± 25%  sched_debug.cfs_rq:/.load.min
      1947 ±  7%     +18.4%       2305 ±  5%  sched_debug.cfs_rq:/.util_est_enqueued.max
     28.04 ± 15%     +54.9%      43.44 ± 21%  sched_debug.cpu.clock.stddev
     28.04 ± 15%     +54.9%      43.44 ± 21%  sched_debug.cpu.clock_task.stddev
      0.92 ±  4%     +10.5%       1.01 ±  3%  sched_debug.cpu.nr_running.stddev
     25.68           -14.5%      21.96 ±  9%  sched_debug.cpu.nr_uninterruptible.avg
     76.92 ±  7%     +24.8%      96.00 ±  6%  sched_debug.cpu.sched_goidle.min
    307.21 ±  2%     -11.7%     271.42 ±  2%  sched_debug.cpu.sched_goidle.stddev
    114197            -1.0%     113083        proc-vmstat.nr_active_anon
     94342            -1.4%      93030        proc-vmstat.nr_anon_pages
     23557 ±  4%      -7.6%      21758 ±  3%  proc-vmstat.nr_dirty
      5070            +3.3%       5239        proc-vmstat.nr_inactive_anon
     23412 ±  4%      -7.6%      21639 ±  3%  proc-vmstat.nr_inactive_file
     56234            -1.0%      55693        proc-vmstat.nr_kernel_stack
      7851            +2.8%       8072        proc-vmstat.nr_mapped
    126171 ±  4%     -32.5%      85223 ± 11%  proc-vmstat.nr_written
    114197            -1.0%     113083        proc-vmstat.nr_zone_active_anon
      5070            +3.3%       5239        proc-vmstat.nr_zone_inactive_anon
     23412 ±  4%      -7.6%      21639 ±  3%  proc-vmstat.nr_zone_inactive_file
     23931 ±  5%      -7.7%      22089 ±  3%  proc-vmstat.nr_zone_write_pending
     21579 ±  9%     -23.2%      16576 ± 25%  proc-vmstat.numa_hint_faults_local
    632388           -11.0%     562909 ±  3%  proc-vmstat.pgfault
   1263693 ±  3%     -10.5%    1130597 ±  3%  proc-vmstat.pgpgout
      1699 ±  2%      -2.0%       1665 ±  2%  proc-vmstat.unevictable_pgs_culled
 3.204e+09 ±  3%     +20.4%  3.858e+09 ±  3%  perf-stat.i.branch-instructions
      1.38            +0.0        1.42        perf-stat.i.branch-miss-rate%
  40999474 ±  3%     +19.6%   49041131 ±  4%  perf-stat.i.branch-misses
      8731 ±  3%      -6.6%       8151        perf-stat.i.context-switches
     12.07 ±  4%     -19.4%       9.73 ±  3%  perf-stat.i.cpi
  2.06e+11            -1.7%  2.024e+11        perf-stat.i.cpu-cycles
   4220143 ±  4%     +20.0%    5062853 ±  3%  perf-stat.i.dTLB-load-misses
  4.91e+09 ±  3%     +20.6%  5.923e+09 ±  3%  perf-stat.i.dTLB-loads
 2.703e+09 ±  3%     +20.8%  3.266e+09 ±  3%  perf-stat.i.dTLB-stores
    385155 ±  8%     +15.6%     445091 ±  4%  perf-stat.i.iTLB-loads
 1.639e+10 ±  3%     +20.5%  1.975e+10 ±  3%  perf-stat.i.instructions
      0.12 ±  3%     +21.4%       0.14 ±  3%  perf-stat.i.ipc
      2.86            -1.9%       2.80        perf-stat.i.metric.GHz
    155.27 ±  3%     +20.2%     186.58 ±  2%  perf-stat.i.metric.M/sec
      3719 ±  2%      +7.3%       3991        perf-stat.i.minor-faults
     77.09            -2.3       74.80        perf-stat.i.node-store-miss-rate%
   7438481 ±  2%      +6.9%    7951338 ±  2%  perf-stat.i.node-store-misses
   1893208 ±  3%     +18.0%    2233804 ±  3%  perf-stat.i.node-stores
      3719 ±  2%      +7.3%       3991        perf-stat.i.page-faults
     12.59 ±  4%     -18.5%      10.26 ±  3%  perf-stat.overall.cpi
      0.00 ±  8%      -0.0        0.00 ±  3%  perf-stat.overall.dTLB-store-miss-rate%
      3860 ±  9%      +6.8%       4121        perf-stat.overall.instructions-per-iTLB-miss
      0.08 ±  4%     +22.7%       0.10 ±  3%  perf-stat.overall.ipc
     79.71            -1.6       78.07        perf-stat.overall.node-store-miss-rate%
 3.187e+09 ±  3%     +20.3%  3.834e+09 ±  3%  perf-stat.ps.branch-instructions
  40731134 ±  3%     +19.5%   48666645 ±  4%  perf-stat.ps.branch-misses
      8685 ±  3%      -6.7%       8102        perf-stat.ps.context-switches
 2.049e+11            -1.9%  2.011e+11        perf-stat.ps.cpu-cycles
   4199062 ±  4%     +19.9%    5034263 ±  3%  perf-stat.ps.dTLB-load-misses
 4.884e+09 ±  3%     +20.5%  5.887e+09 ±  3%  perf-stat.ps.dTLB-loads
 2.689e+09 ±  3%     +20.7%  3.246e+09 ±  3%  perf-stat.ps.dTLB-stores
    381357 ±  8%     +15.4%     439980 ±  3%  perf-stat.ps.iTLB-loads
  1.63e+10 ±  3%     +20.4%  1.962e+10 ±  3%  perf-stat.ps.instructions
      3686 ±  2%      +7.4%       3959        perf-stat.ps.minor-faults
   7401884 ±  2%      +6.7%    7900996 ±  2%  perf-stat.ps.node-store-misses
   1883887 ±  3%     +17.9%    2220702 ±  3%  perf-stat.ps.node-stores
      3686 ±  2%      +7.4%       3959        perf-stat.ps.page-faults
 2.693e+12            -1.2%  2.662e+12        perf-stat.total.instructions
     64184 ±  3%     -16.6%      53552 ±  2%  softirqs.CPU0.TIMER
     60589 ±  2%     -16.7%      50454 ±  4%  softirqs.CPU1.TIMER
     10415 ± 11%     -15.8%       8773 ± 12%  softirqs.CPU10.RCU
     59301 ±  3%     -15.0%      50407 ±  4%  softirqs.CPU10.TIMER
     10493 ±  9%     -16.1%       8801 ± 13%  softirqs.CPU11.RCU
     60567 ±  4%     -17.3%      50092 ±  3%  softirqs.CPU11.TIMER
     10298 ± 10%     -15.8%       8668 ± 14%  softirqs.CPU12.RCU
     59585 ±  4%     -15.6%      50282 ±  4%  softirqs.CPU12.TIMER
     10500 ± 10%     -15.6%       8861 ± 13%  softirqs.CPU13.RCU
     59684 ±  4%     -15.8%      50267 ±  3%  softirqs.CPU13.TIMER
     59617 ±  3%     -15.9%      50167 ±  3%  softirqs.CPU14.TIMER
      9982 ± 10%     -18.8%       8107 ± 13%  softirqs.CPU15.RCU
     59668 ±  3%     -16.1%      50085 ±  3%  softirqs.CPU15.TIMER
     59706 ±  3%     -14.5%      51063 ±  4%  softirqs.CPU16.TIMER
     59376 ±  3%     -14.0%      51075 ±  2%  softirqs.CPU17.TIMER
     10985 ±  7%     -17.1%       9109 ± 11%  softirqs.CPU18.RCU
     60338 ±  3%     -17.0%      50086 ±  2%  softirqs.CPU18.TIMER
     60430 ±  4%     -15.4%      51143        softirqs.CPU19.TIMER
     59171           -15.1%      50253 ±  3%  softirqs.CPU2.TIMER
     59362 ±  5%     -14.5%      50763 ±  4%  softirqs.CPU20.TIMER
     63667 ±  4%     -18.1%      52139 ±  3%  softirqs.CPU21.TIMER
     10829 ± 10%     -14.6%       9248 ± 11%  softirqs.CPU22.RCU
     59879 ±  3%     -16.4%      50057        softirqs.CPU22.TIMER
     59412 ±  4%     -14.8%      50610 ±  3%  softirqs.CPU23.TIMER
     10890 ±  7%     -17.8%       8954 ± 13%  softirqs.CPU24.RCU
     60362 ±  2%     -17.2%      49968 ±  3%  softirqs.CPU24.TIMER
     10686 ± 10%     -16.7%       8898 ± 11%  softirqs.CPU25.RCU
     10590 ±  9%     -16.3%       8859 ±  9%  softirqs.CPU26.RCU
     10657 ±  9%     -16.6%       8889 ± 12%  softirqs.CPU27.RCU
     59522 ±  4%     -16.5%      49685 ±  2%  softirqs.CPU27.TIMER
     10552 ± 10%     -14.6%       9008 ± 13%  softirqs.CPU28.RCU
     59341 ±  3%     -16.4%      49587 ±  3%  softirqs.CPU28.TIMER
     10595 ±  9%     -16.1%       8892 ± 12%  softirqs.CPU29.RCU
     59433 ±  4%     -16.4%      49670 ±  2%  softirqs.CPU29.TIMER
     10645 ±  7%     -19.1%       8608 ± 14%  softirqs.CPU3.RCU
     59761 ±  3%     -16.5%      49909 ±  4%  softirqs.CPU3.TIMER
     59343 ±  3%     -15.8%      49976 ±  3%  softirqs.CPU30.TIMER
     10139 ± 10%     -15.9%       8531 ± 13%  softirqs.CPU31.RCU
      9998 ±  9%     -14.9%       8504 ± 13%  softirqs.CPU32.RCU
     59320 ±  4%     -16.4%      49564 ±  2%  softirqs.CPU32.TIMER
     10150 ±  9%     -15.6%       8567 ± 13%  softirqs.CPU33.RCU
     60084 ±  3%     -17.3%      49677 ±  2%  softirqs.CPU33.TIMER
     59972 ±  4%     -16.9%      49832 ±  2%  softirqs.CPU34.TIMER
     59801 ±  3%     -16.8%      49764 ±  3%  softirqs.CPU35.TIMER
     10485 ± 10%     -13.9%       9031 ± 11%  softirqs.CPU36.RCU
     58968 ±  3%     -14.2%      50566 ±  4%  softirqs.CPU36.TIMER
     59613 ±  3%     -16.0%      50063 ±  3%  softirqs.CPU37.TIMER
     67254 ± 24%     -25.5%      50126 ±  3%  softirqs.CPU38.TIMER
     58187 ±  3%     -12.7%      50775 ±  5%  softirqs.CPU4.TIMER
     12276 ± 22%     -25.8%       9108 ± 16%  softirqs.CPU40.RCU
     67736 ± 16%     -26.5%      49760 ±  3%  softirqs.CPU40.TIMER
     10713 ±  9%     -15.4%       9066 ± 10%  softirqs.CPU41.RCU
     59169 ±  3%     -14.7%      50450 ±  4%  softirqs.CPU41.TIMER
     10673 ±  9%     -18.5%       8696 ± 15%  softirqs.CPU42.RCU
     59029 ±  3%     -16.4%      49357 ±  4%  softirqs.CPU42.TIMER
     58432 ±  3%     -14.8%      49772 ±  3%  softirqs.CPU43.TIMER
     10889 ± 10%     -15.4%       9210 ± 13%  softirqs.CPU44.RCU
     59668 ±  6%     -16.2%      49984 ±  3%  softirqs.CPU44.TIMER
     10674 ± 10%     -16.0%       8963 ± 12%  softirqs.CPU45.RCU
     58719 ±  3%     -14.6%      50149 ±  4%  softirqs.CPU45.TIMER
     10757 ±  8%     -15.2%       9127 ± 11%  softirqs.CPU46.RCU
     58877 ±  3%     -14.6%      50266 ±  3%  softirqs.CPU46.TIMER
     10928 ± 13%     -17.3%       9034 ± 11%  softirqs.CPU47.RCU
     59110 ±  3%     -15.2%      50112 ±  4%  softirqs.CPU47.TIMER
     10752 ± 10%     -17.4%       8886 ± 13%  softirqs.CPU48.RCU
     58637 ±  3%     -15.4%      49619 ±  3%  softirqs.CPU48.TIMER
     11096 ± 11%     -18.8%       9007 ± 12%  softirqs.CPU49.RCU
     59485 ±  4%     -16.0%      49967 ±  2%  softirqs.CPU49.TIMER
     59188 ±  3%     -15.5%      50042 ±  4%  softirqs.CPU5.TIMER
     10707 ± 10%     -16.8%       8904 ± 13%  softirqs.CPU50.RCU
     58785 ±  3%     -15.3%      49778 ±  3%  softirqs.CPU50.TIMER
     10731 ± 10%     -16.8%       8932 ± 14%  softirqs.CPU51.RCU
     58815 ±  4%     -15.7%      49566 ±  3%  softirqs.CPU51.TIMER
     10755 ± 10%     -15.7%       9069 ± 12%  softirqs.CPU52.RCU
     58771 ±  4%     -15.2%      49864 ±  3%  softirqs.CPU52.TIMER
     10827 ±  9%     -15.7%       9123 ± 12%  softirqs.CPU53.RCU
     58683 ±  3%     -14.1%      50390 ±  5%  softirqs.CPU53.TIMER
     10631 ± 12%     -17.7%       8745 ± 16%  softirqs.CPU54.RCU
     59992 ±  4%     -17.3%      49589 ±  2%  softirqs.CPU54.TIMER
     59639 ±  3%     -16.8%      49632 ±  2%  softirqs.CPU55.TIMER
     65164 ± 18%     -24.6%      49130 ±  4%  softirqs.CPU56.TIMER
     59936 ±  4%     -15.4%      50729 ±  5%  softirqs.CPU57.TIMER
     10338 ± 11%     -15.4%       8746 ± 11%  softirqs.CPU58.RCU
     59548 ±  4%     -16.9%      49492 ±  2%  softirqs.CPU58.TIMER
     59693 ±  4%     -15.4%      50526 ±  4%  softirqs.CPU59.TIMER
     10407 ±  9%     -17.2%       8614 ± 13%  softirqs.CPU6.RCU
      9729 ±  8%     -15.5%       8226 ± 12%  softirqs.CPU60.RCU
     60771 ±  2%     -18.5%      49556 ±  2%  softirqs.CPU60.TIMER
     59251 ±  3%     -16.7%      49351 ±  3%  softirqs.CPU61.TIMER
     59165 ±  4%     -16.1%      49637 ±  4%  softirqs.CPU62.TIMER
      9434 ±  7%     -15.4%       7977 ± 12%  softirqs.CPU63.RCU
     59463 ±  3%     -16.3%      49785 ±  3%  softirqs.CPU63.TIMER
      9657 ±  7%     -17.9%       7927 ± 14%  softirqs.CPU64.RCU
     59563 ±  4%     -17.3%      49274 ±  2%  softirqs.CPU64.TIMER
     59069 ±  4%     -16.3%      49427 ±  2%  softirqs.CPU65.TIMER
      9334 ±  9%     -14.3%       7997 ± 14%  softirqs.CPU66.RCU
     59034 ±  4%     -15.8%      49692 ±  2%  softirqs.CPU66.TIMER
      9457 ± 10%     -16.5%       7896 ± 15%  softirqs.CPU67.RCU
     58884 ±  4%     -16.5%      49197 ±  3%  softirqs.CPU67.TIMER
      9165 ± 10%     -11.3%       8128 ± 12%  softirqs.CPU68.RCU
     58785 ±  4%     -14.2%      50453 ±  2%  softirqs.CPU68.TIMER
      9300 ± 10%     -14.2%       7980 ± 13%  softirqs.CPU69.RCU
     59148 ±  4%     -15.0%      50280        softirqs.CPU69.TIMER
     10264 ± 10%     -15.7%       8649 ± 13%  softirqs.CPU7.RCU
     68132 ± 22%     -26.3%      50183 ±  4%  softirqs.CPU7.TIMER
      9381 ± 10%     -13.4%       8124 ± 13%  softirqs.CPU70.RCU
     59209 ±  4%     -15.8%      49863 ±  3%  softirqs.CPU70.TIMER
      9676 ± 10%     -15.4%       8185 ± 13%  softirqs.CPU71.RCU
     58594 ±  3%     -16.4%      48958 ±  2%  softirqs.CPU71.TIMER
     59381 ±  3%     -15.5%      50153 ±  4%  softirqs.CPU8.TIMER
     58928 ±  3%     -14.9%      50162 ±  3%  softirqs.CPU9.TIMER
    747298 ±  9%     -15.0%     635418 ± 13%  softirqs.RCU
   4318190 ±  3%     -15.8%    3634224 ±  3%  softirqs.TIMER
    144.25 ± 11%     -18.5%     117.50 ±  3%  interrupts.35:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc
    331.50 ±  4%     -17.9%     272.00 ±  3%  interrupts.9:IO-APIC.9-fasteoi.acpi
    331831 ±  3%     -17.8%     272770 ±  3%  interrupts.CPU0.LOC:Local_timer_interrupts
    331.50 ±  4%     -17.9%     272.00 ±  3%  interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi
    331972 ±  3%     -17.8%     272743 ±  3%  interrupts.CPU1.LOC:Local_timer_interrupts
      3675 ± 40%     -54.9%       1656 ± 24%  interrupts.CPU1.RES:Rescheduling_interrupts
    331904 ±  3%     -17.8%     272750 ±  3%  interrupts.CPU10.LOC:Local_timer_interrupts
      1641 ±  9%     -18.3%       1341 ±  4%  interrupts.CPU10.RES:Rescheduling_interrupts
    331710 ±  4%     -17.8%     272720 ±  3%  interrupts.CPU11.LOC:Local_timer_interrupts
      8368 ±  5%     -29.0%       5943 ± 27%  interrupts.CPU11.NMI:Non-maskable_interrupts
      8368 ±  5%     -29.0%       5943 ± 27%  interrupts.CPU11.PMI:Performance_monitoring_interrupts
      1816 ± 32%     -24.6%       1369        interrupts.CPU11.RES:Rescheduling_interrupts
    331959 ±  3%     -17.8%     272739 ±  3%  interrupts.CPU12.LOC:Local_timer_interrupts
      1576 ±  7%     -17.0%       1308 ±  5%  interrupts.CPU12.RES:Rescheduling_interrupts
    331990 ±  3%     -17.9%     272702 ±  3%  interrupts.CPU13.LOC:Local_timer_interrupts
    332095 ±  3%     -17.9%     272604 ±  3%  interrupts.CPU14.LOC:Local_timer_interrupts
      1550 ±  5%     -15.0%       1318 ±  5%  interrupts.CPU14.RES:Rescheduling_interrupts
    331980 ±  3%     -17.9%     272702 ±  3%  interrupts.CPU15.LOC:Local_timer_interrupts
      1674 ± 14%     -22.2%       1302 ±  5%  interrupts.CPU15.RES:Rescheduling_interrupts
    331882 ±  3%     -17.8%     272672 ±  3%  interrupts.CPU16.LOC:Local_timer_interrupts
    331937 ±  3%     -17.9%     272611 ±  3%  interrupts.CPU17.LOC:Local_timer_interrupts
    331946 ±  3%     -17.9%     272669 ±  3%  interrupts.CPU18.LOC:Local_timer_interrupts
    331862 ±  3%     -17.9%     272582 ±  3%  interrupts.CPU19.LOC:Local_timer_interrupts
      1647 ±  4%     -17.5%       1359 ±  7%  interrupts.CPU19.RES:Rescheduling_interrupts
    331947 ±  3%     -17.8%     272711 ±  3%  interrupts.CPU2.LOC:Local_timer_interrupts
    331912 ±  3%     -17.8%     272737 ±  3%  interrupts.CPU20.LOC:Local_timer_interrupts
    331944 ±  3%     -17.8%     272722 ±  3%  interrupts.CPU21.LOC:Local_timer_interrupts
    331947 ±  3%     -17.8%     272844 ±  3%  interrupts.CPU22.LOC:Local_timer_interrupts
    331943 ±  3%     -17.9%     272658 ±  3%  interrupts.CPU23.LOC:Local_timer_interrupts
      1830 ± 28%     -30.0%       1282 ±  6%  interrupts.CPU23.RES:Rescheduling_interrupts
    331951 ±  3%     -17.9%     272495 ±  3%  interrupts.CPU24.LOC:Local_timer_interrupts
      1624 ±  8%     -19.1%       1314 ±  8%  interrupts.CPU24.RES:Rescheduling_interrupts
    331806 ±  3%     -17.8%     272821 ±  3%  interrupts.CPU25.LOC:Local_timer_interrupts
      1538 ±  6%     -15.9%       1293 ±  5%  interrupts.CPU25.RES:Rescheduling_interrupts
    331962 ±  3%     -17.8%     272740 ±  3%  interrupts.CPU26.LOC:Local_timer_interrupts
      1554 ±  7%     -15.7%       1310 ±  7%  interrupts.CPU26.RES:Rescheduling_interrupts
    331955 ±  3%     -17.8%     272711 ±  3%  interrupts.CPU27.LOC:Local_timer_interrupts
    331942 ±  3%     -17.8%     272716 ±  3%  interrupts.CPU28.LOC:Local_timer_interrupts
    331917 ±  3%     -17.9%     272494 ±  3%  interrupts.CPU29.LOC:Local_timer_interrupts
      1496 ±  6%     -13.0%       1302 ±  5%  interrupts.CPU29.RES:Rescheduling_interrupts
    331980 ±  3%     -17.8%     272727 ±  3%  interrupts.CPU3.LOC:Local_timer_interrupts
      2693 ± 50%     -41.7%       1571 ± 14%  interrupts.CPU3.RES:Rescheduling_interrupts
    331955 ±  3%     -17.8%     272716 ±  3%  interrupts.CPU30.LOC:Local_timer_interrupts
      1488 ±  4%     -15.4%       1260 ±  4%  interrupts.CPU30.RES:Rescheduling_interrupts
    331953 ±  3%     -17.9%     272681 ±  3%  interrupts.CPU31.LOC:Local_timer_interrupts
      1488 ±  5%     -13.4%       1288 ±  5%  interrupts.CPU31.RES:Rescheduling_interrupts
    331936 ±  3%     -17.8%     272702 ±  3%  interrupts.CPU32.LOC:Local_timer_interrupts
      1539 ±  6%     -14.8%       1312 ±  3%  interrupts.CPU32.RES:Rescheduling_interrupts
    331820 ±  4%     -17.8%     272689 ±  3%  interrupts.CPU33.LOC:Local_timer_interrupts
      1519 ±  5%     -15.2%       1288 ±  5%  interrupts.CPU33.RES:Rescheduling_interrupts
    331916 ±  3%     -17.8%     272704 ±  3%  interrupts.CPU34.LOC:Local_timer_interrupts
      1525 ±  3%     -19.2%       1232 ±  8%  interrupts.CPU34.RES:Rescheduling_interrupts
    331951 ±  3%     -17.8%     272723 ±  3%  interrupts.CPU35.LOC:Local_timer_interrupts
      1603 ±  9%     -21.3%       1261 ± 10%  interrupts.CPU35.RES:Rescheduling_interrupts
    331900 ±  3%     -17.8%     272710 ±  3%  interrupts.CPU36.LOC:Local_timer_interrupts
      1548 ±  6%     -16.4%       1295 ±  6%  interrupts.CPU36.RES:Rescheduling_interrupts
    332050 ±  3%     -17.9%     272721 ±  3%  interrupts.CPU37.LOC:Local_timer_interrupts
      1527 ±  6%     -17.8%       1255 ±  6%  interrupts.CPU37.RES:Rescheduling_interrupts
    331861 ±  3%     -17.8%     272736 ±  3%  interrupts.CPU38.LOC:Local_timer_interrupts
      1726 ± 19%     -24.5%       1302 ±  4%  interrupts.CPU38.RES:Rescheduling_interrupts
    331954 ±  3%     -17.8%     272735 ±  3%  interrupts.CPU39.LOC:Local_timer_interrupts
    331942 ±  3%     -17.8%     272726 ±  3%  interrupts.CPU4.LOC:Local_timer_interrupts
    331989 ±  3%     -17.9%     272707 ±  3%  interrupts.CPU40.LOC:Local_timer_interrupts
    331956 ±  3%     -17.8%     272713 ±  3%  interrupts.CPU41.LOC:Local_timer_interrupts
      1758 ± 21%     -26.9%       1285 ±  3%  interrupts.CPU41.RES:Rescheduling_interrupts
    331940 ±  3%     -17.8%     272712 ±  3%  interrupts.CPU42.LOC:Local_timer_interrupts
    331974 ±  3%     -17.9%     272714 ±  3%  interrupts.CPU43.LOC:Local_timer_interrupts
      1553 ±  3%     -15.8%       1308 ±  4%  interrupts.CPU43.RES:Rescheduling_interrupts
    331966 ±  3%     -17.8%     272734 ±  3%  interrupts.CPU44.LOC:Local_timer_interrupts
      1576 ±  8%     -15.7%       1328 ±  6%  interrupts.CPU44.RES:Rescheduling_interrupts
    331963 ±  3%     -17.8%     272727 ±  3%  interrupts.CPU45.LOC:Local_timer_interrupts
      1572 ±  5%     -17.5%       1297 ±  6%  interrupts.CPU45.RES:Rescheduling_interrupts
    331869 ±  3%     -17.8%     272723 ±  3%  interrupts.CPU46.LOC:Local_timer_interrupts
      5123 ± 27%     +58.0%       8095 ±  6%  interrupts.CPU46.NMI:Non-maskable_interrupts
      5123 ± 27%     +58.0%       8095 ±  6%  interrupts.CPU46.PMI:Performance_monitoring_interrupts
      1598 ± 10%     -20.2%       1276 ±  6%  interrupts.CPU46.RES:Rescheduling_interrupts
    331966 ±  3%     -17.8%     272747 ±  3%  interrupts.CPU47.LOC:Local_timer_interrupts
      1689 ± 19%     -20.7%       1340 ±  8%  interrupts.CPU47.RES:Rescheduling_interrupts
    331971 ±  3%     -17.8%     272716 ±  3%  interrupts.CPU48.LOC:Local_timer_interrupts
    331965 ±  3%     -17.8%     272729 ±  3%  interrupts.CPU49.LOC:Local_timer_interrupts
    331962 ±  3%     -17.8%     272755 ±  3%  interrupts.CPU5.LOC:Local_timer_interrupts
      1685 ± 11%     -22.0%       1314 ±  3%  interrupts.CPU5.RES:Rescheduling_interrupts
    331979 ±  3%     -17.8%     272735 ±  3%  interrupts.CPU50.LOC:Local_timer_interrupts
    331958 ±  3%     -17.8%     272723 ±  3%  interrupts.CPU51.LOC:Local_timer_interrupts
    331596 ±  4%     -17.8%     272663 ±  3%  interrupts.CPU52.LOC:Local_timer_interrupts
    331943 ±  3%     -17.8%     272719 ±  3%  interrupts.CPU53.LOC:Local_timer_interrupts
      6200 ± 31%     +30.8%       8111 ±  6%  interrupts.CPU53.NMI:Non-maskable_interrupts
      6200 ± 31%     +30.8%       8111 ±  6%  interrupts.CPU53.PMI:Performance_monitoring_interrupts
    331958 ±  3%     -17.8%     272716 ±  3%  interrupts.CPU54.LOC:Local_timer_interrupts
      1428 ±  3%     -14.2%       1226 ±  3%  interrupts.CPU54.RES:Rescheduling_interrupts
    331815 ±  3%     -17.8%     272659 ±  3%  interrupts.CPU55.LOC:Local_timer_interrupts
      1483 ±  4%     -15.3%       1257 ±  2%  interrupts.CPU55.RES:Rescheduling_interrupts
    331936 ±  3%     -17.8%     272731 ±  3%  interrupts.CPU56.LOC:Local_timer_interrupts
      1474 ±  4%     -14.7%       1258 ±  7%  interrupts.CPU56.RES:Rescheduling_interrupts
    331896 ±  3%     -17.8%     272737 ±  3%  interrupts.CPU57.LOC:Local_timer_interrupts
      1493 ±  5%     -16.2%       1250 ±  7%  interrupts.CPU57.RES:Rescheduling_interrupts
    331939 ±  3%     -17.8%     272719 ±  3%  interrupts.CPU58.LOC:Local_timer_interrupts
      1634 ± 19%     -24.5%       1234 ±  4%  interrupts.CPU58.RES:Rescheduling_interrupts
    331940 ±  3%     -17.8%     272723 ±  3%  interrupts.CPU59.LOC:Local_timer_interrupts
    331923 ±  3%     -17.8%     272731 ±  3%  interrupts.CPU6.LOC:Local_timer_interrupts
    331947 ±  3%     -17.8%     272699 ±  3%  interrupts.CPU60.LOC:Local_timer_interrupts
      1504 ±  9%     -15.3%       1274 ±  6%  interrupts.CPU60.RES:Rescheduling_interrupts
    331957 ±  3%     -17.9%     272582 ±  3%  interrupts.CPU61.LOC:Local_timer_interrupts
      1467 ±  3%     -13.3%       1271 ±  4%  interrupts.CPU61.RES:Rescheduling_interrupts
    331948 ±  3%     -17.8%     272705 ±  3%  interrupts.CPU62.LOC:Local_timer_interrupts
      1466 ±  6%     -15.2%       1243 ±  8%  interrupts.CPU62.RES:Rescheduling_interrupts
    331894 ±  3%     -17.8%     272728 ±  3%  interrupts.CPU63.LOC:Local_timer_interrupts
      1496 ±  4%     -15.8%       1260 ±  4%  interrupts.CPU63.RES:Rescheduling_interrupts
    331931 ±  3%     -17.8%     272872 ±  3%  interrupts.CPU64.LOC:Local_timer_interrupts
      1492 ±  5%     -14.7%       1272 ±  5%  interrupts.CPU64.RES:Rescheduling_interrupts
    331949 ±  3%     -17.8%     272721 ±  3%  interrupts.CPU65.LOC:Local_timer_interrupts
    331920 ±  3%     -17.8%     272703 ±  3%  interrupts.CPU66.LOC:Local_timer_interrupts
      1491 ±  3%     -16.6%       1244 ±  5%  interrupts.CPU66.RES:Rescheduling_interrupts
    331937 ±  3%     -17.8%     272723 ±  3%  interrupts.CPU67.LOC:Local_timer_interrupts
      1515 ±  3%     -17.7%       1247 ±  4%  interrupts.CPU67.RES:Rescheduling_interrupts
    331927 ±  3%     -17.8%     272715 ±  3%  interrupts.CPU68.LOC:Local_timer_interrupts
      1439 ±  3%     -11.6%       1272 ±  5%  interrupts.CPU68.RES:Rescheduling_interrupts
    331938 ±  3%     -17.8%     272709 ±  3%  interrupts.CPU69.LOC:Local_timer_interrupts
      1445 ±  5%     -13.1%       1255 ±  4%  interrupts.CPU69.RES:Rescheduling_interrupts
    144.25 ± 11%     -18.5%     117.50 ±  3%  interrupts.CPU7.35:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc
    331952 ±  3%     -17.8%     272795 ±  3%  interrupts.CPU7.LOC:Local_timer_interrupts
      1646 ± 11%     -20.7%       1305 ±  7%  interrupts.CPU7.RES:Rescheduling_interrupts
    331942 ±  3%     -17.8%     272737 ±  3%  interrupts.CPU70.LOC:Local_timer_interrupts
      1560 ±  7%     -21.4%       1226 ±  2%  interrupts.CPU70.RES:Rescheduling_interrupts
    332216 ±  3%     -17.9%     272830 ±  3%  interrupts.CPU71.LOC:Local_timer_interrupts
      1807 ± 21%     -27.3%       1313 ±  7%  interrupts.CPU71.RES:Rescheduling_interrupts
    331974 ±  3%     -17.8%     272792 ±  3%  interrupts.CPU8.LOC:Local_timer_interrupts
      1894 ± 17%     -28.5%       1355 ±  9%  interrupts.CPU8.RES:Rescheduling_interrupts
    332143 ±  4%     -17.9%     272738 ±  3%  interrupts.CPU9.LOC:Local_timer_interrupts
      1575 ±  8%     -18.8%       1279 ±  5%  interrupts.CPU9.RES:Rescheduling_interrupts
  23899514 ±  3%     -17.8%   19635391 ±  3%  interrupts.LOC:Local_timer_interrupts
    118109 ±  2%     -15.1%     100275 ±  4%  interrupts.RES:Rescheduling_interrupts
     30.36 ±  6%     -10.2       20.14 ± 23%  perf-profile.calltrace.cycles-pp.__mark_inode_dirty.generic_write_end.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter
     31.81 ±  5%      -9.9       21.93 ± 20%  perf-profile.calltrace.cycles-pp.generic_write_end.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter.new_sync_write
     38.42 ±  5%      -9.0       29.44 ± 11%  perf-profile.calltrace.cycles-pp.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter.new_sync_write.vfs_write
     25.90 ±  8%      -7.6       18.33 ± 25%  perf-profile.calltrace.cycles-pp.ext4_dirty_inode.__mark_inode_dirty.generic_write_end.ext4_da_write_end.generic_perform_write
      6.83 ± 10%      -4.4        2.46 ±  7%  perf-profile.calltrace.cycles-pp.__ext4_handle_dirty_metadata.ext4_do_update_inode.ext4_mark_iloc_dirty.__ext4_mark_inode_dirty.ext4_dirty_inode
     12.12 ±  6%      -3.6        8.47 ± 26%  perf-profile.calltrace.cycles-pp.ext4_mark_iloc_dirty.__ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty.generic_write_end
     11.33 ±  8%      -3.6        7.70 ± 27%  perf-profile.calltrace.cycles-pp.ext4_do_update_inode.ext4_mark_iloc_dirty.__ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty
      5.06 ±  9%      -3.1        1.98 ±  8%  perf-profile.calltrace.cycles-pp.jbd2_journal_dirty_metadata.__ext4_handle_dirty_metadata.ext4_do_update_inode.ext4_mark_iloc_dirty.__ext4_mark_inode_dirty
      5.17 ±  8%      -3.0        2.21 ±  7%  perf-profile.calltrace.cycles-pp.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.__ext4_mark_inode_dirty.ext4_dirty_inode
      5.21 ±  8%      -2.9        2.26 ±  7%  perf-profile.calltrace.cycles-pp.__ext4_journal_get_write_access.ext4_reserve_inode_write.__ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty
      5.23 ±  2%      -2.7        2.49 ± 11%  perf-profile.calltrace.cycles-pp.ext4_journal_check_start.__ext4_journal_start_sb.ext4_dirty_inode.__mark_inode_dirty.generic_write_end
      5.21 ±  2%      -2.7        2.52 ± 11%  perf-profile.calltrace.cycles-pp.ext4_journal_check_start.__ext4_journal_start_sb.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter
      5.50 ±  2%      -2.7        2.85 ± 17%  perf-profile.calltrace.cycles-pp.__ext4_journal_start_sb.ext4_dirty_inode.__mark_inode_dirty.generic_write_end.ext4_da_write_end
      3.88 ±  6%      -2.4        1.44 ±  9%  perf-profile.calltrace.cycles-pp.jbd2_journal_stop.__ext4_journal_stop.__mark_inode_dirty.generic_write_end.ext4_da_write_end
      4.03 ±  5%      -2.4        1.66 ±  7%  perf-profile.calltrace.cycles-pp.__ext4_journal_stop.__mark_inode_dirty.generic_write_end.ext4_da_write_end.generic_perform_write
      3.14 ± 19%      -0.7        2.49        perf-profile.calltrace.cycles-pp.ext4_write_checks.ext4_buffered_write_iter.new_sync_write.vfs_write.ksys_write
      2.92 ± 11%      -0.5        2.40 ±  2%  perf-profile.calltrace.cycles-pp.file_update_time.ext4_write_checks.ext4_buffered_write_iter.new_sync_write.vfs_write
      0.73 ±  6%      -0.5        0.26 ±100%  perf-profile.calltrace.cycles-pp.__ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty.generic_update_time.file_update_time
      2.69 ±  3%      -0.4        2.33 ±  2%  perf-profile.calltrace.cycles-pp.generic_update_time.file_update_time.ext4_write_checks.ext4_buffered_write_iter.new_sync_write
      2.69 ±  3%      -0.4        2.33 ±  2%  perf-profile.calltrace.cycles-pp.__mark_inode_dirty.generic_update_time.file_update_time.ext4_write_checks.ext4_buffered_write_iter
      2.31 ±  4%      -0.3        2.03 ±  3%  perf-profile.calltrace.cycles-pp.ext4_dirty_inode.__mark_inode_dirty.generic_update_time.file_update_time.ext4_write_checks
      0.58 ±  4%      -0.2        0.41 ± 57%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.unlink
      0.58 ±  4%      -0.2        0.41 ± 57%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink
      1.37 ±  3%      +0.1        1.51 ±  3%  perf-profile.calltrace.cycles-pp.start_this_handle.jbd2__journal_start.__ext4_journal_start_sb.ext4_dirty_inode.__mark_inode_dirty
      0.91 ±  2%      +0.1        1.05 ±  2%  perf-profile.calltrace.cycles-pp.__dentry_kill.dput.__fput.task_work_run.exit_to_usermode_loop
      1.37 ±  4%      +0.1        1.51 ±  3%  perf-profile.calltrace.cycles-pp.jbd2__journal_start.__ext4_journal_start_sb.ext4_dirty_inode.__mark_inode_dirty.generic_update_time
      0.92 ±  2%      +0.1        1.06 ±  2%  perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe.close
      0.92 ±  2%      +0.1        1.06 ±  2%  perf-profile.calltrace.cycles-pp.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe.close
      0.91 ±  2%      +0.1        1.05        perf-profile.calltrace.cycles-pp.evict.__dentry_kill.dput.__fput.task_work_run
      0.93            +0.1        1.07 ±  2%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.close
      0.93            +0.1        1.07 ±  2%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.close
      0.90 ±  2%      +0.1        1.04        perf-profile.calltrace.cycles-pp.ext4_evict_inode.evict.__dentry_kill.dput.__fput
      0.91 ±  2%      +0.1        1.05        perf-profile.calltrace.cycles-pp.dput.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64
      0.92 ±  2%      +0.1        1.06 ±  2%  perf-profile.calltrace.cycles-pp.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.73 ±  2%      +0.1        0.88 ±  4%  perf-profile.calltrace.cycles-pp.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill.dput
      0.93            +0.1        1.08 ±  2%  perf-profile.calltrace.cycles-pp.close
      0.58 ± 12%      +0.2        0.81 ± 15%  perf-profile.calltrace.cycles-pp.mark_buffer_dirty.__block_commit_write.block_write_end.generic_write_end.ext4_da_write_end
      1.21 ± 15%      +0.4        1.57 ± 14%  perf-profile.calltrace.cycles-pp.block_write_end.generic_write_end.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter
      1.26 ± 14%      +0.4        1.63 ± 13%  perf-profile.calltrace.cycles-pp.__block_commit_write.block_write_end.generic_write_end.ext4_da_write_end.generic_perform_write
      0.13 ±173%      +0.5        0.59 ± 12%  perf-profile.calltrace.cycles-pp.pagecache_get_page.generic_file_buffered_read.new_sync_read.vfs_read.ksys_read
      2.30 ± 16%      +1.6        3.86 ± 16%  perf-profile.calltrace.cycles-pp.stop_this_handle.jbd2_journal_stop.__ext4_journal_stop.ext4_da_write_end.generic_perform_write
      7.13 ±  3%      +3.7       10.79 ±  9%  perf-profile.calltrace.cycles-pp.add_transaction_credits.start_this_handle.jbd2__journal_start.__ext4_journal_start_sb.ext4_da_write_begin
     39.38 ±  3%      +8.8       48.20 ±  8%  perf-profile.calltrace.cycles-pp.__ext4_journal_start_sb.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter.new_sync_write
     44.30            +9.6       53.95 ±  6%  perf-profile.calltrace.cycles-pp.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter.new_sync_write.vfs_write
     33.69 ±  4%     +11.4       45.08 ± 10%  perf-profile.calltrace.cycles-pp.start_this_handle.jbd2__journal_start.__ext4_journal_start_sb.ext4_da_write_begin.generic_perform_write
     33.90 ±  4%     +11.4       45.34 ± 10%  perf-profile.calltrace.cycles-pp.jbd2__journal_start.__ext4_journal_start_sb.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter
     33.08 ±  5%     -10.6       22.49 ± 21%  perf-profile.children.cycles-pp.__mark_inode_dirty
     31.82 ±  5%      -9.9       21.93 ± 20%  perf-profile.children.cycles-pp.generic_write_end
     38.42 ±  5%      -9.0       29.45 ± 11%  perf-profile.children.cycles-pp.ext4_da_write_end
     28.21 ±  7%      -7.9       20.36 ± 23%  perf-profile.children.cycles-pp.ext4_dirty_inode
     10.66 ±  2%      -5.6        5.09 ± 11%  perf-profile.children.cycles-pp.ext4_journal_check_start
     21.03 ± 10%      -5.3       15.72 ± 27%  perf-profile.children.cycles-pp.__ext4_mark_inode_dirty
      7.19 ± 10%      -4.6        2.58 ±  7%  perf-profile.children.cycles-pp.__ext4_handle_dirty_metadata
     12.70 ±  6%      -3.9        8.82 ± 26%  perf-profile.children.cycles-pp.ext4_mark_iloc_dirty
     11.88 ±  8%      -3.9        8.01 ± 27%  perf-profile.children.cycles-pp.ext4_do_update_inode
      5.34 ±  9%      -3.3        2.08 ±  8%  perf-profile.children.cycles-pp.jbd2_journal_dirty_metadata
      5.46 ±  8%      -3.1        2.33 ±  7%  perf-profile.children.cycles-pp.jbd2_journal_get_write_access
      5.50 ±  8%      -3.1        2.39 ±  7%  perf-profile.children.cycles-pp.__ext4_journal_get_write_access
     10.62 ±  7%      -1.6        8.97 ± 12%  perf-profile.children.cycles-pp.jbd2_journal_stop
      3.15 ± 19%      -0.7        2.49        perf-profile.children.cycles-pp.ext4_write_checks
     97.90            -0.6       97.29        perf-profile.children.cycles-pp.do_syscall_64
     97.95            -0.6       97.35        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      2.92 ± 11%      -0.5        2.40 ±  2%  perf-profile.children.cycles-pp.file_update_time
      2.69 ±  3%      -0.4        2.33 ±  2%  perf-profile.children.cycles-pp.generic_update_time
      0.68 ±  7%      -0.2        0.52 ± 20%  perf-profile.children.cycles-pp.rwsem_down_write_slowpath
      0.64 ±  7%      -0.2        0.48 ± 20%  perf-profile.children.cycles-pp.rwsem_optimistic_spin
      0.45 ±  8%      -0.1        0.32 ± 22%  perf-profile.children.cycles-pp.osq_lock
      0.51 ±  8%      -0.1        0.40 ± 11%  perf-profile.children.cycles-pp.do_filp_open
      0.51 ±  8%      -0.1        0.40 ± 11%  perf-profile.children.cycles-pp.path_openat
      0.51 ±  7%      -0.1        0.40 ± 12%  perf-profile.children.cycles-pp.creat
      0.52 ±  7%      -0.1        0.41 ± 11%  perf-profile.children.cycles-pp.do_sys_open
      0.52 ±  7%      -0.1        0.41 ± 11%  perf-profile.children.cycles-pp.do_sys_openat2
      0.14 ± 12%      -0.0        0.10 ± 19%  perf-profile.children.cycles-pp._raw_spin_lock_irq
      0.06 ± 11%      +0.0        0.08 ±  6%  perf-profile.children.cycles-pp.__mod_lruvec_state
      0.08 ±  5%      +0.0        0.10 ± 12%  perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
      0.06 ± 11%      +0.0        0.08 ±  8%  perf-profile.children.cycles-pp.update_cfs_group
      0.11 ±  4%      +0.0        0.13 ±  9%  perf-profile.children.cycles-pp.get_page_from_freelist
      0.14 ±  3%      +0.0        0.17 ±  8%  perf-profile.children.cycles-pp.__alloc_pages_nodemask
      0.08 ±  6%      +0.0        0.10 ±  7%  perf-profile.children.cycles-pp.down_write
      0.07 ± 13%      +0.0        0.09 ± 13%  perf-profile.children.cycles-pp.fsnotify_parent
      0.04 ± 57%      +0.0        0.06 ±  6%  perf-profile.children.cycles-pp.alloc_buffer_head
      0.04 ± 57%      +0.0        0.06 ±  6%  perf-profile.children.cycles-pp.apparmor_file_permission
      0.04 ± 58%      +0.0        0.07 ±  7%  perf-profile.children.cycles-pp.rcu_all_qs
      0.14 ±  9%      +0.0        0.17 ±  5%  perf-profile.children.cycles-pp.task_tick_fair
      0.15 ±  5%      +0.0        0.18 ±  2%  perf-profile.children.cycles-pp.common_file_perm
      0.12 ±  5%      +0.0        0.15 ±  4%  perf-profile.children.cycles-pp._cond_resched
      0.27 ±  7%      +0.0        0.31 ±  4%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.13 ± 13%      +0.0        0.17 ± 12%  perf-profile.children.cycles-pp.__might_sleep
      0.16 ±  7%      +0.0        0.20 ±  6%  perf-profile.children.cycles-pp.scheduler_tick
      0.22 ±  7%      +0.0        0.26 ±  4%  perf-profile.children.cycles-pp.tick_sched_timer
      0.20 ±  7%      +0.0        0.23 ±  4%  perf-profile.children.cycles-pp.tick_sched_handle
      0.20 ±  7%      +0.0        0.23 ±  4%  perf-profile.children.cycles-pp.update_process_times
      0.14 ±  5%      +0.0        0.18 ±  9%  perf-profile.children.cycles-pp.kmem_cache_alloc
      0.14 ±  8%      +0.0        0.18 ±  4%  perf-profile.children.cycles-pp.xas_load
      0.25 ±  4%      +0.1        0.30 ±  6%  perf-profile.children.cycles-pp.memset_erms
      0.18 ±  8%      +0.1        0.24 ±  6%  perf-profile.children.cycles-pp.copyout
      0.03 ±100%      +0.1        0.09 ±  9%  perf-profile.children.cycles-pp.fpregs_assert_state_consistent
      0.25 ±  4%      +0.1        0.31 ±  4%  perf-profile.children.cycles-pp.security_file_permission
      0.35 ±  5%      +0.1        0.42 ± 13%  perf-profile.children.cycles-pp.crc32c_pcl_intel_update
      0.33 ±  3%      +0.1        0.40 ±  6%  perf-profile.children.cycles-pp.copyin
      0.40 ±  8%      +0.1        0.47 ±  3%  perf-profile.children.cycles-pp.truncate_cleanup_page
      0.29 ±  5%      +0.1        0.38 ±  5%  perf-profile.children.cycles-pp.copy_page_to_iter
      0.41 ±  2%      +0.1        0.51 ±  5%  perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic
      0.49 ±  4%      +0.1        0.61 ±  6%  perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      0.91 ±  2%      +0.1        1.05 ±  2%  perf-profile.children.cycles-pp.__dentry_kill
      0.92 ±  2%      +0.1        1.06 ±  2%  perf-profile.children.cycles-pp.task_work_run
      0.91 ±  2%      +0.1        1.05        perf-profile.children.cycles-pp.evict
      0.90 ±  2%      +0.1        1.04        perf-profile.children.cycles-pp.ext4_evict_inode
      0.92 ±  2%      +0.1        1.06 ±  2%  perf-profile.children.cycles-pp.__fput
      0.92 ±  2%      +0.1        1.06        perf-profile.children.cycles-pp.exit_to_usermode_loop
      0.92 ±  2%      +0.1        1.06 ±  2%  perf-profile.children.cycles-pp.dput
      0.73 ±  2%      +0.1        0.88 ±  4%  perf-profile.children.cycles-pp.truncate_inode_pages_range
      0.93            +0.1        1.08 ±  2%  perf-profile.children.cycles-pp.close
      0.64 ±  4%      +0.2        0.80 ±  6%  perf-profile.children.cycles-pp.entry_SYSCALL_64
      0.28 ±  9%      +0.2        0.46 ± 15%  perf-profile.children.cycles-pp.create_empty_buffers
      0.58 ± 12%      +0.2        0.81 ± 15%  perf-profile.children.cycles-pp.mark_buffer_dirty
      0.69 ±  6%      +0.3        0.98 ±  8%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      1.27 ± 14%      +0.4        1.64 ± 13%  perf-profile.children.cycles-pp.block_write_end
      1.26 ± 14%      +0.4        1.63 ± 13%  perf-profile.children.cycles-pp.__block_commit_write
      2.43 ± 16%      +1.6        4.00 ± 16%  perf-profile.children.cycles-pp.stop_this_handle
      7.45 ±  3%      +3.7       11.18 ±  9%  perf-profile.children.cycles-pp.add_transaction_credits
     46.57 ±  2%      +6.2       52.78 ±  7%  perf-profile.children.cycles-pp.__ext4_journal_start_sb
     44.31            +9.7       53.96 ±  6%  perf-profile.children.cycles-pp.ext4_da_write_begin
     35.17 ±  4%     +11.6       46.74 ± 10%  perf-profile.children.cycles-pp.start_this_handle
     35.39 ±  4%     +11.6       47.00 ± 10%  perf-profile.children.cycles-pp.jbd2__journal_start
     10.56 ±  2%      -5.5        5.01 ± 11%  perf-profile.self.cycles-pp.ext4_journal_check_start
      5.31 ±  9%      -3.2        2.07 ±  8%  perf-profile.self.cycles-pp.jbd2_journal_dirty_metadata
      8.12 ±  5%      -3.2        4.89 ± 10%  perf-profile.self.cycles-pp.jbd2_journal_stop
      5.11 ±  9%      -3.1        1.98 ±  9%  perf-profile.self.cycles-pp.jbd2_journal_get_write_access
      1.82 ± 13%      -1.4        0.45 ± 12%  perf-profile.self.cycles-pp.__ext4_handle_dirty_metadata
      0.48 ± 48%      -0.3        0.18 ±  4%  perf-profile.self.cycles-pp.__mark_inode_dirty
      0.45 ±  8%      -0.1        0.32 ± 21%  perf-profile.self.cycles-pp.osq_lock
      0.30 ± 12%      -0.0        0.26 ± 10%  perf-profile.self.cycles-pp.pagecache_get_page
      0.13 ± 14%      -0.0        0.10 ± 19%  perf-profile.self.cycles-pp._raw_spin_lock_irq
      0.08 ±  5%      +0.0        0.10 ±  8%  perf-profile.self.cycles-pp.__fget_light
      0.07 ±  5%      +0.0        0.09 ± 13%  perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
      0.08            +0.0        0.10 ±  8%  perf-profile.self.cycles-pp.iov_iter_copy_from_user_atomic
      0.07 ± 13%      +0.0        0.08 ±  5%  perf-profile.self.cycles-pp.generic_perform_write
      0.06 ±  7%      +0.0        0.08 ± 11%  perf-profile.self.cycles-pp.read
      0.13 ±  7%      +0.0        0.15        perf-profile.self.cycles-pp.common_file_perm
      0.06 ± 11%      +0.0        0.08 ±  8%  perf-profile.self.cycles-pp.update_cfs_group
      0.09 ±  4%      +0.0        0.11 ±  4%  perf-profile.self.cycles-pp.ext4_da_write_end
      0.04 ± 57%      +0.0        0.06 ±  6%  perf-profile.self.cycles-pp._cond_resched
      0.10 ±  8%      +0.0        0.12 ±  4%  perf-profile.self.cycles-pp.xas_load
      0.04 ± 58%      +0.0        0.07 ±  6%  perf-profile.self.cycles-pp.down_write
      0.12 ± 11%      +0.0        0.15 ± 12%  perf-profile.self.cycles-pp.__might_sleep
      0.05 ± 58%      +0.0        0.09 ± 14%  perf-profile.self.cycles-pp.fsnotify_parent
      0.05 ± 58%      +0.0        0.09 ±  7%  perf-profile.self.cycles-pp.vfs_write
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.security_file_permission
      0.24 ±  6%      +0.1        0.29 ±  5%  perf-profile.self.cycles-pp.memset_erms
      0.03 ±100%      +0.1        0.08 ± 13%  perf-profile.self.cycles-pp.fpregs_assert_state_consistent
      0.33 ±  5%      +0.1        0.40 ± 14%  perf-profile.self.cycles-pp.crc32c_pcl_intel_update
      0.11 ± 21%      +0.1        0.18 ± 14%  perf-profile.self.cycles-pp.create_empty_buffers
      0.30 ± 14%      +0.1        0.41 ±  3%  perf-profile.self.cycles-pp.__ext4_journal_stop
      0.48 ±  4%      +0.1        0.60 ±  6%  perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      0.56 ±  4%      +0.1        0.71 ±  6%  perf-profile.self.cycles-pp.entry_SYSCALL_64
      0.42 ± 14%      +0.2        0.63 ± 17%  perf-profile.self.cycles-pp.mark_buffer_dirty
      0.69 ±  6%      +0.3        0.97 ±  7%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      2.42 ± 15%      +1.6        3.98 ± 16%  perf-profile.self.cycles-pp.stop_this_handle
      7.41 ±  3%      +3.7       11.12 ±  9%  perf-profile.self.cycles-pp.add_transaction_credits
     19.54 ±  3%      +8.5       28.08 ±  9%  perf-profile.self.cycles-pp.start_this_handle


                                                                                
                                  aim7.jobs-per-min                             
                                                                                
  160000 +------------------------------------------------------------------+   
         |  O    O  O OO              OO         OO      OO O O             |   
  140000 |-+   O   O       O        O    O  O  O       O          O      O  |   
  120000 |-O  O          O  O OO   O       O  O     O O        O O  O OO  O |   
         |.++.++.+.++.++.+.++.++.+.++.++.+.++.++.++.+.++.++.+.+ .++         |   
  100000 |-+                                                   +            |   
         |                                                                  |   
   80000 |-+                                                                |   
         |                                                                  |   
   60000 |-+                                                                |   
   40000 |-+                                                                |   
         |                                                                  |   
   20000 |-+                                                                |   
         |                                                                  |   
       0 +------------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample



Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Rong Chen


View attachment "config-5.7.0-rc4-00080-g7b97d868b7ab2" of type "text/plain" (202609 bytes)

View attachment "job-script" of type "text/plain" (8023 bytes)

View attachment "job.yaml" of type "text/plain" (5501 bytes)

View attachment "reproduce" of type "text/plain" (1021 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ