lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20201127022018.GA29584@xsang-OptiPlex-9020>
Date:   Fri, 27 Nov 2020 10:20:18 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     Juergen Gross <jgross@...e.com>
Cc:     0day robot <lkp@...el.com>, Andy Lutomirski <luto@...nel.org>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org,
        ying.huang@...el.com, feng.tang@...el.com, zhengjun.xing@...el.com,
        xen-devel@...ts.xenproject.org, x86@...nel.org,
        virtualization@...ts.linux-foundation.org, peterz@...radead.org,
        Juergen Gross <jgross@...e.com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>,
        "H. Peter Anvin" <hpa@...or.com>, Deep Shah <sdeep@...are.com>,
        "VMware, Inc." <pv-drivers@...are.com>,
        Boris Ostrovsky <boris.ostrovsky@...cle.com>,
        Stefano Stabellini <sstabellini@...nel.org>
Subject: [x86]  97e8f0134a:  fio.write_iops 8.6% improvement


Greeting,

FYI, we noticed a 8.6% improvement of fio.write_iops due to commit:


commit: 97e8f0134a2bb794e4885f642724a50979b84f89 ("x86: rework arch_local_irq_restore() to not use popf")
url: https://github.com/0day-ci/linux/commits/Juergen-Gross/x86-major-paravirt-cleanup/20201120-194934


in testcase: fio-basic
on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory
with following parameters:

	disk: 2pmem
	fs: xfs
	mount_option: dax
	runtime: 200s
	nr_task: 50%
	time_based: tb
	rw: randwrite
	bs: 4k
	ioengine: sync
	test_size: 200G
	cpufreq_governor: performance
	ucode: 0x5003003

test-description: Fio is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user.
test-url: https://github.com/axboe/fio

In addition to that, the commit also has significant impact on the following tests:

+------------------+---------------------------------------------------------------------------+
| testcase: change | will-it-scale: will-it-scale.per_process_ops 5.2% improvement             |
| test machine     | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| test parameters  | cpufreq_governor=performance                                              |
|                  | mode=process                                                              |
|                  | nr_task=50%                                                               |
|                  | test=futex1                                                               |
|                  | ucode=0x5003003                                                           |
+------------------+---------------------------------------------------------------------------+




Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/mount_option/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
  4k/gcc-9/performance/2pmem/xfs/sync/x86_64-rhel-8.3/dax/50%/debian-10.4-x86_64-20200603.cgz/200s/randwrite/lkp-csl-2sp6/200G/fio-basic/tb/0x5003003

commit: 
  d625d30a28 ("x86/xen: drop USERGS_SYSRET64 paravirt call")
  97e8f0134a ("x86: rework arch_local_irq_restore() to not use popf")

d625d30a28a4c7a3 97e8f0134a2bb794e4885f64272 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
      0.12 ±  5%      -0.0        0.09 ± 13%  fio.latency_1000us%
      0.32            -0.0        0.31        fio.latency_100ms%
     18.23            -1.8       16.42 ±  2%  fio.latency_100us%
     19.93            +2.4       22.31 ±  4%  fio.latency_20us%
     20.34            -3.5       16.85        fio.latency_250us%
      0.04 ±  4%      -0.0        0.03 ±  5%  fio.latency_2ms%
      2.78 ±  2%      -0.3        2.45 ±  2%  fio.latency_500us%
      0.26            -0.0        0.21        fio.latency_50ms%
     37.13            +3.5       40.61 ±  2%  fio.latency_50us%
      0.61 ±  4%      -0.1        0.52 ±  7%  fio.latency_750us%
    600.25            -4.7%     572.00        fio.time.percent_of_cpu_this_job_got
      1169            -4.8%       1112        fio.time.system_time
   8540510            +2.4%    8746174        fio.time.voluntary_context_switches
  26105864            +8.6%   28354900        fio.workload
    509.80            +8.6%     553.74        fio.write_bw_MBps
    153600           -12.7%     134144        fio.write_clat_90%_us
    220672            -9.5%     199680        fio.write_clat_95%_us
    612352            -9.4%     555008 ±  2%  fio.write_clat_99%_us
    366788            -7.9%     337679        fio.write_clat_mean_us
   3896686            -2.5%    3799973        fio.write_clat_stddev
    130510            +8.6%     141756        fio.write_iops
     15.54            -1.3%      15.35        boot-time.dhcp
      3739 ± 76%     +84.2%       6886 ± 30%  numa-meminfo.node1.PageTables
    934.75 ± 76%     +84.2%       1721 ± 31%  numa-vmstat.node1.nr_page_table_pages
   1128546            +7.6%    1214673        vmstat.io.bo
    254919            +5.4%     268773        vmstat.system.cs
    158457            +1.5%     160806        proc-vmstat.nr_slab_unreclaimable
   1694299            +5.6%    1789326        proc-vmstat.numa_hit
   1662873            +5.7%    1757835        proc-vmstat.numa_local
   6648063            +9.2%    7260774        proc-vmstat.pgalloc_normal
   6403869            +9.8%    7028415        proc-vmstat.pgfree
 2.313e+08            +7.5%  2.487e+08        proc-vmstat.pgpgout
     36071 ±  5%     -10.1%      32430 ±  3%  softirqs.CPU36.RCU
     36623 ±  5%      -9.0%      33322 ±  4%  softirqs.CPU41.RCU
     36414 ±  5%      -8.8%      33202 ±  3%  softirqs.CPU47.RCU
     34248            -9.6%      30945        softirqs.CPU73.RCU
     33882            -9.4%      30687        softirqs.CPU74.RCU
     36889 ±  4%     -12.9%      32132 ±  2%  softirqs.CPU82.RCU
     35183 ±  4%      -8.5%      32189 ±  4%  softirqs.CPU85.RCU
     36146 ±  2%      -9.3%      32785 ±  3%  softirqs.CPU87.RCU
     36673 ±  3%      -9.9%      33030 ±  3%  softirqs.CPU95.RCU
      5942           -30.4%       4138 ± 21%  sched_debug.cfs_rq:/.exec_clock.avg
      4314 ±  2%     -33.6%       2863 ± 25%  sched_debug.cfs_rq:/.exec_clock.min
    773.71 ±  8%     -16.7%     644.43 ±  6%  sched_debug.cfs_rq:/.exec_clock.stddev
    241.46 ± 11%     -27.0%     176.38 ± 26%  sched_debug.cfs_rq:/.load_avg.avg
     27925 ±  5%     -21.0%      22049 ± 17%  sched_debug.cfs_rq:/.min_vruntime.min
    116399           -20.2%      92862 ± 13%  sched_debug.cpu.clock.avg
    116404           -20.2%      92867 ± 13%  sched_debug.cpu.clock.max
    116393           -20.2%      92856 ± 13%  sched_debug.cpu.clock.min
    114940           -20.2%      91709 ± 13%  sched_debug.cpu.clock_task.avg
    115124           -20.2%      91877 ± 13%  sched_debug.cpu.clock_task.max
    110309           -20.8%      87321 ± 14%  sched_debug.cpu.clock_task.min
      5029           -11.6%       4444 ±  7%  sched_debug.cpu.curr->pid.max
    150621 ±  6%     -31.1%     103723 ± 26%  sched_debug.cpu.nr_switches.min
    148865 ±  6%     -31.1%     102577 ± 26%  sched_debug.cpu.sched_count.min
     36388           -27.7%      26315 ± 22%  sched_debug.cpu.sched_goidle.avg
     23703 ±  3%     -35.2%      15364 ± 25%  sched_debug.cpu.sched_goidle.min
     74196 ±  5%     -30.8%      51361 ± 26%  sched_debug.cpu.ttwu_count.min
    116395           -20.2%      92858 ± 13%  sched_debug.cpu_clk
    115878           -20.3%      92361 ± 13%  sched_debug.ktime
    116767           -20.2%      93217 ± 13%  sched_debug.sched_clk
 1.668e+09            +3.3%  1.724e+09        perf-stat.i.branch-instructions
  28584168 ±  9%      +8.4%   30994854 ±  8%  perf-stat.i.branch-misses
  33324787            +7.5%   35813189        perf-stat.i.cache-misses
  83826973 ± 12%     +12.7%   94503492 ± 13%  perf-stat.i.cache-references
    258508            +5.4%     272477        perf-stat.i.context-switches
      2.60 ±  2%      -5.6%       2.45 ±  2%  perf-stat.i.cpi
    674.12 ±  2%      -7.8%     621.30 ±  3%  perf-stat.i.cycles-between-cache-misses
 2.429e+09            +3.9%  2.524e+09        perf-stat.i.dTLB-loads
 1.259e+09            +5.3%  1.326e+09        perf-stat.i.dTLB-stores
     53.99            +2.1       56.12        perf-stat.i.iTLB-load-miss-rate%
   9728741            +5.2%   10234658 ±  3%  perf-stat.i.iTLB-load-misses
 8.601e+09            +3.8%  8.927e+09        perf-stat.i.instructions
      0.39 ±  2%      +5.7%       0.41 ±  2%  perf-stat.i.ipc
     56.92            +4.2%      59.31        perf-stat.i.metric.M/sec
   9006030            +7.1%    9648264        perf-stat.i.node-load-misses
   3237763 ±  2%     +10.6%    3581907 ±  2%  perf-stat.i.node-loads
   2918052 ±  2%      +6.4%    3106057        perf-stat.i.node-store-misses
    582042 ±  3%     +11.5%     649247 ±  4%  perf-stat.i.node-stores
      2.53 ±  2%      -5.2%       2.40 ±  2%  perf-stat.overall.cpi
    656.32 ±  2%      -8.5%     600.78 ±  2%  perf-stat.overall.cycles-between-cache-misses
     53.81            +2.0       55.85        perf-stat.overall.iTLB-load-miss-rate%
      0.39 ±  2%      +5.5%       0.42 ±  2%  perf-stat.overall.ipc
     66759            -4.6%      63713        perf-stat.overall.path-length
 1.666e+09            +3.3%  1.721e+09        perf-stat.ps.branch-instructions
  28522187 ±  9%      +8.4%   30911250 ±  8%  perf-stat.ps.branch-misses
  33163222            +7.5%   35640249        perf-stat.ps.cache-misses
  83518487 ± 12%     +12.7%   94131115 ± 12%  perf-stat.ps.cache-references
    257076            +5.4%     270958        perf-stat.ps.context-switches
 2.425e+09            +3.9%   2.52e+09        perf-stat.ps.dTLB-loads
 1.256e+09            +5.3%  1.323e+09        perf-stat.ps.dTLB-stores
   9695548            +5.2%   10200116 ±  3%  perf-stat.ps.iTLB-load-misses
 8.591e+09            +3.8%  8.916e+09        perf-stat.ps.instructions
   8959319            +7.1%    9599418        perf-stat.ps.node-load-misses
   3223144 ±  2%     +10.6%    3564824 ±  2%  perf-stat.ps.node-loads
   2903781 ±  2%      +6.5%    3091553        perf-stat.ps.node-store-misses
    580260 ±  3%     +11.5%     646965 ±  4%  perf-stat.ps.node-stores
 1.743e+12            +3.7%  1.807e+12        perf-stat.total.instructions
      1131 ±164%     -99.8%       2.25 ±173%  interrupts.79:PCI-MSI.31981612-edge.i40e-eth0-TxRx-43
    714498 ±  2%     +14.7%     819498 ±  2%  interrupts.CAL:Function_call_interrupts
      6256 ±  8%     +31.9%       8251 ±  6%  interrupts.CPU0.CAL:Function_call_interrupts
    306.50 ± 12%     +31.2%     402.00 ±  7%  interrupts.CPU0.RES:Rescheduling_interrupts
      6860 ± 11%     +39.6%       9576 ±  7%  interrupts.CPU1.CAL:Function_call_interrupts
    291.25 ± 16%     +23.7%     360.25 ±  7%  interrupts.CPU1.RES:Rescheduling_interrupts
      6531 ±  4%     +23.6%       8073 ± 12%  interrupts.CPU11.CAL:Function_call_interrupts
      6598 ±  9%     +22.4%       8078 ±  7%  interrupts.CPU12.CAL:Function_call_interrupts
      6250 ±  8%     +34.0%       8372 ±  9%  interrupts.CPU13.CAL:Function_call_interrupts
      6748 ± 11%     +25.4%       8460 ± 10%  interrupts.CPU14.CAL:Function_call_interrupts
      6387 ± 10%     +33.1%       8498 ± 10%  interrupts.CPU16.CAL:Function_call_interrupts
      6562 ± 12%     +21.9%       7996 ±  9%  interrupts.CPU19.CAL:Function_call_interrupts
     78.75 ± 36%     -64.8%      27.75 ± 39%  interrupts.CPU19.TLB:TLB_shootdowns
      7255 ±  4%     +23.5%       8959 ±  7%  interrupts.CPU2.CAL:Function_call_interrupts
      6542 ±  9%     +34.1%       8770 ±  4%  interrupts.CPU20.CAL:Function_call_interrupts
      6278 ±  9%     +37.5%       8635 ±  5%  interrupts.CPU21.CAL:Function_call_interrupts
    243.00 ± 13%     +24.4%     302.25 ±  6%  interrupts.CPU21.RES:Rescheduling_interrupts
    397.25 ± 12%     -21.1%     313.50 ± 18%  interrupts.CPU24.RES:Rescheduling_interrupts
    540.25 ± 26%     +68.0%     907.50 ± 23%  interrupts.CPU26.NMI:Non-maskable_interrupts
    540.25 ± 26%     +68.0%     907.50 ± 23%  interrupts.CPU26.PMI:Performance_monitoring_interrupts
    336.50 ± 11%     -26.4%     247.75 ±  8%  interrupts.CPU29.RES:Rescheduling_interrupts
      7506 ±  3%     +25.4%       9410 ± 11%  interrupts.CPU3.CAL:Function_call_interrupts
      6747 ± 11%     +17.7%       7944 ±  5%  interrupts.CPU39.CAL:Function_call_interrupts
      6517 ± 10%     +31.9%       8593 ±  8%  interrupts.CPU4.CAL:Function_call_interrupts
      1131 ±164%     -99.8%       2.00 ±173%  interrupts.CPU43.79:PCI-MSI.31981612-edge.i40e-eth0-TxRx-43
      1009 ± 10%     -37.5%     631.25 ± 27%  interrupts.CPU43.NMI:Non-maskable_interrupts
      1009 ± 10%     -37.5%     631.25 ± 27%  interrupts.CPU43.PMI:Performance_monitoring_interrupts
      6214 ± 17%     +26.6%       7866 ±  5%  interrupts.CPU48.CAL:Function_call_interrupts
     95.00 ± 34%     -64.5%      33.75 ± 46%  interrupts.CPU49.TLB:TLB_shootdowns
      7046 ± 12%     +31.1%       9235 ± 10%  interrupts.CPU5.CAL:Function_call_interrupts
      7146 ±  9%     +22.1%       8727 ±  2%  interrupts.CPU50.CAL:Function_call_interrupts
      7306 ±  8%     +18.5%       8654 ± 14%  interrupts.CPU51.CAL:Function_call_interrupts
      7609 ± 11%     +26.2%       9601 ±  4%  interrupts.CPU53.CAL:Function_call_interrupts
      7626 ±  2%     +15.1%       8774 ±  9%  interrupts.CPU55.CAL:Function_call_interrupts
      7280 ±  4%     +27.3%       9264 ±  4%  interrupts.CPU57.CAL:Function_call_interrupts
    491.75 ± 22%     +26.5%     622.00 ± 29%  interrupts.CPU57.NMI:Non-maskable_interrupts
    491.75 ± 22%     +26.5%     622.00 ± 29%  interrupts.CPU57.PMI:Performance_monitoring_interrupts
      8002 ± 10%     +13.9%       9117 ±  6%  interrupts.CPU60.CAL:Function_call_interrupts
      7373 ±  8%     +32.7%       9786 ±  4%  interrupts.CPU62.CAL:Function_call_interrupts
      7193 ±  5%     +33.4%       9593 ±  4%  interrupts.CPU63.CAL:Function_call_interrupts
    272.50 ± 12%     +29.0%     351.50 ± 14%  interrupts.CPU63.RES:Rescheduling_interrupts
      7818 ±  6%     +21.6%       9507 ±  5%  interrupts.CPU64.CAL:Function_call_interrupts
      7044 ± 11%     +34.5%       9474 ± 10%  interrupts.CPU65.CAL:Function_call_interrupts
      7602 ±  9%     +21.2%       9216 ±  6%  interrupts.CPU66.CAL:Function_call_interrupts
      7413 ±  4%     +29.0%       9561 ±  6%  interrupts.CPU67.CAL:Function_call_interrupts
    282.00 ± 17%     +23.7%     348.75 ±  5%  interrupts.CPU67.RES:Rescheduling_interrupts
      7334 ±  6%     +24.8%       9155 ±  8%  interrupts.CPU68.CAL:Function_call_interrupts
    748.50 ± 24%     +32.8%     994.25 ±  8%  interrupts.CPU69.NMI:Non-maskable_interrupts
    748.50 ± 24%     +32.8%     994.25 ±  8%  interrupts.CPU69.PMI:Performance_monitoring_interrupts
      6745 ±  8%     +24.7%       8412 ±  9%  interrupts.CPU7.CAL:Function_call_interrupts
      7765 ±  6%     +25.8%       9769 ±  6%  interrupts.CPU70.CAL:Function_call_interrupts
    281.75 ±  8%     +18.4%     333.50 ± 12%  interrupts.CPU70.RES:Rescheduling_interrupts
      7299 ±  6%     +33.6%       9749 ± 12%  interrupts.CPU71.CAL:Function_call_interrupts
      1046 ±  8%     -37.0%     659.50 ± 17%  interrupts.CPU73.NMI:Non-maskable_interrupts
      1046 ±  8%     -37.0%     659.50 ± 17%  interrupts.CPU73.PMI:Performance_monitoring_interrupts
    970.50 ± 10%     -26.4%     714.00 ± 22%  interrupts.CPU75.NMI:Non-maskable_interrupts
    970.50 ± 10%     -26.4%     714.00 ± 22%  interrupts.CPU75.PMI:Performance_monitoring_interrupts
    854.25 ± 19%     -39.6%     515.75 ± 27%  interrupts.CPU76.NMI:Non-maskable_interrupts
    854.25 ± 19%     -39.6%     515.75 ± 27%  interrupts.CPU76.PMI:Performance_monitoring_interrupts
    978.00 ± 24%     -46.2%     525.75 ± 40%  interrupts.CPU77.NMI:Non-maskable_interrupts
    978.00 ± 24%     -46.2%     525.75 ± 40%  interrupts.CPU77.PMI:Performance_monitoring_interrupts
      7666 ±  2%     +16.3%       8914 ±  7%  interrupts.CPU78.CAL:Function_call_interrupts
    888.25 ± 20%     -28.4%     636.00 ± 25%  interrupts.CPU78.NMI:Non-maskable_interrupts
    888.25 ± 20%     -28.4%     636.00 ± 25%  interrupts.CPU78.PMI:Performance_monitoring_interrupts
    829.00 ± 13%     -17.2%     686.75 ± 20%  interrupts.CPU79.NMI:Non-maskable_interrupts
    829.00 ± 13%     -17.2%     686.75 ± 20%  interrupts.CPU79.PMI:Performance_monitoring_interrupts
    314.75 ± 10%     -12.4%     275.75 ±  5%  interrupts.CPU85.RES:Rescheduling_interrupts
    341.50 ± 13%     -17.9%     280.25 ± 13%  interrupts.CPU92.RES:Rescheduling_interrupts
    350.25 ± 11%     -22.6%     271.25 ±  9%  interrupts.CPU94.RES:Rescheduling_interrupts
     15.97 ±  4%      -1.3       14.63 ±  3%  perf-profile.calltrace.cycles-pp.xlog_grant_head_check.xfs_log_reserve.xfs_trans_reserve.xfs_trans_alloc.xfs_iomap_write_direct
     16.73 ±  4%      -1.2       15.49 ±  2%  perf-profile.calltrace.cycles-pp.xfs_log_reserve.xfs_trans_reserve.xfs_trans_alloc.xfs_iomap_write_direct.xfs_direct_write_iomap_begin
     16.78 ±  4%      -1.2       15.55 ±  3%  perf-profile.calltrace.cycles-pp.xfs_trans_reserve.xfs_trans_alloc.xfs_iomap_write_direct.xfs_direct_write_iomap_begin.iomap_apply
     16.87 ±  4%      -1.2       15.65 ±  2%  perf-profile.calltrace.cycles-pp.xfs_trans_alloc.xfs_iomap_write_direct.xfs_direct_write_iomap_begin.iomap_apply.dax_iomap_rw
      7.83 ±  3%      -1.1        6.68 ±  4%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.xlog_grant_head_wait.xlog_grant_head_check.xfs_log_reserve
      7.89 ±  3%      -1.1        6.76 ±  4%  perf-profile.calltrace.cycles-pp._raw_spin_lock.xlog_grant_head_wait.xlog_grant_head_check.xfs_log_reserve.xfs_trans_reserve
      8.81 ±  3%      -1.1        7.74 ±  4%  perf-profile.calltrace.cycles-pp.xlog_grant_head_wait.xlog_grant_head_check.xfs_log_reserve.xfs_trans_reserve.xfs_trans_alloc
      9.79 ±  4%      -1.1        8.71 ±  3%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.xfs_log_space_wake.xfs_log_ticket_ungrant.xfs_log_commit_cil
      9.86 ±  4%      -1.1        8.80 ±  3%  perf-profile.calltrace.cycles-pp._raw_spin_lock.xfs_log_space_wake.xfs_log_ticket_ungrant.xfs_log_commit_cil.__xfs_trans_commit
     10.16 ±  3%      -1.0        9.15 ±  3%  perf-profile.calltrace.cycles-pp.xfs_log_space_wake.xfs_log_ticket_ungrant.xfs_log_commit_cil.__xfs_trans_commit.xfs_iomap_write_direct
     10.83 ±  3%      -0.8        9.99 ±  3%  perf-profile.calltrace.cycles-pp.xfs_log_ticket_ungrant.xfs_log_commit_cil.__xfs_trans_commit.xfs_iomap_write_direct.xfs_direct_write_iomap_begin
      3.89 ±  7%      -0.4        3.45 ±  4%  perf-profile.calltrace.cycles-pp.file_update_time.xfs_file_aio_write_checks.xfs_file_dax_write.new_sync_write.vfs_write
      3.97 ±  7%      -0.4        3.54 ±  4%  perf-profile.calltrace.cycles-pp.xfs_file_aio_write_checks.xfs_file_dax_write.new_sync_write.vfs_write.ksys_write
      3.84 ±  6%      -0.4        3.42 ±  4%  perf-profile.calltrace.cycles-pp.xfs_vn_update_time.file_update_time.xfs_file_aio_write_checks.xfs_file_dax_write.new_sync_write
      2.31 ±  7%      -0.4        1.91 ±  6%  perf-profile.calltrace.cycles-pp.xlog_grant_head_check.xfs_log_reserve.xfs_trans_reserve.xfs_trans_alloc.xfs_vn_update_time
      2.40 ±  7%      -0.4        1.99 ±  5%  perf-profile.calltrace.cycles-pp.xfs_log_reserve.xfs_trans_reserve.xfs_trans_alloc.xfs_vn_update_time.file_update_time
      2.40 ±  6%      -0.4        2.00 ±  5%  perf-profile.calltrace.cycles-pp.xfs_trans_reserve.xfs_trans_alloc.xfs_vn_update_time.file_update_time.xfs_file_aio_write_checks
      2.40 ±  7%      -0.4        2.01 ±  5%  perf-profile.calltrace.cycles-pp.xfs_trans_alloc.xfs_vn_update_time.file_update_time.xfs_file_aio_write_checks.xfs_file_dax_write
      0.96 ±  9%      -0.1        0.83 ±  5%  perf-profile.calltrace.cycles-pp.xfs_log_space_wake.xfs_log_ticket_ungrant.xfs_log_commit_cil.__xfs_trans_commit.xfs_vn_update_time
      0.89 ±  2%      -0.1        0.83 ±  3%  perf-profile.calltrace.cycles-pp.xfs_trans_committed_bulk.xlog_cil_committed.xlog_cil_process_committed.xlog_state_do_callback.xlog_ioend_work
      0.56 ±  4%      +0.1        0.63 ±  3%  perf-profile.calltrace.cycles-pp.xfs_bmbt_init_key_from_rec.xfs_lookup_get_search_key.xfs_btree_lookup.xfs_bmap_add_extent_unwritten_real.xfs_bmapi_convert_unwritten
      0.67 ±  4%      +0.1        0.75 ±  2%  perf-profile.calltrace.cycles-pp.xfs_lookup_get_search_key.xfs_btree_lookup.xfs_bmap_add_extent_unwritten_real.xfs_bmapi_convert_unwritten.xfs_bmapi_write
      0.80 ±  7%      +0.1        0.89 ±  6%  perf-profile.calltrace.cycles-pp.xfs_buf_item_format.xlog_cil_insert_items.xfs_log_commit_cil.__xfs_trans_commit.xfs_iomap_write_direct
      0.69 ±  6%      +0.1        0.78 ±  5%  perf-profile.calltrace.cycles-pp.xlog_state_release_iclog.xlog_write.xlog_cil_push_work.process_one_work.worker_thread
      0.67 ±  4%      +0.1        0.77 ±  8%  perf-profile.calltrace.cycles-pp.pmem_submit_bio.submit_bio_noacct.submit_bio._xfs_buf_ioapply.__xfs_buf_submit
      0.73 ±  4%      +0.1        0.85 ±  7%  perf-profile.calltrace.cycles-pp.submit_bio_noacct.submit_bio._xfs_buf_ioapply.__xfs_buf_submit.xfs_buf_delwri_submit_buffers
      0.73 ±  4%      +0.1        0.86 ±  7%  perf-profile.calltrace.cycles-pp.submit_bio._xfs_buf_ioapply.__xfs_buf_submit.xfs_buf_delwri_submit_buffers.xfsaild_push
      0.69 ±  7%      +0.1        0.82 ±  8%  perf-profile.calltrace.cycles-pp.xfs_iext_lookup_extent.xfs_bmapi_read.xfs_direct_write_iomap_begin.iomap_apply.dax_iomap_rw
      0.74 ±  7%      +0.1        0.88 ±  8%  perf-profile.calltrace.cycles-pp.xfs_bmapi_read.xfs_direct_write_iomap_begin.iomap_apply.dax_iomap_rw.xfs_file_dax_write
      1.21 ±  4%      +0.2        1.36 ±  6%  perf-profile.calltrace.cycles-pp.pmem_submit_bio.submit_bio_noacct.submit_bio.submit_bio_wait.blkdev_issue_zeroout
      1.74 ±  6%      +0.2        1.92 ±  4%  perf-profile.calltrace.cycles-pp.blkdev_issue_zeroout.xfs_bmapi_convert_unwritten.xfs_bmapi_write.xfs_iomap_write_direct.xfs_direct_write_iomap_begin
      1.26 ±  4%      +0.2        1.46 ±  4%  perf-profile.calltrace.cycles-pp.xlog_write.xlog_cil_push_work.process_one_work.worker_thread.kthread
      1.76 ±  4%      +0.2        1.96 ±  4%  perf-profile.calltrace.cycles-pp.xfs_btree_insrec.xfs_btree_insert.xfs_bmap_add_extent_unwritten_real.xfs_bmapi_convert_unwritten.xfs_bmapi_write
      0.42 ± 57%      +0.2        0.62 ±  6%  perf-profile.calltrace.cycles-pp.xfs_cil_prepare_item.xlog_cil_insert_items.xfs_log_commit_cil.__xfs_trans_commit.xfs_iomap_write_direct
      1.80 ±  4%      +0.2        2.02 ±  4%  perf-profile.calltrace.cycles-pp.xfs_btree_insert.xfs_bmap_add_extent_unwritten_real.xfs_bmapi_convert_unwritten.xfs_bmapi_write.xfs_iomap_write_direct
      2.75 ±  3%      +0.2        2.97 ±  5%  perf-profile.calltrace.cycles-pp.xfsaild.kthread.ret_from_fork
      2.75 ±  3%      +0.2        2.97 ±  5%  perf-profile.calltrace.cycles-pp.xfsaild_push.xfsaild.kthread.ret_from_fork
      1.25 ±  5%      +0.2        1.48 ±  2%  perf-profile.calltrace.cycles-pp.dax_iomap_actor.iomap_apply.dax_iomap_rw.xfs_file_dax_write.new_sync_write
      1.39 ±  3%      +0.2        1.63 ±  3%  perf-profile.calltrace.cycles-pp.xlog_cil_push_work.process_one_work.worker_thread.kthread.ret_from_fork
      1.32 ±  4%      +0.3        1.60 ±  5%  perf-profile.calltrace.cycles-pp._xfs_buf_ioapply.__xfs_buf_submit.xfs_buf_delwri_submit_buffers.xfsaild_push.xfsaild
      2.99            +0.3        3.30 ±  2%  perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
      1.96 ±  4%      +0.3        2.29 ±  5%  perf-profile.calltrace.cycles-pp.__xfs_buf_submit.xfs_buf_delwri_submit_buffers.xfsaild_push.xfsaild.kthread
      2.30 ±  4%      +0.3        2.63 ±  5%  perf-profile.calltrace.cycles-pp.xfs_buf_delwri_submit_buffers.xfsaild_push.xfsaild.kthread.ret_from_fork
      3.31            +0.4        3.69 ±  3%  perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
      6.09 ±  2%      +0.6        6.70 ±  4%  perf-profile.calltrace.cycles-pp.ret_from_fork
      6.09 ±  2%      +0.6        6.70 ±  4%  perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
      6.92 ±  4%      +0.8        7.72 ±  3%  perf-profile.calltrace.cycles-pp.xfs_bmap_add_extent_unwritten_real.xfs_bmapi_convert_unwritten.xfs_bmapi_write.xfs_iomap_write_direct.xfs_direct_write_iomap_begin
      8.79 ±  4%      +1.0        9.78 ±  3%  perf-profile.calltrace.cycles-pp.xfs_bmapi_convert_unwritten.xfs_bmapi_write.xfs_iomap_write_direct.xfs_direct_write_iomap_begin.iomap_apply
      9.29 ±  4%      +1.0       10.33 ±  3%  perf-profile.calltrace.cycles-pp.xfs_bmapi_write.xfs_iomap_write_direct.xfs_direct_write_iomap_begin.iomap_apply.dax_iomap_rw
      2.87 ±  5%      +1.4        4.24 ±  7%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.xlog_cil_insert_items.xfs_log_commit_cil.__xfs_trans_commit
      3.03 ±  5%      +1.4        4.45 ±  7%  perf-profile.calltrace.cycles-pp._raw_spin_lock.xlog_cil_insert_items.xfs_log_commit_cil.__xfs_trans_commit.xfs_iomap_write_direct
     18.13 ±  4%      +1.7       19.80 ±  5%  perf-profile.calltrace.cycles-pp.__xfs_trans_commit.xfs_iomap_write_direct.xfs_direct_write_iomap_begin.iomap_apply.dax_iomap_rw
     18.04 ±  4%      +1.7       19.71 ±  5%  perf-profile.calltrace.cycles-pp.xfs_log_commit_cil.__xfs_trans_commit.xfs_iomap_write_direct.xfs_direct_write_iomap_begin.iomap_apply
      4.89 ±  5%      +1.7        6.59 ±  4%  perf-profile.calltrace.cycles-pp.xlog_cil_insert_items.xfs_log_commit_cil.__xfs_trans_commit.xfs_iomap_write_direct.xfs_direct_write_iomap_begin
     18.29 ±  4%      -1.8       16.54 ±  3%  perf-profile.children.cycles-pp.xlog_grant_head_check
     19.12 ±  4%      -1.6       17.49 ±  2%  perf-profile.children.cycles-pp.xfs_log_reserve
     19.18 ±  4%      -1.6       17.55 ±  3%  perf-profile.children.cycles-pp.xfs_trans_reserve
     19.28 ±  4%      -1.6       17.66 ±  2%  perf-profile.children.cycles-pp.xfs_trans_alloc
     11.15 ±  4%      -1.1       10.02 ±  3%  perf-profile.children.cycles-pp.xfs_log_space_wake
      8.81 ±  3%      -1.1        7.74 ±  4%  perf-profile.children.cycles-pp.xlog_grant_head_wait
     11.88 ±  3%      -1.0       10.92 ±  3%  perf-profile.children.cycles-pp.xfs_log_ticket_ungrant
      3.89 ±  7%      -0.4        3.45 ±  4%  perf-profile.children.cycles-pp.file_update_time
      3.97 ±  7%      -0.4        3.54 ±  4%  perf-profile.children.cycles-pp.xfs_file_aio_write_checks
      3.84 ±  6%      -0.4        3.42 ±  4%  perf-profile.children.cycles-pp.xfs_vn_update_time
      0.36 ±  6%      -0.3        0.10 ± 11%  perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
      1.70 ±  3%      -0.2        1.52 ±  5%  perf-profile.children.cycles-pp.xlog_grant_head_wake
      0.30 ±  5%      -0.1        0.23 ± 10%  perf-profile.children.cycles-pp.xfs_buf_item_push
      0.89 ±  2%      -0.1        0.83 ±  3%  perf-profile.children.cycles-pp.xfs_trans_committed_bulk
      0.11 ± 15%      -0.0        0.08 ± 19%  perf-profile.children.cycles-pp.get_next_timer_interrupt
      0.17 ±  2%      -0.0        0.15 ±  5%  perf-profile.children.cycles-pp.lapic_next_deadline
      0.09            +0.0        0.11 ±  6%  perf-profile.children.cycles-pp.xfs_btree_ptr_addr
      0.11 ±  6%      +0.0        0.13 ± 12%  perf-profile.children.cycles-pp.crc_128
      0.16 ± 10%      +0.0        0.19 ±  6%  perf-profile.children.cycles-pp.xfs_trans_ail_update_bulk
      0.10 ± 12%      +0.0        0.13 ±  6%  perf-profile.children.cycles-pp.xfs_trans_dirty_buf
      0.13 ±  6%      +0.0        0.16        perf-profile.children.cycles-pp.irqtime_account_irq
      0.09            +0.0        0.12 ±  3%  perf-profile.children.cycles-pp.xfs_btree_log_block
      0.04 ± 58%      +0.0        0.07 ±  6%  perf-profile.children.cycles-pp.percpu_counter_add_batch
      0.04 ± 58%      +0.0        0.07 ± 12%  perf-profile.children.cycles-pp.xfs_verify_fsbno
      0.04 ± 58%      +0.0        0.07 ± 10%  perf-profile.children.cycles-pp.cpumask_next_and
      0.11 ±  9%      +0.0        0.14 ± 15%  perf-profile.children.cycles-pp.__bio_add_page
      0.14 ± 11%      +0.0        0.17 ± 10%  perf-profile.children.cycles-pp.bio_add_page
      0.10 ±  7%      +0.0        0.13 ± 14%  perf-profile.children.cycles-pp.sysvec_call_function_single
      0.07 ±  6%      +0.0        0.10 ±  7%  perf-profile.children.cycles-pp.xfs_ail_delete_one
      0.14 ± 10%      +0.0        0.18 ±  4%  perf-profile.children.cycles-pp.finish_task_switch
      0.16 ±  5%      +0.0        0.19 ±  9%  perf-profile.children.cycles-pp.xfs_perag_put
      0.09 ± 13%      +0.0        0.12 ± 14%  perf-profile.children.cycles-pp.xfs_ail_check
      0.17 ±  7%      +0.0        0.21 ±  6%  perf-profile.children.cycles-pp.xfs_buf_unlock
      0.15 ±  5%      +0.0        0.20 ±  5%  perf-profile.children.cycles-pp.__slab_free
      0.12 ±  8%      +0.0        0.16 ±  4%  perf-profile.children.cycles-pp.xfs_trans_log_buf
      0.11 ± 10%      +0.0        0.15 ±  8%  perf-profile.children.cycles-pp.asm_sysvec_call_function_single
      0.24 ±  9%      +0.0        0.28 ±  5%  perf-profile.children.cycles-pp.xfs_btree_update
      0.28 ±  7%      +0.0        0.32 ±  5%  perf-profile.children.cycles-pp.orc_find
      0.26 ±  6%      +0.0        0.30 ±  6%  perf-profile.children.cycles-pp.xfs_buf_bio_end_io
      0.33 ±  6%      +0.0        0.38 ±  3%  perf-profile.children.cycles-pp.bio_alloc_bioset
      0.06 ±  7%      +0.1        0.11 ± 10%  perf-profile.children.cycles-pp.xfs_btree_lblock_calc_crc
      0.01 ±173%      +0.1        0.07 ± 25%  perf-profile.children.cycles-pp.timerqueue_del
      0.19 ±  7%      +0.1        0.24 ±  3%  perf-profile.children.cycles-pp.preempt_schedule_common
      0.00            +0.1        0.05 ±  8%  perf-profile.children.cycles-pp._find_next_bit
      0.26 ±  7%      +0.1        0.32 ±  3%  perf-profile.children.cycles-pp._cond_resched
      0.51 ±  4%      +0.1        0.56 ±  5%  perf-profile.children.cycles-pp._xfs_trans_bjoin
      0.14 ± 11%      +0.1        0.20 ±  4%  perf-profile.children.cycles-pp.xfs_buf_item_done
      0.14 ± 10%      +0.1        0.20 ±  5%  perf-profile.children.cycles-pp.xfs_trans_ail_delete
      0.23 ±  9%      +0.1        0.29 ±  5%  perf-profile.children.cycles-pp.__orc_find
      0.39 ±  9%      +0.1        0.45 ±  4%  perf-profile.children.cycles-pp.xfs_buf_item_pin
      0.45 ±  8%      +0.1        0.51 ±  2%  perf-profile.children.cycles-pp.update_sd_lb_stats
      0.46 ±  8%      +0.1        0.52 ±  2%  perf-profile.children.cycles-pp.find_busiest_group
      0.24 ±  8%      +0.1        0.31 ±  7%  perf-profile.children.cycles-pp.__kmalloc
      0.30 ± 11%      +0.1        0.37 ±  7%  perf-profile.children.cycles-pp.schedule_idle
      0.38 ±  6%      +0.1        0.45 ±  6%  perf-profile.children.cycles-pp.down_read
      0.60 ±  6%      +0.1        0.67 ±  4%  perf-profile.children.cycles-pp.xfs_perag_get
      0.54            +0.1        0.61 ±  3%  perf-profile.children.cycles-pp.memmove
      0.46 ±  7%      +0.1        0.53 ±  4%  perf-profile.children.cycles-pp.newidle_balance
      0.57 ±  3%      +0.1        0.65 ±  2%  perf-profile.children.cycles-pp.xfs_bmbt_init_key_from_rec
      0.54 ± 10%      +0.1        0.61 ±  2%  perf-profile.children.cycles-pp.load_balance
      0.83 ±  7%      +0.1        0.91 ±  6%  perf-profile.children.cycles-pp.xfs_buf_item_format
      0.11 ±  7%      +0.1        0.19 ±  2%  perf-profile.children.cycles-pp.up
      0.68 ±  3%      +0.1        0.76 ±  2%  perf-profile.children.cycles-pp.xfs_lookup_get_search_key
      0.20 ±  5%      +0.1        0.28 ±  2%  perf-profile.children.cycles-pp.__wake_up_common_lock
      0.28 ±  3%      +0.1        0.37        perf-profile.children.cycles-pp.xfs_buf_ioend
      0.54 ±  8%      +0.1        0.63 ±  5%  perf-profile.children.cycles-pp.xfs_cil_prepare_item
      0.69 ±  6%      +0.1        0.78 ±  5%  perf-profile.children.cycles-pp.xlog_state_release_iclog
      0.57 ±  8%      +0.1        0.67 ±  5%  perf-profile.children.cycles-pp.unwind_next_frame
      0.89 ±  5%      +0.1        1.01 ±  7%  perf-profile.children.cycles-pp.memcpy_erms
      0.43 ±  6%      +0.1        0.56 ±  7%  perf-profile.children.cycles-pp.__srcu_read_unlock
      0.68 ±  5%      +0.1        0.82 ±  4%  perf-profile.children.cycles-pp.pick_next_task_fair
      0.74 ±  7%      +0.1        0.89 ±  8%  perf-profile.children.cycles-pp.xfs_bmapi_read
      1.09 ±  6%      +0.1        1.23 ±  3%  perf-profile.children.cycles-pp.schedule
      0.81 ±  8%      +0.2        0.97 ±  4%  perf-profile.children.cycles-pp.arch_stack_walk
      0.88 ±  7%      +0.2        1.03 ±  5%  perf-profile.children.cycles-pp.stack_trace_save_tsk
      0.82 ±  7%      +0.2        0.98 ±  7%  perf-profile.children.cycles-pp.xfs_iext_lookup_extent
      1.13 ±  6%      +0.2        1.29 ±  4%  perf-profile.children.cycles-pp.__account_scheduler_latency
      1.46 ±  5%      +0.2        1.63 ±  4%  perf-profile.children.cycles-pp.ttwu_do_activate
      1.45 ±  5%      +0.2        1.62 ±  4%  perf-profile.children.cycles-pp.enqueue_task_fair
      1.74 ±  6%      +0.2        1.92 ±  4%  perf-profile.children.cycles-pp.blkdev_issue_zeroout
      1.38 ±  5%      +0.2        1.56 ±  4%  perf-profile.children.cycles-pp.enqueue_entity
      1.26 ±  4%      +0.2        1.46 ±  3%  perf-profile.children.cycles-pp.xlog_write
      1.76 ±  4%      +0.2        1.97 ±  4%  perf-profile.children.cycles-pp.xfs_btree_insrec
      1.81 ±  4%      +0.2        2.02 ±  4%  perf-profile.children.cycles-pp.xfs_btree_insert
      2.75 ±  3%      +0.2        2.97 ±  5%  perf-profile.children.cycles-pp.xfsaild
      2.75 ±  3%      +0.2        2.97 ±  5%  perf-profile.children.cycles-pp.xfsaild_push
      1.25 ±  5%      +0.2        1.48 ±  2%  perf-profile.children.cycles-pp.dax_iomap_actor
      1.39 ±  3%      +0.2        1.63 ±  3%  perf-profile.children.cycles-pp.xlog_cil_push_work
      1.54 ±  5%      +0.3        1.79 ±  3%  perf-profile.children.cycles-pp.__schedule
      1.32 ±  4%      +0.3        1.60 ±  5%  perf-profile.children.cycles-pp._xfs_buf_ioapply
      2.99            +0.3        3.30 ±  2%  perf-profile.children.cycles-pp.process_one_work
      1.96 ±  3%      +0.3        2.29 ±  5%  perf-profile.children.cycles-pp.__xfs_buf_submit
      2.43 ±  4%      +0.3        2.75 ±  6%  perf-profile.children.cycles-pp.pmem_submit_bio
      2.30 ±  4%      +0.3        2.63 ±  5%  perf-profile.children.cycles-pp.xfs_buf_delwri_submit_buffers
      2.73 ±  4%      +0.4        3.08 ±  5%  perf-profile.children.cycles-pp.submit_bio_noacct
      2.75 ±  4%      +0.4        3.10 ±  5%  perf-profile.children.cycles-pp.submit_bio
      1.52 ±  6%      +0.4        1.89 ±  4%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      3.31            +0.4        3.69 ±  3%  perf-profile.children.cycles-pp.worker_thread
      6.10 ±  2%      +0.6        6.70 ±  4%  perf-profile.children.cycles-pp.ret_from_fork
      6.09 ±  2%      +0.6        6.70 ±  4%  perf-profile.children.cycles-pp.kthread
      6.92 ±  4%      +0.8        7.73 ±  3%  perf-profile.children.cycles-pp.xfs_bmap_add_extent_unwritten_real
      8.79 ±  4%      +1.0        9.78 ±  3%  perf-profile.children.cycles-pp.xfs_bmapi_convert_unwritten
      9.29 ±  4%      +1.0       10.33 ±  3%  perf-profile.children.cycles-pp.xfs_bmapi_write
     19.55 ±  4%      +1.6       21.20 ±  4%  perf-profile.children.cycles-pp.__xfs_trans_commit
     19.46 ±  4%      +1.6       21.11 ±  4%  perf-profile.children.cycles-pp.xfs_log_commit_cil
      5.16 ±  5%      +1.8        6.94 ±  4%  perf-profile.children.cycles-pp.xlog_cil_insert_items
      0.31 ±  7%      -0.2        0.08 ±  8%  perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
      0.35 ±  5%      -0.2        0.14 ± 10%  perf-profile.self.cycles-pp.xlog_grant_head_wake
      0.20 ±  5%      -0.1        0.08 ± 13%  perf-profile.self.cycles-pp.xfs_buf_item_unpin
      0.11 ±  6%      -0.0        0.09 ± 10%  perf-profile.self.cycles-pp.try_to_wake_up
      0.17            -0.0        0.15 ±  5%  perf-profile.self.cycles-pp.lapic_next_deadline
      0.11 ±  3%      +0.0        0.13 ±  5%  perf-profile.self.cycles-pp.xfs_bmbt_key_diff
      0.10 ±  4%      +0.0        0.12 ±  6%  perf-profile.self.cycles-pp._xfs_trans_bjoin
      0.07 ± 10%      +0.0        0.09 ±  8%  perf-profile.self.cycles-pp.irqtime_account_irq
      0.06 ± 11%      +0.0        0.08 ± 15%  perf-profile.self.cycles-pp.pmem_submit_bio
      0.11 ±  6%      +0.0        0.13 ± 12%  perf-profile.self.cycles-pp.crc_128
      0.08 ±  8%      +0.0        0.10 ± 10%  perf-profile.self.cycles-pp.list_sort
      0.10 ± 12%      +0.0        0.12 ±  8%  perf-profile.self.cycles-pp.xfs_trans_dirty_buf
      0.08 ± 15%      +0.0        0.10 ± 10%  perf-profile.self.cycles-pp.xfs_trans_committed_bulk
      0.08 ±  8%      +0.0        0.11 ±  4%  perf-profile.self.cycles-pp.tick_nohz_next_event
      0.14 ±  5%      +0.0        0.17 ±  9%  perf-profile.self.cycles-pp.xfs_cil_prepare_item
      0.11 ±  9%      +0.0        0.14 ± 15%  perf-profile.self.cycles-pp.__bio_add_page
      0.16 ±  7%      +0.0        0.19 ±  8%  perf-profile.self.cycles-pp.xfs_perag_put
      0.08 ± 10%      +0.0        0.12 ± 13%  perf-profile.self.cycles-pp.xfs_ail_check
      0.17 ±  2%      +0.0        0.21 ±  8%  perf-profile.self.cycles-pp.xlog_write
      0.15 ±  5%      +0.0        0.19 ±  3%  perf-profile.self.cycles-pp.__slab_free
      0.25 ± 10%      +0.0        0.29 ±  5%  perf-profile.self.cycles-pp.xfs_buf_bio_end_io
      0.06 ±  9%      +0.0        0.10 ±  8%  perf-profile.self.cycles-pp.xfs_btree_lblock_calc_crc
      0.03 ±100%      +0.1        0.08 ± 10%  perf-profile.self.cycles-pp._xfs_buf_ioapply
      0.23 ±  9%      +0.1        0.29 ±  5%  perf-profile.self.cycles-pp.__orc_find
      0.56 ±  4%      +0.1        0.62 ±  3%  perf-profile.self.cycles-pp.xfs_bmbt_init_key_from_rec
      0.35 ±  8%      +0.1        0.42 ±  6%  perf-profile.self.cycles-pp.down_read
      0.55 ±  7%      +0.1        0.62 ±  3%  perf-profile.self.cycles-pp.xfs_perag_get
      0.54            +0.1        0.61 ±  3%  perf-profile.self.cycles-pp.memmove
      0.49 ±  2%      +0.1        0.57 ±  5%  perf-profile.self.cycles-pp.xfs_btree_lookup
      0.53 ±  9%      +0.1        0.63 ±  4%  perf-profile.self.cycles-pp.xfs_log_commit_cil
      0.43 ±  6%      +0.1        0.55 ±  7%  perf-profile.self.cycles-pp.__srcu_read_unlock
      0.88 ±  5%      +0.1        1.00 ±  8%  perf-profile.self.cycles-pp.memcpy_erms
      0.81 ±  7%      +0.1        0.96 ±  7%  perf-profile.self.cycles-pp.xfs_iext_lookup_extent
      0.75 ± 10%      +0.2        0.92 ±  5%  perf-profile.self.cycles-pp.xfs_log_ticket_ungrant
      1.47 ±  6%      +0.4        1.84 ±  3%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave


                                                                                
                                 fio.write_bw_MBps                              
                                                                                
  600 +---------------------------------------------------------------------+   
      |   O O O O O O O  O O O O O O O O O O O O O O O O                    |   
  500 |.+.+.+.+.+.+.+.+..+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+..+.+.+.+.+.+.+.+.|   
      |                                                                     |   
      |                                                                     |   
  400 |-+                                                                   |   
      |                                                                     |   
  300 |-+                                                                   |   
      |                                                                     |   
  200 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                   fio.write_iops                               
                                                                                
  160000 +------------------------------------------------------------------+   
         |           O                              O                       |   
  140000 |-+.+.O.+.O.+.O.+.+.O O O O O O O.+O O O.+. .+.O  .+. .+.+.+.      |   
  120000 |.+   +   +   +     +.+.+.+.+.+.+  +.+.+   +   +.+   +       +.+.+.|   
         |                                                                  |   
  100000 |-+                                                                |   
         |                                                                  |   
   80000 |-+                                                                |   
         |                                                                  |   
   60000 |-+                                                                |   
   40000 |-+                                                                |   
         |                                                                  |   
   20000 |-+                                                                |   
         |                                                                  |   
       0 +------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                               fio.write_clat_mean_us                           
                                                                                
  400000 +------------------------------------------------------------------+   
         |.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.|   
  350000 |-+ O O O O O O O O O O O O O O O OO O O O O O O                   |   
         |                                                                  |   
  300000 |-+                                                                |   
  250000 |-+                                                                |   
         |                                                                  |   
  200000 |-+                                                                |   
         |                                                                  |   
  150000 |-+                                                                |   
  100000 |-+                                                                |   
         |                                                                  |   
   50000 |-+                                                                |   
         |                                                                  |   
       0 +------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                fio.write_clat_90__us                           
                                                                                
  160000 +------------------------------------------------------------------+   
         |                   +             ++                         +     |   
  140000 |-+ O O O O O O O O O O O O O O O OO O O O O O O                   |   
  120000 |-+                                                                |   
         |                                                                  |   
  100000 |-+                                                                |   
         |                                                                  |   
   80000 |-+                                                                |   
         |                                                                  |   
   60000 |-+                                                                |   
   40000 |-+                                                                |   
         |                                                                  |   
   20000 |-+                                                                |   
         |                                                                  |   
       0 +------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                               fio.latency_250us_                               
                                                                                
  25 +----------------------------------------------------------------------+   
     |                                                                      |   
     |.+.   .+.  .+. .+.       .+.          .+.+.   .+.+.+.   .+..       .+.|   
  20 |-+ +.+   +.   +   +.+.+.+   +.+.+..+.+     +.+       +.+    +.+.+.+   |   
     |   O O O      O O   O O O O   O O  O           O O                    |   
     |         O  O     O         O        O O O O O                        |   
  15 |-+                                                                    |   
     |                                                                      |   
  10 |-+                                                                    |   
     |                                                                      |   
     |                                                                      |   
   5 |-+                                                                    |   
     |                                                                      |   
     |                                                                      |   
   0 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                     fio.workload                               
                                                                                
    3e+07 +-----------------------------------------------------------------+   
          |   O O O O O O OO O O O O O O O O O O O O O O O                  |   
  2.5e+07 |.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.|   
          |                                                                 |   
          |                                                                 |   
    2e+07 |-+                                                               |   
          |                                                                 |   
  1.5e+07 |-+                                                               |   
          |                                                                 |   
    1e+07 |-+                                                               |   
          |                                                                 |   
          |                                                                 |   
    5e+06 |-+                                                               |   
          |                                                                 |   
        0 +-----------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-csl-2ap2: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory




Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Oliver Sang


View attachment "config-5.10.0-rc4-00005-g97e8f0134a2b" of type "text/plain" (170113 bytes)

View attachment "job-script" of type "text/plain" (8472 bytes)

View attachment "job.yaml" of type "text/plain" (5822 bytes)

View attachment "reproduce" of type "text/plain" (956 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ