lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <20210305023016.GA31481@xsang-OptiPlex-9020>
Date:   Fri, 5 Mar 2021 10:30:16 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     "Darrick J. Wong" <djwong@...nel.org>
Cc:     LKML <linux-kernel@...r.kernel.org>,
        "Darrick J. Wong" <darrick.wong@...cle.com>, lkp@...ts.01.org,
        lkp@...el.com, ying.huang@...el.com, feng.tang@...el.com,
        zhengjun.xing@...el.com
Subject: [xfs]  7f83561097:  stress-ng.dnotify.ops_per_sec -73.9% regression


Greeting,

FYI, we noticed a -73.9% regression of stress-ng.dnotify.ops_per_sec due to commit:


commit: 7f835610979ba652213f2a41e60f3e0ad16884bf ("xfs: deferred inode inactivation")
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git vectorized-scrub


in testcase: stress-ng
on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory
with following parameters:

	nr_threads: 10%
	disk: 1HDD
	testtime: 60s
	fs: xfs
	class: filesystem
	test: dnotify
	cpufreq_governor: performance
	ucode: 0x5003006




If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang@...el.com>


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install                job.yaml  # job file is attached in this email
        bin/lkp split-job --compatible job.yaml
        bin/lkp run                    compatible-job.yaml

=========================================================================================
class/compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode:
  filesystem/gcc-9/performance/1HDD/xfs/x86_64-rhel-8.3/10%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp7/dnotify/stress-ng/60s/0x5003006

commit: 
  dfe11e96a9 ("xfs: decide if inode needs inactivation")
  7f83561097 ("xfs: deferred inode inactivation")

dfe11e96a97bf90e 7f835610979ba652213f2a41e60 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
           :6           67%           4:6     kmsg.XFS(sda1):xlog_verify_grant_tail:space>BBTOB(tail_blocks)
           :6           33%           2:6     kmsg.XFS(sdb1):xlog_verify_grant_tail:space>BBTOB(tail_blocks)
         %stddev     %change         %stddev
             \          |                \  
   1237518 ±  3%     -73.4%     328562 ±  2%  stress-ng.dnotify.ops
     20625 ±  3%     -73.9%       5386        stress-ng.dnotify.ops_per_sec
     62.07            +1.6%      63.09        stress-ng.time.elapsed_time
     62.07            +1.6%      63.09        stress-ng.time.elapsed_time.max
  49500612 ±  3%     -73.5%   13142322 ±  2%  stress-ng.time.file_system_outputs
     48525 ±  2%     -79.0%      10174 ±  3%  stress-ng.time.involuntary_context_switches
    698.50           -77.1%     160.17 ±  3%  stress-ng.time.percent_of_cpu_this_job_got
    422.99           -76.8%      98.17 ±  2%  stress-ng.time.system_time
     10.84 ±  3%     -71.3%       3.12        stress-ng.time.user_time
  13273998 ±  2%     -79.9%    2662800 ±  2%  stress-ng.time.voluntary_context_switches
     92.03            +4.5%      96.14        iostat.cpu.idle
      7.64           -53.2%       3.58 ±  8%  iostat.cpu.system
  82062533 ±  5%     -48.2%   42480684 ± 27%  cpuidle.C1.time
  10997236 ±  3%     -71.7%    3116762 ± 24%  cpuidle.C1.usage
   7041678 ± 12%     -90.8%     645474 ± 64%  cpuidle.POLL.time
   1604875 ± 17%     -96.1%      62665 ± 30%  cpuidle.POLL.usage
      0.01 ±141%      +0.1        0.07 ± 17%  mpstat.cpu.all.iowait%
      0.70 ±  2%      +0.4        1.07 ± 16%  mpstat.cpu.all.irq%
      6.31            -5.0        1.35        mpstat.cpu.all.sys%
      0.30            -0.2        0.13 ±  4%  mpstat.cpu.all.usr%
   3042003 ± 24%     -59.9%    1219129 ± 30%  numa-numastat.node0.local_node
   3062529 ± 24%     -59.2%    1248975 ± 29%  numa-numastat.node0.numa_hit
   3594498 ± 24%     -68.3%    1139998 ± 29%  numa-numastat.node1.local_node
   3660378 ± 23%     -67.3%    1196625 ± 27%  numa-numastat.node1.numa_hit
      1.75 ± 61%     -77.6%       0.39 ± 49%  perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
      0.04 ± 12%     -22.4%       0.03 ± 10%  perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      2.83 ± 62%    +241.2%       9.67 ± 39%  perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
      0.04 ± 13%     -20.5%       0.03 ±  8%  perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.38 ±109%     -77.1%       0.09 ± 65%  perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
     92.00            +4.2%      95.83        vmstat.cpu.id
     13.33 ±109%  +1.7e+05%      23035 ±  2%  vmstat.io.bo
   1158975           +89.9%    2200794        vmstat.memory.cache
      7.00           -71.4%       2.00        vmstat.procs.r
    414493 ±  2%     -76.8%      96097        vmstat.system.cs
    201970            -4.9%     192005        vmstat.system.in
      7188           -41.2%       4224 ±  7%  meminfo.Active
      6782           -43.6%       3827 ±  8%  meminfo.Active(anon)
     89028         +1160.2%    1121973 ±  2%  meminfo.KReclaimable
   2810221           +79.1%    5033113        meminfo.Memused
     89028         +1160.2%    1121973 ±  2%  meminfo.SReclaimable
    190545          +396.3%     945621 ±  2%  meminfo.SUnreclaim
     18608           -16.6%      15525 ±  2%  meminfo.Shmem
    279573          +639.6%    2067595 ±  2%  meminfo.Slab
   2918080           +72.8%    5042980        meminfo.max_used_kB
     50144 ±  5%   +1088.3%     595856 ± 36%  numa-meminfo.node0.KReclaimable
   1518744 ±  5%     +68.7%    2561639 ± 21%  numa-meminfo.node0.MemUsed
     50144 ±  5%   +1088.3%     595856 ± 36%  numa-meminfo.node0.SReclaimable
    105038 ±  5%    +377.2%     501234 ± 36%  numa-meminfo.node0.SUnreclaim
    155183 ±  3%    +607.0%    1097091 ± 36%  numa-meminfo.node0.Slab
      5720 ±  5%     -54.8%       2587 ± 20%  numa-meminfo.node1.Active
      5479 ±  4%     -55.0%       2465 ± 20%  numa-meminfo.node1.Active(anon)
     38882 ±  7%   +1255.0%     526872 ± 37%  numa-meminfo.node1.KReclaimable
   1291635 ±  6%     +91.5%    2473090 ± 20%  numa-meminfo.node1.MemUsed
     38882 ±  7%   +1255.0%     526872 ± 37%  numa-meminfo.node1.SReclaimable
     85544 ±  6%    +419.9%     444766 ± 40%  numa-meminfo.node1.SUnreclaim
    124427 ±  4%    +680.9%     971639 ± 38%  numa-meminfo.node1.Slab
   1199424 ± 30%     -62.0%     455439 ± 39%  numa-vmstat.node0.nr_dirtied
     12535 ±  5%   +1090.1%     149184 ± 36%  numa-vmstat.node0.nr_slab_reclaimable
     26271 ±  5%    +377.4%     125417 ± 36%  numa-vmstat.node0.nr_slab_unreclaimable
   2100796 ± 17%     -34.2%    1382512 ± 17%  numa-vmstat.node0.numa_hit
   2055576 ± 17%     -34.4%    1347695 ± 18%  numa-vmstat.node0.numa_local
      1369 ±  4%     -55.0%     616.17 ± 20%  numa-vmstat.node1.nr_active_anon
   1767675 ± 25%     -76.8%     409372 ± 40%  numa-vmstat.node1.nr_dirtied
      9720 ±  7%   +1257.2%     131925 ± 37%  numa-vmstat.node1.nr_slab_reclaimable
     21382 ±  6%    +420.6%     111314 ± 39%  numa-vmstat.node1.nr_slab_unreclaimable
      1369 ±  4%     -55.0%     616.17 ± 20%  numa-vmstat.node1.nr_zone_active_anon
   2583006 ± 18%     -47.5%    1355326 ± 16%  numa-vmstat.node1.numa_hit
   2375790 ± 19%     -52.1%    1137231 ± 20%  numa-vmstat.node1.numa_local
      1695           -43.6%     956.33 ±  8%  proc-vmstat.nr_active_anon
   6187166 ±  3%     -73.5%    1642672 ±  2%  proc-vmstat.nr_dirtied
      4652           -16.6%       3879 ±  2%  proc-vmstat.nr_shmem
     22256         +1161.4%     280733 ±  2%  proc-vmstat.nr_slab_reclaimable
     47647          +396.4%     236532 ±  2%  proc-vmstat.nr_slab_unreclaimable
      1695           -43.6%     956.33 ±  8%  proc-vmstat.nr_zone_active_anon
   6689446 ±  3%     -63.2%    2464182 ±  2%  proc-vmstat.numa_hit
   6602940 ±  3%     -64.0%    2377633 ±  3%  proc-vmstat.numa_local
      3549 ±  2%     -63.5%       1296 ± 20%  proc-vmstat.pgactivate
   6876740 ±  3%     -53.0%    3232013 ±  2%  proc-vmstat.pgalloc_normal
   6743259 ±  3%     -68.8%    2107101 ±  2%  proc-vmstat.pgfree
    806.50 ±117%  +1.8e+05%    1491815 ±  2%  proc-vmstat.pgpgout
      9.01 ±  7%    +102.6%      18.25 ± 85%  perf-stat.i.MPKI
 3.035e+09 ±  3%     -65.1%  1.058e+09        perf-stat.i.branch-instructions
      1.22            +1.1        2.31 ± 79%  perf-stat.i.branch-miss-rate%
  35811212 ±  2%     -52.5%   17004880 ± 22%  perf-stat.i.branch-misses
  40800963 ± 10%     -67.7%   13182099 ± 12%  perf-stat.i.cache-misses
 1.393e+08 ± 10%     -61.8%   53257508 ± 31%  perf-stat.i.cache-references
    428722 ±  2%     -76.9%      99004        perf-stat.i.context-switches
      1.64 ±  3%    +104.9%       3.37 ± 20%  perf-stat.i.cpi
 2.519e+10           -61.4%  9.729e+09 ±  7%  perf-stat.i.cpu-cycles
    140.36 ±  2%     -11.7%     123.98        perf-stat.i.cpu-migrations
    734.66 ±  8%     +50.4%       1105 ± 17%  perf-stat.i.cycles-between-cache-misses
      0.01 ± 21%      +0.2        0.17 ±108%  perf-stat.i.dTLB-load-miss-rate%
 4.277e+09 ±  3%     -66.8%  1.419e+09        perf-stat.i.dTLB-loads
      0.00 ± 15%      +0.0        0.05 ±103%  perf-stat.i.dTLB-store-miss-rate%
 2.483e+09 ±  3%     -68.6%  7.797e+08        perf-stat.i.dTLB-stores
  19790920 ±  2%     -70.8%    5771607 ±  3%  perf-stat.i.iTLB-load-misses
  23141891 ±  3%     -68.0%    7406149 ±  3%  perf-stat.i.iTLB-loads
 1.532e+10 ±  3%     -65.4%  5.294e+09        perf-stat.i.instructions
      0.62 ±  2%     -33.7%       0.41 ±  7%  perf-stat.i.ipc
      0.26           -61.3%       0.10 ±  7%  perf-stat.i.metric.GHz
    103.96 ±  3%     -66.7%      34.66        perf-stat.i.metric.M/sec
      3304 ±  2%      -2.7%       3216        perf-stat.i.minor-faults
     92.84           -19.5       73.36 ±  7%  perf-stat.i.node-load-miss-rate%
  15780181 ± 14%     -77.5%    3554472 ± 15%  perf-stat.i.node-load-misses
   1057190 ±  9%     -31.2%     726824 ±  6%  perf-stat.i.node-loads
     96.01           -25.9       70.11 ±  8%  perf-stat.i.node-store-miss-rate%
   5866625 ± 15%     -72.3%    1627188 ± 16%  perf-stat.i.node-store-misses
    102306 ± 18%    +502.6%     616466 ±  9%  perf-stat.i.node-stores
      3315 ±  2%      -2.7%       3227        perf-stat.i.page-faults
      1.18            +0.4        1.61 ± 22%  perf-stat.overall.branch-miss-rate%
      1.65 ±  3%     +11.6%       1.84 ±  7%  perf-stat.overall.cpi
    624.78 ± 10%     +19.5%     746.52 ± 12%  perf-stat.overall.cycles-between-cache-misses
      0.01 ± 22%      +0.0        0.05 ± 86%  perf-stat.overall.dTLB-load-miss-rate%
      0.00 ± 11%      +0.0        0.01 ± 80%  perf-stat.overall.dTLB-store-miss-rate%
     46.11 ±  2%      -2.3       43.81 ±  2%  perf-stat.overall.iTLB-load-miss-rate%
    774.22           +18.6%     918.08 ±  3%  perf-stat.overall.instructions-per-iTLB-miss
      0.61 ±  2%     -10.0%       0.55 ±  7%  perf-stat.overall.ipc
     93.65           -10.9       82.70 ±  3%  perf-stat.overall.node-load-miss-rate%
     98.26           -26.2       72.10 ±  6%  perf-stat.overall.node-store-miss-rate%
 2.986e+09 ±  3%     -65.0%  1.044e+09        perf-stat.ps.branch-instructions
  35265519 ±  2%     -52.4%   16769053 ± 22%  perf-stat.ps.branch-misses
  40134921 ± 10%     -67.6%   13000789 ± 11%  perf-stat.ps.cache-misses
  1.37e+08 ± 10%     -61.7%   52528482 ± 31%  perf-stat.ps.cache-references
    421720 ±  2%     -76.8%      97743        perf-stat.ps.context-switches
 2.478e+10           -61.3%  9.593e+09 ±  7%  perf-stat.ps.cpu-cycles
    138.14 ±  2%     -11.6%     122.07        perf-stat.ps.cpu-migrations
 4.208e+09 ±  3%     -66.7%    1.4e+09        perf-stat.ps.dTLB-loads
 2.443e+09 ±  3%     -68.5%  7.694e+08        perf-stat.ps.dTLB-stores
  19468460 ±  2%     -70.8%    5693812 ±  3%  perf-stat.ps.iTLB-load-misses
  22764523 ±  3%     -67.9%    7306057 ±  4%  perf-stat.ps.iTLB-loads
 1.507e+10 ±  3%     -65.4%  5.223e+09        perf-stat.ps.instructions
      3256 ±  2%      -2.8%       3166        perf-stat.ps.minor-faults
  15522002 ± 14%     -77.4%    3507715 ± 15%  perf-stat.ps.node-load-misses
   1039975 ±  9%     -31.1%     716845 ±  6%  perf-stat.ps.node-loads
   5770592 ± 14%     -72.2%    1605965 ± 15%  perf-stat.ps.node-store-misses
    100771 ± 18%    +503.9%     608590 ±  9%  perf-stat.ps.node-stores
      3267 ±  2%      -2.7%       3178        perf-stat.ps.page-faults
 9.535e+11 ±  3%     -64.8%  3.357e+11        perf-stat.total.instructions
      1189 ±  7%     +53.8%       1829 ±  7%  slabinfo.Acpi-Parse.active_objs
      1189 ±  7%     +53.8%       1829 ±  7%  slabinfo.Acpi-Parse.num_objs
    356.83 ±  6%     +28.4%     458.33 ± 11%  slabinfo.biovec-64.active_objs
    356.83 ±  6%     +28.4%     458.33 ± 11%  slabinfo.biovec-64.num_objs
      2.33 ± 80%  +22385.7%     524.67 ± 24%  slabinfo.btrfs_ordered_extent.active_objs
      2.33 ± 80%  +22392.9%     524.83 ± 24%  slabinfo.btrfs_ordered_extent.num_objs
      1573 ±  9%     +63.6%       2574 ± 20%  slabinfo.dmaengine-unmap-16.active_objs
      1573 ±  9%     +66.9%       2626 ± 19%  slabinfo.dmaengine-unmap-16.num_objs
      5275           +87.9%       9912 ±  5%  slabinfo.kmalloc-128.active_objs
    166.50           +88.8%     314.33 ±  5%  slabinfo.kmalloc-128.active_slabs
      5332           +88.9%      10075 ±  5%  slabinfo.kmalloc-128.num_objs
    166.50           +88.8%     314.33 ±  5%  slabinfo.kmalloc-128.num_slabs
     35867         +4329.5%    1588746 ±  2%  slabinfo.kmalloc-16.active_objs
    140.17         +4327.5%       6205 ±  2%  slabinfo.kmalloc-16.active_slabs
     35882         +4327.8%    1588803 ±  2%  slabinfo.kmalloc-16.num_objs
    140.17         +4327.5%       6205 ±  2%  slabinfo.kmalloc-16.num_slabs
     71716           +31.4%      94239        slabinfo.kmalloc-32.active_objs
    561.17 ±  2%     +31.1%     735.83        slabinfo.kmalloc-32.active_slabs
     71885 ±  2%     +31.1%      94259        slabinfo.kmalloc-32.num_objs
    561.17 ±  2%     +31.1%     735.83        slabinfo.kmalloc-32.num_slabs
     11784 ±  2%   +8994.3%    1071666 ±  3%  slabinfo.kmalloc-512.active_objs
    368.83 ±  2%   +9339.1%      34814 ±  2%  slabinfo.kmalloc-512.active_slabs
     11819 ±  2%   +9325.8%    1114079 ±  2%  slabinfo.kmalloc-512.num_objs
    368.83 ±  2%   +9339.1%      34814 ±  2%  slabinfo.kmalloc-512.num_slabs
    821.33         +2362.4%      20224 ±  3%  slabinfo.kmalloc-8k.active_objs
    204.83         +2520.0%       5366 ±  3%  slabinfo.kmalloc-8k.active_slabs
    821.33         +2513.8%      21468 ±  3%  slabinfo.kmalloc-8k.num_objs
    204.83         +2520.0%       5366 ±  3%  slabinfo.kmalloc-8k.num_slabs
    449.67 ±  4%   +6105.7%      27905 ±  2%  slabinfo.kmalloc-rcl-256.active_objs
     13.67 ±  5%   +6279.3%     871.83 ±  2%  slabinfo.kmalloc-rcl-256.active_slabs
    449.67 ±  4%   +6105.8%      27905 ±  2%  slabinfo.kmalloc-rcl-256.num_objs
     13.67 ±  5%   +6279.3%     871.83 ±  2%  slabinfo.kmalloc-rcl-256.num_slabs
      4505 ± 12%    +258.4%      16147 ±  3%  slabinfo.numa_policy.active_objs
     72.17 ± 12%    +267.2%     265.00 ±  3%  slabinfo.numa_policy.active_slabs
      4505 ± 12%    +265.3%      16461 ±  3%  slabinfo.numa_policy.num_objs
     72.17 ± 12%    +267.2%     265.00 ±  3%  slabinfo.numa_policy.num_slabs
     16951           +76.2%      29869        slabinfo.radix_tree_node.active_objs
    302.33           +76.3%     533.00        slabinfo.radix_tree_node.active_slabs
     16951           +76.2%      29876        slabinfo.radix_tree_node.num_objs
    302.33           +76.3%     533.00        slabinfo.radix_tree_node.num_slabs
    257.00 ±  7%  +10813.9%      28048 ±  2%  slabinfo.xfs_buf.active_objs
      5.33 ±  8%  +12421.9%     667.83 ±  2%  slabinfo.xfs_buf.active_slabs
    257.00 ±  7%  +10818.9%      28061 ±  2%  slabinfo.xfs_buf.num_objs
      5.33 ±  8%  +12421.9%     667.83 ±  2%  slabinfo.xfs_buf.num_slabs
      1.00        +53550.0%     536.50 ± 24%  slabinfo.xfs_efi_item.active_objs
      1.00        +53566.7%     536.67 ± 24%  slabinfo.xfs_efi_item.num_objs
    129.00         +5911.1%       7754 ± 13%  slabinfo.xfs_icr.active_objs
      2.00         +8366.7%     169.33 ± 13%  slabinfo.xfs_icr.active_slabs
    129.00         +5964.1%       7822 ± 13%  slabinfo.xfs_icr.num_objs
      2.00         +8366.7%     169.33 ± 13%  slabinfo.xfs_icr.num_slabs
    354.00        +2.4e+05%     848335 ±  2%  slabinfo.xfs_ili.active_objs
      8.00        +2.5e+05%      20210 ±  2%  slabinfo.xfs_ili.active_slabs
    354.00        +2.4e+05%     848856 ±  2%  slabinfo.xfs_ili.num_objs
      8.00        +2.5e+05%      20210 ±  2%  slabinfo.xfs_ili.num_slabs
    302.00        +2.8e+05%     847804 ±  2%  slabinfo.xfs_inode.active_objs
      9.00        +2.9e+05%      26509 ±  2%  slabinfo.xfs_inode.active_slabs
    302.00        +2.8e+05%     848315 ±  2%  slabinfo.xfs_inode.num_objs
      9.00        +2.9e+05%      26509 ±  2%  slabinfo.xfs_inode.num_slabs
      1554          +970.0%      16634 ±  2%  softirqs.BLOCK
     21120 ± 11%     -44.0%      11836 ± 13%  softirqs.CPU0.RCU
     19237 ± 11%     -51.3%       9370 ± 32%  softirqs.CPU1.RCU
     15255 ± 25%     -54.6%       6926 ± 19%  softirqs.CPU10.RCU
     15629 ± 26%     -56.6%       6788 ± 31%  softirqs.CPU11.RCU
     14268 ± 29%     -54.6%       6473 ± 24%  softirqs.CPU13.RCU
     14102 ± 18%     -50.1%       7040 ± 21%  softirqs.CPU14.RCU
     13854 ± 32%     -53.5%       6444 ± 20%  softirqs.CPU15.RCU
     14005 ± 21%     -48.4%       7231 ± 17%  softirqs.CPU16.RCU
     10299 ±  7%     -13.3%       8929 ± 11%  softirqs.CPU16.SCHED
     15150 ± 16%     -59.9%       6075 ± 17%  softirqs.CPU17.RCU
     10061 ±  4%     -12.6%       8795 ± 13%  softirqs.CPU17.SCHED
     16886 ± 23%     -61.1%       6563 ± 12%  softirqs.CPU18.RCU
     16459 ±  6%     -56.0%       7238 ± 30%  softirqs.CPU19.RCU
     10118 ±  4%      -7.9%       9318 ±  4%  softirqs.CPU19.SCHED
     19097 ± 18%     -57.4%       8137 ± 21%  softirqs.CPU2.RCU
     14226 ± 16%     -41.9%       8270 ± 24%  softirqs.CPU20.RCU
     17043 ± 25%     -62.4%       6407 ± 25%  softirqs.CPU21.RCU
     17148 ± 11%     -63.4%       6280 ± 14%  softirqs.CPU22.RCU
     17929 ± 21%     -61.4%       6921 ± 34%  softirqs.CPU23.RCU
     10298 ±  3%      -8.1%       9467 ±  3%  softirqs.CPU23.SCHED
     20526 ± 19%     -58.7%       8467 ± 29%  softirqs.CPU24.RCU
     20176 ± 13%     -67.8%       6494 ± 16%  softirqs.CPU25.RCU
     10744 ±  4%     -11.6%       9496 ±  3%  softirqs.CPU25.SCHED
     18899 ±  6%     -64.4%       6732 ± 12%  softirqs.CPU26.RCU
     10509 ±  2%     -14.2%       9018 ± 13%  softirqs.CPU26.SCHED
     21195 ± 14%     -64.0%       7637 ± 19%  softirqs.CPU27.RCU
     11101 ±  8%     -12.1%       9756 ±  4%  softirqs.CPU27.SCHED
     19348 ± 24%     -64.5%       6874 ± 20%  softirqs.CPU28.RCU
     17513 ± 31%     -62.1%       6633 ± 19%  softirqs.CPU29.RCU
     18595 ± 20%     -59.7%       7495 ± 29%  softirqs.CPU3.RCU
     10884 ±  8%     -26.1%       8040 ± 19%  softirqs.CPU3.SCHED
     16140 ± 12%     -55.7%       7145 ± 27%  softirqs.CPU30.RCU
     18129 ± 29%     -65.0%       6338 ± 14%  softirqs.CPU31.RCU
     10353 ±  6%      -8.5%       9478 ±  3%  softirqs.CPU31.SCHED
     19563 ± 31%     -66.1%       6638 ±  9%  softirqs.CPU32.RCU
     18982 ± 25%     -63.3%       6959 ± 17%  softirqs.CPU33.RCU
     18805 ± 27%     -63.9%       6795 ± 26%  softirqs.CPU34.RCU
     16018 ± 34%     -63.9%       5777 ± 12%  softirqs.CPU35.RCU
     15396 ± 19%     -54.5%       7011 ± 21%  softirqs.CPU36.RCU
     18564 ± 23%     -65.4%       6424 ± 18%  softirqs.CPU37.RCU
     18454 ± 22%     -68.7%       5776 ± 10%  softirqs.CPU38.RCU
     19579 ± 20%     -61.5%       7530 ± 25%  softirqs.CPU39.RCU
     16731 ± 26%     -53.1%       7852 ± 40%  softirqs.CPU4.RCU
     16573 ± 12%     -59.5%       6712 ± 29%  softirqs.CPU40.RCU
     18052 ± 21%     -56.2%       7910 ± 28%  softirqs.CPU41.RCU
     18694 ± 13%     -65.3%       6487 ± 19%  softirqs.CPU42.RCU
     16325 ±  9%     -53.6%       7573 ± 11%  softirqs.CPU43.RCU
     18083 ± 15%     -64.1%       6483 ± 23%  softirqs.CPU44.RCU
     10758 ±  6%     -16.4%       8997 ± 10%  softirqs.CPU44.SCHED
     20808 ± 19%     -67.8%       6691 ± 26%  softirqs.CPU45.RCU
     10725 ±  5%     -15.4%       9074 ± 11%  softirqs.CPU45.SCHED
     17044 ± 22%     -61.0%       6649 ± 22%  softirqs.CPU46.RCU
     17817 ± 16%     -66.9%       5894 ± 17%  softirqs.CPU47.RCU
     14312 ± 30%     -60.4%       5670 ± 27%  softirqs.CPU48.RCU
     14880 ±  9%     -58.1%       6241 ± 21%  softirqs.CPU49.RCU
     17117 ± 33%     -52.5%       8129 ± 24%  softirqs.CPU5.RCU
     14530 ± 29%     -51.5%       7043 ± 20%  softirqs.CPU50.RCU
     12832 ± 29%     -46.2%       6908 ±  9%  softirqs.CPU51.RCU
     16355 ± 18%     -59.8%       6580 ± 22%  softirqs.CPU52.RCU
     14136 ± 24%     -58.2%       5909 ± 24%  softirqs.CPU53.RCU
     16432 ± 15%     -62.3%       6195 ± 26%  softirqs.CPU54.RCU
     14701 ± 19%     -55.5%       6539 ± 16%  softirqs.CPU55.RCU
     16125 ± 43%     -63.6%       5874 ± 20%  softirqs.CPU56.RCU
     14335 ± 21%     -53.5%       6661 ± 15%  softirqs.CPU57.RCU
     14374 ± 15%     -56.5%       6257 ± 27%  softirqs.CPU58.RCU
     16381 ± 30%     -57.0%       7038 ± 27%  softirqs.CPU59.RCU
     15884 ± 17%     -52.9%       7484 ± 28%  softirqs.CPU6.RCU
     15020 ± 26%     -55.7%       6649 ± 30%  softirqs.CPU60.RCU
     14556 ± 29%     -60.7%       5722 ± 26%  softirqs.CPU61.RCU
     15214 ± 16%     -61.7%       5825 ± 24%  softirqs.CPU62.RCU
     15253 ± 29%     -61.2%       5919 ± 18%  softirqs.CPU63.RCU
     16039 ± 21%     -62.8%       5962 ± 26%  softirqs.CPU64.RCU
     15525 ± 28%     -61.9%       5915 ± 21%  softirqs.CPU65.RCU
     15185 ± 22%     -64.4%       5399 ± 16%  softirqs.CPU66.RCU
     16461 ± 13%     -66.5%       5521 ± 22%  softirqs.CPU67.RCU
     10286 ±  7%     -23.8%       7833 ± 21%  softirqs.CPU67.SCHED
     14137 ± 23%     -59.7%       5701 ± 14%  softirqs.CPU68.RCU
     14892 ± 20%     -57.2%       6368 ± 17%  softirqs.CPU69.RCU
     17032 ± 23%     -61.7%       6527 ± 20%  softirqs.CPU7.RCU
     16497 ± 24%     -64.8%       5809 ± 17%  softirqs.CPU70.RCU
     14588 ± 26%     -57.8%       6162 ± 31%  softirqs.CPU71.RCU
     20376 ± 22%     -69.7%       6176 ± 13%  softirqs.CPU72.RCU
     16224 ± 21%     -64.1%       5829 ± 25%  softirqs.CPU73.RCU
     16511 ± 18%     -64.9%       5796 ± 11%  softirqs.CPU74.RCU
     17407 ± 25%     -63.0%       6443 ± 22%  softirqs.CPU75.RCU
     16803 ± 24%     -55.0%       7556 ± 34%  softirqs.CPU76.RCU
     17490 ± 24%     -66.2%       5911 ± 16%  softirqs.CPU77.RCU
     18734 ± 23%     -68.1%       5973 ± 18%  softirqs.CPU78.RCU
     17507 ± 24%     -61.0%       6829 ± 24%  softirqs.CPU79.RCU
     14591 ± 19%     -52.3%       6954 ± 19%  softirqs.CPU8.RCU
     17466 ± 40%     -63.4%       6391 ± 31%  softirqs.CPU80.RCU
     16442 ± 20%     -58.2%       6874 ± 25%  softirqs.CPU81.RCU
     15519 ± 18%     -61.0%       6046 ± 26%  softirqs.CPU82.RCU
     18356 ± 16%     -66.9%       6078 ± 17%  softirqs.CPU83.RCU
     17332 ± 19%     -64.6%       6129 ± 27%  softirqs.CPU84.RCU
     15382 ± 24%     -57.6%       6523 ± 26%  softirqs.CPU85.RCU
     12953 ± 12%     -49.8%       6507 ± 26%  softirqs.CPU86.RCU
     17891 ± 29%     -67.7%       5784 ± 18%  softirqs.CPU87.RCU
     15408 ± 23%     -64.7%       5445 ± 13%  softirqs.CPU88.RCU
     17202 ± 15%     -61.7%       6580 ± 35%  softirqs.CPU89.RCU
     14401 ± 24%     -52.7%       6807 ± 21%  softirqs.CPU9.RCU
     14726 ± 24%     -58.2%       6154 ± 17%  softirqs.CPU90.RCU
     17034 ± 21%     -62.1%       6462 ± 45%  softirqs.CPU91.RCU
     18149 ± 14%     -65.8%       6211 ± 24%  softirqs.CPU92.RCU
     10385 ±  5%      -7.0%       9661 ±  4%  softirqs.CPU92.SCHED
     15547 ± 16%     -59.7%       6260 ± 18%  softirqs.CPU93.RCU
     16675 ± 23%     -64.3%       5946 ± 20%  softirqs.CPU94.RCU
     15994 ± 19%     -64.1%       5744 ± 24%  softirqs.CPU95.RCU
   1595836           -60.0%     637995 ±  5%  softirqs.RCU
    979291 ±  2%      -8.7%     893912 ±  2%  softirqs.SCHED
     32415           -46.7%      17273 ±  5%  softirqs.TIMER
     97.50 ± 52%  +30937.3%      30261 ±  2%  interrupts.315:PCI-MSI.376832-edge.ahci[0000:00:17.0]
    711540 ±  9%     -80.3%     140124 ±  3%  interrupts.CAL:Function_call_interrupts
      6893 ± 32%     -70.4%       2041 ± 54%  interrupts.CPU0.CAL:Function_call_interrupts
      8805 ± 30%     -70.2%       2621 ± 48%  interrupts.CPU1.CAL:Function_call_interrupts
    854.00 ± 44%     -71.9%     239.67 ± 37%  interrupts.CPU10.NMI:Non-maskable_interrupts
    854.00 ± 44%     -71.9%     239.67 ± 37%  interrupts.CPU10.PMI:Performance_monitoring_interrupts
    258.67 ± 59%     -58.1%     108.50 ± 64%  interrupts.CPU10.RES:Rescheduling_interrupts
      5312 ± 68%     -76.3%       1256 ± 50%  interrupts.CPU13.CAL:Function_call_interrupts
    769.67 ± 43%     -65.5%     265.83 ± 41%  interrupts.CPU13.NMI:Non-maskable_interrupts
    769.67 ± 43%     -65.5%     265.83 ± 41%  interrupts.CPU13.PMI:Performance_monitoring_interrupts
      5349 ± 39%     -72.4%       1475 ± 29%  interrupts.CPU14.CAL:Function_call_interrupts
    897.17 ± 35%     -71.4%     257.00 ± 71%  interrupts.CPU14.NMI:Non-maskable_interrupts
    897.17 ± 35%     -71.4%     257.00 ± 71%  interrupts.CPU14.PMI:Performance_monitoring_interrupts
      5508 ± 63%     -70.7%       1612 ± 33%  interrupts.CPU16.CAL:Function_call_interrupts
      5513 ± 47%     -78.9%       1164 ± 45%  interrupts.CPU17.CAL:Function_call_interrupts
    917.00 ± 66%     -79.8%     185.67 ± 37%  interrupts.CPU17.NMI:Non-maskable_interrupts
    917.00 ± 66%     -79.8%     185.67 ± 37%  interrupts.CPU17.PMI:Performance_monitoring_interrupts
      6835 ± 65%     -81.6%       1259 ± 31%  interrupts.CPU18.CAL:Function_call_interrupts
      1211 ± 47%     -76.1%     289.50 ± 62%  interrupts.CPU18.NMI:Non-maskable_interrupts
      1211 ± 47%     -76.1%     289.50 ± 62%  interrupts.CPU18.PMI:Performance_monitoring_interrupts
      6187 ± 49%     -72.0%       1730 ± 66%  interrupts.CPU19.CAL:Function_call_interrupts
      8204 ± 36%     -68.9%       2549 ± 48%  interrupts.CPU2.CAL:Function_call_interrupts
    320.50 ± 50%     -54.4%     146.00 ± 68%  interrupts.CPU2.RES:Rescheduling_interrupts
      5016 ± 32%     -59.6%       2028 ± 51%  interrupts.CPU20.CAL:Function_call_interrupts
      1072 ± 48%     -60.6%     422.50 ± 71%  interrupts.CPU20.NMI:Non-maskable_interrupts
      1072 ± 48%     -60.6%     422.50 ± 71%  interrupts.CPU20.PMI:Performance_monitoring_interrupts
      6363 ± 48%     -79.0%       1334 ± 48%  interrupts.CPU21.CAL:Function_call_interrupts
      6133 ± 32%     -78.7%       1305 ± 40%  interrupts.CPU22.CAL:Function_call_interrupts
      1064 ± 55%     -66.9%     352.33 ± 49%  interrupts.CPU22.NMI:Non-maskable_interrupts
      1064 ± 55%     -66.9%     352.33 ± 49%  interrupts.CPU22.PMI:Performance_monitoring_interrupts
      7479 ± 56%     -81.5%       1384 ± 35%  interrupts.CPU23.CAL:Function_call_interrupts
    744.50 ± 80%     -78.4%     160.67 ± 46%  interrupts.CPU23.NMI:Non-maskable_interrupts
    744.50 ± 80%     -78.4%     160.67 ± 46%  interrupts.CPU23.PMI:Performance_monitoring_interrupts
     11389 ± 45%     -82.7%       1970 ± 43%  interrupts.CPU24.CAL:Function_call_interrupts
      1234 ± 54%     -62.2%     467.17 ± 97%  interrupts.CPU24.NMI:Non-maskable_interrupts
      1234 ± 54%     -62.2%     467.17 ± 97%  interrupts.CPU24.PMI:Performance_monitoring_interrupts
     10263 ± 35%     -88.4%       1191 ± 30%  interrupts.CPU25.CAL:Function_call_interrupts
      9960 ± 31%     -87.0%       1292 ± 22%  interrupts.CPU26.CAL:Function_call_interrupts
    231.67 ± 34%     -64.0%      83.50 ±102%  interrupts.CPU26.RES:Rescheduling_interrupts
     13470 ± 51%     -85.7%       1925 ± 40%  interrupts.CPU27.CAL:Function_call_interrupts
      9836 ± 55%     -86.7%       1303 ± 36%  interrupts.CPU28.CAL:Function_call_interrupts
    272.17 ± 52%     -72.6%      74.67 ± 52%  interrupts.CPU28.RES:Rescheduling_interrupts
      8560 ± 62%     -85.7%       1227 ± 42%  interrupts.CPU29.CAL:Function_call_interrupts
      8218 ± 46%     -75.0%       2053 ± 38%  interrupts.CPU3.CAL:Function_call_interrupts
      6712 ± 31%     -72.0%       1877 ± 48%  interrupts.CPU30.CAL:Function_call_interrupts
      9525 ± 55%     -87.9%       1149 ± 41%  interrupts.CPU31.CAL:Function_call_interrupts
    189.33 ± 71%     -68.4%      59.83 ±103%  interrupts.CPU31.RES:Rescheduling_interrupts
     11122 ± 57%     -90.2%       1092 ± 31%  interrupts.CPU32.CAL:Function_call_interrupts
    207.00 ± 47%     -66.3%      69.67 ± 83%  interrupts.CPU32.RES:Rescheduling_interrupts
      9576 ± 51%     -85.0%       1439 ± 40%  interrupts.CPU33.CAL:Function_call_interrupts
    269.00 ± 48%     -71.2%      77.50 ± 66%  interrupts.CPU33.RES:Rescheduling_interrupts
      9270 ± 57%     -88.1%       1101 ± 39%  interrupts.CPU34.CAL:Function_call_interrupts
      6919 ± 80%     -85.9%     978.00 ± 24%  interrupts.CPU35.CAL:Function_call_interrupts
      1037 ± 81%     -75.3%     255.83 ± 68%  interrupts.CPU35.NMI:Non-maskable_interrupts
      1037 ± 81%     -75.3%     255.83 ± 68%  interrupts.CPU35.PMI:Performance_monitoring_interrupts
      6139 ± 49%     -74.8%       1545 ± 59%  interrupts.CPU36.CAL:Function_call_interrupts
    211.67 ± 33%     -67.6%      68.67 ± 99%  interrupts.CPU36.RES:Rescheduling_interrupts
      9270 ± 43%     -86.8%       1225 ± 41%  interrupts.CPU37.CAL:Function_call_interrupts
    882.83 ± 33%     -72.9%     239.00 ± 63%  interrupts.CPU37.NMI:Non-maskable_interrupts
    882.83 ± 33%     -72.9%     239.00 ± 63%  interrupts.CPU37.PMI:Performance_monitoring_interrupts
      9776 ± 33%     -88.6%       1114 ± 38%  interrupts.CPU38.CAL:Function_call_interrupts
      1174 ± 37%     -78.1%     256.67 ± 98%  interrupts.CPU38.NMI:Non-maskable_interrupts
      1174 ± 37%     -78.1%     256.67 ± 98%  interrupts.CPU38.PMI:Performance_monitoring_interrupts
    298.00 ± 65%     -82.9%      50.83 ± 49%  interrupts.CPU38.RES:Rescheduling_interrupts
     10075 ± 43%     -87.5%       1255 ± 47%  interrupts.CPU39.CAL:Function_call_interrupts
    262.33 ± 37%     -61.5%     101.00 ± 89%  interrupts.CPU39.RES:Rescheduling_interrupts
      8423 ± 55%     -72.9%       2281 ± 73%  interrupts.CPU4.CAL:Function_call_interrupts
      6552 ± 33%     -75.7%       1594 ± 44%  interrupts.CPU40.CAL:Function_call_interrupts
      1047 ± 67%     -85.5%     151.33 ± 39%  interrupts.CPU40.NMI:Non-maskable_interrupts
      1047 ± 67%     -85.5%     151.33 ± 39%  interrupts.CPU40.PMI:Performance_monitoring_interrupts
    217.00 ± 51%     -71.4%      62.17 ±100%  interrupts.CPU40.RES:Rescheduling_interrupts
      8027 ± 44%     -73.1%       2159 ± 81%  interrupts.CPU41.CAL:Function_call_interrupts
      1055 ± 37%     -75.6%     257.17 ± 51%  interrupts.CPU41.NMI:Non-maskable_interrupts
      1055 ± 37%     -75.6%     257.17 ± 51%  interrupts.CPU41.PMI:Performance_monitoring_interrupts
    217.00 ± 54%     -63.1%      80.00 ±102%  interrupts.CPU41.RES:Rescheduling_interrupts
      9163 ± 32%     -83.3%       1530 ± 44%  interrupts.CPU42.CAL:Function_call_interrupts
    240.33 ± 53%     -61.3%      93.00 ± 83%  interrupts.CPU42.RES:Rescheduling_interrupts
      7210 ± 33%     -76.4%       1700 ± 37%  interrupts.CPU43.CAL:Function_call_interrupts
     10002 ± 50%     -87.6%       1243 ± 57%  interrupts.CPU44.CAL:Function_call_interrupts
      1466 ± 62%     -85.9%     207.17 ± 46%  interrupts.CPU44.NMI:Non-maskable_interrupts
      1466 ± 62%     -85.9%     207.17 ± 46%  interrupts.CPU44.PMI:Performance_monitoring_interrupts
    177.33 ± 47%     -66.2%      60.00 ± 86%  interrupts.CPU44.RES:Rescheduling_interrupts
     10433 ± 39%     -85.9%       1467 ± 50%  interrupts.CPU45.CAL:Function_call_interrupts
      1436 ± 47%     -82.5%     251.67 ±127%  interrupts.CPU45.NMI:Non-maskable_interrupts
      1436 ± 47%     -82.5%     251.67 ±127%  interrupts.CPU45.PMI:Performance_monitoring_interrupts
    292.33 ± 54%     -83.8%      47.50 ± 81%  interrupts.CPU45.RES:Rescheduling_interrupts
      7492 ± 48%     -81.9%       1359 ± 51%  interrupts.CPU46.CAL:Function_call_interrupts
    199.83 ± 42%     -78.3%      43.33 ± 95%  interrupts.CPU46.RES:Rescheduling_interrupts
      8400 ± 38%     -84.9%       1265 ± 33%  interrupts.CPU47.CAL:Function_call_interrupts
    234.83 ± 66%     -89.9%      23.83 ± 48%  interrupts.CPU47.RES:Rescheduling_interrupts
      6680 ± 64%     -82.0%       1203 ± 36%  interrupts.CPU48.CAL:Function_call_interrupts
    138.00 ± 59%     -61.4%      53.33 ± 62%  interrupts.CPU48.RES:Rescheduling_interrupts
      5466 ± 44%     -76.0%       1310 ± 41%  interrupts.CPU49.CAL:Function_call_interrupts
      5247 ± 49%     -63.8%       1900 ± 31%  interrupts.CPU50.CAL:Function_call_interrupts
      3954 ± 50%     -58.8%       1628 ± 29%  interrupts.CPU51.CAL:Function_call_interrupts
      7995 ± 52%     -81.6%       1471 ± 28%  interrupts.CPU52.CAL:Function_call_interrupts
      1120 ± 48%     -70.4%     332.00 ± 62%  interrupts.CPU52.NMI:Non-maskable_interrupts
      1120 ± 48%     -70.4%     332.00 ± 62%  interrupts.CPU52.PMI:Performance_monitoring_interrupts
    237.17 ± 49%     -65.8%      81.17 ± 92%  interrupts.CPU52.RES:Rescheduling_interrupts
      5449 ± 48%     -76.7%       1267 ± 46%  interrupts.CPU53.CAL:Function_call_interrupts
      1126 ± 72%     -72.7%     307.50 ± 42%  interrupts.CPU53.NMI:Non-maskable_interrupts
      1126 ± 72%     -72.7%     307.50 ± 42%  interrupts.CPU53.PMI:Performance_monitoring_interrupts
      6957 ± 39%     -80.0%       1389 ± 53%  interrupts.CPU54.CAL:Function_call_interrupts
      5747 ± 48%     -75.1%       1429 ± 35%  interrupts.CPU55.CAL:Function_call_interrupts
    825.00 ± 70%     -73.0%     223.00 ± 59%  interrupts.CPU55.NMI:Non-maskable_interrupts
    825.00 ± 70%     -73.0%     223.00 ± 59%  interrupts.CPU55.PMI:Performance_monitoring_interrupts
      7399 ± 83%     -84.1%       1174 ± 43%  interrupts.CPU56.CAL:Function_call_interrupts
      5267 ± 56%     -74.9%       1321 ± 34%  interrupts.CPU57.CAL:Function_call_interrupts
    208.83 ± 31%     -61.5%      80.50 ± 57%  interrupts.CPU57.RES:Rescheduling_interrupts
      5624 ± 34%     -74.6%       1430 ± 41%  interrupts.CPU58.CAL:Function_call_interrupts
    888.17 ± 33%     -78.6%     190.50 ± 42%  interrupts.CPU58.NMI:Non-maskable_interrupts
    888.17 ± 33%     -78.6%     190.50 ± 42%  interrupts.CPU58.PMI:Performance_monitoring_interrupts
    208.17 ± 71%     -63.4%      76.17 ± 39%  interrupts.CPU58.RES:Rescheduling_interrupts
      7364 ± 65%     -77.1%       1689 ± 55%  interrupts.CPU59.CAL:Function_call_interrupts
      5509 ± 24%     -68.9%       1714 ± 40%  interrupts.CPU6.CAL:Function_call_interrupts
      6142 ± 60%     -72.8%       1670 ± 45%  interrupts.CPU60.CAL:Function_call_interrupts
      5785 ± 63%     -68.1%       1844 ± 72%  interrupts.CPU61.CAL:Function_call_interrupts
      5382 ± 26%     -81.6%     987.67 ± 28%  interrupts.CPU62.CAL:Function_call_interrupts
      1039 ± 41%     -75.8%     251.83 ± 35%  interrupts.CPU62.NMI:Non-maskable_interrupts
      1039 ± 41%     -75.8%     251.83 ± 35%  interrupts.CPU62.PMI:Performance_monitoring_interrupts
      7007 ± 56%     -82.2%       1247 ± 31%  interrupts.CPU63.CAL:Function_call_interrupts
    216.50 ± 65%     -77.8%      48.17 ± 69%  interrupts.CPU63.RES:Rescheduling_interrupts
      5893 ± 48%     -78.3%       1278 ± 60%  interrupts.CPU64.CAL:Function_call_interrupts
    210.17 ± 45%     -73.9%      54.83 ± 62%  interrupts.CPU64.RES:Rescheduling_interrupts
      6246 ± 87%     -80.9%       1190 ± 44%  interrupts.CPU65.CAL:Function_call_interrupts
      1025 ± 57%     -80.8%     197.00 ± 26%  interrupts.CPU65.NMI:Non-maskable_interrupts
      1025 ± 57%     -80.8%     197.00 ± 26%  interrupts.CPU65.PMI:Performance_monitoring_interrupts
      5079 ± 46%     -81.0%     965.67 ± 29%  interrupts.CPU66.CAL:Function_call_interrupts
    946.17 ± 17%     -76.3%     224.50 ± 58%  interrupts.CPU66.NMI:Non-maskable_interrupts
    946.17 ± 17%     -76.3%     224.50 ± 58%  interrupts.CPU66.PMI:Performance_monitoring_interrupts
      7107 ± 50%     -85.6%       1026 ± 47%  interrupts.CPU67.CAL:Function_call_interrupts
    827.50 ± 58%     -67.2%     271.83 ± 66%  interrupts.CPU67.NMI:Non-maskable_interrupts
    827.50 ± 58%     -67.2%     271.83 ± 66%  interrupts.CPU67.PMI:Performance_monitoring_interrupts
    880.50 ± 43%     -70.7%     257.83 ± 36%  interrupts.CPU68.NMI:Non-maskable_interrupts
    880.50 ± 43%     -70.7%     257.83 ± 36%  interrupts.CPU68.PMI:Performance_monitoring_interrupts
    564.50 ± 73%     -65.8%     192.83 ± 36%  interrupts.CPU69.NMI:Non-maskable_interrupts
    564.50 ± 73%     -65.8%     192.83 ± 36%  interrupts.CPU69.PMI:Performance_monitoring_interrupts
      7747 ± 57%     -78.8%       1644 ± 38%  interrupts.CPU7.CAL:Function_call_interrupts
      6315 ± 44%     -83.3%       1054 ± 37%  interrupts.CPU70.CAL:Function_call_interrupts
      1262 ± 46%     -71.1%     364.67 ± 76%  interrupts.CPU70.NMI:Non-maskable_interrupts
      1262 ± 46%     -71.1%     364.67 ± 76%  interrupts.CPU70.PMI:Performance_monitoring_interrupts
    219.33 ± 84%     -74.2%      56.67 ± 97%  interrupts.CPU70.RES:Rescheduling_interrupts
      5991 ± 59%     -81.9%       1082 ± 58%  interrupts.CPU71.CAL:Function_call_interrupts
    688.17 ± 47%     -78.1%     151.00 ± 41%  interrupts.CPU71.NMI:Non-maskable_interrupts
    688.17 ± 47%     -78.1%     151.00 ± 41%  interrupts.CPU71.PMI:Performance_monitoring_interrupts
     11770 ± 46%     -90.7%       1089 ± 31%  interrupts.CPU72.CAL:Function_call_interrupts
      1531 ± 44%     -85.6%     220.67 ± 81%  interrupts.CPU72.NMI:Non-maskable_interrupts
      1531 ± 44%     -85.6%     220.67 ± 81%  interrupts.CPU72.PMI:Performance_monitoring_interrupts
    193.17 ± 39%     -76.3%      45.83 ± 81%  interrupts.CPU72.RES:Rescheduling_interrupts
      6517 ± 45%     -82.2%       1157 ± 71%  interrupts.CPU73.CAL:Function_call_interrupts
      7510 ± 37%     -76.5%       1763 ± 83%  interrupts.CPU74.CAL:Function_call_interrupts
      8386 ± 52%     -82.4%       1474 ± 36%  interrupts.CPU75.CAL:Function_call_interrupts
      8057 ± 51%     -76.9%       1864 ± 75%  interrupts.CPU76.CAL:Function_call_interrupts
      8828 ± 55%     -87.8%       1076 ± 30%  interrupts.CPU77.CAL:Function_call_interrupts
    186.33 ± 33%     -74.6%      47.33 ± 74%  interrupts.CPU77.RES:Rescheduling_interrupts
     10049 ± 53%     -80.2%       1985 ± 95%  interrupts.CPU78.CAL:Function_call_interrupts
    176.67 ± 31%     -72.5%      48.67 ± 51%  interrupts.CPU78.RES:Rescheduling_interrupts
      7695 ± 52%     -83.1%       1297 ± 37%  interrupts.CPU79.CAL:Function_call_interrupts
      1178 ± 49%     -73.6%     311.50 ± 86%  interrupts.CPU8.NMI:Non-maskable_interrupts
      1178 ± 49%     -73.6%     311.50 ± 86%  interrupts.CPU8.PMI:Performance_monitoring_interrupts
      8394 ± 80%     -83.8%       1359 ± 76%  interrupts.CPU80.CAL:Function_call_interrupts
      8136 ± 48%     -82.5%       1427 ± 42%  interrupts.CPU81.CAL:Function_call_interrupts
      6688 ± 41%     -81.1%       1263 ± 36%  interrupts.CPU82.CAL:Function_call_interrupts
      9699 ± 45%     -90.1%     960.83 ± 29%  interrupts.CPU83.CAL:Function_call_interrupts
    900.67 ± 56%     -76.7%     210.00 ± 29%  interrupts.CPU83.NMI:Non-maskable_interrupts
    900.67 ± 56%     -76.7%     210.00 ± 29%  interrupts.CPU83.PMI:Performance_monitoring_interrupts
    198.17 ± 78%     -74.9%      49.83 ± 80%  interrupts.CPU83.RES:Rescheduling_interrupts
      9055 ± 46%     -88.0%       1086 ± 55%  interrupts.CPU84.CAL:Function_call_interrupts
      6343 ± 48%     -76.5%       1491 ± 54%  interrupts.CPU85.CAL:Function_call_interrupts
      4596 ± 37%     -71.4%       1312 ± 52%  interrupts.CPU86.CAL:Function_call_interrupts
      9733 ± 59%     -87.3%       1239 ± 66%  interrupts.CPU87.CAL:Function_call_interrupts
    753.00 ± 42%     -77.4%     170.17 ± 59%  interrupts.CPU87.NMI:Non-maskable_interrupts
    753.00 ± 42%     -77.4%     170.17 ± 59%  interrupts.CPU87.PMI:Performance_monitoring_interrupts
    193.67 ± 59%     -78.1%      42.33 ±112%  interrupts.CPU87.RES:Rescheduling_interrupts
      7232 ± 52%     -85.7%       1037 ± 28%  interrupts.CPU88.CAL:Function_call_interrupts
    974.67 ± 46%     -80.8%     186.83 ± 68%  interrupts.CPU88.NMI:Non-maskable_interrupts
    974.67 ± 46%     -80.8%     186.83 ± 68%  interrupts.CPU88.PMI:Performance_monitoring_interrupts
      7769 ± 36%     -80.2%       1538 ± 76%  interrupts.CPU89.CAL:Function_call_interrupts
      5783 ± 45%     -77.7%       1290 ± 19%  interrupts.CPU90.CAL:Function_call_interrupts
    815.17 ± 45%     -60.2%     324.50 ± 62%  interrupts.CPU90.NMI:Non-maskable_interrupts
    815.17 ± 45%     -60.2%     324.50 ± 62%  interrupts.CPU90.PMI:Performance_monitoring_interrupts
      8128 ± 55%     -80.8%       1562 ± 84%  interrupts.CPU91.CAL:Function_call_interrupts
    212.00 ± 50%     -74.6%      53.83 ±102%  interrupts.CPU91.RES:Rescheduling_interrupts
      9186 ± 35%     -83.8%       1485 ± 42%  interrupts.CPU92.CAL:Function_call_interrupts
      1485 ± 46%     -81.6%     273.33 ± 63%  interrupts.CPU92.NMI:Non-maskable_interrupts
      1485 ± 46%     -81.6%     273.33 ± 63%  interrupts.CPU92.PMI:Performance_monitoring_interrupts
      6946 ± 35%     -79.4%       1427 ± 33%  interrupts.CPU93.CAL:Function_call_interrupts
      1245 ± 45%     -78.4%     269.00 ± 93%  interrupts.CPU93.NMI:Non-maskable_interrupts
      1245 ± 45%     -78.4%     269.00 ± 93%  interrupts.CPU93.PMI:Performance_monitoring_interrupts
      7836 ± 51%     -86.1%       1088 ± 65%  interrupts.CPU94.CAL:Function_call_interrupts
      7555 ± 47%     -87.9%     911.33 ± 53%  interrupts.CPU95.CAL:Function_call_interrupts
     90841 ±  2%     -65.6%      31239 ± 24%  interrupts.NMI:Non-maskable_interrupts
     90841 ±  2%     -65.6%      31239 ± 24%  interrupts.PMI:Performance_monitoring_interrupts
     18910 ± 18%     -61.9%       7206 ± 10%  interrupts.RES:Rescheduling_interrupts
     55.89           -23.0       32.91 ± 28%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
     50.93           -21.3       29.59 ± 29%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
     24.70           -15.6        9.06 ± 28%  perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
     15.02 ±  2%     -14.5        0.51 ± 45%  perf-profile.calltrace.cycles-pp.destroy_inode.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
     14.15 ±  2%     -14.1        0.00        perf-profile.calltrace.cycles-pp.xfs_fs_destroy_inode.destroy_inode.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
      9.17 ± 38%      -9.2        0.00        perf-profile.calltrace.cycles-pp.xfs_inactive.xfs_fs_destroy_inode.destroy_inode.do_unlinkat.do_syscall_64
      8.01 ± 24%      -8.0        0.00        perf-profile.calltrace.cycles-pp.xfs_inactive_ifree.xfs_inactive.xfs_fs_destroy_inode.destroy_inode.do_unlinkat
      6.15 ±  6%      -6.2        0.00        perf-profile.calltrace.cycles-pp.xfs_ifree.xfs_inactive_ifree.xfs_inactive.xfs_fs_destroy_inode.destroy_inode
      4.00 ± 70%      -3.0        1.02 ±223%  perf-profile.calltrace.cycles-pp.xfs_create_tmpfile.path_openat.do_filp_open.do_sys_openat2.do_sys_open
      4.27 ± 70%      -3.0        1.29 ±223%  perf-profile.calltrace.cycles-pp.xfs_vn_lookup.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
      4.25 ± 70%      -3.0        1.28 ±223%  perf-profile.calltrace.cycles-pp.xfs_remove.xfs_vn_lookup.vfs_unlink.do_unlinkat.do_syscall_64
      3.85 ± 70%      -2.9        0.98 ±223%  perf-profile.calltrace.cycles-pp.__xfs_trans_commit.xfs_create_tmpfile.path_openat.do_filp_open.do_sys_openat2
      4.11 ± 70%      -2.8        1.31 ±223%  perf-profile.calltrace.cycles-pp.xfs_create.xfs_create.path_openat.do_filp_open.do_sys_openat2
      3.59 ± 70%      -2.6        0.95 ±223%  perf-profile.calltrace.cycles-pp.xfs_log_commit_cil.__xfs_trans_commit.xfs_create_tmpfile.path_openat.do_filp_open
      4.84 ±  2%      -1.6        3.25 ± 26%  perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
      4.75 ±  2%      -1.6        3.17 ± 26%  perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
      3.70 ±  3%      -1.3        2.39 ± 24%  perf-profile.calltrace.cycles-pp.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
      3.48 ±  3%      -1.2        2.23 ± 24%  perf-profile.calltrace.cycles-pp.__fput.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
      2.82 ±  3%      -1.0        1.83 ± 31%  perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.79 ±  3%      -1.0        1.82 ± 31%  perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.47 ±  3%      -0.9        1.61 ± 31%  perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.36 ±  2%      -0.8        1.56 ± 31%  perf-profile.calltrace.cycles-pp.xfs_file_buffered_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
      2.14 ±  2%      -0.8        1.38 ± 31%  perf-profile.calltrace.cycles-pp.iomap_file_buffered_write.xfs_file_buffered_write.new_sync_write.vfs_write.ksys_write
      2.13 ±  2%      -0.8        1.37 ± 31%  perf-profile.calltrace.cycles-pp.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_write.new_sync_write.vfs_write
      1.55 ±  6%      -0.6        0.94 ± 47%  perf-profile.calltrace.cycles-pp.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.47 ±  5%      -0.6        0.89 ± 47%  perf-profile.calltrace.cycles-pp.filp_close.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.83 ±  3%      -0.5        1.31 ± 29%  perf-profile.calltrace.cycles-pp.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.35 ±  3%      -0.5        0.84 ± 47%  perf-profile.calltrace.cycles-pp.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_write.new_sync_write
      1.16 ±  3%      -0.5        0.65 ± 49%  perf-profile.calltrace.cycles-pp.__x64_sys_rename.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.27 ±  4%      -0.5        0.77 ± 46%  perf-profile.calltrace.cycles-pp.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.11 ±  3%      -0.5        0.62 ± 49%  perf-profile.calltrace.cycles-pp.do_renameat2.__x64_sys_rename.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.22 ±  3%      -0.5        0.73 ± 46%  perf-profile.calltrace.cycles-pp.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.90 ±  2%      -0.5        0.41 ± 73%  perf-profile.calltrace.cycles-pp.vfs_rename.do_renameat2.__x64_sys_rename.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.19 ±  3%      -0.5        0.72 ± 46%  perf-profile.calltrace.cycles-pp.fcntl_dirnotify.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.33 ±  3%      -0.4        0.89 ± 47%  perf-profile.calltrace.cycles-pp.truncate_inode_pages_range.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.72 ±  3%      -0.4        0.28 ±100%  perf-profile.calltrace.cycles-pp.do_fchmodat.__x64_sys_chmod.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.72 ±  3%      -0.4        0.28 ±100%  perf-profile.calltrace.cycles-pp.__x64_sys_chmod.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.73 ±  8%      -0.4        0.30 ±101%  perf-profile.calltrace.cycles-pp.do_dentry_open.do_open.path_openat.do_filp_open.do_sys_openat2
      1.07 ±  5%      -0.4        0.66 ± 49%  perf-profile.calltrace.cycles-pp.do_open.path_openat.do_filp_open.do_sys_openat2.do_sys_open
      0.79 ±  7%      -0.4        0.40 ± 72%  perf-profile.calltrace.cycles-pp.dnotify_flush.filp_close.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.88 ±  3%      -0.3        0.56 ± 47%  perf-profile.calltrace.cycles-pp.arch_do_signal_or_restart.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
      0.87 ±  3%      -0.3        0.60 ± 47%  perf-profile.calltrace.cycles-pp.__pagevec_release.truncate_inode_pages_range.evict.do_unlinkat.do_syscall_64
      0.00            +0.9        0.89 ± 36%  perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      0.56 ±  5%      +0.9        1.47 ± 32%  perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      0.52 ± 45%      +1.0        1.52 ± 19%  perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry
      0.18 ±141%      +1.0        1.19 ± 25%  perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      0.64 ±  7%      +1.1        1.74 ± 21%  perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +1.1        1.14 ± 29%  perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      0.59 ± 45%      +1.2        1.79 ± 16%  perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary
      0.58 ±  7%      +1.4        2.02 ± 43%  perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      0.17 ±141%      +1.5        1.67 ± 40%  perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
      0.18 ±141%      +1.6        1.77 ± 45%  perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      0.80 ±  7%      +2.3        3.08 ± 52%  perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +2.4        2.44 ± 32%  perf-profile.calltrace.cycles-pp.xfs_inactive_worker.process_one_work.worker_thread.kthread.ret_from_fork
      2.06 ±  7%      +3.5        5.52 ± 24%  perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
      0.00            +3.8        3.81 ± 22%  perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
      0.00            +3.8        3.85 ± 22%  perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
      0.70 ±  6%      +3.9        4.65 ± 17%  perf-profile.calltrace.cycles-pp.ret_from_fork
      0.70 ±  6%      +3.9        4.65 ± 17%  perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
      1.63 ±  6%      +4.0        5.64 ± 36%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      1.65 ±  7%      +4.1        5.77 ± 37%  perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
     32.24 ±  2%      +5.9       38.13 ± 17%  perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
      2.75 ±  6%      +6.2        8.99 ± 37%  perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
      3.98 ±  3%      +9.2       13.21 ± 30%  perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
     37.00           +14.1       51.11 ± 19%  perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
     37.61 ±  2%     +15.4       52.96 ± 18%  perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     42.01           +18.4       60.44 ± 16%  perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     42.03           +18.5       60.49 ± 16%  perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
     42.02           +18.5       60.49 ± 16%  perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     42.30           +18.8       61.08 ± 16%  perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
     55.99           -22.8       33.22 ± 28%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
     51.01           -21.1       29.89 ± 28%  perf-profile.children.cycles-pp.do_syscall_64
     24.71           -15.7        9.06 ± 28%  perf-profile.children.cycles-pp.do_unlinkat
     15.02 ±  2%     -14.5        0.55 ± 28%  perf-profile.children.cycles-pp.destroy_inode
     14.15 ±  2%     -14.1        0.06 ± 47%  perf-profile.children.cycles-pp.xfs_fs_destroy_inode
     11.73 ± 14%      -9.5        2.23 ± 30%  perf-profile.children.cycles-pp.xfs_inactive
     16.98            -6.8       10.19 ± 29%  perf-profile.children.cycles-pp.__xfs_trans_commit
     16.57            -6.5       10.04 ± 29%  perf-profile.children.cycles-pp.xfs_log_commit_cil
      8.01 ± 24%      -6.4        1.66 ± 33%  perf-profile.children.cycles-pp.xfs_inactive_ifree
      6.25 ±  5%      -5.3        0.99 ± 35%  perf-profile.children.cycles-pp.xfs_ifree
      4.50 ± 15%      -3.8        0.69 ± 79%  perf-profile.children.cycles-pp.xfs_difree
      4.01 ± 70%      -3.0        1.02 ±223%  perf-profile.children.cycles-pp.xfs_create_tmpfile
      6.21 ± 14%      -2.8        3.45 ± 36%  perf-profile.children.cycles-pp.xfs_buf_get_map
      5.89 ±  3%      -2.5        3.37 ± 19%  perf-profile.children.cycles-pp._raw_spin_lock
      8.79 ± 17%      -2.5        6.29 ± 30%  perf-profile.children.cycles-pp.xfs_remove
      6.88 ±  9%      -2.3        4.54 ± 29%  perf-profile.children.cycles-pp.xfs_trans_read_buf_map
      2.46 ± 10%      -2.0        0.44 ± 37%  perf-profile.children.cycles-pp.xfs_iunlink_remove
      5.88 ±  2%      -2.0        3.87 ± 28%  perf-profile.children.cycles-pp.xlog_cil_insert_items
      3.21 ± 39%      -2.0        1.26 ± 58%  perf-profile.children.cycles-pp.xfs_ialloc_read_agi
      2.19 ±  4%      -1.8        0.39 ± 33%  perf-profile.children.cycles-pp.xfs_itruncate_extents_flags
      3.34 ± 11%      -1.8        1.56 ± 36%  perf-profile.children.cycles-pp.xfs_release
      1.99 ±  4%      -1.7        0.27 ± 34%  perf-profile.children.cycles-pp.xfs_difree_inobt
      4.83 ±  7%      -1.7        3.16 ± 29%  perf-profile.children.cycles-pp.xfs_buf_read_map
      4.51 ±  7%      -1.6        2.91 ± 29%  perf-profile.children.cycles-pp.xfs_buf_find
      4.86 ±  2%      -1.6        3.26 ± 26%  perf-profile.children.cycles-pp.syscall_exit_to_user_mode
      4.76 ±  2%      -1.6        3.18 ± 26%  perf-profile.children.cycles-pp.exit_to_user_mode_prepare
      2.15 ± 11%      -1.5        0.66 ± 28%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      3.71 ±  3%      -1.3        2.39 ± 25%  perf-profile.children.cycles-pp.task_work_run
      1.47 ±  6%      -1.3        0.18 ± 37%  perf-profile.children.cycles-pp.xfs_defer_finish
      1.44 ±  6%      -1.3        0.15 ± 49%  perf-profile.children.cycles-pp.xfs_defer_trans_roll
      3.49 ±  3%      -1.3        2.23 ± 24%  perf-profile.children.cycles-pp.__fput
      2.59 ±  2%      -1.2        1.37 ± 28%  perf-profile.children.cycles-pp.xfs_trans_alloc
      3.43 ±  6%      -1.1        2.28 ± 29%  perf-profile.children.cycles-pp.xfs_btree_lookup
      1.36 ±  6%      -1.1        0.23 ± 37%  perf-profile.children.cycles-pp.xfs_trans_roll
      1.51 ± 52%      -1.1        0.40 ±136%  perf-profile.children.cycles-pp.xfs_trans_add_item
      2.83 ±  3%      -1.0        1.85 ± 30%  perf-profile.children.cycles-pp.ksys_write
      1.14 ± 17%      -1.0        0.17 ± 35%  perf-profile.children.cycles-pp.xfs_difree_finobt
      2.80 ±  3%      -1.0        1.84 ± 30%  perf-profile.children.cycles-pp.vfs_write
      2.15 ±  3%      -0.9        1.29 ± 24%  perf-profile.children.cycles-pp.__schedule
      2.10 ±  5%      -0.9        1.25 ± 35%  perf-profile.children.cycles-pp.up
      2.47 ±  3%      -0.8        1.62 ± 31%  perf-profile.children.cycles-pp.new_sync_write
      2.39 ±  3%      -0.8        1.57 ± 32%  perf-profile.children.cycles-pp.xfs_file_buffered_write
      2.14 ±  2%      -0.8        1.38 ± 31%  perf-profile.children.cycles-pp.iomap_file_buffered_write
      2.13 ±  2%      -0.8        1.37 ± 31%  perf-profile.children.cycles-pp.iomap_apply
      1.40 ±  5%      -0.7        0.66 ± 28%  perf-profile.children.cycles-pp.xfs_log_ticket_ungrant
      1.61 ±  7%      -0.7        0.93 ± 28%  perf-profile.children.cycles-pp.down
      1.67 ±  4%      -0.7        1.00 ± 30%  perf-profile.children.cycles-pp.try_to_wake_up
      0.82 ± 16%      -0.7        0.16 ± 19%  perf-profile.children.cycles-pp.poll_idle
      1.57 ±  6%      -0.7        0.92 ± 28%  perf-profile.children.cycles-pp.__down
      1.32 ±  5%      -0.6        0.75 ± 29%  perf-profile.children.cycles-pp.schedule_timeout
      1.54 ±  3%      -0.5        1.01 ± 32%  perf-profile.children.cycles-pp.__percpu_counter_compare
      1.55 ±  6%      -0.5        1.02 ± 28%  perf-profile.children.cycles-pp.__x64_sys_close
      1.36 ±  5%      -0.5        0.84 ± 25%  perf-profile.children.cycles-pp.schedule
      1.83 ±  3%      -0.5        1.31 ± 29%  perf-profile.children.cycles-pp.evict
      1.47 ±  5%      -0.5        0.97 ± 27%  perf-profile.children.cycles-pp.filp_close
      1.42 ±  3%      -0.5        0.95 ± 31%  perf-profile.children.cycles-pp.__percpu_counter_sum
      1.16 ±  3%      -0.5        0.69 ± 33%  perf-profile.children.cycles-pp.__x64_sys_rename
      1.35 ±  3%      -0.5        0.90 ± 31%  perf-profile.children.cycles-pp.iomap_write_actor
      1.11 ±  3%      -0.5        0.66 ± 33%  perf-profile.children.cycles-pp.do_renameat2
      1.28 ±  3%      -0.4        0.83 ± 27%  perf-profile.children.cycles-pp.__x64_sys_fcntl
      1.22 ±  3%      -0.4        0.79 ± 27%  perf-profile.children.cycles-pp.do_fcntl
      1.20 ±  3%      -0.4        0.78 ± 26%  perf-profile.children.cycles-pp.fcntl_dirnotify
      0.88 ±  6%      -0.4        0.47 ± 30%  perf-profile.children.cycles-pp.down_read
      0.91 ±  5%      -0.4        0.51 ± 26%  perf-profile.children.cycles-pp.schedule_idle
      1.33 ±  3%      -0.4        0.95 ± 30%  perf-profile.children.cycles-pp.truncate_inode_pages_range
      0.90 ±  2%      -0.4        0.52 ± 33%  perf-profile.children.cycles-pp.vfs_rename
      1.07 ±  5%      -0.4        0.71 ± 33%  perf-profile.children.cycles-pp.do_open
      0.46 ±  4%      -0.4        0.10 ± 47%  perf-profile.children.cycles-pp.__xfs_bunmapi
      0.91 ±  3%      -0.3        0.61 ± 30%  perf-profile.children.cycles-pp.kmem_cache_free
      0.52 ± 31%      -0.3        0.23 ± 23%  perf-profile.children.cycles-pp.xfs_btree_check_sblock
      0.82            -0.3        0.54 ± 30%  perf-profile.children.cycles-pp.xfs_inode_item_format
      0.79 ±  7%      -0.3        0.51 ± 28%  perf-profile.children.cycles-pp.dnotify_flush
      0.67 ± 11%      -0.3        0.39 ± 26%  perf-profile.children.cycles-pp.__radix_tree_lookup
      0.88 ±  3%      -0.3        0.61 ± 28%  perf-profile.children.cycles-pp.arch_do_signal_or_restart
      0.73 ±  8%      -0.3        0.48 ± 31%  perf-profile.children.cycles-pp.do_dentry_open
      0.72 ±  3%      -0.3        0.47 ± 25%  perf-profile.children.cycles-pp.do_fchmodat
      0.63 ±  2%      -0.3        0.37 ± 31%  perf-profile.children.cycles-pp.xfs_rename
      0.48 ± 27%      -0.3        0.23 ± 51%  perf-profile.children.cycles-pp.percpu_counter_add_batch
      0.67 ±  6%      -0.3        0.41 ± 32%  perf-profile.children.cycles-pp.xfs_buffered_write_iomap_begin
      0.72 ±  3%      -0.3        0.47 ± 25%  perf-profile.children.cycles-pp.__x64_sys_chmod
      0.64 ±  4%      -0.2        0.40 ± 26%  perf-profile.children.cycles-pp.chmod_common
      0.59 ±  8%      -0.2        0.36 ± 29%  perf-profile.children.cycles-pp.dequeue_task_fair
      0.87 ±  3%      -0.2        0.64 ± 31%  perf-profile.children.cycles-pp.__pagevec_release
      0.62 ±  6%      -0.2        0.39 ± 32%  perf-profile.children.cycles-pp.xfs_trans_alloc_icreate
      0.61 ±  4%      -0.2        0.38 ± 28%  perf-profile.children.cycles-pp.notify_change
      0.74 ±  7%      -0.2        0.52 ± 33%  perf-profile.children.cycles-pp.fsnotify
      0.32 ±  9%      -0.2        0.09 ± 47%  perf-profile.children.cycles-pp.xfs_btree_increment
      0.56 ± 14%      -0.2        0.34 ± 28%  perf-profile.children.cycles-pp.ttwu_do_activate
      0.55 ± 14%      -0.2        0.33 ± 27%  perf-profile.children.cycles-pp.enqueue_task_fair
      0.40 ± 29%      -0.2        0.19 ± 59%  perf-profile.children.cycles-pp.xfs_mod_fdblocks
      0.53 ±  9%      -0.2        0.32 ± 29%  perf-profile.children.cycles-pp.dequeue_entity
      0.51 ±  5%      -0.2        0.30 ± 31%  perf-profile.children.cycles-pp.__list_add_valid
      0.34 ±  5%      -0.2        0.14 ± 18%  perf-profile.children.cycles-pp.up_read
      0.63 ±  2%      -0.2        0.43 ± 30%  perf-profile.children.cycles-pp.iomap_write_begin
      0.56 ±  8%      -0.2        0.37 ± 34%  perf-profile.children.cycles-pp.__fsnotify_parent
      0.45 ±  4%      -0.2        0.26 ± 24%  perf-profile.children.cycles-pp.xfs_setattr_nonsize
      0.58 ±  4%      -0.2        0.40 ± 31%  perf-profile.children.cycles-pp.fsnotify_put_mark
      0.59 ±  3%      -0.2        0.41 ± 29%  perf-profile.children.cycles-pp.grab_cache_page_write_begin
      0.58 ±  3%      -0.2        0.40 ± 30%  perf-profile.children.cycles-pp.pagecache_get_page
      0.43 ± 11%      -0.2        0.25 ± 28%  perf-profile.children.cycles-pp.enqueue_entity
      0.47 ±  7%      -0.2        0.30 ± 33%  perf-profile.children.cycles-pp.iomap_write_end
      0.49 ± 12%      -0.2        0.32 ± 23%  perf-profile.children.cycles-pp.pick_next_task_fair
      0.39 ±  8%      -0.2        0.23 ± 39%  perf-profile.children.cycles-pp.xfs_dir2_sf_addname
      0.27 ±  4%      -0.2        0.11 ± 29%  perf-profile.children.cycles-pp.down_write
      0.23 ± 11%      -0.2        0.07 ± 50%  perf-profile.children.cycles-pp.queue_delayed_work_on
      0.35 ± 10%      -0.2        0.20 ± 42%  perf-profile.children.cycles-pp.xfs_bmapi_reserve_delalloc
      0.51 ±  4%      -0.1        0.37 ± 16%  perf-profile.children.cycles-pp.link_path_walk
      0.37 ±  4%      -0.1        0.23 ± 22%  perf-profile.children.cycles-pp.fsnotify_add_mark_locked
      0.33 ±  5%      -0.1        0.19 ± 39%  perf-profile.children.cycles-pp.xfs_trans_log_buf
      0.42 ±  5%      -0.1        0.28 ± 28%  perf-profile.children.cycles-pp.__x64_sys_rt_sigreturn
      0.41 ±  5%      -0.1        0.29 ± 23%  perf-profile.children.cycles-pp.filename_parentat
      0.45 ±  5%      -0.1        0.32 ± 27%  perf-profile.children.cycles-pp.__setup_rt_frame
      0.35 ±  9%      -0.1        0.23 ± 34%  perf-profile.children.cycles-pp.iomap_set_page_dirty
      0.28 ±  6%      -0.1        0.16 ± 34%  perf-profile.children.cycles-pp.update_curr
      0.38 ±  5%      -0.1        0.26 ± 22%  perf-profile.children.cycles-pp.path_parentat
      0.26 ± 11%      -0.1        0.15 ± 31%  perf-profile.children.cycles-pp.xfs_agino_range
      0.33 ± 10%      -0.1        0.21 ± 26%  perf-profile.children.cycles-pp.update_load_avg
      0.36 ±  6%      -0.1        0.25 ± 32%  perf-profile.children.cycles-pp.dput
      0.24 ±  4%      -0.1        0.13 ± 35%  perf-profile.children.cycles-pp.xfs_inobt_init_cursor
      0.28 ±  4%      -0.1        0.17 ± 33%  perf-profile.children.cycles-pp.xfs_dir2_sf_removename
      0.22 ±  5%      -0.1        0.11 ± 33%  perf-profile.children.cycles-pp.xfs_droplink
      0.32 ±  9%      -0.1        0.21 ± 31%  perf-profile.children.cycles-pp.security_file_free
      0.33 ±  8%      -0.1        0.23 ± 37%  perf-profile.children.cycles-pp.lru_add_drain
      0.22 ± 10%      -0.1        0.11 ± 35%  perf-profile.children.cycles-pp.up_write
      0.32 ±  8%      -0.1        0.22 ± 38%  perf-profile.children.cycles-pp.lru_add_drain_cpu
      0.31 ±  7%      -0.1        0.20 ± 37%  perf-profile.children.cycles-pp.inode_permission
      0.31 ± 10%      -0.1        0.21 ± 31%  perf-profile.children.cycles-pp.apparmor_file_free_security
      0.37 ± 16%      -0.1        0.27 ±  9%  perf-profile.children.cycles-pp._raw_spin_lock_irq
      0.26 ± 10%      -0.1        0.16 ± 32%  perf-profile.children.cycles-pp.mutex_lock
      0.23 ± 13%      -0.1        0.14 ± 34%  perf-profile.children.cycles-pp.__mnt_want_write
      0.20 ± 11%      -0.1        0.10 ± 30%  perf-profile.children.cycles-pp.__switch_to
      0.37 ±  9%      -0.1        0.28 ± 27%  perf-profile.children.cycles-pp.security_file_alloc
      0.15 ± 10%      -0.1        0.06 ± 73%  perf-profile.children.cycles-pp.finish_task_switch
      0.20 ±  7%      -0.1        0.11 ± 54%  perf-profile.children.cycles-pp.delete_from_page_cache_batch
      0.30 ± 10%      -0.1        0.21 ± 38%  perf-profile.children.cycles-pp.__pagevec_lru_add
      0.35 ±  9%      -0.1        0.26 ± 20%  perf-profile.children.cycles-pp.___might_sleep
      0.27 ± 10%      -0.1        0.18 ± 25%  perf-profile.children.cycles-pp.mnt_want_write
      0.33 ±  3%      -0.1        0.25 ± 29%  perf-profile.children.cycles-pp.cpumask_next
      0.20 ± 11%      -0.1        0.11 ± 45%  perf-profile.children.cycles-pp.locked_inode_to_wb_and_lock_list
      0.27 ±  5%      -0.1        0.19 ± 37%  perf-profile.children.cycles-pp.memset_erms
      0.22 ± 10%      -0.1        0.13 ± 43%  perf-profile.children.cycles-pp.errseq_sample
      0.30 ±  8%      -0.1        0.22 ± 30%  perf-profile.children.cycles-pp.copy_fpstate_to_sigframe
      0.21 ±  3%      -0.1        0.13 ± 48%  perf-profile.children.cycles-pp.__cond_resched
      0.33 ±  8%      -0.1        0.25 ± 32%  perf-profile.children.cycles-pp.send_sigio
      0.28 ±  5%      -0.1        0.19 ± 29%  perf-profile.children.cycles-pp.restore_sigcontext
      0.30 ±  6%      -0.1        0.22 ± 33%  perf-profile.children.cycles-pp.add_to_page_cache_lru
      0.26 ±  3%      -0.1        0.18 ± 45%  perf-profile.children.cycles-pp.__might_sleep
      0.28 ±  6%      -0.1        0.20 ± 32%  perf-profile.children.cycles-pp.__add_to_page_cache_locked
      0.29 ±  8%      -0.1        0.21 ± 25%  perf-profile.children.cycles-pp.apparmor_file_alloc_security
      0.13 ± 17%      -0.1        0.06 ± 54%  perf-profile.children.cycles-pp.xfs_lock_two_inodes
      0.11 ± 15%      -0.1        0.04 ±100%  perf-profile.children.cycles-pp.mutex_unlock
      0.14 ±  6%      -0.1        0.07 ± 50%  perf-profile.children.cycles-pp.drop_nlink
      0.11 ± 29%      -0.1        0.04 ± 71%  perf-profile.children.cycles-pp.xfs_inobt_get_maxrecs
      0.17 ±  7%      -0.1        0.10 ± 49%  perf-profile.children.cycles-pp.__switch_to_asm
      0.17 ±  5%      -0.1        0.10 ± 27%  perf-profile.children.cycles-pp.fsnotify_init_mark
      0.21 ±  5%      -0.1        0.15 ± 26%  perf-profile.children.cycles-pp.__fpu__restore_sig
      0.18 ± 11%      -0.1        0.12 ± 51%  perf-profile.children.cycles-pp.__x64_sys_unlink
      0.23 ±  7%      -0.1        0.16 ± 25%  perf-profile.children.cycles-pp.get_signal
      0.14 ± 13%      -0.1        0.08 ± 49%  perf-profile.children.cycles-pp.fsnotify_mark_destroy_workfn
      0.19 ± 10%      -0.1        0.12 ± 46%  perf-profile.children.cycles-pp.down_trylock
      0.22 ±  4%      -0.1        0.15 ± 31%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      0.18 ±  5%      -0.1        0.12 ± 46%  perf-profile.children.cycles-pp.generic_permission
      0.10 ± 16%      -0.1        0.04 ± 72%  perf-profile.children.cycles-pp.update_cfs_group
      0.19 ±  8%      -0.1        0.13 ± 29%  perf-profile.children.cycles-pp.inode_io_list_del
      0.15 ± 13%      -0.1        0.09 ± 49%  perf-profile.children.cycles-pp.xfs_bmap_add_extent_hole_delay
      0.15 ±  7%      -0.1        0.09 ± 27%  perf-profile.children.cycles-pp.fsnotify_get_group
      0.15 ±  9%      -0.1        0.09 ± 50%  perf-profile.children.cycles-pp.__lookup_hash
      0.15 ±  9%      -0.1        0.10 ± 49%  perf-profile.children.cycles-pp.make_kuid
      0.15 ± 14%      -0.1        0.09 ± 45%  perf-profile.children.cycles-pp.queue_work_on
      0.18 ± 11%      -0.1        0.13 ± 30%  perf-profile.children.cycles-pp.xfs_ialloc_ag_select
      0.13 ± 20%      -0.1        0.08 ± 51%  perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic
      0.10            -0.1        0.05 ± 73%  perf-profile.children.cycles-pp.rcu_all_qs
      0.13 ±  2%      -0.1        0.08 ± 51%  perf-profile.children.cycles-pp.fpu__clear
      0.11 ± 18%      -0.1        0.06 ± 49%  perf-profile.children.cycles-pp.ttwu_do_wakeup
      0.09 ± 14%      -0.1        0.03 ±100%  perf-profile.children.cycles-pp.asm_sysvec_call_function_single
      0.15 ±  8%      -0.1        0.10 ± 20%  perf-profile.children.cycles-pp.map_id_up
      0.14 ± 13%      -0.1        0.09 ± 28%  perf-profile.children.cycles-pp.fsnotify_find_mark
      0.20 ± 13%      -0.0        0.15 ± 29%  perf-profile.children.cycles-pp.__mutex_lock
      0.10 ±  9%      -0.0        0.06 ± 52%  perf-profile.children.cycles-pp.fsnotify_final_mark_destroy
      0.14 ±  5%      -0.0        0.10 ± 46%  perf-profile.children.cycles-pp.__entry_text_start
      0.10 ±  9%      -0.0        0.05 ± 71%  perf-profile.children.cycles-pp.lock_page_lruvec_irqsave
      0.15 ±  9%      -0.0        0.10 ± 26%  perf-profile.children.cycles-pp.dequeue_signal
      0.08 ± 19%      -0.0        0.04 ± 73%  perf-profile.children.cycles-pp.task_work_add
      0.16 ± 11%      -0.0        0.12 ± 30%  perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      0.10 ± 17%      -0.0        0.06 ± 49%  perf-profile.children.cycles-pp.check_preempt_curr
      0.10 ± 17%      -0.0        0.06 ± 51%  perf-profile.children.cycles-pp.xfs_iext_insert_raw
      0.14 ±  5%      -0.0        0.09 ± 46%  perf-profile.children.cycles-pp.refcount_dec_and_lock
      0.11 ± 11%      -0.0        0.07 ± 48%  perf-profile.children.cycles-pp.alloc_fd
      0.11 ±  6%      -0.0        0.07 ± 48%  perf-profile.children.cycles-pp.lockref_put_return
      0.07 ± 10%      -0.0        0.03 ±102%  perf-profile.children.cycles-pp.d_lookup
      0.11 ± 11%      -0.0        0.07 ± 47%  perf-profile.children.cycles-pp.xfs_trans_alloc_ichange
      0.10 ± 19%      -0.0        0.07 ± 24%  perf-profile.children.cycles-pp.fput_many
      0.07 ± 11%      -0.0        0.05 ± 45%  perf-profile.children.cycles-pp.___perf_sw_event
      0.06 ± 14%      +0.1        0.12 ± 20%  perf-profile.children.cycles-pp.page_counter_try_charge
      0.00            +0.1        0.06 ± 11%  perf-profile.children.cycles-pp.crc32c_pcl_intel_update
      0.06 ± 13%      +0.1        0.12 ± 16%  perf-profile.children.cycles-pp.__memcg_kmem_charge
      0.12 ±  4%      +0.1        0.19 ± 15%  perf-profile.children.cycles-pp.__slab_free
      0.04 ± 71%      +0.1        0.11 ± 30%  perf-profile.children.cycles-pp.call_cpuidle
      0.00            +0.1        0.07 ± 31%  perf-profile.children.cycles-pp.trigger_load_balance
      0.00            +0.1        0.08 ± 24%  perf-profile.children.cycles-pp.execve
      0.00            +0.1        0.08 ± 24%  perf-profile.children.cycles-pp.__x64_sys_execve
      0.00            +0.1        0.08 ± 24%  perf-profile.children.cycles-pp.do_execveat_common
      0.00            +0.1        0.09 ± 35%  perf-profile.children.cycles-pp.rcu_dynticks_eqs_exit
      0.00            +0.1        0.09 ± 42%  perf-profile.children.cycles-pp.exc_page_fault
      0.00            +0.1        0.09 ± 40%  perf-profile.children.cycles-pp.asm_exc_page_fault
      0.00            +0.1        0.10 ± 48%  perf-profile.children.cycles-pp.memcpy_toio
      0.07 ±  5%      +0.1        0.17 ± 88%  perf-profile.children.cycles-pp.cpuidle_governor_latency_req
      0.03 ±102%      +0.1        0.13 ± 49%  perf-profile.children.cycles-pp.rcu_eqs_enter
      0.11 ± 13%      +0.1        0.21 ± 33%  perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
      0.11 ± 13%      +0.1        0.22 ± 31%  perf-profile.children.cycles-pp.arch_cpu_idle_enter
      0.04 ± 71%      +0.1        0.15 ± 26%  perf-profile.children.cycles-pp.xfs_cil_prepare_item
      0.07 ±  7%      +0.1        0.18 ± 67%  perf-profile.children.cycles-pp.rcu_eqs_exit
      0.06 ± 13%      +0.1        0.18 ± 29%  perf-profile.children.cycles-pp.__intel_pmu_enable_all
      0.07 ± 12%      +0.1        0.20 ± 51%  perf-profile.children.cycles-pp.update_blocked_averages
      0.15 ± 13%      +0.1        0.28 ± 22%  perf-profile.children.cycles-pp._raw_spin_trylock
      0.13 ± 10%      +0.1        0.27 ± 26%  perf-profile.children.cycles-pp.update_sd_lb_stats
      0.08 ±  8%      +0.1        0.21 ± 57%  perf-profile.children.cycles-pp.run_rebalance_domains
      0.02 ±142%      +0.1        0.16 ± 60%  perf-profile.children.cycles-pp.update_irq_load_avg
      0.14 ± 10%      +0.1        0.28 ± 26%  perf-profile.children.cycles-pp.find_busiest_group
      0.09 ±  7%      +0.1        0.23 ± 67%  perf-profile.children.cycles-pp.rcu_idle_exit
      0.04 ± 45%      +0.1        0.19 ± 39%  perf-profile.children.cycles-pp.arch_scale_freq_tick
      0.00            +0.2        0.16 ± 92%  perf-profile.children.cycles-pp.timerqueue_del
      0.11 ± 15%      +0.2        0.27 ± 27%  perf-profile.children.cycles-pp.calc_global_load_tick
      0.04 ± 73%      +0.2        0.20 ± 44%  perf-profile.children.cycles-pp.hrtimer_next_event_without
      0.08 ±  8%      +0.2        0.25 ± 69%  perf-profile.children.cycles-pp.rcu_sched_clock_irq
      0.00            +0.2        0.17 ± 33%  perf-profile.children.cycles-pp.xfs_buf_offset
      0.04 ± 72%      +0.2        0.21 ± 42%  perf-profile.children.cycles-pp.get_next_timer_interrupt
      0.11 ± 69%      +0.2        0.29 ± 35%  perf-profile.children.cycles-pp.xfs_next_bit
      0.01 ±223%      +0.2        0.20 ±101%  perf-profile.children.cycles-pp.__remove_hrtimer
      0.00            +0.2        0.20 ± 38%  perf-profile.children.cycles-pp.kmem_alloc_large
      0.01 ±223%      +0.2        0.22 ± 66%  perf-profile.children.cycles-pp.__hrtimer_next_event_base
      0.00            +0.2        0.21 ± 30%  perf-profile.children.cycles-pp.xfs_iunlink_insert_backref
      0.16 ± 11%      +0.2        0.37 ± 33%  perf-profile.children.cycles-pp.load_balance
      0.00            +0.2        0.21 ± 41%  perf-profile.children.cycles-pp.xfs_inode_set_inactive_tag
      0.20 ±  3%      +0.2        0.44 ± 30%  perf-profile.children.cycles-pp.native_sched_clock
      0.06 ± 11%      +0.3        0.31 ± 51%  perf-profile.children.cycles-pp.io_serial_in
      0.21 ±  4%      +0.3        0.47 ± 30%  perf-profile.children.cycles-pp.sched_clock
      0.00            +0.3        0.27 ± 51%  perf-profile.children.cycles-pp.xfs_trans_committed_bulk
      0.34 ±  6%      +0.3        0.63 ± 23%  perf-profile.children.cycles-pp.native_irq_return_iret
      0.23 ±  4%      +0.3        0.53 ± 29%  perf-profile.children.cycles-pp.sched_clock_cpu
      0.19 ±  5%      +0.3        0.48 ± 36%  perf-profile.children.cycles-pp.read_tsc
      0.09 ±  5%      +0.3        0.38 ± 51%  perf-profile.children.cycles-pp.serial8250_console_putchar
      0.09 ±  5%      +0.3        0.39 ± 49%  perf-profile.children.cycles-pp.wait_for_xmitr
      0.11 ±  6%      +0.3        0.42 ± 37%  perf-profile.children.cycles-pp.ktime_get_update_offsets_now
      0.09 ±  4%      +0.3        0.40 ± 51%  perf-profile.children.cycles-pp.uart_console_write
      0.00            +0.3        0.32 ± 37%  perf-profile.children.cycles-pp.xlog_write
      0.09            +0.3        0.40 ± 49%  perf-profile.children.cycles-pp.serial8250_console_write
      0.00            +0.3        0.32 ± 25%  perf-profile.children.cycles-pp.allocate_slab
      0.18 ±  6%      +0.3        0.50 ± 39%  perf-profile.children.cycles-pp.irqtime_account_irq
      0.27 ± 53%      +0.3        0.59 ± 50%  perf-profile.children.cycles-pp.start_kernel
      0.09 ±  5%      +0.3        0.43 ± 49%  perf-profile.children.cycles-pp.asm_sysvec_irq_work
      0.09 ±  5%      +0.3        0.43 ± 49%  perf-profile.children.cycles-pp.sysvec_irq_work
      0.09 ±  5%      +0.3        0.43 ± 49%  perf-profile.children.cycles-pp.__sysvec_irq_work
      0.09 ±  5%      +0.3        0.43 ± 49%  perf-profile.children.cycles-pp.irq_work_run
      0.09 ±  5%      +0.3        0.43 ± 49%  perf-profile.children.cycles-pp.irq_work_single
      0.09 ±  5%      +0.3        0.43 ± 49%  perf-profile.children.cycles-pp.printk
      0.09 ±  5%      +0.3        0.43 ± 49%  perf-profile.children.cycles-pp.vprintk_emit
      0.09 ±  5%      +0.3        0.43 ± 49%  perf-profile.children.cycles-pp.console_unlock
      0.23 ± 12%      +0.3        0.57 ± 24%  perf-profile.children.cycles-pp.tick_nohz_irq_exit
      0.06 ± 14%      +0.3        0.41 ± 27%  perf-profile.children.cycles-pp.___slab_alloc
      0.06 ± 11%      +0.3        0.41 ± 27%  perf-profile.children.cycles-pp.__slab_alloc
      0.10 ±  5%      +0.4        0.45 ± 51%  perf-profile.children.cycles-pp.irq_work_run_list
      0.17 ± 14%      +0.4        0.53 ± 27%  perf-profile.children.cycles-pp.lapic_next_deadline
      0.00            +0.4        0.37 ± 23%  perf-profile.children.cycles-pp.xfs_inode_alloc
      0.25 ± 10%      +0.4        0.64 ± 28%  perf-profile.children.cycles-pp.rebalance_domains
      0.15 ±  7%      +0.4        0.55 ± 49%  perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
      0.00            +0.4        0.44 ± 35%  perf-profile.children.cycles-pp.xlog_cil_push_work
      0.00            +0.5        0.51 ± 55%  perf-profile.children.cycles-pp.xfsaild
      0.00            +0.5        0.51 ± 55%  perf-profile.children.cycles-pp.xfsaild_push
      1.04 ±  2%      +0.5        1.58 ± 13%  perf-profile.children.cycles-pp.__softirqentry_text_start
      0.24 ± 16%      +0.5        0.79 ± 67%  perf-profile.children.cycles-pp.tick_irq_enter
      0.25 ± 16%      +0.6        0.81 ± 66%  perf-profile.children.cycles-pp.irq_enter_rcu
      0.31 ±  4%      +0.6        0.95 ± 34%  perf-profile.children.cycles-pp.scheduler_tick
      0.86 ±  4%      +0.8        1.71 ± 25%  perf-profile.children.cycles-pp.irq_exit_rcu
      0.60 ± 12%      +0.9        1.55 ± 19%  perf-profile.children.cycles-pp.tick_nohz_next_event
      0.71 ±  7%      +1.1        1.81 ± 20%  perf-profile.children.cycles-pp.clockevents_program_event
      0.68 ± 12%      +1.1        1.82 ± 16%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      0.20 ± 25%      +1.1        1.34 ± 29%  perf-profile.children.cycles-pp.xfs_dialloc_ag_update_inobt
      0.58 ±  7%      +1.2        1.78 ± 38%  perf-profile.children.cycles-pp.update_process_times
      0.59 ±  7%      +1.3        1.87 ± 42%  perf-profile.children.cycles-pp.tick_sched_handle
      0.69 ±  7%      +1.4        2.13 ± 41%  perf-profile.children.cycles-pp.tick_sched_timer
      1.13 ±  9%      +1.5        2.67 ± 19%  perf-profile.children.cycles-pp.ktime_get
      0.96 ±  6%      +2.3        3.24 ± 50%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.00            +2.4        2.42 ± 31%  perf-profile.children.cycles-pp.xfs_inactive_inode
      0.00            +2.4        2.44 ± 32%  perf-profile.children.cycles-pp.xfs_inactive_worker
      0.00            +2.4        2.44 ± 32%  perf-profile.children.cycles-pp.xfs_inode_walk_ag
      2.06 ±141%      +3.0        5.01 ± 56%  perf-profile.children.cycles-pp.xfs_dir_ialloc
      2.09 ±  7%      +3.5        5.59 ± 24%  perf-profile.children.cycles-pp.menu_select
      0.33 ±  8%      +3.5        3.85 ± 22%  perf-profile.children.cycles-pp.worker_thread
      0.28 ±  6%      +3.5        3.81 ± 22%  perf-profile.children.cycles-pp.process_one_work
      0.70 ±  6%      +3.9        4.65 ± 17%  perf-profile.children.cycles-pp.kthread
      0.70 ±  6%      +4.0        4.65 ± 17%  perf-profile.children.cycles-pp.ret_from_fork
      1.89 ±  6%      +4.0        5.90 ± 35%  perf-profile.children.cycles-pp.hrtimer_interrupt
      1.91 ±  6%      +4.1        6.03 ± 36%  perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
     32.41 ±  2%      +5.8       38.23 ± 17%  perf-profile.children.cycles-pp.intel_idle
      3.33 ±  5%      +6.1        9.48 ± 35%  perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
      4.01 ±  4%      +7.7       11.73 ± 30%  perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
     37.84           +15.6       53.46 ± 18%  perf-profile.children.cycles-pp.cpuidle_enter_state
     37.84           +15.6       53.48 ± 18%  perf-profile.children.cycles-pp.cpuidle_enter
     42.03           +18.5       60.49 ± 16%  perf-profile.children.cycles-pp.start_secondary
     42.30           +18.8       61.08 ± 16%  perf-profile.children.cycles-pp.secondary_startup_64_no_verify
     42.30           +18.8       61.08 ± 16%  perf-profile.children.cycles-pp.cpu_startup_entry
     42.29           +18.8       61.08 ± 16%  perf-profile.children.cycles-pp.do_idle
      2.11 ± 10%      -1.5        0.65 ± 28%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
      3.84 ±  3%      -1.1        2.78 ± 19%  perf-profile.self.cycles-pp._raw_spin_lock
      1.82 ±  3%      -1.0        0.84 ± 30%  perf-profile.self.cycles-pp.xfs_log_commit_cil
      1.35 ±  3%      -0.8        0.53 ± 25%  perf-profile.self.cycles-pp.xfs_log_ticket_ungrant
      0.78 ± 16%      -0.7        0.12 ± 21%  perf-profile.self.cycles-pp.poll_idle
      1.03 ±  5%      -0.6        0.45 ± 24%  perf-profile.self.cycles-pp.xlog_cil_insert_items
      0.81 ±  6%      -0.4        0.42 ± 31%  perf-profile.self.cycles-pp.down_read
      0.89 ±  4%      -0.3        0.56 ± 31%  perf-profile.self.cycles-pp.__percpu_counter_sum
      0.50 ±  7%      -0.3        0.19 ± 37%  perf-profile.self.cycles-pp.xfs_trans_log_inode
      0.39 ± 48%      -0.3        0.09 ±156%  perf-profile.self.cycles-pp.__xfs_trans_commit
      0.66 ± 11%      -0.3        0.39 ± 26%  perf-profile.self.cycles-pp.__radix_tree_lookup
      0.81 ±  4%      -0.3        0.56 ± 24%  perf-profile.self.cycles-pp.xfs_buf_find
      0.47 ± 26%      -0.2        0.22 ± 51%  perf-profile.self.cycles-pp.percpu_counter_add_batch
      0.34 ± 26%      -0.2        0.13 ± 34%  perf-profile.self.cycles-pp.xfs_trans_alloc
      0.27 ± 39%      -0.2        0.06 ± 90%  perf-profile.self.cycles-pp.xfs_log_reserve
      0.34 ±  6%      -0.2        0.14 ± 17%  perf-profile.self.cycles-pp.up_read
      0.64 ±  3%      -0.2        0.44 ± 31%  perf-profile.self.cycles-pp.xfs_inode_item_format
      0.57 ±  3%      -0.2        0.37 ± 25%  perf-profile.self.cycles-pp.kmem_cache_free
      0.50 ±  5%      -0.2        0.30 ± 32%  perf-profile.self.cycles-pp.__list_add_valid
      0.34 ±  8%      -0.2        0.15 ± 27%  perf-profile.self.cycles-pp.xfs_trans_add_item
      0.59 ±  5%      -0.2        0.40 ± 26%  perf-profile.self.cycles-pp.kmem_cache_alloc
      0.23 ±  9%      -0.2        0.07 ± 48%  perf-profile.self.cycles-pp.queue_delayed_work_on
      0.35 ±  3%      -0.2        0.20 ± 30%  perf-profile.self.cycles-pp.__schedule
      0.36 ±  5%      -0.2        0.21 ± 30%  perf-profile.self.cycles-pp.xfs_btree_lookup
      0.19 ±  6%      -0.1        0.05 ± 71%  perf-profile.self.cycles-pp.down_write
      0.22 ± 48%      -0.1        0.10 ± 24%  perf-profile.self.cycles-pp.__xfs_btree_check_sblock
      0.32 ±  7%      -0.1        0.20 ± 36%  perf-profile.self.cycles-pp.fsnotify
      0.13 ±  5%      -0.1        0.03 ±102%  perf-profile.self.cycles-pp.__xfs_bunmapi
      0.21 ±  9%      -0.1        0.11 ± 35%  perf-profile.self.cycles-pp.up_write
      0.36 ± 16%      -0.1        0.26 ± 11%  perf-profile.self.cycles-pp._raw_spin_lock_irq
      0.30 ± 10%      -0.1        0.21 ± 32%  perf-profile.self.cycles-pp.apparmor_file_free_security
      0.24 ± 11%      -0.1        0.14 ± 35%  perf-profile.self.cycles-pp.mutex_lock
      0.19 ± 14%      -0.1        0.09 ± 46%  perf-profile.self.cycles-pp.xfs_agino_range
      0.34 ±  9%      -0.1        0.24 ± 19%  perf-profile.self.cycles-pp.___might_sleep
      0.23 ± 12%      -0.1        0.14 ± 34%  perf-profile.self.cycles-pp.__mnt_want_write
      0.18 ±  7%      -0.1        0.09 ± 51%  perf-profile.self.cycles-pp.xfs_create
      0.19 ±  8%      -0.1        0.10 ± 28%  perf-profile.self.cycles-pp.xfs_trans_read_buf_map
      0.19 ± 10%      -0.1        0.10 ± 28%  perf-profile.self.cycles-pp.__switch_to
      0.21 ± 10%      -0.1        0.13 ± 43%  perf-profile.self.cycles-pp.errseq_sample
      0.27 ±  8%      -0.1        0.19 ± 25%  perf-profile.self.cycles-pp.apparmor_file_alloc_security
      0.11 ± 21%      -0.1        0.03 ±106%  perf-profile.self.cycles-pp.xfs_read_agi
      0.22 ±  2%      -0.1        0.14 ± 18%  perf-profile.self.cycles-pp.link_path_walk
      0.11 ± 15%      -0.1        0.04 ±100%  perf-profile.self.cycles-pp.mutex_unlock
      0.12 ± 19%      -0.1        0.05 ± 76%  perf-profile.self.cycles-pp.__percpu_counter_compare
      0.14 ±  7%      -0.1        0.07 ± 52%  perf-profile.self.cycles-pp.drop_nlink
      0.11 ± 25%      -0.1        0.04 ± 70%  perf-profile.self.cycles-pp.xfs_inobt_get_maxrecs
      0.17 ±  7%      -0.1        0.10 ± 49%  perf-profile.self.cycles-pp.__switch_to_asm
      0.22 ±  3%      -0.1        0.15 ± 31%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      0.11 ±  6%      -0.1        0.05 ± 74%  perf-profile.self.cycles-pp.pagecache_get_page
      0.14 ± 12%      -0.1        0.08 ± 45%  perf-profile.self.cycles-pp.try_to_wake_up
      0.15 ±  7%      -0.1        0.09 ± 27%  perf-profile.self.cycles-pp.fsnotify_get_group
      0.10 ± 13%      -0.1        0.04 ± 72%  perf-profile.self.cycles-pp.update_cfs_group
      0.08 ± 17%      -0.1        0.03 ±100%  perf-profile.self.cycles-pp.schedule
      0.15 ±  7%      -0.1        0.10 ± 51%  perf-profile.self.cycles-pp.map_id_range_down
      0.14 ± 16%      -0.1        0.08 ± 46%  perf-profile.self.cycles-pp.queue_work_on
      0.08 ± 22%      -0.1        0.03 ±100%  perf-profile.self.cycles-pp.__fsnotify_parent
      0.12 ± 11%      -0.1        0.07 ± 45%  perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.14 ± 18%      -0.1        0.09 ± 28%  perf-profile.self.cycles-pp.enqueue_entity
      0.10 ±  6%      -0.0        0.05 ± 46%  perf-profile.self.cycles-pp.xfs_remove
      0.12 ± 11%      -0.0        0.08 ± 20%  perf-profile.self.cycles-pp.xfs_buffered_write_iomap_begin
      0.14 ±  5%      -0.0        0.10 ± 47%  perf-profile.self.cycles-pp.__entry_text_start
      0.08 ± 20%      -0.0        0.04 ± 71%  perf-profile.self.cycles-pp.task_work_add
      0.10 ± 18%      -0.0        0.06 ± 56%  perf-profile.self.cycles-pp.xfs_dir2_sf_addname
      0.12 ± 11%      -0.0        0.08 ± 33%  perf-profile.self.cycles-pp.do_dentry_open
      0.14 ±  5%      -0.0        0.09 ± 20%  perf-profile.self.cycles-pp.map_id_up
      0.10 ± 13%      -0.0        0.05 ± 49%  perf-profile.self.cycles-pp.fsnotify_add_mark_locked
      0.14 ±  9%      -0.0        0.10 ± 27%  perf-profile.self.cycles-pp.__fpu__restore_sig
      0.16 ± 10%      -0.0        0.11 ± 31%  perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      0.10 ±  9%      -0.0        0.06 ± 47%  perf-profile.self.cycles-pp.lockref_put_return
      0.09 ± 10%      -0.0        0.06 ± 47%  perf-profile.self.cycles-pp.generic_permission
      0.06 ± 17%      +0.0        0.10 ± 20%  perf-profile.self.cycles-pp.page_counter_try_charge
      0.12 ±  4%      +0.1        0.18 ± 14%  perf-profile.self.cycles-pp.__slab_free
      0.03 ±100%      +0.1        0.11 ± 30%  perf-profile.self.cycles-pp.call_cpuidle
      0.00            +0.1        0.08 ± 38%  perf-profile.self.cycles-pp.rcu_dynticks_eqs_exit
      0.10 ± 12%      +0.1        0.19 ± 20%  perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
      0.23 ±  5%      +0.1        0.32 ± 22%  perf-profile.self.cycles-pp.do_idle
      0.03 ± 99%      +0.1        0.12 ± 24%  perf-profile.self.cycles-pp.xfs_cil_prepare_item
      0.00            +0.1        0.10 ± 48%  perf-profile.self.cycles-pp.memcpy_toio
      0.09 ± 12%      +0.1        0.19 ± 28%  perf-profile.self.cycles-pp.update_sd_lb_stats
      0.00            +0.1        0.10 ± 33%  perf-profile.self.cycles-pp.asm_sysvec_apic_timer_interrupt
      0.00            +0.1        0.12 ± 27%  perf-profile.self.cycles-pp.allocate_slab
      0.06 ± 13%      +0.1        0.18 ± 29%  perf-profile.self.cycles-pp.__intel_pmu_enable_all
      0.00            +0.1        0.12 ± 62%  perf-profile.self.cycles-pp.perf_mux_hrtimer_handler
      0.15 ± 13%      +0.1        0.28 ± 22%  perf-profile.self.cycles-pp._raw_spin_trylock
      0.00            +0.1        0.13 ± 55%  perf-profile.self.cycles-pp.hrtimer_interrupt
      0.00            +0.1        0.13 ± 83%  perf-profile.self.cycles-pp.__hrtimer_run_queues
      0.02 ±142%      +0.1        0.15 ± 62%  perf-profile.self.cycles-pp.update_irq_load_avg
      0.00            +0.1        0.14 ± 38%  perf-profile.self.cycles-pp.xfs_buf_offset
      0.07 ± 10%      +0.1        0.22 ± 72%  perf-profile.self.cycles-pp.rcu_sched_clock_irq
      0.04 ± 45%      +0.1        0.19 ± 39%  perf-profile.self.cycles-pp.arch_scale_freq_tick
      0.11 ± 15%      +0.2        0.26 ± 28%  perf-profile.self.cycles-pp.calc_global_load_tick
      0.10 ± 67%      +0.2        0.27 ± 35%  perf-profile.self.cycles-pp.xfs_next_bit
      0.12 ±  7%      +0.2        0.30 ± 42%  perf-profile.self.cycles-pp.irqtime_account_irq
      0.00            +0.2        0.19 ± 37%  perf-profile.self.cycles-pp.xlog_write
      0.01 ±223%      +0.2        0.20 ± 74%  perf-profile.self.cycles-pp.__hrtimer_next_event_base
      0.00            +0.2        0.20 ± 32%  perf-profile.self.cycles-pp.xfs_iunlink_insert_backref
      0.12 ± 19%      +0.2        0.33 ± 24%  perf-profile.self.cycles-pp.update_process_times
      0.15 ± 11%      +0.2        0.36 ± 13%  perf-profile.self.cycles-pp.tick_nohz_next_event
      0.19 ±  3%      +0.2        0.42 ± 29%  perf-profile.self.cycles-pp.native_sched_clock
      0.06 ± 11%      +0.3        0.31 ± 51%  perf-profile.self.cycles-pp.io_serial_in
      0.09 ± 10%      +0.3        0.35 ± 34%  perf-profile.self.cycles-pp.ktime_get_update_offsets_now
      0.19 ±  6%      +0.3        0.46 ± 36%  perf-profile.self.cycles-pp.read_tsc
      0.34 ±  6%      +0.3        0.62 ± 22%  perf-profile.self.cycles-pp.native_irq_return_iret
      0.16 ± 13%      +0.4        0.53 ± 27%  perf-profile.self.cycles-pp.lapic_next_deadline
      0.97 ± 10%      +1.3        2.25 ± 19%  perf-profile.self.cycles-pp.ktime_get
      1.30 ±  9%      +2.2        3.55 ± 31%  perf-profile.self.cycles-pp.menu_select
      1.36 ±  9%      +3.0        4.33 ± 38%  perf-profile.self.cycles-pp.cpuidle_enter_state
     32.41 ±  2%      +5.8       38.22 ± 17%  perf-profile.self.cycles-pp.intel_idle


                                                                                
                            stress-ng.time.user_time                            
                                                                                
  12 +----------------------------------------------------------------------+   
  11 |-+::    +. +.   .+.     +.   +. .+.++.   + :  + +  .+.+.   +.+.   +. .|   
     |  : :  +  +  +.+   ++. +  + +  +      +.+  +.+   ++     +.+    +.+  + |   
  10 |.+  +.+               +    +                                          |   
   9 |-+                                                                    |   
     |                                                                      |   
   8 |-+                                                                    |   
   7 |-+                                                                    |   
   6 |-+                                                                    |   
     |                                                                      |   
   5 |-+                                                                    |   
   4 |-+                                                                    |   
     |    O                                     O  O O OO O                 |   
   3 |-O O  O O OO O O O OO O O OO O O O OO O O  O          O               |   
   2 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                            stress-ng.time.system_time                          
                                                                                
  450 +---------------------------------------------------------------------+   
      |.+.++   +.++.+.+.++.+.+.++.+.+.++.+.+.++.+.+.++.+.+.++.+.+.++.+.+.++.|   
  400 |-+                                                                   |   
  350 |-+                                                                   |   
      |                                                                     |   
  300 |-+                                                                   |   
      |                                                                     |   
  250 |-+                                                                   |   
      |                                                                     |   
  200 |-+                                                                   |   
  150 |-+                                                                   |   
      |                                                                     |   
  100 |-+ OO                                    O O OO O O OO               |   
      | O    O O OO O O OO O O OO O O OO O O OO                             |   
   50 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                    stress-ng.time.percent_of_cpu_this_job_got                  
                                                                                
  800 +---------------------------------------------------------------------+   
      |                                                                     |   
  700 |.+.++.+.+.++.+.+.++.+.+.++.+.+.++.+.+.++.+.+.++.+.+.++.+.+.++.+.+.++.|   
      |                                                                     |   
  600 |-+                                                                   |   
      |                                                                     |   
  500 |-+                                                                   |   
      |                                                                     |   
  400 |-+                                                                   |   
      |                                                                     |   
  300 |-+                                                                   |   
      |                                                                     |   
  200 |-+                                                                   |   
      | O OO O O OO O O OO O O OO O O OO O O OO O O OO O O OO               |   
  100 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                       stress-ng.time.voluntary_context_switches                
                                                                                
  1.6e+07 +-----------------------------------------------------------------+   
          |                                                                 |   
  1.4e+07 |-+        .+   .+         .+. +. .+   .++. .++.         .+       |   
          |.++.+.++.+  :.+  +.++.+.++   +  +  +.+    +    +.++.++.+  +.+.++.|   
  1.2e+07 |-+          +                                                    |   
          |                                                                 |   
    1e+07 |-+                                                               |   
          |                                                                 |   
    8e+06 |-+                                                               |   
          |                                                                 |   
    6e+06 |-+                                                               |   
          |                                                                 |   
    4e+06 |-+                                                               |   
          | OO O OO O OO O OO OO O OO O OO O OO    O O OO O OO              |   
    2e+06 +-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                     stress-ng.time.involuntary_context_switches                
                                                                                
  55000 +-------------------------------------------------------------------+   
  50000 |.+:+ .+.  .+.+ .+.  .+.+.+     +. + :.+. .++. .++.          +.     |   
        | +  +   ++    +   ++      +.+.+  +  +   +    +    +.++.+.+.+  +.++.|   
  45000 |-+                                                                 |   
  40000 |-+                                                                 |   
        |                                                                   |   
  35000 |-+                                                                 |   
  30000 |-+                                                                 |   
  25000 |-+                                                                 |   
        |                                                                   |   
  20000 |-+                                                                 |   
  15000 |-+                                                                 |   
        |                                          OO   O  O                |   
  10000 |-OO O O OO O OO O OO O O OO O OO O OO O O    O  O   O              |   
   5000 +-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                          stress-ng.time.file_system_outputs                    
                                                                                
  5.5e+07 +-----------------------------------------------------------------+   
          |.    .++. .+ .+.+ .+ .+.  .+. +. .+   .+   .+ .+.    +.+.+   .+  |   
    5e+07 |-++.+    +  +    +  +   ++   +  +  +.+  +.+  +   ++.+     +.+  :+|   
  4.5e+07 |-+                                                             + |   
          |                                                                 |   
    4e+07 |-+                                                               |   
  3.5e+07 |-+                                                               |   
          |                                                                 |   
    3e+07 |-+                                                               |   
  2.5e+07 |-+                                                               |   
          |                                                                 |   
    2e+07 |-+                                                               |   
  1.5e+07 |-+                                                               |   
          |  O O                    O    O      O OO O OO O OO              |   
    1e+07 +-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                 stress-ng.dnotify.ops                          
                                                                                
  1.4e+06 +-----------------------------------------------------------------+   
          |.    .++. .+   .+    .+.  .+. +. .+   .+   .+        +. .+       |   
  1.2e+06 |-++.+    +  +.+  +.++   ++   +  +  +.+  +.+  +.+.++.+  +  +.+.+ +|   
          |                                                               + |   
          |                                                                 |   
    1e+06 |-+                                                               |   
          |                                                                 |   
   800000 |-+                                                               |   
          |                                                                 |   
   600000 |-+                                                               |   
          |                                                                 |   
          |                                                                 |   
   400000 |-+                                     OO O  O   O               |   
          | OO O OO O OO O OO OO O OO O OO O OO O      O  O  O              |   
   200000 +-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                            stress-ng.dnotify.ops_per_sec                       
                                                                                
  24000 +-------------------------------------------------------------------+   
  22000 |-+         +.                           +    +                     |   
        |. +. .+.+ +  ++.+. +. .+.  .+. +. .++. + +  + + +.+.  .+.+.+ .+.+ .|   
  20000 |-+  +    +        +  +   ++   +  +    +   ++   +    ++      +    + |   
  18000 |-+                                                                 |   
        |                                                                   |   
  16000 |-+                                                                 |   
  14000 |-+                                                                 |   
  12000 |-+                                                                 |   
        |                                                                   |   
  10000 |-+                                                                 |   
   8000 |-+                                                                 |   
        |                                                                   |   
   6000 |-+O O                     O    O      O O OO O OO O O              |   
   4000 +-------------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample



Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


---
0DAY/LKP+ Test Infrastructure                   Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org       Intel Corporation

Thanks,
Oliver Sang


View attachment "config-5.12.0-rc1-00064-g7f835610979b" of type "text/plain" (172871 bytes)

View attachment "job-script" of type "text/plain" (8405 bytes)

View attachment "job.yaml" of type "text/plain" (5780 bytes)

View attachment "reproduce" of type "text/plain" (554 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ