lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Date:   Thu, 13 May 2021 23:30:51 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     Christoph Hellwig <hch@....de>
Cc:     Jens Axboe <axboe@...nel.dk>, LKML <linux-kernel@...r.kernel.org>,
        lkp@...ts.01.org, lkp@...el.com, ying.huang@...el.com,
        feng.tang@...el.com, zhengjun.xing@...el.com
Subject: [block]  c76f48eb5c:  stress-ng.loop.ops_per_sec -99.9% regression



Greeting,

FYI, we noticed a -99.9% regression of stress-ng.loop.ops_per_sec due to commit:


commit: c76f48eb5c084b1e15c931ae8cc1826cd771d70d ("block: take bd_mutex around delete_partitions in del_gendisk")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master


in testcase: stress-ng
on test machine: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 192G memory
with following parameters:

	nr_threads: 10%
	disk: 1HDD
	testtime: 60s
	fs: ext4
	class: os
	test: loop
	cpufreq_governor: performance
	ucode: 0x5003006


In addition to that, the commit also has significant impact on the following tests:

+------------------+--------------------------------------------------------------------------------+
| testcase: change | stress-ng: stress-ng.loop.ops_per_sec -99.4% regression                        |
| test machine     | 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory |
| test parameters  | class=device                                                                   |
|                  | cpufreq_governor=performance                                                   |
|                  | disk=1HDD                                                                      |
|                  | nr_threads=100%                                                                |
|                  | test=loop                                                                      |
|                  | testtime=60s                                                                   |
|                  | ucode=0x5003006                                                                |
+------------------+--------------------------------------------------------------------------------+
| testcase: change | stress-ng: stress-ng.loop.ops_per_sec -99.5% regression                        |
| test machine     | 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory |
| test parameters  | class=device                                                                   |
|                  | cpufreq_governor=performance                                                   |
|                  | disk=1HDD                                                                      |
|                  | nr_threads=100%                                                                |
|                  | testtime=60s                                                                   |
|                  | ucode=0x5003006                                                                |
+------------------+--------------------------------------------------------------------------------+


If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang@...el.com>


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install                job.yaml  # job file is attached in this email
        bin/lkp split-job --compatible job.yaml  # generate the yaml file for lkp run
        bin/lkp run                    generated-yaml-file

=========================================================================================
class/compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode:
  os/gcc-9/performance/1HDD/ext4/x86_64-rhel-8.3/10%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp5/loop/stress-ng/60s/0x5003006

commit: 
  d3c4a43d92 ("block: refactor blk_drop_partitions")
  c76f48eb5c ("block: take bd_mutex around delete_partitions in del_gendisk")

d3c4a43d9291279c c76f48eb5c084b1e15c931ae8cc 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
      1460 ±  3%     -99.2%      12.17 ± 28%  stress-ng.loop.ops
     24.28 ±  3%     -99.9%       0.03 ± 47%  stress-ng.loop.ops_per_sec
     62.31          +634.2%     457.48 ± 18%  stress-ng.time.elapsed_time
     62.31          +634.2%     457.48 ± 18%  stress-ng.time.elapsed_time.max
      3687 ±  4%     -99.0%      35.67 ± 21%  stress-ng.time.involuntary_context_switches
      9790            -1.4%       9650        stress-ng.time.minor_page_faults
     36.33 ±  4%    -100.0%       0.00        stress-ng.time.percent_of_cpu_this_job_got
     22.97 ±  4%     -98.9%       0.25 ± 26%  stress-ng.time.system_time
     54565 ± 14%     -98.1%       1048 ± 59%  stress-ng.time.voluntary_context_switches
      2.11 ±  3%     -16.6%       1.76 ±  2%  iostat.cpu.system
     15.50            -1.0%      15.34        boot-time.dhcp
      2249            -4.0%       2160 ±  2%  boot-time.idle
    179396 ± 17%    +228.3%     588985 ± 41%  numa-numastat.node0.local_node
    214097 ±  5%    +193.4%     628215 ± 39%  numa-numastat.node0.numa_hit
    106.77 ±  6%    +365.7%     497.26 ± 17%  uptime.boot
      9731 ±  6%    +378.1%      46529 ± 17%  uptime.idle
      0.58 ±  9%      -0.6        0.02 ± 10%  mpstat.cpu.all.iowait%
      1.30 ±  4%      +0.3        1.62 ±  2%  mpstat.cpu.all.irq%
      0.59 ±  5%      -0.6        0.03 ±  4%  mpstat.cpu.all.sys%
      0.08 ±  4%      -0.1        0.01 ± 11%  mpstat.cpu.all.usr%
      1571 ±  4%     -99.9%       1.83 ± 37%  vmstat.io.bi
      5186 ±  4%     -68.7%       1620 ±  3%  vmstat.memory.buff
      8151 ±  3%     -83.1%       1376        vmstat.system.cs
    189078            +1.4%     191737        vmstat.system.in
   5929338 ±  5%     -50.7%    2922966 ±  4%  cpuidle.C1.time
     67292 ±  6%     -29.7%      47309 ± 11%  cpuidle.C1.usage
 4.335e+09 ± 32%    +725.3%  3.578e+10 ± 25%  cpuidle.C1E.time
  10004764 ± 19%    +681.6%   78192921 ± 20%  cpuidle.C1E.usage
     41558 ±  2%    +181.5%     116998 ± 16%  cpuidle.POLL.usage
      1011 ± 35%     -76.9%     234.00 ± 58%  numa-vmstat.node0.nr_active_file
      9.17 ±223%   +1625.5%     158.17 ± 59%  numa-vmstat.node0.nr_mlock
     12430 ±  8%     -15.6%      10487 ±  5%  numa-vmstat.node0.nr_slab_reclaimable
     29413 ±  6%     -22.2%      22871 ±  6%  numa-vmstat.node0.nr_slab_unreclaimable
      1011 ± 35%     -76.9%     234.00 ± 58%  numa-vmstat.node0.nr_zone_active_file
    540.50 ± 10%    +266.8%       1982 ± 22%  numa-vmstat.node1.nr_active_anon
      8778 ±  4%     -11.8%       7745 ±  5%  numa-vmstat.node1.nr_kernel_stack
     12.67 ±223%    +923.7%     129.67 ± 49%  numa-vmstat.node1.nr_mlock
    540.50 ± 10%    +266.8%       1982 ± 22%  numa-vmstat.node1.nr_zone_active_anon
      4040 ± 35%     -76.8%     938.50 ± 58%  numa-meminfo.node0.Active(file)
     49724 ±  8%     -15.6%      41952 ±  5%  numa-meminfo.node0.KReclaimable
   1517822 ±  2%     -24.9%    1139967 ±  5%  numa-meminfo.node0.MemUsed
     39.67 ±206%   +1497.1%     633.50 ± 59%  numa-meminfo.node0.Mlocked
     49724 ±  8%     -15.6%      41952 ±  5%  numa-meminfo.node0.SReclaimable
    117651 ±  6%     -22.2%      91486 ±  6%  numa-meminfo.node0.SUnreclaim
    167376 ±  6%     -20.3%     133439 ±  4%  numa-meminfo.node0.Slab
      4333 ± 38%    +107.3%       8982 ± 19%  numa-meminfo.node1.Active
      2162 ± 10%    +266.9%       7935 ± 22%  numa-meminfo.node1.Active(anon)
      8787 ±  4%     -11.9%       7741 ±  5%  numa-meminfo.node1.KernelStack
     64.33 ±  4%     -30.3%      44.83 ± 12%  turbostat.Avg_MHz
      1639 ±  2%     -36.9%       1034        turbostat.Bzy_MHz
     62850 ±  4%     -38.4%      38729 ± 17%  turbostat.C1
      0.09 ±  7%      -0.1        0.01 ± 44%  turbostat.C1%
  10004024 ± 19%    +681.6%   78191627 ± 20%  turbostat.C1E
      0.82 ±173%     -96.7%       0.03 ± 17%  turbostat.CPU%c6
     46.67 ±  3%      -8.2%      42.83 ±  3%  turbostat.CoreTmp
  12568851 ±  2%    +602.3%   88274702 ± 18%  turbostat.IRQ
     48.00 ±  2%      -7.3%      44.50 ±  2%  turbostat.PkgTmp
    121.79           -30.8%      84.34 ±  5%  turbostat.PkgWatt
      9534 ±  3%     +40.2%      13369 ± 10%  meminfo.Active
      3329 ±  2%    +241.9%      11383 ± 11%  meminfo.Active(anon)
      6204 ±  3%     -68.0%       1986 ±  4%  meminfo.Active(file)
     94153 ± 11%     +86.3%     175387 ±  2%  meminfo.AnonHugePages
      5114 ±  4%     -68.3%       1619 ±  3%  meminfo.Buffers
    950.67 ±  6%     +41.6%       1345 ± 22%  meminfo.Inactive(file)
     18709           -11.4%      16572 ±  2%  meminfo.KernelStack
   2861608           -15.8%    2408092        meminfo.Memused
     93.50 ±207%   +1134.0%       1153 ± 54%  meminfo.Mlocked
      5593           -12.1%       4917        meminfo.PageTables
     23126           +36.2%      31510 ±  4%  meminfo.Shmem
    832.33 ±  2%    +241.9%       2845 ± 11%  proc-vmstat.nr_active_anon
      1553 ±  3%     -68.1%     496.17 ±  4%  proc-vmstat.nr_active_file
     65178            -7.5%      60279        proc-vmstat.nr_anon_pages
     73.67 ±  8%     +62.4%     119.67 ± 21%  proc-vmstat.nr_dirtied
     70173            -7.1%      65156        proc-vmstat.nr_inactive_anon
    240.67 ±  7%     +39.6%     336.00 ± 22%  proc-vmstat.nr_inactive_file
     18703           -11.4%      16574 ±  2%  proc-vmstat.nr_kernel_stack
     10130            -3.2%       9806        proc-vmstat.nr_mapped
     22.67 ±213%   +1171.3%     288.17 ± 54%  proc-vmstat.nr_mlock
      1396           -12.2%       1226        proc-vmstat.nr_page_table_pages
      5781           +36.2%       7877 ±  4%  proc-vmstat.nr_shmem
     24121            -6.2%      22619        proc-vmstat.nr_slab_reclaimable
     49566            -7.2%      45987        proc-vmstat.nr_slab_unreclaimable
     60.83           +85.2%     112.67 ± 27%  proc-vmstat.nr_written
    832.33 ±  2%    +241.9%       2845 ± 11%  proc-vmstat.nr_zone_active_anon
      1553 ±  3%     -68.1%     496.17 ±  4%  proc-vmstat.nr_zone_active_file
     70173            -7.1%      65156        proc-vmstat.nr_zone_inactive_anon
    240.67 ±  7%     +39.6%     336.00 ± 22%  proc-vmstat.nr_zone_inactive_file
    467764          +154.3%    1189540 ± 14%  proc-vmstat.numa_hit
    379011          +190.9%    1102732 ± 15%  proc-vmstat.numa_local
     88753            -2.2%      86808        proc-vmstat.numa_other
      7810 ±130%    +218.2%      24852 ± 86%  proc-vmstat.numa_pte_updates
      9485 ±  3%     -65.3%       3294 ± 10%  proc-vmstat.pgactivate
    562092          +123.2%    1254854 ± 13%  proc-vmstat.pgalloc_normal
    232165 ±  2%    +475.4%    1335901 ± 17%  proc-vmstat.pgfault
    424165          +192.7%    1241418 ± 13%  proc-vmstat.pgfree
    104618 ±  3%     -99.0%       1040 ± 27%  proc-vmstat.pgpgin
    465899 ± 15%    +562.5%    3086805 ± 28%  proc-vmstat.pgpgout
     15086 ±  2%    +490.4%      89066 ± 18%  proc-vmstat.pgreuse
     10368 ± 45%    +261.8%      37510 ± 19%  sched_debug.cfs_rq:/.load.avg
    437084 ± 53%     +76.5%     771654 ±  4%  sched_debug.cfs_rq:/.load.max
     55417 ± 46%    +158.9%     143481 ±  9%  sched_debug.cfs_rq:/.load.stddev
    423.26 ± 56%     -87.2%      53.98 ± 15%  sched_debug.cfs_rq:/.load_avg.avg
     20090 ± 66%     -92.4%       1536 ± 10%  sched_debug.cfs_rq:/.load_avg.max
      2552 ± 65%     -91.0%     230.91 ± 12%  sched_debug.cfs_rq:/.load_avg.stddev
    236.93 ± 32%     -68.3%      75.01 ± 15%  sched_debug.cfs_rq:/.runnable_avg.avg
      1063 ± 17%     -36.6%     673.90 ±  7%  sched_debug.cfs_rq:/.runnable_avg.max
    225.09 ± 23%     -41.2%     132.27 ±  8%  sched_debug.cfs_rq:/.runnable_avg.stddev
    236.00 ± 32%     -68.3%      74.90 ± 15%  sched_debug.cfs_rq:/.util_avg.avg
      1063 ± 17%     -36.6%     673.50 ±  7%  sched_debug.cfs_rq:/.util_avg.max
    224.45 ± 23%     -41.1%     132.15 ±  8%  sched_debug.cfs_rq:/.util_avg.stddev
     18.88 ± 32%     -66.4%       6.34 ± 21%  sched_debug.cfs_rq:/.util_est_enqueued.avg
    392.50 ± 17%     -64.3%     140.07 ± 22%  sched_debug.cfs_rq:/.util_est_enqueued.max
     74.78 ± 25%     -65.3%      25.92 ± 14%  sched_debug.cfs_rq:/.util_est_enqueued.stddev
    831245 ±  4%     +12.1%     931711        sched_debug.cpu.avg_idle.avg
   1368408 ± 11%     -21.6%    1072417 ±  2%  sched_debug.cpu.avg_idle.max
    235908 ± 17%     -38.0%     146247 ± 11%  sched_debug.cpu.avg_idle.stddev
     67132 ±  7%    +255.4%     238559 ± 15%  sched_debug.cpu.clock.avg
     67137 ±  7%    +255.3%     238562 ± 15%  sched_debug.cpu.clock.max
     67126 ±  7%    +255.4%     238555 ± 15%  sched_debug.cpu.clock.min
      3.23 ± 25%     -32.5%       2.18 ±  5%  sched_debug.cpu.clock.stddev
     66636 ±  6%    +252.6%     234974 ± 15%  sched_debug.cpu.clock_task.avg
     66777 ±  6%    +252.3%     235236 ± 15%  sched_debug.cpu.clock_task.max
     62002 ±  7%    +271.3%     230245 ± 15%  sched_debug.cpu.clock_task.min
    212.96 ± 19%     -35.4%     137.64 ±  9%  sched_debug.cpu.curr->pid.avg
      3577 ± 14%    +111.2%       7556 ± 12%  sched_debug.cpu.curr->pid.max
    712.24 ±  8%     +24.2%     884.78 ± 10%  sched_debug.cpu.curr->pid.stddev
    697232 ±  8%     -21.9%     544502        sched_debug.cpu.max_idle_balance_cost.max
     27525 ± 21%     -73.9%       7190 ± 30%  sched_debug.cpu.max_idle_balance_cost.stddev
      0.00 ± 20%     -44.7%       0.00 ±  3%  sched_debug.cpu.next_balance.stddev
      0.09 ± 21%     -57.9%       0.04 ±  9%  sched_debug.cpu.nr_running.avg
      0.27 ± 12%     -35.6%       0.17 ±  3%  sched_debug.cpu.nr_running.stddev
     15057 ± 24%    +329.0%      64594 ± 21%  sched_debug.cpu.nr_switches.max
      2284 ± 22%    +219.1%       7290 ± 19%  sched_debug.cpu.nr_switches.stddev
      0.07 ± 34%     +65.8%       0.12 ±  4%  sched_debug.cpu.nr_uninterruptible.avg
     67129 ±  7%    +255.4%     238555 ± 15%  sched_debug.cpu_clk
     66635 ±  7%    +257.3%     238060 ± 15%  sched_debug.ktime
     67490 ±  7%    +254.0%     238918 ± 15%  sched_debug.sched_clk
 3.418e+08 ±  3%     -48.7%  1.754e+08 ±  7%  perf-stat.i.branch-instructions
   5554030 ± 33%     -47.2%    2935198 ±  9%  perf-stat.i.branch-misses
  21039609 ± 33%     -42.6%   12083131 ±  8%  perf-stat.i.cache-references
      8238 ±  3%     -83.7%       1341        perf-stat.i.context-switches
      3.59 ±  3%     +28.8%       4.62 ± 10%  perf-stat.i.cpi
 5.649e+09 ±  6%     -32.4%  3.818e+09 ± 18%  perf-stat.i.cpu-cycles
    160.13           -38.7%      98.17        perf-stat.i.cpu-migrations
      1768 ± 15%     -23.5%       1352 ±  8%  perf-stat.i.cycles-between-cache-misses
 4.183e+08 ±  2%     -45.5%  2.282e+08 ±  4%  perf-stat.i.dTLB-loads
  2.07e+08 ±  2%     -40.7%  1.228e+08 ±  3%  perf-stat.i.dTLB-stores
 1.627e+09 ±  3%     -48.2%  8.418e+08 ±  7%  perf-stat.i.instructions
      1017 ±  9%     -47.8%     530.70 ±  7%  perf-stat.i.instructions-per-iTLB-miss
      0.29 ±  4%     -22.5%       0.22 ± 10%  perf-stat.i.ipc
     11.04 ±  3%     -80.2%       2.19 ± 17%  perf-stat.i.major-faults
      0.06 ±  6%     -32.4%       0.04 ± 18%  perf-stat.i.metric.GHz
    248.80 ± 29%     -40.4%     148.35 ±  7%  perf-stat.i.metric.K/sec
     10.07 ±  2%     -45.6%       5.48 ±  5%  perf-stat.i.metric.M/sec
      3169           -11.5%       2806        perf-stat.i.minor-faults
    491422 ±  7%     -51.9%     236374 ±  3%  perf-stat.i.node-load-misses
     85385 ±  7%     -48.8%      43677 ± 34%  perf-stat.i.node-loads
    270636 ±  7%     -87.0%      35097 ±  8%  perf-stat.i.node-store-misses
     65751 ± 18%     -90.4%       6330 ± 48%  perf-stat.i.node-stores
      3212           -12.6%       2808        perf-stat.i.page-faults
      3.47 ±  3%     +29.8%       4.51 ± 11%  perf-stat.overall.cpi
      1746 ± 14%     -22.7%       1349 ±  8%  perf-stat.overall.cycles-between-cache-misses
      1018 ± 10%     -47.8%     531.43 ±  7%  perf-stat.overall.instructions-per-iTLB-miss
      0.29 ±  4%     -22.2%       0.22 ± 10%  perf-stat.overall.ipc
 3.364e+08 ±  3%     -48.0%  1.751e+08 ±  7%  perf-stat.ps.branch-instructions
   5466172 ± 33%     -46.4%    2928478 ±  9%  perf-stat.ps.branch-misses
  20711633 ± 33%     -41.8%   12055873 ±  8%  perf-stat.ps.cache-references
      8111 ±  3%     -83.5%       1338        perf-stat.ps.context-switches
     94513            +1.3%      95782        perf-stat.ps.cpu-clock
 5.561e+09 ±  6%     -31.5%   3.81e+09 ± 18%  perf-stat.ps.cpu-cycles
    157.63           -37.9%      97.94        perf-stat.ps.cpu-migrations
 4.117e+08 ±  2%     -44.7%  2.276e+08 ±  4%  perf-stat.ps.dTLB-loads
 2.037e+08 ±  2%     -39.9%  1.225e+08 ±  3%  perf-stat.ps.dTLB-stores
 1.601e+09 ±  3%     -47.5%  8.399e+08 ±  7%  perf-stat.ps.instructions
     10.91 ±  3%     -79.9%       2.19 ± 18%  perf-stat.ps.major-faults
      3119           -10.3%       2800        perf-stat.ps.minor-faults
    483738 ±  7%     -51.2%     235839 ±  3%  perf-stat.ps.node-load-misses
     84032 ±  7%     -48.1%      43579 ± 34%  perf-stat.ps.node-loads
    266371 ±  7%     -86.9%      35018 ±  8%  perf-stat.ps.node-store-misses
     64694 ± 18%     -90.2%       6316 ± 48%  perf-stat.ps.node-stores
      3162           -11.4%       2802        perf-stat.ps.page-faults
     94513            +1.3%      95782        perf-stat.ps.task-clock
 1.033e+11 ±  3%    +275.1%  3.875e+11 ± 22%  perf-stat.total.instructions
      0.00 ±223%    +812.5%       0.01 ±  8%  perf-sched.sch_delay.avg.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
      0.00 ±223%    +622.2%       0.01 ± 30%  perf-sched.sch_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      0.00 ±223%   +1240.0%       0.01 ±  8%  perf-sched.sch_delay.avg.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
      0.00 ± 58%    +428.6%       0.01 ±  6%  perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.00 ± 35%    +182.6%       0.01 ± 31%  perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown]
      0.01 ± 25%     +86.7%       0.01 ±  7%  perf-sched.sch_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      0.78 ± 52%     -98.4%       0.01 ±  3%  perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
      0.00 ±223%   +1387.5%       0.02 ± 34%  perf-sched.sch_delay.max.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
      0.00 ±223%   +1840.0%       0.02 ± 23%  perf-sched.sch_delay.max.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64
      0.00 ± 49%    +507.4%       0.03 ± 10%  perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.01 ± 25%    +204.4%       0.02 ± 13%  perf-sched.sch_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      2.86 ±  2%     -11.2%       2.54 ±  2%  perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
      0.02 ± 70%     -77.5%       0.00 ±  7%  perf-sched.total_sch_delay.average.ms
      9.11 ± 68%     -72.1%       2.54 ±  2%  perf-sched.total_sch_delay.max.ms
      0.64 ± 22%  +35705.1%     229.93 ±  6%  perf-sched.total_wait_and_delay.average.ms
    607.17 ± 16%    +970.0%       6496 ±  6%  perf-sched.total_wait_and_delay.count.ms
     14.55 ± 16%  +68571.9%       9993        perf-sched.total_wait_and_delay.max.ms
      0.62 ± 22%  +36944.6%     229.92 ±  6%  perf-sched.total_wait_time.average.ms
     13.56 ± 15%  +73578.5%       9993        perf-sched.total_wait_time.max.ms
      0.10 ±113%  +2.8e+05%     265.78 ±  4%  perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.04 ± 12%  +1.8e+05%      75.77 ±  2%  perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.09 ± 62%  +19218.8%      18.00        perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
      2.92 ± 12%    +702.1%      23.39 ± 38%  perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      2.07 ± 24%  +34144.2%     709.14        perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
      1.58 ±121%  +28865.6%     456.79 ±  3%  perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
      5.33 ± 41%   +4475.0%     244.00        perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
    264.17 ± 28%    +704.0%       2123 ±  2%  perf-sched.wait_and_delay.count.pipe_read.new_sync_read.vfs_read.ksys_read
    158.17 ± 14%    +567.8%       1056        perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
      4.17 ± 25%  +13852.0%     581.33 ± 84%  perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
    158.00 ± 14%    +691.0%       1249        perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
      4.83 ± 49%  +12186.2%     593.83 ±  2%  perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
      0.31 ±119%  +7.6e+05%       2368 ± 60%  perf-sched.wait_and_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.95 ±113%  +1.1e+05%       1006        perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      6.92 ± 72%  +1.4e+05%       9993        perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
      4.98         +6280.0%     317.66 ± 26%  perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
     13.34 ± 18%  +16218.6%       2176 ±120%  perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
      5.25 ±102%  +1.7e+05%       8791 ±  5%  perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork
      0.21 ±223%    +498.7%       1.27 ± 10%  perf-sched.wait_time.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      0.10 ±114%  +2.8e+05%     265.77 ±  4%  perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.04 ± 12%  +1.9e+05%      75.77 ±  2%  perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.09 ± 62%  +19218.6%      18.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
      0.96 ± 51%  +13556.5%     131.06 ± 65%  perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      2.91 ± 12%    +703.1%      23.38 ± 38%  perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      2.06 ± 24%  +34407.7%     709.13        perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
      0.80 ±223%  +57092.1%     456.77 ±  3%  perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
      0.42 ±223%    +498.5%       2.54 ± 10%  perf-sched.wait_time.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      0.31 ±120%  +7.6e+05%       2368 ± 60%  perf-sched.wait_time.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.88 ±106%  +1.1e+05%       1006        perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      6.92 ± 72%  +1.4e+05%       9993        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
      2.63 ± 51%  +15737.4%     416.97 ± 28%  perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      4.97         +6285.5%     317.65 ± 26%  perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
     13.33 ± 18%  +16224.4%       2176 ±120%  perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
      2.40 ±223%  +3.7e+05%       8791 ±  5%  perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork
      3620 ±  6%     -55.1%       1626 ± 10%  slabinfo.Acpi-Parse.active_objs
      3620 ±  6%     -55.1%       1626 ± 10%  slabinfo.Acpi-Parse.num_objs
      1563           -83.7%     255.17 ± 18%  slabinfo.bdev_cache.active_objs
      1563           -83.7%     255.17 ± 18%  slabinfo.bdev_cache.num_objs
      2509 ±  7%     -43.5%       1418 ±  3%  slabinfo.buffer_head.active_objs
      2526 ±  7%     -42.8%       1445 ±  3%  slabinfo.buffer_head.num_objs
    101957           +10.7%     112908        slabinfo.dentry.active_objs
      2455           +10.3%       2707        slabinfo.dentry.active_slabs
    103132           +10.3%     113736        slabinfo.dentry.num_objs
      2455           +10.3%       2707        slabinfo.dentry.num_slabs
      3687 ±  4%     -13.6%       3187 ±  2%  slabinfo.dmaengine-unmap-16.active_objs
      3695 ±  5%     -13.7%       3188 ±  2%  slabinfo.dmaengine-unmap-16.num_objs
      2479 ±  3%     -91.9%     200.83 ± 11%  slabinfo.dquot.active_objs
      2479 ±  3%     -91.9%     200.83 ± 11%  slabinfo.dquot.num_objs
      5044 ± 15%     -66.2%       1702 ±  7%  slabinfo.ext4_extent_status.active_objs
      5044 ± 15%     -66.2%       1702 ±  7%  slabinfo.ext4_extent_status.num_objs
      2280 ±  5%     -45.3%       1247 ±  8%  slabinfo.file_lock_cache.active_objs
      2280 ±  5%     -45.3%       1247 ±  8%  slabinfo.file_lock_cache.num_objs
      6722 ±  7%     -46.9%       3567 ±  8%  slabinfo.fsnotify_mark_connector.active_objs
      6722 ±  7%     -46.9%       3567 ±  8%  slabinfo.fsnotify_mark_connector.num_objs
     76665           -10.2%      68851        slabinfo.inode_cache.active_objs
      1427           -10.4%       1279        slabinfo.inode_cache.active_slabs
     77097           -10.4%      69104        slabinfo.inode_cache.num_objs
      1427           -10.4%       1279        slabinfo.inode_cache.num_slabs
     99282           -14.2%      85192        slabinfo.kernfs_node_cache.active_objs
      3106           -14.3%       2661        slabinfo.kernfs_node_cache.active_slabs
     99425           -14.3%      85192        slabinfo.kernfs_node_cache.num_objs
      3106           -14.3%       2661        slabinfo.kernfs_node_cache.num_slabs
     45597           -19.8%      36570 ±  2%  slabinfo.kmalloc-16.num_objs
      7111           -19.7%       5710 ±  2%  slabinfo.kmalloc-192.num_objs
      6524            -8.8%       5949        slabinfo.kmalloc-1k.active_objs
      7525 ±  2%     -19.4%       6064 ±  2%  slabinfo.kmalloc-1k.num_objs
      8074            -9.1%       7340        slabinfo.kmalloc-2k.num_objs
      2245           -13.9%       1934        slabinfo.kmalloc-4k.num_objs
      6161           +26.4%       7788 ±  2%  slabinfo.kmalloc-96.active_objs
      7117           +11.3%       7921 ±  2%  slabinfo.kmalloc-96.num_objs
      3261           -84.0%     522.50 ± 18%  slabinfo.kmalloc-rcl-192.active_objs
      3261           -83.8%     529.17 ± 17%  slabinfo.kmalloc-rcl-192.num_objs
      5425 ±  2%     -11.5%       4802        slabinfo.kmalloc-rcl-512.active_objs
      5432 ±  2%     -11.6%       4802        slabinfo.kmalloc-rcl-512.num_objs
      6594 ±  6%     -23.4%       5049 ±  4%  slabinfo.kmalloc-rcl-64.active_objs
      6594 ±  6%     -23.4%       5049 ±  4%  slabinfo.kmalloc-rcl-64.num_objs
      2679 ±  5%     -13.0%       2330 ±  4%  slabinfo.kmalloc-rcl-96.active_objs
      2679 ±  5%     -13.0%       2330 ±  4%  slabinfo.kmalloc-rcl-96.num_objs
      6254           -76.7%       1459 ± 19%  slabinfo.mbcache.active_objs
      6254           -76.7%       1459 ± 19%  slabinfo.mbcache.num_objs
      1847 ±  7%     -18.7%       1501 ±  7%  slabinfo.pool_workqueue.active_objs
      1847 ±  7%     -18.7%       1501 ±  7%  slabinfo.pool_workqueue.num_objs
     13669           +11.3%      15221        slabinfo.proc_inode_cache.active_objs
     13669           +11.4%      15229        slabinfo.proc_inode_cache.num_objs
      1241           -88.1%     147.83 ± 23%  slabinfo.request_queue.active_objs
      1241           -88.1%     147.83 ± 23%  slabinfo.request_queue.num_objs
    536.50 ±  8%     -38.7%     328.83 ±  6%  slabinfo.skbuff_fclone_cache.active_objs
    536.50 ±  8%     -38.7%     328.83 ±  6%  slabinfo.skbuff_fclone_cache.num_objs
      4376 ±  4%      -9.2%       3974 ±  5%  slabinfo.skbuff_head_cache.active_objs
      4380 ±  4%      -9.2%       3979 ±  5%  slabinfo.skbuff_head_cache.num_objs
      1417            -9.5%       1283 ±  2%  slabinfo.task_struct.active_objs
      1421            -9.3%       1289 ±  2%  slabinfo.task_struct.active_slabs
      1421            -9.3%       1289 ±  2%  slabinfo.task_struct.num_objs
      1421            -9.3%       1289 ±  2%  slabinfo.task_struct.num_slabs
      4881 ±  4%     -22.0%       3805 ±  2%  slabinfo.trace_event_file.active_objs
      4881 ±  4%     -22.0%       3805 ±  2%  slabinfo.trace_event_file.num_objs
     11001           +47.7%      16252 ±  2%  slabinfo.vmap_area.active_objs
     11013           +48.4%      16346 ±  2%  slabinfo.vmap_area.num_objs
    130.67 ±  2%    +603.3%     919.00 ± 18%  interrupts.9:IO-APIC.9-fasteoi.acpi
     63882           +19.8%      76523 ±  5%  interrupts.CAL:Function_call_interrupts
    130205 ±  2%    +605.2%     918262 ± 18%  interrupts.CPU0.LOC:Local_timer_interrupts
    130.67 ±  2%    +603.3%     919.00 ± 18%  interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi
    129768 ±  2%    +607.6%     918227 ± 18%  interrupts.CPU1.LOC:Local_timer_interrupts
    129809 ±  2%    +607.4%     918306 ± 18%  interrupts.CPU10.LOC:Local_timer_interrupts
    231.50 ± 26%     -50.1%     115.50 ± 29%  interrupts.CPU10.NMI:Non-maskable_interrupts
    231.50 ± 26%     -50.1%     115.50 ± 29%  interrupts.CPU10.PMI:Performance_monitoring_interrupts
    129961 ±  2%    +606.5%     918153 ± 18%  interrupts.CPU11.LOC:Local_timer_interrupts
    129679 ±  2%    +608.1%     918235 ± 18%  interrupts.CPU12.LOC:Local_timer_interrupts
    129750 ±  2%    +607.7%     918201 ± 18%  interrupts.CPU13.LOC:Local_timer_interrupts
    129813 ±  2%    +607.4%     918271 ± 18%  interrupts.CPU14.LOC:Local_timer_interrupts
    129839 ±  2%    +607.2%     918158 ± 18%  interrupts.CPU15.LOC:Local_timer_interrupts
    130107 ±  2%    +605.7%     918148 ± 18%  interrupts.CPU16.LOC:Local_timer_interrupts
    201.67 ± 37%     -44.3%     112.33 ± 28%  interrupts.CPU16.NMI:Non-maskable_interrupts
    201.67 ± 37%     -44.3%     112.33 ± 28%  interrupts.CPU16.PMI:Performance_monitoring_interrupts
    129746 ±  2%    +607.6%     918150 ± 18%  interrupts.CPU17.LOC:Local_timer_interrupts
    180.83 ± 21%     -32.1%     122.83 ± 14%  interrupts.CPU17.NMI:Non-maskable_interrupts
    180.83 ± 21%     -32.1%     122.83 ± 14%  interrupts.CPU17.PMI:Performance_monitoring_interrupts
    129807 ±  2%    +607.4%     918216 ± 18%  interrupts.CPU18.LOC:Local_timer_interrupts
    129658 ±  2%    +608.1%     918137 ± 18%  interrupts.CPU19.LOC:Local_timer_interrupts
    130039 ±  2%    +606.1%     918169 ± 18%  interrupts.CPU2.LOC:Local_timer_interrupts
    129817 ±  2%    +607.3%     918152 ± 18%  interrupts.CPU20.LOC:Local_timer_interrupts
    129833 ±  2%    +607.2%     918126 ± 18%  interrupts.CPU21.LOC:Local_timer_interrupts
    212.17 ± 22%     -41.7%     123.67 ± 14%  interrupts.CPU21.NMI:Non-maskable_interrupts
    212.17 ± 22%     -41.7%     123.67 ± 14%  interrupts.CPU21.PMI:Performance_monitoring_interrupts
    130043 ±  2%    +606.0%     918137 ± 18%  interrupts.CPU22.LOC:Local_timer_interrupts
    129779 ±  2%    +607.5%     918243 ± 18%  interrupts.CPU23.LOC:Local_timer_interrupts
    129704 ±  2%    +608.0%     918300 ± 18%  interrupts.CPU24.LOC:Local_timer_interrupts
    129760 ±  2%    +607.6%     918160 ± 18%  interrupts.CPU25.LOC:Local_timer_interrupts
    129929 ±  2%    +606.7%     918147 ± 18%  interrupts.CPU26.LOC:Local_timer_interrupts
    130089 ±  2%    +606.1%     918569 ± 18%  interrupts.CPU27.LOC:Local_timer_interrupts
    129924 ±  2%    +606.7%     918215 ± 18%  interrupts.CPU28.LOC:Local_timer_interrupts
    208.83 ± 28%     -48.4%     107.83 ± 20%  interrupts.CPU28.NMI:Non-maskable_interrupts
    208.83 ± 28%     -48.4%     107.83 ± 20%  interrupts.CPU28.PMI:Performance_monitoring_interrupts
    130113 ±  2%    +605.7%     918249 ± 18%  interrupts.CPU29.LOC:Local_timer_interrupts
    237.17 ± 17%     -54.1%     108.83 ± 21%  interrupts.CPU29.NMI:Non-maskable_interrupts
    237.17 ± 17%     -54.1%     108.83 ± 21%  interrupts.CPU29.PMI:Performance_monitoring_interrupts
    129811 ±  2%    +607.3%     918200 ± 18%  interrupts.CPU3.LOC:Local_timer_interrupts
    179.50 ± 22%     -35.8%     115.17 ± 28%  interrupts.CPU3.NMI:Non-maskable_interrupts
    179.50 ± 22%     -35.8%     115.17 ± 28%  interrupts.CPU3.PMI:Performance_monitoring_interrupts
    129720 ±  2%    +607.7%     918076 ± 18%  interrupts.CPU30.LOC:Local_timer_interrupts
    129820 ±  2%    +607.2%     918141 ± 18%  interrupts.CPU31.LOC:Local_timer_interrupts
    211.00 ± 39%     -48.9%     107.83 ± 20%  interrupts.CPU31.NMI:Non-maskable_interrupts
    211.00 ± 39%     -48.9%     107.83 ± 20%  interrupts.CPU31.PMI:Performance_monitoring_interrupts
    129818 ±  2%    +607.2%     918137 ± 18%  interrupts.CPU32.LOC:Local_timer_interrupts
    129630 ±  2%    +608.3%     918129 ± 18%  interrupts.CPU33.LOC:Local_timer_interrupts
    271.00 ± 37%     -55.3%     121.17 ± 19%  interrupts.CPU33.NMI:Non-maskable_interrupts
    271.00 ± 37%     -55.3%     121.17 ± 19%  interrupts.CPU33.PMI:Performance_monitoring_interrupts
    129914 ±  2%    +606.7%     918104 ± 18%  interrupts.CPU34.LOC:Local_timer_interrupts
    215.83 ± 28%     -48.6%     110.83 ± 30%  interrupts.CPU34.NMI:Non-maskable_interrupts
    215.83 ± 28%     -48.6%     110.83 ± 30%  interrupts.CPU34.PMI:Performance_monitoring_interrupts
    129965 ±  2%    +606.5%     918175 ± 18%  interrupts.CPU35.LOC:Local_timer_interrupts
    248.50 ± 32%     -51.7%     120.00 ± 40%  interrupts.CPU35.NMI:Non-maskable_interrupts
    248.50 ± 32%     -51.7%     120.00 ± 40%  interrupts.CPU35.PMI:Performance_monitoring_interrupts
    129653 ±  2%    +608.2%     918185 ± 18%  interrupts.CPU36.LOC:Local_timer_interrupts
    211.50 ± 51%     -43.9%     118.67 ± 17%  interrupts.CPU36.NMI:Non-maskable_interrupts
    211.50 ± 51%     -43.9%     118.67 ± 17%  interrupts.CPU36.PMI:Performance_monitoring_interrupts
    129872 ±  2%    +606.9%     918073 ± 18%  interrupts.CPU37.LOC:Local_timer_interrupts
    130036 ±  2%    +606.0%     918109 ± 18%  interrupts.CPU38.LOC:Local_timer_interrupts
    129772 ±  2%    +607.5%     918121 ± 18%  interrupts.CPU39.LOC:Local_timer_interrupts
    229.50 ± 18%     -46.3%     123.33 ± 16%  interrupts.CPU39.NMI:Non-maskable_interrupts
    229.50 ± 18%     -46.3%     123.33 ± 16%  interrupts.CPU39.PMI:Performance_monitoring_interrupts
    129712 ±  2%    +607.8%     918166 ± 18%  interrupts.CPU4.LOC:Local_timer_interrupts
    129642 ±  2%    +608.2%     918097 ± 18%  interrupts.CPU40.LOC:Local_timer_interrupts
    129853 ±  2%    +607.1%     918180 ± 18%  interrupts.CPU41.LOC:Local_timer_interrupts
    129809 ±  2%    +607.3%     918146 ± 18%  interrupts.CPU42.LOC:Local_timer_interrupts
    229.67 ± 18%     -46.4%     123.00 ± 15%  interrupts.CPU42.NMI:Non-maskable_interrupts
    229.67 ± 18%     -46.4%     123.00 ± 15%  interrupts.CPU42.PMI:Performance_monitoring_interrupts
    129837 ±  2%    +607.1%     918143 ± 18%  interrupts.CPU43.LOC:Local_timer_interrupts
    129909 ±  2%    +606.7%     918129 ± 18%  interrupts.CPU44.LOC:Local_timer_interrupts
    129605 ±  2%    +608.4%     918084 ± 18%  interrupts.CPU45.LOC:Local_timer_interrupts
    129749 ±  2%    +607.6%     918143 ± 18%  interrupts.CPU46.LOC:Local_timer_interrupts
    129735 ±  2%    +607.8%     918262 ± 18%  interrupts.CPU47.LOC:Local_timer_interrupts
    192.00 ± 36%     -41.6%     112.17 ± 31%  interrupts.CPU47.NMI:Non-maskable_interrupts
    192.00 ± 36%     -41.6%     112.17 ± 31%  interrupts.CPU47.PMI:Performance_monitoring_interrupts
    692.33 ± 14%     -17.9%     568.17 ± 12%  interrupts.CPU48.CAL:Function_call_interrupts
    130023 ±  2%    +606.1%     918092 ± 18%  interrupts.CPU48.LOC:Local_timer_interrupts
    129752 ±  2%    +607.6%     918126 ± 18%  interrupts.CPU49.LOC:Local_timer_interrupts
    129983 ±  2%    +606.4%     918178 ± 18%  interrupts.CPU5.LOC:Local_timer_interrupts
    129727 ±  2%    +607.8%     918247 ± 18%  interrupts.CPU50.LOC:Local_timer_interrupts
    712.33 ± 12%     -15.1%     604.50        interrupts.CPU51.CAL:Function_call_interrupts
    130043 ±  2%    +606.1%     918209 ± 18%  interrupts.CPU51.LOC:Local_timer_interrupts
    186.17 ± 19%     -44.2%     103.83 ± 36%  interrupts.CPU51.NMI:Non-maskable_interrupts
    186.17 ± 19%     -44.2%     103.83 ± 36%  interrupts.CPU51.PMI:Performance_monitoring_interrupts
    129948 ±  2%    +606.6%     918175 ± 18%  interrupts.CPU52.LOC:Local_timer_interrupts
    129691 ±  2%    +608.0%     918219 ± 18%  interrupts.CPU53.LOC:Local_timer_interrupts
    129788 ±  2%    +607.4%     918157 ± 18%  interrupts.CPU54.LOC:Local_timer_interrupts
    129762 ±  2%    +607.6%     918216 ± 18%  interrupts.CPU55.LOC:Local_timer_interrupts
    129912 ±  2%    +606.8%     918157 ± 18%  interrupts.CPU56.LOC:Local_timer_interrupts
    198.33 ± 20%     -47.7%     103.67 ± 34%  interrupts.CPU56.NMI:Non-maskable_interrupts
    198.33 ± 20%     -47.7%     103.67 ± 34%  interrupts.CPU56.PMI:Performance_monitoring_interrupts
    129876 ±  2%    +607.0%     918166 ± 18%  interrupts.CPU57.LOC:Local_timer_interrupts
    130047 ±  2%    +606.1%     918203 ± 18%  interrupts.CPU58.LOC:Local_timer_interrupts
    200.50 ± 28%     -43.1%     114.00 ± 27%  interrupts.CPU58.NMI:Non-maskable_interrupts
    200.50 ± 28%     -43.1%     114.00 ± 27%  interrupts.CPU58.PMI:Performance_monitoring_interrupts
    129733 ±  2%    +607.8%     918265 ± 18%  interrupts.CPU59.LOC:Local_timer_interrupts
    227.67 ± 31%     -50.9%     111.83 ± 29%  interrupts.CPU59.NMI:Non-maskable_interrupts
    227.67 ± 31%     -50.9%     111.83 ± 29%  interrupts.CPU59.PMI:Performance_monitoring_interrupts
    129820 ±  2%    +607.3%     918176 ± 18%  interrupts.CPU6.LOC:Local_timer_interrupts
    129865 ±  2%    +607.0%     918129 ± 18%  interrupts.CPU60.LOC:Local_timer_interrupts
    130055 ±  2%    +606.0%     918166 ± 18%  interrupts.CPU61.LOC:Local_timer_interrupts
    130068 ±  2%    +606.0%     918225 ± 18%  interrupts.CPU62.LOC:Local_timer_interrupts
    129969 ±  2%    +606.5%     918211 ± 18%  interrupts.CPU63.LOC:Local_timer_interrupts
    129777 ±  2%    +607.6%     918348 ± 18%  interrupts.CPU64.LOC:Local_timer_interrupts
    130112 ±  2%    +605.6%     918129 ± 18%  interrupts.CPU65.LOC:Local_timer_interrupts
    189.50 ± 18%     -52.5%      90.00 ± 37%  interrupts.CPU65.NMI:Non-maskable_interrupts
    189.50 ± 18%     -52.5%      90.00 ± 37%  interrupts.CPU65.PMI:Performance_monitoring_interrupts
    691.17 ± 14%     -13.1%     600.33        interrupts.CPU66.CAL:Function_call_interrupts
    129784 ±  2%    +607.6%     918338 ± 18%  interrupts.CPU66.LOC:Local_timer_interrupts
    129998 ±  2%    +606.3%     918232 ± 18%  interrupts.CPU67.LOC:Local_timer_interrupts
    129843 ±  2%    +607.4%     918473 ± 18%  interrupts.CPU68.LOC:Local_timer_interrupts
    130013 ±  2%    +606.2%     918188 ± 18%  interrupts.CPU69.LOC:Local_timer_interrupts
    200.17 ± 30%     -50.8%      98.50 ± 43%  interrupts.CPU69.NMI:Non-maskable_interrupts
    200.17 ± 30%     -50.8%      98.50 ± 43%  interrupts.CPU69.PMI:Performance_monitoring_interrupts
    129906 ±  2%    +606.8%     918171 ± 18%  interrupts.CPU7.LOC:Local_timer_interrupts
    129911 ±  2%    +606.9%     918285 ± 18%  interrupts.CPU70.LOC:Local_timer_interrupts
    130068 ±  2%    +605.9%     918182 ± 18%  interrupts.CPU71.LOC:Local_timer_interrupts
    191.33 ± 33%     -51.3%      93.17 ± 48%  interrupts.CPU71.NMI:Non-maskable_interrupts
    191.33 ± 33%     -51.3%      93.17 ± 48%  interrupts.CPU71.PMI:Performance_monitoring_interrupts
    130115 ±  2%    +605.8%     918315 ± 18%  interrupts.CPU72.LOC:Local_timer_interrupts
    190.50 ± 33%     -55.4%      85.00 ± 45%  interrupts.CPU72.NMI:Non-maskable_interrupts
    190.50 ± 33%     -55.4%      85.00 ± 45%  interrupts.CPU72.PMI:Performance_monitoring_interrupts
    129715 ±  2%    +607.8%     918137 ± 18%  interrupts.CPU73.LOC:Local_timer_interrupts
    191.33 ± 35%     -63.1%      70.67 ± 49%  interrupts.CPU73.NMI:Non-maskable_interrupts
    191.33 ± 35%     -63.1%      70.67 ± 49%  interrupts.CPU73.PMI:Performance_monitoring_interrupts
    129851 ±  2%    +607.1%     918129 ± 18%  interrupts.CPU74.LOC:Local_timer_interrupts
    129791 ±  2%    +607.4%     918208 ± 18%  interrupts.CPU75.LOC:Local_timer_interrupts
    176.00 ± 43%     -51.5%      85.33 ± 50%  interrupts.CPU75.NMI:Non-maskable_interrupts
    176.00 ± 43%     -51.5%      85.33 ± 50%  interrupts.CPU75.PMI:Performance_monitoring_interrupts
    129736 ±  2%    +607.7%     918200 ± 18%  interrupts.CPU76.LOC:Local_timer_interrupts
    231.00 ± 37%     -62.9%      85.67 ± 51%  interrupts.CPU76.NMI:Non-maskable_interrupts
    231.00 ± 37%     -62.9%      85.67 ± 51%  interrupts.CPU76.PMI:Performance_monitoring_interrupts
    129718 ±  2%    +607.8%     918150 ± 18%  interrupts.CPU77.LOC:Local_timer_interrupts
    238.83 ± 19%     -60.0%      95.50 ± 47%  interrupts.CPU77.NMI:Non-maskable_interrupts
    238.83 ± 19%     -60.0%      95.50 ± 47%  interrupts.CPU77.PMI:Performance_monitoring_interrupts
    129718 ±  2%    +607.9%     918239 ± 18%  interrupts.CPU78.LOC:Local_timer_interrupts
    196.00 ± 27%     -56.0%      86.33 ± 52%  interrupts.CPU78.NMI:Non-maskable_interrupts
    196.00 ± 27%     -56.0%      86.33 ± 52%  interrupts.CPU78.PMI:Performance_monitoring_interrupts
    129804 ±  2%    +607.3%     918154 ± 18%  interrupts.CPU79.LOC:Local_timer_interrupts
    130013 ±  2%    +606.3%     918220 ± 18%  interrupts.CPU8.LOC:Local_timer_interrupts
    220.67 ±  7%     -38.8%     135.00 ± 25%  interrupts.CPU8.NMI:Non-maskable_interrupts
    220.67 ±  7%     -38.8%     135.00 ± 25%  interrupts.CPU8.PMI:Performance_monitoring_interrupts
    129665 ±  2%    +608.1%     918223 ± 18%  interrupts.CPU80.LOC:Local_timer_interrupts
    129836 ±  2%    +607.3%     918391 ± 18%  interrupts.CPU81.LOC:Local_timer_interrupts
    258.17 ± 38%     -62.2%      97.67 ± 51%  interrupts.CPU81.NMI:Non-maskable_interrupts
    258.17 ± 38%     -62.2%      97.67 ± 51%  interrupts.CPU81.PMI:Performance_monitoring_interrupts
    129706 ±  2%    +607.9%     918130 ± 18%  interrupts.CPU82.LOC:Local_timer_interrupts
    129821 ±  2%    +607.4%     918412 ± 18%  interrupts.CPU83.LOC:Local_timer_interrupts
    248.00 ± 29%     -57.9%     104.50 ± 59%  interrupts.CPU83.NMI:Non-maskable_interrupts
    248.00 ± 29%     -57.9%     104.50 ± 59%  interrupts.CPU83.PMI:Performance_monitoring_interrupts
    129799 ±  2%    +607.4%     918206 ± 18%  interrupts.CPU84.LOC:Local_timer_interrupts
    130074 ±  2%    +605.9%     918158 ± 18%  interrupts.CPU85.LOC:Local_timer_interrupts
    217.67 ± 32%     -58.3%      90.83 ± 36%  interrupts.CPU85.NMI:Non-maskable_interrupts
    217.67 ± 32%     -58.3%      90.83 ± 36%  interrupts.CPU85.PMI:Performance_monitoring_interrupts
    130033 ±  2%    +606.2%     918269 ± 18%  interrupts.CPU86.LOC:Local_timer_interrupts
    129953 ±  2%    +606.6%     918233 ± 18%  interrupts.CPU87.LOC:Local_timer_interrupts
    207.00 ± 22%     -45.3%     113.17 ± 28%  interrupts.CPU87.NMI:Non-maskable_interrupts
    207.00 ± 22%     -45.3%     113.17 ± 28%  interrupts.CPU87.PMI:Performance_monitoring_interrupts
    129884 ±  2%    +607.0%     918226 ± 18%  interrupts.CPU88.LOC:Local_timer_interrupts
    268.00 ± 29%     -60.8%     105.00 ± 40%  interrupts.CPU88.NMI:Non-maskable_interrupts
    268.00 ± 29%     -60.8%     105.00 ± 40%  interrupts.CPU88.PMI:Performance_monitoring_interrupts
    129693 ±  2%    +607.9%     918113 ± 18%  interrupts.CPU89.LOC:Local_timer_interrupts
    129901 ±  2%    +606.9%     918318 ± 18%  interrupts.CPU9.LOC:Local_timer_interrupts
    229.67 ± 27%     -51.0%     112.50 ± 27%  interrupts.CPU9.NMI:Non-maskable_interrupts
    229.67 ± 27%     -51.0%     112.50 ± 27%  interrupts.CPU9.PMI:Performance_monitoring_interrupts
    129994 ±  2%    +606.4%     918251 ± 18%  interrupts.CPU90.LOC:Local_timer_interrupts
    129941 ±  2%    +606.5%     918072 ± 18%  interrupts.CPU91.LOC:Local_timer_interrupts
    129710 ±  2%    +607.8%     918131 ± 18%  interrupts.CPU92.LOC:Local_timer_interrupts
    668.33 ±  5%     -10.8%     596.17        interrupts.CPU93.CAL:Function_call_interrupts
    129951 ±  2%    +606.6%     918176 ± 18%  interrupts.CPU93.LOC:Local_timer_interrupts
    129873 ±  2%    +606.9%     918117 ± 18%  interrupts.CPU94.LOC:Local_timer_interrupts
    129755 ±  2%    +607.6%     918133 ± 18%  interrupts.CPU95.LOC:Local_timer_interrupts
    188.33 ± 31%     -53.5%      87.67 ± 57%  interrupts.CPU95.NMI:Non-maskable_interrupts
    188.33 ± 31%     -53.5%      87.67 ± 57%  interrupts.CPU95.PMI:Performance_monitoring_interrupts
     20.83 ±  6%    +110.4%      43.83 ± 75%  interrupts.CPU95.TLB:TLB_shootdowns
  12466595 ±  2%    +607.1%   88146744 ± 18%  interrupts.LOC:Local_timer_interrupts
     18867 ±  3%     -44.1%      10549 ± 24%  interrupts.NMI:Non-maskable_interrupts
     18867 ±  3%     -44.1%      10549 ± 24%  interrupts.PMI:Performance_monitoring_interrupts
     19145 ±  3%     -70.7%       5607 ± 15%  softirqs.BLOCK
     11175 ± 14%    +104.8%      22886 ± 24%  softirqs.CPU0.RCU
     10931 ± 14%    +433.1%      58276 ± 16%  softirqs.CPU0.SCHED
      9814 ± 17%    +121.6%      21747 ± 24%  softirqs.CPU1.RCU
      9403 ±  5%    +511.5%      57504 ± 15%  softirqs.CPU1.SCHED
      8834 ±  4%    +130.4%      20353 ± 23%  softirqs.CPU10.RCU
      8184 ±  8%    +587.9%      56299 ± 18%  softirqs.CPU10.SCHED
      8683 ±  8%    +134.3%      20349 ± 22%  softirqs.CPU11.RCU
      9062 ±  6%    +485.5%      53058 ± 18%  softirqs.CPU11.SCHED
      8791 ± 10%    +131.7%      20369 ± 23%  softirqs.CPU12.RCU
      8415 ±  7%    +544.1%      54201 ± 18%  softirqs.CPU12.SCHED
      8447 ±  3%    +140.7%      20331 ± 22%  softirqs.CPU13.RCU
      8588 ±  6%    +546.8%      55556 ± 17%  softirqs.CPU13.SCHED
      8591 ±  4%    +137.0%      20360 ± 25%  softirqs.CPU14.RCU
      8106 ±  9%    +585.3%      55554 ± 18%  softirqs.CPU14.SCHED
      9257 ± 16%    +120.0%      20369 ± 23%  softirqs.CPU15.RCU
      8212 ± 10%    +581.9%      55998 ± 17%  softirqs.CPU15.SCHED
      8190 ±  2%    +150.9%      20545 ± 21%  softirqs.CPU16.RCU
      8988 ±  2%    +526.0%      56269 ± 17%  softirqs.CPU16.SCHED
      8548 ±  9%    +143.5%      20818 ± 20%  softirqs.CPU17.RCU
      8558 ±  7%    +517.1%      52817 ± 28%  softirqs.CPU17.SCHED
      9188 ± 17%    +111.0%      19391 ± 21%  softirqs.CPU18.RCU
      8898 ±  9%    +527.0%      55799 ± 17%  softirqs.CPU18.SCHED
      8336 ±  3%    +146.7%      20563 ± 21%  softirqs.CPU19.RCU
      8822 ±  9%    +532.3%      55785 ± 17%  softirqs.CPU19.SCHED
      8926 ±  7%    +131.7%      20684 ± 21%  softirqs.CPU2.RCU
      8473 ±  9%    +508.6%      51573 ± 22%  softirqs.CPU2.SCHED
      9135 ± 19%    +124.6%      20513 ± 21%  softirqs.CPU20.RCU
      8369 ± 14%    +566.8%      55802 ± 17%  softirqs.CPU20.SCHED
      8300 ±  2%    +152.7%      20977 ± 17%  softirqs.CPU21.RCU
      8736 ±  6%    +538.6%      55791 ± 17%  softirqs.CPU21.SCHED
      8330 ±  4%    +144.0%      20324 ± 20%  softirqs.CPU22.RCU
      8457 ± 12%    +562.6%      56038 ± 18%  softirqs.CPU22.SCHED
      8638 ±  3%    +140.3%      20761 ± 21%  softirqs.CPU23.RCU
      8214 ± 22%    +579.3%      55797 ± 17%  softirqs.CPU23.SCHED
      8514 ±  4%    +148.2%      21136 ± 22%  softirqs.CPU24.RCU
      8571 ± 10%    +552.1%      55898 ± 18%  softirqs.CPU24.SCHED
      8185 ±  3%    +148.1%      20310 ± 22%  softirqs.CPU25.RCU
      8150 ± 12%    +552.0%      53142 ± 17%  softirqs.CPU25.SCHED
      8319 ±  3%    +144.3%      20322 ± 22%  softirqs.CPU26.RCU
      8827 ± 11%    +527.1%      55356 ± 17%  softirqs.CPU26.SCHED
      8325 ±  4%    +145.5%      20438 ± 22%  softirqs.CPU27.RCU
      8135 ± 22%    +580.6%      55362 ± 17%  softirqs.CPU27.SCHED
      8278 ±  3%    +147.8%      20513 ± 21%  softirqs.CPU28.RCU
      8718 ±  7%    +546.5%      56363 ± 19%  softirqs.CPU28.SCHED
      8329 ±  3%    +147.8%      20643 ± 22%  softirqs.CPU29.RCU
      8940 ±  6%    +469.2%      50889 ± 23%  softirqs.CPU29.SCHED
      8607 ±  5%    +136.8%      20379 ± 23%  softirqs.CPU3.RCU
      8101 ± 18%    +588.5%      55771 ± 17%  softirqs.CPU3.SCHED
      8376 ±  5%    +143.4%      20383 ± 22%  softirqs.CPU30.RCU
      7887 ± 17%    +546.4%      50988 ± 22%  softirqs.CPU30.SCHED
      8460 ±  7%    +135.8%      19952 ± 22%  softirqs.CPU31.RCU
      8584 ±  6%    +542.7%      55169 ± 18%  softirqs.CPU31.SCHED
      8869 ±  9%    +134.6%      20807 ± 23%  softirqs.CPU32.RCU
      8582 ± 19%    +543.5%      55232 ± 18%  softirqs.CPU32.SCHED
      8928 ±  5%    +132.4%      20748 ± 22%  softirqs.CPU33.RCU
      8457 ±  5%    +553.0%      55232 ± 18%  softirqs.CPU33.SCHED
      9426 ± 10%    +122.5%      20972 ± 22%  softirqs.CPU34.RCU
      7906 ±  9%    +598.4%      55216 ± 17%  softirqs.CPU34.SCHED
      9106 ±  6%    +131.0%      21039 ± 23%  softirqs.CPU35.RCU
      8816 ±  6%    +525.0%      55101 ± 18%  softirqs.CPU35.SCHED
      8727 ±  5%    +138.0%      20774 ± 24%  softirqs.CPU36.RCU
      8171 ± 12%    +520.1%      50674 ± 23%  softirqs.CPU36.SCHED
      8639 ±  4%    +143.0%      20994 ± 23%  softirqs.CPU37.RCU
      8867 ±  3%    +469.6%      50508 ± 23%  softirqs.CPU37.SCHED
      8578 ±  5%    +142.5%      20806 ± 22%  softirqs.CPU38.RCU
      9056          +476.2%      52184 ± 28%  softirqs.CPU38.SCHED
      8737 ±  6%    +139.1%      20890 ± 22%  softirqs.CPU39.RCU
      9217 ±  2%    +476.9%      53172 ± 24%  softirqs.CPU39.SCHED
      8329 ±  9%    +137.7%      19798 ± 28%  softirqs.CPU4.RCU
      8510 ±  8%    +544.7%      54860 ± 20%  softirqs.CPU4.SCHED
      8974 ±  8%    +138.4%      21393 ± 21%  softirqs.CPU40.RCU
      7294 ± 18%    +659.8%      55422 ± 17%  softirqs.CPU40.SCHED
      8877 ±  8%    +137.3%      21064 ± 22%  softirqs.CPU41.RCU
      8739 ±  4%    +501.2%      52540 ± 28%  softirqs.CPU41.SCHED
      8672 ±  7%    +138.2%      20654 ± 21%  softirqs.CPU42.RCU
      7964 ± 10%    +591.8%      55101 ± 17%  softirqs.CPU42.SCHED
      8918 ±  8%    +134.6%      20923 ± 21%  softirqs.CPU43.RCU
      8468 ±  9%    +550.8%      55111 ± 17%  softirqs.CPU43.SCHED
      8606 ±  9%    +141.0%      20741 ± 21%  softirqs.CPU44.RCU
      8285 ± 13%    +565.0%      55094 ± 17%  softirqs.CPU44.SCHED
      8685 ±  8%    +141.2%      20946 ± 22%  softirqs.CPU45.RCU
      9083          +505.6%      55004 ± 17%  softirqs.CPU45.SCHED
      8732 ±  8%    +137.6%      20750 ± 22%  softirqs.CPU46.RCU
      8562 ±  8%    +545.8%      55296 ± 17%  softirqs.CPU46.SCHED
      8970 ±  7%    +145.0%      21978 ± 22%  softirqs.CPU47.RCU
      9064 ±  2%    +227.5%      29687 ± 61%  softirqs.CPU47.SCHED
      8288 ±  3%    +156.3%      21242 ± 26%  softirqs.CPU48.RCU
      7591 ± 24%    +382.2%      36604 ± 44%  softirqs.CPU48.SCHED
      8450 ±  6%    +137.9%      20104 ± 25%  softirqs.CPU49.RCU
      8107 ± 18%    +606.5%      57274 ± 17%  softirqs.CPU49.SCHED
      8474 ±  4%    +139.7%      20310 ± 22%  softirqs.CPU5.RCU
      8404 ± 15%    +568.5%      56185 ± 17%  softirqs.CPU5.SCHED
      8253 ±  6%    +141.7%      19947 ± 22%  softirqs.CPU50.RCU
      8581 ±  3%    +506.4%      52041 ± 22%  softirqs.CPU50.SCHED
      9540 ± 19%    +112.8%      20304 ± 23%  softirqs.CPU51.RCU
      7495 ± 21%    +663.3%      57211 ± 17%  softirqs.CPU51.SCHED
      8396 ±  7%    +145.1%      20577 ± 24%  softirqs.CPU52.RCU
      8705 ± 10%    +452.9%      48130 ± 24%  softirqs.CPU52.SCHED
      8294 ±  3%    +146.9%      20480 ± 22%  softirqs.CPU53.RCU
      9382 ±  4%    +439.0%      50573 ± 31%  softirqs.CPU53.SCHED
      8772 ±  7%    +135.4%      20653 ± 21%  softirqs.CPU54.RCU
      8818 ±  8%    +499.1%      52833 ± 22%  softirqs.CPU54.SCHED
      8658 ±  5%    +128.7%      19801 ± 23%  softirqs.CPU55.RCU
      7268 ± 18%    +681.8%      56826 ± 17%  softirqs.CPU55.SCHED
      8153 ±  5%    +145.6%      20027 ± 21%  softirqs.CPU56.RCU
      8284 ±  7%    +559.8%      54661 ± 25%  softirqs.CPU56.SCHED
      8338 ±  7%    +141.8%      20164 ± 22%  softirqs.CPU57.RCU
      9493 ±  5%    +503.8%      57318 ± 17%  softirqs.CPU57.SCHED
      8497 ±  6%    +137.9%      20210 ± 23%  softirqs.CPU58.RCU
      9041 ±  8%    +533.7%      57298 ± 17%  softirqs.CPU58.SCHED
      8451 ±  6%    +140.7%      20340 ± 23%  softirqs.CPU59.RCU
      8112 ± 12%    +610.3%      57622 ± 17%  softirqs.CPU59.SCHED
      8981 ±  6%    +121.6%      19900 ± 28%  softirqs.CPU6.RCU
      8239 ± 14%    +576.4%      55728 ± 17%  softirqs.CPU6.SCHED
      8278 ±  5%    +142.7%      20090 ± 23%  softirqs.CPU60.RCU
      8934 ± 10%    +540.1%      57190 ± 17%  softirqs.CPU60.SCHED
      8126 ±  5%    +144.2%      19843 ± 23%  softirqs.CPU61.RCU
      8236 ± 12%    +532.2%      52069 ± 23%  softirqs.CPU61.SCHED
      8127 ±  4%    +141.7%      19640 ± 23%  softirqs.CPU62.RCU
      8282 ± 13%    +582.6%      56541 ± 17%  softirqs.CPU62.SCHED
      8496 ±  5%    +134.4%      19918 ± 24%  softirqs.CPU63.RCU
      7647 ± 19%    +642.6%      56793 ± 18%  softirqs.CPU63.SCHED
      7950 ±  3%    +148.2%      19731 ± 25%  softirqs.CPU64.RCU
      8853 ±  9%    +560.7%      58491 ± 19%  softirqs.CPU64.SCHED
      8030 ±  6%    +162.1%      21044 ± 20%  softirqs.CPU65.RCU
      7387 ± 21%    +599.0%      51641 ± 26%  softirqs.CPU65.SCHED
      8410 ±  5%    +146.6%      20740 ± 21%  softirqs.CPU66.RCU
      7111 ± 25%    +708.2%      57476 ± 17%  softirqs.CPU66.SCHED
      8307 ±  6%    +130.6%      19157 ± 27%  softirqs.CPU67.RCU
      7607 ± 20%    +648.7%      56958 ± 17%  softirqs.CPU67.SCHED
      8596 ±  6%    +136.7%      20344 ± 21%  softirqs.CPU68.RCU
      7018 ± 18%    +721.4%      57646 ± 17%  softirqs.CPU68.SCHED
      7862 ±  4%    +161.6%      20571 ± 21%  softirqs.CPU69.RCU
      8966 ±  4%    +495.4%      53385 ± 24%  softirqs.CPU69.SCHED
      8841 ±  6%    +132.3%      20542 ± 21%  softirqs.CPU7.RCU
      8643 ±  5%    +547.3%      55948 ± 17%  softirqs.CPU7.SCHED
      8159 ±  2%    +154.4%      20758 ± 20%  softirqs.CPU70.RCU
      8672 ±  9%    +527.0%      54379 ± 33%  softirqs.CPU70.SCHED
      8174          +154.7%      20820 ± 22%  softirqs.CPU71.RCU
      7791 ± 17%    +575.7%      52641 ± 21%  softirqs.CPU71.SCHED
      8178 ±  3%    +161.3%      21368 ± 17%  softirqs.CPU72.RCU
      9215 ±  3%    +501.0%      55387 ± 24%  softirqs.CPU72.SCHED
      8152 ±  8%    +143.9%      19888 ± 22%  softirqs.CPU73.RCU
      8220 ±  5%    +535.5%      52236 ± 23%  softirqs.CPU73.SCHED
      7866 ±  3%    +155.3%      20084 ± 22%  softirqs.CPU74.RCU
      8635 ±  6%    +526.1%      54069 ± 28%  softirqs.CPU74.SCHED
      7902 ±  5%    +151.2%      19853 ± 18%  softirqs.CPU75.RCU
      8882 ±  8%    +544.6%      57255 ± 17%  softirqs.CPU75.SCHED
      7970 ±  4%    +148.9%      19837 ± 21%  softirqs.CPU76.RCU
      8922 ±  9%    +542.1%      57294 ± 17%  softirqs.CPU76.SCHED
      8052 ±  5%    +154.1%      20458 ± 22%  softirqs.CPU77.RCU
      8645 ±  5%    +565.2%      57506 ± 17%  softirqs.CPU77.SCHED
      8108 ±  5%    +150.9%      20346 ± 22%  softirqs.CPU78.RCU
      7930 ± 18%    +623.2%      57347 ± 17%  softirqs.CPU78.SCHED
      8107 ±  4%    +150.2%      20284 ± 23%  softirqs.CPU79.RCU
      7219 ± 16%    +688.6%      56928 ± 18%  softirqs.CPU79.SCHED
      8585 ±  4%    +136.0%      20264 ± 21%  softirqs.CPU8.RCU
      8973 ±  3%    +519.7%      55608 ± 17%  softirqs.CPU8.SCHED
      8426 ±  9%    +133.0%      19633 ± 23%  softirqs.CPU80.RCU
      8670 ± 11%    +558.0%      57051 ± 17%  softirqs.CPU80.SCHED
      8484 ±  7%    +135.2%      19957 ± 23%  softirqs.CPU81.RCU
      8310 ±  8%    +586.4%      57041 ± 17%  softirqs.CPU81.SCHED
      8504 ±  8%    +139.7%      20381 ± 21%  softirqs.CPU82.RCU
      8552 ± 11%    +567.8%      57112 ± 17%  softirqs.CPU82.SCHED
      8563 ±  7%    +135.4%      20153 ± 23%  softirqs.CPU83.RCU
      8273 ± 15%    +533.2%      52388 ± 23%  softirqs.CPU83.SCHED
      8629 ±  6%    +129.5%      19802 ± 23%  softirqs.CPU84.RCU
      7551 ± 22%    +663.9%      57682 ± 15%  softirqs.CPU84.SCHED
      8656 ±  6%    +130.1%      19913 ± 23%  softirqs.CPU85.RCU
      7092 ± 16%    +703.8%      57005 ± 17%  softirqs.CPU85.SCHED
      8408 ±  7%    +114.6%      18042 ± 26%  softirqs.CPU86.RCU
      7370 ± 24%    +675.5%      57157 ± 17%  softirqs.CPU86.SCHED
      8420 ±  6%    +133.0%      19617 ± 22%  softirqs.CPU87.RCU
      8399 ± 14%    +575.6%      56744 ± 17%  softirqs.CPU87.SCHED
      8655 ±  8%    +132.5%      20125 ± 23%  softirqs.CPU88.RCU
      8868 ±  5%    +502.5%      53433 ± 20%  softirqs.CPU88.SCHED
      8533 ±  6%    +128.2%      19477 ± 28%  softirqs.CPU89.RCU
      9111 ±  5%    +529.4%      57344 ± 17%  softirqs.CPU89.SCHED
      8711 ±  4%    +134.9%      20467 ± 22%  softirqs.CPU9.RCU
      7848 ± 19%    +609.3%      55666 ± 17%  softirqs.CPU9.SCHED
      8386 ±  7%    +130.9%      19363 ± 24%  softirqs.CPU90.RCU
      7657 ± 17%    +634.1%      56215 ± 18%  softirqs.CPU90.SCHED
      8616 ± 10%    +124.5%      19342 ± 28%  softirqs.CPU91.RCU
      8592 ± 10%    +562.5%      56928 ± 17%  softirqs.CPU91.SCHED
      8381 ±  7%    +129.5%      19231 ± 23%  softirqs.CPU92.RCU
      8386 ± 13%    +574.5%      56566 ± 18%  softirqs.CPU92.SCHED
      8362 ± 11%    +134.8%      19633 ± 23%  softirqs.CPU93.RCU
      8893 ±  9%    +540.7%      56981 ± 17%  softirqs.CPU93.SCHED
      8595 ±  6%    +133.0%      20026 ± 23%  softirqs.CPU94.RCU
      8231 ± 13%    +596.8%      57360 ± 17%  softirqs.CPU94.SCHED
      8974 ±  8%    +138.7%      21420 ± 22%  softirqs.CPU95.RCU
      8231 ±  9%    +408.0%      41821 ± 40%  softirqs.CPU95.SCHED
      1478 ± 64%    +914.6%      15001 ± 52%  softirqs.NET_RX
    821537 ±  4%    +138.2%    1956977 ± 22%  softirqs.RCU
    808773 ±  2%    +548.7%    5246138 ± 17%  softirqs.SCHED
     17352 ±  2%     +74.5%      30278 ± 17%  softirqs.TIMER
     18.64 ±  4%     -18.6        0.00        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
     16.75 ±  5%     -16.7        0.00        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
     13.99 ±  3%     -14.0        0.00        perf-profile.calltrace.cycles-pp.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
     11.59 ±  8%     -11.6        0.00        perf-profile.calltrace.cycles-pp.ret_from_fork
     11.58 ±  8%     -11.6        0.00        perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
     11.22 ±  8%     -11.2        0.00        perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
     11.17 ±  8%     -11.2        0.00        perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
     10.70 ±  8%     -10.7        0.00        perf-profile.calltrace.cycles-pp.drm_fb_helper_damage_work.process_one_work.worker_thread.kthread.ret_from_fork
     10.34 ±  8%     -10.3        0.00        perf-profile.calltrace.cycles-pp.memcpy_toio.drm_fb_helper_damage_work.process_one_work.worker_thread.kthread
     10.30 ±  3%     -10.3        0.00        perf-profile.calltrace.cycles-pp.block_ioctl.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
     10.30 ±  3%     -10.3        0.00        perf-profile.calltrace.cycles-pp.blkdev_ioctl.block_ioctl.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
     10.26 ±  3%     -10.3        0.00        perf-profile.calltrace.cycles-pp.lo_ioctl.blkdev_ioctl.block_ioctl.__x64_sys_ioctl.do_syscall_64
      8.23 ±  6%      -8.2        0.00        perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.printk.set_capacity_and_notify.cold.loop_set_size
      7.66 ±  6%      -7.7        0.00        perf-profile.calltrace.cycles-pp.serial8250_console_write.console_unlock.vprintk_emit.printk.set_capacity_and_notify.cold
      7.29 ±  6%      -6.9        0.37 ± 70%  perf-profile.calltrace.cycles-pp.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit.printk
      6.78 ±  5%      -6.4        0.36 ± 71%  perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock
      6.78 ±  5%      -6.4        0.36 ± 71%  perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit
      5.40 ±  5%      -5.3        0.09 ±223%  perf-profile.calltrace.cycles-pp.io_serial_in.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write
      5.17 ±  9%      -5.2        0.00        perf-profile.calltrace.cycles-pp.loop_configure.lo_ioctl.blkdev_ioctl.block_ioctl.__x64_sys_ioctl
      5.13 ±  9%      -5.1        0.00        perf-profile.calltrace.cycles-pp.printk.set_capacity_and_notify.cold.loop_set_size.loop_configure.lo_ioctl
      5.13 ±  9%      -5.1        0.00        perf-profile.calltrace.cycles-pp.vprintk_emit.printk.set_capacity_and_notify.cold.loop_set_size.loop_configure
      5.13 ±  9%      -5.1        0.00        perf-profile.calltrace.cycles-pp.loop_set_size.loop_configure.lo_ioctl.blkdev_ioctl.block_ioctl
      5.13 ±  9%      -5.1        0.00        perf-profile.calltrace.cycles-pp.set_capacity_and_notify.cold.loop_set_size.loop_configure.lo_ioctl.blkdev_ioctl
      0.76 ± 11%      +0.5        1.25 ± 15%  perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      0.70 ± 45%      +0.5        1.22 ±  4%  perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      0.72 ± 45%      +0.5        1.25 ±  4%  perf-profile.calltrace.cycles-pp.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      0.79 ± 12%      +0.6        1.38 ± 11%  perf-profile.calltrace.cycles-pp.lapic_next_deadline.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      0.69 ± 17%      +0.6        1.31 ± 15%  perf-profile.calltrace.cycles-pp.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.17 ±141%      +0.7        0.82 ± 14%  perf-profile.calltrace.cycles-pp.load_balance.rebalance_domains.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt
      0.95 ± 18%      +0.7        1.63 ± 14%  perf-profile.calltrace.cycles-pp.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      0.49 ± 81%      +0.7        1.18 ±  5%  perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64_no_verify
      0.49 ± 81%      +0.7        1.18 ±  5%  perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_kernel.secondary_startup_64_no_verify
      0.49 ± 81%      +0.7        1.18 ±  5%  perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64_no_verify
      0.40 ± 71%      +0.7        1.11 ± 13%  perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.99 ± 16%      +0.7        1.71 ± 13%  perf-profile.calltrace.cycles-pp.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      0.00            +0.7        0.73 ± 18%  perf-profile.calltrace.cycles-pp.calc_global_load_tick.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer
      1.75 ± 10%      +0.8        2.57 ± 14%  perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      1.07 ±  8%      +0.9        1.95 ±  9%  perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      0.72 ±  8%      +0.9        1.64 ± 17%  perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      2.24 ± 10%      +1.2        3.40 ± 13%  perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      2.09 ±  8%      +1.5        3.60 ± 10%  perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
      2.16 ±  8%      +1.5        3.71 ±  9%  perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      0.86 ± 46%      +1.6        2.48 ± 13%  perf-profile.calltrace.cycles-pp.timekeeping_max_deferment.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle
      1.72 ± 11%      +1.9        3.59 ± 12%  perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      2.52 ±  7%      +2.0        4.50 ±  9%  perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      1.68 ± 12%      +2.3        3.96 ±  8%  perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry
      2.59 ±  9%      +2.5        5.08 ± 11%  perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      2.08 ±  9%      +2.5        4.62 ±  8%  perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary
      3.85 ±  6%      +2.9        6.77 ±  9%  perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      7.69 ±  4%      +6.3       14.01 ± 10%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      7.83 ±  4%      +6.4       14.22 ±  9%  perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
     12.42 ±  5%      +9.0       21.41 ± 10%  perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
      5.56 ± 16%     +12.1       17.64 ±  5%  perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     57.43           +13.5       70.94 ±  2%  perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
     59.52 ±  2%     +18.1       77.59        perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     18.30 ±  9%     +19.3       37.61 ± 13%  perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
     66.15 ±  3%     +30.7       96.87        perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     66.17 ±  3%     +30.7       96.92        perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
     66.17 ±  3%     +30.7       96.92        perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     66.79 ±  2%     +31.3       98.10        perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
     20.85 ±  4%     -19.8        1.01 ± 14%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
     18.54 ±  5%     -17.6        0.98 ± 13%  perf-profile.children.cycles-pp.do_syscall_64
     14.03 ±  3%     -14.0        0.00        perf-profile.children.cycles-pp.__x64_sys_ioctl
     11.60 ±  8%     -11.3        0.30 ± 11%  perf-profile.children.cycles-pp.ret_from_fork
     11.58 ±  8%     -11.3        0.30 ± 13%  perf-profile.children.cycles-pp.kthread
     11.22 ±  8%     -11.1        0.15 ±  7%  perf-profile.children.cycles-pp.worker_thread
     11.17 ±  8%     -11.0        0.14 ±  6%  perf-profile.children.cycles-pp.process_one_work
     10.70 ±  8%     -10.6        0.12 ±  7%  perf-profile.children.cycles-pp.drm_fb_helper_damage_work
     10.69 ±  8%     -10.6        0.12 ±  7%  perf-profile.children.cycles-pp.memcpy_toio
     10.34 ±  3%     -10.3        0.00        perf-profile.children.cycles-pp.block_ioctl
     10.34 ±  3%     -10.3        0.00        perf-profile.children.cycles-pp.blkdev_ioctl
     10.30 ±  3%     -10.3        0.00        perf-profile.children.cycles-pp.lo_ioctl
      9.73 ±  4%      -9.7        0.00        perf-profile.children.cycles-pp.loop_set_size
      9.73 ±  4%      -9.7        0.00        perf-profile.children.cycles-pp.set_capacity_and_notify.cold
     10.24 ±  4%      -9.7        0.58 ± 16%  perf-profile.children.cycles-pp.printk
     10.24 ±  4%      -9.7        0.58 ± 16%  perf-profile.children.cycles-pp.vprintk_emit
      8.72 ±  6%      -8.1        0.58 ± 16%  perf-profile.children.cycles-pp.console_unlock
      8.12 ±  6%      -7.6        0.54 ± 16%  perf-profile.children.cycles-pp.serial8250_console_write
      7.74 ±  6%      -7.2        0.53 ± 16%  perf-profile.children.cycles-pp.uart_console_write
      7.57 ±  5%      -7.0        0.53 ± 17%  perf-profile.children.cycles-pp.wait_for_xmitr
      7.21 ±  6%      -6.7        0.51 ± 17%  perf-profile.children.cycles-pp.serial8250_console_putchar
      6.69 ± 10%      -6.7        0.00        perf-profile.children.cycles-pp.__mutex_lock
      6.03 ±  6%      -5.6        0.44 ± 25%  perf-profile.children.cycles-pp.io_serial_in
      5.17 ±  9%      -5.2        0.00        perf-profile.children.cycles-pp.loop_configure
      3.15 ± 23%      -3.0        0.12 ± 43%  perf-profile.children.cycles-pp.do_sys_open
      3.15 ± 23%      -3.0        0.12 ± 43%  perf-profile.children.cycles-pp.do_sys_openat2
      3.00 ± 25%      -2.9        0.12 ± 37%  perf-profile.children.cycles-pp.do_filp_open
      2.99 ± 25%      -2.9        0.12 ± 37%  perf-profile.children.cycles-pp.path_openat
      1.52 ±  6%      -1.4        0.09 ± 80%  perf-profile.children.cycles-pp.delay_tsc
      0.45 ± 13%      -0.3        0.12 ± 26%  perf-profile.children.cycles-pp.rcu_core
      0.40 ± 12%      -0.3        0.10 ± 51%  perf-profile.children.cycles-pp.__schedule
      0.32 ± 37%      -0.3        0.06 ± 50%  perf-profile.children.cycles-pp.walk_component
      0.32 ± 10%      -0.3        0.07 ± 56%  perf-profile.children.cycles-pp.schedule
      0.28 ±  8%      -0.2        0.04 ± 75%  perf-profile.children.cycles-pp.kmem_cache_alloc
      0.25 ± 16%      -0.2        0.05 ± 78%  perf-profile.children.cycles-pp.link_path_walk
      0.21 ± 18%      -0.2        0.05 ± 74%  perf-profile.children.cycles-pp.newidle_balance
      0.20 ± 20%      -0.1        0.09 ± 31%  perf-profile.children.cycles-pp.note_gp_changes
      0.26 ± 14%      -0.1        0.16 ± 30%  perf-profile.children.cycles-pp.ksys_read
      0.25 ± 15%      -0.1        0.16 ± 30%  perf-profile.children.cycles-pp.vfs_read
      0.17 ± 16%      -0.1        0.08 ± 58%  perf-profile.children.cycles-pp.poll_idle
      0.09 ±  6%      +0.1        0.15 ± 18%  perf-profile.children.cycles-pp.exec_binprm
      0.09 ±  8%      +0.1        0.14 ± 17%  perf-profile.children.cycles-pp.load_elf_binary
      0.03 ±100%      +0.1        0.09 ± 20%  perf-profile.children.cycles-pp.zap_pte_range
      0.11 ± 16%      +0.1        0.17 ± 15%  perf-profile.children.cycles-pp.trigger_load_balance
      0.07 ± 27%      +0.1        0.14 ± 47%  perf-profile.children.cycles-pp.rb_next
      0.10 ±  7%      +0.1        0.16 ± 17%  perf-profile.children.cycles-pp.bprm_execve
      0.03 ±100%      +0.1        0.09 ± 17%  perf-profile.children.cycles-pp.unmap_page_range
      0.03 ±103%      +0.1        0.10 ± 29%  perf-profile.children.cycles-pp.__do_sys_clone
      0.03 ±101%      +0.1        0.10 ± 15%  perf-profile.children.cycles-pp.tick_program_event
      0.09 ± 19%      +0.1        0.16 ± 26%  perf-profile.children.cycles-pp.rcu_dynticks_eqs_enter
      0.13 ± 13%      +0.1        0.20 ± 20%  perf-profile.children.cycles-pp.execve
      0.13 ± 13%      +0.1        0.20 ± 20%  perf-profile.children.cycles-pp.__x64_sys_execve
      0.13 ± 13%      +0.1        0.20 ± 20%  perf-profile.children.cycles-pp.do_execveat_common
      0.03 ±100%      +0.1        0.10 ± 15%  perf-profile.children.cycles-pp.unmap_vmas
      0.07 ± 47%      +0.1        0.14 ± 20%  perf-profile.children.cycles-pp.tick_sched_do_timer
      0.16 ± 19%      +0.1        0.24 ± 19%  perf-profile.children.cycles-pp.rcu_eqs_enter
      0.16 ± 18%      +0.1        0.24 ± 20%  perf-profile.children.cycles-pp.cpuidle_governor_latency_req
      0.05 ± 75%      +0.1        0.12 ± 22%  perf-profile.children.cycles-pp.__libc_fork
      0.09 ± 20%      +0.1        0.18 ± 24%  perf-profile.children.cycles-pp.mmput
      0.09 ± 20%      +0.1        0.18 ± 24%  perf-profile.children.cycles-pp.exit_mmap
      0.12 ±  5%      +0.1        0.22 ± 24%  perf-profile.children.cycles-pp.__hrtimer_get_next_event
      0.19 ± 17%      +0.1        0.30 ± 16%  perf-profile.children.cycles-pp.update_irq_load_avg
      0.14 ± 33%      +0.1        0.25 ± 10%  perf-profile.children.cycles-pp.timerqueue_add
      0.13 ± 29%      +0.1        0.24 ± 12%  perf-profile.children.cycles-pp.hrtimer_get_next_event
      0.28 ± 19%      +0.1        0.40 ± 20%  perf-profile.children.cycles-pp.rcu_idle_exit
      0.13 ± 27%      +0.1        0.26 ± 19%  perf-profile.children.cycles-pp.call_cpuidle
      0.26 ± 15%      +0.1        0.40 ± 15%  perf-profile.children.cycles-pp.hrtimer_update_next_event
      0.16 ± 28%      +0.1        0.31 ±  7%  perf-profile.children.cycles-pp.enqueue_hrtimer
      0.27 ± 19%      +0.2        0.42 ± 18%  perf-profile.children.cycles-pp.__hrtimer_next_event_base
      0.11 ± 19%      +0.2        0.27 ± 54%  perf-profile.children.cycles-pp.__handle_mm_fault
      0.11 ± 21%      +0.2        0.28 ± 55%  perf-profile.children.cycles-pp.handle_mm_fault
      0.13 ± 19%      +0.2        0.30 ± 48%  perf-profile.children.cycles-pp.exc_page_fault
      0.13 ± 19%      +0.2        0.30 ± 48%  perf-profile.children.cycles-pp.do_user_addr_fault
      0.34 ± 14%      +0.2        0.52 ± 14%  perf-profile.children.cycles-pp.update_rq_clock
      0.14 ± 17%      +0.2        0.32 ± 47%  perf-profile.children.cycles-pp.asm_exc_page_fault
      0.28 ± 16%      +0.2        0.48 ± 23%  perf-profile.children.cycles-pp.__intel_pmu_enable_all
      0.28 ±  7%      +0.2        0.47 ± 12%  perf-profile.children.cycles-pp.arch_scale_freq_tick
      0.32 ±  5%      +0.2        0.52 ± 13%  perf-profile.children.cycles-pp.hrtimer_next_event_without
      0.29 ± 21%      +0.2        0.53 ± 11%  perf-profile.children.cycles-pp.get_next_timer_interrupt
      0.62 ± 14%      +0.3        0.87 ± 16%  perf-profile.children.cycles-pp.load_balance
      0.46 ±  5%      +0.3        0.73 ± 15%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      0.21 ± 11%      +0.3        0.51 ± 21%  perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
      0.21 ± 10%      +0.3        0.53 ± 21%  perf-profile.children.cycles-pp.arch_cpu_idle_enter
      0.54 ± 12%      +0.3        0.87 ± 15%  perf-profile.children.cycles-pp.native_sched_clock
      0.57 ± 11%      +0.4        0.94 ± 15%  perf-profile.children.cycles-pp.sched_clock
      0.30 ± 16%      +0.4        0.68 ± 21%  perf-profile.children.cycles-pp._raw_spin_trylock
      0.33 ± 23%      +0.4        0.74 ± 18%  perf-profile.children.cycles-pp.calc_global_load_tick
      0.66 ± 12%      +0.4        1.09 ± 16%  perf-profile.children.cycles-pp.sched_clock_cpu
      0.81 ± 17%      +0.4        1.25 ±  4%  perf-profile.children.cycles-pp.tick_nohz_irq_exit
      0.79 ± 11%      +0.5        1.29 ± 15%  perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
      0.66 ± 16%      +0.5        1.17 ± 18%  perf-profile.children.cycles-pp.read_tsc
      0.59 ± 10%      +0.5        1.13 ± 14%  perf-profile.children.cycles-pp.ktime_get_update_offsets_now
      0.62 ± 43%      +0.6        1.18 ±  5%  perf-profile.children.cycles-pp.start_kernel
      0.80 ±  5%      +0.6        1.38 ± 12%  perf-profile.children.cycles-pp.native_irq_return_iret
      0.83 ± 10%      +0.6        1.43 ± 11%  perf-profile.children.cycles-pp.lapic_next_deadline
      0.70 ± 13%      +0.6        1.30 ± 15%  perf-profile.children.cycles-pp.irqtime_account_irq
      0.98 ± 18%      +0.7        1.67 ± 13%  perf-profile.children.cycles-pp.tick_irq_enter
      1.01 ± 16%      +0.7        1.74 ± 13%  perf-profile.children.cycles-pp.irq_enter_rcu
      1.84 ± 11%      +0.8        2.62 ± 14%  perf-profile.children.cycles-pp.__softirqentry_text_start
      1.13 ±  7%      +0.9        2.00 ±  9%  perf-profile.children.cycles-pp.scheduler_tick
      0.74 ±  8%      +0.9        1.67 ± 17%  perf-profile.children.cycles-pp.rebalance_domains
      2.30 ±  9%      +1.2        3.49 ± 13%  perf-profile.children.cycles-pp.irq_exit_rcu
      2.18 ±  8%      +1.5        3.65 ± 10%  perf-profile.children.cycles-pp.update_process_times
      2.24 ±  8%      +1.5        3.75 ±  9%  perf-profile.children.cycles-pp.tick_sched_handle
      0.95 ± 24%      +1.6        2.52 ± 14%  perf-profile.children.cycles-pp.timekeeping_max_deferment
      2.62 ±  7%      +1.9        4.54 ±  9%  perf-profile.children.cycles-pp.tick_sched_timer
      1.71 ± 11%      +2.3        4.03 ±  8%  perf-profile.children.cycles-pp.tick_nohz_next_event
      2.65 ±  9%      +2.5        5.16 ± 11%  perf-profile.children.cycles-pp.clockevents_program_event
      2.11 ±  9%      +2.6        4.69 ±  8%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      4.00 ±  6%      +2.9        6.88 ±  9%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      3.64 ±  6%      +3.3        6.98 ± 10%  perf-profile.children.cycles-pp.ktime_get
      7.90 ±  4%      +6.3       14.19 ± 10%  perf-profile.children.cycles-pp.hrtimer_interrupt
      8.04 ±  4%      +6.3       14.38 ± 10%  perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
     12.70 ±  5%      +9.0       21.70 ± 10%  perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
      5.64 ± 16%     +12.2       17.88 ±  5%  perf-profile.children.cycles-pp.menu_select
     16.15 ±  6%     +13.9       30.08 ± 12%  perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
     60.05           +18.5       78.52        perf-profile.children.cycles-pp.cpuidle_enter_state
     60.06           +18.5       78.56        perf-profile.children.cycles-pp.cpuidle_enter
     66.17 ±  3%     +30.7       96.92        perf-profile.children.cycles-pp.start_secondary
     66.79 ±  2%     +31.3       98.10        perf-profile.children.cycles-pp.secondary_startup_64_no_verify
     66.79 ±  2%     +31.3       98.10        perf-profile.children.cycles-pp.cpu_startup_entry
     66.79 ±  2%     +31.3       98.10        perf-profile.children.cycles-pp.do_idle
     10.60 ±  8%     -10.5        0.12 ±  7%  perf-profile.self.cycles-pp.memcpy_toio
      6.03 ±  6%      -5.6        0.44 ± 25%  perf-profile.self.cycles-pp.io_serial_in
      1.52 ±  6%      -1.4        0.09 ± 80%  perf-profile.self.cycles-pp.delay_tsc
      0.17 ± 15%      -0.1        0.07 ± 54%  perf-profile.self.cycles-pp.poll_idle
      0.04 ± 71%      +0.0        0.08 ± 29%  perf-profile.self.cycles-pp.tick_sched_timer
      0.06 ± 52%      +0.1        0.11 ± 10%  perf-profile.self.cycles-pp.scheduler_tick
      0.03 ±103%      +0.1        0.09 ± 11%  perf-profile.self.cycles-pp.tick_nohz_get_sleep_length
      0.01 ±223%      +0.1        0.07 ± 11%  perf-profile.self.cycles-pp.update_ts_time_stats
      0.03 ±101%      +0.1        0.09 ± 13%  perf-profile.self.cycles-pp.tick_program_event
      0.08 ± 17%      +0.1        0.14 ± 23%  perf-profile.self.cycles-pp.__hrtimer_get_next_event
      0.09 ± 20%      +0.1        0.15 ± 25%  perf-profile.self.cycles-pp.rcu_dynticks_eqs_enter
      0.07 ± 51%      +0.1        0.13 ± 18%  perf-profile.self.cycles-pp.load_balance
      0.08 ± 16%      +0.1        0.15 ± 28%  perf-profile.self.cycles-pp.tick_irq_enter
      0.08 ± 27%      +0.1        0.14 ± 35%  perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
      0.08 ± 30%      +0.1        0.15 ± 17%  perf-profile.self.cycles-pp.timerqueue_add
      0.07 ± 30%      +0.1        0.14 ± 27%  perf-profile.self.cycles-pp.get_next_timer_interrupt
      0.00            +0.1        0.08 ± 33%  perf-profile.self.cycles-pp.irq_enter_rcu
      0.15 ± 11%      +0.1        0.25 ± 16%  perf-profile.self.cycles-pp.perf_mux_hrtimer_handler
      0.09 ± 35%      +0.1        0.18 ± 31%  perf-profile.self.cycles-pp.rebalance_domains
      0.18 ± 20%      +0.1        0.28 ± 25%  perf-profile.self.cycles-pp.asm_sysvec_apic_timer_interrupt
      0.19 ± 18%      +0.1        0.30 ± 16%  perf-profile.self.cycles-pp.update_irq_load_avg
      0.13 ± 27%      +0.1        0.25 ± 18%  perf-profile.self.cycles-pp.call_cpuidle
      0.14 ± 31%      +0.1        0.27 ± 13%  perf-profile.self.cycles-pp.__hrtimer_run_queues
      0.28 ± 16%      +0.2        0.48 ± 23%  perf-profile.self.cycles-pp.__intel_pmu_enable_all
      0.28 ±  7%      +0.2        0.47 ± 12%  perf-profile.self.cycles-pp.arch_scale_freq_tick
      0.32 ± 11%      +0.2        0.53 ±  8%  perf-profile.self.cycles-pp.do_idle
      0.44 ±  7%      +0.3        0.72 ± 16%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.20 ± 11%      +0.3        0.50 ± 20%  perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
      0.51 ± 12%      +0.3        0.83 ± 14%  perf-profile.self.cycles-pp.native_sched_clock
      0.30 ± 16%      +0.4        0.68 ± 20%  perf-profile.self.cycles-pp._raw_spin_trylock
      0.32 ± 22%      +0.4        0.73 ± 18%  perf-profile.self.cycles-pp.calc_global_load_tick
      0.40 ± 19%      +0.4        0.83 ± 18%  perf-profile.self.cycles-pp.update_process_times
      0.38 ± 16%      +0.4        0.81 ± 13%  perf-profile.self.cycles-pp.irqtime_account_irq
      0.48 ± 12%      +0.4        0.91 ± 15%  perf-profile.self.cycles-pp.ktime_get_update_offsets_now
      0.41 ± 19%      +0.5        0.87 ± 15%  perf-profile.self.cycles-pp.tick_nohz_next_event
      0.64 ± 15%      +0.5        1.13 ± 18%  perf-profile.self.cycles-pp.read_tsc
      0.79 ±  5%      +0.6        1.38 ± 12%  perf-profile.self.cycles-pp.native_irq_return_iret
      0.83 ± 11%      +0.6        1.42 ± 11%  perf-profile.self.cycles-pp.lapic_next_deadline
      0.94 ± 24%      +1.6        2.52 ± 14%  perf-profile.self.cycles-pp.timekeeping_max_deferment
      3.07 ±  6%      +3.0        6.02 ±  9%  perf-profile.self.cycles-pp.ktime_get
      3.32 ± 25%      +9.5       12.86 ±  9%  perf-profile.self.cycles-pp.menu_select
      5.01 ± 27%     +12.2       17.20 ± 12%  perf-profile.self.cycles-pp.cpuidle_enter_state


                                                                                
                          perf-sched.total_wait_time.max.ms                     
                                                                                
  10000 +-------------------------------------------------------------------+   
   9000 |-+                                                                 |   
        |                                                                   |   
   8000 |-+                                                                 |   
   7000 |-+                                                                 |   
        |                                                                   |   
   6000 |-+                                                                 |   
   5000 |-+                                                                 |   
   4000 |-+                                                                 |   
        |                                                                   |   
   3000 |-+                                                                 |   
   2000 |-+                                                                 |   
        |                                                                   |   
   1000 |-+                                                                 |   
      0 +-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                       perf-sched.total_wait_time.average.ms                    
                                                                                
  300 +---------------------------------------------------------------------+   
      |                                                                     |   
  250 |-+ O              O   O  O  OOOO                                     |   
      |  O OO O OO OO OO  O O OO OO    O   O                                |   
      |OO      O  O  O                                                      |   
  200 |-+               O               O O                                 |   
      |                                                                     |   
  150 |-+                                                                   |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
   50 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                     perf-sched.total_wait_and_delay.count.ms                   
                                                                                
  8000 +--------------------------------------------------------------------+   
       |                                 O                                  |   
  7000 |OO     O   O  OOO                                                   |   
  6000 |-+OOOOO  OO OO     OO OO  OO O OO  O                                |   
       |                  O  O  O   O O                                     |   
  5000 |-+                                                                  |   
       |                                                                    |   
  4000 |-+                                                                  |   
       |                                                                    |   
  3000 |-+                                                                  |   
  2000 |-+                                                                  |   
       |                                                                    |   
  1000 |-+                                                                  |   
       |+ +++ ++.+++   +++ ++++++.+++ +++ +++ +++ .++++ ++++++++  +.++++++++|   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                       perf-sched.total_wait_and_delay.max.ms                   
                                                                                
  10000 +-------------------------------------------------------------------+   
   9000 |-+                                                                 |   
        |                                                                   |   
   8000 |-+                                                                 |   
   7000 |-+                                                                 |   
        |                                                                   |   
   6000 |-+                                                                 |   
   5000 |-+                                                                 |   
   4000 |-+                                                                 |   
        |                                                                   |   
   3000 |-+                                                                 |   
   2000 |-+                                                                 |   
        |                                                                   |   
   1000 |-+                                                                 |   
      0 +-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                    perf-sched.total_wait_and_delay.average.ms                  
                                                                                
  300 +---------------------------------------------------------------------+   
      |                                                                     |   
  250 |-+ O     O  O     O   O  O  OOOO                                     |   
      |  O OO O  O  O OO  O O OO OO    O   O                                |   
      |OO      O  O  O                                                      |   
  200 |-+               O               O O                                 |   
      |                                                                     |   
  150 |-+                                                                   |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
   50 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
      perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read    
                                                                                
  1200 +--------------------------------------------------------------------+   
       |                                                                    |   
  1000 |OOOOOO OOOOOOOOOOOOO OOOOOOOOOOOO OOO                               |   
       |                                                                    |   
       |                                                                    |   
   800 |-+                                                                  |   
       |                                                                    |   
   600 |-+                                                                  |   
       |                                                                    |   
   400 |-+                                                                  |   
       |                                                                    |   
       |                                                                    |   
   200 |-+                                                                  |   
       |                                                                    |   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
     perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read     
                                                                                
  90 +----------------------------------------------------------------------+   
     | O      OOOOOO   O O  O   O   O O                                     |   
  80 |-+             O    OO   O  O    O O O                                |   
  70 |-+   OO         O O    O   O        O                                 |   
     |                                                                      |   
  60 |-+                                                                    |   
  50 |-+                                                                    |   
     |                                                                      |   
  40 |-+                                                                    |   
  30 |-+                                                                    |   
     |                                                                      |   
  20 |-+                                                                    |   
  10 |-+                                                                    |   
     |                                                                      |   
   0 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
    perf-sched.wait_and_delay.count.pipe_read.new_sync_read.vfs_read.ksys_read  
                                                                                
  2500 +--------------------------------------------------------------------+   
       |    O          O O                                                  |   
       |     O        O    OO OO  OO    OOOO                                |   
  2000 |-O     O OOOOO  O    O  O    O                                      |   
       |O OO  O           O         O OO                                    |   
       |                                                                    |   
  1500 |-+                                                                  |   
       |                                                                    |   
  1000 |-+                                                                  |   
       |                                                                    |   
       |                                                                    |   
   500 |-+                                                                  |   
       |                     + +                    +++           +  ++     |   
       |  +++ ++.+++    +  ++ ::+.++  +++ ++  +++ .+   :+++ ++ +  :+ : + +++|   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
    perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 
                                                                                
  1200 +--------------------------------------------------------------------+   
       |                                                                    |   
  1000 |OOOOOOOO OOOOOOOOOOOOOOOO OOOOOOOOOO                                |   
       |                                                                    |   
       |                                                                    |   
   800 |-+                                                                  |   
       |                                                                    |   
   600 |-+                                                                  |   
       |                                                                    |   
   400 |-+                                                                  |   
       |                                                                    |   
       |                                                                    |   
   200 |-+                                                                  |   
       |                                                                    |   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
   perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read  
                                                                                
  90 +----------------------------------------------------------------------+   
     | O      OOOOOO  O  O  O  O    O O                                     |   
  80 |-+            O     OO  O  O     OO O                                 |   
  70 |-+  O O        O  O    O  O        O                                  |   
     |                                                                      |   
  60 |-+                                                                    |   
  50 |-+                                                                    |   
     |                                                                      |   
  40 |-+                                                                    |   
  30 |-+                                                                    |   
     |                                                                      |   
  20 |-+                                                                    |   
  10 |-+                                                                    |   
     |                                                                      |   
   0 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
        perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork     
                                                                                
  6000 +--------------------------------------------------------------------+   
       |  O                                                                 |   
  5000 |-+                          O                                       |   
       |                                                                    |   
       |                                                                    |   
  4000 |-+                                                                  |   
       |                                O                                   |   
  3000 |-+                                                                  |   
       |                                                                    |   
  2000 |-+                                                                  |   
       |                                                                    |   
       |                                                                    |   
  1000 |OO OOO OOOOOOOOOOOOO OOOOOOO OOO  OOO                               |   
       |                                                                    |   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
        perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork     
                                                                                
  800 +---------------------------------------------------------------------+   
      |OOOOO OOOOOOOO O OOOOOOOOOOO O OO OO                                 |   
  700 |-+            O               O  O  O                                |   
  600 |-+                                                                   |   
      |                                                                     |   
  500 |-+                                                                   |   
      |                                                                     |   
  400 |-+                                                                   |   
      |                                                                     |   
  300 |-+                                                                   |   
  200 |-+                                                                   |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
      perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork   
                                                                                
  1400 +--------------------------------------------------------------------+   
       |OOOO  OO O OO OOOOO  O  O   OOOOO  O                                |   
  1200 |-+  OO    O  O     OO OO  OO     OO                                 |   
       |                                                                    |   
  1000 |-+                                                                  |   
       |                                                                    |   
   800 |-+                                                                  |   
       |                                                                    |   
   600 |-+                                                                  |   
       |                                                                    |   
   400 |-+                                                                  |   
       |                                                                    |   
   200 |-+                                            +                     |   
       |++++++++.++++++++++++++++.++++++++++++++++.+++ ++++++++++++.++++++++|   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
     perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork   
                                                                                
  6000 +--------------------------------------------------------------------+   
       |  O                                                                 |   
  5000 |-+                          O                                       |   
       |                                                                    |   
       |                                                                    |   
  4000 |-+                                                                  |   
       |                                O                                   |   
  3000 |-+                                                                  |   
       |                                                                    |   
  2000 |-+                                                                  |   
       |                                                                    |   
       |                                                                    |   
  1000 |OO OOOOO OOOOOOOOOOOOOOOO OO OOO OOO                                |   
       |                                                                    |   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
     perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork   
                                                                                
  800 +---------------------------------------------------------------------+   
      |OOOOOO OOOOOOO OOOOO OOOOOOOO OO O O                                 |   
  700 |-+            O              O  O   O                                |   
  600 |-+                                                                   |   
      |                                                                     |   
  500 |-+                                                                   |   
      |                                                                     |   
  400 |-+                                                                   |   
      |                                                                     |   
  300 |-+                                                                   |   
  200 |-+                                                                   |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  10000 +-------------------------------------------------------------------+   
   9000 |-+                                                                 |   
        |                                                                   |   
   8000 |-+                                                                 |   
   7000 |-+                                                                 |   
        |                                                                   |   
   6000 |-+                                                                 |   
   5000 |-+                                                                 |   
   4000 |-+                                                                 |   
        |                                                                   |   
   3000 |-+                                                                 |   
   2000 |-+                                                                 |   
        |                                                                   |   
   1000 |-+                                                                 |   
      0 +-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  20 +----------------------------------------------------------------------+   
  18 |O+OOO O O OOO OOO OOO OOO OOO OOO OO OOO OOO O                        |   
     |                                                                      |   
  16 |-+                                                                    |   
  14 |-+                                                                    |   
     |                                                                      |   
  12 |-+                                                                    |   
  10 |-+                                                                    |   
   8 |-+                                                                    |   
     |                                                                      |   
   6 |-+                                                                    |   
   4 |-+                                                                    |   
     |                                                                      |   
   2 |-+                                                                    |   
   0 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  1100 +--------------------------------------------------------------------+   
  1000 |OOOO  OO OOOOOOOOOO  O  O   OOOOO  O                                |   
       |    OO             OO OO  OO     OO                                 |   
   900 |-+                                                                  |   
   800 |-+                                                                  |   
   700 |-+                                                                  |   
   600 |-+                                                                  |   
       |                                                                    |   
   500 |-+                                                                  |   
   400 |-+                                                                  |   
   300 |-+                                                                  |   
   200 |-+                                                                  |   
       |                                                                    |   
   100 |++++++++.++++++++++++++++.++++++++++++++++.++++++++++++++++.++++++++|   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  10000 +-------------------------------------------------------------------+   
   9000 |-+                                                                 |   
        |                                                                   |   
   8000 |-+                                                                 |   
   7000 |-+                                                                 |   
        |                                                                   |   
   6000 |-+                                                                 |   
   5000 |-+                                                                 |   
   4000 |-+                                                                 |   
        |                                                                   |   
   3000 |-+                                                                 |   
   2000 |-+                                                                 |   
        |                                                                   |   
   1000 |-+                                                                 |   
      0 +-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  20 +----------------------------------------------------------------------+   
  18 |O+OOO O O OOO OOO OOO OOO OOO OOO OO OOO OOO O                        |   
     |                                                                      |   
  16 |-+                                                                    |   
  14 |-+                                                                    |   
     |                                                                      |   
  12 |-+                                                                    |   
  10 |-+                                                                    |   
   8 |-+                                                                    |   
     |                                                                      |   
   6 |-+                                                                    |   
   4 |-+                                                                    |   
     |                                                                      |   
   2 |-+                                                                    |   
   0 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
           perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork      
                                                                                
  10000 +-------------------------------------------------------------------+   
   9000 |O+  OOO        O              O O                                  |   
        |  OO   O O O OO OOOOOOOOOO OOO O  OO                               |   
   8000 |-O      O   O             O                                        |   
   7000 |-+                                                                 |   
        |                                                                   |   
   6000 |-+                                                                 |   
   5000 |-+                                                                 |   
   4000 |-+                                                                 |   
        |                                                                   |   
   3000 |-+                                                                 |   
   2000 |-+                                                                 |   
        |                                                                   |   
   1000 |-+                                                                 |   
      0 +-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
          perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork       
                                                                                
  600 +---------------------------------------------------------------------+   
      |                                                                     |   
  500 |-+              O     O  O                                           |   
      | O O O   O  O  O  OO         O O O O                                 |   
      |O O O   O OO OO  O   O OO OOO O O                                    |   
  400 |-+     O                            O                                |   
      |                                                                     |   
  300 |-+                                                                   |   
      |                                                                     |   
  200 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
        perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork     
                                                                                
  700 +---------------------------------------------------------------------+   
      |                     O  O                                            |   
  600 |OOOOOO   OO OOOOOOOO  OO OOOOOOO O  O                                |   
      |       OO  O                    O  O                                 |   
  500 |-+                                                                   |   
      |                                                                     |   
  400 |-+                                                                   |   
      |                                                                     |   
  300 |-+                                                                   |   
      |                                                                     |   
  200 |-+                                                                   |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
        perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork    
                                                                                
  10000 +-------------------------------------------------------------------+   
   9000 |O+  OOO        O              O O                                  |   
        |  OO   O O O OO OOOOOOOOOO OOO O  OO                               |   
   8000 |-O      O   O             O                                        |   
   7000 |-+                                                                 |   
        |                                                                   |   
   6000 |-+                                                                 |   
   5000 |-+                                                                 |   
   4000 |-+                                                                 |   
        |                                                                   |   
   3000 |-+                                                                 |   
   2000 |-+                                                                 |   
        |                                                                   |   
   1000 |-+                                                                 |   
      0 +-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
       perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork     
                                                                                
  600 +---------------------------------------------------------------------+   
      |                                                                     |   
  500 |-+              O     O  O     O                                     |   
      | O O O   O  O  O  OO         O   O O                                 |   
      |O O O   O OO OO  O   O OO OOO O O                                    |   
  400 |-+     O                            O                                |   
      |                                                                     |   
  300 |-+                                                                   |   
      |                                                                     |   
  200 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  2500 +--------------------------------------------------------------------+   
       |                                                                    |   
       |                                  O                                 |   
  2000 |-+                                                                  |   
       |                                                                    |   
       |                                                                    |   
  1500 |-+                                                                  |   
       | O                               O                                  |   
  1000 |-+               O                                                  |   
       |O                                                                   |   
       |  O    O   O    O                                                   |   
   500 |-+   O        OO                                                    |   
       |   OO O  OO OO    OOOOOOO OOOOOOO  O                                |   
       |                                                                    |   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  300 +---------------------------------------------------------------------+   
      |O O     O  O  OOOO   O  O  OOOO O  OO                                |   
  250 |-O OOO O OO OO    OO  OO OO    O O                                   |   
      |                                                                     |   
      |                                                                     |   
  200 |-+                                                                   |   
      |                                                                     |   
  150 |-+                                                                   |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
   50 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  250 +---------------------------------------------------------------------+   
      |                                                                     |   
      |                                                                     |   
  200 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
  150 |-+                                                                   |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
   50 |-+                                                                   |   
      |                                                                     |   
      |  +            +       +   +  +     +  +                    +        |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  300 +---------------------------------------------------------------------+   
      |O O     O  O  OOOO   O  O  OOOO O  OO                                |   
  250 |-O OOO O OO OO    OO  OO OO    O O                                   |   
      |                                                                     |   
      |                                                                     |   
  200 |-+                                                                   |   
      |                                                                     |   
  150 |-+                                                                   |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
   50 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
  perf-sched.wait_time.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 
                                                                                
  16 +----------------------------------------------------------------------+   
     |         O  O                                                         |   
  14 |OOOOO   O OO OOOO OOOOO OO O  O OOO O                                 |   
  12 |-+    OO               O  O O  O   O                                  |   
     |                                                                      |   
  10 |-+                                                                    |   
     |                                                                      |   
   8 |-+                                                                    |   
     |                                                                      |   
   6 |-+                                                                    |   
   4 |-+                                                                    |   
     |                                                                      |   
   2 |-+                                                                    |   
     |                                                                      |   
   0 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
   perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
                                                                                
  0.9 +---------------------------------------------------------------------+   
      | O O   OO OO OO   O   O  O  OOO                                      |   
  0.8 |O+O      O  O   O    O OO OO   OO   O                                |   
  0.7 |-+  OO         O O O             O O                                 |   
      |                                                                     |   
  0.6 |-+                                                                   |   
  0.5 |-+                                                                   |   
      |                                                                     |   
  0.4 |-+                                                                   |   
  0.3 |-+                                                                   |   
      |                                                                     |   
  0.2 |-+                                                                   |   
  0.1 |-+                                                                   |   
      |                  +    +                           +  +          +   |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
perf-sched.wait_and_delay.count.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_6
                                                                                
  250 +---------------------------------------------------------------------+   
      |O O     OOOOOO  O O            O                                     |   
      | O O   O              O  O  O O                                      |   
  200 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
  150 |-+                                                                   |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
   50 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  16 +----------------------------------------------------------------------+   
     |         O  O                                                         |   
  14 |OOOOO   O OO OOOO OOOOO OO O  O OOO O                                 |   
  12 |-+    OO               O  O O  O   O                                  |   
     |                                                                      |   
  10 |-+                                                                    |   
     |                                                                      |   
   8 |-+                                                                    |   
     |                                                                      |   
   6 |-+                                                                    |   
   4 |-+                                                                    |   
     |                                                                      |   
   2 |-+                                                                    |   
     |                                                                      |   
   0 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  0.9 +---------------------------------------------------------------------+   
      |OOO     O OO OO O O          O                                       |   
  0.8 |-+       O  O        O OO OO   OO   O                                |   
  0.7 |-+  OO         O O O             O O                                 |   
      |                                                                     |   
  0.6 |-+                                                                   |   
  0.5 |-+                                                                   |   
      |                                                                     |   
  0.4 |-+                                                                   |   
  0.3 |-+                                                                   |   
      |                                                                     |   
  0.2 |-+                                                                   |   
  0.1 |-+                                                                   |   
      |               +  +    +                           +  +          +   |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
    perf-sched.sch_delay.max.ms.schedule_timeout.kcompactd.kthread.ret_from_fork
                                                                                
   0.05 +-------------------------------------------------------------------+   
  0.045 |-+                   O   O                                         |   
        |                              O                                    |   
   0.04 |-+      O  O                                                       |   
  0.035 |-+    O           O                                                |   
        |                                                                   |   
   0.03 |-+                                                                 |   
  0.025 |OOOO     OO OO  O           O  O                                   |   
   0.02 |-+                         O O    O                                |   
        |              O                    O                               |   
  0.015 |-+  OO         O O OO OO  O     OO                                 |   
   0.01 |-+                                                                 |   
        |                                                                   |   
  0.005 |-+                                                                :|   
      0 +-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  40 +----------------------------------------------------------------------+   
     |         O  O        O   O  O       O                                 |   
  35 |-+                                                                    |   
  30 |-+                                                                    |   
     |                                                                      |   
  25 |-+                                                                    |   
     |                                                                      |   
  20 |-+                                                                    |   
     |                                                                      |   
  15 |-+                                                                    |   
  10 |-+                                                                    |   
     |                                                                      |   
   5 |-+                                                                    |   
     |                                                                      |   
   0 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
perf-sched.wait_and_delay.max.ms.schedule_timeout.kcompactd.kthread.ret_from_for
                                                                                
  600 +---------------------------------------------------------------------+   
      |                                                                     |   
  500 |OOOOO OOOOOOOOOO OOOOOOOOOOO OOOOOOOO                                |   
      |                                                                     |   
      |                                                                     |   
  400 |-+                                                                   |   
      |                                                                     |   
  300 |-+                                                                   |   
      |                                                                     |   
  200 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
  100 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
perf-sched.wait_and_delay.avg.ms.schedule_timeout.kcompactd.kthread.ret_from_for
                                                                                
  500 +---------------------------------------------------------------------+   
  450 |OOOOO OOOOOOOOOO OOOOOOOOOOO OOOOOOOO                                |   
      |                                                                     |   
  400 |-+                                                                   |   
  350 |-+                                                                   |   
      |                                                                     |   
  300 |-+                                                                   |   
  250 |-+                                                                   |   
  200 |-+                                                                   |   
      |                                                                     |   
  150 |-+                                                                   |   
  100 |-+                                                                   |   
      |                                                                     |   
   50 |-+                                                                   |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  50 +----------------------------------------------------------------------+   
  45 |-+        O  O    O   O   O O O                                       |   
     |                                     O                                |   
  40 |-+   OO              O   O     O                                      |   
  35 |-+       O  O   O      O   O                                          |   
     |  OO   OO  O     O               O OO                                 |   
  30 |O+             O    O        O  O                                     |   
  25 |-+                                                                    |   
  20 |-O                 O                                                  |   
     |                                                                      |   
  15 |-+                                                                    |   
  10 |-+                                                                    |   
     |                                                                      |   
   5 |-+                                                                    |   
   0 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  450 +---------------------------------------------------------------------+   
      |                   O                                                 |   
  400 |-+                                                                   |   
  350 |O+                  O                                                |   
      |  O    OO  O  O              O  OOOO                                 |   
  300 |-+ O     O  O    O     O  O                                          |   
  250 |-+  O O        O     O  O                                            |   
      |                  O   O  O O   O    O                                |   
  200 |-+        O  O                O                                      |   
  150 |-+                                                                   |   
      |                                                                     |   
  100 |-+                                                                   |   
   50 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  450 +---------------------------------------------------------------------+   
      | O                                                                   |   
  400 |-+                 O                                                 |   
  350 |-+                                                                   |   
      |O             O     O        O  OO                                   |   
  300 |-+OO   OOO OO    O                OO                                 |   
  250 |-+  O O        O     O OO O                                          |   
      |                  O            O    O                                |   
  200 |-+        O  O        O  O O  O                                      |   
  150 |-+                                                                   |   
      |                                                                     |   
  100 |-+                                                                   |   
   50 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  20 +----------------------------------------------------------------------+   
     |                                                                      |   
     |                                                                      |   
  15 |-+                                                                    |   
     |                                                                      |   
     |                                                                      |   
     |                                                                      |   
  10 |-+                                                                    |   
     |                                                                      |   
     |                                                                      |   
   5 |-+                                                                    |   
     |                                                                      |   
     |                                                                      |   
     |                                              .+.+                    |   
   0 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  1000 +--------------------------------------------------------------------+   
   900 |-+                                                                  |   
       |                                                                    |   
   800 |-+                                                                  |   
   700 |-+                                                                  |   
       |                                                                    |   
   600 |-+                                                                  |   
   500 |-+                                                                  |   
   400 |-+                                                                  |   
       |                                                                    |   
   300 |-+                                                                  |   
   200 |-+                                                                  |   
       |                                                                    |   
   100 |-+                                                                  |   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  900 +---------------------------------------------------------------------+   
      |                                                                     |   
  800 |-+                                                                   |   
  700 |-+                                                                   |   
      |                                                                     |   
  600 |-+                                                                   |   
  500 |-+                                                                   |   
      |                                                                     |   
  400 |-+                                                                   |   
  300 |-+                                                                   |   
      |                                                                     |   
  200 |-+                                                                   |   
  100 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  14 +----------------------------------------------------------------------+   
     |O                                                                     |   
  12 |-+ O   O      OO                                                      |   
     | OO O O OOOOOO  O OOOOOOOOOOO OOOOOOO                                 |   
  10 |-+                                                                    |   
     |                                                                      |   
   8 |-+                                                                    |   
     |                                                                      |   
   6 |-+                                                                    |   
     |                                                                      |   
   4 |-+                                                                    |   
     |                                                                      |   
   2 |-+                                                                    |   
     | +  +.    +                  .+  +    +   +        +.+ ++         ++  |   
   0 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  1200 +--------------------------------------------------------------------+   
       |                                                                    |   
  1000 |OOOO OOOOOOOOO OOOOOOOOO OOOOOOOO OOOO                              |   
       |                                                                    |   
       |                                                                    |   
   800 |-+                                                                  |   
       |                                                                    |   
   600 |-+                                                                  |   
       |                                                                    |   
   400 |-+                                                                  |   
       |                                                                    |   
       |                                                                    |   
   200 |-+                                                                  |   
       |                                                                    |   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                                                                
                                                                                
  900 +---------------------------------------------------------------------+   
      | OO  OO OOOO OO  OOOO OOOOOOOO OOOOOOO                               |   
  800 |-+  O  O       OO                                                    |   
  700 |O+                                                                   |   
      |                                                                     |   
  600 |-+                                                                   |   
  500 |-+                                                                   |   
      |                                                                     |   
  400 |-+                                                                   |   
  300 |-+                                                                   |   
      |                                                                     |   
  200 |-+                                                                   |   
  100 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                           stress-ng.time.system_time                           
                                                                                
  30 +----------------------------------------------------------------------+   
     |                                                                      |   
  25 |-++    +        +    ++++ +    +   +   +    +   ++++   +              |   
     |  ::+.+ ++ +++++ + + :   :::   ::++ :  :+ ++ +++    + + :++++++ .++  +|   
     |++ +      +       + +    + ++.+ +   +++  +           +  +      +   ++ |   
  20 |-+                                                                    |   
     |                                                                      |   
  15 |-+                                                                    |   
     |                                                                      |   
  10 |-+                                                                    |   
     |                                                                      |   
     |                                                                      |   
   5 |-+                                                                    |   
     |                                                                      |   
   0 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                            stress-ng.time.elapsed_time                         
                                                                                
  900 +---------------------------------------------------------------------+   
      |                                                                     |   
  800 |-+                                                                   |   
  700 |-+O O   OOOOOO       O  O      O                                     |   
      |                                                                     |   
  600 |-+                                                                   |   
  500 |-O O O        O   O        O    OO OO                                |   
      |                                                                     |   
  400 |-+     O           O                                                 |   
  300 |O+              O           O                                        |   
      |                                                                     |   
  200 |-+                                                                   |   
  100 |-+                                                                   |   
      |++++++.+++++++++++++.+++++++++++++.+++++++++++++.+++++++++++++.++++++|   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                          stress-ng.time.elapsed_time.max                       
                                                                                
  900 +---------------------------------------------------------------------+   
      |                                                                     |   
  800 |-+                                                                   |   
  700 |-+O O   OOOOOO       O  O      O                                     |   
      |                                                                     |   
  600 |-+                                                                   |   
  500 |-O O O        O   O        O    OO OO                                |   
      |                                                                     |   
  400 |-+     O           O                                                 |   
  300 |O+              O           O                                        |   
      |                                                                     |   
  200 |-+                                                                   |   
  100 |-+                                                                   |   
      |++++++.+++++++++++++.+++++++++++++.+++++++++++++.+++++++++++++.++++++|   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                      stress-ng.time.voluntary_context_switches                 
                                                                                
  90000 +-------------------------------------------------------------------+   
        |                      +                                            |   
  80000 |-+                    :+                    +                      |   
  70000 |+++       .+    +     : +  + +      + +     :: +          +    +  +|   
        |: :++    + :  + :    :   : : :++    : :++  : + ::  +  +  +:  + :: :|   
  60000 |-::  +   :  : :+ : ++:   +: ::  +. : ::  + +  : +  :: ::+  :+: :++ |   
  50000 |-:    ++:   ++   + : +    + :     ++ :    +   +  +: +: +   +  :    |   
        | +      +         +         +        +            +  +        +    |   
  40000 |-+                                                                 |   
  30000 |-+                                                                 |   
        |                                                                   |   
  20000 |-+                                                                 |   
  10000 |-+                                                                 |   
        |         O   O   O O                                               |   
      0 +-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                    stress-ng.time.involuntary_context_switches                 
                                                                                
  4500 +--------------------------------------------------------------------+   
       |  +   +       +++   + ++  +   +   +   +   .+   ++++  +              |   
  4000 |-+:: + +.  +++   :+ :+  :+ :  :: + :  :: +  :  :   :+ :++ +.    +  +|   
  3500 |++ ++    ++      + +    +  +++ ++  +++ ++   +++    +  +  +  ++++ ++ |   
       |                                                                    |   
  3000 |-+                                                                  |   
  2500 |-+                                                                  |   
       |                                                                    |   
  2000 |-+                                                                  |   
  1500 |-+                                                                  |   
       |                                                                    |   
  1000 |-+                                                                  |   
   500 |-+                                                                  |   
       |                                                                    |   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                stress-ng.loop.ops                              
                                                                                
  1800 +--------------------------------------------------------------------+   
       |  +   +        ++   +     +   +   +   +   .+      +  +              |   
  1600 |-+:: + :   +  +  :+ :+++ + +  :: + +  :: +  : ++++ :+ :+  +.+     + |   
  1400 |++ ++  +.++ ++   + +    +   ++ ++   ++ ++   ++     +  + ++   +++++ +|   
       |                                                                    |   
  1200 |-+                                                                  |   
  1000 |-+                                                                  |   
       |                                                                    |   
   800 |-+                                                                  |   
   600 |-+                                                                  |   
       |                                                                    |   
   400 |-+                                                                  |   
   200 |-+                                                                  |   
       |                                                                    |   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                           stress-ng.loop.ops_per_sec                           
                                                                                
  30 +----------------------------------------------------------------------+   
     |  +    +       ++    +    +    +   +   +    +      +   +              |   
  25 |-+:: .+ :  +  +  +   :+++ ::   :: + :  :+  + : ++++ + + :+  ++      + |   
     |++ ++   +++ ++    +++    + ++.+ ++  +++  ++  ++      +  + ++  ++.+++ +|   
     |                                                                      |   
  20 |-+                                                                    |   
     |                                                                      |   
  15 |-+                                                                    |   
     |                                                                      |   
  10 |-+                                                                    |   
     |                                                                      |   
     |                                                                      |   
   5 |-+                                                                    |   
     |                                                                      |   
   0 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                   stress-ng.time.percent_of_cpu_this_job_got                   
                                                                                
  42 +----------------------------------------------------------------------+   
     |                :                                                     |   
  41 |-++    +        :    +         +   +   +    +                         |   
  40 |-+:    :       : :   : ++ +    :   :   :    :   + ++   +              |   
     |  :    :       : :   :::: :    :   :   ::   :   ::::   :              |   
  39 |-+:   ::      ++ :   :+ : :    :  ::   ::  ::   :+ :  +: +  +         |   
     |  ::  : :     :  :   :  : :    :: ::   ::  : :  :   : :: :  :         |   
  38 |-: :  + :     :  :   :  : :   : : + : : :  + : :    : :: :: :         |   
     | : : :  :    :    : :    : :  : : : : :  : : : :    : :: ::: :        |   
  37 |-: : :  :  ++:    : :    : :  : ::  : :  ::  :+:    : : ::+: :    +  +|   
  36 |-+ ++   ++ : +    ++:    : :  + ++  : +  ++  + +    ::  :  + ++  +: + |   
     | :        :         :    : : +      : :             ::  :      :+  :: |   
  35 |++        +         +    + ++       ++               :  :      +   +  |   
     |                                                     :  :             |   
  34 +----------------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-csl-2sp7: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory
=========================================================================================
class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode:
  device/gcc-9/performance/1HDD/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp7/loop/stress-ng/60s/0x5003006

commit: 
  d3c4a43d92 ("block: refactor blk_drop_partitions")
  c76f48eb5c ("block: take bd_mutex around delete_partitions in del_gendisk")

d3c4a43d9291279c c76f48eb5c084b1e15c931ae8cc 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
      2558 ±  3%     -96.2%      96.60        stress-ng.loop.ops
     42.24 ±  4%     -99.4%       0.25 ± 24%  stress-ng.loop.ops_per_sec
     62.86          +573.1%     423.13 ± 35%  stress-ng.time.elapsed_time
     62.86          +573.1%     423.13 ± 35%  stress-ng.time.elapsed_time.max
      3605 ±  4%     -96.5%     126.20 ±  6%  stress-ng.time.involuntary_context_switches
     44.50 ±  5%    -100.0%       0.00        stress-ng.time.percent_of_cpu_this_job_got
     28.07 ±  5%     -96.5%       0.99 ±  4%  stress-ng.time.system_time
     32283 ±  6%     -84.4%       5030 ± 35%  stress-ng.time.voluntary_context_switches
   9162176 ± 38%    +605.4%   64629562 ± 61%  cpuidle.C1E.usage
     94.64          +382.6%     456.80 ± 32%  uptime.boot
      8430 ±  2%    +403.8%      42470 ± 33%  uptime.idle
      1.32 ±  9%      +0.7        2.06 ± 28%  mpstat.cpu.all.irq%
      0.62 ±  6%      -0.6        0.03 ±  8%  mpstat.cpu.all.sys%
      0.21 ±  3%      -0.2        0.04 ± 18%  mpstat.cpu.all.usr%
    170429 ± 28%    +244.2%     586547 ± 31%  numa-numastat.node0.local_node
    212256 ±  8%    +203.4%     644004 ± 25%  numa-numastat.node0.numa_hit
    178422 ± 28%    +150.6%     447123 ± 37%  numa-numastat.node1.local_node
      3.00        +16860.0%     508.80 ± 98%  numa-meminfo.node0.Mlocked
      2215 ± 15%    +298.7%       8834 ± 16%  numa-meminfo.node1.Active
      2156 ± 16%    +301.9%       8666 ± 15%  numa-meminfo.node1.Active(anon)
   1363818 ± 11%     -19.9%    1092837 ±  9%  numa-meminfo.node1.MemUsed
    775.83 ± 11%     -99.5%       4.00 ± 41%  vmstat.io.bi
      1.00          -100.0%       0.00        vmstat.procs.r
      6009 ±  3%     -76.5%       1414 ±  2%  vmstat.system.cs
    184272 ±  5%      +4.0%     191637        vmstat.system.in
      0.00       +1.3e+104%     126.60 ± 98%  numa-vmstat.node0.nr_mlock
    539.33 ± 16%    +301.4%       2164 ± 15%  numa-vmstat.node1.nr_active_anon
      0.00         +1e+104%     100.00 ± 89%  numa-vmstat.node1.nr_mlock
    539.33 ± 16%    +301.4%       2164 ± 15%  numa-vmstat.node1.nr_zone_active_anon
    782921 ± 11%     +22.3%     957479 ± 11%  numa-vmstat.node1.numa_hit
      3674 ±  3%    +208.7%      11342 ± 12%  meminfo.Active
      3399 ±  3%    +225.2%      11054 ± 12%  meminfo.Active(anon)
     66602 ±  8%    +157.0%     171153 ±  3%  meminfo.AnonHugePages
   2839586           -14.6%    2424400        meminfo.Memused
      7.00        +12894.3%     909.60 ± 94%  meminfo.Mlocked
     15230           +53.2%      23338 ±  6%  meminfo.Shmem
     72.50 ±  7%     -23.0%      55.80 ± 25%  turbostat.Avg_MHz
      1807           -42.0%       1048        turbostat.Bzy_MHz
   9161353 ± 38%    +605.5%   64628765 ± 61%  turbostat.C1E
     48.00 ±  2%     -12.5%      42.00 ±  3%  turbostat.CoreTmp
  12253447 ±  6%    +570.1%   82113222 ± 35%  turbostat.IRQ
     48.00           -13.3%      41.60 ±  2%  turbostat.PkgTmp
    129.19           -39.2%      78.56 ±  8%  turbostat.PkgWatt
    849.33 ±  3%    +225.5%       2764 ± 12%  proc-vmstat.nr_active_anon
     71723            -4.3%      68656        proc-vmstat.nr_anon_pages
     74766            -4.1%      71672        proc-vmstat.nr_inactive_anon
     20825            -8.6%      19039 ±  2%  proc-vmstat.nr_kernel_stack
      9294            -3.6%       8961        proc-vmstat.nr_mapped
      1.00        +22620.0%     227.20 ± 94%  proc-vmstat.nr_mlock
      3807           +53.2%       5834 ±  6%  proc-vmstat.nr_shmem
     51872            -6.3%      48590        proc-vmstat.nr_slab_unreclaimable
    849.33 ±  3%    +225.5%       2764 ± 12%  proc-vmstat.nr_zone_active_anon
     74766            -4.1%      71672        proc-vmstat.nr_zone_inactive_anon
    474012          +143.8%    1155675 ± 27%  proc-vmstat.numa_hit
    383845          +178.2%    1067930 ± 30%  proc-vmstat.numa_local
     90166            -2.7%      87745        proc-vmstat.numa_other
      7912 ±  3%     -59.8%       3178 ± 19%  proc-vmstat.pgactivate
    609137          +103.7%    1240840 ± 26%  proc-vmstat.pgalloc_normal
    253503 ±  2%    +402.0%    1272525 ± 33%  proc-vmstat.pgfault
    454866 ±  2%    +167.4%    1216354 ± 27%  proc-vmstat.pgfree
     51740 ± 12%     -96.6%       1734 ± 19%  proc-vmstat.pgpgin
     15883 ±  2%    +412.7%      81427 ± 32%  proc-vmstat.pgreuse
      7809 ±  9%     -93.0%     546.40 ± 14%  proc-vmstat.unevictable_pgs_culled
 4.497e+08 ±  2%     -57.9%  1.894e+08 ±  8%  perf-stat.i.branch-instructions
      5951 ±  3%     -77.2%       1358 ±  3%  perf-stat.i.context-switches
      3.58 ±  6%     +64.2%       5.88 ± 27%  perf-stat.i.cpi
 6.279e+09 ±  9%     -22.3%  4.877e+09 ± 29%  perf-stat.i.cpu-cycles
    179.19           -44.7%      99.04        perf-stat.i.cpu-migrations
 5.161e+08 ±  2%     -53.1%  2.421e+08 ±  4%  perf-stat.i.dTLB-loads
 2.389e+08           -46.9%  1.268e+08 ±  3%  perf-stat.i.dTLB-stores
 2.184e+09 ±  2%     -58.1%  9.142e+08 ±  8%  perf-stat.i.instructions
      1567 ±  6%     -53.7%     726.17 ± 26%  perf-stat.i.instructions-per-iTLB-miss
      0.32 ±  5%     -39.8%       0.19 ± 23%  perf-stat.i.ipc
     11.79 ±  3%     -81.1%       2.23 ± 14%  perf-stat.i.major-faults
      0.07 ±  9%     -22.3%       0.05 ± 29%  perf-stat.i.metric.GHz
     12.54 ±  2%     -53.6%       5.81 ±  5%  perf-stat.i.metric.M/sec
      3473 ±  2%     -17.4%       2867        perf-stat.i.minor-faults
     77.21 ±  2%     +11.7       88.91 ±  3%  perf-stat.i.node-load-miss-rate%
    543244 ±  3%     -68.4%     171439 ± 49%  perf-stat.i.node-load-misses
    161064 ±  8%     -83.6%      26449 ± 59%  perf-stat.i.node-loads
    347247 ±  2%     -90.5%      33066 ± 14%  perf-stat.i.node-store-misses
    140453 ± 19%     -88.3%      16414 ± 64%  perf-stat.i.node-stores
      3528 ±  2%     -18.7%       2869        perf-stat.i.page-faults
      2.87 ±  6%     +84.2%       5.29 ± 25%  perf-stat.overall.cpi
      1607 ±  7%     -53.7%     744.06 ± 28%  perf-stat.overall.instructions-per-iTLB-miss
      0.35 ±  6%     -42.3%       0.20 ± 24%  perf-stat.overall.ipc
     77.11 ±  2%      +9.6       86.71 ±  2%  perf-stat.overall.node-load-miss-rate%
 4.427e+08 ±  2%     -57.3%   1.89e+08 ±  8%  perf-stat.ps.branch-instructions
      5858 ±  3%     -76.9%       1355 ±  3%  perf-stat.ps.context-switches
     94511            +1.3%      95756        perf-stat.ps.cpu-clock
  6.18e+09 ±  9%     -21.3%  4.865e+09 ± 29%  perf-stat.ps.cpu-cycles
    176.39           -44.0%      98.79        perf-stat.ps.cpu-migrations
 5.081e+08 ±  2%     -52.4%  2.416e+08 ±  4%  perf-stat.ps.dTLB-loads
 2.351e+08           -46.2%  1.265e+08 ±  3%  perf-stat.ps.dTLB-stores
  2.15e+09 ±  2%     -57.6%  9.123e+08 ±  8%  perf-stat.ps.instructions
     11.74 ±  2%     -80.8%       2.25 ± 15%  perf-stat.ps.major-faults
      3421 ±  2%     -16.4%       2861        perf-stat.ps.minor-faults
    534662 ±  3%     -68.0%     171047 ± 49%  perf-stat.ps.node-load-misses
    158578 ±  8%     -83.4%      26391 ± 59%  perf-stat.ps.node-loads
    341702 ±  2%     -90.3%      32994 ± 14%  perf-stat.ps.node-store-misses
    138263 ± 19%     -88.2%      16380 ± 63%  perf-stat.ps.node-stores
      3475 ±  2%     -17.6%       2863        perf-stat.ps.page-faults
     94511            +1.3%      95756        perf-stat.ps.task-clock
 1.383e+11 ±  2%    +175.0%  3.803e+11 ± 26%  perf-stat.total.instructions
      0.00 ± 41%    +260.0%       0.01 ± 28%  perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.41 ± 19%     -96.3%       0.02 ± 16%  perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
      0.01 ± 24%    +416.3%       0.04 ± 50%  perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.00 ±223%   +1012.7%       0.02 ± 40%  perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      0.02 ± 70%     -57.1%       0.01 ± 32%  perf-sched.total_sch_delay.average.ms
      1.04 ± 30%  +21949.0%     230.30 ±  3%  perf-sched.total_wait_and_delay.average.ms
    568.17 ± 17%   +1022.3%       6376 ±  3%  perf-sched.total_wait_and_delay.count.ms
     16.79 ± 22%  +59410.6%       9991        perf-sched.total_wait_and_delay.max.ms
      1.03 ± 31%  +22284.1%     230.29 ±  3%  perf-sched.total_wait_time.average.ms
     16.53 ± 23%  +60361.8%       9991        perf-sched.total_wait_time.max.ms
      0.06 ± 51%  +4.4e+05%     262.65 ±  4%  perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.04 ±  7%  +1.7e+05%      73.20 ±  3%  perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.09 ± 70%  +20597.0%      18.01        perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
      3.09 ± 46%    +592.5%      21.38 ± 31%  perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      3.00 ± 21%  +23956.9%     721.95 ±  2%  perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
      0.95 ± 95%  +47375.8%     450.31 ±  3%  perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
      7.50 ± 18%   +3161.3%     244.60        perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
    193.00 ± 39%   +1041.6%       2203 ±  3%  perf-sched.wait_and_delay.count.pipe_read.new_sync_read.vfs_read.ksys_read
    169.00 ± 12%    +524.9%       1056        perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
      4.67 ± 52%  +10322.9%     486.40 ± 32%  perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
    169.50 ± 12%    +637.5%       1250        perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
      7.33 ± 15%   +7443.6%     553.20        perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
      0.32 ± 50%  +5.8e+05%       1875 ± 85%  perf-sched.wait_and_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.42 ±108%  +2.4e+05%       1008        perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      6.68 ± 66%  +1.5e+05%       9991        perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
      4.17 ± 44%   +8959.9%     377.80 ± 17%  perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
     16.26 ± 24%   +6103.2%       1008        perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
      6.30 ±100%  +1.4e+05%       8622 ±  6%  perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork
      0.06 ± 53%  +4.6e+05%     262.64 ±  4%  perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.04 ±  7%    +2e+05%      73.19 ±  3%  perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.09 ± 18%    -100.0%       0.00        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.mutex_lock.perf_event_release_kernel.perf_release
      0.09 ± 70%  +20597.0%      18.01        perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
      1.79 ± 52%  +12214.2%     220.38 ± 66%  perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      2.86 ± 46%    +646.5%      21.36 ± 31%  perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      2.99 ± 21%  +24010.2%     721.94 ±  2%  perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
      0.54 ±168%  +83831.5%     450.29 ±  3%  perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
      0.32 ± 50%  +5.9e+05%       1875 ± 85%  perf-sched.wait_time.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.40 ±118%  +2.5e+05%       1008        perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.16 ± 14%    -100.0%       0.00        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.mutex_lock.perf_event_release_kernel.perf_release
      6.68 ± 66%  +1.5e+05%       9991        perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
      8.40 ± 67%   +5505.3%     470.98 ± 10%  perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      4.16 ± 44%   +8971.3%     377.76 ± 17%  perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
     16.25 ± 24%   +6104.9%       1008        perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
      3.88 ±164%  +2.2e+05%       8622 ±  6%  perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork
      8194 ± 47%    +304.4%      33140 ± 19%  sched_debug.cfs_rq:/.load.avg
    309373 ± 59%    +107.0%     640417 ± 10%  sched_debug.cfs_rq:/.load.max
     41911 ± 49%    +196.6%     124297 ± 13%  sched_debug.cfs_rq:/.load.stddev
    754.63 ± 94%     -94.9%      38.33 ± 17%  sched_debug.cfs_rq:/.load_avg.avg
     27363 ± 97%     -97.1%     799.68 ±  6%  sched_debug.cfs_rq:/.load_avg.max
      3768 ± 93%     -96.0%     152.02 ± 10%  sched_debug.cfs_rq:/.load_avg.stddev
    586.67 ± 30%     -74.3%     150.85 ± 21%  sched_debug.cfs_rq:/.removed.load_avg.max
     92.77 ± 46%     -69.8%      27.99 ± 30%  sched_debug.cfs_rq:/.removed.load_avg.stddev
      6.34 ± 67%     -71.1%       1.83 ± 51%  sched_debug.cfs_rq:/.removed.runnable_avg.avg
    265.25 ± 43%     -73.1%      71.45 ± 29%  sched_debug.cfs_rq:/.removed.runnable_avg.max
     36.13 ± 51%     -72.4%       9.97 ± 35%  sched_debug.cfs_rq:/.removed.runnable_avg.stddev
      6.34 ± 67%     -71.1%       1.83 ± 51%  sched_debug.cfs_rq:/.removed.util_avg.avg
    265.25 ± 43%     -73.1%      71.45 ± 29%  sched_debug.cfs_rq:/.removed.util_avg.max
     36.13 ± 51%     -72.4%       9.97 ± 35%  sched_debug.cfs_rq:/.removed.util_avg.stddev
    200.97 ± 27%     -63.8%      72.78 ± 14%  sched_debug.cfs_rq:/.runnable_avg.avg
    992.42 ± 14%     -32.4%     671.37 ±  5%  sched_debug.cfs_rq:/.runnable_avg.max
    225.51 ± 18%     -38.5%     138.66 ±  6%  sched_debug.cfs_rq:/.runnable_avg.stddev
    200.33 ± 27%     -63.7%      72.67 ± 14%  sched_debug.cfs_rq:/.util_avg.avg
    992.25 ± 14%     -32.3%     671.30 ±  5%  sched_debug.cfs_rq:/.util_avg.max
    224.89 ± 18%     -38.4%     138.60 ±  6%  sched_debug.cfs_rq:/.util_avg.stddev
     27.88 ± 24%     -75.6%       6.80 ± 24%  sched_debug.cfs_rq:/.util_est_enqueued.avg
    628.83 ± 29%     -71.6%     178.57 ± 22%  sched_debug.cfs_rq:/.util_est_enqueued.max
    102.27 ± 24%     -72.0%      28.59 ± 21%  sched_debug.cfs_rq:/.util_est_enqueued.stddev
    810654 ±  5%     +14.8%     930886        sched_debug.cpu.avg_idle.avg
    215092 ± 19%     -31.7%     146899 ±  4%  sched_debug.cpu.avg_idle.stddev
     54920 ± 20%    +293.5%     216119 ± 33%  sched_debug.cpu.clock.avg
     54925 ± 20%    +293.5%     216123 ± 33%  sched_debug.cpu.clock.max
     54913 ± 20%    +293.6%     216114 ± 33%  sched_debug.cpu.clock.min
     54407 ± 20%    +289.8%     212090 ± 33%  sched_debug.cpu.clock_task.avg
     54551 ± 20%    +289.3%     212373 ± 33%  sched_debug.cpu.clock_task.max
     49667 ± 22%    +317.2%     207221 ± 34%  sched_debug.cpu.clock_task.min
    239.19 ± 21%     -36.4%     152.14 ±  5%  sched_debug.cpu.curr->pid.avg
      3888 ± 15%     +92.2%       7474 ± 23%  sched_debug.cpu.curr->pid.max
    682415 ±  7%     -20.0%     545972 ±  2%  sched_debug.cpu.max_idle_balance_cost.max
     21948 ± 23%     -73.8%       5743 ± 36%  sched_debug.cpu.max_idle_balance_cost.stddev
      0.00 ± 17%     -50.1%       0.00 ±  7%  sched_debug.cpu.next_balance.stddev
      0.10 ± 26%     -57.9%       0.04 ± 14%  sched_debug.cpu.nr_running.avg
      0.28 ± 14%     -35.1%       0.18 ±  5%  sched_debug.cpu.nr_running.stddev
      3082 ± 22%     +38.0%       4252 ± 25%  sched_debug.cpu.nr_switches.avg
     14311 ± 20%    +407.3%      72606 ± 48%  sched_debug.cpu.nr_switches.max
      1904 ± 19%    +311.4%       7836 ± 44%  sched_debug.cpu.nr_switches.stddev
      0.55 ± 43%    +124.8%       1.23 ±  4%  sched_debug.cpu.nr_uninterruptible.avg
     54916 ± 20%    +293.5%     216115 ± 33%  sched_debug.cpu_clk
     54422 ± 20%    +296.2%     215601 ± 33%  sched_debug.ktime
     55284 ± 20%    +291.6%     216494 ± 33%  sched_debug.sched_clk
      4534 ±  6%     -37.0%       2857 ±  5%  slabinfo.Acpi-Parse.active_objs
      4534 ±  6%     -37.0%       2857 ±  5%  slabinfo.Acpi-Parse.num_objs
      1667           -62.0%     634.20 ± 10%  slabinfo.bdev_cache.active_objs
      1667           -62.0%     634.20 ± 10%  slabinfo.bdev_cache.num_objs
      2916 ±  6%     -32.9%       1957 ± 15%  slabinfo.dmaengine-unmap-16.active_objs
      2916 ±  6%     -32.9%       1957 ± 15%  slabinfo.dmaengine-unmap-16.num_objs
      2500 ±  4%     -31.9%       1701 ±  4%  slabinfo.file_lock_cache.active_objs
      2500 ±  4%     -31.9%       1701 ±  4%  slabinfo.file_lock_cache.num_objs
      8070 ±  6%     -42.9%       4611 ±  8%  slabinfo.fsnotify_mark_connector.active_objs
      8070 ±  6%     -42.9%       4611 ±  8%  slabinfo.fsnotify_mark_connector.num_objs
    109405           -12.6%      95625        slabinfo.kernfs_node_cache.active_objs
      3425           -12.8%       2987        slabinfo.kernfs_node_cache.active_slabs
    109641           -12.8%      95625        slabinfo.kernfs_node_cache.num_objs
      3425           -12.8%       2987        slabinfo.kernfs_node_cache.num_slabs
     46371           -16.0%      38934 ±  3%  slabinfo.kmalloc-16.num_objs
      7232           -16.6%       6034 ±  3%  slabinfo.kmalloc-192.num_objs
      7605 ±  2%     -14.9%       6472 ±  3%  slabinfo.kmalloc-1k.active_objs
      8567 ±  2%     -20.5%       6811 ±  3%  slabinfo.kmalloc-1k.num_objs
      9644 ±  2%      -9.5%       8724 ±  3%  slabinfo.kmalloc-256.active_objs
      9924           -11.0%       8829 ±  2%  slabinfo.kmalloc-256.num_objs
      8117 ±  3%      -9.8%       7321 ±  2%  slabinfo.kmalloc-2k.num_objs
      2235           -10.6%       1997 ±  3%  slabinfo.kmalloc-4k.num_objs
     71195 ±  2%     -17.5%      58702        slabinfo.kmalloc-8.active_objs
     71456 ±  2%     -17.1%      59235        slabinfo.kmalloc-8.num_objs
      5918           +21.1%       7170 ±  2%  slabinfo.kmalloc-96.active_objs
      6963 ±  2%     +14.0%       7937 ±  3%  slabinfo.kmalloc-96.num_objs
      3483           -61.8%       1332 ± 10%  slabinfo.kmalloc-rcl-192.active_objs
      3483           -61.8%       1332 ± 10%  slabinfo.kmalloc-rcl-192.num_objs
      7253           -17.7%       5968        slabinfo.kmalloc-rcl-512.active_objs
      7269           -17.9%       5968        slabinfo.kmalloc-rcl-512.num_objs
      7652 ±  9%     -20.9%       6049 ±  2%  slabinfo.kmalloc-rcl-64.active_objs
      7652 ±  9%     -20.9%       6049 ±  2%  slabinfo.kmalloc-rcl-64.num_objs
      2832 ± 11%     -23.0%       2181 ±  6%  slabinfo.kmalloc-rcl-96.active_objs
      2832 ± 11%     -23.0%       2181 ±  6%  slabinfo.kmalloc-rcl-96.num_objs
     13921           +11.2%      15477        slabinfo.proc_inode_cache.active_objs
     13928           +11.3%      15499        slabinfo.proc_inode_cache.num_objs
      1325           -64.6%     468.60 ±  8%  slabinfo.request_queue.active_objs
      1325           -64.6%     468.60 ±  8%  slabinfo.request_queue.num_objs
      6505           -19.6%       5227 ±  2%  slabinfo.shmem_inode_cache.active_objs
      6505           -19.6%       5227 ±  2%  slabinfo.shmem_inode_cache.num_objs
      4913 ±  2%     -14.8%       4188 ±  6%  slabinfo.skbuff_head_cache.active_objs
      4953 ±  2%     -14.9%       4216 ±  7%  slabinfo.skbuff_head_cache.num_objs
      5254 ±  2%     -14.1%       4514 ±  2%  slabinfo.trace_event_file.active_objs
      5254 ±  2%     -14.1%       4514 ±  2%  slabinfo.trace_event_file.num_objs
     11601           +53.1%      17762 ±  3%  slabinfo.vmap_area.active_objs
     11618           +53.3%      17808 ±  4%  slabinfo.vmap_area.num_objs
    130.67 ±  2%    +554.2%     854.80 ± 35%  interrupts.9:IO-APIC.9-fasteoi.acpi
     64335 ±  2%     +17.8%      75806 ±  8%  interrupts.CAL:Function_call_interrupts
    126231 ±  6%    +576.8%     854314 ± 35%  interrupts.CPU0.LOC:Local_timer_interrupts
     18.83 ± 29%    +235.6%      63.20 ± 76%  interrupts.CPU0.RES:Rescheduling_interrupts
    130.67 ±  2%    +554.2%     854.80 ± 35%  interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi
    126143 ±  7%    +577.3%     854306 ± 35%  interrupts.CPU1.LOC:Local_timer_interrupts
    301.33 ± 26%     -51.7%     145.40 ± 27%  interrupts.CPU1.NMI:Non-maskable_interrupts
    301.33 ± 26%     -51.7%     145.40 ± 27%  interrupts.CPU1.PMI:Performance_monitoring_interrupts
    125919 ±  7%    +578.3%     854111 ± 35%  interrupts.CPU10.LOC:Local_timer_interrupts
    125968 ±  7%    +578.0%     854099 ± 35%  interrupts.CPU11.LOC:Local_timer_interrupts
    126177 ±  7%    +576.9%     854130 ± 35%  interrupts.CPU12.LOC:Local_timer_interrupts
    126017 ±  7%    +577.9%     854234 ± 35%  interrupts.CPU13.LOC:Local_timer_interrupts
    125936 ±  7%    +578.4%     854306 ± 35%  interrupts.CPU14.LOC:Local_timer_interrupts
    126056 ±  7%    +577.6%     854125 ± 35%  interrupts.CPU15.LOC:Local_timer_interrupts
    126016 ±  7%    +577.8%     854100 ± 35%  interrupts.CPU16.LOC:Local_timer_interrupts
    125975 ±  7%    +578.0%     854122 ± 35%  interrupts.CPU17.LOC:Local_timer_interrupts
    126066 ±  7%    +577.5%     854157 ± 35%  interrupts.CPU18.LOC:Local_timer_interrupts
    125915 ±  7%    +578.4%     854231 ± 35%  interrupts.CPU19.LOC:Local_timer_interrupts
    241.67 ± 20%     -41.1%     142.40 ± 42%  interrupts.CPU19.NMI:Non-maskable_interrupts
    241.67 ± 20%     -41.1%     142.40 ± 42%  interrupts.CPU19.PMI:Performance_monitoring_interrupts
    126113 ±  7%    +577.3%     854158 ± 35%  interrupts.CPU2.LOC:Local_timer_interrupts
    126017 ±  7%    +577.8%     854132 ± 35%  interrupts.CPU20.LOC:Local_timer_interrupts
    194.33 ± 31%     -37.9%     120.60 ± 41%  interrupts.CPU20.NMI:Non-maskable_interrupts
    194.33 ± 31%     -37.9%     120.60 ± 41%  interrupts.CPU20.PMI:Performance_monitoring_interrupts
    125982 ±  7%    +578.0%     854166 ± 35%  interrupts.CPU21.LOC:Local_timer_interrupts
    126030 ±  7%    +577.7%     854123 ± 35%  interrupts.CPU22.LOC:Local_timer_interrupts
    126070 ±  7%    +577.6%     854202 ± 35%  interrupts.CPU23.LOC:Local_timer_interrupts
    127041 ±  5%    +572.2%     853950 ± 35%  interrupts.CPU24.LOC:Local_timer_interrupts
    694.00 ± 12%     -16.1%     582.00 ±  5%  interrupts.CPU25.CAL:Function_call_interrupts
    126977 ±  5%    +572.7%     854144 ± 35%  interrupts.CPU25.LOC:Local_timer_interrupts
    127109 ±  5%    +572.0%     854131 ± 35%  interrupts.CPU26.LOC:Local_timer_interrupts
    127002 ±  5%    +572.5%     854109 ± 35%  interrupts.CPU27.LOC:Local_timer_interrupts
    126937 ±  5%    +572.8%     854088 ± 35%  interrupts.CPU28.LOC:Local_timer_interrupts
    225.00 ± 27%     -46.4%     120.60 ± 41%  interrupts.CPU28.NMI:Non-maskable_interrupts
    225.00 ± 27%     -46.4%     120.60 ± 41%  interrupts.CPU28.PMI:Performance_monitoring_interrupts
    127021 ±  5%    +572.6%     854378 ± 35%  interrupts.CPU29.LOC:Local_timer_interrupts
    125901 ±  7%    +578.5%     854289 ± 35%  interrupts.CPU3.LOC:Local_timer_interrupts
    127027 ±  5%    +572.3%     854066 ± 35%  interrupts.CPU30.LOC:Local_timer_interrupts
    693.67 ±  5%     -11.7%     612.20 ±  2%  interrupts.CPU31.CAL:Function_call_interrupts
    127143 ±  5%    +571.8%     854089 ± 35%  interrupts.CPU31.LOC:Local_timer_interrupts
    127006 ±  5%    +572.4%     854043 ± 35%  interrupts.CPU32.LOC:Local_timer_interrupts
    127054 ±  5%    +572.2%     854086 ± 35%  interrupts.CPU33.LOC:Local_timer_interrupts
    127025 ±  5%    +572.5%     854227 ± 35%  interrupts.CPU34.LOC:Local_timer_interrupts
    126986 ±  5%    +572.7%     854252 ± 35%  interrupts.CPU35.LOC:Local_timer_interrupts
    127205 ±  5%    +571.4%     854089 ± 35%  interrupts.CPU36.LOC:Local_timer_interrupts
    127104 ±  5%    +572.0%     854113 ± 35%  interrupts.CPU37.LOC:Local_timer_interrupts
    127080 ±  5%    +572.1%     854115 ± 35%  interrupts.CPU38.LOC:Local_timer_interrupts
    127098 ±  5%    +572.0%     854142 ± 35%  interrupts.CPU39.LOC:Local_timer_interrupts
    125973 ±  7%    +578.1%     854277 ± 35%  interrupts.CPU4.LOC:Local_timer_interrupts
    127058 ±  5%    +572.2%     854041 ± 35%  interrupts.CPU40.LOC:Local_timer_interrupts
    126956 ±  5%    +572.7%     854066 ± 35%  interrupts.CPU41.LOC:Local_timer_interrupts
    237.50 ± 32%     -33.0%     159.20 ± 28%  interrupts.CPU41.NMI:Non-maskable_interrupts
    237.50 ± 32%     -33.0%     159.20 ± 28%  interrupts.CPU41.PMI:Performance_monitoring_interrupts
    127173 ±  5%    +571.6%     854048 ± 35%  interrupts.CPU42.LOC:Local_timer_interrupts
    127129 ±  5%    +571.8%     854074 ± 35%  interrupts.CPU43.LOC:Local_timer_interrupts
    255.00 ± 14%     -49.1%     129.80 ± 50%  interrupts.CPU43.NMI:Non-maskable_interrupts
    255.00 ± 14%     -49.1%     129.80 ± 50%  interrupts.CPU43.PMI:Performance_monitoring_interrupts
    126971 ±  5%    +572.6%     854069 ± 35%  interrupts.CPU44.LOC:Local_timer_interrupts
    680.67 ±  7%     -16.4%     569.00 ±  5%  interrupts.CPU45.CAL:Function_call_interrupts
    126984 ±  5%    +572.6%     854054 ± 35%  interrupts.CPU45.LOC:Local_timer_interrupts
    126996 ±  5%    +572.5%     854065 ± 35%  interrupts.CPU46.LOC:Local_timer_interrupts
    127020 ±  5%    +572.5%     854151 ± 35%  interrupts.CPU47.LOC:Local_timer_interrupts
    126153 ±  7%    +577.1%     854165 ± 35%  interrupts.CPU48.LOC:Local_timer_interrupts
    126196 ±  7%    +576.9%     854204 ± 35%  interrupts.CPU49.LOC:Local_timer_interrupts
    126038 ±  7%    +577.7%     854208 ± 35%  interrupts.CPU5.LOC:Local_timer_interrupts
    188.33 ± 26%     -35.6%     121.20 ± 42%  interrupts.CPU5.NMI:Non-maskable_interrupts
    188.33 ± 26%     -35.6%     121.20 ± 42%  interrupts.CPU5.PMI:Performance_monitoring_interrupts
    126052 ±  7%    +577.7%     854242 ± 35%  interrupts.CPU50.LOC:Local_timer_interrupts
    249.67 ± 24%     -45.1%     137.00 ± 30%  interrupts.CPU50.NMI:Non-maskable_interrupts
    249.67 ± 24%     -45.1%     137.00 ± 30%  interrupts.CPU50.PMI:Performance_monitoring_interrupts
    126056 ±  7%    +577.6%     854210 ± 35%  interrupts.CPU51.LOC:Local_timer_interrupts
    126122 ±  7%    +577.3%     854190 ± 35%  interrupts.CPU52.LOC:Local_timer_interrupts
    126114 ±  7%    +577.2%     854071 ± 35%  interrupts.CPU53.LOC:Local_timer_interrupts
    126093 ±  7%    +577.4%     854189 ± 35%  interrupts.CPU54.LOC:Local_timer_interrupts
    126033 ±  7%    +577.7%     854160 ± 35%  interrupts.CPU55.LOC:Local_timer_interrupts
    126052 ±  7%    +577.6%     854149 ± 35%  interrupts.CPU56.LOC:Local_timer_interrupts
    126164 ±  7%    +577.0%     854160 ± 35%  interrupts.CPU57.LOC:Local_timer_interrupts
    126121 ±  7%    +577.3%     854235 ± 35%  interrupts.CPU58.LOC:Local_timer_interrupts
    126149 ±  7%    +577.1%     854139 ± 35%  interrupts.CPU59.LOC:Local_timer_interrupts
    126054 ±  7%    +577.6%     854153 ± 35%  interrupts.CPU6.LOC:Local_timer_interrupts
    126163 ±  7%    +577.0%     854155 ± 35%  interrupts.CPU60.LOC:Local_timer_interrupts
    126224 ±  7%    +576.7%     854173 ± 35%  interrupts.CPU61.LOC:Local_timer_interrupts
    237.67 ± 25%     -40.9%     140.40 ± 41%  interrupts.CPU61.NMI:Non-maskable_interrupts
    237.67 ± 25%     -40.9%     140.40 ± 41%  interrupts.CPU61.PMI:Performance_monitoring_interrupts
    126076 ±  7%    +577.6%     854298 ± 35%  interrupts.CPU62.LOC:Local_timer_interrupts
    126133 ±  7%    +577.3%     854256 ± 35%  interrupts.CPU63.LOC:Local_timer_interrupts
    126027 ±  7%    +577.7%     854111 ± 35%  interrupts.CPU64.LOC:Local_timer_interrupts
    126134 ±  7%    +577.2%     854227 ± 35%  interrupts.CPU65.LOC:Local_timer_interrupts
    244.33 ± 24%     -36.8%     154.40 ± 26%  interrupts.CPU65.NMI:Non-maskable_interrupts
    244.33 ± 24%     -36.8%     154.40 ± 26%  interrupts.CPU65.PMI:Performance_monitoring_interrupts
    126229 ±  7%    +576.7%     854235 ± 35%  interrupts.CPU66.LOC:Local_timer_interrupts
    126204 ±  7%    +576.9%     854258 ± 35%  interrupts.CPU67.LOC:Local_timer_interrupts
    213.00 ± 19%     -27.7%     154.00 ± 30%  interrupts.CPU67.NMI:Non-maskable_interrupts
    213.00 ± 19%     -27.7%     154.00 ± 30%  interrupts.CPU67.PMI:Performance_monitoring_interrupts
    126118 ±  7%    +577.4%     854274 ± 35%  interrupts.CPU68.LOC:Local_timer_interrupts
    201.17 ± 21%     -33.5%     133.80 ± 27%  interrupts.CPU68.NMI:Non-maskable_interrupts
    201.17 ± 21%     -33.5%     133.80 ± 27%  interrupts.CPU68.PMI:Performance_monitoring_interrupts
    126027 ±  7%    +578.0%     854480 ± 35%  interrupts.CPU69.LOC:Local_timer_interrupts
    126293 ±  7%    +576.3%     854077 ± 35%  interrupts.CPU7.LOC:Local_timer_interrupts
    126122 ±  7%    +577.3%     854218 ± 35%  interrupts.CPU70.LOC:Local_timer_interrupts
    192.00 ± 21%     -27.7%     138.80 ± 37%  interrupts.CPU70.NMI:Non-maskable_interrupts
    192.00 ± 21%     -27.7%     138.80 ± 37%  interrupts.CPU70.PMI:Performance_monitoring_interrupts
    126046 ±  7%    +577.7%     854240 ± 35%  interrupts.CPU71.LOC:Local_timer_interrupts
    127174 ±  5%    +571.5%     853961 ± 35%  interrupts.CPU72.LOC:Local_timer_interrupts
    127044 ±  5%    +572.4%     854189 ± 35%  interrupts.CPU73.LOC:Local_timer_interrupts
    127124 ±  5%    +571.9%     854185 ± 35%  interrupts.CPU74.LOC:Local_timer_interrupts
    127077 ±  5%    +572.1%     854122 ± 35%  interrupts.CPU75.LOC:Local_timer_interrupts
    127146 ±  5%    +571.7%     854070 ± 35%  interrupts.CPU76.LOC:Local_timer_interrupts
    127110 ±  5%    +572.0%     854139 ± 35%  interrupts.CPU77.LOC:Local_timer_interrupts
    127083 ±  5%    +572.1%     854128 ± 35%  interrupts.CPU78.LOC:Local_timer_interrupts
    127160 ±  5%    +571.7%     854154 ± 35%  interrupts.CPU79.LOC:Local_timer_interrupts
    125986 ±  7%    +577.9%     854097 ± 35%  interrupts.CPU8.LOC:Local_timer_interrupts
    127148 ±  5%    +571.7%     854087 ± 35%  interrupts.CPU80.LOC:Local_timer_interrupts
    127217 ±  5%    +571.5%     854274 ± 35%  interrupts.CPU81.LOC:Local_timer_interrupts
    127216 ±  5%    +571.6%     854404 ± 35%  interrupts.CPU82.LOC:Local_timer_interrupts
    127167 ±  5%    +571.6%     854104 ± 35%  interrupts.CPU83.LOC:Local_timer_interrupts
    127120 ±  5%    +571.9%     854118 ± 35%  interrupts.CPU84.LOC:Local_timer_interrupts
    127230 ±  5%    +571.3%     854106 ± 35%  interrupts.CPU85.LOC:Local_timer_interrupts
    127365 ±  5%    +570.6%     854061 ± 35%  interrupts.CPU86.LOC:Local_timer_interrupts
    127209 ±  5%    +571.5%     854226 ± 35%  interrupts.CPU87.LOC:Local_timer_interrupts
    127209 ±  5%    +571.8%     854573 ± 35%  interrupts.CPU88.LOC:Local_timer_interrupts
    127360 ±  5%    +570.6%     854090 ± 35%  interrupts.CPU89.LOC:Local_timer_interrupts
    126109 ±  7%    +577.3%     854152 ± 35%  interrupts.CPU9.LOC:Local_timer_interrupts
    249.50 ± 37%     -44.0%     139.80 ± 39%  interrupts.CPU9.NMI:Non-maskable_interrupts
    249.50 ± 37%     -44.0%     139.80 ± 39%  interrupts.CPU9.PMI:Performance_monitoring_interrupts
    127390 ±  5%    +570.5%     854097 ± 35%  interrupts.CPU90.LOC:Local_timer_interrupts
    127226 ±  5%    +571.5%     854322 ± 35%  interrupts.CPU91.LOC:Local_timer_interrupts
    127152 ±  5%    +571.7%     854076 ± 35%  interrupts.CPU92.LOC:Local_timer_interrupts
    714.17 ± 17%     -17.2%     591.40 ±  2%  interrupts.CPU93.CAL:Function_call_interrupts
    127081 ±  5%    +572.1%     854072 ± 35%  interrupts.CPU93.LOC:Local_timer_interrupts
    127151 ±  5%    +571.8%     854146 ± 35%  interrupts.CPU94.LOC:Local_timer_interrupts
    229.50 ± 22%     -39.9%     138.00 ± 40%  interrupts.CPU94.NMI:Non-maskable_interrupts
    229.50 ± 22%     -39.9%     138.00 ± 40%  interrupts.CPU94.PMI:Performance_monitoring_interrupts
    127115 ±  5%    +572.0%     854207 ± 35%  interrupts.CPU95.LOC:Local_timer_interrupts
    216.67 ± 25%     -36.2%     138.20 ± 39%  interrupts.CPU95.NMI:Non-maskable_interrupts
    216.67 ± 25%     -36.2%     138.20 ± 39%  interrupts.CPU95.PMI:Performance_monitoring_interrupts
  12153208 ±  6%    +574.7%   81999844 ± 35%  interrupts.LOC:Local_timer_interrupts
      0.00       +1.2e+104%     115.20 ± 33%  interrupts.MCP:Machine_check_polls
     19276 ±  4%     -28.1%      13864 ± 29%  interrupts.NMI:Non-maskable_interrupts
     19276 ±  4%     -28.1%      13864 ± 29%  interrupts.PMI:Performance_monitoring_interrupts
     11990 ± 20%    +106.2%      24718 ± 24%  softirqs.CPU0.RCU
     11127 ±  8%    +368.5%      52128 ± 35%  softirqs.CPU0.SCHED
      9048 ±  8%    +164.2%      23907 ± 27%  softirqs.CPU1.RCU
      9713 ± 14%    +452.0%      53621 ± 30%  softirqs.CPU1.SCHED
      8948 ±  9%    +168.0%      23987 ± 29%  softirqs.CPU10.RCU
      9207 ± 10%    +475.0%      52939 ± 31%  softirqs.CPU10.SCHED
      8895 ±  9%    +168.0%      23841 ± 28%  softirqs.CPU11.RCU
      8900 ±  6%    +492.7%      52750 ± 31%  softirqs.CPU11.SCHED
      8276 ±  9%    +537.7%      52781 ± 31%  softirqs.CPU12.SCHED
      8685 ±  7%    +174.1%      23808 ± 27%  softirqs.CPU13.RCU
      9251 ±  4%    +469.9%      52726 ± 31%  softirqs.CPU13.SCHED
      9028 ± 10%    +165.2%      23940 ± 29%  softirqs.CPU14.RCU
      8380 ±  6%    +527.1%      52555 ± 31%  softirqs.CPU14.SCHED
      8989 ±  9%    +158.7%      23256 ± 29%  softirqs.CPU15.RCU
      8629 ±  6%    +468.9%      49097 ± 41%  softirqs.CPU15.SCHED
      8274 ±  6%    +172.6%      22552 ± 30%  softirqs.CPU16.RCU
      8886 ±  7%    +499.9%      53308 ± 31%  softirqs.CPU16.SCHED
      8146 ±  4%    +181.3%      22920 ± 28%  softirqs.CPU17.RCU
      9146 ±  4%    +478.6%      52923 ± 31%  softirqs.CPU17.SCHED
      7715 ±  3%    +150.1%      19297 ± 41%  softirqs.CPU18.RCU
      6730 ± 20%    +618.0%      48325 ± 40%  softirqs.CPU18.SCHED
      8009 ±  3%    +160.7%      20879 ± 35%  softirqs.CPU19.RCU
      8869 ± 10%    +495.7%      52840 ± 31%  softirqs.CPU19.SCHED
      8886 ± 18%    +499.3%      53256 ± 31%  softirqs.CPU2.SCHED
      8229 ±  5%    +174.1%      22554 ± 29%  softirqs.CPU20.RCU
      8380 ±  7%    +532.4%      52998 ± 31%  softirqs.CPU20.SCHED
      8158 ±  5%    +180.1%      22854 ± 31%  softirqs.CPU21.RCU
      8929 ±  7%    +410.7%      45606 ± 48%  softirqs.CPU21.SCHED
      9184 ± 14%    +149.4%      22905 ± 30%  softirqs.CPU22.RCU
      8740 ±  9%    +506.0%      52965 ± 31%  softirqs.CPU22.SCHED
      8322 ±  3%    +172.8%      22706 ± 32%  softirqs.CPU23.RCU
      7366 ± 13%    +620.3%      53058 ± 31%  softirqs.CPU23.SCHED
      8376 ±  5%    +156.5%      21482 ± 34%  softirqs.CPU24.RCU
      8021 ± 12%    +564.1%      53264 ± 31%  softirqs.CPU24.SCHED
      7646 ±  4%    +167.4%      20446 ± 36%  softirqs.CPU25.RCU
      8693 ±  8%    +506.9%      52759 ± 31%  softirqs.CPU25.SCHED
      7470 ±  6%    +172.6%      20364 ± 37%  softirqs.CPU26.RCU
      8881 ±  8%    +440.4%      47995 ± 40%  softirqs.CPU26.SCHED
      7941 ±  7%    +167.0%      21201 ± 31%  softirqs.CPU27.RCU
      8574 ± 10%    +517.0%      52902 ± 30%  softirqs.CPU27.SCHED
      8560 ±  5%    +151.1%      21493 ± 32%  softirqs.CPU28.RCU
      8786 ±  6%    +503.9%      53066 ± 31%  softirqs.CPU28.SCHED
      8010 ±  8%    +167.9%      21463 ± 33%  softirqs.CPU29.RCU
      7578 ± 20%    +543.7%      48781 ± 41%  softirqs.CPU29.SCHED
      9251 ±  8%    +163.6%      24386 ± 26%  softirqs.CPU3.RCU
      8600 ± 10%    +513.3%      52741 ± 31%  softirqs.CPU3.SCHED
      8057 ±  5%    +165.9%      21421 ± 34%  softirqs.CPU30.RCU
      6740 ± 34%    +690.7%      53294 ± 30%  softirqs.CPU30.SCHED
      7713 ±  8%    +173.1%      21066 ± 34%  softirqs.CPU31.RCU
      8424 ± 12%    +527.3%      52847 ± 31%  softirqs.CPU31.SCHED
      8584 ±  8%    +157.3%      22089 ± 30%  softirqs.CPU32.RCU
      8525 ± 11%    +529.0%      53628 ± 29%  softirqs.CPU32.SCHED
      8509 ±  6%    +157.4%      21901 ± 30%  softirqs.CPU33.RCU
      8787 ±  9%    +503.4%      53023 ± 30%  softirqs.CPU33.SCHED
      9203 ± 12%    +145.0%      22551 ± 28%  softirqs.CPU34.RCU
      7167 ± 28%    +638.7%      52945 ± 31%  softirqs.CPU34.SCHED
      8799 ±  7%    +156.0%      22524 ± 29%  softirqs.CPU35.RCU
      8926 ±  5%    +492.1%      52851 ± 30%  softirqs.CPU35.SCHED
      8629 ±  7%    +152.4%      21783 ± 31%  softirqs.CPU36.RCU
      9118 ±  6%    +480.3%      52908 ± 31%  softirqs.CPU36.SCHED
      8645 ±  8%    +152.9%      21865 ± 29%  softirqs.CPU37.RCU
      9224 ±  4%    +471.9%      52750 ± 31%  softirqs.CPU37.SCHED
      8586 ±  7%    +160.0%      22321 ± 29%  softirqs.CPU38.RCU
      8343 ± 17%    +490.1%      49238 ± 18%  softirqs.CPU38.SCHED
      8469 ±  5%    +142.9%      20576 ± 24%  softirqs.CPU39.RCU
      8647 ±  6%    +512.2%      52937 ± 31%  softirqs.CPU39.SCHED
      9134 ±  6%    +153.4%      23148 ± 23%  softirqs.CPU4.RCU
      8433 ± 17%    +527.5%      52919 ± 31%  softirqs.CPU4.SCHED
      8718 ±  8%    +158.8%      22559 ± 28%  softirqs.CPU40.RCU
      8368 ± 11%    +489.4%      49327 ± 40%  softirqs.CPU40.SCHED
      8960 ±  9%    +154.7%      22818 ± 29%  softirqs.CPU41.RCU
      8311 ± 12%    +536.4%      52892 ± 30%  softirqs.CPU41.SCHED
      8472 ±  8%    +143.7%      20649 ± 36%  softirqs.CPU42.RCU
      8410 ± 12%    +471.6%      48068 ± 41%  softirqs.CPU42.SCHED
      8826 ±  9%    +153.6%      22381 ± 29%  softirqs.CPU43.RCU
      8618 ± 14%    +513.0%      52831 ± 31%  softirqs.CPU43.SCHED
      8622 ± 11%    +159.1%      22338 ± 29%  softirqs.CPU44.RCU
      8792 ±  7%    +501.2%      52857 ± 31%  softirqs.CPU44.SCHED
      8618 ±  8%    +156.5%      22104 ± 29%  softirqs.CPU45.RCU
      8866 ±  8%    +495.6%      52802 ± 31%  softirqs.CPU45.SCHED
      8801 ± 10%    +158.8%      22776 ± 30%  softirqs.CPU46.RCU
      8485 ± 10%    +439.2%      45754 ±  5%  softirqs.CPU46.SCHED
      8926 ±  9%    +153.3%      22607 ± 30%  softirqs.CPU47.RCU
      8670 ± 16%    +431.0%      46039 ± 45%  softirqs.CPU47.SCHED
      8019 ±  4%    +159.2%      20788 ± 34%  softirqs.CPU48.RCU
      8097 ±  9%    +311.2%      33294 ± 32%  softirqs.CPU48.SCHED
      8364 ± 11%    +170.3%      22612 ± 26%  softirqs.CPU49.RCU
      8489 ±  7%    +527.0%      53230 ± 32%  softirqs.CPU49.SCHED
      9054 ± 10%    +166.5%      24130 ± 27%  softirqs.CPU5.RCU
      8790 ±  7%    +501.6%      52878 ± 31%  softirqs.CPU5.SCHED
      8376 ±  4%    +157.2%      21540 ± 27%  softirqs.CPU50.RCU
      8395 ± 19%    +489.1%      49462 ± 40%  softirqs.CPU50.SCHED
      8730 ±  6%    +174.7%      23986 ± 26%  softirqs.CPU51.RCU
      8108 ± 10%    +564.6%      53889 ± 33%  softirqs.CPU51.SCHED
      9222 ±  9%    +149.2%      22985 ± 24%  softirqs.CPU52.RCU
      7994 ± 10%    +595.0%      55563 ± 37%  softirqs.CPU52.SCHED
      9393 ±  8%    +150.7%      23543 ± 27%  softirqs.CPU53.RCU
      8616 ± 14%    +525.4%      53882 ± 32%  softirqs.CPU53.SCHED
      9025 ±  9%    +162.0%      23648 ± 28%  softirqs.CPU54.RCU
      8074 ± 14%    +566.0%      53775 ± 32%  softirqs.CPU54.SCHED
      8500 ±  8%    +170.4%      22980 ± 27%  softirqs.CPU55.RCU
      8812 ± 15%    +504.5%      53266 ± 32%  softirqs.CPU55.SCHED
      8646 ±  8%    +169.5%      23298 ± 27%  softirqs.CPU56.RCU
      8666 ± 11%    +521.5%      53863 ± 32%  softirqs.CPU56.SCHED
      8567 ±  7%    +167.1%      22885 ± 28%  softirqs.CPU57.RCU
      8606 ± 10%    +476.8%      49641 ± 42%  softirqs.CPU57.SCHED
      8937 ±  9%    +167.1%      23869 ± 27%  softirqs.CPU58.RCU
      7639 ± 15%    +612.4%      54421 ± 32%  softirqs.CPU58.SCHED
      8777 ±  9%    +169.7%      23673 ± 27%  softirqs.CPU59.RCU
      8513 ± 19%    +532.4%      53839 ± 32%  softirqs.CPU59.SCHED
      9439 ±  9%    +158.8%      24425 ± 26%  softirqs.CPU6.RCU
      7630 ±  9%    +548.1%      49453 ± 40%  softirqs.CPU6.SCHED
      8424 ±  7%    +166.7%      22464 ± 28%  softirqs.CPU60.RCU
      8370 ±  8%    +537.9%      53392 ± 32%  softirqs.CPU60.SCHED
      8895 ±  9%    +161.9%      23300 ± 27%  softirqs.CPU61.RCU
      8149 ± 11%    +527.3%      51121 ± 38%  softirqs.CPU61.SCHED
      8864 ± 12%    +163.0%      23312 ± 27%  softirqs.CPU62.RCU
      8360 ±  8%    +545.1%      53930 ± 32%  softirqs.CPU62.SCHED
      8590 ± 10%    +161.8%      22493 ± 30%  softirqs.CPU63.RCU
      8480 ±  7%    +484.2%      49544 ± 42%  softirqs.CPU63.SCHED
      7934 ±  6%    +183.8%      22517 ± 31%  softirqs.CPU64.RCU
      8260 ±  7%    +551.3%      53800 ± 32%  softirqs.CPU64.SCHED
      8135 ±  5%    +176.5%      22493 ± 29%  softirqs.CPU65.RCU
      8338 ±  8%    +500.2%      50044 ± 42%  softirqs.CPU65.SCHED
      7668 ±  5%    +162.3%      20111 ± 34%  softirqs.CPU66.RCU
      7824 ± 14%    +582.1%      53370 ± 31%  softirqs.CPU66.SCHED
      8484 ± 20%    +149.5%      21171 ± 31%  softirqs.CPU67.RCU
      8050 ± 13%    +562.9%      53367 ± 32%  softirqs.CPU67.SCHED
      7934 ±  4%    +184.2%      22546 ± 29%  softirqs.CPU68.RCU
      9050 ±  3%    +496.6%      53993 ± 32%  softirqs.CPU68.SCHED
      8041 ±  5%    +178.5%      22394 ± 29%  softirqs.CPU69.RCU
      8480 ± 11%    +468.6%      48222 ± 48%  softirqs.CPU69.SCHED
      8986 ±  8%    +163.9%      23719 ± 28%  softirqs.CPU7.RCU
      7811 ± 12%    +572.5%      52534 ± 31%  softirqs.CPU7.SCHED
      8318 ±  7%    +170.5%      22502 ± 28%  softirqs.CPU70.RCU
      8112 ± 10%    +565.0%      53944 ± 32%  softirqs.CPU70.SCHED
      7790 ±  7%    +182.2%      21983 ± 30%  softirqs.CPU71.RCU
      8174 ± 16%    +559.5%      53910 ± 32%  softirqs.CPU71.SCHED
      7782 ±  3%    +169.8%      20993 ± 34%  softirqs.CPU72.RCU
      8655 ±  8%    +489.4%      51020 ± 21%  softirqs.CPU72.SCHED
      7406 ±  5%    +163.5%      19515 ± 39%  softirqs.CPU73.RCU
      8499 ± 10%    +534.6%      53934 ± 31%  softirqs.CPU73.SCHED
      7316 ±  4%    +162.3%      19189 ± 31%  softirqs.CPU74.RCU
      9054 ±  4%    +493.2%      53711 ± 31%  softirqs.CPU74.SCHED
      7731 ±  6%    +151.5%      19446 ± 19%  softirqs.CPU75.RCU
      8411 ±  9%    +546.8%      54400 ± 31%  softirqs.CPU75.SCHED
      7757 ±  6%    +171.3%      21043 ± 33%  softirqs.CPU76.RCU
      8489 ±  6%    +539.1%      54258 ± 31%  softirqs.CPU76.SCHED
      8389 ± 14%    +147.7%      20783 ± 34%  softirqs.CPU77.RCU
      8741 ± 10%    +521.8%      54358 ± 31%  softirqs.CPU77.SCHED
      7971 ±  9%    +165.1%      21131 ± 31%  softirqs.CPU78.RCU
      8870 ±  8%    +511.7%      54263 ± 31%  softirqs.CPU78.SCHED
      7516 ±  7%    +176.3%      20770 ± 33%  softirqs.CPU79.RCU
      7985 ± 15%    +577.1%      54066 ± 31%  softirqs.CPU79.SCHED
      9233 ± 13%    +159.1%      23921 ± 28%  softirqs.CPU8.RCU
      8596 ± 17%    +513.7%      52756 ± 32%  softirqs.CPU8.SCHED
      8593 ±  7%    +148.1%      21319 ± 29%  softirqs.CPU80.RCU
      7578 ± 13%    +616.2%      54278 ± 31%  softirqs.CPU80.SCHED
      8351 ±  5%    +168.5%      22428 ± 26%  softirqs.CPU81.RCU
      8729 ±  7%    +520.6%      54173 ± 31%  softirqs.CPU81.SCHED
      8798 ± 10%    +151.6%      22139 ± 28%  softirqs.CPU82.RCU
      8022 ± 14%    +576.3%      54258 ± 31%  softirqs.CPU82.SCHED
      8546 ±  8%    +157.4%      22000 ± 28%  softirqs.CPU83.RCU
      8757 ±  7%    +518.3%      54146 ± 31%  softirqs.CPU83.SCHED
      8128 ± 10%    +156.9%      20879 ± 28%  softirqs.CPU84.RCU
      8284 ± 10%    +547.1%      53612 ± 31%  softirqs.CPU84.SCHED
      8323 ±  6%    +154.3%      21167 ± 27%  softirqs.CPU85.RCU
      8432 ± 17%    +538.0%      53800 ± 31%  softirqs.CPU85.SCHED
      8489 ± 10%    +156.5%      21772 ± 28%  softirqs.CPU86.RCU
      8023 ± 11%    +531.8%      50688 ± 40%  softirqs.CPU86.SCHED
      8539 ±  6%    +146.8%      21073 ± 30%  softirqs.CPU87.RCU
      8381 ± 10%    +556.2%      54993 ± 30%  softirqs.CPU87.SCHED
      8812 ± 10%    +153.0%      22295 ± 28%  softirqs.CPU88.RCU
      7901 ± 14%    +587.6%      54332 ± 31%  softirqs.CPU88.SCHED
      8992 ± 11%    +145.5%      22076 ± 25%  softirqs.CPU89.RCU
      8814 ± 10%    +518.4%      54509 ± 31%  softirqs.CPU89.SCHED
      8776 ±  7%    +168.3%      23549 ± 28%  softirqs.CPU9.RCU
      9385 ±  5%    +461.5%      52701 ± 31%  softirqs.CPU9.SCHED
      8012 ±  3%    +148.5%      19908 ± 31%  softirqs.CPU90.RCU
      8772 ±  8%    +472.6%      50233 ± 40%  softirqs.CPU90.SCHED
      8607 ±  9%    +153.9%      21857 ± 28%  softirqs.CPU91.RCU
      7992 ±  7%    +577.9%      54180 ± 31%  softirqs.CPU91.SCHED
      8214 ± 12%    +164.2%      21704 ± 28%  softirqs.CPU92.RCU
      7560 ± 18%    +618.0%      54281 ± 31%  softirqs.CPU92.SCHED
      8542 ± 10%    +152.6%      21577 ± 29%  softirqs.CPU93.RCU
      8209 ±  8%    +556.7%      53909 ± 31%  softirqs.CPU93.SCHED
      8954 ± 11%    +151.7%      22540 ± 34%  softirqs.CPU94.RCU
      8108 ± 19%    +568.6%      54216 ± 31%  softirqs.CPU94.SCHED
      8919 ±  7%    +158.4%      23047 ± 27%  softirqs.CPU95.RCU
      8338 ±  8%    +358.3%      38211 ± 65%  softirqs.CPU95.SCHED
    818847 ±  5%    +160.1%    2130203 ± 29%  softirqs.RCU
    811938 ±  2%    +516.6%    5006268 ± 32%  softirqs.SCHED
     20491           +30.3%      26707 ± 26%  softirqs.TIMER
     18.87 ± 10%     -18.9        0.00        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
     18.28 ± 10%     -18.3        0.00        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
     16.15 ±  7%     -16.2        0.00        perf-profile.calltrace.cycles-pp.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
     15.59 ±  4%     -15.6        0.00        perf-profile.calltrace.cycles-pp.ret_from_fork
     15.58 ±  4%     -15.6        0.00        perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
     15.29 ±  5%     -15.3        0.00        perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
     15.25 ±  5%     -15.2        0.00        perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
     14.39 ±  5%     -14.4        0.00        perf-profile.calltrace.cycles-pp.drm_fb_helper_damage_work.process_one_work.worker_thread.kthread.ret_from_fork
     13.94 ±  5%     -13.9        0.00        perf-profile.calltrace.cycles-pp.memcpy_toio.drm_fb_helper_damage_work.process_one_work.worker_thread.kthread
     12.85 ±  6%     -12.8        0.00        perf-profile.calltrace.cycles-pp.block_ioctl.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
     12.85 ±  6%     -12.8        0.00        perf-profile.calltrace.cycles-pp.blkdev_ioctl.block_ioctl.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
     12.77 ±  6%     -12.8        0.00        perf-profile.calltrace.cycles-pp.lo_ioctl.blkdev_ioctl.block_ioctl.__x64_sys_ioctl.do_syscall_64
     10.30 ±  6%     -10.3        0.00        perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.printk.set_capacity_and_notify.cold.loop_set_size
      9.61 ±  6%      -9.6        0.00        perf-profile.calltrace.cycles-pp.serial8250_console_write.console_unlock.vprintk_emit.printk.set_capacity_and_notify.cold
      9.16 ±  6%      -8.8        0.34 ± 81%  perf-profile.calltrace.cycles-pp.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit.printk
      8.47 ±  6%      -8.1        0.33 ± 81%  perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit
      8.47 ±  6%      -8.1        0.33 ± 81%  perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock
      6.83 ±  7%      -6.8        0.00        perf-profile.calltrace.cycles-pp.io_serial_in.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write
      6.41 ± 11%      -6.4        0.00        perf-profile.calltrace.cycles-pp.loop_set_size.lo_ioctl.blkdev_ioctl.block_ioctl.__x64_sys_ioctl
      6.41 ± 11%      -6.4        0.00        perf-profile.calltrace.cycles-pp.set_capacity_and_notify.cold.loop_set_size.lo_ioctl.blkdev_ioctl.block_ioctl
      6.41 ± 11%      -6.4        0.00        perf-profile.calltrace.cycles-pp.printk.set_capacity_and_notify.cold.loop_set_size.lo_ioctl.blkdev_ioctl
      6.41 ± 11%      -6.4        0.00        perf-profile.calltrace.cycles-pp.vprintk_emit.printk.set_capacity_and_notify.cold.loop_set_size.lo_ioctl
      6.30 ± 13%      -6.3        0.00        perf-profile.calltrace.cycles-pp.loop_configure.lo_ioctl.blkdev_ioctl.block_ioctl.__x64_sys_ioctl
      6.27 ± 13%      -6.3        0.00        perf-profile.calltrace.cycles-pp.printk.set_capacity_and_notify.cold.loop_set_size.loop_configure.lo_ioctl
      6.27 ± 13%      -6.3        0.00        perf-profile.calltrace.cycles-pp.vprintk_emit.printk.set_capacity_and_notify.cold.loop_set_size.loop_configure
      6.27 ± 13%      -6.3        0.00        perf-profile.calltrace.cycles-pp.loop_set_size.loop_configure.lo_ioctl.blkdev_ioctl.block_ioctl
      6.27 ± 13%      -6.3        0.00        perf-profile.calltrace.cycles-pp.set_capacity_and_notify.cold.loop_set_size.loop_configure.lo_ioctl.blkdev_ioctl
      0.61 ±  8%      +0.4        1.00 ± 15%  perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.77 ± 10%      +0.5        1.29 ± 14%  perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      0.71 ±  7%      +0.6        1.26 ± 28%  perf-profile.calltrace.cycles-pp.lapic_next_deadline.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      0.68 ±  8%      +0.6        1.27 ±  9%  perf-profile.calltrace.cycles-pp.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      1.82 ±  6%      +0.6        2.43 ± 12%  perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      0.69 ±  7%      +0.6        1.33 ± 12%  perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +0.7        0.69 ± 14%  perf-profile.calltrace.cycles-pp.load_balance.rebalance_domains.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt
      0.84 ± 26%      +0.7        1.53 ± 20%  perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel
      0.85 ± 26%      +0.7        1.58 ± 18%  perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64_no_verify
      0.92 ± 24%      +0.8        1.77 ± 13%  perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_kernel.secondary_startup_64_no_verify
      0.92 ± 24%      +0.8        1.77 ± 13%  perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64_no_verify
      0.92 ± 24%      +0.8        1.77 ± 13%  perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64_no_verify
      1.16 ±  5%      +0.9        2.06 ± 12%  perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      1.02 ± 10%      +0.9        1.92 ± 23%  perf-profile.calltrace.cycles-pp.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      1.05 ±  9%      +0.9        1.99 ± 21%  perf-profile.calltrace.cycles-pp.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      2.24 ±  5%      +0.9        3.19 ± 12%  perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      2.24 ±  5%      +1.7        3.91 ± 12%  perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
      1.37 ± 21%      +1.9        3.24 ± 51%  perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry
      2.36 ±  8%      +1.9        4.25 ± 15%  perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      2.67 ±  7%      +2.2        4.85 ± 14%  perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      1.75 ± 16%      +2.2        3.95 ± 41%  perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary
      4.26 ±  9%      +3.3        7.57 ± 18%  perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      7.68 ±  3%      +5.6       13.32 ± 11%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      7.86 ±  3%      +5.8       13.66 ± 10%  perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
     12.33 ±  3%      +8.7       21.03 ± 10%  perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
      4.23 ± 10%      +9.1       13.37 ± 42%  perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     16.39 ±  8%     +16.1       32.45 ± 21%  perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
     56.29 ±  2%     +20.4       76.71 ± 10%  perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
     57.55 ±  3%     +23.6       81.14 ±  6%  perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     62.83 ±  3%     +33.6       96.47        perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     62.88 ±  3%     +33.7       96.53        perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
     62.87 ±  3%     +33.7       96.53        perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     63.79 ±  3%     +34.5       98.30        perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
     19.91 ±  9%     -19.0        0.89 ± 13%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
     19.24 ±  9%     -18.4        0.84 ± 16%  perf-profile.children.cycles-pp.do_syscall_64
     16.15 ±  7%     -16.2        0.00        perf-profile.children.cycles-pp.__x64_sys_ioctl
     15.61 ±  4%     -15.2        0.38 ± 15%  perf-profile.children.cycles-pp.ret_from_fork
     15.58 ±  4%     -15.2        0.38 ± 15%  perf-profile.children.cycles-pp.kthread
     15.29 ±  5%     -15.1        0.22 ± 16%  perf-profile.children.cycles-pp.worker_thread
     15.25 ±  5%     -15.0        0.22 ± 17%  perf-profile.children.cycles-pp.process_one_work
     14.39 ±  5%     -14.2        0.20 ± 20%  perf-profile.children.cycles-pp.drm_fb_helper_damage_work
     14.37 ±  5%     -14.2        0.20 ± 20%  perf-profile.children.cycles-pp.memcpy_toio
     12.85 ±  6%     -12.8        0.00        perf-profile.children.cycles-pp.block_ioctl
     12.85 ±  6%     -12.8        0.00        perf-profile.children.cycles-pp.blkdev_ioctl
     12.77 ±  6%     -12.8        0.00        perf-profile.children.cycles-pp.lo_ioctl
     12.68 ±  6%     -12.7        0.00        perf-profile.children.cycles-pp.loop_set_size
     12.68 ±  6%     -12.7        0.00        perf-profile.children.cycles-pp.set_capacity_and_notify.cold
     13.18 ±  6%     -12.4        0.79 ± 22%  perf-profile.children.cycles-pp.printk
     13.18 ±  6%     -12.4        0.79 ± 22%  perf-profile.children.cycles-pp.vprintk_emit
     10.76 ±  6%     -10.0        0.79 ± 22%  perf-profile.children.cycles-pp.console_unlock
     10.04 ±  6%      -9.3        0.74 ± 23%  perf-profile.children.cycles-pp.serial8250_console_write
      9.58 ±  6%      -8.9        0.73 ± 23%  perf-profile.children.cycles-pp.uart_console_write
      9.29 ±  6%      -8.6        0.72 ± 23%  perf-profile.children.cycles-pp.wait_for_xmitr
      8.85 ±  6%      -8.2        0.70 ± 23%  perf-profile.children.cycles-pp.serial8250_console_putchar
      7.48 ±  7%      -6.9        0.55 ± 28%  perf-profile.children.cycles-pp.io_serial_in
      6.30 ± 13%      -6.3        0.00        perf-profile.children.cycles-pp.loop_configure
      1.80 ± 14%      -1.6        0.17 ± 36%  perf-profile.children.cycles-pp.delay_tsc
      1.72 ± 46%      -1.6        0.12 ± 28%  perf-profile.children.cycles-pp.do_sys_open
      1.71 ± 46%      -1.6        0.12 ± 28%  perf-profile.children.cycles-pp.do_sys_openat2
      1.60 ± 49%      -1.5        0.10 ± 27%  perf-profile.children.cycles-pp.do_filp_open
      1.59 ± 49%      -1.5        0.10 ± 27%  perf-profile.children.cycles-pp.path_openat
      0.70 ±  8%      -0.7        0.03 ± 82%  perf-profile.children.cycles-pp.vt_console_print
      0.67 ±  8%      -0.6        0.03 ± 82%  perf-profile.children.cycles-pp.lf
      0.67 ±  8%      -0.6        0.03 ± 82%  perf-profile.children.cycles-pp.con_scroll
      0.67 ±  8%      -0.6        0.03 ± 82%  perf-profile.children.cycles-pp.fbcon_scroll
      0.66 ±  9%      -0.6        0.03 ± 82%  perf-profile.children.cycles-pp.fbcon_redraw
      0.65 ± 10%      -0.6        0.03 ± 82%  perf-profile.children.cycles-pp.fbcon_putcs
      0.64 ± 11%      -0.6        0.03 ± 82%  perf-profile.children.cycles-pp.bit_putcs
      0.46 ± 11%      -0.3        0.16 ± 26%  perf-profile.children.cycles-pp.rcu_core
      0.44 ±  4%      -0.3        0.14 ± 32%  perf-profile.children.cycles-pp.__schedule
      0.37 ± 18%      -0.3        0.08 ± 22%  perf-profile.children.cycles-pp.link_path_walk
      0.33 ±  5%      -0.3        0.07 ± 45%  perf-profile.children.cycles-pp.schedule
      0.31 ±  9%      -0.2        0.07 ± 57%  perf-profile.children.cycles-pp.walk_component
      0.20 ± 12%      -0.1        0.10 ± 54%  perf-profile.children.cycles-pp._find_next_bit
      0.23 ± 14%      -0.1        0.16 ± 17%  perf-profile.children.cycles-pp.ksys_read
      0.23 ± 14%      -0.1        0.15 ± 16%  perf-profile.children.cycles-pp.vfs_read
      0.16 ± 18%      -0.0        0.11 ± 15%  perf-profile.children.cycles-pp.new_sync_read
      0.05 ± 45%      +0.0        0.09 ± 31%  perf-profile.children.cycles-pp.rcu_needs_cpu
      0.12 ± 20%      +0.1        0.17 ± 21%  perf-profile.children.cycles-pp._raw_spin_lock_irq
      0.01 ±223%      +0.1        0.06 ± 23%  perf-profile.children.cycles-pp.unmap_page_range
      0.01 ±223%      +0.1        0.06 ± 23%  perf-profile.children.cycles-pp.unmap_vmas
      0.09 ± 14%      +0.1        0.14 ± 19%  perf-profile.children.cycles-pp.bprm_execve
      0.10 ± 18%      +0.1        0.18 ± 24%  perf-profile.children.cycles-pp.do_execveat_common
      0.10 ± 18%      +0.1        0.19 ± 24%  perf-profile.children.cycles-pp.execve
      0.10 ± 18%      +0.1        0.19 ± 24%  perf-profile.children.cycles-pp.__x64_sys_execve
      0.10 ± 16%      +0.1        0.19 ± 28%  perf-profile.children.cycles-pp.rcu_dynticks_eqs_exit
      0.01 ±223%      +0.1        0.12 ± 38%  perf-profile.children.cycles-pp.tick_check_broadcast_expired
      0.16 ± 10%      +0.1        0.27 ± 21%  perf-profile.children.cycles-pp.rcu_eqs_enter
      0.15 ± 48%      +0.1        0.27 ± 18%  perf-profile.children.cycles-pp.call_cpuidle
      0.14 ±  9%      +0.1        0.27 ± 24%  perf-profile.children.cycles-pp.hrtimer_get_next_event
      0.20 ± 22%      +0.1        0.34 ± 26%  perf-profile.children.cycles-pp.update_irq_load_avg
      0.14 ± 18%      +0.1        0.29 ± 38%  perf-profile.children.cycles-pp.__hrtimer_get_next_event
      0.29 ±  7%      +0.2        0.46 ± 28%  perf-profile.children.cycles-pp.__intel_pmu_enable_all
      0.27 ±  7%      +0.2        0.45 ± 12%  perf-profile.children.cycles-pp.arch_scale_freq_tick
      0.35 ± 13%      +0.2        0.56 ± 25%  perf-profile.children.cycles-pp.update_rq_clock
      0.23 ± 17%      +0.2        0.46 ± 35%  perf-profile.children.cycles-pp.rcu_eqs_exit
      0.19 ± 17%      +0.2        0.43 ± 23%  perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
      0.20 ± 17%      +0.3        0.45 ± 22%  perf-profile.children.cycles-pp.arch_cpu_idle_enter
      0.29 ± 12%      +0.3        0.56 ±  9%  perf-profile.children.cycles-pp.hrtimer_next_event_without
      0.30 ± 25%      +0.3        0.61 ± 39%  perf-profile.children.cycles-pp.hrtimer_update_next_event
      0.30 ± 10%      +0.3        0.62 ± 13%  perf-profile.children.cycles-pp.get_next_timer_interrupt
      0.55 ±  5%      +0.3        0.87 ± 14%  perf-profile.children.cycles-pp.native_sched_clock
      0.58 ±  5%      +0.3        0.91 ± 13%  perf-profile.children.cycles-pp.sched_clock
      0.28 ± 18%      +0.3        0.62 ± 29%  perf-profile.children.cycles-pp.__hrtimer_next_event_base
      0.43 ± 49%      +0.4        0.79 ± 22%  perf-profile.children.cycles-pp.irq_work_single
      0.43 ± 49%      +0.4        0.79 ± 22%  perf-profile.children.cycles-pp.irq_work_run
      0.43 ± 49%      +0.4        0.79 ± 22%  perf-profile.children.cycles-pp.asm_sysvec_irq_work
      0.43 ± 49%      +0.4        0.79 ± 22%  perf-profile.children.cycles-pp.sysvec_irq_work
      0.43 ± 49%      +0.4        0.79 ± 22%  perf-profile.children.cycles-pp.__sysvec_irq_work
      0.46 ± 45%      +0.4        0.83 ± 23%  perf-profile.children.cycles-pp.irq_work_run_list
      0.58 ±  7%      +0.4        0.97 ± 24%  perf-profile.children.cycles-pp.read_tsc
      0.63 ±  9%      +0.4        1.02 ± 15%  perf-profile.children.cycles-pp.ktime_get_update_offsets_now
      0.67 ±  5%      +0.4        1.08 ± 12%  perf-profile.children.cycles-pp.sched_clock_cpu
      0.75 ± 13%      +0.5        1.23 ± 25%  perf-profile.children.cycles-pp.native_irq_return_iret
      0.80 ±  8%      +0.5        1.33 ± 14%  perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
      0.74 ±  5%      +0.6        1.30 ± 28%  perf-profile.children.cycles-pp.lapic_next_deadline
      0.70 ±  7%      +0.7        1.40 ± 11%  perf-profile.children.cycles-pp.rebalance_domains
      1.88 ±  5%      +0.7        2.62 ± 13%  perf-profile.children.cycles-pp.__softirqentry_text_start
      0.92 ± 24%      +0.8        1.77 ± 13%  perf-profile.children.cycles-pp.start_kernel
      1.22 ±  4%      +0.9        2.11 ± 12%  perf-profile.children.cycles-pp.scheduler_tick
      1.04 ±  9%      +0.9        1.96 ± 22%  perf-profile.children.cycles-pp.tick_irq_enter
      1.07 ±  8%      +0.9        2.01 ± 21%  perf-profile.children.cycles-pp.irq_enter_rcu
      2.31 ±  5%      +1.1        3.40 ± 13%  perf-profile.children.cycles-pp.irq_exit_rcu
      2.34 ±  5%      +1.7        4.01 ± 12%  perf-profile.children.cycles-pp.update_process_times
      2.44 ±  7%      +1.9        4.32 ± 15%  perf-profile.children.cycles-pp.tick_sched_handle
      1.41 ± 20%      +1.9        3.32 ± 50%  perf-profile.children.cycles-pp.tick_nohz_next_event
      2.78 ±  6%      +2.2        4.95 ± 14%  perf-profile.children.cycles-pp.tick_sched_timer
      1.78 ± 16%      +2.3        4.04 ± 40%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      4.41 ±  9%      +3.3        7.72 ± 18%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      7.92 ±  3%      +5.6       13.51 ± 11%  perf-profile.children.cycles-pp.hrtimer_interrupt
      8.08 ±  3%      +5.8       13.84 ± 11%  perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
     12.66 ±  3%      +8.6       21.21 ± 10%  perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
      4.30 ± 10%      +9.3       13.56 ± 42%  perf-profile.children.cycles-pp.menu_select
     15.27 ±  4%     +11.9       27.18 ± 15%  perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
     58.38 ±  3%     +24.3       82.69 ±  6%  perf-profile.children.cycles-pp.cpuidle_enter_state
     58.40 ±  3%     +24.3       82.73 ±  6%  perf-profile.children.cycles-pp.cpuidle_enter
     62.88 ±  3%     +33.7       96.53        perf-profile.children.cycles-pp.start_secondary
     63.79 ±  3%     +34.5       98.30        perf-profile.children.cycles-pp.secondary_startup_64_no_verify
     63.79 ±  3%     +34.5       98.30        perf-profile.children.cycles-pp.cpu_startup_entry
     63.79 ±  3%     +34.5       98.30        perf-profile.children.cycles-pp.do_idle
     14.29 ±  5%     -14.1        0.20 ± 20%  perf-profile.self.cycles-pp.memcpy_toio
      7.48 ±  7%      -6.9        0.55 ± 28%  perf-profile.self.cycles-pp.io_serial_in
      1.80 ± 14%      -1.6        0.17 ± 36%  perf-profile.self.cycles-pp.delay_tsc
      0.20 ± 13%      -0.1        0.10 ± 53%  perf-profile.self.cycles-pp._find_next_bit
      0.12 ± 17%      +0.1        0.17 ± 20%  perf-profile.self.cycles-pp._raw_spin_lock_irq
      0.04 ± 71%      +0.1        0.09 ± 31%  perf-profile.self.cycles-pp.rcu_needs_cpu
      0.05 ± 45%      +0.1        0.11 ± 35%  perf-profile.self.cycles-pp.tick_sched_timer
      0.10 ± 11%      +0.1        0.16 ± 13%  perf-profile.self.cycles-pp.sched_clock_cpu
      0.02 ±146%      +0.1        0.10 ± 31%  perf-profile.self.cycles-pp.tick_nohz_get_sleep_length
      0.08 ± 36%      +0.1        0.16 ± 42%  perf-profile.self.cycles-pp.get_next_timer_interrupt
      0.07 ± 58%      +0.1        0.16 ± 33%  perf-profile.self.cycles-pp.rcu_idle_exit
      0.14 ± 24%      +0.1        0.24 ± 23%  perf-profile.self.cycles-pp.asm_sysvec_apic_timer_interrupt
      0.01 ±223%      +0.1        0.11 ± 42%  perf-profile.self.cycles-pp.tick_check_broadcast_expired
      0.11 ± 18%      +0.1        0.21 ± 42%  perf-profile.self.cycles-pp.rcu_eqs_exit
      0.15 ± 49%      +0.1        0.27 ± 20%  perf-profile.self.cycles-pp.call_cpuidle
      0.20 ± 22%      +0.1        0.34 ± 26%  perf-profile.self.cycles-pp.update_irq_load_avg
      0.09 ± 21%      +0.2        0.24 ± 24%  perf-profile.self.cycles-pp.rebalance_domains
      0.18 ± 34%      +0.2        0.33 ± 33%  perf-profile.self.cycles-pp.__hrtimer_run_queues
      0.29 ±  7%      +0.2        0.46 ± 28%  perf-profile.self.cycles-pp.__intel_pmu_enable_all
      0.27 ±  7%      +0.2        0.45 ± 12%  perf-profile.self.cycles-pp.arch_scale_freq_tick
      0.26 ± 18%      +0.3        0.54 ± 30%  perf-profile.self.cycles-pp.__hrtimer_next_event_base
      0.29 ± 14%      +0.3        0.59 ± 17%  perf-profile.self.cycles-pp.do_idle
      0.53 ±  5%      +0.3        0.83 ± 16%  perf-profile.self.cycles-pp.native_sched_clock
      0.53 ±  8%      +0.4        0.88 ± 14%  perf-profile.self.cycles-pp.ktime_get_update_offsets_now
      0.56 ±  7%      +0.4        0.93 ± 26%  perf-profile.self.cycles-pp.read_tsc
      0.39 ± 28%      +0.4        0.79 ± 20%  perf-profile.self.cycles-pp.update_process_times
      0.75 ± 12%      +0.5        1.23 ± 25%  perf-profile.self.cycles-pp.native_irq_return_iret
      0.38 ± 23%      +0.5        0.88 ± 55%  perf-profile.self.cycles-pp.tick_nohz_next_event
      0.74 ±  5%      +0.6        1.30 ± 28%  perf-profile.self.cycles-pp.lapic_next_deadline
      2.25 ± 12%      +6.8        9.01 ± 49%  perf-profile.self.cycles-pp.menu_select



***************************************************************************************************
lkp-csl-2sp7: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory
=========================================================================================
class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime/ucode:
  device/gcc-9/performance/1HDD/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp7/stress-ng/60s/0x5003006

commit: 
  d3c4a43d92 ("block: refactor blk_drop_partitions")
  c76f48eb5c ("block: take bd_mutex around delete_partitions in del_gendisk")

d3c4a43d9291279c c76f48eb5c084b1e15c931ae8cc 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
      2529 ±  4%     -96.2%      96.00        stress-ng.loop.ops
     41.73 ±  5%     -99.5%       0.20 ± 29%  stress-ng.loop.ops_per_sec
    363.20          +140.9%     875.00 ± 22%  stress-ng.time.elapsed_time
    363.20          +140.9%     875.00 ± 22%  stress-ng.time.elapsed_time.max
      6817           -56.5%       2968 ± 21%  stress-ng.time.percent_of_cpu_this_job_got
      1.13 ±  5%      +7.6%       1.22 ±  6%  perf-sched.wait_time.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      2.26 ±  5%      +7.6%       2.44 ±  6%  perf-sched.wait_time.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
    394.07          +129.9%     905.77 ± 21%  uptime.boot
     11779          +408.6%      59917 ± 30%  uptime.idle
     27.48          +144.1%      67.10 ±  9%  iostat.cpu.idle
     70.73           -54.5%      32.18 ± 19%  iostat.cpu.system
      1.66 ±  7%     -59.9%       0.67 ± 20%  iostat.cpu.user
 5.435e+09 ± 57%    +615.3%  3.887e+10 ± 67%  cpuidle.C1E.time
  15572783 ± 25%    +512.4%   95371028 ± 46%  cpuidle.C1E.usage
   2101854 ±  3%     +21.9%    2562555 ±  5%  cpuidle.POLL.time
    150130 ±  4%    +103.6%     305659 ± 21%  cpuidle.POLL.usage
    268316 ±  2%     +26.2%     338533 ±  4%  meminfo.Active
    266431 ±  2%     +26.5%     336952 ±  4%  meminfo.Active(anon)
     47894           -14.3%      41032        meminfo.Mapped
    946.14 ± 81%     +86.1%       1760 ± 22%  meminfo.Mlocked
    286690 ±  2%     +23.2%     353170 ±  4%  meminfo.Shmem
     59208 ±  7%     +19.1%      70545 ±  7%  numa-meminfo.node0.KReclaimable
     59208 ±  7%     +19.1%      70545 ±  7%  numa-meminfo.node0.SReclaimable
    198077 ± 26%     +51.2%     299434 ± 16%  numa-meminfo.node1.Active
    197163 ± 27%     +51.5%     298631 ± 16%  numa-meminfo.node1.Active(anon)
    208761 ± 26%     +47.6%     308206 ± 16%  numa-meminfo.node1.Shmem
     27.00          +146.3%      66.50 ±  9%  vmstat.cpu.id
     70.14           -54.6%      31.83 ± 20%  vmstat.cpu.sy
      8363 ± 20%     -54.8%       3776 ± 27%  vmstat.io.bi
     75.71           -56.9%      32.67 ± 20%  vmstat.procs.r
     21166 ±  3%     -53.8%       9779 ± 19%  vmstat.system.cs
     28.87           +38.5       67.40 ±  9%  mpstat.cpu.all.idle%
      0.12 ± 11%      -0.1        0.06 ± 24%  mpstat.cpu.all.iowait%
      0.90 ±  2%      +0.6        1.49 ± 18%  mpstat.cpu.all.irq%
      0.77 ±  3%      -0.4        0.40 ± 15%  mpstat.cpu.all.soft%
     67.70           -37.7       29.99 ± 20%  mpstat.cpu.all.sys%
      1.64 ±  7%      -1.0        0.66 ± 20%  mpstat.cpu.all.usr%
     14804 ±  7%     +19.1%      17637 ±  7%  numa-vmstat.node0.nr_slab_reclaimable
  50776578 ±  2%     -23.4%   38912311 ±  4%  numa-vmstat.node0.numa_hit
  50580358 ±  2%     -23.4%   38750194 ±  4%  numa-vmstat.node0.numa_local
     49262 ± 27%     +51.5%      74630 ± 16%  numa-vmstat.node1.nr_active_anon
     52198 ± 26%     +47.6%      77069 ± 16%  numa-vmstat.node1.nr_shmem
     49262 ± 27%     +51.5%      74630 ± 16%  numa-vmstat.node1.nr_zone_active_anon
  54530015 ±  5%     -22.9%   42027883 ± 11%  numa-vmstat.node1.numa_hit
  54388335 ±  5%     -23.1%   41848232 ± 11%  numa-vmstat.node1.numa_local
     66623 ±  2%     +26.5%      84248 ±  4%  proc-vmstat.nr_active_anon
    317296            +5.2%     333673        proc-vmstat.nr_file_pages
     21141            -8.4%      19355 ±  2%  proc-vmstat.nr_kernel_stack
     12110           -14.0%      10411        proc-vmstat.nr_mapped
    235.71 ± 81%     +86.7%     440.00 ± 22%  proc-vmstat.nr_mlock
     71671 ±  2%     +23.2%      88300 ±  4%  proc-vmstat.nr_shmem
     30854 ±  2%      +9.5%      33788        proc-vmstat.nr_slab_reclaimable
     68656            -6.3%      64331        proc-vmstat.nr_slab_unreclaimable
     66623 ±  2%     +26.5%      84248 ±  4%  proc-vmstat.nr_zone_active_anon
      7780 ±  8%     -92.4%     590.50 ± 19%  proc-vmstat.unevictable_pgs_culled
      2039           -55.0%     917.67 ± 19%  turbostat.Avg_MHz
     73.33           -38.5       34.79 ± 18%  turbostat.Busy%
      2788            -5.5%       2635 ±  2%  turbostat.Bzy_MHz
      0.15 ± 14%      -0.1        0.06 ± 21%  turbostat.C1%
  15569558 ± 25%    +512.5%   95367656 ± 46%  turbostat.C1E
     15.52 ± 57%     +28.8       44.34 ± 53%  turbostat.C1E%
     25.80 ±  3%    +140.3%      62.00 ± 13%  turbostat.CPU%c1
  72387783          +136.1%  1.709e+08 ± 21%  turbostat.IRQ
    236.00           -38.5%     145.10 ±  9%  turbostat.PkgWatt
    150.82            -2.2%     147.55        turbostat.RAMWatt
      1.20 ± 35%      +1.5        2.70 ± 62%  perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.kernfs_dop_revalidate.lookup_fast.walk_component
      1.26 ± 32%      +1.5        2.78 ± 61%  perf-profile.calltrace.cycles-pp.__mutex_lock.kernfs_dop_revalidate.lookup_fast.walk_component.link_path_walk
      1.32 ± 31%      +1.5        2.86 ± 59%  perf-profile.calltrace.cycles-pp.kernfs_dop_revalidate.lookup_fast.walk_component.link_path_walk.path_openat
      1.39 ± 29%      +1.5        2.93 ± 58%  perf-profile.calltrace.cycles-pp.lookup_fast.walk_component.link_path_walk.path_openat.do_filp_open
      1.69 ± 27%      +1.8        3.44 ± 56%  perf-profile.calltrace.cycles-pp.walk_component.link_path_walk.path_openat.do_filp_open.do_sys_openat2
      1.76 ± 35%      +2.1        3.88 ± 62%  perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.kernfs_iop_permission.inode_permission.link_path_walk
      1.85 ± 33%      +2.1        3.98 ± 61%  perf-profile.calltrace.cycles-pp.__mutex_lock.kernfs_iop_permission.inode_permission.link_path_walk.path_openat
      1.94 ± 31%      +2.2        4.11 ± 60%  perf-profile.calltrace.cycles-pp.kernfs_iop_permission.inode_permission.link_path_walk.path_openat.do_filp_open
      2.01 ± 30%      +2.2        4.17 ± 59%  perf-profile.calltrace.cycles-pp.inode_permission.link_path_walk.path_openat.do_filp_open.do_sys_openat2
      3.75 ± 28%      +3.9        7.67 ± 57%  perf-profile.calltrace.cycles-pp.link_path_walk.path_openat.do_filp_open.do_sys_openat2.do_sys_open
      0.31 ± 27%      -0.1        0.19 ± 17%  perf-profile.children.cycles-pp.start_kernel
      0.08 ± 36%      +0.0        0.12 ± 26%  perf-profile.children.cycles-pp.__vunmap
      0.03 ±116%      +0.1        0.09 ± 26%  perf-profile.children.cycles-pp.remove_vm_area
      0.04 ± 89%      +0.1        0.09 ± 45%  perf-profile.children.cycles-pp.__mutex_unlock_slowpath
      0.07 ± 76%      +0.1        0.15 ± 23%  perf-profile.children.cycles-pp.new_sync_read
      0.22 ± 29%      +0.2        0.43 ± 55%  perf-profile.children.cycles-pp.kernfs_iop_get_link
      0.23 ± 28%      +0.2        0.44 ± 54%  perf-profile.children.cycles-pp.pick_link
      0.48 ± 23%      +0.5        0.97 ± 59%  perf-profile.children.cycles-pp.__x64_sys_getdents64
      0.47 ± 24%      +0.5        0.97 ± 59%  perf-profile.children.cycles-pp.iterate_dir
      0.43 ± 27%      +0.5        0.93 ± 61%  perf-profile.children.cycles-pp.kernfs_fop_readdir
      1.86 ± 23%      +1.6        3.48 ± 54%  perf-profile.children.cycles-pp.lookup_fast
      2.08 ± 22%      +1.6        3.71 ± 52%  perf-profile.children.cycles-pp.walk_component
      1.65 ± 27%      +1.7        3.31 ± 57%  perf-profile.children.cycles-pp.kernfs_dop_revalidate
      2.42 ± 26%      +2.3        4.71 ± 56%  perf-profile.children.cycles-pp.inode_permission
      2.27 ± 28%      +2.3        4.58 ± 58%  perf-profile.children.cycles-pp.kernfs_iop_permission
      4.26 ± 24%      +3.8        8.02 ± 55%  perf-profile.children.cycles-pp.link_path_walk
      3896 ±  4%     -26.1%       2881 ±  5%  slabinfo.Acpi-Parse.active_objs
      3896 ±  4%     -26.1%       2881 ±  5%  slabinfo.Acpi-Parse.num_objs
      2695 ± 13%     +17.5%       3166 ± 11%  slabinfo.buffer_head.active_objs
      2695 ± 13%     +17.5%       3166 ± 11%  slabinfo.buffer_head.num_objs
    121171           +12.7%     136550 ±  2%  slabinfo.dentry.active_objs
      2954           +11.5%       3293 ±  2%  slabinfo.dentry.active_slabs
    124128           +11.5%     138350 ±  2%  slabinfo.dentry.num_objs
      2954           +11.5%       3293 ±  2%  slabinfo.dentry.num_slabs
     65205 ±  2%     -32.1%      44268 ±  6%  slabinfo.filp.active_objs
      2069 ±  2%     -27.8%       1493 ±  6%  slabinfo.filp.active_slabs
     66223 ±  2%     -27.8%      47810 ±  6%  slabinfo.filp.num_objs
      2069 ±  2%     -27.8%       1493 ±  6%  slabinfo.filp.num_slabs
     87249           +16.1%     101301 ±  2%  slabinfo.inode_cache.active_objs
      1674           +12.8%       1888 ±  2%  slabinfo.inode_cache.active_slabs
     90457           +12.8%     102020 ±  2%  slabinfo.inode_cache.num_objs
      1674           +12.8%       1888 ±  2%  slabinfo.inode_cache.num_slabs
     44095           -16.5%      36822 ±  4%  slabinfo.kmalloc-16.num_objs
      6886           -17.4%       5685 ±  3%  slabinfo.kmalloc-192.num_objs
     73187           -18.6%      59605        slabinfo.kmalloc-8.active_objs
     73828           -17.3%      61093        slabinfo.kmalloc-8.num_objs
      6458           +18.4%       7647 ±  3%  slabinfo.kmalloc-96.active_objs
      7259            +9.8%       7974 ±  2%  slabinfo.kmalloc-96.num_objs
    995.29           -65.3%     345.33 ±  9%  slabinfo.request_queue.active_objs
    995.29           -65.3%     345.33 ±  9%  slabinfo.request_queue.num_objs
     31473 ±  3%     -43.8%      17678 ± 10%  slabinfo.shmem_inode_cache.active_objs
    690.00 ±  3%     -44.0%     386.67 ± 10%  slabinfo.shmem_inode_cache.active_slabs
     31769 ±  3%     -44.0%      17803 ± 10%  slabinfo.shmem_inode_cache.num_objs
    690.00 ±  3%     -44.0%     386.67 ± 10%  slabinfo.shmem_inode_cache.num_slabs
      1313 ±  5%     +13.3%       1487 ±  4%  slabinfo.task_group.active_objs
      1313 ±  5%     +13.3%       1487 ±  4%  slabinfo.task_group.num_objs
     17797 ± 10%     +25.3%      22297 ±  5%  slabinfo.vmap_area.active_objs
     18750 ±  7%     +23.7%      23198 ±  4%  slabinfo.vmap_area.num_objs
     16852 ± 17%     +76.0%      29667 ± 11%  sched_debug.cfs_rq:/.load.avg
    422015 ± 23%     +46.9%     620094 ±  8%  sched_debug.cfs_rq:/.load.max
      6078           -52.8%       2870 ± 19%  sched_debug.cfs_rq:/.load.min
     56970 ± 25%     +92.8%     109819 ±  9%  sched_debug.cfs_rq:/.load.stddev
      8.49 ± 27%     -57.5%       3.61 ± 32%  sched_debug.cfs_rq:/.load_avg.min
  10716828           -15.1%    9102611 ±  3%  sched_debug.cfs_rq:/.min_vruntime.avg
  10941313           -15.0%    9295274 ±  3%  sched_debug.cfs_rq:/.min_vruntime.max
  10319971           -15.0%    8767619 ±  3%  sched_debug.cfs_rq:/.min_vruntime.min
      0.61           -47.5%       0.32 ± 15%  sched_debug.cfs_rq:/.nr_running.avg
      0.57           -53.0%       0.27 ± 19%  sched_debug.cfs_rq:/.nr_running.min
      0.13 ± 11%     +41.8%       0.18 ±  9%  sched_debug.cfs_rq:/.nr_running.stddev
    652.07           -50.8%     320.85 ± 16%  sched_debug.cfs_rq:/.runnable_avg.avg
      1180 ±  4%     -27.8%     852.72 ±  8%  sched_debug.cfs_rq:/.runnable_avg.max
    587.84           -53.1%     275.81 ± 19%  sched_debug.cfs_rq:/.runnable_avg.min
    641.37           -50.7%     316.10 ± 17%  sched_debug.cfs_rq:/.util_avg.avg
    958.02 ±  3%     -19.0%     776.10 ±  8%  sched_debug.cfs_rq:/.util_avg.max
    570.18           -52.8%     269.25 ± 19%  sched_debug.cfs_rq:/.util_avg.min
     73.27 ±  5%     +21.9%      89.33 ±  5%  sched_debug.cfs_rq:/.util_avg.stddev
    287.72 ±  2%     -52.6%     136.32 ± 19%  sched_debug.cfs_rq:/.util_est_enqueued.avg
    565.65 ± 14%     -42.8%     323.83 ± 22%  sched_debug.cfs_rq:/.util_est_enqueued.max
    138.49 ± 10%     -55.5%      61.65 ± 23%  sched_debug.cfs_rq:/.util_est_enqueued.min
     71.94 ± 14%     -44.3%      40.08 ± 21%  sched_debug.cfs_rq:/.util_est_enqueued.stddev
    801363 ±  2%     +10.5%     885168 ±  2%  sched_debug.cpu.avg_idle.avg
    209867          +121.5%     464779 ± 20%  sched_debug.cpu.clock.avg
    209880          +121.5%     464786 ± 20%  sched_debug.cpu.clock.max
    209856          +121.5%     464772 ± 20%  sched_debug.cpu.clock.min
      7.26 ±  7%     -41.7%       4.23 ± 15%  sched_debug.cpu.clock.stddev
    207703          +120.1%     457127 ± 20%  sched_debug.cpu.clock_task.avg
    207939          +120.0%     457508 ± 20%  sched_debug.cpu.clock_task.max
    202738          +123.0%     452135 ± 20%  sched_debug.cpu.clock_task.min
     15311 ±  2%     -37.4%       9584 ± 10%  sched_debug.cpu.curr->pid.avg
     23205 ±  2%     +32.1%      30656 ±  8%  sched_debug.cpu.curr->pid.max
     10827 ± 13%     -44.4%       6021 ± 39%  sched_debug.cpu.curr->pid.min
      1468 ± 13%     +79.3%       2634 ± 15%  sched_debug.cpu.curr->pid.stddev
      0.00 ±  4%     -60.0%       0.00 ± 10%  sched_debug.cpu.next_balance.stddev
      0.62           -51.3%       0.30 ± 18%  sched_debug.cpu.nr_running.avg
      1.65 ±  6%     -21.2%       1.30 ±  5%  sched_debug.cpu.nr_running.max
      0.57           -53.0%       0.27 ± 19%  sched_debug.cpu.nr_running.min
      0.19 ±  7%     -16.2%       0.16 ±  6%  sched_debug.cpu.nr_running.stddev
     65242 ±  4%     +12.0%      73099 ±  2%  sched_debug.cpu.nr_switches.avg
    142006 ±  8%     +69.7%     241000 ± 23%  sched_debug.cpu.nr_switches.max
      9433 ± 10%    +126.9%      21401 ± 26%  sched_debug.cpu.nr_switches.stddev
      0.77 ±  9%     +41.1%       1.08 ±  9%  sched_debug.cpu.nr_uninterruptible.avg
     75.72 ±  8%     +12.5%      85.20 ±  5%  sched_debug.cpu.nr_uninterruptible.stddev
    209854          +121.5%     464772 ± 20%  sched_debug.cpu_clk
    209359          +121.8%     464277 ± 20%  sched_debug.ktime
    210224          +121.3%     465139 ± 20%  sched_debug.sched_clk
     10.07 ± 14%    +145.3%      24.70 ± 72%  perf-stat.i.MPKI
 7.403e+09           -55.9%  3.267e+09 ± 20%  perf-stat.i.branch-instructions
      0.82 ± 19%      +1.9        2.73 ± 81%  perf-stat.i.branch-miss-rate%
  32971236 ±  3%     -45.9%   17833407 ± 28%  perf-stat.i.branch-misses
  62721218 ±  4%     -54.9%   28285991 ± 17%  perf-stat.i.cache-misses
 2.427e+08 ±  4%     -50.4%  1.203e+08 ± 22%  perf-stat.i.cache-references
     20953 ±  3%     -54.1%       9620 ± 19%  perf-stat.i.context-switches
 1.957e+11           -55.4%  8.718e+10 ± 20%  perf-stat.i.cpu-cycles
      1558 ±  3%     -52.5%     740.39 ± 21%  perf-stat.i.cpu-migrations
     98025 ±  3%     -55.7%      43407 ± 23%  perf-stat.i.cycles-between-cache-misses
   3089148 ±  5%     -38.6%    1897995 ± 38%  perf-stat.i.dTLB-load-misses
 7.857e+09           -55.4%  3.508e+09 ± 19%  perf-stat.i.dTLB-loads
   2948479 ± 16%     -54.4%    1344801 ± 24%  perf-stat.i.dTLB-store-misses
 2.368e+09 ±  2%     -54.7%  1.072e+09 ± 18%  perf-stat.i.dTLB-stores
     69.91           -14.0       55.89 ±  5%  perf-stat.i.iTLB-load-miss-rate%
  13760398 ±  2%     -52.4%    6546686 ± 16%  perf-stat.i.iTLB-load-misses
   6194204 ±  6%     -42.4%    3566092 ± 12%  perf-stat.i.iTLB-loads
 3.095e+10           -55.6%  1.375e+10 ± 20%  perf-stat.i.instructions
      4193 ±  4%     -49.1%       2135 ± 16%  perf-stat.i.instructions-per-iTLB-miss
      3.24           -54.5%       1.48 ± 17%  perf-stat.i.major-faults
      2.04           -55.4%       0.91 ± 20%  perf-stat.i.metric.GHz
    186.35           -55.4%      83.10 ± 19%  perf-stat.i.metric.M/sec
    587004 ±  4%     -57.5%     249485 ± 21%  perf-stat.i.minor-faults
     83.95            +4.5       88.41 ±  2%  perf-stat.i.node-load-miss-rate%
   9800429 ±  2%     -55.9%    4326887 ± 18%  perf-stat.i.node-load-misses
   1818557 ±  3%     -55.7%     805771 ± 19%  perf-stat.i.node-loads
   7074657 ±  2%     -57.3%    3022001 ± 20%  perf-stat.i.node-store-misses
   1170221           -56.4%     510653 ± 21%  perf-stat.i.node-stores
    587343 ±  4%     -57.5%     249623 ± 21%  perf-stat.i.page-faults
     69.07 ±  2%      -4.4       64.70 ±  2%  perf-stat.overall.iTLB-load-miss-rate%
 7.379e+09           -55.6%  3.277e+09 ± 19%  perf-stat.ps.branch-instructions
  32913748 ±  3%     -45.7%   17887460 ± 28%  perf-stat.ps.branch-misses
  62364281 ±  4%     -54.6%   28319563 ± 17%  perf-stat.ps.cache-misses
 2.413e+08 ±  4%     -50.1%  1.203e+08 ± 22%  perf-stat.ps.cache-references
     21227 ±  3%     -53.9%       9778 ± 19%  perf-stat.ps.context-switches
 1.945e+11           -55.2%  8.723e+10 ± 20%  perf-stat.ps.cpu-cycles
      1581 ±  3%     -52.3%     754.75 ± 21%  perf-stat.ps.cpu-migrations
   3068889 ±  5%     -38.2%    1896861 ± 38%  perf-stat.ps.dTLB-load-misses
 7.831e+09           -55.1%  3.518e+09 ± 19%  perf-stat.ps.dTLB-loads
   2919101 ± 16%     -54.1%    1341135 ± 24%  perf-stat.ps.dTLB-store-misses
 2.359e+09 ±  2%     -54.5%  1.075e+09 ± 18%  perf-stat.ps.dTLB-stores
  13732191 ±  2%     -52.1%    6573899 ± 16%  perf-stat.ps.iTLB-load-misses
   6153152 ±  6%     -42.1%    3562626 ± 12%  perf-stat.ps.iTLB-loads
 3.085e+10           -55.3%  1.379e+10 ± 19%  perf-stat.ps.instructions
      3.25           -54.1%       1.49 ± 17%  perf-stat.ps.major-faults
    581421 ±  4%     -57.2%     248941 ± 21%  perf-stat.ps.minor-faults
   9768625 ±  2%     -55.5%    4342219 ± 18%  perf-stat.ps.node-load-misses
   1814279 ±  3%     -55.4%     809363 ± 19%  perf-stat.ps.node-loads
   7049268 ±  3%     -57.0%    3032742 ± 20%  perf-stat.ps.node-store-misses
   1169828           -56.1%     514019 ± 21%  perf-stat.ps.node-stores
    581769 ±  4%     -57.2%     249084 ± 21%  perf-stat.ps.page-faults
 1.124e+13            +2.9%  1.156e+13        perf-stat.total.instructions
    729.71          +140.4%       1754 ± 22%  interrupts.9:IO-APIC.9-fasteoi.acpi
    729357          +140.6%    1754773 ± 21%  interrupts.CPU0.LOC:Local_timer_interrupts
    729.71          +140.4%       1754 ± 22%  interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi
    729821          +140.4%    1754679 ± 21%  interrupts.CPU1.LOC:Local_timer_interrupts
    729386          +140.5%    1754509 ± 22%  interrupts.CPU10.LOC:Local_timer_interrupts
    729412          +140.5%    1754514 ± 21%  interrupts.CPU11.LOC:Local_timer_interrupts
    729031          +140.6%    1754160 ± 21%  interrupts.CPU12.LOC:Local_timer_interrupts
    729475          +140.5%    1754273 ± 21%  interrupts.CPU13.LOC:Local_timer_interrupts
    729102          +140.6%    1754493 ± 21%  interrupts.CPU14.LOC:Local_timer_interrupts
    729717          +140.5%    1754612 ± 21%  interrupts.CPU15.LOC:Local_timer_interrupts
    729627          +140.4%    1754285 ± 21%  interrupts.CPU16.LOC:Local_timer_interrupts
    729274          +140.7%    1755011 ± 21%  interrupts.CPU17.LOC:Local_timer_interrupts
    729806          +140.4%    1754723 ± 21%  interrupts.CPU18.LOC:Local_timer_interrupts
    729574          +140.5%    1754478 ± 21%  interrupts.CPU19.LOC:Local_timer_interrupts
    729426          +140.5%    1754493 ± 21%  interrupts.CPU2.LOC:Local_timer_interrupts
    729244          +140.5%    1754033 ± 21%  interrupts.CPU20.LOC:Local_timer_interrupts
    729711          +140.5%    1754632 ± 22%  interrupts.CPU21.LOC:Local_timer_interrupts
    729366          +140.5%    1754262 ± 21%  interrupts.CPU22.LOC:Local_timer_interrupts
    729480          +140.5%    1754577 ± 21%  interrupts.CPU23.LOC:Local_timer_interrupts
    729151          +140.7%    1754712 ± 21%  interrupts.CPU24.LOC:Local_timer_interrupts
      9912 ± 10%     -19.0%       8026 ±  4%  interrupts.CPU25.CAL:Function_call_interrupts
    729242          +140.6%    1754726 ± 21%  interrupts.CPU25.LOC:Local_timer_interrupts
    729368          +140.5%    1754187 ± 21%  interrupts.CPU26.LOC:Local_timer_interrupts
    729423          +140.5%    1754291 ± 21%  interrupts.CPU27.LOC:Local_timer_interrupts
    729379          +140.5%    1754139 ± 21%  interrupts.CPU28.LOC:Local_timer_interrupts
    729699          +140.4%    1754474 ± 21%  interrupts.CPU29.LOC:Local_timer_interrupts
    729552          +140.5%    1754499 ± 22%  interrupts.CPU3.LOC:Local_timer_interrupts
    729031          +140.6%    1754303 ± 21%  interrupts.CPU30.LOC:Local_timer_interrupts
    729307          +140.6%    1754496 ± 21%  interrupts.CPU31.LOC:Local_timer_interrupts
    729071          +140.7%    1754789 ± 21%  interrupts.CPU32.LOC:Local_timer_interrupts
    729226          +140.6%    1754795 ± 21%  interrupts.CPU33.LOC:Local_timer_interrupts
    728976          +140.6%    1754102 ± 21%  interrupts.CPU34.LOC:Local_timer_interrupts
    729691          +140.4%    1754505 ± 21%  interrupts.CPU35.LOC:Local_timer_interrupts
    729166          +140.6%    1754201 ± 21%  interrupts.CPU36.LOC:Local_timer_interrupts
    729342          +140.6%    1754462 ± 21%  interrupts.CPU37.LOC:Local_timer_interrupts
    729316          +140.5%    1754167 ± 21%  interrupts.CPU38.LOC:Local_timer_interrupts
    728849          +140.7%    1754698 ± 21%  interrupts.CPU39.LOC:Local_timer_interrupts
    729704          +140.5%    1754890 ± 21%  interrupts.CPU4.LOC:Local_timer_interrupts
    729240          +140.5%    1754038 ± 21%  interrupts.CPU40.LOC:Local_timer_interrupts
    729104          +140.6%    1754432 ± 21%  interrupts.CPU41.LOC:Local_timer_interrupts
    729329          +140.6%    1754621 ± 21%  interrupts.CPU42.LOC:Local_timer_interrupts
    729588          +140.5%    1754591 ± 22%  interrupts.CPU43.LOC:Local_timer_interrupts
    649.43 ±  2%    +209.9%       2012 ±128%  interrupts.CPU43.RES:Rescheduling_interrupts
    729351          +140.5%    1754314 ± 22%  interrupts.CPU44.LOC:Local_timer_interrupts
    729075          +140.7%    1754891 ± 22%  interrupts.CPU45.LOC:Local_timer_interrupts
    729366          +140.5%    1754386 ± 21%  interrupts.CPU46.LOC:Local_timer_interrupts
    728947          +140.7%    1754603 ± 22%  interrupts.CPU47.LOC:Local_timer_interrupts
    729443          +140.5%    1754092 ± 21%  interrupts.CPU48.LOC:Local_timer_interrupts
    729689          +140.5%    1754547 ± 21%  interrupts.CPU49.LOC:Local_timer_interrupts
    729492          +140.6%    1754896 ± 21%  interrupts.CPU5.LOC:Local_timer_interrupts
    729592          +140.4%    1754124 ± 22%  interrupts.CPU50.LOC:Local_timer_interrupts
    728964          +140.7%    1754460 ± 21%  interrupts.CPU51.LOC:Local_timer_interrupts
    729788          +140.4%    1754626 ± 21%  interrupts.CPU52.LOC:Local_timer_interrupts
    729841          +140.4%    1754643 ± 21%  interrupts.CPU53.LOC:Local_timer_interrupts
    729323          +140.6%    1754713 ± 22%  interrupts.CPU54.LOC:Local_timer_interrupts
    729344          +140.6%    1754682 ± 22%  interrupts.CPU55.LOC:Local_timer_interrupts
    729348          +140.6%    1755026 ± 22%  interrupts.CPU56.LOC:Local_timer_interrupts
    729338          +140.6%    1754492 ± 21%  interrupts.CPU57.LOC:Local_timer_interrupts
    729337          +140.6%    1754608 ± 21%  interrupts.CPU58.LOC:Local_timer_interrupts
    729804          +140.4%    1754565 ± 22%  interrupts.CPU59.LOC:Local_timer_interrupts
    729680          +140.5%    1755106 ± 22%  interrupts.CPU6.LOC:Local_timer_interrupts
    729475          +140.5%    1754650 ± 21%  interrupts.CPU60.LOC:Local_timer_interrupts
    729498          +140.5%    1754537 ± 21%  interrupts.CPU61.LOC:Local_timer_interrupts
    729337          +140.5%    1754304 ± 21%  interrupts.CPU62.LOC:Local_timer_interrupts
    729965          +140.4%    1754554 ± 22%  interrupts.CPU63.LOC:Local_timer_interrupts
    729696          +140.4%    1754455 ± 21%  interrupts.CPU64.LOC:Local_timer_interrupts
    729412          +140.5%    1754535 ± 21%  interrupts.CPU65.LOC:Local_timer_interrupts
    729532          +140.5%    1754692 ± 21%  interrupts.CPU66.LOC:Local_timer_interrupts
    729629          +140.5%    1754529 ± 21%  interrupts.CPU67.LOC:Local_timer_interrupts
    729573          +140.5%    1754492 ± 21%  interrupts.CPU68.LOC:Local_timer_interrupts
    729095          +140.7%    1754764 ± 21%  interrupts.CPU69.LOC:Local_timer_interrupts
    729350          +140.6%    1754840 ± 22%  interrupts.CPU7.LOC:Local_timer_interrupts
    729768          +140.4%    1754460 ± 21%  interrupts.CPU70.LOC:Local_timer_interrupts
    729482          +140.5%    1754368 ± 21%  interrupts.CPU71.LOC:Local_timer_interrupts
    729257          +140.6%    1754349 ± 21%  interrupts.CPU72.LOC:Local_timer_interrupts
    729577          +140.5%    1754329 ± 21%  interrupts.CPU73.LOC:Local_timer_interrupts
    729420          +140.5%    1754481 ± 22%  interrupts.CPU74.LOC:Local_timer_interrupts
    729164          +140.6%    1754477 ± 22%  interrupts.CPU75.LOC:Local_timer_interrupts
    729256          +140.6%    1754386 ± 21%  interrupts.CPU76.LOC:Local_timer_interrupts
    729258          +140.6%    1754817 ± 21%  interrupts.CPU77.LOC:Local_timer_interrupts
    729618          +140.4%    1754329 ± 21%  interrupts.CPU78.LOC:Local_timer_interrupts
    729195          +140.6%    1754203 ± 21%  interrupts.CPU79.LOC:Local_timer_interrupts
    729523          +140.5%    1754652 ± 21%  interrupts.CPU8.LOC:Local_timer_interrupts
    729515          +140.5%    1754515 ± 22%  interrupts.CPU80.LOC:Local_timer_interrupts
    729269          +140.5%    1754189 ± 21%  interrupts.CPU81.LOC:Local_timer_interrupts
    729281          +140.5%    1754169 ± 22%  interrupts.CPU82.LOC:Local_timer_interrupts
    729217          +140.6%    1754321 ± 21%  interrupts.CPU83.LOC:Local_timer_interrupts
    729641          +140.5%    1754553 ± 22%  interrupts.CPU84.LOC:Local_timer_interrupts
    728947          +140.7%    1754445 ± 22%  interrupts.CPU85.LOC:Local_timer_interrupts
    729242          +140.6%    1754268 ± 21%  interrupts.CPU86.LOC:Local_timer_interrupts
    742.00 ± 14%     -18.7%     603.00 ±  6%  interrupts.CPU86.RES:Rescheduling_interrupts
    729720          +140.4%    1754273 ± 21%  interrupts.CPU87.LOC:Local_timer_interrupts
    729673          +140.4%    1754450 ± 21%  interrupts.CPU88.LOC:Local_timer_interrupts
    729216          +140.6%    1754474 ± 21%  interrupts.CPU89.LOC:Local_timer_interrupts
    729263          +140.6%    1754418 ± 22%  interrupts.CPU9.LOC:Local_timer_interrupts
    729400          +140.5%    1754383 ± 21%  interrupts.CPU90.LOC:Local_timer_interrupts
    729249          +140.6%    1754544 ± 21%  interrupts.CPU91.LOC:Local_timer_interrupts
    728980          +140.7%    1754421 ± 22%  interrupts.CPU92.LOC:Local_timer_interrupts
    729036          +140.6%    1754104 ± 22%  interrupts.CPU93.LOC:Local_timer_interrupts
    729038          +140.6%    1754296 ± 21%  interrupts.CPU94.LOC:Local_timer_interrupts
    728704          +140.8%    1754436 ± 21%  interrupts.CPU95.LOC:Local_timer_interrupts
  70020802          +140.5%  1.684e+08 ± 21%  interrupts.LOC:Local_timer_interrupts
     96.00          +133.3%     224.00 ± 20%  interrupts.MCP:Machine_check_polls
     87063 ±  3%     +29.3%     112565 ± 10%  softirqs.CPU0.RCU
     19291 ±  4%    +319.2%      80871 ± 27%  softirqs.CPU0.SCHED
     84306 ±  2%     +33.0%     112095 ±  8%  softirqs.CPU1.RCU
     17098 ±  4%    +361.2%      78858 ± 28%  softirqs.CPU1.SCHED
     83314 ±  2%     +33.5%     111188 ±  7%  softirqs.CPU10.RCU
     16366 ±  5%    +305.9%      66436 ± 35%  softirqs.CPU10.SCHED
     82645 ±  3%     +34.1%     110839 ±  7%  softirqs.CPU11.RCU
     16820 ±  4%    +366.0%      78389 ± 28%  softirqs.CPU11.SCHED
     81897 ±  3%     +34.9%     110444 ±  7%  softirqs.CPU12.RCU
     16463 ±  5%    +373.9%      78020 ± 28%  softirqs.CPU12.SCHED
     82114           +33.9%     109972 ±  7%  softirqs.CPU13.RCU
     16319 ±  5%    +379.3%      78218 ± 28%  softirqs.CPU13.SCHED
     81428 ±  4%     +33.5%     108711 ±  9%  softirqs.CPU14.RCU
     16037 ± 10%    +387.7%      78222 ± 29%  softirqs.CPU14.SCHED
     81014 ±  3%     +36.2%     110327 ±  7%  softirqs.CPU15.RCU
     16132 ±  5%    +384.7%      78200 ± 29%  softirqs.CPU15.SCHED
     18325 ±114%    -100.0%       7.17 ± 66%  softirqs.CPU16.BLOCK
     83479 ±  3%     +31.8%     110030 ± 10%  softirqs.CPU16.RCU
     17442 ±  3%    +348.3%      78195 ± 29%  softirqs.CPU16.SCHED
     81902 ±  2%     +35.1%     110635 ±  8%  softirqs.CPU17.RCU
     15903 ± 10%    +395.5%      78796 ± 28%  softirqs.CPU17.SCHED
     80731 ±  3%     +34.3%     108404 ± 10%  softirqs.CPU18.RCU
     16741 ±  4%    +369.3%      78561 ± 28%  softirqs.CPU18.SCHED
     82414 ±  2%     +30.1%     107231 ±  7%  softirqs.CPU19.RCU
     16125 ±  4%    +385.7%      78330 ± 29%  softirqs.CPU19.SCHED
     82439 ±  2%     +34.5%     110902 ±  9%  softirqs.CPU2.RCU
     14985 ± 10%    +426.2%      78854 ± 28%  softirqs.CPU2.SCHED
     81327 ±  4%     +37.6%     111899 ±  8%  softirqs.CPU20.RCU
     16566 ±  7%    +361.9%      76527 ± 27%  softirqs.CPU20.SCHED
     81841 ±  4%     +33.6%     109362 ±  8%  softirqs.CPU21.RCU
     16005 ±  6%    +388.5%      78192 ± 29%  softirqs.CPU21.SCHED
     80566 ±  4%     +36.4%     109861 ±  8%  softirqs.CPU22.RCU
     15971 ±  6%    +391.2%      78454 ± 29%  softirqs.CPU22.SCHED
     81863 ±  2%     +32.4%     108376 ± 10%  softirqs.CPU23.RCU
     16727 ±  2%    +368.0%      78281 ± 29%  softirqs.CPU23.SCHED
     87993 ±  3%     +27.5%     112191 ± 10%  softirqs.CPU24.RCU
     15464 ± 11%    +427.7%      81612 ± 32%  softirqs.CPU24.SCHED
     85959 ±  3%     +30.9%     112555 ±  8%  softirqs.CPU25.RCU
     16429 ±  4%    +380.6%      78965 ± 28%  softirqs.CPU25.SCHED
     83219 ±  6%     +33.2%     110840 ±  8%  softirqs.CPU26.RCU
     15982 ±  5%    +394.5%      79039 ± 28%  softirqs.CPU26.SCHED
     82086 ±  5%     +35.2%     110947 ±  9%  softirqs.CPU27.RCU
     16031 ±  8%    +393.7%      79148 ± 28%  softirqs.CPU27.SCHED
     84842 ±  3%     +31.6%     111610 ±  8%  softirqs.CPU28.RCU
     15500 ± 11%    +415.0%      79828 ± 27%  softirqs.CPU28.SCHED
     85066 ±  3%     +31.2%     111577 ±  8%  softirqs.CPU29.RCU
     15856 ±  5%    +397.6%      78904 ± 28%  softirqs.CPU29.SCHED
     82344 ±  4%     +36.9%     112702 ±  8%  softirqs.CPU3.RCU
     15260 ±  8%    +403.2%      76794 ± 32%  softirqs.CPU3.SCHED
     83594 ±  5%     +31.3%     109800 ±  7%  softirqs.CPU30.RCU
     16551 ±  3%    +376.6%      78882 ± 28%  softirqs.CPU30.SCHED
     82947 ±  7%     +32.4%     109833 ± 10%  softirqs.CPU31.RCU
     16149 ±  7%    +387.1%      78673 ± 28%  softirqs.CPU31.SCHED
     84732 ±  3%     +34.8%     114256 ±  8%  softirqs.CPU32.RCU
     15988 ± 10%    +395.7%      79247 ± 28%  softirqs.CPU32.SCHED
     83430 ±  5%     +36.3%     113685 ±  8%  softirqs.CPU33.RCU
     15340 ±  8%    +413.8%      78821 ± 28%  softirqs.CPU33.SCHED
     85678 ±  3%     +34.3%     115060 ±  8%  softirqs.CPU34.RCU
     15503 ± 10%    +409.5%      78994 ± 28%  softirqs.CPU34.SCHED
     84635 ±  5%     +35.2%     114436 ±  8%  softirqs.CPU35.RCU
     16547 ±  3%    +377.2%      78974 ± 28%  softirqs.CPU35.SCHED
     85010 ±  3%     +29.4%     109964 ±  7%  softirqs.CPU36.RCU
     15756 ± 11%    +405.6%      79667 ± 28%  softirqs.CPU36.SCHED
     83869 ±  5%     +33.5%     111951 ±  7%  softirqs.CPU37.RCU
     15843 ±  4%    +396.9%      78720 ± 28%  softirqs.CPU37.SCHED
     83148 ±  6%     +33.5%     111005 ±  8%  softirqs.CPU38.RCU
     15656 ±  8%    +406.2%      79252 ± 29%  softirqs.CPU38.SCHED
     84300 ±  6%     +31.8%     111124 ±  8%  softirqs.CPU39.RCU
     15964 ±  3%    +396.7%      79287 ± 29%  softirqs.CPU39.SCHED
     82975 ±  2%     +32.9%     110261 ±  8%  softirqs.CPU4.RCU
     16608 ±  2%    +327.3%      70972 ± 22%  softirqs.CPU4.SCHED
     84792 ±  4%     +33.7%     113386 ± 10%  softirqs.CPU40.RCU
     16338 ±  5%    +382.5%      78834 ± 28%  softirqs.CPU40.SCHED
     86177 ±  3%     +33.8%     115303 ±  8%  softirqs.CPU41.RCU
     15868 ±  6%    +398.2%      79060 ± 28%  softirqs.CPU41.SCHED
     83097 ±  4%     +32.8%     110348 ± 11%  softirqs.CPU42.RCU
     16953 ±  6%    +363.6%      78603 ± 28%  softirqs.CPU42.SCHED
     81892 ±  6%     +38.4%     113349 ±  9%  softirqs.CPU43.RCU
     16027 ±  4%    +392.2%      78886 ± 28%  softirqs.CPU43.SCHED
     85013 ±  5%     +33.9%     113856 ±  8%  softirqs.CPU44.RCU
     16114 ±  3%    +369.7%      75691 ± 36%  softirqs.CPU44.SCHED
     85689 ±  3%     +33.9%     114751 ±  8%  softirqs.CPU45.RCU
     15457 ±  6%    +410.3%      78873 ± 28%  softirqs.CPU45.SCHED
     84736 ±  5%     +33.7%     113258 ±  7%  softirqs.CPU46.RCU
     15502 ±  8%    +409.0%      78901 ± 28%  softirqs.CPU46.SCHED
     84317 ±  6%     +34.1%     113053 ±  8%  softirqs.CPU47.RCU
     15752 ±  5%    +400.1%      78785 ± 28%  softirqs.CPU47.SCHED
     82686 ±  2%     +30.3%     107716 ±  8%  softirqs.CPU48.RCU
     15481 ±  6%    +407.8%      78607 ± 29%  softirqs.CPU48.SCHED
     84297 ±  2%     +33.9%     112856 ±  7%  softirqs.CPU49.RCU
     16254 ±  7%    +370.8%      76526 ± 24%  softirqs.CPU49.SCHED
     82208 ±  4%     +33.5%     109751 ±  9%  softirqs.CPU5.RCU
     15599 ± 11%    +401.8%      78281 ± 28%  softirqs.CPU5.SCHED
     82634           +33.1%     110003 ±  8%  softirqs.CPU50.RCU
     16018 ±  8%    +396.3%      79502 ± 28%  softirqs.CPU50.SCHED
     82336           +34.7%     110920 ±  9%  softirqs.CPU51.RCU
     15972 ± 11%    +395.5%      79147 ± 29%  softirqs.CPU51.SCHED
     82684 ±  2%     +33.9%     110738 ±  7%  softirqs.CPU52.RCU
     16511 ±  6%    +383.4%      79821 ± 29%  softirqs.CPU52.SCHED
     81199 ±  2%     +37.0%     111229 ±  8%  softirqs.CPU53.RCU
     15638 ±  9%    +411.4%      79970 ± 29%  softirqs.CPU53.SCHED
     81315 ±  5%     +37.3%     111643 ±  8%  softirqs.CPU54.RCU
     15661 ±  3%    +411.3%      80073 ± 29%  softirqs.CPU54.SCHED
     80028 ±  4%     +35.8%     108711 ±  7%  softirqs.CPU55.RCU
     15215 ±  6%    +419.3%      79017 ± 29%  softirqs.CPU55.SCHED
     82528 ±  2%     +33.1%     109875 ±  8%  softirqs.CPU56.RCU
     17018          +341.1%      75064 ± 35%  softirqs.CPU56.SCHED
     81572 ±  2%     +34.0%     109280 ±  8%  softirqs.CPU57.RCU
     16588 ±  2%    +380.4%      79689 ± 29%  softirqs.CPU57.SCHED
     81401 ±  5%     +35.9%     110657 ±  7%  softirqs.CPU58.RCU
     15302 ± 10%    +421.6%      79818 ± 29%  softirqs.CPU58.SCHED
     82077 ±  2%     +35.7%     111382 ±  7%  softirqs.CPU59.RCU
     16677 ±  3%    +378.4%      79782 ± 29%  softirqs.CPU59.SCHED
     83340 ±  3%     +32.8%     110671 ±  7%  softirqs.CPU6.RCU
     15960 ±  5%    +391.2%      78396 ± 28%  softirqs.CPU6.SCHED
     80697 ±  5%     +36.8%     110422 ±  7%  softirqs.CPU60.RCU
     16259 ±  4%    +387.5%      79262 ± 29%  softirqs.CPU60.SCHED
     82538           +34.9%     111338 ±  7%  softirqs.CPU61.RCU
     16199 ±  6%    +389.2%      79245 ± 29%  softirqs.CPU61.SCHED
     82543 ±  3%     +33.4%     110132 ±  7%  softirqs.CPU62.RCU
     16458 ±  5%    +382.5%      79413 ± 26%  softirqs.CPU62.SCHED
     82032 ±  3%     +31.5%     107864 ±  6%  softirqs.CPU63.RCU
     15885 ±  6%    +397.0%      78956 ± 28%  softirqs.CPU63.SCHED
     80600 ±  2%     +33.7%     107756 ±  9%  softirqs.CPU64.RCU
     16161 ±  4%    +393.9%      79827 ± 29%  softirqs.CPU64.SCHED
     81896           +36.0%     111401 ±  8%  softirqs.CPU65.RCU
     16678 ±  6%    +379.4%      79956 ± 29%  softirqs.CPU65.SCHED
     81249           +32.7%     107847 ± 11%  softirqs.CPU66.RCU
     16606 ±  4%    +378.5%      79459 ± 29%  softirqs.CPU66.SCHED
     81245           +34.4%     109209 ±  9%  softirqs.CPU67.RCU
     15815 ±  6%    +398.3%      78813 ± 28%  softirqs.CPU67.SCHED
     82868 ±  2%     +32.6%     109916 ± 10%  softirqs.CPU68.RCU
     16448 ±  5%    +385.7%      79887 ± 29%  softirqs.CPU68.SCHED
     82275 ±  2%     +31.6%     108287 ± 14%  softirqs.CPU69.RCU
     16462 ±  4%    +384.6%      79776 ± 28%  softirqs.CPU69.SCHED
     82833 ±  2%     +33.8%     110811 ±  7%  softirqs.CPU7.RCU
     16627 ±  4%    +370.5%      78234 ± 29%  softirqs.CPU7.SCHED
     81081 ±  3%     +34.1%     108765 ±  8%  softirqs.CPU70.RCU
     16056 ±  5%    +398.4%      80026 ± 29%  softirqs.CPU70.SCHED
     81280 ±  2%     +30.4%     105970 ± 10%  softirqs.CPU71.RCU
     15086 ± 10%    +404.2%      76060 ± 36%  softirqs.CPU71.SCHED
     86835 ±  3%     +27.3%     110583 ±  7%  softirqs.CPU72.RCU
     16615 ±  6%    +386.3%      80794 ± 29%  softirqs.CPU72.SCHED
     83217 ±  4%     +33.7%     111256 ±  7%  softirqs.CPU73.RCU
     16465 ±  4%    +358.8%      75551 ± 27%  softirqs.CPU73.SCHED
     83785 ±  3%     +29.5%     108513 ±  9%  softirqs.CPU74.RCU
     16285 ±  6%    +394.5%      80525 ± 28%  softirqs.CPU74.SCHED
     84189 ±  3%     +32.5%     111582 ±  8%  softirqs.CPU75.RCU
     16334 ±  3%    +393.5%      80617 ± 29%  softirqs.CPU75.SCHED
     83617 ±  4%     +32.3%     110632 ±  8%  softirqs.CPU76.RCU
     16102 ±  8%    +398.8%      80312 ± 28%  softirqs.CPU76.SCHED
     81501 ±  3%     +37.4%     111963 ±  8%  softirqs.CPU77.RCU
     16794 ±  2%    +380.6%      80720 ± 28%  softirqs.CPU77.SCHED
     83423 ±  4%     +31.9%     109996 ±  6%  softirqs.CPU78.RCU
     15669 ±  6%    +413.2%      80411 ± 29%  softirqs.CPU78.SCHED
     82996 ±  5%     +33.9%     111140 ±  8%  softirqs.CPU79.RCU
     15504 ±  7%    +414.3%      79737 ± 28%  softirqs.CPU79.SCHED
     82892 ±  2%     +34.4%     111392 ±  7%  softirqs.CPU8.RCU
     16163 ±  8%    +382.8%      78036 ± 28%  softirqs.CPU8.SCHED
     85019 ±  4%     +31.2%     111541 ±  8%  softirqs.CPU80.RCU
     16185 ±  5%    +399.0%      80763 ± 29%  softirqs.CPU80.SCHED
     84231 ±  5%     +32.9%     111927 ±  7%  softirqs.CPU81.RCU
     15720 ±  5%    +408.8%      79983 ± 29%  softirqs.CPU81.SCHED
     84979 ±  5%     +31.3%     111570 ±  9%  softirqs.CPU82.RCU
     15307 ±  8%    +424.5%      80284 ± 28%  softirqs.CPU82.SCHED
     85474 ±  4%     +31.5%     112430 ±  8%  softirqs.CPU83.RCU
     15885 ±  9%    +404.9%      80202 ± 28%  softirqs.CPU83.SCHED
     85221 ±  3%     +27.0%     108248 ±  9%  softirqs.CPU84.RCU
     15286 ±  8%    +418.6%      79265 ± 28%  softirqs.CPU84.SCHED
     84922 ±  4%     +31.4%     111599 ±  8%  softirqs.CPU85.RCU
     16013 ±  6%    +396.8%      79559 ± 28%  softirqs.CPU85.SCHED
     84635 ±  5%     +28.5%     108760 ±  9%  softirqs.CPU86.RCU
     15451 ±  6%    +423.5%      80888 ± 29%  softirqs.CPU86.SCHED
     83811 ±  5%     +33.0%     111434 ±  8%  softirqs.CPU87.RCU
     16446 ±  4%    +385.9%      79921 ± 28%  softirqs.CPU87.SCHED
     83285 ±  6%     +34.2%     111810 ±  9%  softirqs.CPU88.RCU
     16261 ±  5%    +392.7%      80122 ± 28%  softirqs.CPU88.SCHED
     84421 ±  5%     +33.1%     112339 ±  8%  softirqs.CPU89.RCU
     16499 ±  4%    +386.8%      80325 ± 28%  softirqs.CPU89.SCHED
     82076           +35.9%     111539 ±  8%  softirqs.CPU9.RCU
     16530 ±  5%    +374.2%      78396 ± 29%  softirqs.CPU9.SCHED
     84286 ±  3%     +27.8%     107732 ± 10%  softirqs.CPU90.RCU
     15897 ±  6%    +401.9%      79783 ± 28%  softirqs.CPU90.SCHED
     85201 ±  3%     +29.8%     110581 ± 11%  softirqs.CPU91.RCU
     16102 ±  5%    +399.4%      80420 ± 27%  softirqs.CPU91.SCHED
     85122 ±  3%     +32.6%     112838 ±  8%  softirqs.CPU92.RCU
     16271 ±  4%    +394.4%      80443 ± 29%  softirqs.CPU92.SCHED
     84908 ±  5%     +31.8%     111888 ±  7%  softirqs.CPU93.RCU
     15528 ±  8%    +415.1%      79984 ± 28%  softirqs.CPU93.SCHED
     84414 ±  4%     +34.4%     113461 ±  8%  softirqs.CPU94.RCU
     16174 ±  6%    +394.7%      80018 ± 29%  softirqs.CPU94.SCHED
     83695 ±  4%     +33.5%     111697 ±  7%  softirqs.CPU95.RCU
     15609 ±  9%    +413.2%      80111 ± 29%  softirqs.CPU95.SCHED
   7993548 ±  2%     +33.2%   10647607 ±  8%  softirqs.RCU
   1547228          +389.6%    7574654 ± 28%  softirqs.SCHED
    102796 ±  2%     +21.4%     124764 ±  7%  softirqs.TIMER





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


---
0DAY/LKP+ Test Infrastructure                   Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org       Intel Corporation

Thanks,
Oliver Sang


View attachment "config-5.12.0-rc4-00024-gc76f48eb5c08" of type "text/plain" (172837 bytes)

View attachment "job-script" of type "text/plain" (8116 bytes)

View attachment "job.yaml" of type "text/plain" (5668 bytes)

View attachment "reproduce" of type "text/plain" (532 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ