lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20200701091913.GB3874@shao2-debian>
Date:   Wed, 1 Jul 2020 17:19:13 +0800
From:   kernel test robot <rong.a.chen@...el.com>
To:     peter.puhov@...aro.org
Cc:     linux-kernel@...r.kernel.org, peter.puhov@...aro.org,
        robert.foley@...aro.org, Ingo Molnar <mingo@...hat.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Juri Lelli <juri.lelli@...hat.com>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        Dietmar Eggemann <dietmar.eggemann@....com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Ben Segall <bsegall@...gle.com>, Mel Gorman <mgorman@...e.de>,
        lkp@...ts.01.org
Subject: [sched/fair] 0b9730e694: vm-scalability.throughput 7.7% improvement

Greeting,

FYI, we noticed a 7.7% improvement of vm-scalability.throughput due to commit:


commit: 0b9730e694e8c89df348e59e8d93b4985489ffee ("[PATCH] sched/fair: update_pick_idlest() Select group with lowest group_util when idle_cpus are equal")
url: https://github.com/0day-ci/linux/commits/peter-puhov-linaro-org/sched-fair-update_pick_idlest-Select-group-with-lowest-group_util-when-idle_cpus-are-equal/20200617-004854


in testcase: vm-scalability
on test machine: 104 threads Skylake with 192G memory
with following parameters:

	runtime: 300s
	size: 8T
	test: anon-w-seq-mt
	cpufreq_governor: performance
	ucode: 0x2006906

test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us.
test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/

In addition to that, the commit also has significant impact on the following tests:

+------------------+----------------------------------------------------------------------+
| testcase: change | vm-scalability: vm-scalability.throughput -7.9% regression           |
| test machine     | 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory |
| test parameters  | cpufreq_governor=performance                                         |
|                  | runtime=300s                                                         |
|                  | size=512G                                                            |
|                  | test=anon-w-rand-hugetlb                                             |
|                  | ucode=0x5002f01                                                      |
+------------------+----------------------------------------------------------------------+




Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase/ucode:
  gcc-9/performance/x86_64-rhel-7.6/debian-x86_64-20191114.cgz/300s/8T/lkp-skl-fpga01/anon-w-seq-mt/vm-scalability/0x2006906

commit: 
  87e867b426 ("sched/pelt: Cleanup PELT divider")
  0b9730e694 ("sched/fair: update_pick_idlest() Select group with lowest group_util when idle_cpus are equal")

87e867b4269f29da 0b9730e694e8c89df348e59e8d9 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
      0.01           +11.9%       0.01 ±  5%  vm-scalability.free_time
    538577            -1.8%     529143        vm-scalability.median
      4.26 ± 20%      +4.3        8.56 ±  8%  vm-scalability.median_stddev%
      3.11 ±  4%      +6.8        9.88 ±  7%  vm-scalability.stddev%
  56815424            +7.7%   61205029        vm-scalability.throughput
    242763 ±  7%     -58.5%     100640 ±  5%  vm-scalability.time.involuntary_context_switches
  13544465            -5.8%   12752486        vm-scalability.time.minor_page_faults
      8173           -18.6%       6655        vm-scalability.time.percent_of_cpu_this_job_got
     10998           -18.4%       8977        vm-scalability.time.system_time
     13712           -18.6%      11155        vm-scalability.time.user_time
    629591 ±  2%     -19.3%     508253        vm-scalability.time.voluntary_context_switches
 1.463e+10            -5.9%  1.378e+10        vm-scalability.workload
 1.219e+08 ±166%     -97.6%    2921999 ±  9%  cpuidle.C1.time
   7655935 ± 32%     +90.6%   14594155 ± 20%  cpuidle.C1E.usage
     47783 ± 37%     +78.3%      85182 ± 11%  cpuidle.POLL.usage
     19.83 ±  3%     +14.7       34.56        mpstat.cpu.all.idle%
      0.00 ± 11%      -0.0        0.00 ± 16%  mpstat.cpu.all.soft%
     36.14            -6.5       29.67        mpstat.cpu.all.sys%
     44.02            -8.2       35.78        mpstat.cpu.all.usr%
     20.00 ±  3%     +72.5%      34.50        vmstat.cpu.id
     43.50           -19.5%      35.00        vmstat.cpu.us
     85.25           -21.1%      67.25        vmstat.procs.r
     10286 ±  4%     -22.3%       7994        vmstat.system.cs
    217561            -1.2%     214961        vmstat.system.in
    524.75 ±  5%     +28.6%     675.00 ±  9%  slabinfo.UDP.active_objs
    524.75 ±  5%     +28.6%     675.00 ±  9%  slabinfo.UDP.num_objs
      6790 ±  2%      +8.5%       7367 ±  2%  slabinfo.kmalloc-2k.active_objs
      6840 ±  2%      +8.4%       7414 ±  2%  slabinfo.kmalloc-2k.num_objs
      1068 ±  2%     +26.3%       1349 ±  5%  slabinfo.task_group.active_objs
      1068 ±  2%     +26.3%       1349 ±  5%  slabinfo.task_group.num_objs
  42637428           -23.8%   32497308        meminfo.Active
  42637045           -23.8%   32496919        meminfo.Active(anon)
  34573140           -22.5%   26785639        meminfo.AnonHugePages
  42547114           -23.9%   32388157        meminfo.AnonPages
  85538597           -18.4%   69791362        meminfo.Committed_AS
  45201309           -22.5%   35025896        meminfo.Memused
     79011           -22.1%      61526        meminfo.PageTables
  20748538 ±  3%     -30.0%   14525085 ±  9%  numa-meminfo.node0.Active
  20748299 ±  3%     -30.0%   14524731 ±  9%  numa-meminfo.node0.Active(anon)
  16858671 ±  3%     -29.3%   11925466 ± 10%  numa-meminfo.node0.AnonHugePages
  20716928 ±  3%     -30.1%   14486516 ±  9%  numa-meminfo.node0.AnonPages
  21996436 ±  3%     -28.4%   15759062 ±  9%  numa-meminfo.node0.MemUsed
     38539 ±  2%     -26.5%      28330 ±  8%  numa-meminfo.node0.PageTables
  22305061 ±  5%     -17.2%   18479482 ± 10%  numa-meminfo.node1.Active
  22304917 ±  5%     -17.2%   18479448 ± 10%  numa-meminfo.node1.Active(anon)
  18167081 ±  5%     -16.4%   15196131 ±  9%  numa-meminfo.node1.AnonHugePages
  22247308 ±  5%     -17.3%   18409202 ± 10%  numa-meminfo.node1.AnonPages
  23623142 ±  4%     -16.3%   19775516 ±  9%  numa-meminfo.node1.MemUsed
     41615 ±  4%     -17.0%      34527 ±  8%  numa-meminfo.node1.PageTables
   5250950 ±  3%     -32.0%    3568295 ±  8%  numa-vmstat.node0.nr_active_anon
   5242969 ±  3%     -32.2%    3556656 ±  8%  numa-vmstat.node0.nr_anon_pages
      8253 ±  4%     -30.6%       5726 ±  9%  numa-vmstat.node0.nr_anon_transparent_hugepages
      9616 ±  3%     -27.7%       6948 ±  8%  numa-vmstat.node0.nr_page_table_pages
   5249215 ±  3%     -32.0%    3567541 ±  8%  numa-vmstat.node0.nr_zone_active_anon
   5655438 ±  4%      -9.6%    5111268 ±  2%  numa-vmstat.node0.numa_hit
   5483453 ±  4%      -9.9%    4942508 ±  2%  numa-vmstat.node0.numa_local
   5608225 ±  5%     -20.0%    4486804 ± 10%  numa-vmstat.node1.nr_active_anon
   5594460 ±  5%     -20.1%    4470975 ± 10%  numa-vmstat.node1.nr_anon_pages
      8890 ±  5%     -18.6%       7237 ± 10%  numa-vmstat.node1.nr_anon_transparent_hugepages
     10446 ±  4%     -19.5%       8404 ±  9%  numa-vmstat.node1.nr_page_table_pages
   5606844 ±  5%     -20.0%    4486341 ± 10%  numa-vmstat.node1.nr_zone_active_anon
  10736731           -24.1%    8149870        proc-vmstat.nr_active_anon
  10714236           -24.2%    8119391        proc-vmstat.nr_anon_pages
     17081           -23.3%      13103        proc-vmstat.nr_anon_transparent_hugepages
    464.50 ±  2%      +7.1%     497.25        proc-vmstat.nr_dirtied
   3751388            +6.9%    4011879        proc-vmstat.nr_dirty_background_threshold
   7511950            +6.9%    8033567        proc-vmstat.nr_dirty_threshold
  37803600            +6.9%   40399954        proc-vmstat.nr_free_pages
    530.25            +2.5%     543.75        proc-vmstat.nr_inactive_file
     20048           -22.6%      15522        proc-vmstat.nr_page_table_pages
     44177            +1.0%      44603        proc-vmstat.nr_slab_unreclaimable
    430.50 ±  2%      +6.1%     456.75        proc-vmstat.nr_written
  10736723           -24.1%    8149865        proc-vmstat.nr_zone_active_anon
    530.25            +2.5%     543.75        proc-vmstat.nr_zone_inactive_file
      3830 ± 86%    +135.6%       9022 ± 23%  proc-vmstat.numa_hint_faults_local
  20727730            -5.5%   19581418        proc-vmstat.numa_hit
     17667           -13.9%      15209 ±  7%  proc-vmstat.numa_huge_pte_updates
  20693962            -5.5%   19547594        proc-vmstat.numa_local
   9108038           -13.9%    7845548 ±  7%  proc-vmstat.numa_pte_updates
     14037 ±  2%     -12.0%      12353 ±  2%  proc-vmstat.pgactivate
  3.26e+09            -5.9%  3.069e+09        proc-vmstat.pgalloc_normal
  14355270            -5.4%   13580863        proc-vmstat.pgfault
 3.257e+09            -5.8%  3.068e+09        proc-vmstat.pgfree
   6338210            -5.9%    5966713        proc-vmstat.thp_fault_alloc
    109340 ± 10%     -24.6%      82456 ±  2%  sched_debug.cfs_rq:/.exec_clock.min
      2092 ± 19%    +411.2%      10698 ± 28%  sched_debug.cfs_rq:/.exec_clock.stddev
     18497 ±  3%     +28.1%      23703 ±  4%  sched_debug.cfs_rq:/.load.avg
    872088 ± 13%     +20.9%    1054326 ± 11%  sched_debug.cfs_rq:/.load.max
      5167 ± 23%     -75.5%       1267 ±112%  sched_debug.cfs_rq:/.load.min
     89568 ± 10%     +32.1%     118317 ± 11%  sched_debug.cfs_rq:/.load.stddev
      3.71 ± 22%     -75.3%       0.92 ± 77%  sched_debug.cfs_rq:/.load_avg.min
  11899740 ±  9%     -32.1%    8081747        sched_debug.cfs_rq:/.min_vruntime.avg
  12268244 ±  9%     -26.1%    9072332 ±  2%  sched_debug.cfs_rq:/.min_vruntime.max
  11092219 ± 11%     -39.5%    6713734 ±  3%  sched_debug.cfs_rq:/.min_vruntime.min
    250035 ±  9%    +216.5%     791257 ± 28%  sched_debug.cfs_rq:/.min_vruntime.stddev
      0.73 ±  5%     -37.6%       0.45 ± 23%  sched_debug.cfs_rq:/.nr_running.avg
      0.53 ± 23%     -76.2%       0.12 ±110%  sched_debug.cfs_rq:/.nr_running.min
      0.19 ± 17%     +77.6%       0.33 ±  6%  sched_debug.cfs_rq:/.nr_running.stddev
     42.00 ± 22%     -71.2%      12.10 ± 15%  sched_debug.cfs_rq:/.nr_spread_over.avg
    251.91 ± 24%     -50.0%     125.96 ± 31%  sched_debug.cfs_rq:/.nr_spread_over.max
     60.77 ± 21%     -67.4%      19.81 ± 26%  sched_debug.cfs_rq:/.nr_spread_over.stddev
     63.48 ±129%    +301.6%     254.96 ± 33%  sched_debug.cfs_rq:/.removed.load_avg.max
     63.52 ±129%    +301.4%     254.96 ± 33%  sched_debug.cfs_rq:/.removed.runnable_avg.max
    894.04 ± 10%     -37.2%     561.73 ± 19%  sched_debug.cfs_rq:/.runnable_avg.avg
      2625 ± 35%     -41.9%       1524 ± 10%  sched_debug.cfs_rq:/.runnable_avg.max
    459.19 ± 23%     -87.4%      57.88 ±114%  sched_debug.cfs_rq:/.runnable_avg.min
    104083 ±151%    +620.3%     749680 ± 48%  sched_debug.cfs_rq:/.spread0.avg
    472076 ± 35%    +268.7%    1740379 ± 29%  sched_debug.cfs_rq:/.spread0.max
    250006 ±  9%    +216.4%     791111 ± 28%  sched_debug.cfs_rq:/.spread0.stddev
    718.36 ±  8%     -35.1%     466.32 ± 23%  sched_debug.cfs_rq:/.util_avg.avg
      1335 ± 10%     -17.4%       1103 ±  3%  sched_debug.cfs_rq:/.util_avg.max
    295.38 ± 26%     -89.1%      32.25 ± 83%  sched_debug.cfs_rq:/.util_avg.min
    169.25 ± 11%     +60.7%     272.04 ±  8%  sched_debug.cfs_rq:/.util_avg.stddev
      0.53 ± 23%     -76.2%       0.12 ±110%  sched_debug.cfs_rq:/.util_est_enqueued.min
    188416 ± 13%     -17.1%     156147 ±  7%  sched_debug.cpu.avg_idle.stddev
     18.94 ± 26%     -66.6%       6.33 ± 31%  sched_debug.cpu.clock.stddev
     18.94 ± 26%     -66.6%       6.33 ± 31%  sched_debug.cpu.clock_task.stddev
     10205 ±  5%     -43.0%       5814 ± 23%  sched_debug.cpu.curr->pid.avg
    502.67 ± 36%     -70.5%     148.21 ±127%  sched_debug.cpu.curr->pid.min
      2734 ± 32%     +47.9%       4043 ±  9%  sched_debug.cpu.curr->pid.stddev
      0.75 ±  8%     -42.2%       0.43 ± 24%  sched_debug.cpu.nr_running.avg
      2.27 ± 34%     -41.4%       1.33 ± 15%  sched_debug.cpu.nr_running.max
      0.53 ± 23%     -76.2%       0.12 ±110%  sched_debug.cpu.nr_running.min
    157.77 ± 33%     -67.4%      51.50 ± 28%  sched_debug.cpu.nr_uninterruptible.max
   -102.27           -54.7%     -46.33        sched_debug.cpu.nr_uninterruptible.min
     31.78 ± 14%     -38.6%      19.50 ± 19%  sched_debug.cpu.nr_uninterruptible.stddev
      4080 ± 12%     +19.6%       4882 ±  2%  sched_debug.cpu.sched_goidle.avg
     11397 ± 12%     +52.1%      17336 ±  6%  sched_debug.cpu.sched_goidle.max
      1460 ± 18%     +87.1%       2731 ±  7%  sched_debug.cpu.sched_goidle.stddev
      2074 ± 16%     -53.7%     960.45 ± 11%  sched_debug.cpu.ttwu_local.avg
     18659 ± 25%     -49.8%       9357 ± 53%  sched_debug.cpu.ttwu_local.max
      2942 ± 16%     -64.8%       1036 ± 51%  sched_debug.cpu.ttwu_local.stddev
 5.038e+10            -5.9%   4.74e+10        perf-stat.i.branch-instructions
      0.04 ±  2%      +0.0        0.04 ±  9%  perf-stat.i.branch-miss-rate%
     73.81            -6.2       67.60        perf-stat.i.cache-miss-rate%
 5.959e+08           -13.0%  5.186e+08        perf-stat.i.cache-misses
 7.992e+08            -5.9%  7.521e+08        perf-stat.i.cache-references
     10301 ±  4%     -22.6%       7970        perf-stat.i.context-switches
      1.39           -12.0%       1.23        perf-stat.i.cpi
 2.211e+11           -16.8%  1.839e+11        perf-stat.i.cpu-cycles
    316.70 ±  2%     -13.5%     273.91        perf-stat.i.cpu-migrations
    481094 ±  3%      -9.2%     436689 ±  3%  perf-stat.i.dTLB-load-misses
 3.953e+10            -5.8%  3.725e+10        perf-stat.i.dTLB-loads
      0.00            -0.0        0.00        perf-stat.i.dTLB-store-miss-rate%
    378091           -11.8%     333435        perf-stat.i.dTLB-store-misses
 1.152e+10            -5.8%  1.085e+10        perf-stat.i.dTLB-stores
     28.57 ±  3%     -12.0       16.59 ±  4%  perf-stat.i.iTLB-load-miss-rate%
    268142           -23.3%     205627 ±  4%  perf-stat.i.iTLB-load-misses
    804233 ±  2%     +47.1%    1183170        perf-stat.i.iTLB-loads
 1.577e+11            -5.7%  1.487e+11        perf-stat.i.instructions
    615208 ±  2%     +24.2%     764150 ±  3%  perf-stat.i.instructions-per-iTLB-miss
      0.72           +14.8%       0.83        perf-stat.i.ipc
      2.12           -17.0%       1.76        perf-stat.i.metric.GHz
      1.02            -4.9%       0.97 ±  2%  perf-stat.i.metric.K/sec
    980.01            -6.0%     921.69        perf-stat.i.metric.M/sec
     47481            -5.9%      44680        perf-stat.i.minor-faults
  75840108 ±  2%      -4.8%   72212836        perf-stat.i.node-stores
     47481            -5.9%      44681        perf-stat.i.page-faults
      0.02            +0.0        0.02        perf-stat.overall.branch-miss-rate%
     74.57            -5.6       68.93        perf-stat.overall.cache-miss-rate%
      1.40           -11.8%       1.24        perf-stat.overall.cpi
    370.98            -4.4%     354.58        perf-stat.overall.cycles-between-cache-misses
      0.00            -0.0        0.00        perf-stat.overall.dTLB-store-miss-rate%
     25.03 ±  2%     -10.2       14.80 ±  4%  perf-stat.overall.iTLB-load-miss-rate%
    588828           +22.8%     722834 ±  3%  perf-stat.overall.instructions-per-iTLB-miss
      0.71           +13.4%       0.81        perf-stat.overall.ipc
 5.007e+10            -5.9%  4.709e+10        perf-stat.ps.branch-instructions
 5.922e+08           -13.0%  5.151e+08        perf-stat.ps.cache-misses
 7.942e+08            -5.9%  7.473e+08        perf-stat.ps.cache-references
     10230 ±  4%     -22.4%       7939        perf-stat.ps.context-switches
 2.197e+11           -16.9%  1.826e+11        perf-stat.ps.cpu-cycles
    314.22 ±  2%     -13.5%     271.91        perf-stat.ps.cpu-migrations
    477788 ±  3%      -9.0%     434647 ±  3%  perf-stat.ps.dTLB-load-misses
 3.928e+10            -5.8%    3.7e+10        perf-stat.ps.dTLB-loads
    375508           -11.6%     331801        perf-stat.ps.dTLB-store-misses
 1.144e+10            -5.8%  1.078e+10        perf-stat.ps.dTLB-stores
    266199 ±  2%     -23.1%     204665 ±  4%  perf-stat.ps.iTLB-load-misses
    797608 ±  2%     +47.7%    1178072        perf-stat.ps.iTLB-loads
 1.567e+11            -5.8%  1.477e+11        perf-stat.ps.instructions
     47130            -5.6%      44470        perf-stat.ps.minor-faults
  75363222 ±  2%      -4.8%   71710440        perf-stat.ps.node-stores
     47130            -5.6%      44470        perf-stat.ps.page-faults
 4.747e+13            -5.4%  4.491e+13        perf-stat.total.instructions
     78.02 ±  2%      -3.5       74.52 ±  2%  perf-profile.calltrace.cycles-pp.do_access
      0.64            -0.0        0.60        perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.clear_page_erms.clear_subpage.clear_huge_page.do_huge_pmd_anonymous_page
      0.85 ± 12%      +0.1        0.99 ±  4%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write._fini
      0.85 ± 12%      +0.1        0.99 ±  4%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write._fini
      0.85 ± 12%      +0.1        0.99 ±  4%  perf-profile.calltrace.cycles-pp.write._fini
      0.85 ± 12%      +0.1        0.99 ±  4%  perf-profile.calltrace.cycles-pp._fini
      0.84 ± 12%      +0.1        0.99 ±  4%  perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.devkmsg_emit.devkmsg_write.cold.new_sync_write
      0.63 ± 14%      +0.1        0.78 ±  5%  perf-profile.calltrace.cycles-pp.serial8250_console_write.console_unlock.vprintk_emit.devkmsg_emit.devkmsg_write.cold
      0.84 ± 12%      +0.1        0.99 ±  4%  perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write._fini
      0.84 ± 12%      +0.1        0.99 ±  4%  perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
      0.84 ± 12%      +0.1        0.99 ±  4%  perf-profile.calltrace.cycles-pp.vprintk_emit.devkmsg_emit.devkmsg_write.cold.new_sync_write.vfs_write
      0.84 ± 12%      +0.1        0.99 ±  4%  perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.84 ± 12%      +0.1        0.99 ±  4%  perf-profile.calltrace.cycles-pp.devkmsg_write.cold.new_sync_write.vfs_write.ksys_write.do_syscall_64
      0.84 ± 12%      +0.1        0.99 ±  4%  perf-profile.calltrace.cycles-pp.devkmsg_emit.devkmsg_write.cold.new_sync_write.vfs_write.ksys_write
      0.45 ± 59%      +0.3        0.71 ±  5%  perf-profile.calltrace.cycles-pp.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit.devkmsg_emit
      0.29 ±100%      +0.3        0.62 ±  5%  perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock
      0.29 ±100%      +0.3        0.62 ±  5%  perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit
      0.14 ±173%      +0.4        0.56 ±  5%  perf-profile.calltrace.cycles-pp.io_serial_in.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write
      1.61 ± 59%      +4.6        6.25 ± 10%  perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
      1.72 ± 56%      +4.7        6.43 ± 10%  perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
      1.73 ± 56%      +4.7        6.45 ± 10%  perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
      1.78 ± 55%      +4.8        6.53 ± 10%  perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
      1.78 ± 55%      +4.8        6.53 ± 10%  perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
      1.78 ± 55%      +4.8        6.53 ± 10%  perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
      1.79 ± 55%      +4.8        6.64 ±  9%  perf-profile.calltrace.cycles-pp.secondary_startup_64
     66.80 ±  2%      -2.5       64.31 ±  2%  perf-profile.children.cycles-pp.do_access
      1.98 ±  5%      -0.3        1.70 ±  4%  perf-profile.children.cycles-pp.___might_sleep
      0.14 ± 10%      -0.0        0.11 ±  4%  perf-profile.children.cycles-pp.prepare_exit_to_usermode
      0.12 ± 12%      -0.0        0.10 ±  9%  perf-profile.children.cycles-pp.__softirqentry_text_start
      0.07 ± 26%      +0.0        0.10 ±  4%  perf-profile.children.cycles-pp.io_serial_out
      0.11 ± 13%      +0.0        0.16 ±  2%  perf-profile.children.cycles-pp.__free_pages_ok
      0.14 ± 10%      +0.1        0.20 ±  4%  perf-profile.children.cycles-pp.release_pages
      0.13 ± 24%      +0.1        0.19 ±  8%  perf-profile.children.cycles-pp.munmap
      0.13 ± 13%      +0.1        0.20 ±  2%  perf-profile.children.cycles-pp.tlb_finish_mmu
      0.13 ± 13%      +0.1        0.20 ±  2%  perf-profile.children.cycles-pp.tlb_flush_mmu
      0.15 ± 11%      +0.1        0.23 ±  2%  perf-profile.children.cycles-pp.__do_munmap
      0.15 ± 13%      +0.1        0.23        perf-profile.children.cycles-pp.unmap_region
      0.15 ± 11%      +0.1        0.24        perf-profile.children.cycles-pp.__vm_munmap
      0.15 ± 11%      +0.1        0.24        perf-profile.children.cycles-pp.__x64_sys_munmap
      0.27 ±  7%      +0.1        0.36 ±  5%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      0.25 ± 10%      +0.1        0.34 ±  4%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      0.01 ±173%      +0.1        0.11 ± 34%  perf-profile.children.cycles-pp.start_kernel
      0.53 ±  9%      +0.1        0.63 ±  6%  perf-profile.children.cycles-pp.serial8250_console_putchar
      0.57 ± 10%      +0.1        0.69 ±  5%  perf-profile.children.cycles-pp.wait_for_xmitr
      0.52 ± 12%      +0.1        0.64 ±  5%  perf-profile.children.cycles-pp.io_serial_in
      0.60 ± 11%      +0.1        0.72 ±  6%  perf-profile.children.cycles-pp.uart_console_write
      0.85 ± 12%      +0.1        0.99 ±  4%  perf-profile.children.cycles-pp._fini
      0.85 ± 12%      +0.1        0.99 ±  4%  perf-profile.children.cycles-pp.devkmsg_write.cold
      0.85 ± 12%      +0.1        0.99 ±  4%  perf-profile.children.cycles-pp.devkmsg_emit
      0.86 ± 11%      +0.1        1.01 ±  4%  perf-profile.children.cycles-pp.console_unlock
      0.65 ± 12%      +0.1        0.79 ±  5%  perf-profile.children.cycles-pp.serial8250_console_write
      0.87 ± 11%      +0.2        1.02 ±  4%  perf-profile.children.cycles-pp.vprintk_emit
      1.18 ± 10%      +0.2        1.37 ±  3%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      1.18 ± 10%      +0.2        1.37 ±  3%  perf-profile.children.cycles-pp.do_syscall_64
      0.02 ±173%      +0.7        0.72 ±112%  perf-profile.children.cycles-pp.osq_lock
      0.02 ±173%      +0.7        0.76 ±112%  perf-profile.children.cycles-pp.rwsem_optimistic_spin
      0.03 ±173%      +0.7        0.77 ±109%  perf-profile.children.cycles-pp.rwsem_down_read_slowpath
      1.63 ± 60%      +4.7        6.36 ± 10%  perf-profile.children.cycles-pp.intel_idle
      1.78 ± 55%      +4.8        6.53 ± 10%  perf-profile.children.cycles-pp.start_secondary
      1.75 ± 57%      +4.8        6.56 ±  9%  perf-profile.children.cycles-pp.cpuidle_enter
      1.75 ± 57%      +4.8        6.56 ±  9%  perf-profile.children.cycles-pp.cpuidle_enter_state
      1.79 ± 55%      +4.8        6.64 ±  9%  perf-profile.children.cycles-pp.do_idle
      1.79 ± 55%      +4.8        6.64 ±  9%  perf-profile.children.cycles-pp.secondary_startup_64
      1.79 ± 55%      +4.8        6.64 ±  9%  perf-profile.children.cycles-pp.cpu_startup_entry
      1.69 ±  5%      -0.2        1.46 ±  4%  perf-profile.self.cycles-pp.___might_sleep
      0.54 ±  4%      -0.1        0.46 ±  5%  perf-profile.self.cycles-pp.rmqueue
      0.10 ± 12%      -0.0        0.08 ±  5%  perf-profile.self.cycles-pp.update_curr
      0.07 ± 26%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.io_serial_out
      0.10 ± 15%      +0.0        0.15 ±  2%  perf-profile.self.cycles-pp.__free_pages_ok
      0.25 ± 10%      +0.1        0.34 ±  4%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
      0.51 ± 13%      +0.1        0.64 ±  5%  perf-profile.self.cycles-pp.io_serial_in
      0.02 ±173%      +0.7        0.72 ±112%  perf-profile.self.cycles-pp.osq_lock
      1.63 ± 60%      +4.7        6.36 ± 10%  perf-profile.self.cycles-pp.intel_idle
     24783 ±  6%     -39.4%      15018 ±  4%  softirqs.CPU0.RCU
     14460 ±  6%     +38.0%      19955 ±  4%  softirqs.CPU0.SCHED
     22296 ±  4%     -35.2%      14448 ±  7%  softirqs.CPU1.RCU
     13585 ±  3%     +44.3%      19605 ±  8%  softirqs.CPU1.SCHED
     23996 ±  7%     -42.3%      13839 ±  5%  softirqs.CPU10.RCU
     12300 ±  2%     +46.1%      17970 ±  9%  softirqs.CPU10.SCHED
     22392 ±  7%     -40.5%      13332 ±  6%  softirqs.CPU100.RCU
     12060 ±  3%     +22.1%      14731 ±  3%  softirqs.CPU100.SCHED
     22300 ±  7%     -39.2%      13558 ±  5%  softirqs.CPU101.RCU
     12071 ±  2%     +31.1%      15826 ±  9%  softirqs.CPU101.SCHED
     22418 ±  7%     -40.3%      13382 ±  5%  softirqs.CPU102.RCU
     12155 ±  3%     +31.8%      16015 ± 11%  softirqs.CPU102.SCHED
     22475 ±  7%     -40.4%      13390 ±  5%  softirqs.CPU103.RCU
     12115           +20.4%      14584 ± 12%  softirqs.CPU103.SCHED
     23395 ±  8%     -40.5%      13917 ±  5%  softirqs.CPU11.RCU
     11880 ±  4%     +59.6%      18961 ± 11%  softirqs.CPU11.SCHED
     23634 ±  8%     -40.8%      13982 ±  6%  softirqs.CPU12.RCU
     11937 ±  5%     +59.8%      19081 ±  5%  softirqs.CPU12.SCHED
     23617 ±  8%     -41.4%      13838 ±  5%  softirqs.CPU13.RCU
     11641 ±  6%     +59.3%      18543 ±  5%  softirqs.CPU13.SCHED
     23654 ±  8%     -40.7%      14023 ±  4%  softirqs.CPU14.RCU
     12132           +44.2%      17495 ±  7%  softirqs.CPU14.SCHED
     24073 ±  9%     -41.5%      14088 ±  4%  softirqs.CPU15.RCU
     11844 ±  4%     +58.9%      18817 ±  6%  softirqs.CPU15.SCHED
     24318 ±  7%     -43.5%      13748 ±  3%  softirqs.CPU16.RCU
     11233 ±  7%     +77.0%      19886 ±  7%  softirqs.CPU16.SCHED
     24480 ±  7%     -42.5%      14069 ±  5%  softirqs.CPU17.RCU
     11689 ±  5%     +61.9%      18920 ± 12%  softirqs.CPU17.SCHED
     24552 ±  5%     -42.6%      14088 ±  6%  softirqs.CPU18.RCU
     11941 ±  5%     +59.1%      18997 ±  7%  softirqs.CPU18.SCHED
     23989 ±  9%     -41.5%      14026 ±  5%  softirqs.CPU19.RCU
     12033 ±  4%     +53.2%      18433 ±  8%  softirqs.CPU19.SCHED
     24304 ± 10%     -38.8%      14884 ±  9%  softirqs.CPU2.RCU
     12366 ±  4%     +57.5%      19473 ± 11%  softirqs.CPU2.SCHED
     24320 ± 10%     -42.0%      14097 ±  5%  softirqs.CPU20.RCU
     11599 ±  8%     +57.6%      18277 ± 10%  softirqs.CPU20.SCHED
     24150 ±  8%     -41.6%      14096 ±  4%  softirqs.CPU21.RCU
     11680 ±  5%     +63.4%      19083 ±  7%  softirqs.CPU21.SCHED
     24166 ±  8%     -42.3%      13939 ±  6%  softirqs.CPU22.RCU
     11618 ±  4%     +59.9%      18577 ±  8%  softirqs.CPU22.SCHED
     24129 ±  8%     -41.3%      14159 ±  5%  softirqs.CPU23.RCU
     11998 ±  3%     +60.7%      19277 ±  6%  softirqs.CPU23.SCHED
     23975 ±  8%     -41.4%      14058 ±  5%  softirqs.CPU24.RCU
     11973 ±  4%     +55.6%      18629 ±  7%  softirqs.CPU24.SCHED
     23862 ±  8%     -39.5%      14434 ±  5%  softirqs.CPU25.RCU
     11391 ±  6%     +69.2%      19275 ± 11%  softirqs.CPU25.SCHED
     21729 ±  9%     -44.2%      12128 ±  3%  softirqs.CPU26.RCU
     13617 ±  9%     +20.6%      16422 ±  4%  softirqs.CPU26.SCHED
     21141 ±  8%     -42.0%      12267 ±  5%  softirqs.CPU27.RCU
     11987 ±  3%     +33.8%      16034 ± 12%  softirqs.CPU27.SCHED
     22180 ±  6%     -44.7%      12257 ±  3%  softirqs.CPU28.RCU
     11774 ±  4%     +35.8%      15993 ± 10%  softirqs.CPU28.SCHED
     21072 ±  7%     -35.1%      13666 ± 18%  softirqs.CPU29.RCU
     11848 ±  6%     +30.6%      15471 ± 10%  softirqs.CPU29.SCHED
     23495 ±  8%     -39.7%      14157 ±  5%  softirqs.CPU3.RCU
     12036 ±  5%     +63.9%      19727 ±  7%  softirqs.CPU3.SCHED
     23130 ±  6%     -38.6%      14201 ±  7%  softirqs.CPU30.RCU
     12174 ±  3%     +31.8%      16041 ±  8%  softirqs.CPU30.SCHED
     23136 ±  7%     -37.7%      14423 ±  5%  softirqs.CPU31.RCU
     12262 ±  2%     +31.5%      16120 ± 12%  softirqs.CPU31.SCHED
     23472 ±  8%     -40.1%      14060 ±  5%  softirqs.CPU32.RCU
     12287 ±  2%     +31.8%      16198 ± 11%  softirqs.CPU32.SCHED
     23564 ±  7%     -40.8%      13960 ± 10%  softirqs.CPU33.RCU
     12392 ±  3%     +29.9%      16092 ±  9%  softirqs.CPU33.SCHED
     22893 ± 12%     -39.0%      13968 ±  6%  softirqs.CPU34.RCU
     12028 ±  2%     +33.9%      16111 ±  9%  softirqs.CPU34.SCHED
     23270 ±  7%     -39.2%      14139 ±  7%  softirqs.CPU35.RCU
     11869 ±  5%     +31.6%      15625 ±  5%  softirqs.CPU35.SCHED
     23159 ±  7%     -39.3%      14051 ±  6%  softirqs.CPU36.RCU
     12208 ±  2%     +32.1%      16125 ± 10%  softirqs.CPU36.SCHED
     24507 ±  9%     -41.6%      14315 ±  4%  softirqs.CPU37.RCU
     12179 ±  4%     +24.5%      15169 ±  2%  softirqs.CPU37.SCHED
     23516 ±  6%     -40.9%      13902 ±  5%  softirqs.CPU38.RCU
     12160 ±  3%     +32.8%      16151 ± 13%  softirqs.CPU38.SCHED
     23688 ±  7%     -40.6%      14064 ±  6%  softirqs.CPU39.RCU
     12139 ±  2%     +32.2%      16044 ± 11%  softirqs.CPU39.SCHED
     23739 ± 10%     -41.0%      14008 ±  6%  softirqs.CPU4.RCU
     11691 ±  3%     +59.5%      18643 ±  7%  softirqs.CPU4.SCHED
     23418 ±  7%     -40.5%      13930 ±  6%  softirqs.CPU40.RCU
     11982 ±  2%     +35.9%      16279 ± 12%  softirqs.CPU40.SCHED
     23222 ±  7%     -40.9%      13717 ±  9%  softirqs.CPU41.RCU
     12059 ±  3%     +32.8%      16017 ±  8%  softirqs.CPU41.SCHED
     22787 ±  8%     -38.2%      14076 ±  6%  softirqs.CPU42.RCU
     12076 ±  3%     +33.1%      16073 ± 15%  softirqs.CPU42.SCHED
     23099 ±  6%     -38.9%      14107 ±  5%  softirqs.CPU43.RCU
     12205 ±  3%     +29.7%      15824 ±  9%  softirqs.CPU43.SCHED
     23538 ±  6%     -40.6%      13981 ±  7%  softirqs.CPU44.RCU
     12362 ±  4%     +27.7%      15788 ±  9%  softirqs.CPU44.SCHED
     21975 ±  8%     -41.1%      12934 ±  5%  softirqs.CPU45.RCU
     12162 ±  2%     +30.4%      15859 ±  7%  softirqs.CPU45.SCHED
     21884 ±  8%     -39.7%      13193 ±  7%  softirqs.CPU46.RCU
     12047 ±  3%     +32.6%      15970 ± 11%  softirqs.CPU46.SCHED
     22283 ±  6%     -41.6%      13014 ±  7%  softirqs.CPU47.RCU
     12534 ±  7%     +27.1%      15927 ± 13%  softirqs.CPU47.SCHED
     21918 ±  7%     -40.4%      13066 ±  4%  softirqs.CPU48.RCU
     12033 ±  2%     +31.5%      15824 ±  7%  softirqs.CPU48.SCHED
     21793 ±  7%     -41.0%      12847 ±  4%  softirqs.CPU49.RCU
     12045 ±  3%     +34.8%      16233 ± 10%  softirqs.CPU49.SCHED
     23337 ±  7%     -39.8%      14038 ±  7%  softirqs.CPU5.RCU
     12008 ±  3%     +62.9%      19559 ±  9%  softirqs.CPU5.SCHED
     22065 ±  7%     -41.6%      12892 ±  4%  softirqs.CPU50.RCU
     12190 ±  2%     +31.8%      16067 ± 13%  softirqs.CPU50.SCHED
     21952 ±  8%     -41.3%      12894 ±  5%  softirqs.CPU51.RCU
     10932 ±  6%     +37.5%      15029 ±  8%  softirqs.CPU51.SCHED
     23038 ±  8%     -43.1%      13103 ±  5%  softirqs.CPU52.RCU
     11553 ±  7%     +55.7%      17982 ±  7%  softirqs.CPU52.SCHED
     22728 ±  5%     -42.1%      13151 ±  6%  softirqs.CPU53.RCU
     11702 ±  3%     +64.1%      19208 ±  8%  softirqs.CPU53.SCHED
     23652 ±  9%     -43.7%      13315 ±  5%  softirqs.CPU54.RCU
     11916 ±  3%     +46.7%      17481 ± 10%  softirqs.CPU54.SCHED
     23314 ±  7%     -42.1%      13496 ±  5%  softirqs.CPU55.RCU
     11909 ±  4%     +65.3%      19687 ±  9%  softirqs.CPU55.SCHED
     23410 ±  8%     -41.9%      13598 ±  6%  softirqs.CPU56.RCU
     11350 ±  9%     +57.3%      17850 ±  9%  softirqs.CPU56.SCHED
     23567 ±  6%     -42.8%      13485 ±  5%  softirqs.CPU57.RCU
     11618 ±  4%     +59.6%      18544 ±  9%  softirqs.CPU57.SCHED
     23485 ±  8%     -41.6%      13707 ±  5%  softirqs.CPU58.RCU
     11883 ±  2%     +62.9%      19362 ±  7%  softirqs.CPU58.SCHED
     23186 ±  8%     -42.8%      13264 ±  4%  softirqs.CPU59.RCU
     11650 ±  3%     +57.5%      18347 ± 11%  softirqs.CPU59.SCHED
     23423 ±  8%     -40.5%      13935 ±  5%  softirqs.CPU6.RCU
     11840 ±  4%     +56.1%      18481 ±  6%  softirqs.CPU6.SCHED
     23408 ±  7%     -41.8%      13618 ±  5%  softirqs.CPU60.RCU
     11389 ±  6%     +63.9%      18671 ±  5%  softirqs.CPU60.SCHED
     23640 ±  8%     -41.6%      13807 ±  4%  softirqs.CPU61.RCU
     12138 ±  3%     +50.3%      18243 ±  5%  softirqs.CPU61.SCHED
     23653 ±  9%     -42.4%      13618 ±  5%  softirqs.CPU62.RCU
     11576 ±  7%     +63.7%      18953 ±  6%  softirqs.CPU62.SCHED
     23693 ±  8%     -41.6%      13845 ±  5%  softirqs.CPU63.RCU
     11820 ±  4%     +58.8%      18770 ± 12%  softirqs.CPU63.SCHED
     24003 ±  7%     -42.8%      13723 ±  4%  softirqs.CPU64.RCU
     11040 ±  7%     +73.6%      19168 ±  7%  softirqs.CPU64.SCHED
     23497 ±  8%     -41.9%      13641 ±  4%  softirqs.CPU65.RCU
     11428 ±  6%     +57.8%      18034 ± 16%  softirqs.CPU65.SCHED
     23861 ±  7%     -41.5%      13954 ±  6%  softirqs.CPU66.RCU
     12081 ±  4%     +49.6%      18074 ± 13%  softirqs.CPU66.SCHED
     23392 ±  8%     -41.4%      13696 ±  5%  softirqs.CPU67.RCU
     11894 ±  2%     +60.0%      19032 ±  7%  softirqs.CPU67.SCHED
     23361 ±  8%     -44.2%      13034 ±  4%  softirqs.CPU68.RCU
     11659 ±  4%     +58.8%      18519 ±  7%  softirqs.CPU68.SCHED
    101304 ±  2%     +18.8%     120355 ± 18%  softirqs.CPU68.TIMER
     23359 ±  8%     -41.0%      13776 ±  4%  softirqs.CPU69.RCU
     11613 ±  5%     +68.0%      19509 ±  6%  softirqs.CPU69.SCHED
     23524 ±  6%     -40.8%      13915 ±  5%  softirqs.CPU7.RCU
     11877 ±  2%     +56.5%      18589 ± 10%  softirqs.CPU7.SCHED
     23404 ±  8%     -40.8%      13857 ±  4%  softirqs.CPU70.RCU
     11517 ±  8%     +66.3%      19153 ±  6%  softirqs.CPU70.SCHED
     23447 ±  8%     -41.5%      13720 ±  4%  softirqs.CPU71.RCU
     11737 ±  3%     +57.0%      18422 ±  5%  softirqs.CPU71.SCHED
     23646 ±  7%     -41.8%      13752 ±  3%  softirqs.CPU72.RCU
     11853 ±  2%     +58.4%      18773 ±  6%  softirqs.CPU72.SCHED
     23309 ±  7%     -40.8%      13806 ±  6%  softirqs.CPU73.RCU
     11029 ± 10%     +67.8%      18507 ±  4%  softirqs.CPU73.SCHED
     23671 ±  9%     -42.1%      13710 ±  4%  softirqs.CPU74.RCU
     11840 ±  3%     +62.3%      19222 ±  5%  softirqs.CPU74.SCHED
     20587 ± 10%     -42.0%      11940 ±  4%  softirqs.CPU75.RCU
     11410 ±  5%     +63.7%      18680 ±  9%  softirqs.CPU75.SCHED
     20714 ± 11%     -42.0%      12014 ±  5%  softirqs.CPU76.RCU
     11749 ±  5%     +53.2%      18005 ±  9%  softirqs.CPU76.SCHED
     20628 ± 11%     -43.4%      11684 ±  3%  softirqs.CPU77.RCU
     11581 ±  2%     +62.1%      18770 ±  7%  softirqs.CPU77.SCHED
     23137 ±  7%     -42.9%      13211 ±  5%  softirqs.CPU78.RCU
     12028 ±  2%     +28.6%      15465 ±  2%  softirqs.CPU78.SCHED
     23382 ±  8%     -41.9%      13588 ±  6%  softirqs.CPU79.RCU
     11926 ±  2%     +29.5%      15448 ±  9%  softirqs.CPU79.SCHED
     23564 ±  8%     -38.5%      14491 ±  5%  softirqs.CPU8.RCU
     12048 ±  5%     +41.0%      16989 ± 11%  softirqs.CPU8.SCHED
     23147 ±  8%     -41.3%      13593 ±  4%  softirqs.CPU80.RCU
     11618 ±  3%     +29.6%      15056 ±  3%  softirqs.CPU80.SCHED
     23143 ±  7%     -41.2%      13619 ±  4%  softirqs.CPU81.RCU
     11834 ±  3%     +30.2%      15410 ± 11%  softirqs.CPU81.SCHED
     25618 ± 15%     -47.3%      13489 ±  5%  softirqs.CPU82.RCU
     12173 ±  2%     +28.0%      15581 ±  7%  softirqs.CPU82.SCHED
     23905 ±  8%     -33.7%      15859 ± 14%  softirqs.CPU83.RCU
     11861 ±  2%     +35.0%      16013 ± 10%  softirqs.CPU83.SCHED
     23375 ±  8%     -40.6%      13878 ±  4%  softirqs.CPU84.RCU
     11894 ±  3%     +31.3%      15618 ± 12%  softirqs.CPU84.SCHED
     23267 ±  7%     -37.6%      14529 ±  9%  softirqs.CPU85.RCU
     11945 ±  2%     +30.3%      15559 ± 10%  softirqs.CPU85.SCHED
     22207 ± 15%     -38.3%      13694 ±  5%  softirqs.CPU86.RCU
     12102 ±  4%     +29.7%      15701 ± 10%  softirqs.CPU86.SCHED
     23301 ±  7%     -41.2%      13693 ±  6%  softirqs.CPU87.RCU
     11843 ±  5%     +28.7%      15238 ±  7%  softirqs.CPU87.SCHED
     23403 ±  7%     -39.9%      14063 ±  6%  softirqs.CPU88.RCU
     12155 ±  2%     +23.8%      15046 ±  3%  softirqs.CPU88.SCHED
     23389 ±  6%     -40.1%      14019 ±  7%  softirqs.CPU89.RCU
     11801 ±  5%     +31.2%      15484 ± 10%  softirqs.CPU89.SCHED
     23478 ±  8%     -40.3%      14006 ±  6%  softirqs.CPU9.RCU
     12089 ±  3%     +49.8%      18115 ±  9%  softirqs.CPU9.SCHED
     22491 ±  7%     -40.0%      13504 ±  5%  softirqs.CPU90.RCU
     11867 ±  5%     +36.2%      16166 ± 11%  softirqs.CPU90.SCHED
     22365 ±  7%     -41.0%      13204 ±  6%  softirqs.CPU91.RCU
     11943           +29.7%      15496 ± 11%  softirqs.CPU91.SCHED
     22456 ±  6%     -40.8%      13293 ±  5%  softirqs.CPU92.RCU
     11755 ±  6%     +33.7%      15715 ± 11%  softirqs.CPU92.SCHED
     22637 ±  8%     -42.3%      13052 ± 10%  softirqs.CPU93.RCU
     12258           +29.2%      15843 ±  9%  softirqs.CPU93.SCHED
     21694 ± 10%     -38.0%      13446 ±  7%  softirqs.CPU94.RCU
     11727 ±  5%     +35.6%      15901 ±  8%  softirqs.CPU94.SCHED
     22623 ±  8%     -40.4%      13487 ±  6%  softirqs.CPU95.RCU
     11591 ±  8%     +36.2%      15786 ± 10%  softirqs.CPU95.SCHED
     22778 ±  7%     -39.7%      13737 ±  7%  softirqs.CPU96.RCU
     11996 ±  2%     +33.9%      16059 ± 11%  softirqs.CPU96.SCHED
     22540 ±  8%     -39.5%      13633 ±  7%  softirqs.CPU97.RCU
     12054 ±  3%     +35.3%      16311 ±  8%  softirqs.CPU97.SCHED
     22033 ±  7%     -39.4%      13357 ±  6%  softirqs.CPU98.RCU
     11664 ±  5%     +33.6%      15587 ±  8%  softirqs.CPU98.SCHED
     21947 ±  6%     -39.4%      13290 ±  7%  softirqs.CPU99.RCU
     11548 ±  4%     +27.0%      14665 ±  3%  softirqs.CPU99.SCHED
   2406782 ±  7%     -40.9%    1422634 ±  5%  softirqs.RCU
   1241423           +44.4%    1793178 ±  2%  softirqs.SCHED
   2606651           -33.7%    1729139 ±  2%  interrupts.CAL:Function_call_interrupts
     24001 ± 10%     -29.2%      17001 ± 14%  interrupts.CPU0.CAL:Function_call_interrupts
      1294 ± 17%     -30.9%     895.00 ± 10%  interrupts.CPU0.RES:Rescheduling_interrupts
     22837 ± 12%     -35.8%      14672 ±  5%  interrupts.CPU0.TLB:TLB_shootdowns
     24970 ± 10%     -38.0%      15470 ±  5%  interrupts.CPU1.CAL:Function_call_interrupts
      5304 ±  8%     -29.7%       3727 ± 16%  interrupts.CPU1.NMI:Non-maskable_interrupts
      5304 ±  8%     -29.7%       3727 ± 16%  interrupts.CPU1.PMI:Performance_monitoring_interrupts
     23793 ±  9%     -37.0%      14981 ±  5%  interrupts.CPU1.TLB:TLB_shootdowns
     24945 ±  7%     -31.6%      17071 ±  5%  interrupts.CPU10.CAL:Function_call_interrupts
     24160 ±  9%     -31.9%      16456 ±  5%  interrupts.CPU10.TLB:TLB_shootdowns
     26539 ±  7%     -35.3%      17177 ±  6%  interrupts.CPU100.CAL:Function_call_interrupts
     26164 ±  8%     -35.3%      16921 ±  7%  interrupts.CPU100.TLB:TLB_shootdowns
     25753 ±  7%     -32.2%      17454 ±  5%  interrupts.CPU101.CAL:Function_call_interrupts
     25432 ±  8%     -33.8%      16844 ±  5%  interrupts.CPU101.TLB:TLB_shootdowns
     27130 ± 11%     -38.4%      16704 ±  6%  interrupts.CPU102.CAL:Function_call_interrupts
      2191 ± 67%     -89.7%     226.25 ± 39%  interrupts.CPU102.RES:Rescheduling_interrupts
     26628 ± 11%     -38.1%      16482 ±  6%  interrupts.CPU102.TLB:TLB_shootdowns
     26565 ±  9%     -34.8%      17330 ±  7%  interrupts.CPU103.CAL:Function_call_interrupts
     26254 ± 10%     -37.0%      16534 ±  5%  interrupts.CPU103.TLB:TLB_shootdowns
     24169 ± 12%     -27.6%      17506 ± 12%  interrupts.CPU11.CAL:Function_call_interrupts
     23839 ± 12%     -32.5%      16102 ±  7%  interrupts.CPU11.TLB:TLB_shootdowns
     23893 ± 12%     -27.1%      17427 ±  5%  interrupts.CPU12.CAL:Function_call_interrupts
     23697 ± 12%     -29.9%      16603 ±  2%  interrupts.CPU12.TLB:TLB_shootdowns
     23622 ± 11%     -28.4%      16906 ±  7%  interrupts.CPU13.CAL:Function_call_interrupts
     23316 ± 12%     -31.3%      16028 ±  6%  interrupts.CPU13.TLB:TLB_shootdowns
     23806 ± 10%     -31.4%      16327 ±  2%  interrupts.CPU14.CAL:Function_call_interrupts
      5317 ±  9%     -42.1%       3076 ± 25%  interrupts.CPU14.NMI:Non-maskable_interrupts
      5317 ±  9%     -42.1%       3076 ± 25%  interrupts.CPU14.PMI:Performance_monitoring_interrupts
     23778 ± 11%     -32.7%      16001 ±  3%  interrupts.CPU14.TLB:TLB_shootdowns
     24616 ±  8%     -34.0%      16246 ±  3%  interrupts.CPU15.CAL:Function_call_interrupts
      5349 ±  7%     -23.8%       4076 ± 17%  interrupts.CPU15.NMI:Non-maskable_interrupts
      5349 ±  7%     -23.8%       4076 ± 17%  interrupts.CPU15.PMI:Performance_monitoring_interrupts
     23748 ± 10%     -32.7%      15984 ±  3%  interrupts.CPU15.TLB:TLB_shootdowns
     23957 ±  9%     -32.1%      16275 ±  4%  interrupts.CPU16.CAL:Function_call_interrupts
     23627 ± 10%     -34.3%      15528 ±  4%  interrupts.CPU16.TLB:TLB_shootdowns
     23923 ± 12%     -30.2%      16704 ±  3%  interrupts.CPU17.CAL:Function_call_interrupts
     23564 ± 13%     -31.2%      16223 ±  3%  interrupts.CPU17.TLB:TLB_shootdowns
     24408 ± 10%     -27.5%      17705 ±  4%  interrupts.CPU18.CAL:Function_call_interrupts
     23948 ± 12%     -31.0%      16523 ±  5%  interrupts.CPU18.TLB:TLB_shootdowns
     24397 ±  8%     -35.4%      15757 ±  6%  interrupts.CPU19.CAL:Function_call_interrupts
     23493 ± 10%     -35.2%      15217 ±  7%  interrupts.CPU19.TLB:TLB_shootdowns
     24196 ± 11%     -32.2%      16405 ±  5%  interrupts.CPU2.CAL:Function_call_interrupts
     23837 ± 11%     -32.9%      15993 ±  6%  interrupts.CPU2.TLB:TLB_shootdowns
     24461 ±  9%     -31.9%      16656 ±  4%  interrupts.CPU20.CAL:Function_call_interrupts
     23888 ± 10%     -35.2%      15476 ±  6%  interrupts.CPU20.TLB:TLB_shootdowns
     24856 ±  8%     -35.1%      16123 ±  6%  interrupts.CPU21.CAL:Function_call_interrupts
     23805 ± 11%     -34.2%      15658 ±  5%  interrupts.CPU21.TLB:TLB_shootdowns
     24069 ±  9%     -33.6%      15980 ±  4%  interrupts.CPU22.CAL:Function_call_interrupts
     23714 ± 10%     -33.8%      15697 ±  5%  interrupts.CPU22.TLB:TLB_shootdowns
     24007 ± 10%     -34.2%      15807 ±  9%  interrupts.CPU23.CAL:Function_call_interrupts
     23529 ± 12%     -35.5%      15178 ±  8%  interrupts.CPU23.TLB:TLB_shootdowns
     24680 ±  9%     -36.0%      15788 ±  5%  interrupts.CPU24.CAL:Function_call_interrupts
     24159 ± 10%     -35.6%      15557 ±  5%  interrupts.CPU24.TLB:TLB_shootdowns
     24361 ±  9%     -33.8%      16126 ±  4%  interrupts.CPU25.CAL:Function_call_interrupts
     23727 ± 11%     -33.4%      15803 ±  4%  interrupts.CPU25.TLB:TLB_shootdowns
     26819 ± 11%     -39.8%      16144 ± 13%  interrupts.CPU26.CAL:Function_call_interrupts
     26314 ± 10%     -41.7%      15352 ± 12%  interrupts.CPU26.TLB:TLB_shootdowns
     26161 ± 11%     -37.7%      16294 ±  5%  interrupts.CPU27.CAL:Function_call_interrupts
     25778 ± 13%     -38.7%      15811 ±  6%  interrupts.CPU27.TLB:TLB_shootdowns
     26169 ±  9%     -38.0%      16231 ±  5%  interrupts.CPU28.CAL:Function_call_interrupts
      1105 ± 75%     -59.0%     452.75 ± 38%  interrupts.CPU28.RES:Rescheduling_interrupts
     26027 ± 11%     -38.3%      16047 ±  6%  interrupts.CPU28.TLB:TLB_shootdowns
     25973 ±  9%     -35.4%      16783 ±  4%  interrupts.CPU29.CAL:Function_call_interrupts
     25752 ±  9%     -35.9%      16520 ±  4%  interrupts.CPU29.TLB:TLB_shootdowns
     24234 ±  9%     -33.6%      16103 ±  4%  interrupts.CPU3.CAL:Function_call_interrupts
     23495 ± 10%     -33.5%      15615 ±  2%  interrupts.CPU3.TLB:TLB_shootdowns
     26185 ±  7%     -34.9%      17057 ±  7%  interrupts.CPU30.CAL:Function_call_interrupts
     25521 ± 10%     -33.9%      16871 ±  7%  interrupts.CPU30.TLB:TLB_shootdowns
     26134 ±  9%     -33.8%      17303 ±  2%  interrupts.CPU31.CAL:Function_call_interrupts
     25715 ± 10%     -33.2%      17174 ±  2%  interrupts.CPU31.TLB:TLB_shootdowns
     26017 ±  6%     -37.8%      16183 ±  2%  interrupts.CPU32.CAL:Function_call_interrupts
     25924 ±  8%     -38.3%      15991 ±  2%  interrupts.CPU32.TLB:TLB_shootdowns
     25759 ±  8%     -35.4%      16648 ±  5%  interrupts.CPU33.CAL:Function_call_interrupts
     25721 ±  9%     -36.0%      16458 ±  5%  interrupts.CPU33.TLB:TLB_shootdowns
     26350 ± 10%     -28.1%      18942 ± 26%  interrupts.CPU34.CAL:Function_call_interrupts
     26361 ± 10%     -39.8%      15880 ±  3%  interrupts.CPU34.TLB:TLB_shootdowns
     25688 ±  9%     -26.4%      18897 ± 19%  interrupts.CPU35.CAL:Function_call_interrupts
      5730 ±  9%     -35.3%       3707 ± 34%  interrupts.CPU35.NMI:Non-maskable_interrupts
      5730 ±  9%     -35.3%       3707 ± 34%  interrupts.CPU35.PMI:Performance_monitoring_interrupts
     25465 ±  9%     -36.9%      16078 ±  7%  interrupts.CPU35.TLB:TLB_shootdowns
     26476 ±  9%     -36.5%      16818 ±  8%  interrupts.CPU36.CAL:Function_call_interrupts
     25299 ±  9%     -37.1%      15921 ±  7%  interrupts.CPU36.TLB:TLB_shootdowns
     26120 ±  8%     -35.7%      16798 ±  6%  interrupts.CPU37.CAL:Function_call_interrupts
     26122 ±  9%     -36.7%      16526 ±  7%  interrupts.CPU37.TLB:TLB_shootdowns
     25800 ±  8%     -34.7%      16856 ±  5%  interrupts.CPU38.CAL:Function_call_interrupts
     25485 ± 11%     -35.2%      16502 ±  5%  interrupts.CPU38.TLB:TLB_shootdowns
     25674 ± 10%     -36.6%      16273 ±  4%  interrupts.CPU39.CAL:Function_call_interrupts
     25434 ± 10%     -37.1%      15988 ±  5%  interrupts.CPU39.TLB:TLB_shootdowns
     23756 ± 10%     -30.5%      16499 ±  6%  interrupts.CPU4.CAL:Function_call_interrupts
      5274 ±  8%     -28.7%       3758 ± 21%  interrupts.CPU4.NMI:Non-maskable_interrupts
      5274 ±  8%     -28.7%       3758 ± 21%  interrupts.CPU4.PMI:Performance_monitoring_interrupts
     23181 ± 12%     -31.9%      15782 ±  2%  interrupts.CPU4.TLB:TLB_shootdowns
     25540 ±  7%     -36.2%      16282 ±  4%  interrupts.CPU40.CAL:Function_call_interrupts
     25341 ±  8%     -37.3%      15880 ±  5%  interrupts.CPU40.TLB:TLB_shootdowns
     25598 ±  7%     -36.3%      16306 ±  5%  interrupts.CPU41.CAL:Function_call_interrupts
     25128 ±  9%     -36.0%      16079 ±  7%  interrupts.CPU41.TLB:TLB_shootdowns
     25322 ±  9%     -33.6%      16802 ±  3%  interrupts.CPU42.CAL:Function_call_interrupts
      5805 ±  5%     -22.8%       4484 ± 25%  interrupts.CPU42.NMI:Non-maskable_interrupts
      5805 ±  5%     -22.8%       4484 ± 25%  interrupts.CPU42.PMI:Performance_monitoring_interrupts
     25311 ± 10%     -36.3%      16117 ±  6%  interrupts.CPU42.TLB:TLB_shootdowns
     25461 ±  9%     -34.5%      16688 ±  5%  interrupts.CPU43.CAL:Function_call_interrupts
     25467 ±  9%     -36.6%      16142 ±  3%  interrupts.CPU43.TLB:TLB_shootdowns
     25772 ±  7%     -36.4%      16388 ±  5%  interrupts.CPU44.CAL:Function_call_interrupts
     25648 ±  9%     -36.7%      16229 ±  6%  interrupts.CPU44.TLB:TLB_shootdowns
     26004 ±  8%     -34.5%      17032 ±  7%  interrupts.CPU45.CAL:Function_call_interrupts
     25761 ± 10%     -37.6%      16064 ±  9%  interrupts.CPU45.TLB:TLB_shootdowns
     26517 ±  9%     -38.3%      16362 ±  5%  interrupts.CPU46.CAL:Function_call_interrupts
     26327 ± 10%     -38.7%      16142 ±  6%  interrupts.CPU46.TLB:TLB_shootdowns
     26088 ±  8%     -37.4%      16319 ±  5%  interrupts.CPU47.CAL:Function_call_interrupts
     25758 ± 10%     -38.0%      15973 ±  7%  interrupts.CPU47.TLB:TLB_shootdowns
     26074 ±  9%     -36.3%      16607 ±  4%  interrupts.CPU48.CAL:Function_call_interrupts
     25765 ± 11%     -36.6%      16343 ±  4%  interrupts.CPU48.TLB:TLB_shootdowns
     26773 ± 11%     -36.7%      16942 ±  6%  interrupts.CPU49.CAL:Function_call_interrupts
     25748 ± 10%     -35.1%      16708 ±  6%  interrupts.CPU49.TLB:TLB_shootdowns
     24208 ± 12%     -31.6%      16548 ±  6%  interrupts.CPU5.CAL:Function_call_interrupts
     23384 ± 13%     -33.7%      15501 ±  7%  interrupts.CPU5.TLB:TLB_shootdowns
     26594 ±  9%     -38.1%      16472 ±  4%  interrupts.CPU50.CAL:Function_call_interrupts
     26592 ±  9%     -38.6%      16337 ±  5%  interrupts.CPU50.TLB:TLB_shootdowns
     26067 ± 10%     -37.9%      16184 ±  4%  interrupts.CPU51.CAL:Function_call_interrupts
     26036 ± 11%     -38.1%      16110 ±  4%  interrupts.CPU51.TLB:TLB_shootdowns
     25324 ±  9%     -34.4%      16614 ±  4%  interrupts.CPU52.CAL:Function_call_interrupts
     23960 ± 10%     -34.1%      15780 ±  3%  interrupts.CPU52.TLB:TLB_shootdowns
     23969 ± 15%     -32.0%      16294 ±  2%  interrupts.CPU53.CAL:Function_call_interrupts
     23230 ± 13%     -30.2%      16225 ±  2%  interrupts.CPU53.TLB:TLB_shootdowns
     24615 ± 11%     -33.8%      16298        interrupts.CPU54.CAL:Function_call_interrupts
     23770 ± 12%     -32.8%      15983        interrupts.CPU54.TLB:TLB_shootdowns
     23405 ± 12%     -28.8%      16671 ±  2%  interrupts.CPU55.CAL:Function_call_interrupts
      5218 ±  8%     -28.0%       3756 ± 17%  interrupts.CPU55.NMI:Non-maskable_interrupts
      5218 ±  8%     -28.0%       3756 ± 17%  interrupts.CPU55.PMI:Performance_monitoring_interrupts
     22815 ± 12%     -29.0%      16192 ±  3%  interrupts.CPU55.TLB:TLB_shootdowns
     23722 ± 10%     -24.2%      17984 ± 13%  interrupts.CPU56.CAL:Function_call_interrupts
     22985 ± 12%     -30.4%      16003 ±  5%  interrupts.CPU56.TLB:TLB_shootdowns
     24226 ± 10%     -34.7%      15813 ±  6%  interrupts.CPU57.CAL:Function_call_interrupts
      5293 ±  8%     -26.5%       3891 ± 24%  interrupts.CPU57.NMI:Non-maskable_interrupts
      5293 ±  8%     -26.5%       3891 ± 24%  interrupts.CPU57.PMI:Performance_monitoring_interrupts
     23393 ± 12%     -33.3%      15613 ±  5%  interrupts.CPU57.TLB:TLB_shootdowns
     24006 ± 10%     -28.6%      17134 ±  9%  interrupts.CPU58.CAL:Function_call_interrupts
      5430 ±  8%     -38.2%       3357 ± 21%  interrupts.CPU58.NMI:Non-maskable_interrupts
      5430 ±  8%     -38.2%       3357 ± 21%  interrupts.CPU58.PMI:Performance_monitoring_interrupts
     23223 ± 11%     -32.0%      15781 ±  8%  interrupts.CPU58.TLB:TLB_shootdowns
     24840 ± 15%     -31.8%      16951 ±  6%  interrupts.CPU59.CAL:Function_call_interrupts
     23203 ± 13%     -31.0%      16014 ±  7%  interrupts.CPU59.TLB:TLB_shootdowns
     24857 ±  8%     -26.7%      18211 ± 13%  interrupts.CPU6.CAL:Function_call_interrupts
     24091 ±  8%     -32.6%      16236 ±  6%  interrupts.CPU6.TLB:TLB_shootdowns
     24617 ±  8%     -35.6%      15856 ±  5%  interrupts.CPU60.CAL:Function_call_interrupts
      5369 ±  5%     -25.8%       3983 ± 19%  interrupts.CPU60.NMI:Non-maskable_interrupts
      5369 ±  5%     -25.8%       3983 ± 19%  interrupts.CPU60.PMI:Performance_monitoring_interrupts
     23658 ± 10%     -33.9%      15630 ±  5%  interrupts.CPU60.TLB:TLB_shootdowns
     24148 ± 11%     -32.7%      16245 ±  6%  interrupts.CPU61.CAL:Function_call_interrupts
     23107 ± 13%     -30.9%      15966 ±  6%  interrupts.CPU61.TLB:TLB_shootdowns
     23325 ± 11%     -27.6%      16891 ± 10%  interrupts.CPU62.CAL:Function_call_interrupts
     22813 ± 12%     -31.3%      15661 ±  6%  interrupts.CPU62.TLB:TLB_shootdowns
     24771 ± 10%     -32.3%      16772 ±  5%  interrupts.CPU63.CAL:Function_call_interrupts
     23865 ± 11%     -34.1%      15738 ±  4%  interrupts.CPU63.TLB:TLB_shootdowns
     24478 ± 10%     -31.2%      16848 ±  4%  interrupts.CPU64.CAL:Function_call_interrupts
      1760 ±106%     -80.2%     348.75 ±  6%  interrupts.CPU64.RES:Rescheduling_interrupts
     23902 ± 11%     -32.4%      16164 ±  4%  interrupts.CPU64.TLB:TLB_shootdowns
     23545 ± 11%     -34.0%      15530 ±  6%  interrupts.CPU65.CAL:Function_call_interrupts
      5257 ±  8%     -29.0%       3733 ± 23%  interrupts.CPU65.NMI:Non-maskable_interrupts
      5257 ±  8%     -29.0%       3733 ± 23%  interrupts.CPU65.PMI:Performance_monitoring_interrupts
     22799 ± 11%     -34.3%      14975 ±  6%  interrupts.CPU65.TLB:TLB_shootdowns
     23472 ± 10%     -32.9%      15750 ±  6%  interrupts.CPU66.CAL:Function_call_interrupts
     23353 ± 11%     -34.7%      15244 ±  4%  interrupts.CPU66.TLB:TLB_shootdowns
     23826 ± 11%     -34.0%      15725 ±  6%  interrupts.CPU67.CAL:Function_call_interrupts
      5536 ±  7%     -27.2%       4030 ± 29%  interrupts.CPU67.NMI:Non-maskable_interrupts
      5536 ±  7%     -27.2%       4030 ± 29%  interrupts.CPU67.PMI:Performance_monitoring_interrupts
     23164 ± 12%     -35.4%      14968 ±  8%  interrupts.CPU67.TLB:TLB_shootdowns
     23689 ± 10%     -25.3%      17687 ± 19%  interrupts.CPU68.CAL:Function_call_interrupts
     23134 ± 12%     -33.3%      15428 ±  7%  interrupts.CPU68.TLB:TLB_shootdowns
     23593 ± 12%     -31.6%      16143 ±  5%  interrupts.CPU69.CAL:Function_call_interrupts
     22725 ± 14%     -33.1%      15208 ±  4%  interrupts.CPU69.TLB:TLB_shootdowns
     25294 ±  8%     -34.6%      16544 ±  2%  interrupts.CPU7.CAL:Function_call_interrupts
     23919 ± 10%     -32.1%      16237 ±  3%  interrupts.CPU7.TLB:TLB_shootdowns
     23343 ± 12%     -28.0%      16800 ±  5%  interrupts.CPU70.CAL:Function_call_interrupts
     23114 ± 13%     -30.8%      15998 ±  6%  interrupts.CPU70.TLB:TLB_shootdowns
     23701 ±  9%     -31.3%      16293 ±  6%  interrupts.CPU71.CAL:Function_call_interrupts
     23442 ± 10%     -31.6%      16028 ±  6%  interrupts.CPU71.TLB:TLB_shootdowns
     24572 ± 10%     -36.0%      15729 ±  6%  interrupts.CPU72.CAL:Function_call_interrupts
     23959 ± 11%     -35.8%      15389 ±  6%  interrupts.CPU72.TLB:TLB_shootdowns
     23571 ± 10%     -29.4%      16653 ±  5%  interrupts.CPU73.CAL:Function_call_interrupts
     23044 ± 12%     -28.5%      16475 ±  5%  interrupts.CPU73.TLB:TLB_shootdowns
     23446 ± 11%     -31.7%      16013 ±  8%  interrupts.CPU74.CAL:Function_call_interrupts
     22939 ± 12%     -32.0%      15596 ±  8%  interrupts.CPU74.TLB:TLB_shootdowns
     24200 ± 11%     -32.0%      16449 ±  3%  interrupts.CPU75.CAL:Function_call_interrupts
      5274 ±  8%     -20.4%       4196 ± 18%  interrupts.CPU75.NMI:Non-maskable_interrupts
      5274 ±  8%     -20.4%       4196 ± 18%  interrupts.CPU75.PMI:Performance_monitoring_interrupts
     23798 ± 11%     -32.1%      16150 ±  3%  interrupts.CPU75.TLB:TLB_shootdowns
     23724 ± 11%     -27.1%      17289 ±  4%  interrupts.CPU76.CAL:Function_call_interrupts
     23417 ± 12%     -27.1%      17060 ±  5%  interrupts.CPU76.TLB:TLB_shootdowns
     23835 ± 10%     -31.9%      16224 ±  6%  interrupts.CPU77.CAL:Function_call_interrupts
     23439 ± 12%     -33.3%      15624 ±  5%  interrupts.CPU77.TLB:TLB_shootdowns
     26028 ±  6%     -35.7%      16742 ±  5%  interrupts.CPU78.CAL:Function_call_interrupts
     25322 ± 10%     -35.6%      16297 ±  3%  interrupts.CPU78.TLB:TLB_shootdowns
     25722 ±  7%     -31.5%      17617 ±  5%  interrupts.CPU79.CAL:Function_call_interrupts
     25377 ±  9%     -34.2%      16689 ±  2%  interrupts.CPU79.TLB:TLB_shootdowns
     24086 ±  8%     -32.0%      16370 ±  5%  interrupts.CPU8.CAL:Function_call_interrupts
     23679 ± 10%     -31.9%      16128 ±  5%  interrupts.CPU8.TLB:TLB_shootdowns
     26378 ±  8%     -35.1%      17117 ±  3%  interrupts.CPU80.CAL:Function_call_interrupts
     25853 ±  9%     -34.8%      16850 ±  3%  interrupts.CPU80.TLB:TLB_shootdowns
     25298 ±  8%     -33.1%      16918 ±  6%  interrupts.CPU81.CAL:Function_call_interrupts
     25002 ±  9%     -32.8%      16802 ±  8%  interrupts.CPU81.TLB:TLB_shootdowns
     24700 ±  8%     -32.8%      16607 ±  4%  interrupts.CPU82.CAL:Function_call_interrupts
     24643 ±  8%     -33.9%      16297 ±  5%  interrupts.CPU82.TLB:TLB_shootdowns
     26133 ±  6%     -37.8%      16266 ±  5%  interrupts.CPU83.CAL:Function_call_interrupts
      6087 ±  6%     -18.6%       4952 ± 13%  interrupts.CPU83.NMI:Non-maskable_interrupts
      6087 ±  6%     -18.6%       4952 ± 13%  interrupts.CPU83.PMI:Performance_monitoring_interrupts
     25696 ± 10%     -37.5%      16053 ±  5%  interrupts.CPU83.TLB:TLB_shootdowns
     25825 ±  8%     -35.8%      16567 ±  6%  interrupts.CPU84.CAL:Function_call_interrupts
     25859 ±  8%     -37.0%      16286 ±  7%  interrupts.CPU84.TLB:TLB_shootdowns
     25611 ±  8%     -35.5%      16526 ±  6%  interrupts.CPU85.CAL:Function_call_interrupts
     25292 ±  8%     -35.7%      16260 ±  7%  interrupts.CPU85.TLB:TLB_shootdowns
     25606 ± 10%     -34.9%      16657 ±  6%  interrupts.CPU86.CAL:Function_call_interrupts
     25394 ± 10%     -35.5%      16370 ±  7%  interrupts.CPU86.TLB:TLB_shootdowns
     25580 ±  6%     -37.2%      16070 ±  5%  interrupts.CPU87.CAL:Function_call_interrupts
     25419 ±  8%     -38.2%      15698 ±  7%  interrupts.CPU87.TLB:TLB_shootdowns
     25493 ±  8%     -35.7%      16396 ±  7%  interrupts.CPU88.CAL:Function_call_interrupts
     24751 ±  8%     -34.5%      16217 ±  8%  interrupts.CPU88.TLB:TLB_shootdowns
     26074 ±  4%     -34.4%      17094 ±  4%  interrupts.CPU89.CAL:Function_call_interrupts
     25342 ±  8%     -33.1%      16944 ±  5%  interrupts.CPU89.TLB:TLB_shootdowns
     24062 ± 10%     -31.6%      16466 ±  6%  interrupts.CPU9.CAL:Function_call_interrupts
     23408 ± 11%     -30.3%      16319 ±  6%  interrupts.CPU9.TLB:TLB_shootdowns
     25583 ±  8%     -35.5%      16512 ±  8%  interrupts.CPU90.CAL:Function_call_interrupts
     25215 ±  9%     -35.7%      16215 ±  9%  interrupts.CPU90.TLB:TLB_shootdowns
     25536 ±  8%     -37.1%      16073 ±  7%  interrupts.CPU91.CAL:Function_call_interrupts
     25420 ±  8%     -37.3%      15935 ±  8%  interrupts.CPU91.TLB:TLB_shootdowns
     25510 ±  5%     -35.4%      16473 ±  4%  interrupts.CPU92.CAL:Function_call_interrupts
     24921 ±  8%     -34.1%      16418 ±  5%  interrupts.CPU92.TLB:TLB_shootdowns
     25076 ± 10%     -34.4%      16461 ±  7%  interrupts.CPU93.CAL:Function_call_interrupts
     24717 ± 10%     -34.9%      16097 ±  8%  interrupts.CPU93.TLB:TLB_shootdowns
     25460 ±  8%     -34.4%      16708 ±  4%  interrupts.CPU94.CAL:Function_call_interrupts
     25217 ±  9%     -34.6%      16497 ±  5%  interrupts.CPU94.TLB:TLB_shootdowns
     26120 ±  8%     -36.0%      16709 ±  7%  interrupts.CPU95.CAL:Function_call_interrupts
     25864 ±  9%     -36.0%      16552 ±  8%  interrupts.CPU95.TLB:TLB_shootdowns
     26369 ±  4%     -36.2%      16836 ±  7%  interrupts.CPU96.CAL:Function_call_interrupts
     25684 ±  9%     -36.2%      16383 ±  8%  interrupts.CPU96.TLB:TLB_shootdowns
     26552 ±  3%     -36.3%      16913 ±  5%  interrupts.CPU97.CAL:Function_call_interrupts
     25821 ±  9%     -35.6%      16616 ±  5%  interrupts.CPU97.TLB:TLB_shootdowns
     26734 ±  9%     -37.8%      16632 ±  4%  interrupts.CPU98.CAL:Function_call_interrupts
     26525 ±  9%     -38.4%      16349 ±  4%  interrupts.CPU98.TLB:TLB_shootdowns
     26309 ±  8%     -38.2%      16255 ±  6%  interrupts.CPU99.CAL:Function_call_interrupts
     26023 ±  9%     -38.8%      15923 ±  7%  interrupts.CPU99.TLB:TLB_shootdowns
    555164 ±  5%     -12.0%     488658 ±  3%  interrupts.NMI:Non-maskable_interrupts
    555164 ±  5%     -12.0%     488658 ±  3%  interrupts.PMI:Performance_monitoring_interrupts
     92265 ± 16%     -48.1%      47888 ±  7%  interrupts.RES:Rescheduling_interrupts
   2555541           -34.6%    1670205 ±  2%  interrupts.TLB:TLB_shootdowns


                                                                                
                               vm-scalability.throughput                        
                                                                                
  6.2e+07 +-----------------------------------------------------------------+   
          |                      O            O           O  O      O  O    |   
  6.1e+07 |-+       O    O         O            O  O    O         O         |   
          |    O                      O    O         O         O            |   
          |       O                                                         |   
    6e+07 |-+                                                               |   
          |            O      O                                             |   
  5.9e+07 |-+               O                                               |   
          |  O                                                              |   
  5.8e+07 |-+                                                               |   
          |                                                                 |   
          |..                                       .+..                    |   
  5.7e+07 |-+    .+.+..      .+..       +..   +.  .+    +.  .+.+..    .+.+..|   
          |  +.+.      +.+..+    +.+.. +    ..  +.        +.      +.+.      |   
  5.6e+07 +-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                            vm-scalability.time.user_time                       
                                                                                
  14500 +-------------------------------------------------------------------+   
        |            +..+.          .+.  .+..                               |   
  14000 |-.+.+..+   +     +..  .+.+.   +.    +.          .+.  .+..+.  .+.  .|   
  13500 |.+      + +         +.                +..+.+..+.   +.      +.   +. |   
        |         +                                                         |   
  13000 |-+                                                                 |   
        |                                                                   |   
  12500 |-+                                                                 |   
        |                                                                   |   
  12000 |-+                                                                 |   
  11500 |-+                                                                 |   
        |       O O  O            O       O    O  O O          O            |   
  11000 |-+O O          O O  O       O O     O         O  O O     O O  O    |   
        |                       O                                           |   
  10500 +-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                           vm-scalability.time.system_time                      
                                                                                
  11500 +-------------------------------------------------------------------+   
        |            +..+.+..                             +           .+    |   
  11000 |-.+.+..+. ..        +..+.+..+.  .+.. .+..       + :     .+.+.  +   |   
        |.        +                    +.    +    +.+.. +  :  .+.        +..|   
        |                                              +    +.              |   
  10500 |-+                                                                 |   
        |                                                                   |   
  10000 |-+                                                                 |   
        |                                                                   |   
   9500 |-+                                                                 |   
        |         O                                                         |   
        |    O  O    O  O         O       O  O O  O    O  O       O O       |   
   9000 |-+O              O            O            O       O  O       O    |   
        |                    O  O    O                                      |   
   8500 +-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                  vm-scalability.time.percent_of_cpu_this_job_got               
                                                                                
  8600 +--------------------------------------------------------------------+   
  8400 |-+          +..+..+                                                 |   
       | .+.+..+.. +       +  .+..+.+..  .+.             +..  .+.+..  .+.   |   
  8200 |.+        +         +.         +.   +..+.+..+.. +   +.      +.   +..|   
  8000 |-+                                             +                    |   
  7800 |-+                                                                  |   
  7600 |-+                                                                  |   
       |                                                                    |   
  7400 |-+                                                                  |   
  7200 |-+                                                                  |   
  7000 |-+                                                                  |   
  6800 |-+        O                              O                          |   
       |    O  O    O  O          O    O  O O  O    O  O O     O O  O       |   
  6600 |-+O               O O       O                       O          O    |   
  6400 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                  vm-scalability.time.involuntary_context_switches              
                                                                                
  280000 +------------------------------------------------------------------+   
  260000 |-+       +..+             : :   : :    .+..+.      +.          +  |   
         |+   +.. +    :      +.    :  :  : :   +      +..  +  +    +.. : + |   
  240000 |-+ +   +     :    ..  +..+   : :   :..           +    +  :    :  +|   
  220000 |-++           +..+            +    +            +      + :   +    |   
         |                                                        +         |   
  200000 |-+                                                                |   
  180000 |-+                                                                |   
  160000 |-+                                                                |   
         |                                                                  |   
  140000 |-+                                                                |   
  120000 |-+     O O               O              O                         |   
         |            O O     O                        O          O         |   
  100000 |-+O O            O         O  O  O O  O    O    O  O O    O  O    |   
   80000 +------------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-csl-2sp6: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase/ucode:
  gcc-9/performance/x86_64-rhel-7.6/debian-x86_64-20191114.cgz/300s/512G/lkp-csl-2sp6/anon-w-rand-hugetlb/vm-scalability/0x5002f01

commit: 
  87e867b426 ("sched/pelt: Cleanup PELT divider")
  0b9730e694 ("sched/fair: update_pick_idlest() Select group with lowest group_util when idle_cpus are equal")

87e867b4269f29da 0b9730e694e8c89df348e59e8d9 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
          6:4         -120%           1:4     perf-profile.calltrace.cycles-pp.error_entry
          5:4         -109%           1:4     perf-profile.calltrace.cycles-pp.sync_regs.error_entry
          6:4         -117%           1:4     perf-profile.children.cycles-pp.error_entry
          0:4          -12%           0:4     perf-profile.self.cycles-pp.error_entry
         %stddev     %change         %stddev
             \          |                \  
      0.00 ±  9%     +27.1%       0.00 ±  6%  vm-scalability.free_time
     77440           -10.2%      69530 ±  2%  vm-scalability.median
   7580390            -7.9%    6980302 ±  2%  vm-scalability.throughput
    173.87 ±  3%     +20.9%     210.28 ±  5%  vm-scalability.time.elapsed_time
    173.87 ±  3%     +20.9%     210.28 ±  5%  vm-scalability.time.elapsed_time.max
      8729            -7.2%       8099 ±  4%  vm-scalability.time.percent_of_cpu_this_job_got
      1231 ±  4%     +22.5%       1508 ±  6%  vm-scalability.time.system_time
     13939 ±  2%     +11.2%      15494 ±  4%  vm-scalability.time.user_time
      3286            +7.8%       3542 ±  3%  vm-scalability.time.voluntary_context_switches
    790851 ±  3%     -17.6%     651546 ±  6%  meminfo.max_used_kB
      9.14 ± 14%      +6.5       15.68 ± 23%  mpstat.cpu.all.idle%
    484919 ±  5%     +19.5%     579550 ±  6%  numa-numastat.node1.local_node
    506709 ±  4%     +16.8%     592066 ±  5%  numa-numastat.node1.numa_hit
      9.75 ± 13%     +66.7%      16.25 ± 23%  vmstat.cpu.id
     82.00            -7.9%      75.50 ±  4%  vmstat.cpu.us
      6.79 ±154%     +28.3       35.12 ± 82%  perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
      6.79 ±154%     +28.4       35.16 ± 82%  perf-profile.children.cycles-pp.intel_idle
      6.79 ±154%     +28.4       35.16 ± 82%  perf-profile.self.cycles-pp.intel_idle
    116660 ± 13%    +103.1%     236933 ±  7%  numa-meminfo.node0.Active
    115715 ± 13%    +103.6%     235539 ±  7%  numa-meminfo.node0.Active(anon)
     64046 ± 40%    +131.7%     148384 ±  6%  numa-meminfo.node0.AnonHugePages
    112193 ± 13%    +107.7%     233064 ±  6%  numa-meminfo.node0.AnonPages
    166875 ±  9%     -68.3%      52918 ± 33%  numa-meminfo.node1.Active
    165953 ±  9%     -68.4%      52464 ± 33%  numa-meminfo.node1.Active(anon)
     86244 ± 32%     -83.0%      14632 ± 78%  numa-meminfo.node1.AnonHugePages
    161863 ±  9%     -75.9%      39060 ± 39%  numa-meminfo.node1.AnonPages
     28921 ± 13%    +103.6%      58883 ±  7%  numa-vmstat.node0.nr_active_anon
     28043 ± 13%    +107.8%      58265 ±  6%  numa-vmstat.node0.nr_anon_pages
     28921 ± 13%    +103.6%      58883 ±  7%  numa-vmstat.node0.nr_zone_active_anon
    829325 ± 10%     -15.3%     702160 ±  7%  numa-vmstat.node0.numa_local
     85543 ± 88%     +99.5%     170657 ±  4%  numa-vmstat.node0.numa_other
     41512 ±  9%     -68.4%      13118 ± 33%  numa-vmstat.node1.nr_active_anon
     40467 ±  9%     -75.9%       9765 ± 39%  numa-vmstat.node1.nr_anon_pages
     41512 ±  9%     -68.4%      13118 ± 33%  numa-vmstat.node1.nr_zone_active_anon
    772937 ± 11%     +21.1%     936063 ±  4%  numa-vmstat.node1.numa_local
     70404            +2.3%      72002        proc-vmstat.nr_active_anon
   3533411            +6.1%    3750236 ±  3%  proc-vmstat.nr_dirty_background_threshold
   7075463            +6.1%    7509642 ±  3%  proc-vmstat.nr_dirty_threshold
  35670935            +6.1%   37842330 ±  3%  proc-vmstat.nr_free_pages
      1747            -2.6%       1702        proc-vmstat.nr_page_table_pages
     38184            +5.0%      40094        proc-vmstat.nr_shmem
     70404            +2.3%      72002        proc-vmstat.nr_zone_active_anon
   1073926            +7.3%    1152673 ±  2%  proc-vmstat.numa_hit
   1042788            +7.5%    1121456 ±  2%  proc-vmstat.numa_local
   1046912            +9.6%    1147398 ±  3%  proc-vmstat.pgfault
      2621 ± 60%     +84.0%       4823 ± 12%  sched_debug.cfs_rq:/.exec_clock.stddev
      3.28 ± 28%     +73.5%       5.69 ± 12%  sched_debug.cfs_rq:/.nr_spread_over.avg
    573715 ± 41%     +63.1%     935910 ± 36%  sched_debug.cfs_rq:/.spread0.max
    409.84 ±  9%     +40.7%     576.84 ±  7%  sched_debug.cfs_rq:/.util_est_enqueued.avg
    961.69 ±  4%     +18.6%       1140 ± 10%  sched_debug.cfs_rq:/.util_est_enqueued.max
      0.60 ± 26%   +3648.3%      22.65 ± 40%  sched_debug.cfs_rq:/.util_est_enqueued.min
    381.58 ±  2%     -18.9%     309.54 ±  7%  sched_debug.cfs_rq:/.util_est_enqueued.stddev
      8.80 ± 36%     +65.6%      14.56 ± 29%  sched_debug.cpu.clock.stddev
      8.80 ± 36%     +65.6%      14.56 ± 29%  sched_debug.cpu.clock_task.stddev
     97.79 ± 35%     -68.3%      31.00 ± 10%  sched_debug.cpu.nr_uninterruptible.max
      1752 ± 18%     +49.7%       2623 ± 28%  sched_debug.cpu.sched_goidle.max
     35.00 ± 21%     +47.1%      51.50 ± 15%  sched_debug.cpu.sched_goidle.min
    287.52 ± 35%     +50.1%     431.68 ± 16%  sched_debug.cpu.sched_goidle.stddev
     11.29            -1.0%      11.18        perf-stat.i.MPKI
 1.672e+10 ±  3%     -17.1%  1.386e+10 ±  6%  perf-stat.i.branch-instructions
   6038973 ±  4%     -13.7%    5211681 ±  2%  perf-stat.i.branch-misses
 7.919e+08 ±  3%     -17.2%  6.556e+08 ±  6%  perf-stat.i.cache-misses
  8.13e+08 ±  3%     -17.2%  6.731e+08 ±  6%  perf-stat.i.cache-references
 2.417e+11            -7.2%  2.244e+11 ±  4%  perf-stat.i.cpu-cycles
    171.33 ±  5%     -11.5%     151.59 ±  3%  perf-stat.i.cpu-migrations
 1.987e+10 ±  3%     -17.1%  1.647e+10 ±  6%  perf-stat.i.dTLB-loads
 7.962e+09 ±  3%     -17.0%  6.605e+09 ±  6%  perf-stat.i.dTLB-stores
     88.28 ±  2%      -8.8       79.50 ±  5%  perf-stat.i.iTLB-load-miss-rate%
    238164 ± 16%     +86.6%     444313 ± 23%  perf-stat.i.iTLB-loads
  7.33e+10 ±  3%     -17.1%   6.08e+10 ±  6%  perf-stat.i.instructions
    101312 ±  3%     -20.1%      80899 ±  5%  perf-stat.i.instructions-per-iTLB-miss
      0.31            -8.4%       0.28 ±  3%  perf-stat.i.ipc
      2.52            -7.2%       2.34 ±  4%  perf-stat.i.metric.GHz
    480.61 ±  3%     -17.0%     398.76 ±  6%  perf-stat.i.metric.M/sec
      5778 ±  3%      -9.4%       5234        perf-stat.i.minor-faults
     31.34 ±  5%      +5.9       37.27 ±  5%  perf-stat.i.node-load-miss-rate%
    402344 ±  5%      +8.6%     436835 ±  2%  perf-stat.i.node-load-misses
   1389803 ±  8%     -20.5%    1105497 ±  7%  perf-stat.i.node-loads
     12.30 ± 25%     +16.6       28.86 ± 12%  perf-stat.i.node-store-miss-rate%
  77615511 ± 19%    +115.7%  1.674e+08 ±  2%  perf-stat.i.node-store-misses
 7.092e+08 ±  5%     -31.8%  4.839e+08 ±  9%  perf-stat.i.node-stores
      5778 ±  3%      -9.4%       5235        perf-stat.i.page-faults
      3.31 ±  2%     +12.2%       3.72 ±  4%  perf-stat.overall.cpi
    306.56 ±  2%     +12.4%     344.68 ±  4%  perf-stat.overall.cycles-between-cache-misses
     80.61 ±  3%     -10.6       70.04 ±  5%  perf-stat.overall.iTLB-load-miss-rate%
     73133 ±  3%     -19.0%      59242 ±  6%  perf-stat.overall.instructions-per-iTLB-miss
      0.30 ±  2%     -10.8%       0.27 ±  4%  perf-stat.overall.ipc
     22.19 ± 10%      +6.2       28.43 ±  6%  perf-stat.overall.node-load-miss-rate%
     10.04 ± 22%     +15.9       25.90 ±  8%  perf-stat.overall.node-store-miss-rate%
 1.651e+10 ±  3%     -16.9%  1.372e+10 ±  5%  perf-stat.ps.branch-instructions
   5954776 ±  3%     -13.2%    5169398 ±  2%  perf-stat.ps.branch-misses
 7.822e+08 ±  3%     -17.1%  6.489e+08 ±  5%  perf-stat.ps.cache-misses
 8.032e+08 ±  3%     -17.0%  6.662e+08 ±  5%  perf-stat.ps.cache-references
 2.396e+11            -6.8%  2.233e+11 ±  4%  perf-stat.ps.cpu-cycles
    169.26 ±  5%     -11.2%     150.27 ±  3%  perf-stat.ps.cpu-migrations
 1.963e+10 ±  3%     -16.9%   1.63e+10 ±  5%  perf-stat.ps.dTLB-loads
 7.864e+09 ±  3%     -16.9%  6.536e+09 ±  5%  perf-stat.ps.dTLB-stores
    239234 ± 16%     +84.6%     441573 ± 21%  perf-stat.ps.iTLB-loads
  7.24e+10 ±  3%     -16.9%  6.017e+10 ±  5%  perf-stat.ps.instructions
      5854            -9.3%       5309 ±  2%  perf-stat.ps.minor-faults
    404234 ±  5%      +9.0%     440545 ±  3%  perf-stat.ps.node-load-misses
   1427369 ±  9%     -21.9%    1114896 ±  8%  perf-stat.ps.node-loads
  77404942 ± 19%    +114.7%  1.662e+08 ±  3%  perf-stat.ps.node-store-misses
 6.997e+08 ±  6%     -31.6%  4.784e+08 ±  8%  perf-stat.ps.node-stores
      5854            -9.3%       5309 ±  2%  perf-stat.ps.page-faults
     62610 ±  4%     +50.6%      94283 ± 24%  softirqs.CPU1.TIMER
     60394 ±  3%     +20.2%      72598 ±  5%  softirqs.CPU10.TIMER
     61763 ±  5%     +34.0%      82736 ± 20%  softirqs.CPU11.TIMER
     61376 ±  3%     +19.5%      73373 ±  4%  softirqs.CPU12.TIMER
     61136 ±  4%     +19.9%      73315 ±  5%  softirqs.CPU13.TIMER
     60791 ±  3%     +18.8%      72243 ±  5%  softirqs.CPU14.TIMER
     60820 ±  3%     +18.4%      72036 ±  4%  softirqs.CPU15.TIMER
     60770 ±  3%     +18.6%      72067 ±  5%  softirqs.CPU16.TIMER
      3909 ± 10%     +93.7%       7574 ± 27%  softirqs.CPU17.SCHED
     61374 ±  3%     +19.0%      73048 ±  5%  softirqs.CPU17.TIMER
     61378 ±  3%     +18.0%      72401 ±  4%  softirqs.CPU18.TIMER
     60787 ±  3%     +19.5%      72656 ±  5%  softirqs.CPU19.TIMER
      3497 ± 11%    +116.3%       7564 ± 23%  softirqs.CPU20.SCHED
     61469 ±  4%     +17.7%      72350 ±  5%  softirqs.CPU21.TIMER
     61016 ±  3%     +17.5%      71676 ±  4%  softirqs.CPU22.TIMER
     60803 ±  4%     +20.1%      73009 ±  5%  softirqs.CPU23.TIMER
     62363 ±  3%     +18.7%      74038 ±  7%  softirqs.CPU24.TIMER
     62147 ±  3%     +18.9%      73919 ±  7%  softirqs.CPU25.TIMER
     62025 ±  3%     +18.6%      73534 ±  7%  softirqs.CPU26.TIMER
     61692 ±  3%     +19.8%      73915 ±  8%  softirqs.CPU27.TIMER
     63030 ±  4%     +18.3%      74562 ±  6%  softirqs.CPU28.TIMER
     62007 ±  3%     +20.0%      74390 ±  8%  softirqs.CPU29.TIMER
     61993 ±  4%     +16.7%      72335 ±  5%  softirqs.CPU3.TIMER
     61879 ±  3%     +19.0%      73643 ±  7%  softirqs.CPU30.TIMER
     62098 ±  3%     +19.3%      74059 ±  7%  softirqs.CPU31.TIMER
     61928 ±  2%     +18.7%      73511 ±  7%  softirqs.CPU32.TIMER
      3831 ±  6%     +77.5%       6802 ± 30%  softirqs.CPU33.SCHED
     61915 ±  3%     +19.1%      73752 ±  7%  softirqs.CPU33.TIMER
      4195 ± 18%     +59.8%       6705 ± 32%  softirqs.CPU34.SCHED
     61875 ±  3%     +19.8%      74096 ±  8%  softirqs.CPU34.TIMER
     61825 ±  3%     +19.1%      73620 ±  6%  softirqs.CPU35.TIMER
     62079 ±  4%     +17.9%      73195 ±  7%  softirqs.CPU36.TIMER
     63178 ±  6%     +16.4%      73528 ±  7%  softirqs.CPU37.TIMER
     62506 ±  3%     +17.2%      73282 ±  7%  softirqs.CPU38.TIMER
     61931 ±  3%     +18.1%      73113 ±  7%  softirqs.CPU39.TIMER
     62059 ±  3%     +16.7%      72406 ±  5%  softirqs.CPU4.TIMER
     62091 ±  4%     +17.2%      72780 ±  7%  softirqs.CPU40.TIMER
     62208 ±  3%     +18.4%      73646 ±  7%  softirqs.CPU41.TIMER
     61786 ±  4%     +18.7%      73322 ±  6%  softirqs.CPU42.TIMER
     61397 ±  3%     +18.4%      72707 ±  6%  softirqs.CPU43.TIMER
     61922 ±  3%     +18.8%      73569 ±  6%  softirqs.CPU44.TIMER
     61990 ±  3%     +21.6%      75363 ±  9%  softirqs.CPU45.TIMER
     61781 ±  4%     +17.9%      72813 ±  7%  softirqs.CPU46.TIMER
     62311 ±  4%     +17.4%      73172 ±  7%  softirqs.CPU47.TIMER
     63175 ±  3%     +15.3%      72836 ±  5%  softirqs.CPU48.TIMER
      4376 ± 13%     +82.0%       7962 ± 22%  softirqs.CPU49.SCHED
     61299 ±  3%     +22.7%      75220 ±  6%  softirqs.CPU49.TIMER
     63297 ±  5%     +15.1%      72854 ±  5%  softirqs.CPU5.TIMER
     63494 ±  5%     +13.6%      72161 ±  5%  softirqs.CPU50.TIMER
     60467 ±  4%     +19.8%      72425 ±  5%  softirqs.CPU51.TIMER
     62426 ±  3%     +15.8%      72299 ±  5%  softirqs.CPU52.TIMER
      3730 ± 23%    +104.5%       7627 ± 27%  softirqs.CPU53.SCHED
     60268 ±  4%     +20.6%      72704 ±  6%  softirqs.CPU53.TIMER
     60384 ±  3%     +20.2%      72596 ±  5%  softirqs.CPU54.TIMER
     59884 ±  4%     +21.3%      72641 ±  5%  softirqs.CPU55.TIMER
     61465 ±  3%     +18.1%      72593 ±  5%  softirqs.CPU56.TIMER
     60000 ±  4%     +36.9%      82133 ± 22%  softirqs.CPU57.TIMER
     59859 ±  4%     +21.4%      72652 ±  3%  softirqs.CPU58.TIMER
     60460 ±  4%     +20.3%      72740 ±  5%  softirqs.CPU59.TIMER
     62593 ±  6%     +16.6%      72976 ±  5%  softirqs.CPU6.TIMER
     60623 ±  3%     +18.9%      72092 ±  5%  softirqs.CPU60.TIMER
     60870 ±  4%     +18.6%      72194 ±  5%  softirqs.CPU61.TIMER
     60338 ±  4%     +20.8%      72913 ±  3%  softirqs.CPU62.TIMER
     60023 ±  4%     +19.2%      71519 ±  4%  softirqs.CPU63.TIMER
     60896 ±  4%     +19.2%      72590 ±  5%  softirqs.CPU64.TIMER
     60505 ±  4%     +20.1%      72677 ±  5%  softirqs.CPU65.TIMER
     60947 ±  4%     +18.4%      72137 ±  5%  softirqs.CPU66.TIMER
     60324 ±  3%     +19.5%      72100 ±  5%  softirqs.CPU67.TIMER
     61520 ±  7%     +16.6%      71744 ±  5%  softirqs.CPU68.TIMER
     63228 ±  5%     +13.9%      72028 ±  5%  softirqs.CPU69.TIMER
     60844 ±  4%     +19.1%      72437 ±  4%  softirqs.CPU7.TIMER
     60471 ±  4%     +18.2%      71500 ±  4%  softirqs.CPU70.TIMER
     61537 ±  3%     +17.8%      72482 ±  6%  softirqs.CPU71.TIMER
     62250 ±  2%     +18.3%      73620 ±  7%  softirqs.CPU72.TIMER
     61938 ±  4%     +17.3%      72639 ±  7%  softirqs.CPU73.TIMER
     61631 ±  3%     +17.2%      72205 ±  7%  softirqs.CPU74.TIMER
     61498 ±  4%     +17.6%      72297 ±  7%  softirqs.CPU75.TIMER
     61189 ±  3%     +18.8%      72709 ±  7%  softirqs.CPU76.TIMER
     62192 ±  4%     +18.0%      73396 ±  8%  softirqs.CPU77.TIMER
     61744 ±  3%     +17.1%      72325 ±  7%  softirqs.CPU78.TIMER
     62638 ±  3%     +15.6%      72388 ±  7%  softirqs.CPU79.TIMER
     61442 ±  4%     +17.3%      72049 ±  5%  softirqs.CPU8.TIMER
     61741 ±  3%     +17.2%      72379 ±  7%  softirqs.CPU80.TIMER
     63379 ±  2%     +14.7%      72714 ±  7%  softirqs.CPU81.TIMER
     62133 ±  3%     +17.1%      72734 ±  7%  softirqs.CPU82.TIMER
     62184 ±  3%     +17.2%      72868 ±  7%  softirqs.CPU83.TIMER
     61565 ±  4%     +18.5%      72962 ±  7%  softirqs.CPU84.TIMER
     61814 ±  3%     +17.6%      72723 ±  7%  softirqs.CPU85.TIMER
     61845 ±  3%     +17.1%      72417 ±  7%  softirqs.CPU86.TIMER
     62581 ±  5%     +16.4%      72822 ±  7%  softirqs.CPU87.TIMER
     61505 ±  3%     +18.2%      72684 ±  6%  softirqs.CPU88.TIMER
     62105 ±  3%     +16.6%      72437 ±  7%  softirqs.CPU89.TIMER
     61379 ±  5%     +19.1%      73092 ±  5%  softirqs.CPU9.TIMER
     61306 ±  3%     +18.1%      72423 ±  7%  softirqs.CPU90.TIMER
     61228 ±  3%     +18.1%      72309 ±  7%  softirqs.CPU91.TIMER
     61437 ±  2%     +17.0%      71906 ±  7%  softirqs.CPU92.TIMER
     61562 ±  3%     +17.1%      72076 ±  7%  softirqs.CPU93.TIMER
     61909 ±  3%     +16.4%      72071 ±  7%  softirqs.CPU94.TIMER
    397809 ±  7%     +51.8%     603839 ± 19%  softirqs.SCHED
   5953086 ±  3%     +18.2%    7036827 ±  6%  softirqs.TIMER
    147.25 ±  6%     +25.5%     184.75 ± 12%  interrupts.35:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc
      1.00 ±100%   +6325.0%      64.25 ± 92%  interrupts.60:PCI-MSI.31981593-edge.i40e-eth0-TxRx-24
    349317 ±  3%     +21.1%     422991 ±  6%  interrupts.CPU0.LOC:Local_timer_interrupts
      7610 ±  4%     -31.3%       5230 ± 33%  interrupts.CPU0.NMI:Non-maskable_interrupts
      7610 ±  4%     -31.3%       5230 ± 33%  interrupts.CPU0.PMI:Performance_monitoring_interrupts
    349158 ±  3%     +21.2%     423344 ±  6%  interrupts.CPU1.LOC:Local_timer_interrupts
    349286 ±  3%     +21.2%     423243 ±  6%  interrupts.CPU10.LOC:Local_timer_interrupts
    464.00 ±  2%     +16.6%     541.00 ± 16%  interrupts.CPU11.CAL:Function_call_interrupts
    348627 ±  3%     +21.4%     423349 ±  6%  interrupts.CPU11.LOC:Local_timer_interrupts
      7746           -24.5%       5849 ± 27%  interrupts.CPU11.NMI:Non-maskable_interrupts
      7746           -24.5%       5849 ± 27%  interrupts.CPU11.PMI:Performance_monitoring_interrupts
    195.50 ± 10%     +34.0%     262.00 ± 22%  interrupts.CPU11.RES:Rescheduling_interrupts
    349624 ±  3%     +21.0%     423121 ±  6%  interrupts.CPU12.LOC:Local_timer_interrupts
      6575 ± 24%     -36.7%       4162 ± 33%  interrupts.CPU12.NMI:Non-maskable_interrupts
      6575 ± 24%     -36.7%       4162 ± 33%  interrupts.CPU12.PMI:Performance_monitoring_interrupts
    349667 ±  3%     +21.0%     423134 ±  6%  interrupts.CPU13.LOC:Local_timer_interrupts
    458.25 ±  2%     +68.4%     771.75 ± 60%  interrupts.CPU14.CAL:Function_call_interrupts
    349672 ±  3%     +21.1%     423341 ±  6%  interrupts.CPU14.LOC:Local_timer_interrupts
    180.50 ±  6%     +21.5%     219.25 ±  5%  interrupts.CPU14.RES:Rescheduling_interrupts
    349277 ±  3%     +21.1%     422914 ±  6%  interrupts.CPU15.LOC:Local_timer_interrupts
    349462 ±  3%     +21.1%     423128 ±  6%  interrupts.CPU16.LOC:Local_timer_interrupts
    349570 ±  3%     +21.1%     423387 ±  6%  interrupts.CPU17.LOC:Local_timer_interrupts
    349561 ±  3%     +21.1%     423252 ±  6%  interrupts.CPU18.LOC:Local_timer_interrupts
    349579 ±  3%     +21.0%     423069 ±  6%  interrupts.CPU19.LOC:Local_timer_interrupts
    348985 ±  3%     +21.3%     423252 ±  6%  interrupts.CPU2.LOC:Local_timer_interrupts
    349387 ±  3%     +21.1%     423015 ±  6%  interrupts.CPU20.LOC:Local_timer_interrupts
    349572 ±  3%     +21.1%     423268 ±  6%  interrupts.CPU21.LOC:Local_timer_interrupts
    349446 ±  3%     +21.1%     423343 ±  6%  interrupts.CPU22.LOC:Local_timer_interrupts
    349125 ±  3%     +21.2%     423279 ±  6%  interrupts.CPU23.LOC:Local_timer_interrupts
      0.50 ±100%  +12750.0%      64.25 ± 92%  interrupts.CPU24.60:PCI-MSI.31981593-edge.i40e-eth0-TxRx-24
    349355 ±  3%     +21.0%     422848 ±  6%  interrupts.CPU24.LOC:Local_timer_interrupts
    349310 ±  3%     +21.1%     423069 ±  6%  interrupts.CPU25.LOC:Local_timer_interrupts
      1588 ± 40%     -65.5%     547.50 ±  3%  interrupts.CPU26.CAL:Function_call_interrupts
    349129 ±  3%     +21.2%     423009 ±  6%  interrupts.CPU26.LOC:Local_timer_interrupts
    349286 ±  3%     +21.1%     423073 ±  6%  interrupts.CPU27.LOC:Local_timer_interrupts
    349099 ±  3%     +21.2%     422962 ±  6%  interrupts.CPU28.LOC:Local_timer_interrupts
    349383 ±  3%     +21.1%     423047 ±  6%  interrupts.CPU29.LOC:Local_timer_interrupts
    349417 ±  3%     +21.1%     423123 ±  6%  interrupts.CPU3.LOC:Local_timer_interrupts
    349141 ±  3%     +21.1%     422972 ±  6%  interrupts.CPU30.LOC:Local_timer_interrupts
    183.00 ±  7%     +21.4%     222.25 ±  2%  interrupts.CPU30.RES:Rescheduling_interrupts
    349301 ±  3%     +21.1%     423145 ±  6%  interrupts.CPU31.LOC:Local_timer_interrupts
    348747 ±  3%     +21.3%     423017 ±  6%  interrupts.CPU32.LOC:Local_timer_interrupts
    349092 ±  3%     +21.2%     423139 ±  6%  interrupts.CPU33.LOC:Local_timer_interrupts
    349215 ±  3%     +21.2%     423088 ±  6%  interrupts.CPU34.LOC:Local_timer_interrupts
    349257 ±  3%     +21.1%     422903 ±  6%  interrupts.CPU35.LOC:Local_timer_interrupts
    349192 ±  3%     +21.1%     423034 ±  6%  interrupts.CPU36.LOC:Local_timer_interrupts
    171.50 ±  2%     +24.9%     214.25 ±  7%  interrupts.CPU36.RES:Rescheduling_interrupts
    349125 ±  3%     +21.2%     423014 ±  6%  interrupts.CPU37.LOC:Local_timer_interrupts
    349295 ±  3%     +21.1%     423030 ±  6%  interrupts.CPU38.LOC:Local_timer_interrupts
    348979 ±  3%     +21.2%     422869 ±  6%  interrupts.CPU39.LOC:Local_timer_interrupts
    171.00 ±  6%     +29.8%     222.00 ± 15%  interrupts.CPU39.RES:Rescheduling_interrupts
    349387 ±  3%     +21.1%     423056 ±  6%  interrupts.CPU4.LOC:Local_timer_interrupts
      7633 ±  3%     -27.2%       5557 ± 20%  interrupts.CPU4.NMI:Non-maskable_interrupts
      7633 ±  3%     -27.2%       5557 ± 20%  interrupts.CPU4.PMI:Performance_monitoring_interrupts
    349285 ±  3%     +21.1%     423025 ±  6%  interrupts.CPU40.LOC:Local_timer_interrupts
    179.25 ±  9%     +13.0%     202.50 ±  7%  interrupts.CPU40.RES:Rescheduling_interrupts
    349237 ±  3%     +21.1%     423008 ±  6%  interrupts.CPU41.LOC:Local_timer_interrupts
    349280 ±  3%     +21.1%     422986 ±  6%  interrupts.CPU42.LOC:Local_timer_interrupts
    349364 ±  3%     +21.1%     423033 ±  6%  interrupts.CPU43.LOC:Local_timer_interrupts
    166.00 ±  9%     +22.0%     202.50 ±  8%  interrupts.CPU43.RES:Rescheduling_interrupts
    349207 ±  3%     +21.1%     423039 ±  6%  interrupts.CPU44.LOC:Local_timer_interrupts
    349270 ±  3%     +21.1%     423126 ±  6%  interrupts.CPU45.LOC:Local_timer_interrupts
    165.00 ±  6%     +15.3%     190.25 ±  5%  interrupts.CPU45.RES:Rescheduling_interrupts
    349398 ±  3%     +21.1%     423046 ±  6%  interrupts.CPU46.LOC:Local_timer_interrupts
    170.75 ±  8%     +21.8%     208.00 ±  6%  interrupts.CPU46.RES:Rescheduling_interrupts
    349353 ±  3%     +21.1%     423026 ±  6%  interrupts.CPU47.LOC:Local_timer_interrupts
    174.75 ± 15%     +21.6%     212.50 ± 11%  interrupts.CPU47.RES:Rescheduling_interrupts
    349297 ±  3%     +21.2%     423328 ±  6%  interrupts.CPU48.LOC:Local_timer_interrupts
    349418 ±  3%     +21.1%     423220 ±  6%  interrupts.CPU49.LOC:Local_timer_interrupts
    349291 ±  3%     +21.2%     423207 ±  6%  interrupts.CPU5.LOC:Local_timer_interrupts
    349211 ±  3%     +21.2%     423232 ±  6%  interrupts.CPU50.LOC:Local_timer_interrupts
    349086 ±  3%     +21.2%     423216 ±  6%  interrupts.CPU51.LOC:Local_timer_interrupts
      7606 ±  5%     -47.4%       4004 ± 33%  interrupts.CPU51.NMI:Non-maskable_interrupts
      7606 ±  5%     -47.4%       4004 ± 33%  interrupts.CPU51.PMI:Performance_monitoring_interrupts
    349419 ±  3%     +21.0%     422768 ±  6%  interrupts.CPU52.LOC:Local_timer_interrupts
      7778           -18.9%       6309 ±  5%  interrupts.CPU52.NMI:Non-maskable_interrupts
      7778           -18.9%       6309 ±  5%  interrupts.CPU52.PMI:Performance_monitoring_interrupts
    349468 ±  3%     +21.0%     422917 ±  6%  interrupts.CPU53.LOC:Local_timer_interrupts
    349573 ±  3%     +21.1%     423194 ±  6%  interrupts.CPU54.LOC:Local_timer_interrupts
      7807           -18.7%       6350 ±  7%  interrupts.CPU54.NMI:Non-maskable_interrupts
      7807           -18.7%       6350 ±  7%  interrupts.CPU54.PMI:Performance_monitoring_interrupts
    349365 ±  3%     +21.1%     423236 ±  6%  interrupts.CPU55.LOC:Local_timer_interrupts
      7556 ±  6%     -37.1%       4751 ± 32%  interrupts.CPU55.NMI:Non-maskable_interrupts
      7556 ±  6%     -37.1%       4751 ± 32%  interrupts.CPU55.PMI:Performance_monitoring_interrupts
    349268 ±  3%     +21.2%     423209 ±  6%  interrupts.CPU56.LOC:Local_timer_interrupts
    349325 ±  3%     +21.1%     423164 ±  6%  interrupts.CPU57.LOC:Local_timer_interrupts
    349320 ±  3%     +21.1%     423127 ±  6%  interrupts.CPU58.LOC:Local_timer_interrupts
    349440 ±  3%     +21.2%     423350 ±  6%  interrupts.CPU59.LOC:Local_timer_interrupts
    349249 ±  3%     +21.2%     423191 ±  6%  interrupts.CPU6.LOC:Local_timer_interrupts
    349499 ±  3%     +21.0%     423031 ±  6%  interrupts.CPU60.LOC:Local_timer_interrupts
    349496 ±  3%     +21.1%     423202 ±  6%  interrupts.CPU61.LOC:Local_timer_interrupts
    475.00 ±  5%     +12.5%     534.25 ±  7%  interrupts.CPU62.CAL:Function_call_interrupts
    349547 ±  3%     +21.0%     423065 ±  6%  interrupts.CPU62.LOC:Local_timer_interrupts
      7557 ±  6%     -29.1%       5356 ± 24%  interrupts.CPU62.NMI:Non-maskable_interrupts
      7557 ±  6%     -29.1%       5356 ± 24%  interrupts.CPU62.PMI:Performance_monitoring_interrupts
    348271 ±  3%     +21.5%     423010 ±  6%  interrupts.CPU63.LOC:Local_timer_interrupts
    349332 ±  3%     +21.2%     423400 ±  6%  interrupts.CPU64.LOC:Local_timer_interrupts
    349226 ±  3%     +21.1%     422998 ±  6%  interrupts.CPU65.LOC:Local_timer_interrupts
      7595 ±  5%     -27.6%       5501 ± 17%  interrupts.CPU65.NMI:Non-maskable_interrupts
      7595 ±  5%     -27.6%       5501 ± 17%  interrupts.CPU65.PMI:Performance_monitoring_interrupts
    349538 ±  3%     +21.1%     423214 ±  6%  interrupts.CPU66.LOC:Local_timer_interrupts
      7552 ±  6%     -30.2%       5272 ± 23%  interrupts.CPU66.NMI:Non-maskable_interrupts
      7552 ±  6%     -30.2%       5272 ± 23%  interrupts.CPU66.PMI:Performance_monitoring_interrupts
    349560 ±  3%     +21.1%     423298 ±  6%  interrupts.CPU67.LOC:Local_timer_interrupts
      7556 ±  6%     -20.7%       5989 ±  7%  interrupts.CPU67.NMI:Non-maskable_interrupts
      7556 ±  6%     -20.7%       5989 ±  7%  interrupts.CPU67.PMI:Performance_monitoring_interrupts
    349516 ±  3%     +21.0%     423040 ±  6%  interrupts.CPU68.LOC:Local_timer_interrupts
      7665 ±  3%     -28.7%       5466 ± 29%  interrupts.CPU68.NMI:Non-maskable_interrupts
      7665 ±  3%     -28.7%       5466 ± 29%  interrupts.CPU68.PMI:Performance_monitoring_interrupts
    349088 ±  3%     +21.2%     423047 ±  6%  interrupts.CPU69.LOC:Local_timer_interrupts
    147.25 ±  6%     +25.5%     184.75 ± 12%  interrupts.CPU7.35:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc
    349480 ±  3%     +21.1%     423165 ±  6%  interrupts.CPU7.LOC:Local_timer_interrupts
    348916 ±  3%     +21.3%     423169 ±  6%  interrupts.CPU70.LOC:Local_timer_interrupts
    349575 ±  3%     +21.1%     423199 ±  6%  interrupts.CPU71.LOC:Local_timer_interrupts
      7565 ±  6%     -28.9%       5381 ± 23%  interrupts.CPU71.NMI:Non-maskable_interrupts
      7565 ±  6%     -28.9%       5381 ± 23%  interrupts.CPU71.PMI:Performance_monitoring_interrupts
    349025 ±  3%     +21.2%     422912 ±  6%  interrupts.CPU72.LOC:Local_timer_interrupts
    349214 ±  3%     +21.1%     422921 ±  6%  interrupts.CPU73.LOC:Local_timer_interrupts
    349156 ±  3%     +21.2%     423037 ±  6%  interrupts.CPU74.LOC:Local_timer_interrupts
    349051 ±  3%     +21.1%     422808 ±  6%  interrupts.CPU75.LOC:Local_timer_interrupts
    349153 ±  3%     +21.1%     422847 ±  6%  interrupts.CPU76.LOC:Local_timer_interrupts
    349043 ±  3%     +21.2%     422957 ±  6%  interrupts.CPU77.LOC:Local_timer_interrupts
    349075 ±  3%     +21.2%     422931 ±  6%  interrupts.CPU78.LOC:Local_timer_interrupts
    349151 ±  3%     +21.1%     422951 ±  6%  interrupts.CPU79.LOC:Local_timer_interrupts
    186.25 ± 10%     -18.0%     152.75 ± 11%  interrupts.CPU79.RES:Rescheduling_interrupts
    349688 ±  3%     +21.0%     423133 ±  6%  interrupts.CPU8.LOC:Local_timer_interrupts
      7543 ±  5%     -13.9%       6492 ± 10%  interrupts.CPU8.NMI:Non-maskable_interrupts
      7543 ±  5%     -13.9%       6492 ± 10%  interrupts.CPU8.PMI:Performance_monitoring_interrupts
    189.75 ±  6%     +25.3%     237.75 ± 11%  interrupts.CPU8.RES:Rescheduling_interrupts
    349068 ±  3%     +21.2%     422917 ±  6%  interrupts.CPU80.LOC:Local_timer_interrupts
    349046 ±  3%     +21.2%     422898 ±  6%  interrupts.CPU81.LOC:Local_timer_interrupts
    349015 ±  3%     +21.2%     422981 ±  6%  interrupts.CPU82.LOC:Local_timer_interrupts
    349046 ±  3%     +21.1%     422783 ±  6%  interrupts.CPU83.LOC:Local_timer_interrupts
    348881 ±  3%     +21.2%     422860 ±  6%  interrupts.CPU84.LOC:Local_timer_interrupts
    348773 ±  3%     +21.3%     422902 ±  6%  interrupts.CPU85.LOC:Local_timer_interrupts
    349137 ±  3%     +21.1%     422949 ±  6%  interrupts.CPU86.LOC:Local_timer_interrupts
    348905 ±  3%     +21.2%     422939 ±  6%  interrupts.CPU87.LOC:Local_timer_interrupts
    349051 ±  3%     +21.2%     422898 ±  6%  interrupts.CPU88.LOC:Local_timer_interrupts
    348781 ±  3%     +21.2%     422861 ±  6%  interrupts.CPU89.LOC:Local_timer_interrupts
    348918 ±  3%     +21.3%     423363 ±  6%  interrupts.CPU9.LOC:Local_timer_interrupts
    349140 ±  3%     +21.1%     422939 ±  6%  interrupts.CPU90.LOC:Local_timer_interrupts
    348953 ±  3%     +21.2%     422872 ±  6%  interrupts.CPU91.LOC:Local_timer_interrupts
    349321 ±  3%     +21.0%     422821 ±  6%  interrupts.CPU92.LOC:Local_timer_interrupts
    348937 ±  3%     +21.2%     422932 ±  6%  interrupts.CPU93.LOC:Local_timer_interrupts
    349340 ±  3%     +21.1%     422977 ±  6%  interrupts.CPU94.LOC:Local_timer_interrupts
    349401 ±  3%     +21.1%     423128 ±  6%  interrupts.CPU95.LOC:Local_timer_interrupts
  33527497 ±  3%     +21.1%   40614868 ±  6%  interrupts.LOC:Local_timer_interrupts
    666191 ±  4%      -7.3%     617233 ±  4%  interrupts.NMI:Non-maskable_interrupts
    666191 ±  4%      -7.3%     617233 ±  4%  interrupts.PMI:Performance_monitoring_interrupts





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Rong Chen


View attachment "config-5.8.0-rc1-00023-g0b9730e694e8c" of type "text/plain" (206210 bytes)

View attachment "job-script" of type "text/plain" (7524 bytes)

View attachment "job.yaml" of type "text/plain" (5141 bytes)

View attachment "reproduce" of type "text/plain" (8520 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ