lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [day] [month] [year] [list]
Date:   Mon, 18 Mar 2019 21:35:55 +0800
From:   kernel test robot <rong.a.chen@...el.com>
To:     David Howells <dhowells@...hat.com>
Cc:     Hugh Dickins <hughd@...gle.com>,
        LKML <linux-kernel@...r.kernel.org>,
        David Howells <dhowells@...hat.com>, lkp@...org
Subject: [LKP] [tmpfs, devtmpfs, ramfs, rootfs]  9d65ac8a35:
 vm-scalability.median -19.4% regression

Greeting,

FYI, we noticed a -19.4% regression of vm-scalability.median due to commit:


commit: 9d65ac8a35a75c7334cfa0f855215c91bb4e752b ("tmpfs, devtmpfs, ramfs, rootfs: Convert to fs_context")
https://git.kernel.org/cgit/linux/kernel/git/dhowells/linux-fs.git mount-api-viro

in testcase: vm-scalability
on test machine: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory
with following parameters:

	runtime: 300s
	size: 16G
	test: shm-pread-rand
	cpufreq_governor: performance

test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us.
test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/

In addition to that, the commit also has significant impact on the following tests:

+------------------+------------------------------------------------------------------------+
| testcase: change | vm-scalability: vm-scalability.median 17.0% improvement                |
| test machine     | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory  |
| test parameters  | cpufreq_governor=performance                                           |
|                  | runtime=300s                                                           |
|                  | size=256G                                                              |
|                  | test=lru-shm-rand                                                      |
+------------------+------------------------------------------------------------------------+
| testcase: change | vm-scalability: vm-scalability.median -20.3% regression                |
| test machine     | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory  |
| test parameters  | cpufreq_governor=performance                                           |
|                  | runtime=300s                                                           |
|                  | size=16G                                                               |
|                  | test=shm-xread-rand                                                    |
+------------------+------------------------------------------------------------------------+
| testcase: change | vm-scalability: vm-scalability.median -3.4% regression                 |
| test machine     | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory  |
| test parameters  | cpufreq_governor=performance                                           |
|                  | runtime=300s                                                           |
|                  | size=2T                                                                |
|                  | test=shm-xread-seq-mt                                                  |
+------------------+------------------------------------------------------------------------+
| testcase: change | vm-scalability: vm-scalability.median -40.5% regression                |
| test machine     | 160 threads Intel(R) Xeon(R) CPU E7-8890 v4 @ 2.20GHz with 256G memory |
| test parameters  | cpufreq_governor=performance                                           |
|                  | runtime=300s                                                           |
|                  | size=16G                                                               |
|                  | test=shm-pread-rand                                                    |
|                  | ucode=0xb00002e                                                        |
+------------------+------------------------------------------------------------------------+


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase:
  gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-2018-04-03.cgz/300s/16G/lkp-bdw-ep2/shm-pread-rand/vm-scalability

commit: 
  8055fb607e ("vfs: Convert smackfs to fs_context")
  9d65ac8a35 ("tmpfs, devtmpfs, ramfs, rootfs: Convert to fs_context")

8055fb607ebc82ea 9d65ac8a35a75c7334cfa0f8552 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
     35.50           -50.0%      17.74        vm-scalability.free_time
     58652           -19.4%      47285        vm-scalability.median
   5153844           -19.5%    4147169        vm-scalability.throughput
    345.52            -6.6%     322.82        vm-scalability.time.elapsed_time
    345.52            -6.6%     322.82        vm-scalability.time.elapsed_time.max
    113751 ±  2%      -4.7%     108399 ±  2%  vm-scalability.time.involuntary_context_switches
  65947512           -50.0%   32974438        vm-scalability.time.maximum_resident_set_size
 1.362e+08           -50.0%   68095705        vm-scalability.time.minor_page_faults
      8553            +1.2%       8659        vm-scalability.time.percent_of_cpu_this_job_got
      5020           -49.4%       2539        vm-scalability.time.system_time
     24536            +3.6%      25415        vm-scalability.time.user_time
  1.55e+09           -19.6%  1.245e+09        vm-scalability.workload
   3583491           -19.8%    2873813        pmeter.performance_per_watt
 7.662e+08 ± 18%     -49.4%  3.875e+08 ± 37%  cpuidle.C6.time
    855882 ± 15%     -44.3%     476492 ± 34%  cpuidle.C6.usage
      1.46 ± 14%      -0.2        1.24 ±  8%  perf-profile.calltrace.cycles-pp.rcu_sched_clock_irq.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      0.30 ±100%      +0.4        0.68 ± 10%  perf-profile.calltrace.cycles-pp.read_tsc.ktime_get.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt
      2.83 ±  3%      -1.1        1.70 ±  6%  mpstat.cpu.all.idle%
     16.52            -7.6        8.96        mpstat.cpu.all.sys%
     80.64            +8.7       89.34        mpstat.cpu.all.usr%
     79.50           +10.7%      88.00        vmstat.cpu.us
  64655701           -48.2%   33511191        vmstat.memory.cache
  55573932           +65.7%   92083684        vmstat.memory.free
  10158808 ±  3%     -48.1%    5271382 ±  5%  numa-numastat.node0.local_node
  10167474 ±  3%     -48.1%    5281431 ±  5%  numa-numastat.node0.numa_hit
   9928425 ±  3%     -48.5%    5109232 ±  5%  numa-numastat.node1.local_node
   9937061 ±  3%     -48.5%    5116473 ±  5%  numa-numastat.node1.numa_hit
      9.68           +10.8%      10.72 ±  8%  sched_debug.cfs_rq:/.runnable_load_avg.avg
     24.21 ± 10%    +354.2%     109.96 ± 73%  sched_debug.cfs_rq:/.runnable_load_avg.max
      2.33 ±  9%    +380.7%      11.20 ± 75%  sched_debug.cfs_rq:/.runnable_load_avg.stddev
    130581 ± 18%     +44.1%     188155 ± 17%  sched_debug.cpu.avg_idle.stddev
      2711            +1.1%       2741        turbostat.Avg_MHz
    845415 ± 15%     -44.9%     465446 ± 36%  turbostat.C6
      2.50 ± 18%      -1.2        1.34 ± 37%  turbostat.C6%
      0.94 ± 12%     -47.2%       0.49 ±  8%  turbostat.Pkg%pc2
     26.52            +2.8%      27.25        turbostat.RAMWatt
      3921 ±  5%     -13.9%       3375 ±  8%  slabinfo.kmalloc-rcl-64.active_objs
      3921 ±  5%     -13.9%       3375 ±  8%  slabinfo.kmalloc-rcl-64.num_objs
    742.00 ±  3%     -21.6%     582.00 ±  9%  slabinfo.kmem_cache_node.active_objs
    783.00 ±  3%     -20.4%     623.00 ±  8%  slabinfo.kmem_cache_node.num_objs
    272701           -46.8%     144966        slabinfo.radix_tree_node.active_objs
      4930           -47.2%       2604        slabinfo.radix_tree_node.active_slabs
    276130           -47.2%     145877        slabinfo.radix_tree_node.num_objs
      4930           -47.2%       2604        slabinfo.radix_tree_node.num_slabs
   4569892           -68.8%    1424560        meminfo.Active
   4569576           -68.8%    1424248        meminfo.Active(anon)
  64567907           -48.2%   33424445        meminfo.Cached
  64384458           -49.4%   32588466        meminfo.Committed_AS
  59013843           -47.4%   31022617        meminfo.Inactive
  59012549           -47.4%   31021325        meminfo.Inactive(anon)
    222415           -33.3%     148303        meminfo.KReclaimable
  58958501           -47.5%   30968794        meminfo.Mapped
  54904833           +66.6%   91471162        meminfo.MemAvailable
  55411629           +66.1%   92014991        meminfo.MemFree
  76480902           -47.9%   39877539        meminfo.Memused
  10903221           -49.3%    5527712        meminfo.PageTables
    222415           -33.3%     148303        meminfo.SReclaimable
  63339328           -49.2%   32202719        meminfo.Shmem
    351616           -20.9%     278049        meminfo.Slab
    238022           -44.9%     131123        meminfo.max_used_kB
   1141190           -68.8%     355914        proc-vmstat.nr_active_anon
   1368629           +66.7%    2281555        proc-vmstat.nr_dirty_background_threshold
   2740606           +66.7%    4568693        proc-vmstat.nr_dirty_threshold
  16135961           -48.2%    8356035        proc-vmstat.nr_file_pages
  13860684           +66.0%   23003344        proc-vmstat.nr_free_pages
  14748066           -47.4%    7755141        proc-vmstat.nr_inactive_anon
  14734654           -47.5%    7742396        proc-vmstat.nr_mapped
   2724876           -49.3%    1382176        proc-vmstat.nr_page_table_pages
  15828555           -49.1%    8050342        proc-vmstat.nr_shmem
     55601           -33.3%      37078        proc-vmstat.nr_slab_reclaimable
   1141190           -68.8%     355914        proc-vmstat.nr_zone_active_anon
  14748066           -47.4%    7755141        proc-vmstat.nr_zone_inactive_anon
  20129588           -48.2%   10426124        proc-vmstat.numa_hit
  20112280           -48.2%   10408815        proc-vmstat.numa_local
  16498333           -50.0%    8255747        proc-vmstat.pgactivate
  20220521           -48.1%   10503690        proc-vmstat.pgalloc_normal
 1.371e+08           -49.7%   68925453        proc-vmstat.pgfault
  19940928           -48.5%   10269082 ±  2%  proc-vmstat.pgfree
    222.75 ± 30%     +57.7%     351.25 ± 40%  interrupts.46:IR-PCI-MSI.1572876-edge.eth0-TxRx-12
    214.75 ± 30%     -26.2%     158.50        interrupts.76:IR-PCI-MSI.1572904-edge.eth0-TxRx-40
    174.25 ±  2%      -9.0%     158.50        interrupts.84:IR-PCI-MSI.1572912-edge.eth0-TxRx-48
    546.00 ±  2%      -8.9%     497.50        interrupts.9:IR-IO-APIC.9-fasteoi.acpi
    314943            -5.9%     296436        interrupts.CAL:Function_call_interrupts
    546.00 ±  2%      -8.9%     497.50        interrupts.CPU1.9:IR-IO-APIC.9-fasteoi.acpi
    222.75 ± 30%     +57.7%     351.25 ± 40%  interrupts.CPU12.46:IR-PCI-MSI.1572876-edge.eth0-TxRx-12
    937.75 ± 45%     -68.9%     292.00 ±106%  interrupts.CPU13.RES:Rescheduling_interrupts
    165.25 ± 82%    +146.3%     407.00 ± 16%  interrupts.CPU26.RES:Rescheduling_interrupts
      4933 ± 34%     +59.4%       7861        interrupts.CPU33.NMI:Non-maskable_interrupts
      4933 ± 34%     +59.4%       7861        interrupts.CPU33.PMI:Performance_monitoring_interrupts
      4951 ± 34%     +59.2%       7880        interrupts.CPU35.NMI:Non-maskable_interrupts
      4951 ± 34%     +59.2%       7880        interrupts.CPU35.PMI:Performance_monitoring_interrupts
    318.50 ± 59%    +142.9%     773.50 ± 48%  interrupts.CPU37.RES:Rescheduling_interrupts
    214.75 ± 30%     -26.2%     158.50        interrupts.CPU40.76:IR-PCI-MSI.1572904-edge.eth0-TxRx-40
    174.25 ±  2%      -9.0%     158.50        interrupts.CPU48.84:IR-PCI-MSI.1572912-edge.eth0-TxRx-48
    259.25 ± 90%     -75.5%      63.50 ± 84%  interrupts.CPU52.RES:Rescheduling_interrupts
     62.75 ±  3%    +549.8%     407.75 ± 48%  interrupts.CPU56.RES:Rescheduling_interrupts
    101.25 ± 18%     -41.5%      59.25 ± 41%  interrupts.CPU65.RES:Rescheduling_interrupts
    112.50 ± 88%     -77.3%      25.50 ± 58%  interrupts.CPU78.RES:Rescheduling_interrupts
   2329031 ±  2%     -68.6%     732077 ±  6%  numa-meminfo.node0.Active
   2328876 ±  2%     -68.6%     731939 ±  6%  numa-meminfo.node0.Active(anon)
  32528442           -47.6%   17046796        numa-meminfo.node0.FilePages
  29716367           -46.7%   15825723        numa-meminfo.node0.Inactive
  29715713           -46.7%   15825283        numa-meminfo.node0.Inactive(anon)
  29686207           -46.8%   15780849        numa-meminfo.node0.Mapped
  27255472 ±  4%     +66.0%   45237231        numa-meminfo.node0.MemFree
  38598666 ±  2%     -46.6%   20610335 ±  3%  numa-meminfo.node0.MemUsed
   5516070 ±  9%     -44.9%    3039561 ±  9%  numa-meminfo.node0.PageTables
  31920558           -48.5%   16434497        numa-meminfo.node0.Shmem
   2288090 ±  3%     -69.1%     706425 ±  6%  numa-meminfo.node1.Active
   2287930 ±  3%     -69.1%     706252 ±  6%  numa-meminfo.node1.Active(anon)
  32045048           -48.9%   16388773        numa-meminfo.node1.FilePages
  29254750           -48.1%   15193020        numa-meminfo.node1.Inactive
  29254110           -48.1%   15192167        numa-meminfo.node1.Inactive(anon)
  29229499           -48.1%   15183674        numa-meminfo.node1.Mapped
  28158551 ±  3%     +66.1%   46765793        numa-meminfo.node1.MemFree
  37879839 ±  2%     -49.1%   19279170 ±  3%  numa-meminfo.node1.MemUsed
   5382524 ± 10%     -53.7%    2490391 ± 11%  numa-meminfo.node1.PageTables
  31423310           -49.8%   15778300        numa-meminfo.node1.Shmem
    581468 ±  2%     -68.6%     182804 ±  6%  numa-vmstat.node0.nr_active_anon
   8125960           -47.5%    4262789        numa-vmstat.node0.nr_file_pages
   6821064 ±  4%     +65.8%   11307564        numa-vmstat.node0.nr_free_pages
   7423531           -46.7%    3957573        numa-vmstat.node0.nr_inactive_anon
   7416171           -46.8%    3946658        numa-vmstat.node0.nr_mapped
   1377979 ±  9%     -44.8%     760546 ±  9%  numa-vmstat.node0.nr_page_table_pages
   7973989           -48.5%    4109714        numa-vmstat.node0.nr_shmem
    581468 ±  2%     -68.6%     182804 ±  6%  numa-vmstat.node0.nr_zone_active_anon
   7423531           -46.7%    3957573        numa-vmstat.node0.nr_zone_inactive_anon
  10222865 ±  3%     -45.7%    5555074 ±  3%  numa-vmstat.node0.numa_hit
  10213690 ±  3%     -45.7%    5544531 ±  3%  numa-vmstat.node0.numa_local
    571252 ±  3%     -69.1%     176447 ±  6%  numa-vmstat.node1.nr_active_anon
   8005128           -48.8%    4098150        numa-vmstat.node1.nr_file_pages
   7046569 ±  3%     +65.9%   11690035        numa-vmstat.node1.nr_free_pages
   7308130           -48.0%    3799139        numa-vmstat.node1.nr_inactive_anon
   7302096           -48.0%    3797203        numa-vmstat.node1.nr_mapped
   1344848 ±  9%     -53.7%     623009 ± 11%  numa-vmstat.node1.nr_page_table_pages
   7849694           -49.7%    3945532        numa-vmstat.node1.nr_shmem
    571251 ±  3%     -69.1%     176446 ±  6%  numa-vmstat.node1.nr_zone_active_anon
   7308130           -48.0%    3799139        numa-vmstat.node1.nr_zone_inactive_anon
   9965690 ±  3%     -47.2%    5258034 ±  3%  numa-vmstat.node1.numa_hit
   9781491 ±  3%     -48.1%    5075833 ±  4%  numa-vmstat.node1.numa_local
    140703 ±  3%     -14.3%     120593 ±  6%  softirqs.CPU10.TIMER
    142725 ±  2%     -16.8%     118771 ±  5%  softirqs.CPU11.TIMER
    145496 ±  3%     -17.4%     120236 ±  5%  softirqs.CPU12.TIMER
    142496 ±  4%     -15.8%     119968 ±  5%  softirqs.CPU13.TIMER
    147287 ±  5%     -15.9%     123852 ±  7%  softirqs.CPU14.TIMER
    149316 ±  2%     -15.3%     126451 ±  7%  softirqs.CPU15.TIMER
    143888 ±  5%     -13.9%     123840 ±  7%  softirqs.CPU16.TIMER
    141273 ±  4%     -15.1%     120003 ±  5%  softirqs.CPU19.TIMER
    146948 ±  6%     -15.4%     124264 ±  7%  softirqs.CPU2.TIMER
    146064 ±  4%     -14.8%     124416 ±  7%  softirqs.CPU20.TIMER
    144060 ±  5%     -15.6%     121575 ±  6%  softirqs.CPU3.TIMER
    146000 ±  2%     -15.4%     123551 ±  6%  softirqs.CPU4.TIMER
     34170 ± 19%     +25.2%      42774 ±  7%  softirqs.CPU45.RCU
    143930 ±  5%     -13.6%     124408 ±  8%  softirqs.CPU46.TIMER
    145203 ±  5%     -15.6%     122528 ±  6%  softirqs.CPU47.TIMER
    143932 ±  4%     -15.1%     122182 ±  6%  softirqs.CPU48.TIMER
    142245 ±  4%     -14.2%     122017 ±  6%  softirqs.CPU49.TIMER
    141972 ±  4%     -14.1%     121995 ±  6%  softirqs.CPU5.TIMER
    145814 ±  4%     -13.5%     126067 ±  8%  softirqs.CPU50.TIMER
    143490 ±  5%     -13.2%     124617 ±  8%  softirqs.CPU51.TIMER
    144034           -15.8%     121348 ±  5%  softirqs.CPU52.TIMER
    140485 ±  3%     -14.0%     120835 ±  6%  softirqs.CPU54.TIMER
    140156 ±  2%     -15.0%     119090 ±  5%  softirqs.CPU55.TIMER
    142404 ±  3%     -16.3%     119214 ±  6%  softirqs.CPU56.TIMER
    141461 ±  4%     -16.1%     118734 ±  5%  softirqs.CPU57.TIMER
    145017 ±  5%     -15.2%     122909 ±  7%  softirqs.CPU58.TIMER
    146559 ±  4%     -13.5%     126754 ±  7%  softirqs.CPU59.TIMER
    146090 ±  5%     -14.2%     125326 ±  8%  softirqs.CPU6.TIMER
    143474 ±  4%     -14.6%     122493 ±  7%  softirqs.CPU60.TIMER
    138424 ±  5%     -13.5%     119705 ±  6%  softirqs.CPU61.TIMER
    141324 ±  4%     -15.2%     119813 ±  6%  softirqs.CPU63.TIMER
    146399 ±  3%     -15.6%     123495 ±  7%  softirqs.CPU64.TIMER
    144796 ±  5%     -13.6%     125089 ±  8%  softirqs.CPU7.TIMER
    144444           -16.0%     121322 ±  4%  softirqs.CPU8.TIMER
    144366 ±  3%     -14.5%     123476 ±  6%  softirqs.CPU9.TIMER
    335692           -15.1%     285135 ±  2%  softirqs.SCHED
  12267090 ±  3%      -9.0%   11168809 ±  2%  softirqs.TIMER
     27.35            +2.9%      28.15        perf-stat.i.MPKI
 8.823e+09            -1.4%  8.697e+09        perf-stat.i.branch-instructions
      0.15 ±  4%      -0.0        0.10 ±  8%  perf-stat.i.branch-miss-rate%
  10706236           -29.4%    7553569        perf-stat.i.branch-misses
     63.60            +4.2       67.76        perf-stat.i.cache-miss-rate%
 6.597e+08            +8.5%  7.154e+08        perf-stat.i.cache-misses
 9.614e+08            +5.4%  1.013e+09        perf-stat.i.cache-references
      6.44            +1.4%       6.53        perf-stat.i.cpi
 2.379e+11            +1.2%  2.408e+11        perf-stat.i.cpu-cycles
    657.07 ±  2%     -23.0%     505.74        perf-stat.i.cycles-between-cache-misses
      4.43            +0.4        4.84        perf-stat.i.dTLB-load-miss-rate%
 4.713e+08           +12.3%  5.294e+08        perf-stat.i.dTLB-load-misses
 3.296e+09            +5.8%  3.489e+09        perf-stat.i.dTLB-stores
    888133 ±  3%     -41.6%     518784 ±  6%  perf-stat.i.iTLB-load-misses
 3.784e+10            -1.3%  3.734e+10        perf-stat.i.instructions
      0.17            -4.7%       0.16        perf-stat.i.ipc
    394814           -46.2%     212352        perf-stat.i.minor-faults
     37.04            -3.7       33.29 ±  3%  perf-stat.i.node-load-miss-rate%
 1.848e+08           +10.2%  2.036e+08 ±  3%  perf-stat.i.node-load-misses
 4.608e+08            +9.3%  5.038e+08 ±  2%  perf-stat.i.node-loads
     32.56 ± 20%     -10.4       22.18 ± 16%  perf-stat.i.node-store-miss-rate%
   2252868           -44.9%    1240402        perf-stat.i.node-store-misses
   1551162           -41.4%     908499 ±  2%  perf-stat.i.node-stores
    394819           -46.2%     212359        perf-stat.i.page-faults
     25.34            +6.9%      27.09        perf-stat.overall.MPKI
      0.12            -0.0        0.09        perf-stat.overall.branch-miss-rate%
     68.59            +2.0       70.60        perf-stat.overall.cache-miss-rate%
      6.28            +2.6%       6.45        perf-stat.overall.cpi
    361.41            -6.7%     337.04        perf-stat.overall.cycles-between-cache-misses
      4.21            +0.5        4.73        perf-stat.overall.dTLB-load-miss-rate%
     42463 ±  3%     +69.3%      71884 ±  6%  perf-stat.overall.instructions-per-iTLB-miss
      0.16            -2.6%       0.16        perf-stat.overall.ipc
     59.20            -1.5       57.72        perf-stat.overall.node-store-miss-rate%
      8436           +14.7%       9672        perf-stat.overall.path-length
 8.808e+09            -1.5%  8.676e+09        perf-stat.ps.branch-instructions
  10767486           -29.4%    7602885        perf-stat.ps.branch-misses
 6.563e+08            +8.5%  7.123e+08        perf-stat.ps.cache-misses
  9.57e+08            +5.4%  1.009e+09        perf-stat.ps.cache-references
 2.372e+11            +1.2%  2.401e+11        perf-stat.ps.cpu-cycles
 4.688e+08           +12.4%   5.27e+08        perf-stat.ps.dTLB-load-misses
 3.283e+09            +5.9%  3.475e+09        perf-stat.ps.dTLB-stores
    890294 ±  3%     -41.6%     520318 ±  6%  perf-stat.ps.iTLB-load-misses
 3.777e+10            -1.4%  3.724e+10        perf-stat.ps.instructions
    395820           -46.2%     213017        perf-stat.ps.minor-faults
 1.839e+08           +10.2%  2.028e+08 ±  3%  perf-stat.ps.node-load-misses
 4.583e+08            +9.4%  5.015e+08 ±  2%  perf-stat.ps.node-loads
   2263230           -44.9%    1247828        perf-stat.ps.node-store-misses
   1559702           -41.4%     914092 ±  2%  perf-stat.ps.node-stores
    395820           -46.2%     213017        perf-stat.ps.page-faults
 1.307e+13            -7.9%  1.205e+13        perf-stat.total.instructions


                                                                                
                            vm-scalability.time.user_time                       
                                                                                
  25600 +-+-----------------------------------------------------------------+   
        O O O  O     OO   O                                                 |   
  25400 +-+  O   O O    O   OO O O O OO O O O OO O O                        |   
        |                                                                   |   
  25200 +-+                                                                 |   
        |                                                                   |   
  25000 +-+                                                                 |   
        |                                                                   |   
  24800 +-+                                                   +             |   
        |                                                    +:             |   
  24600 +-+                                                 +  :           +|   
        |  .++. .+   +   .+.  .+.+.    .+.+. .+            +   :.+.+.+.+ .+ |   
  24400 +-+    +  + + +.+   ++     +.++     +  +. .+.+.++.+    +        +   |   
        |          +                             +                          |   
  24200 +-+-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                          vm-scalability.time.system_time                       
                                                                                
  5500 +-+------------------------------------------------------------------+   
       |       .+.+.                                                        |   
  5000 +-+.++.+     +.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.+.+.+ .+.+.+.+.++.+.|   
       |                                                     +              |   
       |                                                                    |   
  4500 +-+                                                                  |   
       |                                                                    |   
  4000 +-+                                                                  |   
       |                                                                    |   
  3500 +-+                                                                  |   
       |                                                                    |   
       |                                                                    |   
  3000 +-+                                                                  |   
       |                                                                    |   
  2500 O-O-OO-O-O-O-O-OO-O-O-O-OO-O-O-O-O-OO-O-O-O-O------------------------+   
                                                                                
                                                                                                                                                                
                  vm-scalability.time.percent_of_cpu_this_job_got               
                                                                                
  8680 +-+------------------------------------------------------------------+   
       |   OO O        O   O    O                                           |   
  8660 O-O      O O O O  O   O O  O O O O OO O O O O                        |   
  8640 +-+                                                                  |   
       |                                                                    |   
  8620 +-+                                                                  |   
       |                                                                    |   
  8600 +-+                                                                  |   
       |                                                                    |   
  8580 +-+                                                                  |   
  8560 +-+  +    .+.            +        .+                                 |   
       |. .+ + .+   +. +. .+. .+ + .+. .+  :.+.+.    .+.+.+.++.+.+. .+.++.+.|   
  8540 +-+    +       +  +   +    +   +    +     +.++              +        |   
       |                                                                    |   
  8520 +-+------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                         vm-scalability.time.elapsed_time                       
                                                                                
  350 +-+-------------------------------------------------------------------+   
      |       .+.                                                           |   
  345 +-+.+.++   +.+.+.++.+.+.+.+.++.+.+.+.+.+.++.+.+.+.+.++.+.+.+.+.++.+.+.|   
      |                                                                     |   
      |                                                                     |   
  340 +-+                                                                   |   
      |                                                                     |   
  335 +-+                                                                   |   
      |                                                                     |   
  330 +-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
  325 +-+        O O   OO          O O       O                              |   
      O O O OO O     O    O O O O O    O O O   OO O                         |   
  320 +-+-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                       vm-scalability.time.elapsed_time.max                     
                                                                                
  350 +-+-------------------------------------------------------------------+   
      |       .+.                                                           |   
  345 +-+.+.++   +.+.+.++.+.+.+.+.++.+.+.+.+.+.++.+.+.+.+.++.+.+.+.+.++.+.+.|   
      |                                                                     |   
      |                                                                     |   
  340 +-+                                                                   |   
      |                                                                     |   
  335 +-+                                                                   |   
      |                                                                     |   
  330 +-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
  325 +-+        O O   OO          O O       O                              |   
      O O O OO O     O    O O O O O    O O O   OO O                         |   
  320 +-+-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                     vm-scalability.time.maximum_resident_set_size              
                                                                                
    7e+07 +-+---------------------------------------------------------------+   
          |.++.+.+.++.+.+.++.+.+.++.+.+.++.+.++.+.+.++.+.+.++.+.+.++.+.+.++.|   
  6.5e+07 +-+                                                               |   
    6e+07 +-+                                                               |   
          |                                                                 |   
  5.5e+07 +-+                                                               |   
          |                                                                 |   
    5e+07 +-+                                                               |   
          |                                                                 |   
  4.5e+07 +-+                                                               |   
    4e+07 +-+                                                               |   
          |                                                                 |   
  3.5e+07 +-+                                                               |   
          O OO O O OO O O OO O O OO O O OO O OO O O O                       |   
    3e+07 +-+---------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                         vm-scalability.time.minor_page_faults                  
                                                                                
  1.4e+08 +-+---------------------------------------------------------------+   
          |.++.+.+.++.+.+.++.+.+.++.+.+.++.+.++.+.+.++.+.+.++.+.+.++.+.+.++.|   
  1.3e+08 +-+                                                               |   
  1.2e+08 +-+                                                               |   
          |                                                                 |   
  1.1e+08 +-+                                                               |   
          |                                                                 |   
    1e+08 +-+                                                               |   
          |                                                                 |   
    9e+07 +-+                                                               |   
    8e+07 +-+                                                               |   
          |                                                                 |   
    7e+07 +-+                                                               |   
          O OO O O OO O O OO O O OO O O OO O OO O O O                       |   
    6e+07 +-+---------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                               vm-scalability.throughput                        
                                                                                
  5.4e+06 +-+---------------------------------------------------------------+   
          |                                                                 |   
  5.2e+06 +-+ .+.+. +. .+.  .+.   +.+.+.  .+.++.+. .+ .+.+. +.+.+.++.+. .++.|   
          | ++     +  +   ++   +.+      ++        +  +     +           +    |   
    5e+06 +-+                                                               |   
          |                                                                 |   
  4.8e+06 +-+                                                               |   
          |                                                                 |   
  4.6e+06 +-+                                                               |   
          |                                                                 |   
  4.4e+06 +-+                                                               |   
          |                                                                 |   
  4.2e+06 O-+  O   O      OO O   O    O  O O    O O                         |   
          | OO   O  O O O      O  O O   O    OO     O                       |   
    4e+06 +-+---------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                            vm-scalability.free_time                            
                                                                                
  38 +-+--------------------------------------------------------------------+   
  36 +-+     .+.+.     .+.                         .+.      .+.+.           |   
     |.+.+.++     +.+.+   ++.+.+.+.+.+.+.++.+.+.+.+   ++.+.+     +.+.++.+.+.|   
  34 +-+                                                                    |   
  32 +-+                                                                    |   
  30 +-+                                                                    |   
  28 +-+                                                                    |   
     |                                                                      |   
  26 +-+                                                                    |   
  24 +-+                                                                    |   
  22 +-+                                                                    |   
  20 +-+                                                                    |   
     |                                                                      |   
  18 O-O O OO O O O O O O OO O O O O O O OO O O O O                         |   
  16 +-+--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                vm-scalability.median                           
                                                                                
  60000 +-+-----------------------------------------------------------------+   
        |.   +.+. .+. +.+. .+   .+.+.+   .+.+.++. .+. .++.+.+.++.+.+.+.++.+.|   
  58000 +-+.+    +   +    +  +.+      +.+        +   +                      |   
        |                                                                   |   
  56000 +-+                                                                 |   
        |                                                                   |   
  54000 +-+                                                                 |   
        |                                                                   |   
  52000 +-+                                                                 |   
        |                                                                   |   
  50000 +-+                                                                 |   
        |                                                                   |   
  48000 O-+      O      O O O  O     O  O      O                            |   
        | O OO O   O OO      O   O    O   O O O  O O                        |   
  46000 +-+------------------------O----------------------------------------+   
                                                                                
                                                                                                                                                                
                                vm-scalability.workload                         
                                                                                
   1.6e+09 +-+--------------------------------------------------------------+   
           |.                .+.      +.      .+.           .+     +.       |   
  1.55e+09 +-++.+.+.++.+.++.+   ++.+.+  +.+.++   ++.+.++.+.+  +.+.+  +.+.++.|   
   1.5e+09 +-+                                                              |   
           |                                                                |   
  1.45e+09 +-+                                                              |   
           |                                                                |   
   1.4e+09 +-+                                                              |   
           |                                                                |   
  1.35e+09 +-+                                                              |   
   1.3e+09 +-+                                                              |   
           |                                                                |   
  1.25e+09 O-OO O O OO O OO O O  O    O O O O  O OO                         |   
           |                    O  O O       O      O                       |   
   1.2e+09 +-+--------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                              pmeter.performance_per_watt                       
                                                                                
  3.7e+06 +-+---------------------------------------------------------------+   
          |  +.+.               .++.     +.+. +.            +. .+. +.       |   
  3.6e+06 +-+    +.++.+.+.++.+.+    +.+.+    +  +.+.++.+.+.+  +   +  +.+.++.|   
  3.5e+06 +-+                                                               |   
          |                                                                 |   
  3.4e+06 +-+                                                               |   
  3.3e+06 +-+                                                               |   
          |                                                                 |   
  3.2e+06 +-+                                                               |   
  3.1e+06 +-+                                                               |   
          |                                                                 |   
    3e+06 +-+                                                               |   
  2.9e+06 O-O             O  O   O                                          |   
          |  O O O OO O O  O   O  O O O OO O OO O O O                       |   
  2.8e+06 +-+---------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-bdw-ep2: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase:
  gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-2018-04-03.cgz/300s/256G/lkp-bdw-ep2/lru-shm-rand/vm-scalability

commit: 
  8055fb607e ("vfs: Convert smackfs to fs_context")
  9d65ac8a35 ("tmpfs, devtmpfs, ramfs, rootfs: Convert to fs_context")

8055fb607ebc82ea 9d65ac8a35a75c7334cfa0f8552 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
           :4           25%           1:4     dmesg.WARNING:at#for_ip_interrupt_entry/0x
         %stddev     %change         %stddev
             \          |                \  
      0.03           -52.0%       0.01        vm-scalability.free_time
     54285           +17.0%      63517        vm-scalability.median
      0.00 ± 27%    +218.7%       0.01 ±  3%  vm-scalability.median_stddev
   4776085           +16.8%    5576527        vm-scalability.throughput
    187.64            -6.7%     175.08        vm-scalability.time.elapsed_time
    187.64            -6.7%     175.08        vm-scalability.time.elapsed_time.max
    750512           -49.9%     376070        vm-scalability.time.maximum_resident_set_size
      5850            -8.4%       5361        vm-scalability.time.percent_of_cpu_this_job_got
    880.16            -6.5%     823.38        vm-scalability.time.system_time
     10097           -15.2%       8563        vm-scalability.time.user_time
     10295           +98.8%      20463        vm-scalability.time.voluntary_context_switches
     33.44            +5.3       38.74        mpstat.cpu.all.idle%
      0.00 ±100%      +0.0        0.00 ± 46%  mpstat.cpu.all.iowait%
    375.53            -1.5%     369.95        pmeter.Average_Active_Power
   1593388            +1.6%    1619557        pmeter.performance_per_watt
      1880            -7.6%       1737        turbostat.Avg_MHz
    184.21            -1.7%     181.07        turbostat.PkgWatt
     23.07            -5.2%      21.88        turbostat.RAMWatt
     33.75           +15.6%      39.00        vmstat.cpu.id
     60.00            -8.8%      54.75        vmstat.cpu.us
  55054325           -49.4%   27836911        vmstat.memory.cache
  75933593           +35.9%  1.032e+08        vmstat.memory.free
      1985           +43.8%       2855 ±  4%  vmstat.system.cs
     11984 ±  2%      +8.9%      13047        slabinfo.cred_jar.active_objs
     11984 ±  2%      +8.9%      13047        slabinfo.cred_jar.num_objs
      4107           +10.4%       4536        slabinfo.pid.active_objs
      4107           +10.4%       4536        slabinfo.pid.num_objs
    238266           -47.5%     125045        slabinfo.radix_tree_node.active_objs
      4278           -47.3%       2254        slabinfo.radix_tree_node.active_slabs
    239600           -47.3%     126312        slabinfo.radix_tree_node.num_objs
      4278           -47.3%       2254        slabinfo.radix_tree_node.num_slabs
    681.50 ±  3%     +35.4%     923.00 ± 10%  slabinfo.skbuff_ext_cache.active_objs
    681.50 ±  3%     +35.4%     923.00 ± 10%  slabinfo.skbuff_ext_cache.num_objs
  54983067           -49.6%   27699207        meminfo.Cached
  53998108           -50.6%   26652818        meminfo.Committed_AS
  53743330           -50.7%   26469267        meminfo.Inactive
  53741872           -50.8%   26467690        meminfo.Inactive(anon)
    201706           -32.7%     135684        meminfo.KReclaimable
  41309028           -54.4%   18841465        meminfo.Mapped
  75285897           +36.3%  1.026e+08        meminfo.MemAvailable
  75802933           +36.1%  1.032e+08        meminfo.MemFree
  56089597           -48.8%   28702330        meminfo.Memused
     91718           -50.0%      45872        meminfo.PageTables
    201706           -32.7%     135684        meminfo.SReclaimable
  53753761           -50.7%   26477700        meminfo.Shmem
    335668           -19.0%     272009        meminfo.Slab
    370692           -44.8%     204627        meminfo.max_used_kB
   6859775 ±  2%     -49.9%    3434155 ±  2%  numa-vmstat.node0.nr_file_pages
   9437339 ±  2%     +36.4%   12875285        numa-vmstat.node0.nr_free_pages
   6702347 ±  3%     -51.1%    3277184 ±  2%  numa-vmstat.node0.nr_inactive_anon
   5127207           -53.6%    2379043        numa-vmstat.node0.nr_mapped
     11593 ±  2%     -47.1%       6132        numa-vmstat.node0.nr_page_table_pages
   6703873 ±  3%     -51.1%    3279148 ±  2%  numa-vmstat.node0.nr_shmem
     26487 ±  3%     -27.4%      19231 ±  4%  numa-vmstat.node0.nr_slab_reclaimable
   6702334 ±  3%     -51.1%    3277168 ±  2%  numa-vmstat.node0.nr_zone_inactive_anon
   6855232 ±  3%     -49.1%    3488488 ±  3%  numa-vmstat.node1.nr_file_pages
   9544721 ±  2%     +35.4%   12924651        numa-vmstat.node1.nr_free_pages
   6702112 ±  3%     -50.2%    3337309 ±  3%  numa-vmstat.node1.nr_inactive_anon
   5115345           -54.6%    2324824        numa-vmstat.node1.nr_mapped
     11143 ±  2%     -51.8%       5368        numa-vmstat.node1.nr_page_table_pages
   6703548 ±  3%     -50.2%    3337861 ±  3%  numa-vmstat.node1.nr_shmem
     23866 ±  4%     -38.3%      14718 ±  6%  numa-vmstat.node1.nr_slab_reclaimable
   6702099 ±  3%     -50.2%    3337298 ±  3%  numa-vmstat.node1.nr_zone_inactive_anon
     78855 ±  6%     -12.1%      69301 ± 10%  softirqs.CPU22.TIMER
     20329 ±  9%     +16.7%      23727 ±  3%  softirqs.CPU28.RCU
     77548 ±  5%     -14.1%      66583 ± 11%  softirqs.CPU34.TIMER
     78632 ±  5%     -13.7%      67887 ± 11%  softirqs.CPU36.TIMER
     77969 ±  5%     -12.0%      68650 ± 10%  softirqs.CPU37.TIMER
     77859 ±  6%     -12.8%      67887 ± 11%  softirqs.CPU41.TIMER
     78802 ±  6%     -13.6%      68051 ± 11%  softirqs.CPU42.TIMER
      9417 ±  4%      +5.1%       9898 ±  5%  softirqs.CPU58.SCHED
     81061 ±  9%     -14.4%      69374 ±  8%  softirqs.CPU65.TIMER
      9601            +9.9%      10552 ±  7%  softirqs.CPU70.SCHED
     77194 ±  5%     -14.2%      66260 ± 11%  softirqs.CPU78.TIMER
     78250 ±  6%     -12.6%      68392 ± 10%  softirqs.CPU80.TIMER
     77574 ±  6%     -12.4%      67951 ± 10%  softirqs.CPU81.TIMER
     76944 ±  5%     -15.2%      65222 ± 13%  softirqs.CPU82.TIMER
     77169 ±  7%     -13.5%      66716 ± 11%  softirqs.CPU85.TIMER
     77530 ±  7%     -13.0%      67466 ± 11%  softirqs.CPU86.TIMER
    120.25           +58.8%     191.00        proc-vmstat.nr_dirtied
   1876748           +36.4%    2559252        proc-vmstat.nr_dirty_background_threshold
   3758164           +36.4%    5124848        proc-vmstat.nr_dirty_threshold
  13747341           -49.5%    6938038        proc-vmstat.nr_file_pages
  18949663           +36.1%   25784733        proc-vmstat.nr_free_pages
  13436775           -50.7%    6629895        proc-vmstat.nr_inactive_anon
    364.25            +8.1%     393.75        proc-vmstat.nr_inactive_file
  10320581           -54.7%    4678579        proc-vmstat.nr_mapped
     22830           -50.3%      11339        proc-vmstat.nr_page_table_pages
  13439754           -50.7%    6632403        proc-vmstat.nr_shmem
     50372           -32.7%      33900        proc-vmstat.nr_slab_reclaimable
     33490            +1.8%      34079        proc-vmstat.nr_slab_unreclaimable
    120.00           +42.9%     171.50        proc-vmstat.nr_written
  13436775           -50.7%    6629895        proc-vmstat.nr_zone_inactive_anon
    364.25            +8.1%     393.75        proc-vmstat.nr_zone_inactive_file
    496.75 ± 13%     +88.5%     936.50 ± 28%  proc-vmstat.numa_huge_pte_updates
    275761 ± 15%     +83.7%     506466 ± 26%  proc-vmstat.numa_pte_updates
    201.00 ± 57%     -52.6%      95.25 ±  6%  interrupts.41:IR-PCI-MSI.1572871-edge.eth0-TxRx-7
    206.75 ± 34%     -53.0%      97.25 ±  7%  interrupts.49:IR-PCI-MSI.1572879-edge.eth0-TxRx-15
    152.00 ± 63%     -42.9%      86.75        interrupts.52:IR-PCI-MSI.1572882-edge.eth0-TxRx-18
    124.75 ± 38%     -30.3%      87.00        interrupts.95:IR-PCI-MSI.1572923-edge.eth0-TxRx-59
      5664 ± 24%     -44.5%       3146 ± 33%  interrupts.CPU10.NMI:Non-maskable_interrupts
      5664 ± 24%     -44.5%       3146 ± 33%  interrupts.CPU10.PMI:Performance_monitoring_interrupts
    638.00 ± 45%     -81.1%     120.50 ± 63%  interrupts.CPU11.RES:Rescheduling_interrupts
    171.75 ± 40%     -65.9%      58.50 ± 38%  interrupts.CPU13.RES:Rescheduling_interrupts
    206.75 ± 34%     -53.0%      97.25 ±  7%  interrupts.CPU15.49:IR-PCI-MSI.1572879-edge.eth0-TxRx-15
    226.50 ±  8%     -55.1%     101.75 ± 41%  interrupts.CPU17.RES:Rescheduling_interrupts
    152.00 ± 63%     -42.9%      86.75        interrupts.CPU18.52:IR-PCI-MSI.1572882-edge.eth0-TxRx-18
    369764 ±  4%      -7.8%     341029 ±  2%  interrupts.CPU22.LOC:Local_timer_interrupts
    128.00 ± 33%    +224.8%     415.75 ± 62%  interrupts.CPU38.RES:Rescheduling_interrupts
      1876 ±  7%     -29.2%       1328 ± 12%  interrupts.CPU43.RES:Rescheduling_interrupts
     82.25 ± 21%    +241.3%     280.75 ± 40%  interrupts.CPU49.RES:Rescheduling_interrupts
    326.00 ± 55%     -72.3%      90.25 ± 79%  interrupts.CPU51.RES:Rescheduling_interrupts
    124.75 ± 38%     -30.3%      87.00        interrupts.CPU59.95:IR-PCI-MSI.1572923-edge.eth0-TxRx-59
    201.00 ± 57%     -52.6%      95.25 ±  6%  interrupts.CPU7.41:IR-PCI-MSI.1572871-edge.eth0-TxRx-7
    116.75 ± 27%    +490.6%     689.50 ±103%  interrupts.CPU70.RES:Rescheduling_interrupts
     60.50 ± 53%    +339.3%     265.75 ± 48%  interrupts.CPU72.RES:Rescheduling_interrupts
     48.50 ± 64%    +862.4%     466.75 ± 60%  interrupts.CPU73.RES:Rescheduling_interrupts
    262.00 ± 14%    +112.4%     556.50 ± 11%  interrupts.TLB:TLB_shootdowns
  27405811 ±  2%     -49.9%   13728230 ±  2%  numa-meminfo.node0.FilePages
  26777106 ±  2%     -51.1%   13101235 ±  2%  numa-meminfo.node0.Inactive
  26776084 ±  2%     -51.1%   13100338 ±  2%  numa-meminfo.node0.Inactive(anon)
    105830 ±  3%     -27.3%      76923 ±  4%  numa-meminfo.node0.KReclaimable
  20465391           -53.7%    9474078        numa-meminfo.node0.Mapped
  37783048           +36.3%   51509577        numa-meminfo.node0.MemFree
  28064518 ±  2%     -48.9%   14344562 ±  2%  numa-meminfo.node0.MemUsed
     46263 ±  2%     -47.1%      24460        numa-meminfo.node0.PageTables
    105830 ±  3%     -27.3%      76923 ±  4%  numa-meminfo.node0.SReclaimable
  26782195 ±  2%     -51.1%   13108197 ±  2%  numa-meminfo.node0.Shmem
    179458 ±  3%     -13.1%     156022 ±  2%  numa-meminfo.node0.Slab
  27392441 ±  3%     -49.1%   13933317 ±  3%  numa-meminfo.node1.FilePages
  26780342 ±  3%     -50.2%   13329288 ±  3%  numa-meminfo.node1.Inactive
  26779908 ±  3%     -50.2%   13328609 ±  3%  numa-meminfo.node1.Inactive(anon)
     95194 ±  4%     -38.2%      58811 ±  6%  numa-meminfo.node1.KReclaimable
  20436709           -54.7%    9260997        numa-meminfo.node1.Mapped
  38207989 ±  2%     +35.4%   51719340        numa-meminfo.node1.MemFree
  27836974 ±  3%     -48.6%   14319050 ±  3%  numa-meminfo.node1.MemUsed
     44414 ±  3%     -51.8%      21393        numa-meminfo.node1.PageTables
     95194 ±  4%     -38.2%      58811 ±  6%  numa-meminfo.node1.SReclaimable
  26785705 ±  3%     -50.2%   13330801 ±  3%  numa-meminfo.node1.Shmem
    155528 ±  4%     -25.4%     116025 ±  3%  numa-meminfo.node1.Slab
     22.15 ±128%     -22.2        0.00        perf-profile.calltrace.cycles-pp.apic_timer_interrupt
     20.96 ±128%     -21.0        0.00        perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt
     16.69 ±127%     -16.7        0.00        perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      5.32 ± 44%      -5.3        0.00        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
      5.32 ± 44%      -5.3        0.00        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.03 ± 51%      -5.0        0.00        perf-profile.calltrace.cycles-pp.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.03 ± 51%      -5.0        0.00        perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64
      5.03 ± 51%      -5.0        0.00        perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.03 ± 51%      -5.0        0.00        perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.01 ± 51%      -5.0        0.00        perf-profile.calltrace.cycles-pp.unmap_vmas.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap
      4.99 ± 51%      -5.0        0.00        perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.unmap_region.__do_munmap.__vm_munmap
      3.85 ± 63%      -3.7        0.19 ±173%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle
      2.81 ± 39%      -2.3        0.52 ± 59%  perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.31 ± 33%      -2.0        0.29 ±101%  perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock
      2.31 ± 33%      -2.0        0.29 ±101%  perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit
      0.00            +1.3        1.25 ± 20%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma
      0.00            +1.3        1.27 ± 19%  perf-profile.calltrace.cycles-pp.clear_page_erms.shmem_getpage_gfp.shmem_fault.__do_fault.__handle_mm_fault
      0.00            +1.3        1.28 ± 20%  perf-profile.calltrace.cycles-pp._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.shmem_alloc_page
      0.00            +1.5        1.51 ±  6%  perf-profile.calltrace.cycles-pp.nrand48_r
      0.00            +1.7        1.65 ± 20%  perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.shmem_alloc_page.shmem_alloc_and_acct_page
      0.00            +1.7        1.72 ± 20%  perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.alloc_pages_vma.shmem_alloc_page.shmem_alloc_and_acct_page.shmem_getpage_gfp
      0.00            +1.8        1.76 ± 20%  perf-profile.calltrace.cycles-pp.alloc_pages_vma.shmem_alloc_page.shmem_alloc_and_acct_page.shmem_getpage_gfp.shmem_fault
      0.00            +1.8        1.81 ± 20%  perf-profile.calltrace.cycles-pp.shmem_alloc_page.shmem_alloc_and_acct_page.shmem_getpage_gfp.shmem_fault.__do_fault
      0.00            +2.1        2.08 ± 20%  perf-profile.calltrace.cycles-pp.shmem_alloc_and_acct_page.shmem_getpage_gfp.shmem_fault.__do_fault.__handle_mm_fault
      0.00            +3.2        3.20 ±  6%  perf-profile.calltrace.cycles-pp.do_rw_once
      0.00            +3.2        3.22 ± 20%  perf-profile.calltrace.cycles-pp.filemap_map_pages.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
      0.00            +4.8        4.83 ± 19%  perf-profile.calltrace.cycles-pp.shmem_getpage_gfp.shmem_fault.__do_fault.__handle_mm_fault.handle_mm_fault
      0.00            +4.9        4.92 ± 19%  perf-profile.calltrace.cycles-pp.shmem_fault.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault
      0.00            +4.9        4.94 ± 19%  perf-profile.calltrace.cycles-pp.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
      0.15 ±173%      +8.3        8.48 ± 19%  perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
      0.00            +8.6        8.56 ± 19%  perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault.do_access
      0.00            +8.8        8.81 ± 19%  perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault.do_access
      0.00            +8.9        8.88 ± 19%  perf-profile.calltrace.cycles-pp.do_page_fault.page_fault.do_access
      0.00            +9.7        9.74 ± 22%  perf-profile.calltrace.cycles-pp.page_fault.do_access
      0.00           +86.0       86.00 ±  2%  perf-profile.calltrace.cycles-pp.do_access
     10007 ± 14%     -25.9%       7412 ±  9%  sched_debug.cfs_rq:/.load.avg
      7701 ± 14%     -20.4%       6130 ± 11%  sched_debug.cfs_rq:/.load.min
    300.23 ± 15%     +65.7%     497.54 ± 14%  sched_debug.cfs_rq:/.load_avg.max
   5164591 ± 15%     -58.4%    2147060 ± 33%  sched_debug.cfs_rq:/.min_vruntime.avg
   5239400 ± 15%     -58.1%    2195327 ± 33%  sched_debug.cfs_rq:/.min_vruntime.max
   5015129 ± 15%     -59.9%    2013293 ± 34%  sched_debug.cfs_rq:/.min_vruntime.min
     44715 ±  9%     -26.1%      33058 ± 26%  sched_debug.cfs_rq:/.min_vruntime.stddev
    211.40 ± 60%    +122.0%     469.33 ± 15%  sched_debug.cfs_rq:/.removed.load_avg.max
      9848 ± 60%    +121.4%      21804 ± 15%  sched_debug.cfs_rq:/.removed.runnable_sum.max
    101.17 ± 63%    +135.7%     238.46 ± 15%  sched_debug.cfs_rq:/.removed.util_avg.max
     16.56 ± 66%    +109.8%      34.74 ± 20%  sched_debug.cfs_rq:/.removed.util_avg.stddev
      9.01 ± 11%     -25.2%       6.75 ±  9%  sched_debug.cfs_rq:/.runnable_load_avg.avg
      9959 ± 15%     -27.2%       7246 ±  9%  sched_debug.cfs_rq:/.runnable_weight.avg
      7701 ± 14%     -20.4%       6130 ± 11%  sched_debug.cfs_rq:/.runnable_weight.min
    206600 ± 12%     -38.7%     126726 ± 44%  sched_debug.cfs_rq:/.spread0.max
     44588 ±  9%     -26.0%      33012 ± 26%  sched_debug.cfs_rq:/.spread0.stddev
    117594 ± 11%     -38.3%      72503 ± 17%  sched_debug.cpu.clock.avg
    117603 ± 11%     -38.3%      72514 ± 17%  sched_debug.cpu.clock.max
    117587 ± 11%     -38.4%      72490 ± 17%  sched_debug.cpu.clock.min
      4.58 ± 29%     +87.3%       8.58 ± 21%  sched_debug.cpu.clock.stddev
    117594 ± 11%     -38.3%      72503 ± 17%  sched_debug.cpu.clock_task.avg
    117603 ± 11%     -38.3%      72514 ± 17%  sched_debug.cpu.clock_task.max
    117587 ± 11%     -38.4%      72490 ± 17%  sched_debug.cpu.clock_task.min
      4.58 ± 29%     +87.3%       8.58 ± 21%  sched_debug.cpu.clock_task.stddev
      8.45 ±  4%     -19.0%       6.84 ± 10%  sched_debug.cpu.cpu_load[0].avg
      8.50 ±  4%     -18.5%       6.93 ±  9%  sched_debug.cpu.cpu_load[1].avg
      8.61 ±  6%     -17.5%       7.11 ±  8%  sched_debug.cpu.cpu_load[2].avg
     46.00 ± 44%     -52.5%      21.83 ±  8%  sched_debug.cpu.cpu_load[2].max
      8.71 ±  7%     -14.1%       7.48 ±  7%  sched_debug.cpu.cpu_load[3].avg
    636.96 ±  9%     -34.8%     415.44 ± 25%  sched_debug.cpu.curr->pid.stddev
      9061 ±  4%     -17.8%       7449 ±  9%  sched_debug.cpu.load.avg
      7701 ± 14%     -20.4%       6130 ± 11%  sched_debug.cpu.load.min
     88165 ± 14%     -50.9%      43299 ± 29%  sched_debug.cpu.nr_load_updates.avg
     94734 ± 13%     -47.3%      49931 ± 24%  sched_debug.cpu.nr_load_updates.max
     82786 ± 16%     -52.5%      39357 ± 33%  sched_debug.cpu.nr_load_updates.min
      0.22 ±  8%     +13.2%       0.25 ±  9%  sched_debug.cpu.nr_running.stddev
      2596 ± 12%     -26.1%       1919 ± 21%  sched_debug.cpu.nr_switches.avg
      0.01 ± 34%     +66.7%       0.01 ± 28%  sched_debug.cpu.nr_uninterruptible.avg
    117588 ± 11%     -38.4%      72490 ± 17%  sched_debug.cpu_clk
    113551 ± 11%     -39.7%      68449 ± 18%  sched_debug.ktime
    118269 ± 11%     -38.1%      73184 ± 17%  sched_debug.sched_clk
 9.024e+09            +7.9%   9.74e+09        perf-stat.i.branch-instructions
  19866561 ±  7%     +14.5%   22748472 ±  3%  perf-stat.i.branch-misses
     54.24           -10.3       43.91        perf-stat.i.cache-miss-rate%
 5.969e+08           -13.0%  5.193e+08        perf-stat.i.cache-misses
 8.353e+08            +5.1%  8.778e+08        perf-stat.i.cache-references
      1934           +45.9%       2822 ±  4%  perf-stat.i.context-switches
      3.58           -15.5%       3.02        perf-stat.i.cpi
 1.652e+11            -7.7%  1.524e+11        perf-stat.i.cpu-cycles
     65.64           +35.7%      89.04        perf-stat.i.cpu-migrations
      2.98            -0.5        2.50        perf-stat.i.dTLB-load-miss-rate%
 4.598e+08            -6.6%  4.293e+08        perf-stat.i.dTLB-load-misses
  1.13e+10            +8.0%   1.22e+10        perf-stat.i.dTLB-loads
 4.402e+09            +8.4%  4.774e+09        perf-stat.i.dTLB-stores
    207400 ± 15%     +44.9%     300452 ± 16%  perf-stat.i.iTLB-loads
 3.969e+10            +8.2%  4.296e+10        perf-stat.i.instructions
      0.33 ±  2%     +11.4%       0.37 ±  2%  perf-stat.i.ipc
    692836            +8.6%     752662        perf-stat.i.minor-faults
 5.794e+08           -14.0%  4.984e+08        perf-stat.i.node-loads
    692963            +8.6%     752688        perf-stat.i.page-faults
     21.03            -2.8%      20.43        perf-stat.overall.MPKI
     71.36           -12.2       59.15        perf-stat.overall.cache-miss-rate%
      4.16           -14.7%       3.55        perf-stat.overall.cpi
    277.07            +6.0%     293.58        perf-stat.overall.cycles-between-cache-misses
      3.90            -0.5        3.40        perf-stat.overall.dTLB-load-miss-rate%
     90.35            -3.2       87.17 ±  2%  perf-stat.overall.iTLB-load-miss-rate%
      0.24           +17.2%       0.28        perf-stat.overall.ipc
      0.69 ± 26%      +0.4        1.06 ±  6%  perf-stat.overall.node-load-miss-rate%
     12552            +1.1%      12694        perf-stat.overall.path-length
 8.991e+09            +8.0%  9.714e+09        perf-stat.ps.branch-instructions
  19970181 ±  7%     +13.7%   22702354 ±  3%  perf-stat.ps.branch-misses
 5.936e+08           -12.8%  5.177e+08        perf-stat.ps.cache-misses
 8.318e+08            +5.2%  8.753e+08        perf-stat.ps.cache-references
      1933           +45.4%       2811 ±  4%  perf-stat.ps.context-switches
 1.645e+11            -7.6%   1.52e+11        perf-stat.ps.cpu-cycles
     65.91           +35.2%      89.09        perf-stat.ps.cpu-migrations
 4.571e+08            -6.4%  4.279e+08        perf-stat.ps.dTLB-load-misses
 1.127e+10            +8.0%  1.217e+10        perf-stat.ps.dTLB-loads
 4.393e+09            +8.4%  4.763e+09        perf-stat.ps.dTLB-stores
    206653 ± 15%     +44.2%     297961 ± 16%  perf-stat.ps.iTLB-loads
 3.956e+10            +8.3%  4.285e+10        perf-stat.ps.instructions
    705038            +7.4%     757158        perf-stat.ps.minor-faults
  5.76e+08           -13.7%  4.968e+08        perf-stat.ps.node-loads
    705038            +7.4%     757158        perf-stat.ps.page-faults
  7.45e+12            +1.1%  7.535e+12        perf-stat.total.instructions



***************************************************************************************************
lkp-bdw-ep2: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase:
  gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-2018-04-03.cgz/300s/16G/lkp-bdw-ep2/shm-xread-rand/vm-scalability

commit: 
  8055fb607e ("vfs: Convert smackfs to fs_context")
  9d65ac8a35 ("tmpfs, devtmpfs, ramfs, rootfs: Convert to fs_context")

8055fb607ebc82ea 9d65ac8a35a75c7334cfa0f8552 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
          1:4          -25%            :4     dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
          1:4          -25%            :4     dmesg.WARNING:stack_recursion
         %stddev     %change         %stddev
             \          |                \  
     58903           -20.3%      46946        vm-scalability.median
   5172452           -20.4%    4118440        vm-scalability.throughput
    345.95            -6.6%     323.08        vm-scalability.time.elapsed_time
    345.95            -6.6%     323.08        vm-scalability.time.elapsed_time.max
    111396            -2.5%     108605        vm-scalability.time.involuntary_context_switches
  65947596           -50.0%   32974445        vm-scalability.time.maximum_resident_set_size
 1.362e+08           -50.0%   68098904        vm-scalability.time.minor_page_faults
      8550            +1.3%       8658        vm-scalability.time.percent_of_cpu_this_job_got
      5087           -50.8%       2504        vm-scalability.time.system_time
     24495            +4.0%      25472        vm-scalability.time.user_time
 1.555e+09           -20.4%  1.238e+09        vm-scalability.workload
   3581144           -20.2%    2857829        pmeter.performance_per_watt
 5.284e+08 ± 28%     -68.8%   1.65e+08 ±100%  cpuidle.C3.time
   1327550 ± 17%     -62.5%     497618 ± 68%  cpuidle.C3.usage
      2.96 ±  3%      -1.2        1.71 ±  3%  mpstat.cpu.all.idle%
     16.71            -7.9        8.83        mpstat.cpu.all.sys%
     80.33            +9.1       89.46        mpstat.cpu.all.usr%
     79.25           +11.4%      88.25        vmstat.cpu.us
  64612119           -48.1%   33517986        vmstat.memory.cache
  55623598           +65.5%   92078368        vmstat.memory.free
   9977135           -48.5%    5142019 ±  4%  numa-numastat.node0.local_node
   9982886           -48.3%    5157733 ±  4%  numa-numastat.node0.numa_hit
  10111629           -48.2%    5237252 ±  4%  numa-numastat.node1.local_node
  10123206           -48.2%    5238834 ±  4%  numa-numastat.node1.numa_hit
     59844 ±  6%     +22.5%      73308 ±  9%  sched_debug.cfs_rq:/.min_vruntime.stddev
     59581 ±  6%     +22.5%      72962 ±  9%  sched_debug.cfs_rq:/.spread0.stddev
      1101 ±  8%     +19.1%       1312 ±  9%  sched_debug.cpu.nr_load_updates.stddev
      3.75 ± 15%     -20.2%       2.99 ±  9%  sched_debug.cpu.nr_uninterruptible.stddev
     21350 ±  4%      -8.9%      19448 ±  4%  slabinfo.filp.active_objs
     21407 ±  4%      -8.8%      19515 ±  4%  slabinfo.filp.num_objs
    272341           -46.7%     145219        slabinfo.radix_tree_node.active_objs
      4924           -47.0%       2609        slabinfo.radix_tree_node.active_slabs
    275782           -47.0%     146146        slabinfo.radix_tree_node.num_objs
      4924           -47.0%       2609        slabinfo.radix_tree_node.num_slabs
      2709            +1.2%       2741        turbostat.Avg_MHz
   1327367 ± 17%     -62.5%     497393 ± 68%  turbostat.C3
      1.73 ± 28%      -1.2        0.57 ±101%  turbostat.C3%
      1.33 ± 11%     -43.2%       0.76 ± 19%  turbostat.CPU%c1
      1.06 ± 40%     -76.8%       0.24 ±131%  turbostat.CPU%c3
      0.84 ±  6%     -36.3%       0.53 ± 11%  turbostat.Pkg%pc2
     26.27            +3.1%      27.10        turbostat.RAMWatt
      1.72 ±  8%      -0.4        1.29 ± 18%  perf-profile.calltrace.cycles-pp.cmd_record.run_builtin.main.generic_start_main.__strcspn_sse42
      1.29 ±  8%      -0.3        0.96 ± 21%  perf-profile.calltrace.cycles-pp.record__mmap_read_evlist.cmd_record.run_builtin.main.generic_start_main
      1.11 ±  9%      -0.2        0.86 ± 21%  perf-profile.calltrace.cycles-pp.__libc_write.record__pushfn.perf_mmap__push.record__mmap_read_evlist.cmd_record
      0.00            +0.6        0.61 ± 11%  perf-profile.calltrace.cycles-pp.intel_pmu_disable_all.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt
      1.31 ± 64%      +1.3        2.62 ± 34%  perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.printk.irq_work_run_list.irq_work_run
      1.31 ± 64%      +1.5        2.84 ± 28%  perf-profile.calltrace.cycles-pp.printk.irq_work_run_list.irq_work_run.smp_irq_work_interrupt.irq_work_interrupt
      1.31 ± 64%      +1.5        2.84 ± 28%  perf-profile.calltrace.cycles-pp.vprintk_emit.printk.irq_work_run_list.irq_work_run.smp_irq_work_interrupt
      0.57 ±112%      +1.8        2.37 ± 33%  perf-profile.calltrace.cycles-pp.irq_work_interrupt
      0.57 ±112%      +1.8        2.37 ± 33%  perf-profile.calltrace.cycles-pp.smp_irq_work_interrupt.irq_work_interrupt
      0.56 ±111%      +1.8        2.37 ± 33%  perf-profile.calltrace.cycles-pp.irq_work_run_list.irq_work_run.smp_irq_work_interrupt.irq_work_interrupt
      0.56 ±111%      +1.8        2.37 ± 33%  perf-profile.calltrace.cycles-pp.irq_work_run.smp_irq_work_interrupt.irq_work_interrupt
   4619861           -69.2%    1421166        meminfo.Active
   4619538           -69.2%    1420846        meminfo.Active(anon)
  64499750           -48.1%   33455359        meminfo.Cached
  64314136           -49.3%   32618183        meminfo.Committed_AS
   7686210 ±  5%     -13.6%    6642659 ±  7%  meminfo.DirectMap2M
  58895884           -47.3%   31056924        meminfo.Inactive
  58894602           -47.3%   31055638        meminfo.Inactive(anon)
    222659           -33.3%     148512        meminfo.KReclaimable
  58840690           -47.3%   31002708        meminfo.Mapped
  54984886           +66.3%   91436602        meminfo.MemAvailable
  55491527           +65.8%   91980428        meminfo.MemFree
  76401003           -47.8%   39912102        meminfo.Memused
  10891829           -49.2%    5531713        meminfo.PageTables
    222659           -33.3%     148512        meminfo.SReclaimable
  63270909           -49.1%   32234167        meminfo.Shmem
    352176           -21.2%     277620        meminfo.Slab
    237668           -44.8%     131115        meminfo.max_used_kB
   1151943           -69.3%     353978        proc-vmstat.nr_active_anon
   1370541           +66.5%    2281328        proc-vmstat.nr_dirty_background_threshold
   2744434           +66.5%    4568236        proc-vmstat.nr_dirty_threshold
  16120020           -48.1%    8358289        proc-vmstat.nr_file_pages
  13879822           +65.7%   23001086        proc-vmstat.nr_free_pages
  14721414           -47.3%    7759335        proc-vmstat.nr_inactive_anon
  14708048           -47.3%    7746354        proc-vmstat.nr_mapped
   2720928           -49.2%    1382900        proc-vmstat.nr_page_table_pages
  15812549           -49.1%    8052730        proc-vmstat.nr_shmem
     55636           -33.3%      37097        proc-vmstat.nr_slab_reclaimable
   1151943           -69.3%     353978        proc-vmstat.nr_zone_active_anon
  14721414           -47.3%    7759335        proc-vmstat.nr_zone_inactive_anon
  20133051           -48.2%   10423588        proc-vmstat.numa_hit
  20115718           -48.3%   10406285        proc-vmstat.numa_local
  16498700           -50.0%    8255732        proc-vmstat.pgactivate
  20224085           -48.1%   10500895        proc-vmstat.pgalloc_normal
 1.371e+08           -49.7%   68932918        proc-vmstat.pgfault
  20127362           -48.1%   10441339        proc-vmstat.pgfree
    138366 ±  2%     -11.0%     123211 ±  5%  softirqs.CPU0.TIMER
    149690 ± 12%     -18.8%     121621 ±  6%  softirqs.CPU1.TIMER
    137396 ±  2%     -11.0%     122245 ±  7%  softirqs.CPU3.TIMER
     39705 ±  4%      -7.8%      36624        softirqs.CPU34.RCU
    145711 ±  7%     -15.4%     123281 ±  5%  softirqs.CPU4.TIMER
    149597 ±  6%     -16.0%     125654 ±  8%  softirqs.CPU45.TIMER
    140697 ±  3%     -12.1%     123678 ±  5%  softirqs.CPU46.TIMER
    139246 ±  3%     -12.0%     122571 ±  7%  softirqs.CPU47.TIMER
    148109 ± 11%     -17.4%     122387 ±  5%  softirqs.CPU48.TIMER
    144373 ±  3%     -13.8%     124384 ±  6%  softirqs.CPU50.TIMER
    142611 ±  3%     -12.9%     124165 ±  7%  softirqs.CPU51.TIMER
     35209 ±  5%      +7.6%      37869 ±  3%  softirqs.CPU53.RCU
    135035 ±  2%     -11.5%     119518 ±  5%  softirqs.CPU55.TIMER
    143106 ±  3%     -13.5%     123842 ±  6%  softirqs.CPU6.TIMER
    140285 ±  4%     -12.8%     122376 ±  6%  softirqs.CPU60.TIMER
    138943 ±  4%     -13.0%     120840 ±  6%  softirqs.CPU62.TIMER
     32370 ±  6%      -9.9%      29180 ±  3%  softirqs.CPU63.RCU
     36484 ±  7%     -11.2%      32394 ±  3%  softirqs.CPU64.RCU
    140860 ±  2%     -12.5%     123309 ±  7%  softirqs.CPU7.TIMER
    336771           -15.4%     284753 ±  2%  softirqs.SCHED
  12108815            -8.2%   11115408 ±  4%  softirqs.TIMER
    452.50 ± 73%     -62.2%     171.25 ±  3%  interrupts.37:IR-PCI-MSI.1572867-edge.eth0-TxRx-3
    486.50 ±106%     -64.0%     175.25        interrupts.48:IR-PCI-MSI.1572878-edge.eth0-TxRx-14
    174.75 ±  2%      -9.2%     158.75        interrupts.79:IR-PCI-MSI.1572907-edge.eth0-TxRx-43
    332281 ±  8%     -10.6%     296936        interrupts.CAL:Function_call_interrupts
    486.50 ±106%     -64.0%     175.25        interrupts.CPU14.48:IR-PCI-MSI.1572878-edge.eth0-TxRx-14
    300.50 ± 58%    +322.0%       1268 ± 45%  interrupts.CPU14.RES:Rescheduling_interrupts
    256.75 ± 52%    +251.6%     902.75 ± 66%  interrupts.CPU16.RES:Rescheduling_interrupts
    589.50 ± 56%     -83.0%     100.25 ± 50%  interrupts.CPU22.RES:Rescheduling_interrupts
      1365 ± 70%     -87.9%     165.00 ± 84%  interrupts.CPU26.RES:Rescheduling_interrupts
    452.50 ± 73%     -62.2%     171.25 ±  3%  interrupts.CPU3.37:IR-PCI-MSI.1572867-edge.eth0-TxRx-3
      3862 ±  7%     -12.1%       3394        interrupts.CPU30.CAL:Function_call_interrupts
    866.75 ± 75%     -70.1%     258.75 ± 40%  interrupts.CPU30.RES:Rescheduling_interrupts
    598.75 ± 12%     -59.1%     244.75 ± 41%  interrupts.CPU32.RES:Rescheduling_interrupts
    596.00 ± 40%     -65.3%     206.75 ± 33%  interrupts.CPU36.RES:Rescheduling_interrupts
      3847 ±  8%     -11.2%       3418        interrupts.CPU38.CAL:Function_call_interrupts
    956.00 ± 60%     -74.3%     245.50 ± 58%  interrupts.CPU38.RES:Rescheduling_interrupts
    822.75 ± 66%     -68.4%     260.25 ± 83%  interrupts.CPU39.RES:Rescheduling_interrupts
    751.00 ± 49%     -64.8%     264.50 ± 61%  interrupts.CPU40.RES:Rescheduling_interrupts
      3844 ±  8%     -12.5%       3362        interrupts.CPU41.CAL:Function_call_interrupts
    174.75 ±  2%      -9.2%     158.75        interrupts.CPU43.79:IR-PCI-MSI.1572907-edge.eth0-TxRx-43
      4278 ±  3%     -11.8%       3774 ±  3%  interrupts.CPU43.RES:Rescheduling_interrupts
      3865 ±  7%     -11.1%       3437        interrupts.CPU74.CAL:Function_call_interrupts
    252.50 ±147%     -96.9%       7.75 ± 29%  interrupts.CPU85.RES:Rescheduling_interrupts
    139.50 ± 19%     -59.1%      57.00 ±  3%  interrupts.TLB:TLB_shootdowns
   2363099           -71.2%     680111 ±  4%  numa-meminfo.node0.Active
   2362909           -71.2%     679995 ±  4%  numa-meminfo.node0.Active(anon)
    111665 ± 13%     -34.0%      73698 ± 38%  numa-meminfo.node0.AnonHugePages
    148248 ± 15%     -29.2%     105012 ± 31%  numa-meminfo.node0.AnonPages
  32253198           -49.0%   16433879 ±  2%  numa-meminfo.node0.FilePages
  29422844           -48.1%   15256374 ±  2%  numa-meminfo.node0.Inactive
  29422610           -48.2%   15255530 ±  2%  numa-meminfo.node0.Inactive(anon)
    116308 ± 24%     -50.9%      57155 ± 37%  numa-meminfo.node0.KReclaimable
  29376475           -48.1%   15241543 ±  2%  numa-meminfo.node0.Mapped
  27577954           +68.8%   46544763        numa-meminfo.node0.MemFree
  38269612           -49.5%   19322519 ±  4%  numa-meminfo.node0.MemUsed
   5406782 ±  5%     -55.3%    2416882 ± 14%  numa-meminfo.node0.PageTables
    116308 ± 24%     -50.9%      57155 ± 37%  numa-meminfo.node0.SReclaimable
  31637547           -50.0%   15830617 ±  2%  numa-meminfo.node0.Shmem
    188033 ± 14%     -32.4%     127088 ± 18%  numa-meminfo.node0.Slab
   2312677           -67.3%     755238 ±  3%  numa-meminfo.node1.Active
   2312544           -67.4%     755034 ±  3%  numa-meminfo.node1.Active(anon)
  32263670           -47.2%   17036666 ±  2%  numa-meminfo.node1.FilePages
  29433165           -46.3%   15800516 ±  2%  numa-meminfo.node1.Inactive
  29432116           -46.3%   15800075 ±  2%  numa-meminfo.node1.Inactive(anon)
  29424249           -46.4%   15760881 ±  2%  numa-meminfo.node1.Mapped
  27898977           +62.8%   45416266        numa-meminfo.node1.MemFree
  38145985           -46.0%   20608980 ±  3%  numa-meminfo.node1.MemUsed
   5485105 ±  5%     -43.1%    3119610 ± 11%  numa-meminfo.node1.PageTables
  31649432           -48.1%   16417690 ±  2%  numa-meminfo.node1.Shmem
    588255           -71.2%     169477 ±  4%  numa-vmstat.node0.nr_active_anon
     37057 ± 15%     -29.2%      26250 ± 31%  numa-vmstat.node0.nr_anon_pages
   8055464           -49.0%    4105442 ±  2%  numa-vmstat.node0.nr_file_pages
   6904034           +68.6%   11639744        numa-vmstat.node0.nr_free_pages
   7350295           -48.1%    3811386 ±  2%  numa-vmstat.node0.nr_inactive_anon
   7338831           -48.1%    3808003 ±  2%  numa-vmstat.node0.nr_mapped
   1349770 ±  5%     -55.3%     603819 ± 14%  numa-vmstat.node0.nr_page_table_pages
   7901552           -50.0%    3954627 ±  2%  numa-vmstat.node0.nr_shmem
     29044 ± 24%     -50.8%      14282 ± 37%  numa-vmstat.node0.nr_slab_reclaimable
    588255           -71.2%     169477 ±  4%  numa-vmstat.node0.nr_zone_active_anon
   7350295           -48.1%    3811386 ±  2%  numa-vmstat.node0.nr_zone_inactive_anon
  10146868           -47.4%    5340924 ±  3%  numa-vmstat.node0.numa_hit
  10140885           -47.5%    5324695 ±  3%  numa-vmstat.node0.numa_local
    575687           -67.3%     188231 ±  3%  numa-vmstat.node1.nr_active_anon
   8058042           -47.2%    4256182 ±  2%  numa-vmstat.node1.nr_file_pages
   6984514           +62.6%   11357293        numa-vmstat.node1.nr_free_pages
   7352606           -46.3%    3947550 ±  2%  numa-vmstat.node1.nr_inactive_anon
   7350701           -46.4%    3937932 ±  2%  numa-vmstat.node1.nr_mapped
   1369252 ±  5%     -43.1%     779777 ± 11%  numa-vmstat.node1.nr_page_table_pages
   7904483           -48.1%    4101439 ±  2%  numa-vmstat.node1.nr_shmem
    575687           -67.3%     188232 ±  3%  numa-vmstat.node1.nr_zone_active_anon
   7352606           -46.3%    3947550 ±  2%  numa-vmstat.node1.nr_zone_inactive_anon
  10040888           -45.5%    5472600 ±  3%  numa-vmstat.node1.numa_hit
   9853575           -46.3%    5296236 ±  3%  numa-vmstat.node1.numa_local
    187312 ±  3%      -5.8%     176363        numa-vmstat.node1.numa_other
     27.34            +3.1%      28.20        perf-stat.i.MPKI
 8.864e+09            -2.8%   8.62e+09        perf-stat.i.branch-instructions
      0.14 ± 11%      -0.0        0.10 ±  5%  perf-stat.i.branch-miss-rate%
  10769953           -28.8%    7671610 ±  2%  perf-stat.i.branch-misses
     63.11            +4.6       67.74        perf-stat.i.cache-miss-rate%
 6.599e+08            +7.4%   7.09e+08        perf-stat.i.cache-misses
 9.689e+08            +3.7%  1.005e+09        perf-stat.i.cache-references
      6.40            +3.1%       6.60        perf-stat.i.cpi
 2.378e+11            +1.3%  2.408e+11        perf-stat.i.cpu-cycles
     50.96 ±  9%     +18.2%      60.24 ±  4%  perf-stat.i.cpu-migrations
    661.83           -24.2%     501.47        perf-stat.i.cycles-between-cache-misses
      4.41            +0.5        4.87        perf-stat.i.dTLB-load-miss-rate%
  4.72e+08           +11.5%  5.265e+08        perf-stat.i.dTLB-load-misses
 3.311e+09            +4.5%   3.46e+09        perf-stat.i.dTLB-stores
   1137454 ± 40%     -52.2%     543479 ± 20%  perf-stat.i.iTLB-load-misses
     29208 ± 20%     -27.1%      21301 ±  7%  perf-stat.i.iTLB-loads
 3.802e+10            -2.6%  3.701e+10        perf-stat.i.instructions
      0.17            -5.5%       0.16        perf-stat.i.ipc
    394369           -46.2%     212356        perf-stat.i.minor-faults
 1.777e+08           +16.3%  2.067e+08 ±  5%  perf-stat.i.node-load-misses
   2262667           -45.2%    1240313        perf-stat.i.node-store-misses
   1570312           -43.7%     883357        perf-stat.i.node-stores
    394373           -46.2%     212361        perf-stat.i.page-faults
     25.41            +6.7%      27.11        perf-stat.overall.MPKI
      0.12            -0.0        0.09 ±  3%  perf-stat.overall.branch-miss-rate%
     68.09            +2.4       70.53        perf-stat.overall.cache-miss-rate%
      6.25            +4.1%       6.50        perf-stat.overall.cpi
    361.17            -5.8%     340.10        perf-stat.overall.cycles-between-cache-misses
      4.20            +0.5        4.74        perf-stat.overall.dTLB-load-miss-rate%
     97.33            -1.2       96.11        perf-stat.overall.iTLB-load-miss-rate%
     37498 ± 27%     +86.9%      70100 ± 15%  perf-stat.overall.instructions-per-iTLB-miss
      0.16            -3.9%       0.15        perf-stat.overall.ipc
      8461           +14.1%       9653        perf-stat.overall.path-length
 8.848e+09            -2.8%  8.599e+09        perf-stat.ps.branch-instructions
  10826438           -28.7%    7715583 ±  2%  perf-stat.ps.branch-misses
 6.566e+08            +7.5%  7.059e+08        perf-stat.ps.cache-misses
 9.644e+08            +3.8%  1.001e+09        perf-stat.ps.cache-references
 2.371e+11            +1.2%    2.4e+11        perf-stat.ps.cpu-cycles
     50.79 ±  9%     +18.2%      60.04 ±  4%  perf-stat.ps.cpu-migrations
 4.695e+08           +11.6%  5.242e+08        perf-stat.ps.dTLB-load-misses
 3.297e+09            +4.5%  3.447e+09        perf-stat.ps.dTLB-stores
   1138634 ± 40%     -52.2%     544577 ± 20%  perf-stat.ps.iTLB-load-misses
     29201 ± 20%     -27.0%      21308 ±  7%  perf-stat.ps.iTLB-loads
 3.795e+10            -2.7%  3.692e+10        perf-stat.ps.instructions
    395278           -46.1%     212887        perf-stat.ps.minor-faults
 1.768e+08           +16.4%  2.058e+08 ±  5%  perf-stat.ps.node-load-misses
   2273468           -45.1%    1248526        perf-stat.ps.node-store-misses
   1578735           -43.7%     888814        perf-stat.ps.node-stores
    395278           -46.1%     212887        perf-stat.ps.page-faults
 1.316e+13            -9.2%  1.195e+13        perf-stat.total.instructions



***************************************************************************************************
lkp-bdw-ep2: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase:
  gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-2018-04-03.cgz/300s/2T/lkp-bdw-ep2/shm-xread-seq-mt/vm-scalability

commit: 
  8055fb607e ("vfs: Convert smackfs to fs_context")
  9d65ac8a35 ("tmpfs, devtmpfs, ramfs, rootfs: Convert to fs_context")

8055fb607ebc82ea 9d65ac8a35a75c7334cfa0f8552 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    196632            -3.4%     190033        vm-scalability.median
  17303638            -3.4%   16722968        vm-scalability.throughput
    305.58           -35.0%     198.68        vm-scalability.time.elapsed_time
    305.58           -35.0%     198.68        vm-scalability.time.elapsed_time.max
     51178 ±  3%     -35.2%      33182 ±  2%  vm-scalability.time.involuntary_context_switches
  50826393           -35.1%   32976181        vm-scalability.time.maximum_resident_set_size
 1.918e+08           -35.1%  1.245e+08        vm-scalability.time.minor_page_faults
      4352           -35.0%       2830        vm-scalability.time.system_time
      2028           -35.1%       1317        vm-scalability.time.user_time
 2.055e+08           -35.2%  1.332e+08        vm-scalability.time.voluntary_context_switches
 5.198e+09           -37.2%  3.264e+09        vm-scalability.workload
     26.86 ± 13%     -14.1%      23.07        boot-time.dhcp
  14360030           -37.1%    9036532        pmeter.performance_per_watt
  26669923           -33.6%   17698248        vmstat.memory.cache
      5.28            +0.5%       5.31        perf-stat.overall.MPKI
      3897            +3.4%       4029        perf-stat.overall.path-length
 2.026e+13           -35.1%  1.315e+13        perf-stat.total.instructions
  11133423           -35.0%    7236823        numa-numastat.node0.local_node
  11140682           -35.0%    7241254        numa-numastat.node0.numa_hit
  11164060           -35.1%    7246687        numa-numastat.node1.local_node
  11174175           -35.0%    7259603        numa-numastat.node1.numa_hit
  1.37e+08           -36.3%   87306795 ±  2%  turbostat.C1
  98317400 ±  2%     -33.7%   65227313 ±  3%  turbostat.C1E
   3598865 ±  4%     -30.4%    2503778 ± 13%  turbostat.C3
    624349 ± 18%     -52.6%     296068 ± 42%  turbostat.C6
  57984931           -34.5%   37968501        turbostat.IRQ
    618.75 ±  4%     +28.1%     792.75 ± 11%  slabinfo.file_lock_cache.active_objs
    618.75 ±  4%     +28.1%     792.75 ± 11%  slabinfo.file_lock_cache.num_objs
    116482           -30.4%      81085        slabinfo.radix_tree_node.active_objs
      2081           -30.4%       1448        slabinfo.radix_tree_node.active_slabs
    116572           -30.4%      81165        slabinfo.radix_tree_node.num_objs
      2081           -30.4%       1448        slabinfo.radix_tree_node.num_slabs
 1.043e+10 ±  2%     -34.1%  6.874e+09        cpuidle.C1.time
  1.37e+08           -36.3%   87314564 ±  2%  cpuidle.C1.usage
 8.491e+09 ±  3%     -36.1%  5.423e+09        cpuidle.C1E.time
  98317790 ±  2%     -33.7%   65227663 ±  3%  cpuidle.C1E.usage
   3599022 ±  4%     -30.4%    2504208 ± 13%  cpuidle.C3.usage
  4.81e+08 ± 22%     -51.2%  2.345e+08 ± 47%  cpuidle.C6.time
    639440 ± 18%     -52.1%     306262 ± 40%  cpuidle.C6.usage
 1.554e+08           -35.0%  1.011e+08        cpuidle.POLL.time
  41971317 ±  2%     -35.1%   27250494 ±  2%  cpuidle.POLL.usage
    171264           -11.7%     151295        meminfo.AnonHugePages
  26614721           -33.6%   17675359        meminfo.Cached
  26464318           -33.9%   17494063        meminfo.Committed_AS
  25363226           -35.2%   16441470        meminfo.Inactive
  25361926           -35.2%   16440177        meminfo.Inactive(anon)
    130801           -16.1%     109804        meminfo.KReclaimable
  24900911           -35.2%   16124722        meminfo.Mapped
  27556910           -32.5%   18604729        meminfo.Memused
     53313           -32.3%      36085        meminfo.PageTables
    130801           -16.1%     109804        meminfo.SReclaimable
  25386096           -35.2%   16454135        meminfo.Shmem
  13374369           -33.6%    8875998        numa-meminfo.node0.FilePages
  12746980           -35.2%    8253690        numa-meminfo.node0.Inactive
  12746045           -35.3%    8252964        numa-meminfo.node0.Inactive(anon)
  12511393           -35.3%    8091983        numa-meminfo.node0.Mapped
  13923310           -32.2%    9433317        numa-meminfo.node0.MemUsed
     26875 ±  4%     -30.5%      18681 ±  5%  numa-meminfo.node0.PageTables
  12758310           -35.2%    8262068        numa-meminfo.node0.Shmem
  13267153           -33.7%    8791711        numa-meminfo.node1.FilePages
  12642015           -35.3%    8179158        numa-meminfo.node1.Inactive
  12641650           -35.3%    8178592        numa-meminfo.node1.Inactive(anon)
     59135 ± 10%     -19.1%      47862 ±  9%  numa-meminfo.node1.KReclaimable
  12416665           -35.3%    8029295        numa-meminfo.node1.Mapped
  13659837           -32.9%    9163031        numa-meminfo.node1.MemUsed
     26452 ±  4%     -34.2%      17416 ±  6%  numa-meminfo.node1.PageTables
     59135 ± 10%     -19.1%      47862 ±  9%  numa-meminfo.node1.SReclaimable
  12653543           -35.3%    8183377        numa-meminfo.node1.Shmem
   3342451           -33.6%    2220433        numa-vmstat.node0.nr_file_pages
   3185349           -35.2%    2064637        numa-vmstat.node0.nr_inactive_anon
   3127155           -35.3%    2022977        numa-vmstat.node0.nr_mapped
      6720 ±  4%     -30.4%       4674 ±  5%  numa-vmstat.node0.nr_page_table_pages
   3188436           -35.2%    2066951        numa-vmstat.node0.nr_shmem
   3185348           -35.2%    2064635        numa-vmstat.node0.nr_zone_inactive_anon
   6174756           -31.8%    4213820        numa-vmstat.node0.numa_hit
   6167259           -31.7%    4209178        numa-vmstat.node0.numa_local
   3315635           -33.7%    2199341        numa-vmstat.node1.nr_file_pages
   3159256           -35.2%    2046053        numa-vmstat.node1.nr_inactive_anon
   3103543           -35.3%    2007364        numa-vmstat.node1.nr_mapped
      6615 ±  4%     -34.1%       4359 ±  6%  numa-vmstat.node1.nr_page_table_pages
   3162233           -35.3%    2047258        numa-vmstat.node1.nr_shmem
     14782 ± 10%     -19.1%      11965 ±  9%  numa-vmstat.node1.nr_slab_reclaimable
   3159256           -35.2%    2046052        numa-vmstat.node1.nr_zone_inactive_anon
   6053005           -33.3%    4037539        numa-vmstat.node1.numa_hit
   5867194           -34.4%    3849828        numa-vmstat.node1.numa_local
     65705            -4.1%      63028        proc-vmstat.nr_active_anon
   2588819            +8.7%    2813043        proc-vmstat.nr_dirty_background_threshold
   5183970            +8.7%    5632966        proc-vmstat.nr_dirty_threshold
   6657180           -33.7%    4414871        proc-vmstat.nr_file_pages
  26080478            +8.6%   28326011        proc-vmstat.nr_free_pages
   6343697           -35.3%    4105826        proc-vmstat.nr_inactive_anon
   6229857           -35.3%    4029578        proc-vmstat.nr_mapped
     13336           -32.3%       9026        proc-vmstat.nr_page_table_pages
   6349763           -35.3%    4109305        proc-vmstat.nr_shmem
     32703           -16.1%      27439        proc-vmstat.nr_slab_reclaimable
     65705            -4.1%      63028        proc-vmstat.nr_zone_active_anon
   6343697           -35.3%    4105826        proc-vmstat.nr_zone_inactive_anon
    106482 ±  5%     -36.3%      67843 ±  4%  proc-vmstat.numa_hint_faults
     77337 ±  6%     -36.2%      49325 ±  5%  proc-vmstat.numa_hint_faults_local
  22342557           -35.0%   14525355        proc-vmstat.numa_hit
  22325169           -35.0%   14508002        proc-vmstat.numa_local
  22424699           -34.9%   14601937        proc-vmstat.pgalloc_normal
 1.926e+08           -35.1%   1.25e+08        proc-vmstat.pgfault
  21941855 ±  3%     -34.9%   14276115 ±  2%  proc-vmstat.pgfree
     10484 ± 11%     -26.3%       7724 ±  9%  sched_debug.cfs_rq:/.load.avg
     87787 ± 84%     -52.8%      41420 ±  3%  sched_debug.cfs_rq:/.load.max
     21156 ± 32%     -29.4%      14934 ±  4%  sched_debug.cfs_rq:/.load.stddev
    829431           -39.3%     503257        sched_debug.cfs_rq:/.min_vruntime.avg
    885846           -37.4%     554313        sched_debug.cfs_rq:/.min_vruntime.max
    812888           -39.6%     490860        sched_debug.cfs_rq:/.min_vruntime.min
      0.24 ±  9%     -20.8%       0.19 ±  9%  sched_debug.cfs_rq:/.nr_running.avg
     10403 ± 11%     -25.5%       7752 ±  9%  sched_debug.cfs_rq:/.runnable_weight.avg
     86935 ± 85%     -55.1%      39045        sched_debug.cfs_rq:/.runnable_weight.max
     21021 ± 32%     -29.3%      14857 ±  3%  sched_debug.cfs_rq:/.runnable_weight.stddev
    101.34           +17.4%     118.95 ±  4%  sched_debug.cfs_rq:/.util_avg.stddev
     56.89 ±  7%     -24.1%      43.20 ±  5%  sched_debug.cfs_rq:/.util_est_enqueued.avg
    109.31 ±  4%     -12.6%      95.54        sched_debug.cfs_rq:/.util_est_enqueued.stddev
    195835           +24.1%     242941        sched_debug.cpu.avg_idle.avg
    347207 ±  4%     +40.1%     486352 ± 12%  sched_debug.cpu.avg_idle.max
     24654 ±  7%     +25.9%      31044 ±  3%  sched_debug.cpu.avg_idle.min
     62728 ±  2%     +35.0%      84672 ±  5%  sched_debug.cpu.avg_idle.stddev
    188539           -33.7%     125063        sched_debug.cpu.clock.avg
    188544           -33.7%     125068        sched_debug.cpu.clock.max
    188535           -33.7%     125059        sched_debug.cpu.clock.min
    188539           -33.7%     125063        sched_debug.cpu.clock_task.avg
    188544           -33.7%     125068        sched_debug.cpu.clock_task.max
    188535           -33.7%     125059        sched_debug.cpu.clock_task.min
      1.48 ± 10%     +23.5%       1.83 ±  4%  sched_debug.cpu.cpu_load[4].avg
     46.33 ± 23%     +51.5%      70.19 ± 14%  sched_debug.cpu.cpu_load[4].max
      5.42 ± 16%     +43.6%       7.78 ± 12%  sched_debug.cpu.cpu_load[4].stddev
      5561 ±  3%     -24.8%       4179        sched_debug.cpu.curr->pid.max
      1024 ±  5%     -17.9%     841.52 ±  2%  sched_debug.cpu.curr->pid.stddev
     11106 ±  7%     -24.9%       8336 ±  5%  sched_debug.cpu.load.avg
    129322 ±110%     -67.5%      41984        sched_debug.cpu.load.max
     24962 ± 51%     -38.6%      15338 ±  2%  sched_debug.cpu.load.stddev
    156214           -38.7%      95691        sched_debug.cpu.nr_load_updates.avg
    163995           -38.0%     101745        sched_debug.cpu.nr_load_updates.max
    154518           -39.0%      94297        sched_debug.cpu.nr_load_updates.min
      0.42 ±  3%      -7.1%       0.39 ±  2%  sched_debug.cpu.nr_running.stddev
   2318901           -40.3%    1383764        sched_debug.cpu.nr_switches.avg
   2333427           -40.2%    1395703        sched_debug.cpu.nr_switches.max
   2271583           -40.8%    1344471        sched_debug.cpu.nr_switches.min
      9275 ± 10%     -25.3%       6932 ±  3%  sched_debug.cpu.nr_switches.stddev
    797.75 ± 20%     -47.2%     421.50 ± 27%  sched_debug.cpu.nr_uninterruptible.max
    -83.38           -40.5%     -49.62        sched_debug.cpu.nr_uninterruptible.min
    103.08 ± 19%     -46.9%      54.75 ± 26%  sched_debug.cpu.nr_uninterruptible.stddev
    188535           -33.7%     125059        sched_debug.cpu_clk
    184490           -34.4%     121011        sched_debug.ktime
    189336           -33.6%     125753        sched_debug.sched_clk
     23387 ± 11%     -46.4%      12529 ±  8%  softirqs.CPU0.RCU
     53140 ±  2%     -32.1%      36067 ±  3%  softirqs.CPU0.SCHED
    105272           -33.0%      70585        softirqs.CPU0.TIMER
     23286 ±  6%     -43.3%      13211 ±  4%  softirqs.CPU1.RCU
     49935           -34.1%      32901        softirqs.CPU1.SCHED
     22010 ±  5%     -40.8%      13029 ±  8%  softirqs.CPU10.RCU
     48822           -34.2%      32141        softirqs.CPU10.SCHED
     90827           -33.3%      60569        softirqs.CPU10.TIMER
     21040 ±  3%     -38.2%      13007 ±  5%  softirqs.CPU11.RCU
     48693           -34.0%      32144        softirqs.CPU11.SCHED
     91941           -33.8%      60905        softirqs.CPU11.TIMER
     21612 ±  2%     -37.5%      13506 ± 11%  softirqs.CPU12.RCU
     48797           -34.0%      32190        softirqs.CPU12.SCHED
     91069           -33.2%      60853        softirqs.CPU12.TIMER
     21786 ±  4%     -39.5%      13186 ±  5%  softirqs.CPU13.RCU
     48857           -34.1%      32216        softirqs.CPU13.SCHED
     90779           -32.7%      61120        softirqs.CPU13.TIMER
     22220 ±  7%     -39.3%      13479 ±  7%  softirqs.CPU14.RCU
     49143           -34.2%      32358        softirqs.CPU14.SCHED
     91204           -33.5%      60616        softirqs.CPU14.TIMER
     23692 ±  5%     -38.4%      14592 ±  5%  softirqs.CPU15.RCU
     49566           -34.3%      32549        softirqs.CPU15.SCHED
     91630           -33.3%      61144        softirqs.CPU15.TIMER
     24311 ±  8%     -40.2%      14529 ±  8%  softirqs.CPU16.RCU
     49337           -34.3%      32437        softirqs.CPU16.SCHED
     91471           -33.5%      60868        softirqs.CPU16.TIMER
     23302 ±  4%     -40.3%      13901 ±  8%  softirqs.CPU17.RCU
     49014           -34.6%      32038        softirqs.CPU17.SCHED
     90966           -33.9%      60098        softirqs.CPU17.TIMER
     21200 ± 14%     -36.2%      13518 ±  9%  softirqs.CPU18.RCU
     49057           -34.5%      32148        softirqs.CPU18.SCHED
     90381           -32.8%      60699        softirqs.CPU18.TIMER
     24113 ±  4%     -41.4%      14141 ±  7%  softirqs.CPU19.RCU
     48660           -34.3%      31980        softirqs.CPU19.SCHED
     91470 ±  2%     -34.1%      60247        softirqs.CPU19.TIMER
     23240 ± 10%     -37.5%      14527 ± 13%  softirqs.CPU2.RCU
     49136           -34.5%      32178        softirqs.CPU2.SCHED
     93879 ±  5%     -35.5%      60529        softirqs.CPU2.TIMER
     23082 ±  6%     -39.3%      14017 ±  7%  softirqs.CPU20.RCU
     49277           -34.1%      32471        softirqs.CPU20.SCHED
     90735           -33.2%      60574        softirqs.CPU20.TIMER
     23969 ±  3%     -44.7%      13245 ± 19%  softirqs.CPU21.RCU
     49551           -34.4%      32527        softirqs.CPU21.SCHED
     91464           -33.4%      60882 ±  2%  softirqs.CPU21.TIMER
     23511 ±  2%     -35.2%      15246 ± 10%  softirqs.CPU22.RCU
     49923           -34.1%      32886        softirqs.CPU22.SCHED
     94860 ±  2%     -30.4%      66066 ±  3%  softirqs.CPU22.TIMER
     23929 ±  6%     -36.2%      15266 ±  8%  softirqs.CPU23.RCU
     49349           -34.4%      32378        softirqs.CPU23.SCHED
     90384           -33.4%      60213        softirqs.CPU23.TIMER
     23887           -41.6%      13950 ± 10%  softirqs.CPU24.RCU
     49063           -34.1%      32350        softirqs.CPU24.SCHED
     90207           -33.1%      60367        softirqs.CPU24.TIMER
     22154 ±  5%     -34.6%      14489 ± 10%  softirqs.CPU25.RCU
     48884           -34.6%      31954        softirqs.CPU25.SCHED
     90168           -33.6%      59883        softirqs.CPU25.TIMER
     23604 ±  5%     -40.7%      13995 ± 11%  softirqs.CPU26.RCU
     48998           -34.3%      32190        softirqs.CPU26.SCHED
     90213           -33.6%      59892        softirqs.CPU26.TIMER
     23856 ±  6%     -41.1%      14051 ± 11%  softirqs.CPU27.RCU
     49515 ±  2%     -35.2%      32087        softirqs.CPU27.SCHED
    107476 ± 27%     -43.9%      60265        softirqs.CPU27.TIMER
     23823 ±  6%     -42.5%      13695 ± 10%  softirqs.CPU28.RCU
     49693           -34.8%      32375        softirqs.CPU28.SCHED
     91425           -34.2%      60152        softirqs.CPU28.TIMER
     23682 ±  3%     -42.7%      13580 ±  7%  softirqs.CPU29.RCU
     49091           -34.5%      32166        softirqs.CPU29.SCHED
     90088           -33.4%      59978        softirqs.CPU29.TIMER
     21696 ±  2%     -40.8%      12836 ±  6%  softirqs.CPU3.RCU
     48756           -34.2%      32071        softirqs.CPU3.SCHED
     91187           -33.8%      60335        softirqs.CPU3.TIMER
     22322 ±  2%     -27.9%      16090 ± 15%  softirqs.CPU30.RCU
     48881           -34.5%      32036        softirqs.CPU30.SCHED
     89824           -33.1%      60087        softirqs.CPU30.TIMER
     22881 ±  2%     -37.4%      14324 ±  6%  softirqs.CPU31.RCU
     48763           -34.2%      32079        softirqs.CPU31.SCHED
     89807           -33.0%      60132        softirqs.CPU31.TIMER
     22369 ±  5%     -34.2%      14711 ±  5%  softirqs.CPU32.RCU
     48932           -34.4%      32089        softirqs.CPU32.SCHED
     90136           -33.7%      59771        softirqs.CPU32.TIMER
     24078 ±  2%     -39.2%      14637 ± 10%  softirqs.CPU33.RCU
     48900           -34.7%      31929        softirqs.CPU33.SCHED
     90381           -33.8%      59859        softirqs.CPU33.TIMER
     24794           -40.5%      14755 ±  2%  softirqs.CPU34.RCU
     48702           -34.2%      32059        softirqs.CPU34.SCHED
     89981           -33.4%      59949        softirqs.CPU34.TIMER
     23387 ±  3%     -42.0%      13563 ± 10%  softirqs.CPU35.RCU
     49175 ±  2%     -35.3%      31829        softirqs.CPU35.SCHED
    107101 ± 27%     -44.3%      59642        softirqs.CPU35.TIMER
     25396 ±  3%     -42.0%      14725 ±  6%  softirqs.CPU36.RCU
     49373           -34.3%      32419        softirqs.CPU36.SCHED
     90586           -33.6%      60171        softirqs.CPU36.TIMER
     24764           -38.0%      15347 ±  5%  softirqs.CPU37.RCU
     49240           -34.7%      32174        softirqs.CPU37.SCHED
     90693           -34.1%      59782        softirqs.CPU37.TIMER
     23986 ±  7%     -39.0%      14637 ±  4%  softirqs.CPU38.RCU
     49241           -34.7%      32166        softirqs.CPU38.SCHED
     91208           -34.2%      60034        softirqs.CPU38.TIMER
     25146 ±  3%     -39.7%      15168 ± 18%  softirqs.CPU39.RCU
     48951           -34.5%      32041        softirqs.CPU39.SCHED
     90295           -33.7%      59848        softirqs.CPU39.TIMER
     23881 ± 17%     -43.3%      13530 ±  9%  softirqs.CPU4.RCU
     48681           -33.6%      32306        softirqs.CPU4.SCHED
     90505           -33.1%      60561        softirqs.CPU4.TIMER
     23356 ±  2%     -38.3%      14402 ±  8%  softirqs.CPU40.RCU
     48722           -34.2%      32044        softirqs.CPU40.SCHED
     90099           -33.4%      60004        softirqs.CPU40.TIMER
     22071 ± 15%     -35.1%      14332 ±  4%  softirqs.CPU41.RCU
     49308           -34.5%      32280        softirqs.CPU41.SCHED
     92092 ±  2%     -34.6%      60238        softirqs.CPU41.TIMER
     24707 ±  4%     -37.5%      15453 ±  7%  softirqs.CPU42.RCU
     49464           -34.2%      32567 ±  2%  softirqs.CPU42.SCHED
     24420           -39.6%      14741 ± 13%  softirqs.CPU43.RCU
     49611           -34.5%      32473        softirqs.CPU43.SCHED
     91848           -34.1%      60524 ±  2%  softirqs.CPU43.TIMER
     19410           -42.4%      11182 ±  5%  softirqs.CPU44.RCU
     48878           -34.0%      32259        softirqs.CPU44.SCHED
     91387           -32.4%      61814 ±  3%  softirqs.CPU44.TIMER
     24101 ±  7%     -27.8%      17398 ±  8%  softirqs.CPU45.RCU
     49242           -34.0%      32508        softirqs.CPU45.SCHED
     92561           -28.0%      66676 ±  2%  softirqs.CPU45.TIMER
     23188           -40.1%      13895 ±  7%  softirqs.CPU46.RCU
     48916           -34.3%      32116        softirqs.CPU46.SCHED
     90678           -33.5%      60335        softirqs.CPU46.TIMER
     22800           -39.4%      13821 ±  9%  softirqs.CPU47.RCU
     48560           -34.1%      31984        softirqs.CPU47.SCHED
     91331           -34.1%      60166        softirqs.CPU47.TIMER
     23352 ±  6%     -39.1%      14232 ± 10%  softirqs.CPU48.RCU
     48942           -34.3%      32146        softirqs.CPU48.SCHED
     91248           -34.1%      60136        softirqs.CPU48.TIMER
     23095 ±  6%     -40.4%      13758 ± 11%  softirqs.CPU49.RCU
     48810           -34.6%      31944        softirqs.CPU49.SCHED
     90747           -33.8%      60083        softirqs.CPU49.TIMER
     23325 ±  5%     -41.0%      13769 ±  5%  softirqs.CPU5.RCU
     48805           -34.2%      32094        softirqs.CPU5.SCHED
     91617           -33.9%      60564        softirqs.CPU5.TIMER
     23644 ±  4%     -39.9%      14205 ± 10%  softirqs.CPU50.RCU
     48990           -34.2%      32236        softirqs.CPU50.SCHED
     90692           -33.2%      60554        softirqs.CPU50.TIMER
     23686 ±  7%     -35.6%      15252 ± 11%  softirqs.CPU51.RCU
     48956           -34.1%      32281        softirqs.CPU51.SCHED
     91604           -34.0%      60504        softirqs.CPU51.TIMER
     22432           -33.9%      14818 ± 11%  softirqs.CPU52.RCU
     48387           -33.6%      32125        softirqs.CPU52.SCHED
     90864           -33.4%      60497        softirqs.CPU52.TIMER
     23386 ±  3%     -42.4%      13472 ±  5%  softirqs.CPU53.RCU
     48886           -34.5%      32024        softirqs.CPU53.SCHED
     90647           -33.5%      60254        softirqs.CPU53.TIMER
     22531 ±  4%     -30.9%      15559 ± 20%  softirqs.CPU54.RCU
     48895           -34.4%      32072        softirqs.CPU54.SCHED
     90922           -33.1%      60853        softirqs.CPU54.TIMER
     22676 ±  3%     -41.7%      13215 ±  8%  softirqs.CPU55.RCU
     48288           -33.9%      31913        softirqs.CPU55.SCHED
     90143           -33.0%      60354        softirqs.CPU55.TIMER
     24427 ±  3%     -44.0%      13683 ±  7%  softirqs.CPU56.RCU
     49382 ±  2%     -34.9%      32138        softirqs.CPU56.SCHED
     91201 ±  2%     -33.5%      60678        softirqs.CPU56.TIMER
     24291 ±  7%     -43.1%      13816 ± 10%  softirqs.CPU57.RCU
     48663           -34.3%      31962        softirqs.CPU57.SCHED
     90697           -33.2%      60624        softirqs.CPU57.TIMER
     24129 ±  4%     -41.8%      14036 ±  7%  softirqs.CPU58.RCU
     49018           -34.4%      32175        softirqs.CPU58.SCHED
     90610           -32.9%      60790        softirqs.CPU58.TIMER
     24746 ±  5%     -46.0%      13369 ±  7%  softirqs.CPU59.RCU
     49153           -34.0%      32426        softirqs.CPU59.SCHED
     90884           -33.1%      60813        softirqs.CPU59.TIMER
     22574 ±  2%     -42.6%      12956 ±  6%  softirqs.CPU6.RCU
     49235           -33.8%      32587        softirqs.CPU6.SCHED
     91160           -33.2%      60871        softirqs.CPU6.TIMER
     24608           -42.8%      14077 ±  7%  softirqs.CPU60.RCU
     49107           -34.1%      32338        softirqs.CPU60.SCHED
     90958           -33.5%      60470        softirqs.CPU60.TIMER
     22744 ±  4%     -39.5%      13770 ±  7%  softirqs.CPU61.RCU
     48815           -34.5%      31981        softirqs.CPU61.SCHED
     90275           -33.6%      59942        softirqs.CPU61.TIMER
     22717 ±  5%     -36.3%      14470 ±  5%  softirqs.CPU62.RCU
     49262           -35.0%      32004        softirqs.CPU62.SCHED
    107526 ± 26%     -44.5%      59722        softirqs.CPU62.TIMER
     22121 ±  5%     -38.5%      13593 ± 13%  softirqs.CPU63.RCU
     48424           -34.2%      31850        softirqs.CPU63.SCHED
     89901           -32.9%      60354 ±  2%  softirqs.CPU63.TIMER
     23264 ±  2%     -39.4%      14093 ±  6%  softirqs.CPU64.RCU
     49293           -34.5%      32299        softirqs.CPU64.SCHED
     90911           -32.4%      61449 ±  2%  softirqs.CPU64.TIMER
     24129 ±  5%     -43.3%      13684 ±  8%  softirqs.CPU65.RCU
     49149           -34.0%      32419        softirqs.CPU65.SCHED
     25946 ±  2%     -41.5%      15171 ±  6%  softirqs.CPU66.RCU
     49445           -35.2%      32022        softirqs.CPU66.SCHED
     92374           -34.8%      60194        softirqs.CPU66.TIMER
     25811 ±  2%     -42.0%      14962 ±  6%  softirqs.CPU67.RCU
     49183           -34.6%      32190        softirqs.CPU67.SCHED
     90248           -33.4%      60104        softirqs.CPU67.TIMER
     25134 ±  4%     -39.9%      15098 ±  8%  softirqs.CPU68.RCU
     48943           -34.1%      32271        softirqs.CPU68.SCHED
     90171           -33.3%      60139        softirqs.CPU68.TIMER
     24403 ±  3%     -37.2%      15327 ±  6%  softirqs.CPU69.RCU
     48832           -34.7%      31872        softirqs.CPU69.SCHED
     90258           -34.3%      59311        softirqs.CPU69.TIMER
     21852 ±  2%     -38.8%      13364 ±  7%  softirqs.CPU7.RCU
     48924           -33.9%      32319        softirqs.CPU7.SCHED
     90908           -33.1%      60776        softirqs.CPU7.TIMER
     24793           -38.9%      15140 ±  8%  softirqs.CPU70.RCU
     48807           -34.5%      31953        softirqs.CPU70.SCHED
     90038           -33.8%      59613        softirqs.CPU70.TIMER
     23670 ± 18%     -37.0%      14906 ± 14%  softirqs.CPU71.RCU
     48849           -34.8%      31858        softirqs.CPU71.SCHED
     89585           -33.6%      59455        softirqs.CPU71.TIMER
     25855 ±  6%     -41.1%      15217 ± 10%  softirqs.CPU72.RCU
     49310           -34.6%      32228        softirqs.CPU72.SCHED
     90872           -34.3%      59745        softirqs.CPU72.TIMER
     25080 ±  5%     -41.8%      14604 ±  2%  softirqs.CPU73.RCU
     48907           -34.9%      31856        softirqs.CPU73.SCHED
     90081           -34.1%      59359        softirqs.CPU73.TIMER
     24380 ±  3%     -39.6%      14717 ±  5%  softirqs.CPU74.RCU
     48841           -34.5%      31975        softirqs.CPU74.SCHED
     89711           -33.6%      59559        softirqs.CPU74.TIMER
     20706 ±  3%     -37.8%      12883 ±  8%  softirqs.CPU75.RCU
     48773           -34.8%      31822        softirqs.CPU75.SCHED
     90101           -34.0%      59435        softirqs.CPU75.TIMER
     20385 ±  5%     -41.7%      11878 ±  4%  softirqs.CPU76.RCU
     48652           -34.5%      31876        softirqs.CPU76.SCHED
     90132           -34.2%      59276        softirqs.CPU76.TIMER
     20442 ±  2%     -41.4%      11984 ± 12%  softirqs.CPU77.RCU
     48664           -34.7%      31799        softirqs.CPU77.SCHED
     90078           -33.9%      59562        softirqs.CPU77.TIMER
     20243 ±  6%     -38.5%      12444 ±  7%  softirqs.CPU78.RCU
     48677           -34.8%      31717        softirqs.CPU78.SCHED
     89889           -34.0%      59362        softirqs.CPU78.TIMER
     19018 ± 11%     -36.0%      12162 ± 10%  softirqs.CPU79.RCU
     48810           -35.0%      31748        softirqs.CPU79.SCHED
     91123           -34.8%      59406        softirqs.CPU79.TIMER
     21869 ±  4%     -37.4%      13695 ±  6%  softirqs.CPU8.RCU
     48748           -33.9%      32209        softirqs.CPU8.SCHED
     90966           -33.2%      60779        softirqs.CPU8.TIMER
     20992 ±  2%     -41.1%      12368 ±  7%  softirqs.CPU80.RCU
     49104           -34.6%      32122        softirqs.CPU80.SCHED
     91143           -34.6%      59581        softirqs.CPU80.TIMER
     20507 ±  3%     -38.0%      12723 ± 11%  softirqs.CPU81.RCU
     49167           -34.8%      32054        softirqs.CPU81.SCHED
     90831           -34.7%      59322        softirqs.CPU81.TIMER
     23199 ± 12%     -46.1%      12507 ± 11%  softirqs.CPU82.RCU
     48908           -34.5%      32028        softirqs.CPU82.SCHED
     91215           -34.5%      59701        softirqs.CPU82.TIMER
     20694 ±  5%     -39.6%      12499 ± 13%  softirqs.CPU83.RCU
     48561           -34.5%      31797        softirqs.CPU83.SCHED
     89836           -34.1%      59238        softirqs.CPU83.TIMER
     20754           -41.2%      12210 ± 17%  softirqs.CPU84.RCU
     48475           -34.3%      31837        softirqs.CPU84.SCHED
     89895           -33.9%      59460        softirqs.CPU84.TIMER
     23132 ± 17%     -36.5%      14690 ± 23%  softirqs.CPU85.RCU
     49159           -34.3%      32273        softirqs.CPU85.SCHED
    108932 ± 25%     -44.9%      60058        softirqs.CPU85.TIMER
     20899 ±  6%     -43.5%      11811 ± 15%  softirqs.CPU86.RCU
     49019           -34.3%      32188        softirqs.CPU86.SCHED
     90158           -33.5%      59974        softirqs.CPU86.TIMER
     20364 ±  3%     -39.3%      12364 ± 10%  softirqs.CPU87.RCU
     48874           -34.1%      32209        softirqs.CPU87.SCHED
     22130 ±  4%     -38.5%      13603 ± 10%  softirqs.CPU9.RCU
     48933           -34.2%      32203        softirqs.CPU9.SCHED
     91213           -33.3%      60833        softirqs.CPU9.TIMER
   2029638 ±  2%     -39.6%    1226445 ±  6%  softirqs.RCU
   4316876           -34.4%    2833655        softirqs.SCHED
   8077949           -33.6%    5364474        softirqs.TIMER
    300.25           -34.7%     196.00        interrupts.168:IR-PCI-MSI.512000-edge.ahci[0000:00:1f.2]
    636.75 ± 89%     -82.4%     111.75 ±  8%  interrupts.37:IR-PCI-MSI.1572867-edge.eth0-TxRx-3
    194.50 ± 13%     -40.2%     116.25 ±  9%  interrupts.39:IR-PCI-MSI.1572869-edge.eth0-TxRx-5
    313.75 ± 48%     -51.2%     153.00 ± 41%  interrupts.41:IR-PCI-MSI.1572871-edge.eth0-TxRx-7
    181.00 ± 13%     -40.7%     107.25 ±  2%  interrupts.42:IR-PCI-MSI.1572872-edge.eth0-TxRx-8
    234.25 ± 38%     -46.2%     126.00 ± 15%  interrupts.43:IR-PCI-MSI.1572873-edge.eth0-TxRx-9
    168.00 ±  2%     -27.2%     122.25 ± 13%  interrupts.44:IR-PCI-MSI.1572874-edge.eth0-TxRx-10
    170.50 ±  5%     -29.8%     119.75 ±  7%  interrupts.45:IR-PCI-MSI.1572875-edge.eth0-TxRx-11
    248.75 ± 43%     -51.8%     120.00 ± 21%  interrupts.47:IR-PCI-MSI.1572877-edge.eth0-TxRx-13
    175.75 ±  5%     -38.3%     108.50 ±  6%  interrupts.48:IR-PCI-MSI.1572878-edge.eth0-TxRx-14
    151.50           -34.7%      99.00 ±  2%  interrupts.50:IR-PCI-MSI.1572880-edge.eth0-TxRx-16
    151.25           -34.4%      99.25 ±  2%  interrupts.51:IR-PCI-MSI.1572881-edge.eth0-TxRx-17
    151.25           -35.0%      98.25        interrupts.53:IR-PCI-MSI.1572883-edge.eth0-TxRx-19
    155.50 ±  4%     -36.8%      98.25        interrupts.54:IR-PCI-MSI.1572884-edge.eth0-TxRx-20
    150.25           -34.4%      98.50        interrupts.55:IR-PCI-MSI.1572885-edge.eth0-TxRx-21
    150.50           -33.2%     100.50 ±  4%  interrupts.56:IR-PCI-MSI.1572886-edge.eth0-TxRx-22
    150.00           -34.0%      99.00 ±  2%  interrupts.57:IR-PCI-MSI.1572887-edge.eth0-TxRx-23
    151.25           -26.9%     110.50 ± 20%  interrupts.58:IR-PCI-MSI.1572888-edge.eth0-TxRx-24
    150.00           -34.7%      98.00        interrupts.59:IR-PCI-MSI.1572889-edge.eth0-TxRx-25
    150.00           -34.5%      98.25        interrupts.60:IR-PCI-MSI.1572890-edge.eth0-TxRx-26
    150.00           -34.8%      97.75        interrupts.61:IR-PCI-MSI.1572891-edge.eth0-TxRx-27
    150.00           -30.8%     103.75 ±  8%  interrupts.63:IR-PCI-MSI.1572893-edge.eth0-TxRx-29
    152.25 ±  2%     -35.8%      97.75        interrupts.64:IR-PCI-MSI.1572894-edge.eth0-TxRx-30
    150.00           -34.8%      97.75        interrupts.65:IR-PCI-MSI.1572895-edge.eth0-TxRx-31
    151.00           -35.3%      97.75        interrupts.66:IR-PCI-MSI.1572896-edge.eth0-TxRx-32
    150.50           -35.0%      97.75        interrupts.67:IR-PCI-MSI.1572897-edge.eth0-TxRx-33
    150.00           -34.0%      99.00 ±  2%  interrupts.68:IR-PCI-MSI.1572898-edge.eth0-TxRx-34
    150.00           -34.8%      97.75        interrupts.69:IR-PCI-MSI.1572899-edge.eth0-TxRx-35
    150.25           -34.6%      98.25        interrupts.70:IR-PCI-MSI.1572900-edge.eth0-TxRx-36
    150.00           -34.5%      98.25        interrupts.71:IR-PCI-MSI.1572901-edge.eth0-TxRx-37
    150.00           -34.0%      99.00        interrupts.74:IR-PCI-MSI.1572902-edge.eth0-TxRx-38
    150.00           -34.8%      97.75        interrupts.75:IR-PCI-MSI.1572903-edge.eth0-TxRx-39
    150.00           -34.8%      97.75        interrupts.76:IR-PCI-MSI.1572904-edge.eth0-TxRx-40
    173.50 ± 22%     -41.9%     100.75 ±  5%  interrupts.77:IR-PCI-MSI.1572905-edge.eth0-TxRx-41
    150.00           -34.8%      97.75        interrupts.78:IR-PCI-MSI.1572906-edge.eth0-TxRx-42
    150.50           -35.0%      97.75        interrupts.79:IR-PCI-MSI.1572907-edge.eth0-TxRx-43
    155.75 ±  5%     -31.9%     106.00 ±  9%  interrupts.80:IR-PCI-MSI.1572908-edge.eth0-TxRx-44
    152.00           -34.5%      99.50 ±  2%  interrupts.81:IR-PCI-MSI.1572909-edge.eth0-TxRx-45
    150.25           -32.3%     101.75 ±  7%  interrupts.82:IR-PCI-MSI.1572910-edge.eth0-TxRx-46
    151.00           -34.3%      99.25 ±  2%  interrupts.83:IR-PCI-MSI.1572911-edge.eth0-TxRx-47
    173.50 ± 20%     -43.2%      98.50 ±  2%  interrupts.84:IR-PCI-MSI.1572912-edge.eth0-TxRx-48
    153.00 ±  3%     -34.6%     100.00 ±  4%  interrupts.85:IR-PCI-MSI.1572913-edge.eth0-TxRx-49
    158.00 ±  2%     -35.9%     101.25 ±  6%  interrupts.86:IR-PCI-MSI.1572914-edge.eth0-TxRx-50
    150.75           -33.5%     100.25 ±  4%  interrupts.87:IR-PCI-MSI.1572915-edge.eth0-TxRx-51
    151.00           -35.3%      97.75        interrupts.88:IR-PCI-MSI.1572916-edge.eth0-TxRx-52
    151.75           -33.9%     100.25 ±  5%  interrupts.89:IR-PCI-MSI.1572917-edge.eth0-TxRx-53
    154.00 ±  3%     -34.6%     100.75 ±  3%  interrupts.90:IR-PCI-MSI.1572918-edge.eth0-TxRx-54
    152.75           -36.0%      97.75        interrupts.92:IR-PCI-MSI.1572920-edge.eth0-TxRx-56
    150.75           -35.2%      97.75        interrupts.93:IR-PCI-MSI.1572921-edge.eth0-TxRx-57
    150.25           -34.9%      97.75        interrupts.94:IR-PCI-MSI.1572922-edge.eth0-TxRx-58
    152.25           -35.8%      97.75        interrupts.95:IR-PCI-MSI.1572923-edge.eth0-TxRx-59
    152.75 ±  2%     -36.0%      97.75        interrupts.96:IR-PCI-MSI.1572924-edge.eth0-TxRx-60
    150.75           -34.8%      98.25        interrupts.97:IR-PCI-MSI.1572925-edge.eth0-TxRx-61
    151.00           -35.1%      98.00        interrupts.98:IR-PCI-MSI.1572926-edge.eth0-TxRx-62
    468.50           -31.9%     319.00 ±  4%  interrupts.9:IR-IO-APIC.9-fasteoi.acpi
   1509329 ±  2%     -33.9%     998065 ±  2%  interrupts.CAL:Function_call_interrupts
     17097 ±  2%     -33.4%      11385 ±  2%  interrupts.CPU0.CAL:Function_call_interrupts
    613505           -34.9%     399429        interrupts.CPU0.LOC:Local_timer_interrupts
      8520 ±  2%     -34.4%       5585 ±  3%  interrupts.CPU0.RES:Rescheduling_interrupts
     15342 ±  2%     -34.4%      10057 ±  4%  interrupts.CPU0.TLB:TLB_shootdowns
    468.50           -31.9%     319.00 ±  4%  interrupts.CPU1.9:IR-IO-APIC.9-fasteoi.acpi
     17378           -34.8%      11337 ±  3%  interrupts.CPU1.CAL:Function_call_interrupts
    613323           -34.9%     399178        interrupts.CPU1.LOC:Local_timer_interrupts
      7227 ±  2%     -32.2%       4898 ±  4%  interrupts.CPU1.RES:Rescheduling_interrupts
     15741 ±  2%     -36.6%       9977 ±  3%  interrupts.CPU1.TLB:TLB_shootdowns
    168.00 ±  2%     -27.2%     122.25 ± 13%  interrupts.CPU10.44:IR-PCI-MSI.1572874-edge.eth0-TxRx-10
     17112 ±  2%     -34.0%      11300 ±  2%  interrupts.CPU10.CAL:Function_call_interrupts
    612934           -34.9%     399312        interrupts.CPU10.LOC:Local_timer_interrupts
      7060 ±  6%     -31.3%       4851 ± 10%  interrupts.CPU10.RES:Rescheduling_interrupts
     15429 ±  4%     -35.1%      10019 ±  3%  interrupts.CPU10.TLB:TLB_shootdowns
    170.50 ±  5%     -29.8%     119.75 ±  7%  interrupts.CPU11.45:IR-PCI-MSI.1572875-edge.eth0-TxRx-11
     16995 ±  2%     -33.1%      11364 ±  2%  interrupts.CPU11.CAL:Function_call_interrupts
    612683           -34.8%     399339        interrupts.CPU11.LOC:Local_timer_interrupts
      6990 ±  3%     -31.2%       4810 ±  7%  interrupts.CPU11.RES:Rescheduling_interrupts
     15186 ±  3%     -34.3%       9972 ±  2%  interrupts.CPU11.TLB:TLB_shootdowns
     16997 ±  2%     -33.3%      11338 ±  2%  interrupts.CPU12.CAL:Function_call_interrupts
    613339           -34.9%     399170        interrupts.CPU12.LOC:Local_timer_interrupts
      6990 ±  4%     -34.5%       4581 ±  5%  interrupts.CPU12.RES:Rescheduling_interrupts
     15108 ±  2%     -33.7%      10016 ±  2%  interrupts.CPU12.TLB:TLB_shootdowns
    248.75 ± 43%     -51.8%     120.00 ± 21%  interrupts.CPU13.47:IR-PCI-MSI.1572877-edge.eth0-TxRx-13
     16929 ±  3%     -33.5%      11263 ±  2%  interrupts.CPU13.CAL:Function_call_interrupts
    613134           -34.9%     399371        interrupts.CPU13.LOC:Local_timer_interrupts
      6704           -29.0%       4760 ±  9%  interrupts.CPU13.RES:Rescheduling_interrupts
     15192 ±  3%     -34.9%       9897 ±  3%  interrupts.CPU13.TLB:TLB_shootdowns
    175.75 ±  5%     -38.3%     108.50 ±  6%  interrupts.CPU14.48:IR-PCI-MSI.1572878-edge.eth0-TxRx-14
     16953 ±  3%     -32.9%      11371 ±  2%  interrupts.CPU14.CAL:Function_call_interrupts
    613112           -34.9%     399101        interrupts.CPU14.LOC:Local_timer_interrupts
      7159 ±  3%     -33.5%       4763 ± 10%  interrupts.CPU14.RES:Rescheduling_interrupts
     15301 ±  4%     -34.3%      10046 ±  3%  interrupts.CPU14.TLB:TLB_shootdowns
     17050 ±  3%     -34.3%      11210 ±  2%  interrupts.CPU15.CAL:Function_call_interrupts
    613127           -34.9%     399291        interrupts.CPU15.LOC:Local_timer_interrupts
      6944 ±  3%     -31.0%       4791 ±  4%  interrupts.CPU15.RES:Rescheduling_interrupts
     15282 ±  4%     -34.8%       9966 ±  2%  interrupts.CPU15.TLB:TLB_shootdowns
    151.50           -34.7%      99.00 ±  2%  interrupts.CPU16.50:IR-PCI-MSI.1572880-edge.eth0-TxRx-16
     16919 ±  2%     -33.9%      11190        interrupts.CPU16.CAL:Function_call_interrupts
    613265           -34.9%     399128        interrupts.CPU16.LOC:Local_timer_interrupts
      7560 ±  5%     -30.1%       5288 ± 16%  interrupts.CPU16.RES:Rescheduling_interrupts
     15173 ±  3%     -35.1%       9849        interrupts.CPU16.TLB:TLB_shootdowns
    151.25           -34.4%      99.25 ±  2%  interrupts.CPU17.51:IR-PCI-MSI.1572881-edge.eth0-TxRx-17
     17067 ±  2%     -34.6%      11155 ±  2%  interrupts.CPU17.CAL:Function_call_interrupts
    613275           -35.0%     398883        interrupts.CPU17.LOC:Local_timer_interrupts
      7592 ±  7%     -35.3%       4915 ± 11%  interrupts.CPU17.RES:Rescheduling_interrupts
     15381 ±  2%     -36.3%       9799 ±  3%  interrupts.CPU17.TLB:TLB_shootdowns
     17064 ±  2%     -34.4%      11192 ±  3%  interrupts.CPU18.CAL:Function_call_interrupts
    612709           -34.9%     399089        interrupts.CPU18.LOC:Local_timer_interrupts
      7251 ±  4%     -33.4%       4827 ±  8%  interrupts.CPU18.RES:Rescheduling_interrupts
     15243 ±  3%     -35.4%       9843 ±  3%  interrupts.CPU18.TLB:TLB_shootdowns
    151.25           -35.0%      98.25        interrupts.CPU19.53:IR-PCI-MSI.1572883-edge.eth0-TxRx-19
     17054 ±  2%     -34.1%      11246 ±  2%  interrupts.CPU19.CAL:Function_call_interrupts
    612999           -34.9%     398997        interrupts.CPU19.LOC:Local_timer_interrupts
      7466 ±  7%     -32.5%       5037 ± 13%  interrupts.CPU19.RES:Rescheduling_interrupts
     15258 ±  2%     -35.0%       9920 ±  2%  interrupts.CPU19.TLB:TLB_shootdowns
     17162 ±  2%     -34.4%      11265 ±  2%  interrupts.CPU2.CAL:Function_call_interrupts
    613357           -34.9%     399326        interrupts.CPU2.LOC:Local_timer_interrupts
      7000           -34.7%       4569        interrupts.CPU2.RES:Rescheduling_interrupts
     15308 ±  3%     -34.7%       9989 ±  3%  interrupts.CPU2.TLB:TLB_shootdowns
    155.50 ±  4%     -36.8%      98.25        interrupts.CPU20.54:IR-PCI-MSI.1572884-edge.eth0-TxRx-20
     16990 ±  2%     -32.6%      11443 ±  2%  interrupts.CPU20.CAL:Function_call_interrupts
    613086           -34.9%     398881        interrupts.CPU20.LOC:Local_timer_interrupts
      7580 ±  6%     -34.3%       4981 ±  7%  interrupts.CPU20.RES:Rescheduling_interrupts
     15329 ±  4%     -34.3%      10066 ±  4%  interrupts.CPU20.TLB:TLB_shootdowns
    150.25           -34.4%      98.50        interrupts.CPU21.55:IR-PCI-MSI.1572885-edge.eth0-TxRx-21
     17255           -33.2%      11519 ±  2%  interrupts.CPU21.CAL:Function_call_interrupts
    613366           -35.0%     398819        interrupts.CPU21.LOC:Local_timer_interrupts
      7764 ±  6%     -30.2%       5418 ± 15%  interrupts.CPU21.RES:Rescheduling_interrupts
     15569 ±  2%     -34.2%      10252 ±  3%  interrupts.CPU21.TLB:TLB_shootdowns
    150.50           -33.2%     100.50 ±  4%  interrupts.CPU22.56:IR-PCI-MSI.1572886-edge.eth0-TxRx-22
     17100 ±  4%     -35.9%      10961        interrupts.CPU22.CAL:Function_call_interrupts
    613054           -34.9%     398855        interrupts.CPU22.LOC:Local_timer_interrupts
      7160 ±  3%     -30.4%       4981 ±  6%  interrupts.CPU22.RES:Rescheduling_interrupts
     15282 ±  5%     -36.4%       9717 ±  3%  interrupts.CPU22.TLB:TLB_shootdowns
    150.00           -34.0%      99.00 ±  2%  interrupts.CPU23.57:IR-PCI-MSI.1572887-edge.eth0-TxRx-23
     16840 ±  2%     -33.3%      11236 ±  3%  interrupts.CPU23.CAL:Function_call_interrupts
    613099           -34.9%     399413        interrupts.CPU23.LOC:Local_timer_interrupts
      6909 ±  2%     -34.8%       4505 ±  2%  interrupts.CPU23.RES:Rescheduling_interrupts
     15233 ±  3%     -34.9%       9923 ±  3%  interrupts.CPU23.TLB:TLB_shootdowns
    151.25           -26.9%     110.50 ± 20%  interrupts.CPU24.58:IR-PCI-MSI.1572888-edge.eth0-TxRx-24
     16924 ±  4%     -33.6%      11232 ±  2%  interrupts.CPU24.CAL:Function_call_interrupts
    613290           -34.9%     399537        interrupts.CPU24.LOC:Local_timer_interrupts
      6875 ±  2%     -35.2%       4455        interrupts.CPU24.RES:Rescheduling_interrupts
     15151 ±  4%     -34.6%       9909 ±  3%  interrupts.CPU24.TLB:TLB_shootdowns
    150.00           -34.7%      98.00        interrupts.CPU25.59:IR-PCI-MSI.1572889-edge.eth0-TxRx-25
     17010 ±  3%     -33.5%      11318 ±  2%  interrupts.CPU25.CAL:Function_call_interrupts
    613085           -34.9%     399411        interrupts.CPU25.LOC:Local_timer_interrupts
      6855 ±  2%     -35.4%       4426        interrupts.CPU25.RES:Rescheduling_interrupts
     15283 ±  3%     -34.4%      10020 ±  3%  interrupts.CPU25.TLB:TLB_shootdowns
    150.00           -34.5%      98.25        interrupts.CPU26.60:IR-PCI-MSI.1572890-edge.eth0-TxRx-26
     16806           -33.6%      11158 ±  3%  interrupts.CPU26.CAL:Function_call_interrupts
    613174           -34.9%     399071        interrupts.CPU26.LOC:Local_timer_interrupts
      3493           -37.5%       2181 ± 34%  interrupts.CPU26.NMI:Non-maskable_interrupts
      3493           -37.5%       2181 ± 34%  interrupts.CPU26.PMI:Performance_monitoring_interrupts
      6778           -34.7%       4424 ±  3%  interrupts.CPU26.RES:Rescheduling_interrupts
     14997 ±  2%     -34.8%       9782 ±  4%  interrupts.CPU26.TLB:TLB_shootdowns
    150.00           -34.8%      97.75        interrupts.CPU27.61:IR-PCI-MSI.1572891-edge.eth0-TxRx-27
     16913 ±  2%     -34.2%      11121        interrupts.CPU27.CAL:Function_call_interrupts
    613360           -34.8%     399689        interrupts.CPU27.LOC:Local_timer_interrupts
      6802 ±  3%     -34.3%       4472        interrupts.CPU27.RES:Rescheduling_interrupts
     15301 ±  2%     -36.0%       9800 ±  3%  interrupts.CPU27.TLB:TLB_shootdowns
     16746 ±  2%     -32.1%      11370 ±  3%  interrupts.CPU28.CAL:Function_call_interrupts
    613302           -34.9%     399103        interrupts.CPU28.LOC:Local_timer_interrupts
      6817           -35.2%       4419        interrupts.CPU28.RES:Rescheduling_interrupts
     14986 ±  2%     -32.7%      10081 ±  4%  interrupts.CPU28.TLB:TLB_shootdowns
    150.00           -30.8%     103.75 ±  8%  interrupts.CPU29.63:IR-PCI-MSI.1572893-edge.eth0-TxRx-29
     16799 ±  3%     -32.8%      11290 ±  2%  interrupts.CPU29.CAL:Function_call_interrupts
    613087           -34.9%     399053        interrupts.CPU29.LOC:Local_timer_interrupts
      7091           -35.9%       4544 ±  2%  interrupts.CPU29.RES:Rescheduling_interrupts
     15128 ±  3%     -34.1%       9969 ±  3%  interrupts.CPU29.TLB:TLB_shootdowns
    636.75 ± 89%     -82.4%     111.75 ±  8%  interrupts.CPU3.37:IR-PCI-MSI.1572867-edge.eth0-TxRx-3
     17112           -33.7%      11350 ±  3%  interrupts.CPU3.CAL:Function_call_interrupts
    613344           -34.9%     399272        interrupts.CPU3.LOC:Local_timer_interrupts
      6958           -35.2%       4512 ±  2%  interrupts.CPU3.RES:Rescheduling_interrupts
     15326 ±  2%     -34.7%      10005 ±  3%  interrupts.CPU3.TLB:TLB_shootdowns
    152.25 ±  2%     -35.8%      97.75        interrupts.CPU30.64:IR-PCI-MSI.1572894-edge.eth0-TxRx-30
     16811 ±  4%     -32.4%      11366 ±  3%  interrupts.CPU30.CAL:Function_call_interrupts
    612821           -34.9%     399173        interrupts.CPU30.LOC:Local_timer_interrupts
      6808           -34.2%       4481 ±  2%  interrupts.CPU30.RES:Rescheduling_interrupts
     15137 ±  3%     -33.8%      10017 ±  5%  interrupts.CPU30.TLB:TLB_shootdowns
    150.00           -34.8%      97.75        interrupts.CPU31.65:IR-PCI-MSI.1572895-edge.eth0-TxRx-31
     16767 ±  3%     -34.3%      11024 ±  2%  interrupts.CPU31.CAL:Function_call_interrupts
    612828           -34.8%     399360        interrupts.CPU31.LOC:Local_timer_interrupts
      6698           -34.4%       4395 ±  3%  interrupts.CPU31.RES:Rescheduling_interrupts
     14995 ±  3%     -35.1%       9735 ±  3%  interrupts.CPU31.TLB:TLB_shootdowns
    151.00           -35.3%      97.75        interrupts.CPU32.66:IR-PCI-MSI.1572896-edge.eth0-TxRx-32
     16901 ±  3%     -34.7%      11030 ±  3%  interrupts.CPU32.CAL:Function_call_interrupts
    613222           -34.9%     399170        interrupts.CPU32.LOC:Local_timer_interrupts
      6642           -34.3%       4367        interrupts.CPU32.RES:Rescheduling_interrupts
     15186 ±  3%     -35.6%       9786 ±  4%  interrupts.CPU32.TLB:TLB_shootdowns
    150.50           -35.0%      97.75        interrupts.CPU33.67:IR-PCI-MSI.1572897-edge.eth0-TxRx-33
     16916 ±  3%     -33.8%      11195 ±  3%  interrupts.CPU33.CAL:Function_call_interrupts
    613470           -34.9%     399097        interrupts.CPU33.LOC:Local_timer_interrupts
      6647           -35.0%       4320        interrupts.CPU33.RES:Rescheduling_interrupts
     15227 ±  4%     -35.1%       9881 ±  4%  interrupts.CPU33.TLB:TLB_shootdowns
    150.00           -34.0%      99.00 ±  2%  interrupts.CPU34.68:IR-PCI-MSI.1572898-edge.eth0-TxRx-34
     17071 ±  3%     -34.0%      11268 ±  2%  interrupts.CPU34.CAL:Function_call_interrupts
    613242           -34.8%     399545        interrupts.CPU34.LOC:Local_timer_interrupts
      6736           -35.2%       4368 ±  2%  interrupts.CPU34.RES:Rescheduling_interrupts
     15377 ±  3%     -35.8%       9868 ±  3%  interrupts.CPU34.TLB:TLB_shootdowns
    150.00           -34.8%      97.75        interrupts.CPU35.69:IR-PCI-MSI.1572899-edge.eth0-TxRx-35
     16750 ±  2%     -32.7%      11268 ±  3%  interrupts.CPU35.CAL:Function_call_interrupts
    613212           -34.9%     399142        interrupts.CPU35.LOC:Local_timer_interrupts
      6859 ±  2%     -36.8%       4335 ±  2%  interrupts.CPU35.RES:Rescheduling_interrupts
     15208 ±  2%     -34.1%      10025 ±  2%  interrupts.CPU35.TLB:TLB_shootdowns
    150.25           -34.6%      98.25        interrupts.CPU36.70:IR-PCI-MSI.1572900-edge.eth0-TxRx-36
     17078 ±  4%     -34.4%      11209 ±  5%  interrupts.CPU36.CAL:Function_call_interrupts
    613116           -34.9%     399228        interrupts.CPU36.LOC:Local_timer_interrupts
      6787 ±  2%     -34.7%       4430        interrupts.CPU36.RES:Rescheduling_interrupts
     15310 ±  5%     -35.1%       9943 ±  6%  interrupts.CPU36.TLB:TLB_shootdowns
    150.00           -34.5%      98.25        interrupts.CPU37.71:IR-PCI-MSI.1572901-edge.eth0-TxRx-37
     16861 ±  2%     -33.0%      11301        interrupts.CPU37.CAL:Function_call_interrupts
    613110           -34.9%     399095        interrupts.CPU37.LOC:Local_timer_interrupts
      6735           -31.8%       4594 ±  6%  interrupts.CPU37.RES:Rescheduling_interrupts
     15107 ±  3%     -33.7%      10022 ±  2%  interrupts.CPU37.TLB:TLB_shootdowns
    150.00           -34.0%      99.00        interrupts.CPU38.74:IR-PCI-MSI.1572902-edge.eth0-TxRx-38
     17011 ±  2%     -34.5%      11149 ±  2%  interrupts.CPU38.CAL:Function_call_interrupts
    613510           -35.0%     398940        interrupts.CPU38.LOC:Local_timer_interrupts
      6808           -35.2%       4411 ±  3%  interrupts.CPU38.RES:Rescheduling_interrupts
     15210 ±  2%     -35.2%       9859 ±  3%  interrupts.CPU38.TLB:TLB_shootdowns
    150.00           -34.8%      97.75        interrupts.CPU39.75:IR-PCI-MSI.1572903-edge.eth0-TxRx-39
     17110 ±  4%     -34.5%      11212 ±  2%  interrupts.CPU39.CAL:Function_call_interrupts
    613130           -34.9%     399017        interrupts.CPU39.LOC:Local_timer_interrupts
      6651           -34.2%       4380 ±  3%  interrupts.CPU39.RES:Rescheduling_interrupts
     15352 ±  4%     -36.1%       9802 ±  2%  interrupts.CPU39.TLB:TLB_shootdowns
     17310           -34.9%      11277 ±  2%  interrupts.CPU4.CAL:Function_call_interrupts
    613127           -34.9%     398947        interrupts.CPU4.LOC:Local_timer_interrupts
      7093 ±  2%     -36.0%       4537 ±  3%  interrupts.CPU4.RES:Rescheduling_interrupts
     15585 ±  2%     -36.5%       9895 ±  3%  interrupts.CPU4.TLB:TLB_shootdowns
    150.00           -34.8%      97.75        interrupts.CPU40.76:IR-PCI-MSI.1572904-edge.eth0-TxRx-40
     16608 ±  2%     -32.6%      11195 ±  3%  interrupts.CPU40.CAL:Function_call_interrupts
    613122           -34.9%     399262        interrupts.CPU40.LOC:Local_timer_interrupts
      6829           -35.1%       4433 ±  2%  interrupts.CPU40.RES:Rescheduling_interrupts
     14888 ±  3%     -33.1%       9967 ±  4%  interrupts.CPU40.TLB:TLB_shootdowns
    173.50 ± 22%     -41.9%     100.75 ±  5%  interrupts.CPU41.77:IR-PCI-MSI.1572905-edge.eth0-TxRx-41
     16915 ±  3%     -34.0%      11160 ±  3%  interrupts.CPU41.CAL:Function_call_interrupts
    612830           -34.8%     399521        interrupts.CPU41.LOC:Local_timer_interrupts
      6945           -35.8%       4461        interrupts.CPU41.RES:Rescheduling_interrupts
     15089 ±  3%     -34.6%       9861 ±  4%  interrupts.CPU41.TLB:TLB_shootdowns
    150.00           -34.8%      97.75        interrupts.CPU42.78:IR-PCI-MSI.1572906-edge.eth0-TxRx-42
     16853 ±  2%     -34.2%      11095 ±  3%  interrupts.CPU42.CAL:Function_call_interrupts
    613158           -34.9%     399462        interrupts.CPU42.LOC:Local_timer_interrupts
      7142           -36.3%       4548 ±  2%  interrupts.CPU42.RES:Rescheduling_interrupts
     15101 ±  3%     -34.2%       9944 ±  2%  interrupts.CPU42.TLB:TLB_shootdowns
    150.50           -35.0%      97.75        interrupts.CPU43.79:IR-PCI-MSI.1572907-edge.eth0-TxRx-43
     16990 ±  3%     -35.0%      11044 ±  5%  interrupts.CPU43.CAL:Function_call_interrupts
    613206           -34.9%     399419        interrupts.CPU43.LOC:Local_timer_interrupts
      7370 ±  7%     -34.6%       4818 ±  5%  interrupts.CPU43.RES:Rescheduling_interrupts
     15286 ±  3%     -35.6%       9842 ±  4%  interrupts.CPU43.TLB:TLB_shootdowns
    155.75 ±  5%     -31.9%     106.00 ±  9%  interrupts.CPU44.80:IR-PCI-MSI.1572908-edge.eth0-TxRx-44
     17243 ±  3%     -32.9%      11562 ±  2%  interrupts.CPU44.CAL:Function_call_interrupts
    612888           -34.9%     398776        interrupts.CPU44.LOC:Local_timer_interrupts
      7316 ±  4%     -30.8%       5064 ± 16%  interrupts.CPU44.RES:Rescheduling_interrupts
     15311 ±  4%     -34.0%      10110 ±  2%  interrupts.CPU44.TLB:TLB_shootdowns
    152.00           -34.5%      99.50 ±  2%  interrupts.CPU45.81:IR-PCI-MSI.1572909-edge.eth0-TxRx-45
     17412 ±  3%     -34.2%      11449 ±  3%  interrupts.CPU45.CAL:Function_call_interrupts
    613317           -34.9%     399022        interrupts.CPU45.LOC:Local_timer_interrupts
      7481 ±  6%     -32.4%       5058 ± 13%  interrupts.CPU45.RES:Rescheduling_interrupts
     15483 ±  4%     -35.0%      10069 ±  4%  interrupts.CPU45.TLB:TLB_shootdowns
    150.25           -32.3%     101.75 ±  7%  interrupts.CPU46.82:IR-PCI-MSI.1572910-edge.eth0-TxRx-46
     17328 ±  2%     -34.4%      11368 ±  2%  interrupts.CPU46.CAL:Function_call_interrupts
    613203           -34.9%     399405        interrupts.CPU46.LOC:Local_timer_interrupts
      7036 ±  4%     -21.9%       5494 ± 14%  interrupts.CPU46.RES:Rescheduling_interrupts
     15425 ±  3%     -35.8%       9909 ±  4%  interrupts.CPU46.TLB:TLB_shootdowns
    151.00           -34.3%      99.25 ±  2%  interrupts.CPU47.83:IR-PCI-MSI.1572911-edge.eth0-TxRx-47
     17441 ±  3%     -33.7%      11566 ±  2%  interrupts.CPU47.CAL:Function_call_interrupts
    613313           -35.0%     398878        interrupts.CPU47.LOC:Local_timer_interrupts
      7104 ±  8%     -22.3%       5520 ± 12%  interrupts.CPU47.RES:Rescheduling_interrupts
     15665 ±  4%     -35.7%      10072 ±  2%  interrupts.CPU47.TLB:TLB_shootdowns
    173.50 ± 20%     -43.2%      98.50 ±  2%  interrupts.CPU48.84:IR-PCI-MSI.1572912-edge.eth0-TxRx-48
     17185 ±  2%     -33.8%      11376 ±  3%  interrupts.CPU48.CAL:Function_call_interrupts
    613513           -35.0%     398931        interrupts.CPU48.LOC:Local_timer_interrupts
      6800 ±  2%     -15.0%       5781 ± 14%  interrupts.CPU48.RES:Rescheduling_interrupts
     15359 ±  3%     -35.4%       9923 ±  4%  interrupts.CPU48.TLB:TLB_shootdowns
    153.00 ±  3%     -34.6%     100.00 ±  4%  interrupts.CPU49.85:IR-PCI-MSI.1572913-edge.eth0-TxRx-49
     17198 ±  2%     -33.2%      11492 ±  2%  interrupts.CPU49.CAL:Function_call_interrupts
    613362           -34.9%     399274        interrupts.CPU49.LOC:Local_timer_interrupts
      7370 ± 11%     -25.8%       5471 ± 13%  interrupts.CPU49.RES:Rescheduling_interrupts
     15362 ±  2%     -34.6%      10040 ±  3%  interrupts.CPU49.TLB:TLB_shootdowns
    194.50 ± 13%     -40.2%     116.25 ±  9%  interrupts.CPU5.39:IR-PCI-MSI.1572869-edge.eth0-TxRx-5
     17030           -33.0%      11413 ±  3%  interrupts.CPU5.CAL:Function_call_interrupts
    613317           -34.9%     399142        interrupts.CPU5.LOC:Local_timer_interrupts
      6779           -33.4%       4518        interrupts.CPU5.RES:Rescheduling_interrupts
     15330 ±  2%     -34.1%      10109 ±  4%  interrupts.CPU5.TLB:TLB_shootdowns
    158.00 ±  2%     -35.9%     101.25 ±  6%  interrupts.CPU50.86:IR-PCI-MSI.1572914-edge.eth0-TxRx-50
     17405 ±  2%     -35.2%      11274 ±  2%  interrupts.CPU50.CAL:Function_call_interrupts
    613191           -34.9%     399039        interrupts.CPU50.LOC:Local_timer_interrupts
      7383 ±  9%     -18.7%       5999 ± 19%  interrupts.CPU50.RES:Rescheduling_interrupts
     15544 ±  3%     -36.5%       9867 ±  3%  interrupts.CPU50.TLB:TLB_shootdowns
    150.75           -33.5%     100.25 ±  4%  interrupts.CPU51.87:IR-PCI-MSI.1572915-edge.eth0-TxRx-51
     17259 ±  2%     -33.1%      11553 ±  2%  interrupts.CPU51.CAL:Function_call_interrupts
    613375           -34.9%     399496        interrupts.CPU51.LOC:Local_timer_interrupts
      6891 ±  3%     -21.1%       5437 ± 13%  interrupts.CPU51.RES:Rescheduling_interrupts
     15300 ±  3%     -34.2%      10069 ±  3%  interrupts.CPU51.TLB:TLB_shootdowns
    151.00           -35.3%      97.75        interrupts.CPU52.88:IR-PCI-MSI.1572916-edge.eth0-TxRx-52
     17093 ±  3%     -33.7%      11329 ±  2%  interrupts.CPU52.CAL:Function_call_interrupts
    613320           -34.9%     399031        interrupts.CPU52.LOC:Local_timer_interrupts
     15166 ±  4%     -35.1%       9843 ±  3%  interrupts.CPU52.TLB:TLB_shootdowns
    151.75           -33.9%     100.25 ±  5%  interrupts.CPU53.89:IR-PCI-MSI.1572917-edge.eth0-TxRx-53
     17344 ±  3%     -34.7%      11324 ±  2%  interrupts.CPU53.CAL:Function_call_interrupts
    613471           -34.9%     399086        interrupts.CPU53.LOC:Local_timer_interrupts
      6950 ±  4%     -24.8%       5228 ± 10%  interrupts.CPU53.RES:Rescheduling_interrupts
     15438 ±  5%     -35.4%       9975 ±  2%  interrupts.CPU53.TLB:TLB_shootdowns
    154.00 ±  3%     -34.6%     100.75 ±  3%  interrupts.CPU54.90:IR-PCI-MSI.1572918-edge.eth0-TxRx-54
     17340 ±  2%     -34.7%      11324        interrupts.CPU54.CAL:Function_call_interrupts
    613196           -34.9%     399217        interrupts.CPU54.LOC:Local_timer_interrupts
      6492           -33.3%       4328 ±  3%  interrupts.CPU54.RES:Rescheduling_interrupts
     15378 ±  2%     -36.3%       9801        interrupts.CPU54.TLB:TLB_shootdowns
     17001 ±  2%     -33.7%      11267 ±  2%  interrupts.CPU55.CAL:Function_call_interrupts
    613013           -34.9%     399160        interrupts.CPU55.LOC:Local_timer_interrupts
      6656           -33.2%       4448 ±  2%  interrupts.CPU55.RES:Rescheduling_interrupts
     15001 ±  2%     -33.9%       9919 ±  2%  interrupts.CPU55.TLB:TLB_shootdowns
    152.75           -36.0%      97.75        interrupts.CPU56.92:IR-PCI-MSI.1572920-edge.eth0-TxRx-56
     17235 ±  3%     -34.6%      11275 ±  2%  interrupts.CPU56.CAL:Function_call_interrupts
    613166           -34.8%     399584        interrupts.CPU56.LOC:Local_timer_interrupts
      6635           -35.3%       4291 ±  2%  interrupts.CPU56.RES:Rescheduling_interrupts
     15362 ±  4%     -36.0%       9833 ±  2%  interrupts.CPU56.TLB:TLB_shootdowns
    150.75           -35.2%      97.75        interrupts.CPU57.93:IR-PCI-MSI.1572921-edge.eth0-TxRx-57
     17396 ±  2%     -34.4%      11406 ±  2%  interrupts.CPU57.CAL:Function_call_interrupts
    613238           -34.9%     399281        interrupts.CPU57.LOC:Local_timer_interrupts
      6561           -34.7%       4285 ±  2%  interrupts.CPU57.RES:Rescheduling_interrupts
     15512 ±  3%     -35.4%      10026        interrupts.CPU57.TLB:TLB_shootdowns
    150.25           -34.9%      97.75        interrupts.CPU58.94:IR-PCI-MSI.1572922-edge.eth0-TxRx-58
     17247 ±  2%     -33.7%      11428 ±  2%  interrupts.CPU58.CAL:Function_call_interrupts
    612854           -34.9%     399031        interrupts.CPU58.LOC:Local_timer_interrupts
      6662 ±  3%     -34.4%       4372 ±  2%  interrupts.CPU58.RES:Rescheduling_interrupts
     15258 ±  3%     -33.8%      10107 ±  4%  interrupts.CPU58.TLB:TLB_shootdowns
    152.25           -35.8%      97.75        interrupts.CPU59.95:IR-PCI-MSI.1572923-edge.eth0-TxRx-59
     17107 ±  2%     -33.7%      11340 ±  3%  interrupts.CPU59.CAL:Function_call_interrupts
    613119           -34.9%     399361        interrupts.CPU59.LOC:Local_timer_interrupts
      6641           -34.3%       4366 ±  3%  interrupts.CPU59.RES:Rescheduling_interrupts
     15218 ±  3%     -35.4%       9830 ±  4%  interrupts.CPU59.TLB:TLB_shootdowns
     17106 ±  2%     -33.7%      11334 ±  3%  interrupts.CPU6.CAL:Function_call_interrupts
    613305           -34.9%     399214        interrupts.CPU6.LOC:Local_timer_interrupts
      6993           -33.6%       4641        interrupts.CPU6.RES:Rescheduling_interrupts
     15394 ±  3%     -34.9%      10025 ±  4%  interrupts.CPU6.TLB:TLB_shootdowns
    152.75 ±  2%     -36.0%      97.75        interrupts.CPU60.96:IR-PCI-MSI.1572924-edge.eth0-TxRx-60
     17467 ±  2%     -34.7%      11406 ±  2%  interrupts.CPU60.CAL:Function_call_interrupts
    613388           -34.9%     399057        interrupts.CPU60.LOC:Local_timer_interrupts
      6587           -34.5%       4312        interrupts.CPU60.RES:Rescheduling_interrupts
     15489 ±  2%     -35.9%       9926 ±  4%  interrupts.CPU60.TLB:TLB_shootdowns
    150.75           -34.8%      98.25        interrupts.CPU61.97:IR-PCI-MSI.1572925-edge.eth0-TxRx-61
     17174 ±  2%     -33.1%      11483 ±  3%  interrupts.CPU61.CAL:Function_call_interrupts
    613316           -34.9%     399016        interrupts.CPU61.LOC:Local_timer_interrupts
      6753           -37.2%       4238 ±  2%  interrupts.CPU61.RES:Rescheduling_interrupts
     15229 ±  2%     -34.0%      10052 ±  3%  interrupts.CPU61.TLB:TLB_shootdowns
    151.00           -35.1%      98.00        interrupts.CPU62.98:IR-PCI-MSI.1572926-edge.eth0-TxRx-62
     17516 ±  2%     -33.6%      11626        interrupts.CPU62.CAL:Function_call_interrupts
    613770           -35.0%     399191        interrupts.CPU62.LOC:Local_timer_interrupts
      6545 ±  4%     -35.0%       4258        interrupts.CPU62.RES:Rescheduling_interrupts
     15532 ±  4%     -34.6%      10153 ±  2%  interrupts.CPU62.TLB:TLB_shootdowns
     17679           -34.2%      11638        interrupts.CPU63.CAL:Function_call_interrupts
    613265           -34.9%     399043        interrupts.CPU63.LOC:Local_timer_interrupts
      6529           -34.3%       4289 ±  3%  interrupts.CPU63.RES:Rescheduling_interrupts
     15629 ±  2%     -35.4%      10092        interrupts.CPU63.TLB:TLB_shootdowns
     18420           -34.3%      12099        interrupts.CPU64.CAL:Function_call_interrupts
    613325           -35.0%     398840        interrupts.CPU64.LOC:Local_timer_interrupts
      6841 ±  2%     -36.9%       4315        interrupts.CPU64.RES:Rescheduling_interrupts
     16615           -36.3%      10588        interrupts.CPU64.TLB:TLB_shootdowns
    300.25           -34.7%     196.00        interrupts.CPU65.168:IR-PCI-MSI.512000-edge.ahci[0000:00:1f.2]
     20230           -34.6%      13236        interrupts.CPU65.CAL:Function_call_interrupts
    613229           -34.9%     398907        interrupts.CPU65.LOC:Local_timer_interrupts
      6698           -33.2%       4477 ±  3%  interrupts.CPU65.RES:Rescheduling_interrupts
     18313           -35.8%      11754 ±  2%  interrupts.CPU65.TLB:TLB_shootdowns
     17027 ±  2%     -34.0%      11234 ±  2%  interrupts.CPU66.CAL:Function_call_interrupts
    613528           -35.0%     398760        interrupts.CPU66.LOC:Local_timer_interrupts
      6751 ±  2%     -31.9%       4596 ±  2%  interrupts.CPU66.RES:Rescheduling_interrupts
     15165 ±  2%     -35.1%       9835 ±  2%  interrupts.CPU66.TLB:TLB_shootdowns
     17227 ±  2%     -34.2%      11338 ±  3%  interrupts.CPU67.CAL:Function_call_interrupts
    613109           -34.9%     399428        interrupts.CPU67.LOC:Local_timer_interrupts
      6672           -34.6%       4360 ±  3%  interrupts.CPU67.RES:Rescheduling_interrupts
     15363 ±  2%     -35.5%       9911 ±  3%  interrupts.CPU67.TLB:TLB_shootdowns
     17073 ±  4%     -34.2%      11236 ±  4%  interrupts.CPU68.CAL:Function_call_interrupts
    613025           -34.8%     399505        interrupts.CPU68.LOC:Local_timer_interrupts
      7266 ±  9%     -27.9%       5241 ± 20%  interrupts.CPU68.RES:Rescheduling_interrupts
     15088 ±  5%     -35.2%       9779 ±  4%  interrupts.CPU68.TLB:TLB_shootdowns
     17431 ±  2%     -34.5%      11419 ±  3%  interrupts.CPU69.CAL:Function_call_interrupts
    613029           -34.9%     398840        interrupts.CPU69.LOC:Local_timer_interrupts
      6829 ±  6%     -27.3%       4966 ± 18%  interrupts.CPU69.RES:Rescheduling_interrupts
     15503 ±  3%     -35.2%      10039 ±  3%  interrupts.CPU69.TLB:TLB_shootdowns
    313.75 ± 48%     -51.2%     153.00 ± 41%  interrupts.CPU7.41:IR-PCI-MSI.1572871-edge.eth0-TxRx-7
     17180 ±  2%     -34.6%      11241 ±  2%  interrupts.CPU7.CAL:Function_call_interrupts
    613068           -34.8%     399495        interrupts.CPU7.LOC:Local_timer_interrupts
      6808 ±  3%     -33.9%       4499 ±  2%  interrupts.CPU7.RES:Rescheduling_interrupts
     15576 ±  4%     -36.5%       9897 ±  3%  interrupts.CPU7.TLB:TLB_shootdowns
     17157 ±  3%     -32.0%      11669 ±  2%  interrupts.CPU70.CAL:Function_call_interrupts
    613420           -35.0%     398835        interrupts.CPU70.LOC:Local_timer_interrupts
      7200 ±  9%     -26.7%       5279 ± 20%  interrupts.CPU70.RES:Rescheduling_interrupts
     15149 ±  4%     -32.7%      10202        interrupts.CPU70.TLB:TLB_shootdowns
     17262 ±  3%     -35.2%      11192 ±  3%  interrupts.CPU71.CAL:Function_call_interrupts
    613347           -34.9%     399308        interrupts.CPU71.LOC:Local_timer_interrupts
      6664 ±  5%     -26.8%       4878 ± 12%  interrupts.CPU71.RES:Rescheduling_interrupts
     15261 ±  2%     -35.5%       9848 ±  4%  interrupts.CPU71.TLB:TLB_shootdowns
     16973 ±  3%     -32.5%      11449 ±  3%  interrupts.CPU72.CAL:Function_call_interrupts
    613044           -34.9%     398831        interrupts.CPU72.LOC:Local_timer_interrupts
      6953 ±  5%     -22.5%       5388 ± 22%  interrupts.CPU72.RES:Rescheduling_interrupts
     15156 ±  2%     -34.5%       9922 ±  3%  interrupts.CPU72.TLB:TLB_shootdowns
     16935 ±  4%     -33.8%      11209 ±  3%  interrupts.CPU73.CAL:Function_call_interrupts
    613165           -34.9%     398895        interrupts.CPU73.LOC:Local_timer_interrupts
      6883 ±  6%     -23.7%       5255 ± 19%  interrupts.CPU73.RES:Rescheduling_interrupts
     15114 ±  5%     -36.1%       9660 ±  3%  interrupts.CPU73.TLB:TLB_shootdowns
     17073 ±  4%     -34.3%      11210 ±  2%  interrupts.CPU74.CAL:Function_call_interrupts
    612878           -34.9%     399058        interrupts.CPU74.LOC:Local_timer_interrupts
      7117 ±  7%     -27.4%       5170 ± 20%  interrupts.CPU74.RES:Rescheduling_interrupts
     15094 ±  4%     -35.1%       9792 ±  3%  interrupts.CPU74.TLB:TLB_shootdowns
     17094 ±  4%     -33.8%      11312 ±  3%  interrupts.CPU75.CAL:Function_call_interrupts
    613073           -34.9%     399369        interrupts.CPU75.LOC:Local_timer_interrupts
      6853 ±  6%     -29.2%       4854 ± 15%  interrupts.CPU75.RES:Rescheduling_interrupts
     15135 ±  4%     -34.5%       9908 ±  3%  interrupts.CPU75.TLB:TLB_shootdowns
     17122 ±  2%     -34.9%      11145 ±  2%  interrupts.CPU76.CAL:Function_call_interrupts
    613050           -34.9%     398883        interrupts.CPU76.LOC:Local_timer_interrupts
     15173 ±  2%     -36.6%       9625 ±  2%  interrupts.CPU76.TLB:TLB_shootdowns
     17052 ±  3%     -33.7%      11299 ±  3%  interrupts.CPU77.CAL:Function_call_interrupts
    613093           -34.9%     399125        interrupts.CPU77.LOC:Local_timer_interrupts
      6560 ±  4%     -22.1%       5111 ± 16%  interrupts.CPU77.RES:Rescheduling_interrupts
     15198 ±  4%     -35.5%       9807 ±  4%  interrupts.CPU77.TLB:TLB_shootdowns
     17139 ±  3%     -34.2%      11277 ±  2%  interrupts.CPU78.CAL:Function_call_interrupts
    612965           -34.9%     398949        interrupts.CPU78.LOC:Local_timer_interrupts
      6482 ±  3%     -23.8%       4940 ± 16%  interrupts.CPU78.RES:Rescheduling_interrupts
     15040 ±  4%     -35.4%       9709 ±  3%  interrupts.CPU78.TLB:TLB_shootdowns
     17151 ±  3%     -34.2%      11285        interrupts.CPU79.CAL:Function_call_interrupts
    613303           -34.9%     399170        interrupts.CPU79.LOC:Local_timer_interrupts
      6341 ±  3%     -20.4%       5049 ± 14%  interrupts.CPU79.RES:Rescheduling_interrupts
     15184 ±  3%     -35.6%       9786 ±  2%  interrupts.CPU79.TLB:TLB_shootdowns
    181.00 ± 13%     -40.7%     107.25 ±  2%  interrupts.CPU8.42:IR-PCI-MSI.1572872-edge.eth0-TxRx-8
     17199 ±  3%     -34.4%      11289 ±  4%  interrupts.CPU8.CAL:Function_call_interrupts
    613751           -34.9%     399458        interrupts.CPU8.LOC:Local_timer_interrupts
      6973 ±  3%     -32.8%       4687 ±  7%  interrupts.CPU8.RES:Rescheduling_interrupts
     15590 ±  3%     -36.0%       9972 ±  4%  interrupts.CPU8.TLB:TLB_shootdowns
     17166 ±  2%     -34.2%      11291 ±  2%  interrupts.CPU80.CAL:Function_call_interrupts
    613399           -35.0%     399004        interrupts.CPU80.LOC:Local_timer_interrupts
      6620 ±  7%     -24.0%       5029 ± 14%  interrupts.CPU80.RES:Rescheduling_interrupts
     15230 ±  2%     -35.1%       9883 ±  4%  interrupts.CPU80.TLB:TLB_shootdowns
     17030 ±  3%     -34.6%      11138        interrupts.CPU81.CAL:Function_call_interrupts
    612912           -34.9%     398899        interrupts.CPU81.LOC:Local_timer_interrupts
      6721 ±  3%     -23.5%       5140 ± 13%  interrupts.CPU81.RES:Rescheduling_interrupts
     15109 ±  3%     -35.6%       9732        interrupts.CPU81.TLB:TLB_shootdowns
     17216 ±  2%     -33.5%      11445        interrupts.CPU82.CAL:Function_call_interrupts
    613178           -34.9%     399124        interrupts.CPU82.LOC:Local_timer_interrupts
      6755 ±  6%     -24.1%       5127 ± 16%  interrupts.CPU82.RES:Rescheduling_interrupts
     15239 ±  4%     -34.6%       9962 ±  3%  interrupts.CPU82.TLB:TLB_shootdowns
     17064 ±  2%     -33.6%      11324 ±  2%  interrupts.CPU83.CAL:Function_call_interrupts
    613016           -34.9%     398830        interrupts.CPU83.LOC:Local_timer_interrupts
      6518 ±  4%     -24.4%       4924 ± 14%  interrupts.CPU83.RES:Rescheduling_interrupts
     14901 ±  4%     -33.8%       9860 ±  2%  interrupts.CPU83.TLB:TLB_shootdowns
     16943 ±  4%     -33.3%      11307 ±  2%  interrupts.CPU84.CAL:Function_call_interrupts
    613118           -35.0%     398825        interrupts.CPU84.LOC:Local_timer_interrupts
      6408           -34.9%       4174 ±  3%  interrupts.CPU84.RES:Rescheduling_interrupts
     15001 ±  5%     -34.2%       9873 ±  5%  interrupts.CPU84.TLB:TLB_shootdowns
     17026 ±  3%     -33.0%      11409 ±  2%  interrupts.CPU85.CAL:Function_call_interrupts
    613015           -34.8%     399382        interrupts.CPU85.LOC:Local_timer_interrupts
      6570 ±  2%     -36.2%       4194 ±  2%  interrupts.CPU85.RES:Rescheduling_interrupts
     15036 ±  3%     -33.9%       9941 ±  3%  interrupts.CPU85.TLB:TLB_shootdowns
     16974 ±  3%     -32.7%      11421 ±  2%  interrupts.CPU86.CAL:Function_call_interrupts
    613071           -34.9%     398943        interrupts.CPU86.LOC:Local_timer_interrupts
      6707 ±  3%     -39.4%       4065        interrupts.CPU86.RES:Rescheduling_interrupts
     14987 ±  4%     -33.4%       9984 ±  2%  interrupts.CPU86.TLB:TLB_shootdowns
     17508 ±  3%     -34.1%      11539 ±  2%  interrupts.CPU87.CAL:Function_call_interrupts
    612946           -34.9%     398784        interrupts.CPU87.LOC:Local_timer_interrupts
      6458           -34.9%       4206 ±  2%  interrupts.CPU87.RES:Rescheduling_interrupts
     15514 ±  3%     -35.0%      10083 ±  3%  interrupts.CPU87.TLB:TLB_shootdowns
    234.25 ± 38%     -46.2%     126.00 ± 15%  interrupts.CPU9.43:IR-PCI-MSI.1572873-edge.eth0-TxRx-9
     17193 ±  2%     -33.8%      11381 ±  3%  interrupts.CPU9.CAL:Function_call_interrupts
    613335           -34.9%     399343        interrupts.CPU9.LOC:Local_timer_interrupts
      6942 ±  2%     -32.9%       4659 ±  5%  interrupts.CPU9.RES:Rescheduling_interrupts
     15489 ±  3%     -35.3%      10023 ±  5%  interrupts.CPU9.TLB:TLB_shootdowns
  53960435           -34.9%   35125153        interrupts.LOC:Local_timer_interrupts
    608241           -31.1%     419265        interrupts.RES:Rescheduling_interrupts
   1348227 ±  3%     -35.0%     876474 ±  2%  interrupts.TLB:TLB_shootdowns



***************************************************************************************************
lkp-bdw-ex2: 160 threads Intel(R) Xeon(R) CPU E7-8890 v4 @ 2.20GHz with 256G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase/ucode:
  gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-2018-04-03.cgz/300s/16G/lkp-bdw-ex2/shm-pread-rand/vm-scalability/0xb00002e

commit: 
  8055fb607e ("vfs: Convert smackfs to fs_context")
  9d65ac8a35 ("tmpfs, devtmpfs, ramfs, rootfs: Convert to fs_context")

8055fb607ebc82ea 9d65ac8a35a75c7334cfa0f8552 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    215.00 ±  6%     -57.5%      91.46        vm-scalability.free_time
    114519           -40.5%      68132        vm-scalability.median
  21956336           -40.4%   13080254        vm-scalability.throughput
    562.26 ±  2%     -26.2%     414.68        vm-scalability.time.elapsed_time
    562.26 ±  2%     -26.2%     414.68        vm-scalability.time.elapsed_time.max
    243763 ±  3%     -24.7%     183433 ±  3%  vm-scalability.time.involuntary_context_switches
 2.476e+08           -50.0%  1.238e+08        vm-scalability.time.maximum_resident_set_size
 9.302e+08           -50.0%  4.651e+08        vm-scalability.time.minor_page_faults
     17588            +3.0%      18124        vm-scalability.time.percent_of_cpu_this_job_got
     66031 ±  4%     -55.3%      29529 ±  2%  vm-scalability.time.system_time
     32874           +38.8%      45629        vm-scalability.time.user_time
 6.598e+09           -40.4%  3.929e+09        vm-scalability.workload
 2.271e+08 ±160%     -96.1%    8752847 ±134%  cpuidle.C1E.time
   2191304 ±160%     -96.6%      73511 ±118%  cpuidle.C1E.usage
      8.46 ±  4%      -2.8        5.70 ±  3%  mpstat.cpu.all.idle%
     61.08           -24.0       37.06 ±  2%  mpstat.cpu.all.sys%
     30.46 ±  2%     +26.8       57.24        mpstat.cpu.all.usr%
      2379            +2.9%       2448        turbostat.Avg_MHz
   2187139 ±161%     -96.7%      71800 ±121%  turbostat.C1E
 2.084e+08 ±  5%     -23.2%    1.6e+08 ±  3%  turbostat.IRQ
      3.76 ±  8%     -35.2%       2.44 ± 11%  turbostat.Pkg%pc2
     49.49           +13.7%      56.25        turbostat.RAMWatt
     60.50           -39.7%      36.50 ±  2%  vmstat.cpu.sy
     30.00 ±  2%     +88.3%      56.50        vmstat.cpu.us
 2.176e+08           -47.3%  1.148e+08        vmstat.memory.cache
 1.955e+08           +72.4%  3.369e+08        vmstat.memory.free
      1731            -7.7%       1598        vmstat.system.cs
      5172 ±  9%     +18.9%       6147 ±  7%  slabinfo.kmalloc-rcl-64.active_objs
      5172 ±  9%     +18.9%       6147 ±  7%  slabinfo.kmalloc-rcl-64.num_objs
      2256 ±  2%     +80.7%       4077 ± 24%  slabinfo.pool_workqueue.active_objs
      2339 ±  2%     +77.8%       4160 ± 23%  slabinfo.pool_workqueue.num_objs
    953219           -48.1%     495186        slabinfo.radix_tree_node.active_objs
     17762           -48.8%       9086        slabinfo.radix_tree_node.active_slabs
    994715           -48.8%     508833        slabinfo.radix_tree_node.num_objs
     17762           -48.8%       9086        slabinfo.radix_tree_node.num_slabs
  21827780 ±  2%     -49.2%   11094619        numa-numastat.node0.local_node
  21836923 ±  2%     -49.1%   11111700        numa-numastat.node0.numa_hit
   5144327 ±  7%    -100.0%       0.00        numa-numastat.node0.numa_miss
   5153471 ±  7%     -99.7%      17082 ± 15%  numa-numastat.node0.other_node
  21688031           -50.9%   10645806        numa-numastat.node1.local_node
  21702081           -50.9%   10664473        numa-numastat.node1.numa_hit
  21521849           -50.2%   10714943        numa-numastat.node2.local_node
  21540579           -50.2%   10725912        numa-numastat.node2.numa_hit
  16542134           -31.6%   11315887        numa-numastat.node3.local_node
   5144327 ±  7%    -100.0%       0.00        numa-numastat.node3.numa_foreign
  16556145           -31.6%   11325232        numa-numastat.node3.numa_hit
  56607469 ±  4%     -69.7%   17172207        meminfo.Active
  56607225 ±  4%     -69.7%   17171908        meminfo.Active(anon)
 2.171e+08           -47.3%  1.144e+08        meminfo.Cached
     68155 ± 28%    +188.9%     196900        meminfo.CmaFree
 2.267e+08           -48.6%  1.164e+08        meminfo.Committed_AS
 1.595e+08           -39.7%   96248802        meminfo.Inactive
 1.595e+08           -39.7%   96247436        meminfo.Inactive(anon)
    655863           -42.3%     378738        meminfo.KReclaimable
 1.593e+08           -39.7%   96010805        meminfo.Mapped
 1.941e+08           +72.9%  3.355e+08        meminfo.MemAvailable
 1.954e+08           +72.4%  3.369e+08        meminfo.MemFree
 2.998e+08           -47.2%  1.582e+08        meminfo.Memused
      1488 ± 31%     -39.2%     904.50 ± 61%  meminfo.Mlocked
  80775082           -47.6%   42297345        meminfo.PageTables
    655863           -42.3%     378738        meminfo.SReclaimable
 2.159e+08           -47.6%  1.132e+08        meminfo.Shmem
    884970           -31.5%     606435        meminfo.Slab
    629206 ±  2%     -31.4%     431746        meminfo.max_used_kB
    346175 ±  4%    -100.0%       0.00        proc-vmstat.compact_daemon_free_scanned
    346175 ±  4%    -100.0%       0.00        proc-vmstat.compact_free_scanned
    570.00 ± 62%    -100.0%       0.00        proc-vmstat.compact_isolated
  14103663 ±  5%     -69.7%    4275680        proc-vmstat.nr_active_anon
   4841612           +72.8%    8368661        proc-vmstat.nr_dirty_background_threshold
   9695080           +72.8%   16757792        proc-vmstat.nr_dirty_threshold
  54241834           -47.3%   28609921        proc-vmstat.nr_file_pages
     17063 ± 28%    +188.5%      49225        proc-vmstat.nr_free_cma
  48899002           +72.2%   84221154        proc-vmstat.nr_free_pages
  39893226           -39.6%   24091372        proc-vmstat.nr_inactive_anon
  39833592           -39.7%   24032103        proc-vmstat.nr_mapped
    371.75 ± 31%     -39.4%     225.25 ± 61%  proc-vmstat.nr_mlock
  20181469           -47.6%   10577257        proc-vmstat.nr_page_table_pages
  53934410           -47.5%   28304267        proc-vmstat.nr_shmem
    163883           -42.2%      94716        proc-vmstat.nr_slab_reclaimable
  14103663 ±  5%     -69.7%    4275680        proc-vmstat.nr_zone_active_anon
  39893226           -39.6%   24091372        proc-vmstat.nr_zone_inactive_anon
   5144327 ±  7%    -100.0%       0.00        proc-vmstat.numa_foreign
    795.75 ± 64%     -57.6%     337.75 ± 37%  proc-vmstat.numa_hint_faults_local
  81660157           -46.3%   43850512        proc-vmstat.numa_hit
  81604218           -46.3%   43794445        proc-vmstat.numa_local
   5144327 ±  7%    -100.0%       0.00        proc-vmstat.numa_miss
   5200265 ±  7%     -98.9%      56067        proc-vmstat.numa_other
     46220 ± 13%     +29.0%      59622 ± 20%  proc-vmstat.numa_pte_updates
  61922169           -50.0%   30971532        proc-vmstat.pgactivate
  87033754           -49.4%   44002601        proc-vmstat.pgalloc_normal
 9.321e+08           -50.0%  4.665e+08        proc-vmstat.pgfault
  86980887           -49.6%   43814302        proc-vmstat.pgfree
      3767 ±154%    -100.0%       0.00        proc-vmstat.slabs_scanned
  50357480 ±  5%     -27.4%   36579520        sched_debug.cfs_rq:/.min_vruntime.avg
  50560150 ±  5%     -27.4%   36730022        sched_debug.cfs_rq:/.min_vruntime.max
  44524469 ±  5%     -23.4%   34123234        sched_debug.cfs_rq:/.min_vruntime.min
    590148 ± 10%     -57.1%     253421 ±  5%  sched_debug.cfs_rq:/.min_vruntime.stddev
     81.25 ± 57%     +76.9%     143.71        sched_debug.cfs_rq:/.removed.load_avg.max
      3747 ± 57%     +77.5%       6653        sched_debug.cfs_rq:/.removed.runnable_sum.max
     35.17 ± 64%     +94.5%      68.39 ± 10%  sched_debug.cfs_rq:/.removed.util_avg.max
    589754 ± 10%     -57.1%     252737 ±  5%  sched_debug.cfs_rq:/.spread0.stddev
      4.53 ± 59%     -81.1%       0.86        sched_debug.cfs_rq:/.util_est_enqueued.min
    235963 ± 28%     +75.9%     415151 ±  6%  sched_debug.cpu.avg_idle.min
    303930 ±  4%     -22.1%     236626        sched_debug.cpu.clock.avg
    303960 ±  4%     -22.1%     236658        sched_debug.cpu.clock.max
    303901 ±  4%     -22.1%     236595        sched_debug.cpu.clock.min
    303930 ±  4%     -22.1%     236626        sched_debug.cpu.clock_task.avg
    303960 ±  4%     -22.1%     236658        sched_debug.cpu.clock_task.max
    303901 ±  4%     -22.1%     236595        sched_debug.cpu.clock_task.min
     96.87 ±122%     -76.6%      22.68 ± 74%  sched_debug.cpu.cpu_load[1].max
     96.93 ±120%     -76.1%      23.18 ± 61%  sched_debug.cpu.cpu_load[2].max
     15.84 ±150%     -89.5%       1.66 ± 58%  sched_debug.cpu.cpu_load[2].stddev
    111.17 ±103%     -73.4%      29.54 ± 14%  sched_debug.cpu.cpu_load[4].max
     16.80 ±142%     -87.9%       2.03 ± 18%  sched_debug.cpu.cpu_load[4].stddev
      9877 ±  6%     -19.5%       7948 ±  3%  sched_debug.cpu.curr->pid.max
    584.07 ±  7%     -23.1%     448.87 ±  5%  sched_debug.cpu.curr->pid.stddev
    254169 ±  5%     -26.4%     187119        sched_debug.cpu.nr_load_updates.avg
    262340 ±  5%     -25.8%     194755        sched_debug.cpu.nr_load_updates.max
    252688 ±  5%     -26.5%     185657        sched_debug.cpu.nr_load_updates.min
      2718 ±  5%     -25.6%       2022        sched_debug.cpu.nr_switches.avg
     59267 ±  5%     -54.5%      26985 ± 18%  sched_debug.cpu.nr_switches.max
    668.49           -33.8%     442.32 ±  2%  sched_debug.cpu.nr_switches.min
      5785 ±  9%     -48.6%       2976 ±  9%  sched_debug.cpu.nr_switches.stddev
    303901 ±  4%     -22.1%     236595        sched_debug.cpu_clk
    298715 ±  4%     -22.5%     231407        sched_debug.ktime
      2.55 ±  2%     -15.3%       2.16        sched_debug.rt_rq:/.rt_runtime.stddev
    309168 ±  4%     -21.8%     241862        sched_debug.sched_clk
  18471535 ±  5%     -77.0%    4241496 ±  2%  numa-meminfo.node0.Active
  18471412 ±  5%     -77.0%    4241496 ±  2%  numa-meminfo.node0.Active(anon)
     45316 ± 83%     -83.9%       7279 ±169%  numa-meminfo.node0.AnonHugePages
  69167055           -58.7%   28537182        numa-meminfo.node0.FilePages
  50441137           -52.4%   24013491        numa-meminfo.node0.Inactive
  50440681           -52.4%   24013432        numa-meminfo.node0.Inactive(anon)
    175696 ± 16%     -33.9%     116201 ± 21%  numa-meminfo.node0.KReclaimable
  50371492           -52.5%   23946772        numa-meminfo.node0.Mapped
  41199474 ±  2%    +124.0%   92268263        numa-meminfo.node0.MemFree
  90697497           -56.3%   39648428        numa-meminfo.node0.MemUsed
  20892011           -49.3%   10586665        numa-meminfo.node0.PageTables
    175696 ± 16%     -33.9%     116201 ± 21%  numa-meminfo.node0.SReclaimable
  68851086           -59.0%   28236379        numa-meminfo.node0.Shmem
    315510 ±  5%      -4.7%     300790        numa-meminfo.node0.Unevictable
  14364208 ±  4%     -70.7%    4203376        numa-meminfo.node1.Active
  14364116 ±  4%     -70.7%    4203357        numa-meminfo.node1.Active(anon)
  54786209           -48.6%   28143332        numa-meminfo.node1.FilePages
  40191119 ±  2%     -41.1%   23668090        numa-meminfo.node1.Inactive
  40190539 ±  2%     -41.1%   23667849        numa-meminfo.node1.Inactive(anon)
    172189 ± 12%     -67.2%      56471 ±  9%  numa-meminfo.node1.KReclaimable
  40076004           -40.9%   23670206        numa-meminfo.node1.Mapped
  23566368 ±  2%    +156.0%   60331290        numa-meminfo.node1.MemFree
  75508237           -48.7%   38730169        numa-meminfo.node1.MemUsed
  20255361           -48.9%   10354337        numa-meminfo.node1.PageTables
    172189 ± 12%     -67.2%      56471 ±  9%  numa-meminfo.node1.SReclaimable
  54478406           -48.9%   27844350        numa-meminfo.node1.Shmem
    227127 ± 12%     -55.3%     101579 ±  8%  numa-meminfo.node1.Slab
  14137268 ±  5%     -69.3%    4335736        numa-meminfo.node2.Active
  14137268 ±  5%     -69.3%    4335664        numa-meminfo.node2.Active(anon)
  53846613           -47.3%   28403150        numa-meminfo.node2.FilePages
  39490579           -39.6%   23864915        numa-meminfo.node2.Inactive
  39490577           -39.6%   23864569        numa-meminfo.node2.Inactive(anon)
    140893 ± 11%     -44.2%      78596 ± 32%  numa-meminfo.node2.KReclaimable
  39492831           -39.6%   23867473        numa-meminfo.node2.Mapped
 1.239e+08           +28.3%  1.589e+08        numa-meminfo.node2.MemFree
  74275591           -47.2%   39218377        numa-meminfo.node2.MemUsed
  20036743           -47.7%   10479351        numa-meminfo.node2.PageTables
    140893 ± 11%     -44.2%      78596 ± 32%  numa-meminfo.node2.SReclaimable
  53547614           -47.5%   28100479        numa-meminfo.node2.Shmem
    188330 ± 11%     -34.7%     122900 ± 26%  numa-meminfo.node2.Slab
   9814197 ±  4%     -54.8%    4433185        numa-meminfo.node3.Active
   9814168 ±  4%     -54.8%    4432977        numa-meminfo.node3.Active(anon)
  38885936           -24.9%   29201047        numa-meminfo.node3.FilePages
  28797354           -14.7%   24554479        numa-meminfo.node3.Inactive
  28797285           -14.7%   24553758        numa-meminfo.node3.Inactive(anon)
  28739603           -15.2%   24378346        numa-meminfo.node3.Mapped
   7415082 ±  4%    +245.2%   25598601        numa-meminfo.node3.MemFree
  58633632           -31.0%   40450113        numa-meminfo.node3.MemUsed
  19344684           -44.3%   10783400        numa-meminfo.node3.PageTables
     45389 ± 15%     +37.2%      62274 ±  5%  numa-meminfo.node3.SUnreclaim
  38579011           -25.1%   28880887        numa-meminfo.node3.Shmem
     16.01           +44.7%      23.15 ±  2%  perf-stat.i.MPKI
 1.919e+10           -17.3%  1.586e+10        perf-stat.i.branch-instructions
  30643400 ±  3%     -27.7%   22140532 ±  2%  perf-stat.i.branch-misses
     53.75            +7.8       61.51        perf-stat.i.cache-miss-rate%
 5.552e+08 ±  2%     +44.4%  8.019e+08 ±  2%  perf-stat.i.cache-misses
  7.87e+08 ±  2%     +42.9%  1.124e+09 ±  2%  perf-stat.i.cache-references
      1685            -8.5%       1542        perf-stat.i.context-switches
      6.39           +23.8%       7.91        perf-stat.i.cpi
 4.532e+11            +3.2%  4.677e+11        perf-stat.i.cpu-cycles
     35.20           +12.3%      39.54        perf-stat.i.cpu-migrations
      2347 ±  5%     -36.1%       1499 ±  6%  perf-stat.i.cycles-between-cache-misses
      2.07 ±  2%      +1.3        3.37        perf-stat.i.dTLB-load-miss-rate%
 3.278e+08 ±  2%     +55.0%  5.079e+08 ±  2%  perf-stat.i.dTLB-load-misses
 2.133e+10           -14.1%  1.832e+10        perf-stat.i.dTLB-loads
 3.629e+09 ±  2%     +18.2%  4.291e+09        perf-stat.i.dTLB-stores
     82.40 ±  3%      +4.4       86.76        perf-stat.i.iTLB-load-miss-rate%
   3768060 ±  3%     -23.6%    2880473 ±  9%  perf-stat.i.iTLB-load-misses
 8.048e+10           -16.5%  6.722e+10        perf-stat.i.instructions
    476164 ± 21%     -22.8%     367493 ± 13%  perf-stat.i.instructions-per-iTLB-miss
      0.18           -19.1%       0.15        perf-stat.i.ipc
   1685065           -33.5%    1120936        perf-stat.i.minor-faults
     70.03           -11.7       58.35        perf-stat.i.node-load-miss-rate%
 2.147e+08 ±  2%     +50.7%  3.235e+08        perf-stat.i.node-load-misses
 2.767e+08 ±  4%     +57.7%  4.365e+08 ±  5%  perf-stat.i.node-loads
     56.11 ±  2%      -6.8       49.31 ±  3%  perf-stat.i.node-store-miss-rate%
  17332505 ±  4%     -29.8%   12161421        perf-stat.i.node-store-misses
   5218571 ±  3%     -32.7%    3514411        perf-stat.i.node-stores
   1685239           -33.5%    1121158        perf-stat.i.page-faults
      9.29 ±  3%     +72.7%      16.04 ±  2%  perf-stat.overall.MPKI
      0.16 ±  3%      -0.0        0.14 ±  3%  perf-stat.overall.branch-miss-rate%
     69.96            +1.1       71.07        perf-stat.overall.cache-miss-rate%
      5.56           +23.1%       6.84        perf-stat.overall.cpi
    855.85 ±  3%     -29.9%     600.33 ±  2%  perf-stat.overall.cycles-between-cache-misses
      1.43 ±  4%      +1.2        2.58 ±  2%  perf-stat.overall.dTLB-load-miss-rate%
      0.18           -18.8%       0.15        perf-stat.overall.ipc
 1.953e+10           -17.4%  1.614e+10        perf-stat.ps.branch-instructions
  30805276 ±  3%     -27.0%   22503111 ±  2%  perf-stat.ps.branch-misses
 5.312e+08 ±  3%     +46.5%  7.784e+08 ±  3%  perf-stat.ps.cache-misses
 7.592e+08 ±  2%     +44.3%  1.095e+09 ±  2%  perf-stat.ps.cache-references
      1690            -8.4%       1548        perf-stat.ps.context-switches
 4.541e+11            +2.8%  4.669e+11        perf-stat.ps.cpu-cycles
     35.44           +12.2%      39.76        perf-stat.ps.cpu-migrations
 3.126e+08 ±  3%     +57.3%  4.918e+08 ±  2%  perf-stat.ps.dTLB-load-misses
  2.16e+10           -14.2%  1.854e+10        perf-stat.ps.dTLB-loads
 3.525e+09 ±  2%     +19.2%  4.204e+09        perf-stat.ps.dTLB-stores
   3719883 ±  3%     -22.2%    2895285 ±  9%  perf-stat.ps.iTLB-load-misses
 8.174e+10           -16.5%  6.826e+10        perf-stat.ps.instructions
   1656530 ±  2%     -32.2%    1123747        perf-stat.ps.minor-faults
  2.06e+08 ±  2%     +52.6%  3.142e+08        perf-stat.ps.node-load-misses
 2.622e+08 ±  4%     +60.9%   4.22e+08 ±  5%  perf-stat.ps.node-loads
  17222639 ±  4%     -28.9%   12250027        perf-stat.ps.node-store-misses
   5214549 ±  3%     -31.6%    3566921        perf-stat.ps.node-stores
   1656530 ±  2%     -32.2%    1123747        perf-stat.ps.page-faults
 4.603e+13 ±  3%     -38.4%  2.834e+13        perf-stat.total.instructions
   4608245 ±  5%     -76.9%    1063287 ±  2%  numa-vmstat.node0.nr_active_anon
  17297000           -58.8%    7131761        numa-vmstat.node0.nr_file_pages
  10294951 ±  2%    +124.1%   23070382        numa-vmstat.node0.nr_free_pages
  12625019           -52.5%    5997916        numa-vmstat.node0.nr_inactive_anon
    113.50 ± 75%     -87.2%      14.50 ±165%  numa-vmstat.node0.nr_inactive_file
  12607751           -52.6%    5981237        numa-vmstat.node0.nr_mapped
   5222723           -49.3%    2645940        numa-vmstat.node0.nr_page_table_pages
  17218007           -59.0%    7056560        numa-vmstat.node0.nr_shmem
     43945 ± 17%     -33.9%      29046 ± 21%  numa-vmstat.node0.nr_slab_reclaimable
     78877 ±  5%      -4.7%      75197        numa-vmstat.node0.nr_unevictable
   4608244 ±  5%     -76.9%    1063286 ±  2%  numa-vmstat.node0.nr_zone_active_anon
  12625019           -52.5%    5997917        numa-vmstat.node0.nr_zone_inactive_anon
    113.50 ± 75%     -87.2%      14.50 ±165%  numa-vmstat.node0.nr_zone_inactive_file
     78877 ±  5%      -4.7%      75197        numa-vmstat.node0.nr_zone_unevictable
  20496209           -47.2%   10820663        numa-vmstat.node0.numa_hit
  20485842           -47.3%   10802835        numa-vmstat.node0.numa_local
   4355055 ±  7%    -100.0%       0.00        numa-vmstat.node0.numa_miss
   4365424 ±  7%     -99.6%      17829 ± 17%  numa-vmstat.node0.numa_other
   3583662 ±  4%     -70.6%    1053714        numa-vmstat.node1.nr_active_anon
  13700707           -48.7%    7033341        numa-vmstat.node1.nr_file_pages
   5886203 ±  2%    +156.3%   15086088        numa-vmstat.node1.nr_free_pages
  10059153 ±  2%     -41.2%    5911596        numa-vmstat.node1.nr_inactive_anon
  10030599           -41.1%    5912140        numa-vmstat.node1.nr_mapped
   5065074           -48.9%    2587895        numa-vmstat.node1.nr_page_table_pages
  13623756           -48.9%    6958596        numa-vmstat.node1.nr_shmem
     43066 ± 12%     -67.2%      14114 ±  9%  numa-vmstat.node1.nr_slab_reclaimable
   3583661 ±  4%     -70.6%    1053713        numa-vmstat.node1.nr_zone_active_anon
  10059153 ±  2%     -41.2%    5911596        numa-vmstat.node1.nr_zone_inactive_anon
  20377141           -49.2%   10344159        numa-vmstat.node1.numa_hit
  20266558           -49.5%   10229369        numa-vmstat.node1.numa_local
   3527033 ±  5%     -69.2%    1086778        numa-vmstat.node2.nr_active_anon
  13465915           -47.3%    7098298        numa-vmstat.node2.nr_file_pages
  30968456           +28.3%   39739868        numa-vmstat.node2.nr_free_pages
   9884179           -39.7%    5960783        numa-vmstat.node2.nr_inactive_anon
   9884755           -39.7%    5961458        numa-vmstat.node2.nr_mapped
     97.75 ± 52%     -51.7%      47.25 ± 59%  numa-vmstat.node2.nr_mlock
   5010526           -47.7%    2619130        numa-vmstat.node2.nr_page_table_pages
  13391165           -47.6%    7022630        numa-vmstat.node2.nr_shmem
     35241 ± 11%     -44.3%      19645 ± 32%  numa-vmstat.node2.nr_slab_reclaimable
   3527031 ±  5%     -69.2%    1086777        numa-vmstat.node2.nr_zone_active_anon
   9884180           -39.7%    5960783        numa-vmstat.node2.nr_zone_inactive_anon
  20161532           -48.2%   10447582        numa-vmstat.node2.numa_hit
  20046512           -48.4%   10340782 ±  2%  numa-vmstat.node2.numa_local
   2448498 ±  4%     -54.6%    1111165        numa-vmstat.node3.nr_active_anon
   9724748           -25.0%    7297701        numa-vmstat.node3.nr_file_pages
     17049 ± 28%    +188.7%      49225        numa-vmstat.node3.nr_free_cma
   1849256 ±  5%    +246.3%    6403109        numa-vmstat.node3.nr_free_pages
   7207631           -14.9%    6132959        numa-vmstat.node3.nr_inactive_anon
   7193198           -15.3%    6089173        numa-vmstat.node3.nr_mapped
   4837484           -44.3%    2695110        numa-vmstat.node3.nr_page_table_pages
   9648017           -25.2%    7217661        numa-vmstat.node3.nr_shmem
     11347 ± 15%     +37.2%      15568 ±  5%  numa-vmstat.node3.nr_slab_unreclaimable
   2448498 ±  4%     -54.6%    1111165        numa-vmstat.node3.nr_zone_active_anon
   7207633           -14.9%    6132959        numa-vmstat.node3.nr_zone_inactive_anon
   4355515 ±  7%    -100.0%       0.00        numa-vmstat.node3.numa_foreign
  15875416           -31.8%   10829284        numa-vmstat.node3.numa_hit
  15765227           -32.0%   10724232        numa-vmstat.node3.numa_local
     80.75           -79.3        1.45 ± 32%  perf-profile.calltrace.cycles-pp.page_fault
     80.62           -79.2        1.45 ± 32%  perf-profile.calltrace.cycles-pp.do_page_fault.page_fault
     79.91           -78.5        1.41 ± 32%  perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault
     76.84           -75.7        1.18 ± 31%  perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
     75.72           -74.6        1.10 ± 31%  perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
     73.06           -72.3        0.73 ± 62%  perf-profile.calltrace.cycles-pp.filemap_map_pages.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
     13.91 ±  3%     -13.9        0.00        perf-profile.calltrace.cycles-pp.xas_find.filemap_map_pages.__handle_mm_fault.handle_mm_fault.__do_page_fault
     13.02 ±  2%     -13.0        0.00        perf-profile.calltrace.cycles-pp.xas_load.xas_find.filemap_map_pages.__handle_mm_fault.handle_mm_fault
      0.00            +0.6        0.56 ±  4%  perf-profile.calltrace.cycles-pp.__hrtimer_next_event_base.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      0.00            +0.6        0.57 ±  3%  perf-profile.calltrace.cycles-pp.account_entity_dequeue.reweight_entity.task_tick_fair.scheduler_tick.update_process_times
      0.00            +0.6        0.61 ± 10%  perf-profile.calltrace.cycles-pp.tick_program_event.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      0.00            +0.7        0.68 ± 11%  perf-profile.calltrace.cycles-pp.ktime_get.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt
      0.00            +0.8        0.77 ± 13%  perf-profile.calltrace.cycles-pp.__do_execve_file.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
      0.00            +0.8        0.78 ± 12%  perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
      0.00            +0.8        0.78 ± 13%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.execve
      0.00            +0.8        0.78 ± 13%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
      0.00            +0.8        0.78 ± 12%  perf-profile.calltrace.cycles-pp.execve
      0.00            +0.8        0.78 ± 22%  perf-profile.calltrace.cycles-pp.note_gp_changes.rcu_core.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt
      0.00            +0.8        0.79 ± 20%  perf-profile.calltrace.cycles-pp.perf_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +0.9        0.89 ± 23%  perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__read_nocancel
      0.00            +0.9        0.91 ±  8%  perf-profile.calltrace.cycles-pp.read_tsc.ktime_get_update_offsets_now.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      0.00            +0.9        0.92 ±  9%  perf-profile.calltrace.cycles-pp.timerqueue_add.enqueue_hrtimer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt
      0.00            +0.9        0.92 ± 28%  perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
      0.00            +0.9        0.94 ± 26%  perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
      0.00            +0.9        0.95 ± 23%  perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__read_nocancel.perf_evsel__read_counter
      0.00            +1.0        0.95 ± 22%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__read_nocancel.perf_evsel__read_counter.read_counters
      0.00            +1.0        0.96 ± 23%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__read_nocancel.perf_evsel__read_counter.read_counters.process_interval
      0.00            +1.0        1.02 ± 25%  perf-profile.calltrace.cycles-pp.__read_nocancel.perf_evsel__read_counter.read_counters.process_interval.__run_perf_stat
      0.00            +1.0        1.02 ± 15%  perf-profile.calltrace.cycles-pp.native_write_msr.__intel_pmu_enable_all.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt
      0.00            +1.0        1.03 ±  5%  perf-profile.calltrace.cycles-pp.enqueue_hrtimer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      0.00            +1.0        1.03 ± 17%  perf-profile.calltrace.cycles-pp.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write
      0.00            +1.0        1.05 ± 16%  perf-profile.calltrace.cycles-pp.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write.ksys_write
      0.00            +1.1        1.06 ± 11%  perf-profile.calltrace.cycles-pp.perf_event_task_tick.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer
      0.00            +1.1        1.07 ± 17%  perf-profile.calltrace.cycles-pp.generic_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
      0.00            +1.1        1.08 ± 16%  perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +1.1        1.08 ±  7%  perf-profile.calltrace.cycles-pp.rcu_irq_enter.irq_enter.smp_apic_timer_interrupt.apic_timer_interrupt
      0.00            +1.1        1.11 ± 17%  perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
      0.00            +1.1        1.12 ± 12%  perf-profile.calltrace.cycles-pp.__intel_pmu_enable_all.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt
      0.00            +1.1        1.13 ± 20%  perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write.record__pushfn
      0.00            +1.1        1.15 ± 22%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write.record__pushfn.perf_mmap__push
      0.00            +1.1        1.15 ± 22%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write.record__pushfn.perf_mmap__push.record__mmap_read_evlist
      0.00            +1.2        1.23 ± 21%  perf-profile.calltrace.cycles-pp.__libc_write.record__pushfn.perf_mmap__push.record__mmap_read_evlist.cmd_record
      0.00            +1.2        1.23 ± 19%  perf-profile.calltrace.cycles-pp.ret_from_fork
      0.00            +1.2        1.23 ± 19%  perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
      0.00            +1.2        1.24 ± 21%  perf-profile.calltrace.cycles-pp.record__pushfn.perf_mmap__push.record__mmap_read_evlist.cmd_record.run_builtin
      0.00            +1.2        1.25 ± 11%  perf-profile.calltrace.cycles-pp.prepare_exit_to_usermode.swapgs_restore_regs_and_return_to_usermode
      0.00            +1.3        1.30 ±  8%  perf-profile.calltrace.cycles-pp.update_rq_clock.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer
      0.00            +1.4        1.35 ± 10%  perf-profile.calltrace.cycles-pp.timerqueue_del.__remove_hrtimer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt
      0.00            +1.5        1.48 ±  8%  perf-profile.calltrace.cycles-pp.reweight_entity.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle
      0.00            +1.5        1.51 ± 27%  perf-profile.calltrace.cycles-pp.perf_mmap__push.record__mmap_read_evlist.cmd_record.run_builtin.main
      0.00            +1.5        1.52 ± 16%  perf-profile.calltrace.cycles-pp.rcu_core.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
      0.00            +1.5        1.54 ± 10%  perf-profile.calltrace.cycles-pp.update_cfs_group.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle
      0.00            +1.6        1.57 ± 27%  perf-profile.calltrace.cycles-pp.record__mmap_read_evlist.cmd_record.run_builtin.main.generic_start_main
      0.00            +1.7        1.66 ±  5%  perf-profile.calltrace.cycles-pp.run_timer_softirq.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
      0.00            +1.7        1.70 ±  4%  perf-profile.calltrace.cycles-pp.rcu_sched_clock_irq.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      0.00            +1.8        1.81 ± 11%  perf-profile.calltrace.cycles-pp.__remove_hrtimer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      0.00            +1.8        1.82 ± 38%  perf-profile.calltrace.cycles-pp.ktime_get.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt
      0.00            +1.8        1.84 ±  3%  perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle
      0.00            +1.9        1.88 ± 25%  perf-profile.calltrace.cycles-pp.cmd_record.run_builtin.main.generic_start_main.__strcspn_sse42
      0.00            +1.9        1.94 ±  7%  perf-profile.calltrace.cycles-pp.__update_load_avg_se.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle
      0.00            +2.0        1.99 ± 10%  perf-profile.calltrace.cycles-pp.native_apic_msr_eoi_write.smp_apic_timer_interrupt.apic_timer_interrupt
      2.86 ±  2%      +2.1        4.94 ±  5%  perf-profile.calltrace.cycles-pp.swapgs_restore_regs_and_return_to_usermode
      0.00            +2.3        2.32 ± 10%  perf-profile.calltrace.cycles-pp.irq_enter.smp_apic_timer_interrupt.apic_timer_interrupt
      0.00            +2.3        2.34 ± 16%  perf-profile.calltrace.cycles-pp.__strcspn_sse42
      0.00            +2.3        2.34 ± 16%  perf-profile.calltrace.cycles-pp.generic_start_main.__strcspn_sse42
      0.00            +2.3        2.34 ± 16%  perf-profile.calltrace.cycles-pp.main.generic_start_main.__strcspn_sse42
      0.00            +2.3        2.34 ± 16%  perf-profile.calltrace.cycles-pp.run_builtin.main.generic_start_main.__strcspn_sse42
      0.00            +2.4        2.35 ±  8%  perf-profile.calltrace.cycles-pp.native_write_msr.lapic_next_deadline.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt
      0.00            +2.6        2.63 ± 12%  perf-profile.calltrace.cycles-pp.hrtimer_active.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle
      0.00            +2.7        2.67 ± 14%  perf-profile.calltrace.cycles-pp.account_user_time.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      0.00            +2.7        2.72 ±  4%  perf-profile.calltrace.cycles-pp.update_curr.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle
      0.28 ±100%      +2.7        3.00 ± 15%  perf-profile.calltrace.cycles-pp.run_local_timers.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      0.00            +3.2        3.17 ± 11%  perf-profile.calltrace.cycles-pp.interrupt_entry.apic_timer_interrupt
      0.13 ±173%      +3.3        3.39 ±  6%  perf-profile.calltrace.cycles-pp.lapic_next_deadline.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      0.64 ±  5%      +3.6        4.28 ± 18%  perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      0.40 ± 57%      +4.1        4.48 ± 16%  perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
      0.70 ±  8%      +4.1        4.82 ± 14%  perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      0.00            +4.2        4.20 ± 12%  perf-profile.calltrace.cycles-pp.interrupt_entry
      0.50 ± 57%      +5.1        5.55 ± 11%  perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
      0.56 ±100%      +5.6        6.18 ± 14%  perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      1.22 ± 36%      +8.7        9.95 ±  8%  perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      1.98 ± 14%     +13.5       15.46 ±  4%  perf-profile.calltrace.cycles-pp.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer
      2.50 ± 16%     +17.3       19.75 ±  4%  perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      3.71 ± 14%     +25.2       28.93 ±  3%  perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
      3.75 ± 14%     +25.4       29.17 ±  3%  perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt
      4.12 ± 15%     +28.0       32.13 ±  4%  perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      5.44 ± 10%     +36.5       41.96 ±  4%  perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      7.87 ± 12%     +52.8       60.66 ±  2%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      9.07 ± 11%     +63.9       72.98 ±  2%  perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt
      9.49 ± 11%     +66.8       76.26 ±  3%  perf-profile.calltrace.cycles-pp.apic_timer_interrupt
     29145 ±  6%     +24.4%      36253 ± 11%  softirqs.CPU0.RCU
    217767 ±  2%     -27.4%     158118        softirqs.CPU0.TIMER
      9218 ±  7%     -37.1%       5801 ±  4%  softirqs.CPU1.SCHED
    216877           -27.2%     157841 ±  2%  softirqs.CPU1.TIMER
     29224 ±  4%     +22.7%      35849 ±  9%  softirqs.CPU10.RCU
    212823           -27.2%     154948        softirqs.CPU10.TIMER
     28608 ±  2%     +24.4%      35592 ± 11%  softirqs.CPU100.RCU
      9960 ±  9%     -44.3%       5547 ±  2%  softirqs.CPU100.SCHED
    217762 ±  3%     -28.2%     156358 ±  2%  softirqs.CPU100.TIMER
     28221 ±  3%     +25.9%      35543 ±  7%  softirqs.CPU101.RCU
     10030 ±  9%     -44.8%       5531 ±  3%  softirqs.CPU101.SCHED
    213330 ±  2%     -27.1%     155497 ±  2%  softirqs.CPU101.TIMER
     30729 ±  6%     +18.2%      36319 ±  9%  softirqs.CPU102.RCU
    211774 ±  2%     -25.5%     157669 ±  2%  softirqs.CPU102.TIMER
     30529 ±  3%     +21.0%      36927 ±  8%  softirqs.CPU103.RCU
    216947 ±  2%     -25.8%     161011 ±  4%  softirqs.CPU103.TIMER
     30213 ±  3%     +20.0%      36247 ±  8%  softirqs.CPU104.RCU
    213075           -26.9%     155812        softirqs.CPU104.TIMER
    209569           -26.6%     153886        softirqs.CPU105.TIMER
     29186 ±  4%     +18.5%      34574 ±  8%  softirqs.CPU106.RCU
    211805           -27.3%     154034        softirqs.CPU106.TIMER
     28524 ±  4%     +23.1%      35122 ±  8%  softirqs.CPU107.RCU
    210468           -26.9%     153951        softirqs.CPU107.TIMER
     27750 ±  4%     +30.7%      36259 ± 11%  softirqs.CPU108.RCU
    210616           -26.2%     155349        softirqs.CPU108.TIMER
    211359           -26.7%     154900        softirqs.CPU109.TIMER
     27937 ±  2%     +29.2%      36106 ±  8%  softirqs.CPU11.RCU
    211910           -26.4%     155958        softirqs.CPU11.TIMER
     28330 ±  5%     +23.2%      34909 ±  9%  softirqs.CPU110.RCU
    209875 ±  2%     -26.6%     154082        softirqs.CPU110.TIMER
     29845           +24.7%      37207 ± 10%  softirqs.CPU111.RCU
    215365           -26.6%     158058        softirqs.CPU111.TIMER
     28227 ±  4%     +18.5%      33462 ±  9%  softirqs.CPU112.RCU
    212386           -26.0%     157093        softirqs.CPU112.TIMER
     28294 ±  3%     +20.8%      34173 ±  7%  softirqs.CPU113.RCU
    214104 ±  2%     -26.5%     157278        softirqs.CPU113.TIMER
     26006 ±  4%     +19.9%      31187 ± 11%  softirqs.CPU114.RCU
    209981 ±  2%     -25.8%     155737        softirqs.CPU114.TIMER
     26352           +23.8%      32622 ± 10%  softirqs.CPU115.RCU
    211348           -26.9%     154524        softirqs.CPU115.TIMER
     25313           +30.1%      32932 ± 10%  softirqs.CPU116.RCU
    209307 ±  2%     -26.2%     154459        softirqs.CPU116.TIMER
     27790 ±  3%     +21.1%      33645 ±  9%  softirqs.CPU117.RCU
     14353 ± 41%     -61.5%       5526        softirqs.CPU117.SCHED
    216127           -27.8%     156022        softirqs.CPU117.TIMER
     27594           +23.7%      34136 ±  9%  softirqs.CPU118.RCU
    213239           -26.2%     157404        softirqs.CPU118.TIMER
     26734 ±  2%     +26.9%      33914 ± 13%  softirqs.CPU119.RCU
    217805 ±  2%     -28.2%     156393        softirqs.CPU119.TIMER
     28426 ±  3%     +28.9%      36632 ±  7%  softirqs.CPU12.RCU
    211554           -26.1%     156337        softirqs.CPU12.TIMER
    214181           -29.5%     151083 ±  3%  softirqs.CPU120.TIMER
     26400 ±  4%     +19.5%      31545 ±  6%  softirqs.CPU121.RCU
    213728           -28.8%     152225 ±  2%  softirqs.CPU121.TIMER
     26915 ±  6%     +18.0%      31770 ±  8%  softirqs.CPU122.RCU
    215720           -29.2%     152784 ±  2%  softirqs.CPU122.TIMER
     26363 ±  4%     +17.3%      30931 ±  9%  softirqs.CPU123.RCU
    213604           -29.6%     150381 ±  2%  softirqs.CPU123.TIMER
     26061 ±  5%     +15.8%      30190 ±  7%  softirqs.CPU124.RCU
    214491           -29.7%     150877 ±  2%  softirqs.CPU124.TIMER
     24418 ±  9%     +21.2%      29595 ±  7%  softirqs.CPU125.RCU
    234269 ± 15%     -36.6%     148545        softirqs.CPU125.TIMER
    213994           -29.3%     151351 ±  2%  softirqs.CPU126.TIMER
    218150           -29.2%     154451 ±  3%  softirqs.CPU127.TIMER
     30129 ±  6%     +20.3%      36251 ±  4%  softirqs.CPU128.RCU
    212778           -29.8%     149285        softirqs.CPU128.TIMER
     28601 ±  4%     +17.1%      33489 ±  7%  softirqs.CPU129.RCU
    211017           -29.6%     148512        softirqs.CPU129.TIMER
     29009 ±  2%     +27.8%      37076 ± 13%  softirqs.CPU13.RCU
    211553           -25.7%     157235        softirqs.CPU13.TIMER
     30054 ±  2%     +14.6%      34428 ±  5%  softirqs.CPU130.RCU
    211668           -29.5%     149289        softirqs.CPU130.TIMER
    210962           -30.1%     147500        softirqs.CPU131.TIMER
     27424 ±  3%     +25.1%      34313 ± 10%  softirqs.CPU132.RCU
    210509           -29.7%     148005        softirqs.CPU132.TIMER
     28479 ±  5%     +18.3%      33678 ± 10%  softirqs.CPU133.RCU
    209659           -29.4%     148095        softirqs.CPU133.TIMER
     29238           +16.6%      34091 ±  6%  softirqs.CPU134.RCU
    209720           -28.9%     149136        softirqs.CPU134.TIMER
     29823 ±  4%     +22.1%      36417 ± 11%  softirqs.CPU135.RCU
    213130 ±  2%     -28.4%     152702        softirqs.CPU135.TIMER
     29568 ±  4%     +24.1%      36682 ± 10%  softirqs.CPU136.RCU
    213209           -28.6%     152225        softirqs.CPU136.TIMER
     29277 ±  6%     +21.8%      35668 ± 11%  softirqs.CPU137.RCU
    211313           -28.2%     151657        softirqs.CPU137.TIMER
     27785 ±  4%     +22.4%      34021 ±  9%  softirqs.CPU138.RCU
    209255           -29.2%     148208        softirqs.CPU138.TIMER
     29243 ±  7%     +22.5%      35809 ±  9%  softirqs.CPU139.RCU
    209904           -29.1%     148916        softirqs.CPU139.TIMER
     29039 ±  4%     +23.5%      35850 ±  9%  softirqs.CPU14.RCU
    210534           -26.3%     155172        softirqs.CPU14.TIMER
     29218 ±  4%     +18.2%      34550 ±  8%  softirqs.CPU140.RCU
    209494           -29.3%     148163        softirqs.CPU140.TIMER
     29462 ±  4%     +21.4%      35753 ± 10%  softirqs.CPU141.RCU
     10211 ± 24%     -46.8%       5437        softirqs.CPU141.SCHED
    214685           -29.3%     151777        softirqs.CPU141.TIMER
     29923 ±  5%     +21.5%      36371 ±  9%  softirqs.CPU142.RCU
    212589           -28.5%     152001        softirqs.CPU142.TIMER
     11765 ± 41%     -54.5%       5356 ±  3%  softirqs.CPU144.SCHED
    215272 ±  2%     -29.0%     152746 ±  3%  softirqs.CPU144.TIMER
    213970 ±  2%     -27.2%     155763 ±  4%  softirqs.CPU145.TIMER
     28812 ±  4%     +28.2%      36936 ±  7%  softirqs.CPU146.RCU
    214283 ±  2%     -27.6%     155116 ±  4%  softirqs.CPU146.TIMER
    213130 ±  2%     -28.4%     152595 ±  3%  softirqs.CPU147.TIMER
     28775 ±  3%     +20.9%      34792 ± 11%  softirqs.CPU148.RCU
    212692 ±  2%     -28.1%     152999 ±  2%  softirqs.CPU148.TIMER
     27922 ±  7%     +22.4%      34182 ±  7%  softirqs.CPU149.RCU
    210971 ±  2%     -28.8%     150272 ±  2%  softirqs.CPU149.TIMER
     30249 ±  2%     +26.6%      38281 ±  7%  softirqs.CPU15.RCU
    214745           -25.6%     159746        softirqs.CPU15.TIMER
     29659 ±  6%     +24.4%      36891 ±  7%  softirqs.CPU150.RCU
    214015 ±  2%     -27.4%     155400 ±  2%  softirqs.CPU150.TIMER
    214757 ±  2%     -27.3%     156125 ±  3%  softirqs.CPU151.TIMER
    213026 ±  2%     -28.1%     153079        softirqs.CPU152.TIMER
    211168 ±  2%     -28.0%     151973        softirqs.CPU154.TIMER
     27614 ±  4%     +28.8%      35579 ±  9%  softirqs.CPU155.RCU
    211854 ±  2%     -28.2%     152200 ±  2%  softirqs.CPU155.TIMER
    213570 ±  4%     -28.5%     152707 ±  2%  softirqs.CPU156.TIMER
     27943 ±  5%     +25.6%      35103 ± 11%  softirqs.CPU157.RCU
    209587 ±  2%     -27.3%     152304        softirqs.CPU157.TIMER
     28668 ±  7%     +18.8%      34067 ±  8%  softirqs.CPU158.RCU
    211389 ±  3%     -27.8%     152553        softirqs.CPU158.TIMER
     29430 ±  6%     +20.7%      35513 ±  7%  softirqs.CPU159.RCU
    215418 ±  2%     -27.1%     157044        softirqs.CPU159.TIMER
     28001           +25.7%      35196 ± 10%  softirqs.CPU16.RCU
    213858 ±  2%     -25.9%     158518        softirqs.CPU16.TIMER
    213063 ±  2%     -27.6%     154173        softirqs.CPU160.TIMER
    211516 ±  2%     -27.0%     154490 ±  2%  softirqs.CPU161.TIMER
     25585 ±  6%     +23.8%      31686 ±  9%  softirqs.CPU162.RCU
    209829           -26.9%     153281 ±  2%  softirqs.CPU162.TIMER
    211130 ±  2%     -28.3%     151472 ±  2%  softirqs.CPU163.TIMER
     24955 ±  6%     +22.3%      30511 ± 10%  softirqs.CPU164.RCU
    209894 ±  2%     -27.5%     152076        softirqs.CPU164.TIMER
    214094 ±  3%     -27.8%     154645 ±  2%  softirqs.CPU165.TIMER
    212640 ±  2%     -27.5%     154136 ±  2%  softirqs.CPU166.TIMER
    213221 ±  2%     -27.3%     154998 ±  2%  softirqs.CPU167.TIMER
    210353 ±  2%     -25.1%     157593 ±  4%  softirqs.CPU168.TIMER
     27406 ±  2%     +18.0%      32349 ±  5%  softirqs.CPU169.RCU
     10616 ± 28%     -46.2%       5713 ± 11%  softirqs.CPU169.SCHED
    211118 ±  2%     -25.9%     156352 ±  4%  softirqs.CPU169.TIMER
     28289 ±  2%     +23.7%      34995 ±  8%  softirqs.CPU17.RCU
    213930 ±  2%     -25.9%     158532        softirqs.CPU17.TIMER
     27171 ±  5%     +19.7%      32522 ±  7%  softirqs.CPU170.RCU
    211321           -25.2%     158100 ±  5%  softirqs.CPU170.TIMER
     26894 ±  4%     +15.6%      31098 ±  8%  softirqs.CPU171.RCU
    208881           -25.1%     156553 ±  4%  softirqs.CPU171.TIMER
     27119 ±  6%     +19.3%      32360 ± 11%  softirqs.CPU172.RCU
    208501 ±  2%     -25.1%     156093 ±  4%  softirqs.CPU172.TIMER
     27023 ±  4%     +13.8%      30753 ±  8%  softirqs.CPU173.RCU
    208591           -25.0%     156399 ±  4%  softirqs.CPU173.TIMER
     26902 ±  4%     +18.8%      31963 ±  7%  softirqs.CPU174.RCU
    209460           -24.9%     157321 ±  3%  softirqs.CPU174.TIMER
    209711           -24.8%     157732 ±  4%  softirqs.CPU175.TIMER
     29123 ±  2%     +17.2%      34136 ±  8%  softirqs.CPU176.RCU
    210342 ±  2%     -25.1%     157474 ±  3%  softirqs.CPU176.TIMER
     27889 ±  5%     +20.6%      33622 ±  7%  softirqs.CPU177.RCU
    208914           -26.0%     154671 ±  3%  softirqs.CPU177.TIMER
     28702 ±  4%     +17.5%      33727 ±  8%  softirqs.CPU178.RCU
    208987           -25.8%     155136 ±  3%  softirqs.CPU178.TIMER
     26399 ±  4%     +23.1%      32505 ±  7%  softirqs.CPU18.RCU
    212164 ±  3%     -26.3%     156318        softirqs.CPU18.TIMER
     26923 ±  2%     +25.9%      33899 ±  9%  softirqs.CPU180.RCU
    206649           -25.8%     153411 ±  3%  softirqs.CPU180.TIMER
     29044 ±  2%     +16.5%      33846 ±  8%  softirqs.CPU182.RCU
    207100           -24.9%     155487 ±  3%  softirqs.CPU182.TIMER
     29943 ±  4%     +19.6%      35825 ±  9%  softirqs.CPU183.RCU
    211346           -23.8%     161066 ±  4%  softirqs.CPU183.TIMER
     30396           +19.7%      36378 ±  8%  softirqs.CPU184.RCU
    211560 ±  2%     -24.3%     160103 ±  5%  softirqs.CPU184.TIMER
     30131 ±  4%     +20.4%      36266 ± 11%  softirqs.CPU185.RCU
    209560           -26.1%     154798 ±  4%  softirqs.CPU185.TIMER
     28558 ±  2%     +18.0%      33688 ±  6%  softirqs.CPU186.RCU
    208010           -25.3%     155330 ±  4%  softirqs.CPU186.TIMER
     28347 ±  4%     +22.3%      34665 ±  7%  softirqs.CPU187.RCU
    207505 ±  2%     -25.1%     155444 ±  4%  softirqs.CPU187.TIMER
     27965           +22.2%      34182 ±  6%  softirqs.CPU188.RCU
    207078           -25.8%     153553 ±  3%  softirqs.CPU188.TIMER
     29523 ±  3%     +24.9%      36885 ± 11%  softirqs.CPU189.RCU
    209950           -24.9%     157714 ±  4%  softirqs.CPU189.TIMER
     26998 ±  3%     +23.3%      33297 ± 11%  softirqs.CPU19.RCU
     11187 ± 35%     -50.7%       5512        softirqs.CPU19.SCHED
    212227           -26.8%     155367        softirqs.CPU19.TIMER
     31200 ±  4%     +18.8%      37055 ± 10%  softirqs.CPU190.RCU
    210981 ±  2%     -25.1%     158089 ±  4%  softirqs.CPU190.TIMER
     29706 ±  5%     +22.5%      36379 ± 10%  softirqs.CPU191.RCU
      9436 ± 11%     -42.7%       5409        softirqs.CPU191.SCHED
    210329 ±  2%     -25.2%     157288 ±  3%  softirqs.CPU191.TIMER
     33787 ±  6%     +15.9%      39161 ±  5%  softirqs.CPU2.RCU
      9510 ±  9%     -40.0%       5708 ±  5%  softirqs.CPU2.SCHED
    219099 ±  2%     -27.5%     158798 ±  2%  softirqs.CPU2.TIMER
     26575 ±  4%     +28.3%      34085 ± 12%  softirqs.CPU20.RCU
    209894 ±  2%     -26.0%     155372        softirqs.CPU20.TIMER
     28068 ±  2%     +23.8%      34762 ± 10%  softirqs.CPU21.RCU
     11463 ± 43%     -51.9%       5516 ±  3%  softirqs.CPU21.SCHED
    215404           -27.0%     157312        softirqs.CPU21.TIMER
     28546           +21.9%      34806 ±  9%  softirqs.CPU22.RCU
    215758           -26.5%     158668        softirqs.CPU22.TIMER
     27843 ±  5%     +22.1%      33985 ±  9%  softirqs.CPU23.RCU
    232587 ± 12%     -32.1%     158004        softirqs.CPU23.TIMER
    212941           -28.1%     153136 ±  4%  softirqs.CPU24.TIMER
     27069 ±  4%     +18.3%      32009 ±  7%  softirqs.CPU25.RCU
    213723           -29.2%     151408        softirqs.CPU25.TIMER
     27059 ±  5%     +20.9%      32720 ±  8%  softirqs.CPU26.RCU
    213330           -28.3%     152899 ±  2%  softirqs.CPU26.TIMER
    215235           -30.1%     150373        softirqs.CPU27.TIMER
     26865 ±  5%     +16.1%      31187 ± 10%  softirqs.CPU28.RCU
    213328           -29.5%     150306        softirqs.CPU28.TIMER
    216040 ±  2%     -30.9%     149343        softirqs.CPU29.TIMER
     29232 ±  8%     +24.4%      36373 ± 11%  softirqs.CPU3.RCU
    214112 ±  2%     -27.3%     155737        softirqs.CPU3.TIMER
    213977           -28.9%     152069 ±  2%  softirqs.CPU30.TIMER
    216119           -29.3%     152717        softirqs.CPU31.TIMER
     29690 ±  4%     +21.1%      35964 ±  9%  softirqs.CPU32.RCU
    213975           -29.5%     150912        softirqs.CPU32.TIMER
     30031 ±  5%     +16.0%      34848 ±  5%  softirqs.CPU33.RCU
    211751           -29.5%     149377        softirqs.CPU33.TIMER
     29447 ±  2%     +17.1%      34479 ±  6%  softirqs.CPU34.RCU
    211731           -29.1%     150214        softirqs.CPU34.TIMER
    212468           -30.3%     148061        softirqs.CPU35.TIMER
     28685 ±  3%     +21.0%      34706 ± 10%  softirqs.CPU36.RCU
    211353           -29.3%     149392        softirqs.CPU36.TIMER
     29378           +17.2%      34426 ±  8%  softirqs.CPU37.RCU
    210511           -29.1%     149216        softirqs.CPU37.TIMER
     29100 ±  3%     +19.3%      34726 ±  7%  softirqs.CPU38.RCU
    211042           -29.0%     149835        softirqs.CPU38.TIMER
     30780 ±  5%     +20.7%      37166 ± 10%  softirqs.CPU39.RCU
    214297           -28.0%     154259        softirqs.CPU39.TIMER
     28050 ± 10%     +27.3%      35705 ±  9%  softirqs.CPU4.RCU
    230797 ± 13%     -32.2%     156453        softirqs.CPU4.TIMER
     31115 ±  6%     +19.6%      37212 ±  7%  softirqs.CPU40.RCU
    214008           -28.9%     152207        softirqs.CPU40.TIMER
     30461 ±  2%     +21.0%      36863 ±  8%  softirqs.CPU41.RCU
    211998           -28.2%     152240        softirqs.CPU41.TIMER
    209853           -29.2%     148539        softirqs.CPU42.TIMER
    211190           -28.9%     150055        softirqs.CPU43.TIMER
     29662 ±  3%     +17.6%      34883 ±  6%  softirqs.CPU44.RCU
    210298           -29.2%     148797        softirqs.CPU44.TIMER
     30223 ±  2%     +21.3%      36674 ±  9%  softirqs.CPU45.RCU
    214317           -28.7%     152818        softirqs.CPU45.TIMER
     31143 ±  4%     +17.7%      36669 ±  8%  softirqs.CPU46.RCU
    213854           -28.7%     152573        softirqs.CPU46.TIMER
     30210 ±  7%     +16.2%      35106 ± 10%  softirqs.CPU47.RCU
    213016           -26.9%     155816 ±  4%  softirqs.CPU47.TIMER
     12279 ± 32%     -50.8%       6040 ±  4%  softirqs.CPU48.SCHED
    215529 ±  2%     -28.8%     153564 ±  2%  softirqs.CPU48.TIMER
    214485 ±  2%     -27.3%     155985 ±  2%  softirqs.CPU49.TIMER
     29001 ±  6%     +25.0%      36243 ± 11%  softirqs.CPU5.RCU
    213150 ±  2%     -26.5%     156591        softirqs.CPU5.TIMER
     30095 ±  6%     +26.6%      38108 ±  5%  softirqs.CPU50.RCU
    213823 ±  2%     -27.2%     155646 ±  3%  softirqs.CPU50.TIMER
     29786 ±  6%     +18.0%      35152 ±  6%  softirqs.CPU51.RCU
    213572 ±  2%     -27.9%     153947 ±  3%  softirqs.CPU51.TIMER
     29179 ±  7%     +22.9%      35870 ±  8%  softirqs.CPU52.RCU
    211849 ±  2%     -27.6%     153474 ±  2%  softirqs.CPU52.TIMER
     28466 ±  7%     +24.0%      35307 ±  7%  softirqs.CPU53.RCU
    211192 ±  2%     -28.1%     151907 ±  3%  softirqs.CPU53.TIMER
     29840 ±  6%     +24.2%      37051 ±  7%  softirqs.CPU54.RCU
      9283 ±  6%     -41.4%       5436 ±  2%  softirqs.CPU54.SCHED
    214709           -27.8%     154928 ±  2%  softirqs.CPU54.TIMER
    213817           -27.6%     154702 ±  2%  softirqs.CPU55.TIMER
    213041 ±  2%     -27.6%     154179        softirqs.CPU56.TIMER
    210655 ±  2%     -26.3%     155289 ±  4%  softirqs.CPU57.TIMER
     28635 ±  6%     +24.3%      35591 ±  8%  softirqs.CPU58.RCU
    211656 ±  2%     -27.7%     152960        softirqs.CPU58.TIMER
    211521 ±  2%     -28.0%     152333 ±  2%  softirqs.CPU59.TIMER
     30055 ±  6%     +25.7%      37781 ±  8%  softirqs.CPU6.RCU
    214308 ±  2%     -25.9%     158750        softirqs.CPU6.TIMER
     26190 ± 15%     +32.4%      34689 ± 10%  softirqs.CPU60.RCU
    230723 ± 17%     -34.0%     152286        softirqs.CPU60.TIMER
     29128 ±  8%     +20.8%      35172 ±  8%  softirqs.CPU61.RCU
    209907 ±  2%     -27.4%     152496        softirqs.CPU61.TIMER
    211588 ±  2%     -27.9%     152582 ±  2%  softirqs.CPU62.TIMER
     29021 ±  6%     +27.0%      36844 ±  8%  softirqs.CPU63.RCU
    215397 ±  2%     -27.4%     156430 ±  2%  softirqs.CPU63.TIMER
    213662 ±  2%     -26.9%     156219        softirqs.CPU64.TIMER
     28114 ±  6%     +21.2%      34068 ±  9%  softirqs.CPU65.RCU
    212010 ±  2%     -27.0%     154766 ±  2%  softirqs.CPU65.TIMER
     25998 ±  4%     +20.0%      31210 ±  8%  softirqs.CPU66.RCU
    210363           -26.8%     154005 ±  2%  softirqs.CPU66.TIMER
    211451 ±  2%     -27.7%     152858 ±  2%  softirqs.CPU67.TIMER
    211237 ±  2%     -27.9%     152242        softirqs.CPU68.TIMER
    215228 ±  3%     -27.8%     155486 ±  2%  softirqs.CPU69.TIMER
     30439 ±  3%     +24.0%      37741 ± 10%  softirqs.CPU7.RCU
    215223           -25.9%     159580 ±  2%  softirqs.CPU7.TIMER
    213071 ±  2%     -27.2%     155209 ±  2%  softirqs.CPU70.TIMER
    213980 ±  2%     -27.5%     155074 ±  2%  softirqs.CPU71.TIMER
     28881 ±  4%     +13.7%      32843 ±  6%  softirqs.CPU72.RCU
    211907           -25.9%     157058 ±  3%  softirqs.CPU72.TIMER
     29058 ±  6%     +12.5%      32698 ±  7%  softirqs.CPU73.RCU
    211007           -24.8%     158685 ±  4%  softirqs.CPU73.TIMER
     28375 ±  3%     +16.5%      33064 ±  8%  softirqs.CPU74.RCU
    211668 ±  2%     -25.6%     157524 ±  4%  softirqs.CPU74.TIMER
     28125 ±  3%     +12.1%      31521 ±  6%  softirqs.CPU75.RCU
    209642           -25.4%     156436 ±  4%  softirqs.CPU75.TIMER
    209402           -25.9%     155214 ±  3%  softirqs.CPU76.TIMER
     27497 ±  3%     +18.5%      32587 ±  8%  softirqs.CPU77.RCU
    209689           -26.0%     155272 ±  3%  softirqs.CPU77.TIMER
     28087 ±  5%     +14.2%      32086 ±  6%  softirqs.CPU78.RCU
    210243 ±  2%     -25.3%     157057 ±  3%  softirqs.CPU78.TIMER
    210500           -25.0%     157922 ±  3%  softirqs.CPU79.TIMER
     30154 ±  3%     +23.3%      37194 ± 10%  softirqs.CPU8.RCU
    213661           -26.5%     157104        softirqs.CPU8.TIMER
     30205 ±  2%     +18.3%      35728 ±  8%  softirqs.CPU80.RCU
    210461 ±  2%     -24.8%     158370 ±  3%  softirqs.CPU80.TIMER
    209770           -25.4%     156440 ±  4%  softirqs.CPU81.TIMER
     29093 ±  4%     +20.5%      35070 ±  9%  softirqs.CPU82.RCU
    210729           -25.7%     156479 ±  3%  softirqs.CPU82.TIMER
    209439           -23.9%     159332 ±  6%  softirqs.CPU83.TIMER
     27645 ±  3%     +23.8%      34212 ±  8%  softirqs.CPU84.RCU
    205990           -25.2%     154104 ±  4%  softirqs.CPU84.TIMER
     28885 ±  3%     +18.3%      34181 ± 10%  softirqs.CPU85.RCU
    208247           -23.1%     160056 ±  6%  softirqs.CPU85.TIMER
    207440           -24.8%     156032 ±  3%  softirqs.CPU86.TIMER
     29964 ±  2%     +22.7%      36780 ± 10%  softirqs.CPU87.RCU
    211431 ±  2%     -24.0%     160677 ±  4%  softirqs.CPU87.TIMER
     30822 ±  2%     +22.4%      37741 ±  8%  softirqs.CPU88.RCU
    212526 ±  2%     -24.3%     160844 ±  5%  softirqs.CPU88.TIMER
     30358 ±  3%     +22.6%      37218 ± 11%  softirqs.CPU89.RCU
    210377 ±  2%     -24.8%     158292 ±  5%  softirqs.CPU89.TIMER
      9451 ± 11%     -40.0%       5670 ±  3%  softirqs.CPU9.SCHED
    210373           -25.9%     155838        softirqs.CPU9.TIMER
     28924 ±  2%     +20.3%      34802 ± 11%  softirqs.CPU90.RCU
    208994           -25.3%     156215 ±  4%  softirqs.CPU90.TIMER
     29719 ±  5%     +20.0%      35662 ± 13%  softirqs.CPU91.RCU
    209303 ±  2%     -25.0%     156893 ±  4%  softirqs.CPU91.TIMER
     29396 ±  4%     +16.1%      34133 ±  8%  softirqs.CPU92.RCU
    208657 ±  2%     -25.5%     155510 ±  4%  softirqs.CPU92.TIMER
     30129 ±  2%     +23.2%      37117 ±  9%  softirqs.CPU93.RCU
    211711           -24.9%     158963 ±  4%  softirqs.CPU93.TIMER
     30322           +25.9%      38166 ±  8%  softirqs.CPU94.RCU
    212952 ±  2%     -25.2%     159367 ±  4%  softirqs.CPU94.TIMER
     29568 ±  3%     +26.7%      37477 ± 12%  softirqs.CPU95.RCU
    212568 ±  2%     -24.9%     159638 ±  3%  softirqs.CPU95.TIMER
    217764 ±  2%     -27.2%     158456 ±  3%  softirqs.CPU96.TIMER
     29691 ±  4%     +25.1%      37132 ± 10%  softirqs.CPU97.RCU
    215860 ±  2%     -26.9%     157889 ±  2%  softirqs.CPU97.TIMER
     29065 ±  3%     +26.0%      36621 ±  9%  softirqs.CPU98.RCU
    215163 ±  2%     -27.4%     156134 ±  2%  softirqs.CPU98.TIMER
     28538 ±  6%     +24.8%      35613 ±  9%  softirqs.CPU99.RCU
    212827 ±  2%     -26.9%     155544        softirqs.CPU99.TIMER
   5511110 ±  3%     +20.2%    6622475 ±  8%  softirqs.RCU
   1729142           -38.4%    1065258        softirqs.SCHED
  40805530           -27.1%   29737670        softirqs.TIMER
    622.00 ± 33%     -46.4%     333.50 ± 38%  interrupts.112:IR-PCI-MSI.1574914-edge.eth1-TxRx-2
    490.75 ± 69%     -55.5%     218.25 ±  5%  interrupts.113:IR-PCI-MSI.1574915-edge.eth1-TxRx-3
    366.25 ± 32%     -34.8%     238.75 ±  8%  interrupts.117:IR-PCI-MSI.1574919-edge.eth1-TxRx-7
    394.75 ± 30%     -41.0%     232.75 ± 12%  interrupts.124:IR-PCI-MSI.1574922-edge.eth1-TxRx-10
    456.50 ± 60%     -49.7%     229.75 ± 11%  interrupts.125:IR-PCI-MSI.1574923-edge.eth1-TxRx-11
    367.75 ± 38%     -40.0%     220.50 ±  7%  interrupts.127:IR-PCI-MSI.1574925-edge.eth1-TxRx-13
    323.75 ± 11%     -35.0%     210.50 ±  2%  interrupts.128:IR-PCI-MSI.1574926-edge.eth1-TxRx-14
    275.00 ±  2%     -25.1%     206.00        interrupts.130:IR-PCI-MSI.1574928-edge.eth1-TxRx-16
    277.75 ±  2%     -26.8%     203.25        interrupts.131:IR-PCI-MSI.1574929-edge.eth1-TxRx-17
    275.50 ±  2%     -25.2%     206.00 ±  2%  interrupts.132:IR-PCI-MSI.1574930-edge.eth1-TxRx-18
    277.50 ±  2%     -26.8%     203.25        interrupts.133:IR-PCI-MSI.1574931-edge.eth1-TxRx-19
    275.25 ±  2%     -26.2%     203.25        interrupts.134:IR-PCI-MSI.1574932-edge.eth1-TxRx-20
    282.25 ±  2%     -26.9%     206.25 ±  2%  interrupts.135:IR-PCI-MSI.1574933-edge.eth1-TxRx-21
    275.00 ±  2%     -26.1%     203.25        interrupts.136:IR-PCI-MSI.1574934-edge.eth1-TxRx-22
    277.00 ±  2%     -22.6%     214.50 ±  8%  interrupts.137:IR-PCI-MSI.1574935-edge.eth1-TxRx-23
    282.75           -27.3%     205.50        interrupts.138:IR-PCI-MSI.1574936-edge.eth1-TxRx-24
    275.50 ±  2%     -26.2%     203.25        interrupts.140:IR-PCI-MSI.1574938-edge.eth1-TxRx-26
    279.25 ±  2%     -27.2%     203.25        interrupts.141:IR-PCI-MSI.1574939-edge.eth1-TxRx-27
    275.00 ±  2%     -25.0%     206.25 ±  2%  interrupts.142:IR-PCI-MSI.1574940-edge.eth1-TxRx-28
    277.25 ±  2%     -26.7%     203.25        interrupts.143:IR-PCI-MSI.1574941-edge.eth1-TxRx-29
    275.50 ±  2%     -26.1%     203.50        interrupts.144:IR-PCI-MSI.1574942-edge.eth1-TxRx-30
    275.00 ±  2%     -26.1%     203.25        interrupts.145:IR-PCI-MSI.1574943-edge.eth1-TxRx-31
    275.75 ±  2%     -26.3%     203.25        interrupts.146:IR-PCI-MSI.1574944-edge.eth1-TxRx-32
    275.75 ±  2%     -26.2%     203.50        interrupts.147:IR-PCI-MSI.1574945-edge.eth1-TxRx-33
    329.50 ± 27%     -38.2%     203.50        interrupts.149:IR-PCI-MSI.1574947-edge.eth1-TxRx-35
    277.50 ±  2%     -26.8%     203.25        interrupts.150:IR-PCI-MSI.1574948-edge.eth1-TxRx-36
    275.25 ±  2%     -26.0%     203.75        interrupts.151:IR-PCI-MSI.1574949-edge.eth1-TxRx-37
    287.50 ±  5%     -29.3%     203.25        interrupts.152:IR-PCI-MSI.1574950-edge.eth1-TxRx-38
    275.00 ±  2%     -26.0%     203.50        interrupts.153:IR-PCI-MSI.1574951-edge.eth1-TxRx-39
    275.00 ±  2%     -26.1%     203.25        interrupts.154:IR-PCI-MSI.1574952-edge.eth1-TxRx-40
    275.00 ±  2%     -26.1%     203.25        interrupts.155:IR-PCI-MSI.1574953-edge.eth1-TxRx-41
    277.50 ±  2%     -26.7%     203.50        interrupts.156:IR-PCI-MSI.1574954-edge.eth1-TxRx-42
    275.00 ±  2%     -25.7%     204.25        interrupts.157:IR-PCI-MSI.1574955-edge.eth1-TxRx-43
    286.50 ±  6%     -28.9%     203.75        interrupts.158:IR-PCI-MSI.1574956-edge.eth1-TxRx-44
    277.50 ±  3%     -26.7%     203.50        interrupts.159:IR-PCI-MSI.1574957-edge.eth1-TxRx-45
    275.00 ±  2%     -26.1%     203.25        interrupts.160:IR-PCI-MSI.1574958-edge.eth1-TxRx-46
    275.00 ±  2%     -26.1%     203.25        interrupts.161:IR-PCI-MSI.1574959-edge.eth1-TxRx-47
    275.75 ±  2%     -25.5%     205.50        interrupts.163:IR-PCI-MSI.1574961-edge.eth1-TxRx-49
    275.00 ±  2%     -25.9%     203.75        interrupts.164:IR-PCI-MSI.1574962-edge.eth1-TxRx-50
    276.50 ±  3%     -26.5%     203.25        interrupts.165:IR-PCI-MSI.1574963-edge.eth1-TxRx-51
    275.00 ±  2%     -26.1%     203.25        interrupts.166:IR-PCI-MSI.1574964-edge.eth1-TxRx-52
    276.00 ±  2%     -26.4%     203.25        interrupts.167:IR-PCI-MSI.1574965-edge.eth1-TxRx-53
    275.50 ±  2%     -25.3%     205.75        interrupts.168:IR-PCI-MSI.1574966-edge.eth1-TxRx-54
    322.25 ± 24%     -36.9%     203.25        interrupts.169:IR-PCI-MSI.1574967-edge.eth1-TxRx-55
    275.50 ±  2%     -26.0%     203.75        interrupts.170:IR-PCI-MSI.1574968-edge.eth1-TxRx-56
    276.00 ±  2%     -26.2%     203.75        interrupts.171:IR-PCI-MSI.1574969-edge.eth1-TxRx-57
    275.00 ±  2%     -26.1%     203.25        interrupts.172:IR-PCI-MSI.1574970-edge.eth1-TxRx-58
    275.25 ±  2%     -26.2%     203.25        interrupts.173:IR-PCI-MSI.1574971-edge.eth1-TxRx-59
    275.00 ±  2%     -26.1%     203.25        interrupts.174:IR-PCI-MSI.1574972-edge.eth1-TxRx-60
    275.50 ±  2%     -26.1%     203.50        interrupts.175:IR-PCI-MSI.1574973-edge.eth1-TxRx-61
    275.25 ±  2%     -26.1%     203.50        interrupts.176:IR-PCI-MSI.1574974-edge.eth1-TxRx-62
   1130237 ± 13%     -25.8%     838335        interrupts.CAL:Function_call_interrupts
      5168 ±  8%     -15.7%       4359        interrupts.CPU0.CAL:Function_call_interrupts
   1076991 ±  4%     -24.1%     817359 ±  2%  interrupts.CPU0.LOC:Local_timer_interrupts
      5745 ± 13%     -24.4%       4341        interrupts.CPU1.CAL:Function_call_interrupts
   1068882 ±  5%     -23.5%     817312 ±  2%  interrupts.CPU1.LOC:Local_timer_interrupts
    394.75 ± 30%     -41.0%     232.75 ± 12%  interrupts.CPU10.124:IR-PCI-MSI.1574922-edge.eth1-TxRx-10
      5748 ± 13%     -24.5%       4337 ±  2%  interrupts.CPU10.CAL:Function_call_interrupts
   1068622 ±  5%     -23.5%     817378 ±  2%  interrupts.CPU10.LOC:Local_timer_interrupts
      6026 ± 11%     -27.9%       4345        interrupts.CPU100.CAL:Function_call_interrupts
   1070995 ±  5%     -23.7%     817171 ±  2%  interrupts.CPU100.LOC:Local_timer_interrupts
      5836 ± 16%     -25.0%       4379        interrupts.CPU101.CAL:Function_call_interrupts
   1068645 ±  5%     -23.5%     817389 ±  2%  interrupts.CPU101.LOC:Local_timer_interrupts
      5952 ± 13%     -28.4%       4261 ±  6%  interrupts.CPU102.CAL:Function_call_interrupts
   1068969 ±  5%     -23.6%     817155 ±  2%  interrupts.CPU102.LOC:Local_timer_interrupts
      5914 ± 13%     -25.7%       4395        interrupts.CPU103.CAL:Function_call_interrupts
   1068474 ±  5%     -23.5%     817266 ±  2%  interrupts.CPU103.LOC:Local_timer_interrupts
      5936 ± 13%     -26.1%       4386        interrupts.CPU104.CAL:Function_call_interrupts
   1068373 ±  5%     -23.5%     817256 ±  2%  interrupts.CPU104.LOC:Local_timer_interrupts
      5923 ± 13%     -26.1%       4377        interrupts.CPU105.CAL:Function_call_interrupts
   1067765 ±  5%     -23.5%     817318 ±  2%  interrupts.CPU105.LOC:Local_timer_interrupts
      5905 ± 13%     -26.0%       4371        interrupts.CPU106.CAL:Function_call_interrupts
   1068345 ±  5%     -23.5%     817168 ±  2%  interrupts.CPU106.LOC:Local_timer_interrupts
      5876 ± 13%     -24.7%       4422 ±  2%  interrupts.CPU107.CAL:Function_call_interrupts
   1068399 ±  5%     -23.5%     817241 ±  2%  interrupts.CPU107.LOC:Local_timer_interrupts
     26.25 ±106%    +361.0%     121.00 ± 77%  interrupts.CPU107.RES:Rescheduling_interrupts
      5870 ± 13%     -24.8%       4415 ±  2%  interrupts.CPU108.CAL:Function_call_interrupts
   1068321 ±  5%     -23.5%     817069 ±  2%  interrupts.CPU108.LOC:Local_timer_interrupts
      5847 ± 13%     -24.6%       4410 ±  2%  interrupts.CPU109.CAL:Function_call_interrupts
   1068358 ±  5%     -23.5%     817283 ±  2%  interrupts.CPU109.LOC:Local_timer_interrupts
     17.50 ± 69%    +718.6%     143.25 ± 48%  interrupts.CPU109.RES:Rescheduling_interrupts
    456.50 ± 60%     -49.7%     229.75 ± 11%  interrupts.CPU11.125:IR-PCI-MSI.1574923-edge.eth1-TxRx-11
      5727 ± 13%     -24.4%       4329 ±  2%  interrupts.CPU11.CAL:Function_call_interrupts
   1068652 ±  5%     -23.5%     817368 ±  2%  interrupts.CPU11.LOC:Local_timer_interrupts
      5832 ± 13%     -24.5%       4405 ±  2%  interrupts.CPU110.CAL:Function_call_interrupts
   1068111 ±  5%     -23.5%     817267 ±  2%  interrupts.CPU110.LOC:Local_timer_interrupts
      5805 ± 13%     -24.7%       4373 ±  4%  interrupts.CPU111.CAL:Function_call_interrupts
   1068078 ±  5%     -23.5%     817185 ±  2%  interrupts.CPU111.LOC:Local_timer_interrupts
     25.75 ± 73%    +435.0%     137.75 ± 12%  interrupts.CPU111.RES:Rescheduling_interrupts
      5690 ± 11%     -23.1%       4378 ±  2%  interrupts.CPU112.CAL:Function_call_interrupts
   1068016 ±  5%     -23.5%     817236 ±  2%  interrupts.CPU112.LOC:Local_timer_interrupts
      5888 ± 12%     -25.0%       4416 ±  2%  interrupts.CPU113.CAL:Function_call_interrupts
   1068115 ±  5%     -23.5%     817407 ±  2%  interrupts.CPU113.LOC:Local_timer_interrupts
      5754 ± 13%     -24.3%       4355 ±  3%  interrupts.CPU114.CAL:Function_call_interrupts
   1068245 ±  5%     -23.5%     817299 ±  2%  interrupts.CPU114.LOC:Local_timer_interrupts
      5746 ± 13%     -24.1%       4362 ±  2%  interrupts.CPU115.CAL:Function_call_interrupts
   1067729 ±  5%     -23.5%     817198 ±  2%  interrupts.CPU115.LOC:Local_timer_interrupts
      5728 ± 13%     -33.6%       3802 ± 24%  interrupts.CPU116.CAL:Function_call_interrupts
   1067531 ±  5%     -23.4%     817294 ±  2%  interrupts.CPU116.LOC:Local_timer_interrupts
   1067792 ±  5%     -23.5%     817380 ±  2%  interrupts.CPU117.LOC:Local_timer_interrupts
      5967 ± 14%     -36.3%       3800 ± 27%  interrupts.CPU118.CAL:Function_call_interrupts
   1068169 ±  5%     -23.5%     817269 ±  2%  interrupts.CPU118.LOC:Local_timer_interrupts
     57.75 ±126%    +802.6%     521.25 ± 82%  interrupts.CPU118.RES:Rescheduling_interrupts
      5948 ± 14%     -25.5%       4431 ±  2%  interrupts.CPU119.CAL:Function_call_interrupts
   1067702 ±  5%     -23.5%     816641 ±  2%  interrupts.CPU119.LOC:Local_timer_interrupts
      5766 ± 13%     -28.2%       4140 ±  5%  interrupts.CPU12.CAL:Function_call_interrupts
   1068501 ±  5%     -23.5%     817472 ±  2%  interrupts.CPU12.LOC:Local_timer_interrupts
      5940 ± 14%     -25.6%       4419        interrupts.CPU120.CAL:Function_call_interrupts
   1064878 ±  6%     -23.4%     815435 ±  3%  interrupts.CPU120.LOC:Local_timer_interrupts
      5858 ± 14%     -24.6%       4414        interrupts.CPU121.CAL:Function_call_interrupts
   1064719 ±  5%     -23.4%     815464 ±  3%  interrupts.CPU121.LOC:Local_timer_interrupts
      5887 ± 14%     -24.8%       4429        interrupts.CPU122.CAL:Function_call_interrupts
   1064872 ±  6%     -23.4%     815208 ±  3%  interrupts.CPU122.LOC:Local_timer_interrupts
      5980 ± 12%     -25.9%       4428        interrupts.CPU123.CAL:Function_call_interrupts
   1065006 ±  5%     -23.4%     815282 ±  3%  interrupts.CPU123.LOC:Local_timer_interrupts
      5829 ± 14%     -24.2%       4417        interrupts.CPU124.CAL:Function_call_interrupts
   1064512 ±  6%     -23.4%     815486 ±  3%  interrupts.CPU124.LOC:Local_timer_interrupts
      5932 ± 12%     -25.1%       4443        interrupts.CPU125.CAL:Function_call_interrupts
   1064549 ±  6%     -23.4%     815278 ±  3%  interrupts.CPU125.LOC:Local_timer_interrupts
      5954 ± 12%     -24.9%       4472 ±  2%  interrupts.CPU126.CAL:Function_call_interrupts
   1064735 ±  6%     -23.4%     815395 ±  3%  interrupts.CPU126.LOC:Local_timer_interrupts
      5863 ± 12%     -23.9%       4461 ±  2%  interrupts.CPU127.CAL:Function_call_interrupts
   1064451 ±  6%     -23.4%     815262 ±  3%  interrupts.CPU127.LOC:Local_timer_interrupts
      5969 ± 13%     -25.5%       4448 ±  2%  interrupts.CPU128.CAL:Function_call_interrupts
   1064891 ±  6%     -23.4%     815290 ±  3%  interrupts.CPU128.LOC:Local_timer_interrupts
      6043 ± 12%     -26.6%       4438 ±  2%  interrupts.CPU129.CAL:Function_call_interrupts
   1064774 ±  6%     -23.4%     815476 ±  3%  interrupts.CPU129.LOC:Local_timer_interrupts
    367.75 ± 38%     -40.0%     220.50 ±  7%  interrupts.CPU13.127:IR-PCI-MSI.1574925-edge.eth1-TxRx-13
      5717 ± 13%     -24.5%       4319 ±  2%  interrupts.CPU13.CAL:Function_call_interrupts
   1068376 ±  5%     -23.5%     817463 ±  2%  interrupts.CPU13.LOC:Local_timer_interrupts
      6034 ± 12%     -27.5%       4373        interrupts.CPU130.CAL:Function_call_interrupts
   1064663 ±  6%     -23.3%     816165 ±  3%  interrupts.CPU130.LOC:Local_timer_interrupts
      6006 ± 12%     -25.9%       4452 ±  2%  interrupts.CPU131.CAL:Function_call_interrupts
   1064736 ±  6%     -23.4%     815494 ±  3%  interrupts.CPU131.LOC:Local_timer_interrupts
      5985 ± 12%     -25.3%       4468        interrupts.CPU132.CAL:Function_call_interrupts
   1064863 ±  6%     -23.4%     815416 ±  3%  interrupts.CPU132.LOC:Local_timer_interrupts
      5966 ± 12%     -25.3%       4458        interrupts.CPU133.CAL:Function_call_interrupts
   1065070 ±  5%     -23.5%     815295 ±  3%  interrupts.CPU133.LOC:Local_timer_interrupts
      5949 ± 12%     -25.0%       4463 ±  2%  interrupts.CPU134.CAL:Function_call_interrupts
   1064865 ±  6%     -23.4%     815378 ±  3%  interrupts.CPU134.LOC:Local_timer_interrupts
      5941 ± 12%     -24.9%       4465 ±  2%  interrupts.CPU135.CAL:Function_call_interrupts
   1065166 ±  5%     -23.5%     815337 ±  3%  interrupts.CPU135.LOC:Local_timer_interrupts
      5928 ± 13%     -24.5%       4473        interrupts.CPU136.CAL:Function_call_interrupts
   1064619 ±  6%     -23.4%     815310 ±  3%  interrupts.CPU136.LOC:Local_timer_interrupts
      5910 ± 13%     -23.3%       4532 ±  2%  interrupts.CPU137.CAL:Function_call_interrupts
   1064777 ±  6%     -23.4%     815316 ±  3%  interrupts.CPU137.LOC:Local_timer_interrupts
      5898 ± 13%     -23.2%       4527 ±  2%  interrupts.CPU138.CAL:Function_call_interrupts
   1064691 ±  6%     -23.4%     815430 ±  3%  interrupts.CPU138.LOC:Local_timer_interrupts
      5932 ± 14%     -23.5%       4536 ±  2%  interrupts.CPU139.CAL:Function_call_interrupts
   1064513 ±  6%     -23.4%     815356 ±  3%  interrupts.CPU139.LOC:Local_timer_interrupts
    323.75 ± 11%     -35.0%     210.50 ±  2%  interrupts.CPU14.128:IR-PCI-MSI.1574926-edge.eth1-TxRx-14
      5741 ± 13%     -24.0%       4364 ±  3%  interrupts.CPU14.CAL:Function_call_interrupts
   1068480 ±  5%     -23.5%     817445 ±  2%  interrupts.CPU14.LOC:Local_timer_interrupts
      5855 ± 13%     -22.9%       4515 ±  2%  interrupts.CPU140.CAL:Function_call_interrupts
   1064653 ±  6%     -23.4%     815425 ±  3%  interrupts.CPU140.LOC:Local_timer_interrupts
      5759 ± 14%     -21.7%       4508 ±  2%  interrupts.CPU141.CAL:Function_call_interrupts
   1064571 ±  6%     -23.4%     815503 ±  3%  interrupts.CPU141.LOC:Local_timer_interrupts
      5998 ± 15%     -25.2%       4486 ±  2%  interrupts.CPU142.CAL:Function_call_interrupts
   1064679 ±  6%     -23.4%     815652 ±  3%  interrupts.CPU142.LOC:Local_timer_interrupts
      6042 ± 14%     -25.5%       4499 ±  2%  interrupts.CPU143.CAL:Function_call_interrupts
   1064141 ±  5%     -23.4%     815097 ±  3%  interrupts.CPU143.LOC:Local_timer_interrupts
   1065178 ±  6%     -23.5%     815097 ±  3%  interrupts.CPU144.LOC:Local_timer_interrupts
      6094 ± 14%     -26.3%       4493        interrupts.CPU145.CAL:Function_call_interrupts
   1063968 ±  6%     -23.4%     814942 ±  3%  interrupts.CPU145.LOC:Local_timer_interrupts
      6150 ± 16%     -27.2%       4478        interrupts.CPU146.CAL:Function_call_interrupts
   1063995 ±  6%     -23.4%     815284 ±  3%  interrupts.CPU146.LOC:Local_timer_interrupts
      6077 ± 16%     -26.5%       4463        interrupts.CPU147.CAL:Function_call_interrupts
   1063615 ±  6%     -23.4%     815216 ±  3%  interrupts.CPU147.LOC:Local_timer_interrupts
      5995 ± 17%     -25.8%       4449        interrupts.CPU148.CAL:Function_call_interrupts
   1064104 ±  6%     -23.4%     815345 ±  3%  interrupts.CPU148.LOC:Local_timer_interrupts
      6104 ± 16%     -27.4%       4434        interrupts.CPU149.CAL:Function_call_interrupts
   1064084 ±  6%     -23.4%     815304 ±  3%  interrupts.CPU149.LOC:Local_timer_interrupts
      5698 ± 13%     -28.8%       4055 ±  5%  interrupts.CPU15.CAL:Function_call_interrupts
   1068242 ±  5%     -23.5%     817354 ±  2%  interrupts.CPU15.LOC:Local_timer_interrupts
    105.25 ± 68%    +150.6%     263.75 ± 28%  interrupts.CPU15.RES:Rescheduling_interrupts
      6081 ± 16%     -27.2%       4426        interrupts.CPU150.CAL:Function_call_interrupts
   1063905 ±  6%     -23.4%     815031 ±  3%  interrupts.CPU150.LOC:Local_timer_interrupts
      6051 ± 16%     -27.1%       4410        interrupts.CPU151.CAL:Function_call_interrupts
   1064069 ±  6%     -23.4%     815211 ±  3%  interrupts.CPU151.LOC:Local_timer_interrupts
      5970 ± 17%     -26.4%       4393        interrupts.CPU152.CAL:Function_call_interrupts
   1064026 ±  6%     -23.4%     815122 ±  3%  interrupts.CPU152.LOC:Local_timer_interrupts
      6010 ± 16%     -26.4%       4422        interrupts.CPU153.CAL:Function_call_interrupts
   1064010 ±  6%     -23.4%     814655 ±  3%  interrupts.CPU153.LOC:Local_timer_interrupts
      5988 ± 15%     -26.5%       4403        interrupts.CPU154.CAL:Function_call_interrupts
   1064073 ±  6%     -23.4%     814984 ±  3%  interrupts.CPU154.LOC:Local_timer_interrupts
      5971 ± 15%     -26.1%       4415 ±  2%  interrupts.CPU155.CAL:Function_call_interrupts
   1063921 ±  6%     -23.4%     815049 ±  3%  interrupts.CPU155.LOC:Local_timer_interrupts
      5826 ± 12%     -24.3%       4410 ±  2%  interrupts.CPU156.CAL:Function_call_interrupts
   1063807 ±  6%     -23.4%     815023 ±  3%  interrupts.CPU156.LOC:Local_timer_interrupts
      6045 ± 14%     -27.7%       4372        interrupts.CPU157.CAL:Function_call_interrupts
   1064014 ±  6%     -23.4%     815142 ±  3%  interrupts.CPU157.LOC:Local_timer_interrupts
      6074 ± 13%     -27.2%       4419        interrupts.CPU158.CAL:Function_call_interrupts
   1064052 ±  6%     -23.4%     815046 ±  3%  interrupts.CPU158.LOC:Local_timer_interrupts
      6058 ± 13%     -27.2%       4407        interrupts.CPU159.CAL:Function_call_interrupts
   1064269 ±  6%     -23.4%     815019 ±  3%  interrupts.CPU159.LOC:Local_timer_interrupts
     34.75 ± 39%    +136.0%      82.00 ± 28%  interrupts.CPU159.RES:Rescheduling_interrupts
    275.00 ±  2%     -25.1%     206.00        interrupts.CPU16.130:IR-PCI-MSI.1574928-edge.eth1-TxRx-16
      5811 ± 12%     -26.1%       4293 ±  2%  interrupts.CPU16.CAL:Function_call_interrupts
   1068216 ±  5%     -23.5%     817684 ±  2%  interrupts.CPU16.LOC:Local_timer_interrupts
      6048 ± 13%     -27.1%       4406        interrupts.CPU160.CAL:Function_call_interrupts
   1064260 ±  6%     -23.4%     814992 ±  3%  interrupts.CPU160.LOC:Local_timer_interrupts
      6097 ± 12%     -27.1%       4446 ±  2%  interrupts.CPU161.CAL:Function_call_interrupts
   1063884 ±  6%     -23.4%     815123 ±  3%  interrupts.CPU161.LOC:Local_timer_interrupts
      6088 ± 12%     -26.5%       4475 ±  2%  interrupts.CPU162.CAL:Function_call_interrupts
   1064094 ±  6%     -23.4%     815087 ±  3%  interrupts.CPU162.LOC:Local_timer_interrupts
      6095 ± 12%     -26.8%       4462 ±  2%  interrupts.CPU163.CAL:Function_call_interrupts
   1063849 ±  6%     -23.4%     814953 ±  3%  interrupts.CPU163.LOC:Local_timer_interrupts
      6066 ± 12%     -26.5%       4460 ±  2%  interrupts.CPU164.CAL:Function_call_interrupts
   1064007 ±  6%     -23.4%     815019 ±  3%  interrupts.CPU164.LOC:Local_timer_interrupts
      6051 ± 12%     -26.4%       4453 ±  2%  interrupts.CPU165.CAL:Function_call_interrupts
   1064298 ±  6%     -23.4%     815087 ±  3%  interrupts.CPU165.LOC:Local_timer_interrupts
      6021 ± 12%     -26.3%       4435 ±  2%  interrupts.CPU166.CAL:Function_call_interrupts
   1063918 ±  6%     -23.4%     815168 ±  3%  interrupts.CPU166.LOC:Local_timer_interrupts
      6040 ± 11%     -26.6%       4431        interrupts.CPU167.CAL:Function_call_interrupts
   1063652 ±  6%     -23.4%     815042 ±  3%  interrupts.CPU167.LOC:Local_timer_interrupts
      5976 ± 10%     -26.9%       4367        interrupts.CPU168.CAL:Function_call_interrupts
   1065599 ±  5%     -23.5%     815552 ±  3%  interrupts.CPU168.LOC:Local_timer_interrupts
     67.75 ± 42%     -60.9%      26.50 ± 32%  interrupts.CPU168.RES:Rescheduling_interrupts
      5943 ± 10%     -26.6%       4361        interrupts.CPU169.CAL:Function_call_interrupts
   1071958 ±  4%     -23.9%     815242 ±  3%  interrupts.CPU169.LOC:Local_timer_interrupts
    277.75 ±  2%     -26.8%     203.25        interrupts.CPU17.131:IR-PCI-MSI.1574929-edge.eth1-TxRx-17
      5799 ± 11%     -25.3%       4329        interrupts.CPU17.CAL:Function_call_interrupts
   1068555 ±  5%     -23.5%     817348 ±  2%  interrupts.CPU17.LOC:Local_timer_interrupts
      6011 ± 11%     -26.3%       4433 ±  3%  interrupts.CPU170.CAL:Function_call_interrupts
   1065838 ±  5%     -23.5%     815377 ±  3%  interrupts.CPU170.LOC:Local_timer_interrupts
      6009 ± 11%     -28.9%       4272        interrupts.CPU171.CAL:Function_call_interrupts
   1065808 ±  5%     -23.5%     815373 ±  3%  interrupts.CPU171.LOC:Local_timer_interrupts
      6023 ± 10%     -26.9%       4400        interrupts.CPU172.CAL:Function_call_interrupts
   1065763 ±  5%     -23.5%     815383 ±  3%  interrupts.CPU172.LOC:Local_timer_interrupts
      6051 ± 10%     -27.6%       4384        interrupts.CPU173.CAL:Function_call_interrupts
   1065502 ±  5%     -23.5%     815500 ±  3%  interrupts.CPU173.LOC:Local_timer_interrupts
      6035 ± 10%     -29.6%       4247 ±  3%  interrupts.CPU174.CAL:Function_call_interrupts
   1065804 ±  5%     -23.5%     815599 ±  3%  interrupts.CPU174.LOC:Local_timer_interrupts
    166.75 ± 93%     -85.8%      23.75 ± 42%  interrupts.CPU174.RES:Rescheduling_interrupts
      6003 ± 10%     -27.2%       4371        interrupts.CPU175.CAL:Function_call_interrupts
   1065757 ±  5%     -23.5%     815380 ±  3%  interrupts.CPU175.LOC:Local_timer_interrupts
      5975 ± 10%     -26.9%       4366        interrupts.CPU176.CAL:Function_call_interrupts
   1065807 ±  5%     -23.5%     815586 ±  3%  interrupts.CPU176.LOC:Local_timer_interrupts
      5950 ± 10%     -25.9%       4407 ±  2%  interrupts.CPU177.CAL:Function_call_interrupts
   1065642 ±  5%     -23.5%     815627 ±  3%  interrupts.CPU177.LOC:Local_timer_interrupts
      5947 ± 13%     -26.0%       4399 ±  2%  interrupts.CPU178.CAL:Function_call_interrupts
   1065715 ±  5%     -23.5%     815417 ±  3%  interrupts.CPU178.LOC:Local_timer_interrupts
      5975 ± 13%     -26.3%       4404 ±  2%  interrupts.CPU179.CAL:Function_call_interrupts
   1065717 ±  5%     -23.5%     815743 ±  3%  interrupts.CPU179.LOC:Local_timer_interrupts
    275.50 ±  2%     -25.2%     206.00 ±  2%  interrupts.CPU18.132:IR-PCI-MSI.1574930-edge.eth1-TxRx-18
      5788 ± 12%     -25.0%       4339        interrupts.CPU18.CAL:Function_call_interrupts
   1068246 ±  5%     -23.5%     817433 ±  2%  interrupts.CPU18.LOC:Local_timer_interrupts
      5949 ± 13%     -26.2%       4391 ±  2%  interrupts.CPU180.CAL:Function_call_interrupts
   1065442 ±  5%     -23.4%     815794 ±  3%  interrupts.CPU180.LOC:Local_timer_interrupts
      5979 ± 13%     -27.3%       4344 ±  3%  interrupts.CPU181.CAL:Function_call_interrupts
   1065736 ±  5%     -23.5%     815300 ±  3%  interrupts.CPU181.LOC:Local_timer_interrupts
      6071 ± 11%     -29.1%       4306        interrupts.CPU182.CAL:Function_call_interrupts
   1065585 ±  5%     -23.4%     815820 ±  3%  interrupts.CPU182.LOC:Local_timer_interrupts
      6225 ± 14%     -29.4%       4393 ±  2%  interrupts.CPU183.CAL:Function_call_interrupts
   1065689 ±  5%     -23.5%     815521 ±  3%  interrupts.CPU183.LOC:Local_timer_interrupts
      6149 ±  9%     -29.9%       4311 ±  4%  interrupts.CPU184.CAL:Function_call_interrupts
   1065475 ±  5%     -23.4%     815824 ±  3%  interrupts.CPU184.LOC:Local_timer_interrupts
      6308 ± 13%     -30.2%       4405 ±  2%  interrupts.CPU185.CAL:Function_call_interrupts
   1065480 ±  5%     -23.4%     815849 ±  3%  interrupts.CPU185.LOC:Local_timer_interrupts
      6253 ± 13%     -29.6%       4402        interrupts.CPU186.CAL:Function_call_interrupts
   1065724 ±  5%     -23.5%     815776 ±  3%  interrupts.CPU186.LOC:Local_timer_interrupts
      6276 ± 12%     -30.1%       4385        interrupts.CPU187.CAL:Function_call_interrupts
   1065407 ±  5%     -23.5%     815379 ±  3%  interrupts.CPU187.LOC:Local_timer_interrupts
      6265 ± 13%     -30.2%       4371        interrupts.CPU188.CAL:Function_call_interrupts
   1065783 ±  5%     -23.5%     815586 ±  3%  interrupts.CPU188.LOC:Local_timer_interrupts
      6256 ± 12%     -29.2%       4426 ±  2%  interrupts.CPU189.CAL:Function_call_interrupts
   1065486 ±  5%     -23.4%     815869 ±  3%  interrupts.CPU189.LOC:Local_timer_interrupts
     51.50 ± 40%    +248.5%     179.50 ± 41%  interrupts.CPU189.RES:Rescheduling_interrupts
    277.50 ±  2%     -26.8%     203.25        interrupts.CPU19.133:IR-PCI-MSI.1574931-edge.eth1-TxRx-19
      5780 ± 12%     -27.0%       4217 ±  3%  interrupts.CPU19.CAL:Function_call_interrupts
   1067443 ±  5%     -23.4%     817248 ±  2%  interrupts.CPU19.LOC:Local_timer_interrupts
      6260 ± 12%     -30.6%       4343        interrupts.CPU190.CAL:Function_call_interrupts
   1065962 ±  5%     -23.5%     815771 ±  3%  interrupts.CPU190.LOC:Local_timer_interrupts
      5678 ±  2%     -23.2%       4362        interrupts.CPU191.CAL:Function_call_interrupts
   1067845 ±  5%     -23.6%     815805 ±  3%  interrupts.CPU191.LOC:Local_timer_interrupts
    622.00 ± 33%     -46.4%     333.50 ± 38%  interrupts.CPU2.112:IR-PCI-MSI.1574914-edge.eth1-TxRx-2
      5819 ± 14%     -25.0%       4362        interrupts.CPU2.CAL:Function_call_interrupts
   1068877 ±  5%     -23.5%     817462 ±  2%  interrupts.CPU2.LOC:Local_timer_interrupts
    275.25 ±  2%     -26.2%     203.25        interrupts.CPU20.134:IR-PCI-MSI.1574932-edge.eth1-TxRx-20
      5767 ± 12%     -25.4%       4304        interrupts.CPU20.CAL:Function_call_interrupts
   1067225 ±  5%     -23.4%     817645 ±  2%  interrupts.CPU20.LOC:Local_timer_interrupts
    282.25 ±  2%     -26.9%     206.25 ±  2%  interrupts.CPU21.135:IR-PCI-MSI.1574933-edge.eth1-TxRx-21
   1067560 ±  5%     -23.4%     817431 ±  2%  interrupts.CPU21.LOC:Local_timer_interrupts
    275.00 ±  2%     -26.1%     203.25        interrupts.CPU22.136:IR-PCI-MSI.1574934-edge.eth1-TxRx-22
      5735 ± 12%     -25.7%       4260 ±  2%  interrupts.CPU22.CAL:Function_call_interrupts
   1068543 ±  5%     -23.4%     818022 ±  2%  interrupts.CPU22.LOC:Local_timer_interrupts
    277.00 ±  2%     -22.6%     214.50 ±  8%  interrupts.CPU23.137:IR-PCI-MSI.1574935-edge.eth1-TxRx-23
      5761 ± 12%     -25.8%       4274 ±  2%  interrupts.CPU23.CAL:Function_call_interrupts
   1068202 ±  5%     -23.4%     818386 ±  2%  interrupts.CPU23.LOC:Local_timer_interrupts
    282.75           -27.3%     205.50        interrupts.CPU24.138:IR-PCI-MSI.1574936-edge.eth1-TxRx-24
      5578 ± 14%     -24.3%       4225 ±  5%  interrupts.CPU24.CAL:Function_call_interrupts
   1065105 ±  6%     -23.3%     817356 ±  2%  interrupts.CPU24.LOC:Local_timer_interrupts
      5702 ± 12%     -25.1%       4271 ±  3%  interrupts.CPU25.CAL:Function_call_interrupts
   1064814 ±  6%     -23.4%     815355 ±  3%  interrupts.CPU25.LOC:Local_timer_interrupts
    275.50 ±  2%     -26.2%     203.25        interrupts.CPU26.140:IR-PCI-MSI.1574938-edge.eth1-TxRx-26
      5764 ± 14%     -25.2%       4312        interrupts.CPU26.CAL:Function_call_interrupts
   1064800 ±  6%     -23.4%     815374 ±  3%  interrupts.CPU26.LOC:Local_timer_interrupts
    279.25 ±  2%     -27.2%     203.25        interrupts.CPU27.141:IR-PCI-MSI.1574939-edge.eth1-TxRx-27
      5746 ± 14%     -24.7%       4329 ±  2%  interrupts.CPU27.CAL:Function_call_interrupts
   1064852 ±  5%     -23.4%     815240 ±  3%  interrupts.CPU27.LOC:Local_timer_interrupts
    275.00 ±  2%     -25.0%     206.25 ±  2%  interrupts.CPU28.142:IR-PCI-MSI.1574940-edge.eth1-TxRx-28
      5726 ± 15%     -24.6%       4320 ±  2%  interrupts.CPU28.CAL:Function_call_interrupts
   1064388 ±  6%     -23.4%     815276 ±  3%  interrupts.CPU28.LOC:Local_timer_interrupts
    277.25 ±  2%     -26.7%     203.25        interrupts.CPU29.143:IR-PCI-MSI.1574941-edge.eth1-TxRx-29
      5790 ± 14%     -25.6%       4308 ±  2%  interrupts.CPU29.CAL:Function_call_interrupts
   1064453 ±  5%     -23.4%     815262 ±  3%  interrupts.CPU29.LOC:Local_timer_interrupts
    490.75 ± 69%     -55.5%     218.25 ±  5%  interrupts.CPU3.113:IR-PCI-MSI.1574915-edge.eth1-TxRx-3
      5882 ± 13%     -25.9%       4360        interrupts.CPU3.CAL:Function_call_interrupts
   1068550 ±  5%     -23.6%     816515 ±  2%  interrupts.CPU3.LOC:Local_timer_interrupts
    275.50 ±  2%     -26.1%     203.50        interrupts.CPU30.144:IR-PCI-MSI.1574942-edge.eth1-TxRx-30
      5826 ± 13%     -26.1%       4303 ±  2%  interrupts.CPU30.CAL:Function_call_interrupts
   1064980 ±  6%     -23.4%     815263 ±  3%  interrupts.CPU30.LOC:Local_timer_interrupts
    275.00 ±  2%     -26.1%     203.25        interrupts.CPU31.145:IR-PCI-MSI.1574943-edge.eth1-TxRx-31
      5804 ± 13%     -25.7%       4313 ±  2%  interrupts.CPU31.CAL:Function_call_interrupts
   1064921 ±  6%     -23.4%     815292 ±  3%  interrupts.CPU31.LOC:Local_timer_interrupts
    275.75 ±  2%     -26.3%     203.25        interrupts.CPU32.146:IR-PCI-MSI.1574944-edge.eth1-TxRx-32
      5948 ± 12%     -28.0%       4285 ±  2%  interrupts.CPU32.CAL:Function_call_interrupts
   1064930 ±  6%     -23.4%     815278 ±  3%  interrupts.CPU32.LOC:Local_timer_interrupts
    275.75 ±  2%     -26.2%     203.50        interrupts.CPU33.147:IR-PCI-MSI.1574945-edge.eth1-TxRx-33
      5887 ± 14%     -27.2%       4286        interrupts.CPU33.CAL:Function_call_interrupts
   1065309 ±  6%     -23.5%     815316 ±  3%  interrupts.CPU33.LOC:Local_timer_interrupts
      6023 ± 12%     -37.8%       3745 ± 27%  interrupts.CPU34.CAL:Function_call_interrupts
   1065254 ±  6%     -23.4%     816073 ±  3%  interrupts.CPU34.LOC:Local_timer_interrupts
    329.50 ± 27%     -38.2%     203.50        interrupts.CPU35.149:IR-PCI-MSI.1574947-edge.eth1-TxRx-35
      5974 ± 13%     -28.2%       4291        interrupts.CPU35.CAL:Function_call_interrupts
   1065082 ±  5%     -23.5%     815313 ±  3%  interrupts.CPU35.LOC:Local_timer_interrupts
    277.50 ±  2%     -26.8%     203.25        interrupts.CPU36.150:IR-PCI-MSI.1574948-edge.eth1-TxRx-36
      5939 ± 13%     -27.9%       4284        interrupts.CPU36.CAL:Function_call_interrupts
   1064962 ±  6%     -23.4%     815369 ±  3%  interrupts.CPU36.LOC:Local_timer_interrupts
    275.25 ±  2%     -26.0%     203.75        interrupts.CPU37.151:IR-PCI-MSI.1574949-edge.eth1-TxRx-37
      5961 ± 14%     -28.0%       4291        interrupts.CPU37.CAL:Function_call_interrupts
   1064785 ±  5%     -23.4%     815349 ±  3%  interrupts.CPU37.LOC:Local_timer_interrupts
     48.75 ± 62%    +392.3%     240.00 ±107%  interrupts.CPU37.RES:Rescheduling_interrupts
    287.50 ±  5%     -29.3%     203.25        interrupts.CPU38.152:IR-PCI-MSI.1574950-edge.eth1-TxRx-38
      5951 ± 14%     -28.1%       4281        interrupts.CPU38.CAL:Function_call_interrupts
   1065166 ±  6%     -23.5%     815295 ±  3%  interrupts.CPU38.LOC:Local_timer_interrupts
    275.00 ±  2%     -26.0%     203.50        interrupts.CPU39.153:IR-PCI-MSI.1574951-edge.eth1-TxRx-39
      5921 ± 14%     -26.4%       4359        interrupts.CPU39.CAL:Function_call_interrupts
   1064741 ±  6%     -23.4%     815368 ±  3%  interrupts.CPU39.LOC:Local_timer_interrupts
      5864 ± 13%     -25.8%       4352        interrupts.CPU4.CAL:Function_call_interrupts
   1068694 ±  5%     -23.5%     817464 ±  2%  interrupts.CPU4.LOC:Local_timer_interrupts
    275.00 ±  2%     -26.1%     203.25        interrupts.CPU40.154:IR-PCI-MSI.1574952-edge.eth1-TxRx-40
      5925 ± 13%     -26.1%       4379        interrupts.CPU40.CAL:Function_call_interrupts
   1064766 ±  6%     -23.4%     815403 ±  3%  interrupts.CPU40.LOC:Local_timer_interrupts
    275.00 ±  2%     -26.1%     203.25        interrupts.CPU41.155:IR-PCI-MSI.1574953-edge.eth1-TxRx-41
      5898 ± 14%     -25.4%       4402 ±  2%  interrupts.CPU41.CAL:Function_call_interrupts
   1064611 ±  6%     -23.4%     815405 ±  3%  interrupts.CPU41.LOC:Local_timer_interrupts
    277.50 ±  2%     -26.7%     203.50        interrupts.CPU42.156:IR-PCI-MSI.1574954-edge.eth1-TxRx-42
      5900 ± 12%     -25.5%       4398 ±  2%  interrupts.CPU42.CAL:Function_call_interrupts
   1064596 ±  6%     -23.4%     815474 ±  3%  interrupts.CPU42.LOC:Local_timer_interrupts
    275.00 ±  2%     -25.7%     204.25        interrupts.CPU43.157:IR-PCI-MSI.1574955-edge.eth1-TxRx-43
      5882 ± 12%     -25.2%       4402 ±  2%  interrupts.CPU43.CAL:Function_call_interrupts
   1064840 ±  6%     -23.4%     815462 ±  3%  interrupts.CPU43.LOC:Local_timer_interrupts
    316.50 ± 97%     -82.7%      54.75 ± 72%  interrupts.CPU43.RES:Rescheduling_interrupts
    286.50 ±  6%     -28.9%     203.75        interrupts.CPU44.158:IR-PCI-MSI.1574956-edge.eth1-TxRx-44
      5854 ± 12%     -24.4%       4423 ±  2%  interrupts.CPU44.CAL:Function_call_interrupts
   1064759 ±  6%     -23.4%     815360 ±  3%  interrupts.CPU44.LOC:Local_timer_interrupts
    277.50 ±  3%     -26.7%     203.50        interrupts.CPU45.159:IR-PCI-MSI.1574957-edge.eth1-TxRx-45
      5938 ± 15%     -25.6%       4416 ±  2%  interrupts.CPU45.CAL:Function_call_interrupts
   1064747 ±  6%     -23.4%     815284 ±  3%  interrupts.CPU45.LOC:Local_timer_interrupts
    275.00 ±  2%     -26.1%     203.25        interrupts.CPU46.160:IR-PCI-MSI.1574958-edge.eth1-TxRx-46
      5927 ± 15%     -25.7%       4406 ±  2%  interrupts.CPU46.CAL:Function_call_interrupts
   1064796 ±  6%     -23.4%     815290 ±  3%  interrupts.CPU46.LOC:Local_timer_interrupts
    275.00 ±  2%     -26.1%     203.25        interrupts.CPU47.161:IR-PCI-MSI.1574959-edge.eth1-TxRx-47
      5798 ± 17%     -23.6%       4428        interrupts.CPU47.CAL:Function_call_interrupts
   1064870 ±  6%     -23.5%     815152 ±  3%  interrupts.CPU47.LOC:Local_timer_interrupts
   1064667 ±  6%     -23.4%     815225 ±  3%  interrupts.CPU48.LOC:Local_timer_interrupts
    275.75 ±  2%     -25.5%     205.50        interrupts.CPU49.163:IR-PCI-MSI.1574961-edge.eth1-TxRx-49
      5868 ± 15%     -24.5%       4429        interrupts.CPU49.CAL:Function_call_interrupts
   1063758 ±  6%     -23.4%     815150 ±  3%  interrupts.CPU49.LOC:Local_timer_interrupts
      5897 ± 13%     -26.6%       4327        interrupts.CPU5.CAL:Function_call_interrupts
   1068401 ±  5%     -23.5%     817222 ±  2%  interrupts.CPU5.LOC:Local_timer_interrupts
    275.00 ±  2%     -25.9%     203.75        interrupts.CPU50.164:IR-PCI-MSI.1574962-edge.eth1-TxRx-50
      5833 ± 15%     -24.3%       4417        interrupts.CPU50.CAL:Function_call_interrupts
   1064005 ±  6%     -23.4%     815078 ±  3%  interrupts.CPU50.LOC:Local_timer_interrupts
    276.50 ±  3%     -26.5%     203.25        interrupts.CPU51.165:IR-PCI-MSI.1574963-edge.eth1-TxRx-51
      5816 ± 15%     -24.3%       4404        interrupts.CPU51.CAL:Function_call_interrupts
   1063686 ±  6%     -23.4%     815241 ±  3%  interrupts.CPU51.LOC:Local_timer_interrupts
    275.00 ±  2%     -26.1%     203.25        interrupts.CPU52.166:IR-PCI-MSI.1574964-edge.eth1-TxRx-52
      5809 ± 15%     -23.9%       4420        interrupts.CPU52.CAL:Function_call_interrupts
   1063758 ±  6%     -23.4%     815164 ±  3%  interrupts.CPU52.LOC:Local_timer_interrupts
    276.00 ±  2%     -26.4%     203.25        interrupts.CPU53.167:IR-PCI-MSI.1574965-edge.eth1-TxRx-53
      5797 ± 15%     -24.1%       4402        interrupts.CPU53.CAL:Function_call_interrupts
   1063910 ±  6%     -23.4%     815224 ±  3%  interrupts.CPU53.LOC:Local_timer_interrupts
    275.50 ±  2%     -25.3%     205.75        interrupts.CPU54.168:IR-PCI-MSI.1574966-edge.eth1-TxRx-54
      5781 ± 15%     -24.0%       4396        interrupts.CPU54.CAL:Function_call_interrupts
   1064139 ±  6%     -23.4%     815128 ±  3%  interrupts.CPU54.LOC:Local_timer_interrupts
    322.25 ± 24%     -36.9%     203.25        interrupts.CPU55.169:IR-PCI-MSI.1574967-edge.eth1-TxRx-55
      5766 ± 14%     -24.1%       4375        interrupts.CPU55.CAL:Function_call_interrupts
   1063901 ±  6%     -23.4%     815109 ±  3%  interrupts.CPU55.LOC:Local_timer_interrupts
    275.50 ±  2%     -26.0%     203.75        interrupts.CPU56.170:IR-PCI-MSI.1574968-edge.eth1-TxRx-56
      5698 ± 15%     -23.4%       4366        interrupts.CPU56.CAL:Function_call_interrupts
   1063923 ±  6%     -23.4%     815101 ±  3%  interrupts.CPU56.LOC:Local_timer_interrupts
    276.00 ±  2%     -26.2%     203.75        interrupts.CPU57.171:IR-PCI-MSI.1574969-edge.eth1-TxRx-57
      5750 ± 15%     -24.2%       4358        interrupts.CPU57.CAL:Function_call_interrupts
   1063868 ±  6%     -23.4%     814792 ±  3%  interrupts.CPU57.LOC:Local_timer_interrupts
    275.00 ±  2%     -26.1%     203.25        interrupts.CPU58.172:IR-PCI-MSI.1574970-edge.eth1-TxRx-58
      5770 ± 15%     -24.5%       4357        interrupts.CPU58.CAL:Function_call_interrupts
   1063870 ±  6%     -23.4%     815218 ±  3%  interrupts.CPU58.LOC:Local_timer_interrupts
    275.25 ±  2%     -26.2%     203.25        interrupts.CPU59.173:IR-PCI-MSI.1574971-edge.eth1-TxRx-59
      5793 ± 16%     -24.9%       4349        interrupts.CPU59.CAL:Function_call_interrupts
   1063943 ±  6%     -23.4%     815138 ±  3%  interrupts.CPU59.LOC:Local_timer_interrupts
      5810 ± 12%     -33.3%       3878 ± 19%  interrupts.CPU6.CAL:Function_call_interrupts
   1068871 ±  5%     -23.5%     817731 ±  2%  interrupts.CPU6.LOC:Local_timer_interrupts
    275.00 ±  2%     -26.1%     203.25        interrupts.CPU60.174:IR-PCI-MSI.1574972-edge.eth1-TxRx-60
      5693 ± 14%     -23.8%       4339        interrupts.CPU60.CAL:Function_call_interrupts
   1063234 ±  6%     -23.3%     815105 ±  3%  interrupts.CPU60.LOC:Local_timer_interrupts
    275.50 ±  2%     -26.1%     203.50        interrupts.CPU61.175:IR-PCI-MSI.1574973-edge.eth1-TxRx-61
      5758 ± 16%     -24.8%       4330        interrupts.CPU61.CAL:Function_call_interrupts
   1063910 ±  6%     -23.4%     815085 ±  3%  interrupts.CPU61.LOC:Local_timer_interrupts
    275.25 ±  2%     -26.1%     203.50        interrupts.CPU62.176:IR-PCI-MSI.1574974-edge.eth1-TxRx-62
      5756 ± 16%     -24.9%       4324        interrupts.CPU62.CAL:Function_call_interrupts
   1064031 ±  6%     -23.4%     814990 ±  3%  interrupts.CPU62.LOC:Local_timer_interrupts
      5722 ± 15%     -24.6%       4312        interrupts.CPU63.CAL:Function_call_interrupts
   1065971 ±  5%     -23.5%     815015 ±  3%  interrupts.CPU63.LOC:Local_timer_interrupts
      5759 ± 15%     -24.4%       4351        interrupts.CPU64.CAL:Function_call_interrupts
   1063874 ±  6%     -23.4%     815087 ±  3%  interrupts.CPU64.LOC:Local_timer_interrupts
      5750 ± 15%     -24.0%       4370 ±  2%  interrupts.CPU65.CAL:Function_call_interrupts
   1064215 ±  6%     -23.4%     815017 ±  3%  interrupts.CPU65.LOC:Local_timer_interrupts
      5737 ± 15%     -23.8%       4371 ±  3%  interrupts.CPU66.CAL:Function_call_interrupts
   1064010 ±  6%     -23.4%     815282 ±  3%  interrupts.CPU66.LOC:Local_timer_interrupts
      5717 ± 15%     -22.8%       4416 ±  2%  interrupts.CPU67.CAL:Function_call_interrupts
   1063785 ±  6%     -23.4%     815181 ±  3%  interrupts.CPU67.LOC:Local_timer_interrupts
      5613 ± 17%     -22.3%       4360 ±  3%  interrupts.CPU68.CAL:Function_call_interrupts
   1064069 ±  6%     -23.4%     814925 ±  3%  interrupts.CPU68.LOC:Local_timer_interrupts
      5687 ± 15%     -23.3%       4359 ±  3%  interrupts.CPU69.CAL:Function_call_interrupts
   1063564 ±  6%     -23.4%     815054 ±  3%  interrupts.CPU69.LOC:Local_timer_interrupts
    366.25 ± 32%     -34.8%     238.75 ±  8%  interrupts.CPU7.117:IR-PCI-MSI.1574919-edge.eth1-TxRx-7
      5826 ± 13%     -25.9%       4320        interrupts.CPU7.CAL:Function_call_interrupts
   1068409 ±  5%     -23.5%     817390 ±  2%  interrupts.CPU7.LOC:Local_timer_interrupts
      5709 ± 15%     -23.7%       4357 ±  3%  interrupts.CPU70.CAL:Function_call_interrupts
   1063871 ±  6%     -23.4%     815060 ±  3%  interrupts.CPU70.LOC:Local_timer_interrupts
      5695 ± 15%     -23.6%       4349 ±  3%  interrupts.CPU71.CAL:Function_call_interrupts
   1064132 ±  6%     -23.4%     814595 ±  3%  interrupts.CPU71.LOC:Local_timer_interrupts
      5692 ± 13%     -23.5%       4351 ±  3%  interrupts.CPU72.CAL:Function_call_interrupts
   1065764 ±  5%     -23.5%     815695 ±  3%  interrupts.CPU72.LOC:Local_timer_interrupts
      5863 ± 14%     -27.8%       4235 ±  2%  interrupts.CPU73.CAL:Function_call_interrupts
   1066184 ±  5%     -23.5%     816127 ±  3%  interrupts.CPU73.LOC:Local_timer_interrupts
      5840 ± 13%     -25.9%       4327 ±  3%  interrupts.CPU74.CAL:Function_call_interrupts
   1065855 ±  5%     -23.5%     815593 ±  3%  interrupts.CPU74.LOC:Local_timer_interrupts
      5882 ± 12%     -25.6%       4374 ±  2%  interrupts.CPU75.CAL:Function_call_interrupts
   1065844 ±  5%     -23.5%     815713 ±  3%  interrupts.CPU75.LOC:Local_timer_interrupts
      5866 ± 12%     -26.2%       4327 ±  2%  interrupts.CPU76.CAL:Function_call_interrupts
   1065792 ±  5%     -23.5%     815691 ±  3%  interrupts.CPU76.LOC:Local_timer_interrupts
      5805 ± 12%     -25.0%       4356 ±  2%  interrupts.CPU77.CAL:Function_call_interrupts
   1065496 ±  5%     -23.5%     815630 ±  3%  interrupts.CPU77.LOC:Local_timer_interrupts
      5779 ± 12%     -23.7%       4411 ±  4%  interrupts.CPU78.CAL:Function_call_interrupts
   1065678 ±  5%     -23.5%     815169 ±  3%  interrupts.CPU78.LOC:Local_timer_interrupts
      5795 ± 12%     -24.8%       4359 ±  2%  interrupts.CPU79.CAL:Function_call_interrupts
   1065750 ±  5%     -23.5%     815664 ±  3%  interrupts.CPU79.LOC:Local_timer_interrupts
      5769 ± 14%     -25.2%       4317        interrupts.CPU8.CAL:Function_call_interrupts
   1068358 ±  5%     -23.5%     817290 ±  2%  interrupts.CPU8.LOC:Local_timer_interrupts
      5830 ± 14%     -24.6%       4395 ±  2%  interrupts.CPU80.CAL:Function_call_interrupts
   1065810 ±  5%     -23.5%     815187 ±  3%  interrupts.CPU80.LOC:Local_timer_interrupts
      5875 ± 13%     -24.4%       4439        interrupts.CPU81.CAL:Function_call_interrupts
   1065604 ±  5%     -23.4%     815987 ±  3%  interrupts.CPU81.LOC:Local_timer_interrupts
      5965 ± 12%     -25.5%       4443        interrupts.CPU82.CAL:Function_call_interrupts
   1065923 ±  5%     -23.5%     815763 ±  3%  interrupts.CPU82.LOC:Local_timer_interrupts
      5945 ± 13%     -26.1%       4394 ±  2%  interrupts.CPU83.CAL:Function_call_interrupts
   1065777 ±  5%     -23.5%     815539 ±  3%  interrupts.CPU83.LOC:Local_timer_interrupts
      5931 ± 13%     -25.3%       4431        interrupts.CPU84.CAL:Function_call_interrupts
   1065465 ±  5%     -23.5%     815387 ±  3%  interrupts.CPU84.LOC:Local_timer_interrupts
      5920 ± 14%     -25.5%       4408        interrupts.CPU85.CAL:Function_call_interrupts
   1065872 ±  5%     -23.5%     815110 ±  3%  interrupts.CPU85.LOC:Local_timer_interrupts
      5891 ± 14%     -25.1%       4414        interrupts.CPU86.CAL:Function_call_interrupts
   1065640 ±  5%     -23.5%     815544 ±  3%  interrupts.CPU86.LOC:Local_timer_interrupts
      5895 ± 13%     -25.3%       4403        interrupts.CPU87.CAL:Function_call_interrupts
   1065362 ±  5%     -23.4%     815629 ±  3%  interrupts.CPU87.LOC:Local_timer_interrupts
      5939 ± 13%     -26.9%       4341 ±  2%  interrupts.CPU88.CAL:Function_call_interrupts
   1065509 ±  5%     -23.5%     815643 ±  3%  interrupts.CPU88.LOC:Local_timer_interrupts
      5919 ± 13%     -25.7%       4395        interrupts.CPU89.CAL:Function_call_interrupts
   1065465 ±  5%     -23.4%     815615 ±  3%  interrupts.CPU89.LOC:Local_timer_interrupts
      5762 ± 13%     -25.2%       4311        interrupts.CPU9.CAL:Function_call_interrupts
   1067849 ±  5%     -23.5%     817350 ±  2%  interrupts.CPU9.LOC:Local_timer_interrupts
      5940 ± 12%     -26.1%       4389        interrupts.CPU90.CAL:Function_call_interrupts
   1065728 ±  5%     -23.5%     815578 ±  3%  interrupts.CPU90.LOC:Local_timer_interrupts
      5922 ± 13%     -26.1%       4374        interrupts.CPU91.CAL:Function_call_interrupts
   1065455 ±  5%     -23.5%     815494 ±  3%  interrupts.CPU91.LOC:Local_timer_interrupts
      5950 ± 12%     -30.0%       4165 ±  4%  interrupts.CPU92.CAL:Function_call_interrupts
   1065765 ±  5%     -23.4%     815995 ±  3%  interrupts.CPU92.LOC:Local_timer_interrupts
    685.00 ± 97%     -75.9%     164.75 ± 34%  interrupts.CPU92.RES:Rescheduling_interrupts
      6025 ± 14%     -26.3%       4441        interrupts.CPU93.CAL:Function_call_interrupts
   1065657 ±  5%     -23.4%     815771 ±  3%  interrupts.CPU93.LOC:Local_timer_interrupts
      6050 ± 11%     -27.4%       4389        interrupts.CPU94.CAL:Function_call_interrupts
   1065874 ±  5%     -23.5%     815599 ±  3%  interrupts.CPU94.LOC:Local_timer_interrupts
      5956 ± 10%     -30.6%       4133 ±  6%  interrupts.CPU95.CAL:Function_call_interrupts
   1071360 ±  4%     -23.7%     816944 ±  3%  interrupts.CPU95.LOC:Local_timer_interrupts
     12839 ±  5%     -20.9%      10158 ±  6%  interrupts.CPU95.RES:Rescheduling_interrupts
      5968 ± 10%     -28.1%       4292 ±  3%  interrupts.CPU96.CAL:Function_call_interrupts
   1071091 ±  5%     -23.7%     817467 ±  2%  interrupts.CPU96.LOC:Local_timer_interrupts
      6006 ± 11%     -26.8%       4395        interrupts.CPU97.CAL:Function_call_interrupts
   1068538 ±  5%     -23.5%     817381 ±  2%  interrupts.CPU97.LOC:Local_timer_interrupts
      6069 ± 13%     -27.3%       4409        interrupts.CPU98.CAL:Function_call_interrupts
   1068457 ±  5%     -23.5%     817227 ±  2%  interrupts.CPU98.LOC:Local_timer_interrupts
     25.00 ± 26%   +1018.0%     279.50 ± 46%  interrupts.CPU98.RES:Rescheduling_interrupts
      6048 ± 13%     -26.8%       4426        interrupts.CPU99.CAL:Function_call_interrupts
   1068492 ±  5%     -23.5%     817135 ±  2%  interrupts.CPU99.LOC:Local_timer_interrupts
 2.046e+08 ±  5%     -23.5%  1.566e+08 ±  3%  interrupts.LOC:Local_timer_interrupts
    308.50 ± 70%     -84.8%      47.00 ± 54%  interrupts.TLB:TLB_shootdowns





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Rong Chen


View attachment "config-5.0.0-11177-g9d65ac8" of type "text/plain" (192841 bytes)

View attachment "job-script" of type "text/plain" (7547 bytes)

View attachment "job.yaml" of type "text/plain" (5075 bytes)

View attachment "reproduce" of type "text/plain" (913 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ