lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Date:   Tue, 3 Nov 2020 13:03:29 +0800
From:   kernel test robot <rong.a.chen@...el.com>
To:     Joonsoo Kim <iamjoonsoo.kim@....com>
Cc:     Linus Torvalds <torvalds@...ux-foundation.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Johannes Weiner <hannes@...xchg.org>,
        Vlastimil Babka <vbabka@...e.cz>,
        Hugh Dickins <hughd@...gle.com>,
        Matthew Wilcox <willy@...radead.org>,
        Mel Gorman <mgorman@...hsingularity.net>,
        Michal Hocko <mhocko@...nel.org>,
        Minchan Kim <minchan@...nel.org>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org,
        lkp@...el.com, ying.huang@...el.com, feng.tang@...el.com,
        zhengjun.xing@...el.com
Subject: [mm/swap] aae466b005: vm-scalability.throughput 17.2% improvement

Greeting,

FYI, we noticed a 17.2% improvement of vm-scalability.throughput due to commit:


commit: aae466b0052e1888edd1d7f473d4310d64936196 ("mm/swap: implement workingset detection for anonymous LRU")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master


in testcase: vm-scalability
on test machine: 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory
with following parameters:

	thp_enabled: never
	thp_defrag: never
	nr_task: 8
	nr_pmem: 4
	priority: 1
	test: swap-w-seq
	cpufreq_governor: performance
	ucode: 0x16

test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us.
test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/





Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
compiler/cpufreq_governor/kconfig/nr_pmem/nr_task/priority/rootfs/tbox_group/test/testcase/thp_defrag/thp_enabled/ucode:
  gcc-9/performance/x86_64-rhel-8.3/4/8/1/debian-10.4-x86_64-20200603.cgz/lkp-hsw-4ex1/swap-w-seq/vm-scalability/never/never/0x16

commit: 
  3852f6768e ("mm/swapcache: support to handle the shadow entries")
  aae466b005 ("mm/swap: implement workingset detection for anonymous LRU")

3852f6768ede542e aae466b0052e1888edd1d7f473d 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
          2:4            1%           2:4     perf-profile.children.cycles-pp.error_entry
          1:4            2%           2:4     perf-profile.self.cycles-pp.error_entry
         %stddev     %change         %stddev
             \          |                \  
     23.24 ±  3%      +8.6%      25.25        vm-scalability.free_time
    451459 ±  2%     +17.6%     531040 ±  2%  vm-scalability.median
   3613724 ±  2%     +17.2%    4233483 ±  2%  vm-scalability.throughput
    147.90 ±  2%     -11.1%     131.45 ±  2%  vm-scalability.time.elapsed_time
    147.90 ±  2%     -11.1%     131.45 ±  2%  vm-scalability.time.elapsed_time.max
    126092           -40.2%      75396 ±  5%  vm-scalability.time.involuntary_context_switches
    709.75            +3.4%     733.75        vm-scalability.time.percent_of_cpu_this_job_got
    858.01 ±  6%     -12.7%     748.79 ±  3%  vm-scalability.time.system_time
      2555 ± 11%     -86.5%     345.25 ± 19%  vm-scalability.time.voluntary_context_switches
      0.96 ± 22%     +23.2%       1.18 ±  2%  iostat.cpu.user
      0.46 ±  2%      -0.4        0.08 ±  6%  mpstat.cpu.all.soft%
  19058794 ± 73%     -51.3%    9284205 ±108%  numa-numastat.node2.numa_foreign
   1166704           +28.5%    1499626        vmstat.memory.cache
   2470874 ±  2%     +12.5%    2779610 ±  2%  vmstat.swap.so
      5097           -25.6%       3792 ±  3%  vmstat.system.cs
    775916 ±  6%     +15.8%     898759        vmstat.system.in
  3.27e+09 ±  3%      -9.3%  2.964e+09 ±  2%  perf-node.node-load-misses
 1.755e+09 ±  4%     -16.1%  1.473e+09 ±  9%  perf-node.node-loads
     40.00 ±  4%     -62.5%      15.00        perf-node.node-local-store-ratio
 2.337e+09 ±  2%     -39.6%  1.412e+09 ±  2%  perf-node.node-store-misses
 1.594e+09 ±  3%     -83.9%  2.563e+08 ±  2%  perf-node.node-stores
  11725373           +15.5%   13547085 ±  6%  meminfo.DirectMap2M
    179916          +187.0%     516441        meminfo.KReclaimable
   1116912 ±  5%     +19.9%    1339332 ±  4%  meminfo.MemAvailable
    179916          +187.0%     516441        meminfo.SReclaimable
    413707           +80.4%     746239        meminfo.Slab
    218196 ±  2%     +12.4%     245320 ±  2%  meminfo.max_used_kB
     11422 ± 11%    +182.9%      32315 ± 25%  numa-vmstat.node0.nr_slab_reclaimable
     30123           +11.0%      33440 ±  4%  numa-vmstat.node1.nr_page_table_pages
     11318 ± 10%    +187.6%      32549 ± 15%  numa-vmstat.node1.nr_slab_reclaimable
  14498502           +11.0%   16097678 ±  4%  numa-vmstat.node1.nr_vmscan_write
  14498504           +11.0%   16097680 ±  4%  numa-vmstat.node1.nr_written
      2099 ± 74%     -67.5%     681.25 ± 58%  numa-vmstat.node2.nr_active_anon
     11274 ± 13%    +172.3%      30698 ± 11%  numa-vmstat.node2.nr_slab_reclaimable
      2101 ± 74%     -67.6%     681.50 ± 58%  numa-vmstat.node2.nr_zone_active_anon
  11491764 ± 72%     -54.4%    5245129 ±107%  numa-vmstat.node2.numa_foreign
     10784 ±  7%    +210.9%      33532 ±  6%  numa-vmstat.node3.nr_slab_reclaimable
      6117 ±  6%     -15.4%       5175 ±  2%  slabinfo.files_cache.active_objs
      6124 ±  5%     -15.5%       5175 ±  2%  slabinfo.files_cache.num_objs
      1269 ±  3%     +21.3%       1539 ±  5%  slabinfo.khugepaged_mm_slot.active_objs
      1269 ±  3%     +21.3%       1539 ±  5%  slabinfo.khugepaged_mm_slot.num_objs
     12779 ±  3%     -15.2%      10839 ±  4%  slabinfo.pde_opener.active_objs
     12779 ±  3%     -15.2%      10839 ±  4%  slabinfo.pde_opener.num_objs
    166610 ±  2%    +352.8%     754410        slabinfo.radix_tree_node.active_objs
      3337 ±  3%    +318.4%      13961        slabinfo.radix_tree_node.active_slabs
    167671 ±  2%    +366.3%     781839        slabinfo.radix_tree_node.num_objs
      3337 ±  3%    +318.4%      13961        slabinfo.radix_tree_node.num_slabs
      3109 ±  4%     -10.4%       2785        slabinfo.sighand_cache.active_objs
      3131 ±  4%     -10.4%       2807 ±  2%  slabinfo.sighand_cache.num_objs
      8020 ±  4%     -12.5%       7021 ±  2%  slabinfo.task_delay_info.active_objs
      8020 ±  4%     -12.5%       7021 ±  2%  slabinfo.task_delay_info.num_objs
     45388 ± 11%    +184.5%     129110 ± 25%  numa-meminfo.node0.KReclaimable
     45388 ± 11%    +184.5%     129110 ± 25%  numa-meminfo.node0.SReclaimable
    114445 ± 14%     +74.5%     199762 ± 18%  numa-meminfo.node0.Slab
     45359 ±  9%    +187.0%     130162 ± 15%  numa-meminfo.node1.KReclaimable
    120458           +11.0%     133695 ±  4%  numa-meminfo.node1.PageTables
     45359 ±  9%    +187.0%     130162 ± 15%  numa-meminfo.node1.SReclaimable
    109895 ± 16%     +71.3%     188201 ± 13%  numa-meminfo.node1.Slab
      8457 ± 73%     -67.5%       2751 ± 59%  numa-meminfo.node2.Active
      8389 ± 74%     -67.5%       2726 ± 58%  numa-meminfo.node2.Active(anon)
     45582 ± 13%    +169.3%     122741 ± 11%  numa-meminfo.node2.KReclaimable
     45582 ± 13%    +169.3%     122741 ± 11%  numa-meminfo.node2.SReclaimable
     97401 ± 13%     +73.7%     169199 ±  6%  numa-meminfo.node2.Slab
     43179 ±  6%    +210.4%     134010 ±  6%  numa-meminfo.node3.KReclaimable
     43179 ±  6%    +210.4%     134010 ±  6%  numa-meminfo.node3.SReclaimable
     91559 ±  9%    +106.1%     188662 ±  8%  numa-meminfo.node3.Slab
  20729509 ± 16%  -7.8e+06    12912354 ± 25%  syscalls.sys_close.noise.100%
  28960042 ± 11%  -8.2e+06    20719899 ± 14%  syscalls.sys_close.noise.2%
  27766391 ± 11%  -8.4e+06    19401559 ± 15%  syscalls.sys_close.noise.25%
  28908895 ± 11%  -8.2e+06    20661833 ± 14%  syscalls.sys_close.noise.5%
  25426814 ± 13%  -8.5e+06    16899949 ± 18%  syscalls.sys_close.noise.50%
  22869034 ± 14%    -8e+06    14822779 ± 21%  syscalls.sys_close.noise.75%
  32447789 ± 25%     -69.7%    9828013 ± 98%  syscalls.sys_openat.max
 1.654e+09 ± 39%  -1.4e+09   2.337e+08 ± 39%  syscalls.sys_openat.noise.100%
 1.706e+09 ± 38%  -1.4e+09   2.834e+08 ± 32%  syscalls.sys_openat.noise.2%
 1.693e+09 ± 38%  -1.4e+09   2.712e+08 ± 34%  syscalls.sys_openat.noise.25%
 1.706e+09 ± 38%  -1.4e+09   2.829e+08 ± 32%  syscalls.sys_openat.noise.5%
 1.678e+09 ± 38%  -1.4e+09   2.548e+08 ± 36%  syscalls.sys_openat.noise.50%
 1.667e+09 ± 39%  -1.4e+09   2.438e+08 ± 38%  syscalls.sys_openat.noise.75%
      1174 ± 22%     +53.6%       1803 ±  5%  syscalls.sys_write.med
 5.911e+08 ± 64%    +2e+09   2.555e+09 ±  2%  syscalls.sys_write.noise.100%
 6.002e+08 ± 63%    +2e+09    2.56e+09 ±  2%  syscalls.sys_write.noise.2%
 5.985e+08 ± 64%    +2e+09   2.559e+09 ±  2%  syscalls.sys_write.noise.25%
 6.002e+08 ± 63%    +2e+09    2.56e+09 ±  2%  syscalls.sys_write.noise.5%
 5.953e+08 ± 64%    +2e+09   2.558e+09 ±  2%  syscalls.sys_write.noise.50%
  5.93e+08 ± 64%    +2e+09   2.557e+09 ±  2%  syscalls.sys_write.noise.75%
     52505 ±  2%      +8.7%      57048 ±  4%  proc-vmstat.allocstall_movable
   1329701 ± 53%     -76.6%     311376 ± 84%  proc-vmstat.compact_daemon_migrate_scanned
      3092 ±104%     -98.4%      48.75 ± 73%  proc-vmstat.compact_daemon_wake
     91.25 ± 12%     -83.0%      15.50 ± 39%  proc-vmstat.compact_fail
   1470217 ± 46%     -68.7%     459729 ± 57%  proc-vmstat.compact_migrate_scanned
     97.50 ± 13%     -81.3%      18.25 ± 32%  proc-vmstat.compact_stall
      3161 ± 98%     -96.3%     117.50 ± 72%  proc-vmstat.kswapd_low_wmark_hit_quickly
   5191554            -2.4%    5065321        proc-vmstat.nr_anon_pages
   5184927            -2.4%    5059469        proc-vmstat.nr_inactive_anon
    187.75 ±  4%      -9.1%     170.75        proc-vmstat.nr_isolated_anon
    111810            +1.6%     113619        proc-vmstat.nr_page_table_pages
     44657          +188.9%     129025        proc-vmstat.nr_slab_reclaimable
     58446            -1.7%      57450        proc-vmstat.nr_slab_unreclaimable
   5183621            -2.4%    5059220        proc-vmstat.nr_zone_inactive_anon
  56750562 ± 10%     -26.1%   41935745 ± 19%  proc-vmstat.numa_pte_updates
   1625054 ±  9%    -100.0%       0.00        proc-vmstat.pgalloc_dma
   8718205 ±  6%     -40.3%    5207463 ± 26%  proc-vmstat.pgalloc_dma32
 1.346e+08           -30.0%   94227189        proc-vmstat.pgalloc_normal
 1.451e+08           -31.4%   99498336        proc-vmstat.pgfree
     63642 ±  3%      -7.3%      59028 ±  5%  proc-vmstat.slabs_scanned
    456.00 ±127%    +223.0%       1473 ± 21%  proc-vmstat.swap_ra
    345.00 ±165%    +286.0%       1331 ± 20%  proc-vmstat.swap_ra_hit
    621120 ±  3%     -12.0%     546816 ±  2%  proc-vmstat.unevictable_pgs_scanned
  5.92e+09 ±  3%      +6.5%  6.304e+09        perf-stat.i.branch-instructions
  61633368 ±  2%     -24.6%   46490910        perf-stat.i.cache-misses
      5041           -26.5%       3706 ±  3%  perf-stat.i.context-switches
    388.19           -23.9%     295.51        perf-stat.i.cpu-migrations
    809.81 ±  8%     +21.0%     980.06 ±  7%  perf-stat.i.cycles-between-cache-misses
 6.351e+09            +6.7%  6.776e+09 ±  2%  perf-stat.i.dTLB-loads
      0.15 ±  5%      +0.0        0.18 ±  5%  perf-stat.i.dTLB-store-miss-rate%
 3.606e+09 ±  3%      -7.4%  3.339e+09 ±  2%  perf-stat.i.dTLB-stores
   6823772 ±  5%     +14.2%    7794812 ±  8%  perf-stat.i.iTLB-load-misses
   2.4e+10 ±  3%      +4.7%  2.513e+10        perf-stat.i.instructions
    112.11            +3.1%     115.57        perf-stat.i.metric.M/sec
    666516 ±  2%     +12.5%     749805 ±  2%  perf-stat.i.minor-faults
     61.38 ±  2%     +20.6       82.01        perf-stat.i.node-store-miss-rate%
  15267649 ±  4%     -25.5%   11374283 ± 10%  perf-stat.i.node-store-misses
  10435945 ±  6%     -79.8%    2104898 ± 10%  perf-stat.i.node-stores
    666572 ±  2%     +12.5%     749872 ±  2%  perf-stat.i.page-faults
    659.86 ±  6%     +28.5%     848.20 ±  6%  perf-stat.overall.cycles-between-cache-misses
      0.16 ±  3%      +0.0        0.20 ±  3%  perf-stat.overall.dTLB-store-miss-rate%
     59.26 ±  2%     +25.0       84.28        perf-stat.overall.node-store-miss-rate%
      8026            -6.9%       7474        perf-stat.overall.path-length
 5.902e+09 ±  3%      +6.3%  6.274e+09        perf-stat.ps.branch-instructions
  61374057 ±  2%     -24.7%   46230779        perf-stat.ps.cache-misses
      5019           -26.6%       3682 ±  3%  perf-stat.ps.context-switches
    386.19           -24.1%     293.29        perf-stat.ps.cpu-migrations
 6.332e+09            +6.5%  6.744e+09 ±  2%  perf-stat.ps.dTLB-loads
 3.595e+09 ±  3%      -7.6%  3.323e+09 ±  2%  perf-stat.ps.dTLB-stores
   6808353 ±  5%     +14.1%    7769606 ±  8%  perf-stat.ps.iTLB-load-misses
 2.392e+10 ±  2%      +4.6%  2.502e+10        perf-stat.ps.instructions
    666070 ±  2%     +12.2%     747635 ±  2%  perf-stat.ps.minor-faults
  15144045 ±  4%     -25.7%   11246617 ± 10%  perf-stat.ps.node-store-misses
  10417852 ±  6%     -79.9%    2096205 ± 10%  perf-stat.ps.node-stores
    666126 ±  2%     +12.2%     747701 ±  2%  perf-stat.ps.page-faults
 3.555e+12            -6.8%  3.311e+12        perf-stat.total.instructions
     12185 ±  8%    +182.0%      34361 ±  8%  sched_debug.cfs_rq:/.exec_clock.max
      1188 ± 14%     -98.8%      13.90 ± 53%  sched_debug.cfs_rq:/.exec_clock.min
      2713 ± 12%    +171.7%       7373 ±  8%  sched_debug.cfs_rq:/.exec_clock.stddev
    158216 ± 20%     -27.7%     114407 ± 17%  sched_debug.cfs_rq:/.load.stddev
     43.24 ± 15%     -23.4%      33.11 ± 19%  sched_debug.cfs_rq:/.load_avg.avg
      1022 ±  3%     -28.2%     733.92 ±  3%  sched_debug.cfs_rq:/.load_avg.max
    170.08 ±  8%     -25.7%     126.42 ±  9%  sched_debug.cfs_rq:/.load_avg.stddev
    119278 ± 11%    +110.4%     250918 ±  6%  sched_debug.cfs_rq:/.min_vruntime.max
     23859 ± 13%    +115.1%      51319 ± 14%  sched_debug.cfs_rq:/.min_vruntime.stddev
      0.06 ± 48%     -42.5%       0.04 ± 11%  sched_debug.cfs_rq:/.nr_spread_over.avg
      0.23 ± 26%     -32.0%       0.16 ±  8%  sched_debug.cfs_rq:/.nr_spread_over.stddev
    -50753          +175.6%    -139895        sched_debug.cfs_rq:/.spread0.avg
    -83164          +122.8%    -185253        sched_debug.cfs_rq:/.spread0.min
     23868 ± 13%    +115.0%      51318 ± 14%  sched_debug.cfs_rq:/.spread0.stddev
    140364 ± 14%     +23.7%     173587 ±  4%  sched_debug.cpu.avg_idle.stddev
    678.88 ±  4%     +20.0%     814.72 ±  5%  sched_debug.cpu.clock_task.stddev
      0.00 ± 26%    +566.5%       0.00 ± 91%  sched_debug.cpu.next_balance.stddev
      3913           -17.4%       3233 ±  2%  sched_debug.cpu.nr_switches.avg
     14639 ± 12%     +55.0%      22684 ± 19%  sched_debug.cpu.nr_switches.max
      1791 ±  8%     -37.4%       1121 ±  7%  sched_debug.cpu.nr_switches.min
      2060 ±  8%     +27.8%       2633 ±  6%  sched_debug.cpu.nr_switches.stddev
      2394 ±  2%     -28.3%       1716 ±  5%  sched_debug.cpu.sched_count.avg
     10139 ± 24%     +79.3%      18179 ± 24%  sched_debug.cpu.sched_count.max
    961.92 ± 11%     -54.9%     434.25        sched_debug.cpu.sched_count.min
      1375 ± 14%     +47.3%       2025 ± 10%  sched_debug.cpu.sched_count.stddev
    626.15 ±  4%     -19.1%     506.24 ±  5%  sched_debug.cpu.sched_goidle.avg
      4252 ± 25%    +109.7%       8919 ± 24%  sched_debug.cpu.sched_goidle.max
    277.58 ± 11%     -63.6%     101.00 ±  4%  sched_debug.cpu.sched_goidle.min
    554.97 ±  9%     +71.0%     949.07 ± 11%  sched_debug.cpu.sched_goidle.stddev
      1105 ±  2%     -30.2%     771.36 ±  4%  sched_debug.cpu.ttwu_count.avg
      4824 ± 23%     +89.2%       9126 ± 21%  sched_debug.cpu.ttwu_count.max
    364.25 ± 13%     -61.1%     141.83        sched_debug.cpu.ttwu_count.min
    727.66 ±  8%     +44.9%       1054 ±  8%  sched_debug.cpu.ttwu_count.stddev
    754.79           -25.7%     560.55 ±  5%  sched_debug.cpu.ttwu_local.avg
      1844 ±  5%     +94.2%       3581 ± 25%  sched_debug.cpu.ttwu_local.max
    281.33 ± 10%     -50.5%     139.25        sched_debug.cpu.ttwu_local.min
    338.12 ± 10%     +70.6%     576.68 ±  5%  sched_debug.cpu.ttwu_local.stddev
      4.68 ±  4%      -4.0        0.73 ± 44%  perf-profile.calltrace.cycles-pp.add_to_swap_cache.add_to_swap.shrink_page_list.shrink_inactive_list.shrink_lruvec
      6.00 ±  4%      -3.2        2.75 ± 13%  perf-profile.calltrace.cycles-pp.add_to_swap.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node
      1.82 ± 15%      -1.5        0.28 ±100%  perf-profile.calltrace.cycles-pp.prep_new_page.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.do_anonymous_page
      1.74 ±  5%      -1.4        0.29 ±100%  perf-profile.calltrace.cycles-pp.__delete_from_swap_cache.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec
      2.55 ± 11%      -1.4        1.16 ± 12%  perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.do_anonymous_page.__handle_mm_fault
      1.44 ± 15%      -1.0        0.47 ±106%  perf-profile.calltrace.cycles-pp.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      1.43 ± 16%      -1.0        0.47 ±106%  perf-profile.calltrace.cycles-pp.__softirqentry_text_start.asm_call_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt
      1.43 ± 16%      -1.0        0.47 ±106%  perf-profile.calltrace.cycles-pp.asm_call_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      1.55 ± 16%      -0.8        0.70 ± 68%  perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      2.10 ± 15%      -0.5        1.61 ± 12%  perf-profile.calltrace.cycles-pp.mem_cgroup_charge.do_anonymous_page.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
      2.12 ±  8%      +0.4        2.52 ± 10%  perf-profile.calltrace.cycles-pp.end_page_writeback.pmem_rw_page.bdev_write_page.__swap_writepage.pageout
      2.13 ±  5%      +0.4        2.58 ± 11%  perf-profile.calltrace.cycles-pp.try_to_unmap_one.rmap_walk_anon.try_to_unmap.shrink_page_list.shrink_inactive_list
      0.95 ± 26%      +0.5        1.47 ± 13%  perf-profile.calltrace.cycles-pp.swap_writepage.pageout.shrink_page_list.shrink_inactive_list.shrink_lruvec
      1.28 ±  4%      +0.5        1.81 ± 14%  perf-profile.calltrace.cycles-pp.page_counter_uncharge.mem_cgroup_swapout.__remove_mapping.shrink_page_list.shrink_inactive_list
      1.27 ±  4%      +0.5        1.80 ± 14%  perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.mem_cgroup_swapout.__remove_mapping.shrink_page_list
      0.54 ± 71%      +0.9        1.41 ± 10%  perf-profile.calltrace.cycles-pp.__test_set_page_writeback.bdev_write_page.__swap_writepage.pageout.shrink_page_list
      2.52 ±  5%      +0.9        3.44 ± 14%  perf-profile.calltrace.cycles-pp.mem_cgroup_swapout.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec
      5.49 ±  4%      -4.8        0.69 ± 44%  perf-profile.children.cycles-pp.__softirqentry_text_start
      4.76 ±  4%      -3.7        1.09 ± 12%  perf-profile.children.cycles-pp.add_to_swap_cache
      3.74 ±  5%      -3.6        0.16 ± 19%  perf-profile.children.cycles-pp.xas_create_range
      3.74 ±  5%      -3.5        0.27 ± 15%  perf-profile.children.cycles-pp.xas_create
      6.06 ±  4%      -3.3        2.77 ± 13%  perf-profile.children.cycles-pp.add_to_swap
      2.32 ± 11%      -1.6        0.70 ± 44%  perf-profile.children.cycles-pp.do_softirq_own_stack
      2.61 ± 11%      -1.6        1.04 ± 35%  perf-profile.children.cycles-pp.irq_exit_rcu
      5.20 ± 11%      -1.5        3.68 ± 11%  perf-profile.children.cycles-pp.get_page_from_freelist
      3.78 ± 14%      -1.3        2.45 ± 12%  perf-profile.children.cycles-pp.prep_new_page
      3.61 ± 14%      -1.3        2.32 ± 12%  perf-profile.children.cycles-pp.clear_page_erms
      1.21 ±  7%      -1.0        0.21 ± 20%  perf-profile.children.cycles-pp.xas_store
      1.75 ±  5%      -0.9        0.86 ± 13%  perf-profile.children.cycles-pp.__delete_from_swap_cache
      2.17 ± 14%      -0.5        1.63 ± 12%  perf-profile.children.cycles-pp.mem_cgroup_charge
      0.61 ± 17%      -0.5        0.15 ± 25%  perf-profile.children.cycles-pp.worker_thread
      0.58 ± 19%      -0.4        0.13 ± 24%  perf-profile.children.cycles-pp.process_one_work
      0.82 ±  7%      -0.3        0.56 ± 10%  perf-profile.children.cycles-pp.rmqueue
      0.75 ± 13%      -0.2        0.52 ±  8%  perf-profile.children.cycles-pp.mem_cgroup_charge_statistics
      0.55 ±  6%      -0.2        0.37 ± 11%  perf-profile.children.cycles-pp.free_pcppages_bulk
      0.77 ± 13%      -0.2        0.59 ±  5%  perf-profile.children.cycles-pp.__count_memcg_events
      0.19 ± 30%      -0.1        0.04 ± 58%  perf-profile.children.cycles-pp.__libc_fork
      0.48 ± 13%      -0.1        0.34 ± 13%  perf-profile.children.cycles-pp.try_charge
      0.36 ± 13%      -0.1        0.23 ± 14%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.34 ± 12%      -0.1        0.22 ± 13%  perf-profile.children.cycles-pp.do_syscall_64
      0.15 ± 22%      -0.1        0.03 ±100%  perf-profile.children.cycles-pp.__do_sys_clone
      0.15 ± 22%      -0.1        0.03 ±100%  perf-profile.children.cycles-pp._do_fork
      0.21 ±  7%      -0.1        0.11 ± 23%  perf-profile.children.cycles-pp.drm_fb_helper_dirty_work
      0.35 ±  5%      -0.1        0.27 ± 13%  perf-profile.children.cycles-pp.rmqueue_bulk
      0.15 ± 14%      -0.1        0.09 ± 23%  perf-profile.children.cycles-pp.drm_atomic_helper_dirtyfb
      0.15 ± 14%      -0.1        0.09 ± 23%  perf-profile.children.cycles-pp.memcpy_toio
      0.15 ± 14%      -0.1        0.09 ± 23%  perf-profile.children.cycles-pp.drm_atomic_helper_commit
      0.15 ± 14%      -0.1        0.09 ± 23%  perf-profile.children.cycles-pp.commit_tail
      0.15 ± 14%      -0.1        0.09 ± 23%  perf-profile.children.cycles-pp.drm_atomic_helper_commit_tail
      0.15 ± 14%      -0.1        0.09 ± 23%  perf-profile.children.cycles-pp.drm_atomic_helper_commit_planes
      0.15 ± 14%      -0.1        0.09 ± 23%  perf-profile.children.cycles-pp.mgag200_simple_display_pipe_update
      0.15 ± 14%      -0.1        0.09 ± 23%  perf-profile.children.cycles-pp.mgag200_handle_damage
      0.15 ± 14%      -0.1        0.09 ± 23%  perf-profile.children.cycles-pp.drm_fb_memcpy_dstclip
      0.10 ±  8%      -0.0        0.06 ± 14%  perf-profile.children.cycles-pp.schedule
      0.08 ± 19%      +0.0        0.11 ± 12%  perf-profile.children.cycles-pp.get_next_timer_interrupt
      0.14 ±  3%      +0.0        0.17 ± 10%  perf-profile.children.cycles-pp.default_send_IPI_single_phys
      0.12 ±  3%      +0.0        0.16 ±  9%  perf-profile.children.cycles-pp.scan_swap_map_try_ssd_cluster
      0.10 ± 11%      +0.0        0.14 ±  3%  perf-profile.children.cycles-pp.inc_node_page_state
      0.03 ±102%      +0.0        0.08 ±  6%  perf-profile.children.cycles-pp.arch_cpu_idle_enter
      0.03 ±102%      +0.0        0.08 ±  6%  perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
      0.13 ± 11%      +0.0        0.18 ± 14%  perf-profile.children.cycles-pp.cpumask_any_but
      0.23 ±  8%      +0.1        0.29 ± 10%  perf-profile.children.cycles-pp.scan_swap_map_slots
      0.15 ± 12%      +0.1        0.23 ±  7%  perf-profile.children.cycles-pp.lock_page_memcg
      0.31 ±  5%      +0.1        0.39 ±  8%  perf-profile.children.cycles-pp.test_clear_page_writeback
      0.29 ±  6%      +0.1        0.38 ± 11%  perf-profile.children.cycles-pp.get_swap_pages
      0.37 ±  7%      +0.1        0.46 ± 16%  perf-profile.children.cycles-pp.llist_reverse_order
      0.39 ±  8%      +0.1        0.52 ± 11%  perf-profile.children.cycles-pp.tlb_is_not_lazy
      0.35 ±  8%      +0.1        0.48 ± 14%  perf-profile.children.cycles-pp.zswap_frontswap_store
      0.45 ±  2%      +0.2        0.63 ± 14%  perf-profile.children.cycles-pp.page_swapcount
      0.49 ±  3%      +0.2        0.67 ± 14%  perf-profile.children.cycles-pp.try_to_free_swap
      0.82 ±  4%      +0.2        1.01 ± 11%  perf-profile.children.cycles-pp.put_swap_page
      0.55 ±  9%      +0.2        0.79 ± 14%  perf-profile.children.cycles-pp.__frontswap_store
      0.81 ±  6%      +0.3        1.07 ± 13%  perf-profile.children.cycles-pp.get_swap_page
      0.62 ±  7%      +0.3        0.89 ± 17%  perf-profile.children.cycles-pp.page_swap_info
      0.58 ±  8%      +0.3        0.85 ± 17%  perf-profile.children.cycles-pp.swap_set_page_dirty
      2.18 ±  8%      +0.4        2.56 ± 10%  perf-profile.children.cycles-pp.end_page_writeback
      1.05 ±  3%      +0.4        1.45 ± 11%  perf-profile.children.cycles-pp._swap_info_get
      1.04 ±  5%      +0.4        1.44 ± 11%  perf-profile.children.cycles-pp.__test_set_page_writeback
      1.07 ±  6%      +0.4        1.48 ± 13%  perf-profile.children.cycles-pp.swap_writepage
      2.17 ±  5%      +0.4        2.61 ± 11%  perf-profile.children.cycles-pp.try_to_unmap_one
      2.23 ±  5%      +0.5        2.71 ± 15%  perf-profile.children.cycles-pp.__sysvec_call_function_single
      2.52 ±  6%      +0.5        3.05 ± 15%  perf-profile.children.cycles-pp.sysvec_call_function_single
      1.30 ±  4%      +0.5        1.83 ± 14%  perf-profile.children.cycles-pp.page_counter_uncharge
      1.29 ±  4%      +0.5        1.82 ± 14%  perf-profile.children.cycles-pp.page_counter_cancel
      2.56 ±  5%      +0.9        3.46 ± 14%  perf-profile.children.cycles-pp.mem_cgroup_swapout
      3.38 ± 13%      -1.2        2.17 ± 12%  perf-profile.self.cycles-pp.clear_page_erms
      0.77 ± 13%      -0.2        0.59 ±  5%  perf-profile.self.cycles-pp.__count_memcg_events
      0.22 ±  8%      -0.1        0.08 ± 15%  perf-profile.self.cycles-pp.xas_store
      0.32 ± 15%      -0.1        0.18 ± 12%  perf-profile.self.cycles-pp.try_charge
      0.34 ±  9%      -0.1        0.22 ±  9%  perf-profile.self.cycles-pp.mem_cgroup_charge
      0.35 ±  7%      -0.1        0.24 ± 17%  perf-profile.self.cycles-pp.xas_create
      0.11 ± 10%      -0.1        0.04 ± 58%  perf-profile.self.cycles-pp.smp_call_function_many_cond
      0.18 ±  4%      -0.0        0.13 ± 16%  perf-profile.self.cycles-pp.rmqueue_bulk
      0.13 ± 13%      -0.0        0.09 ± 13%  perf-profile.self.cycles-pp.rmqueue
      0.12 ± 15%      -0.0        0.08 ± 14%  perf-profile.self.cycles-pp.page_remove_rmap
      0.15 ± 15%      -0.0        0.12 ± 21%  perf-profile.self.cycles-pp.prep_new_page
      0.09 ± 13%      +0.0        0.12 ±  3%  perf-profile.self.cycles-pp.inc_node_page_state
      0.05 ± 58%      +0.0        0.08 ±  8%  perf-profile.self.cycles-pp.scan_swap_map_slots
      0.01 ±173%      +0.0        0.05 ±  9%  perf-profile.self.cycles-pp.dec_zone_page_state
      0.01 ±173%      +0.0        0.05 ±  9%  perf-profile.self.cycles-pp.mutex_unlock
      0.01 ±173%      +0.0        0.06 ± 20%  perf-profile.self.cycles-pp.irqtime_account_process_tick
      0.13 ± 11%      +0.1        0.19 ± 16%  perf-profile.self.cycles-pp.mem_cgroup_swapout
      0.08 ± 15%      +0.1        0.15 ± 14%  perf-profile.self.cycles-pp.arch_tlbbatch_flush
      0.15 ± 11%      +0.1        0.21 ± 10%  perf-profile.self.cycles-pp.lock_page_memcg
      0.37 ±  7%      +0.1        0.46 ± 16%  perf-profile.self.cycles-pp.llist_reverse_order
      0.18 ± 15%      +0.1        0.28 ± 13%  perf-profile.self.cycles-pp.__frontswap_store
      0.57            +0.1        0.67 ± 11%  perf-profile.self.cycles-pp.__delete_from_swap_cache
      0.60 ±  5%      +0.1        0.70 ± 12%  perf-profile.self.cycles-pp.add_to_swap_cache
      0.33 ±  7%      +0.1        0.45 ± 13%  perf-profile.self.cycles-pp.zswap_frontswap_store
      0.45 ±  2%      +0.1        0.57 ± 14%  perf-profile.self.cycles-pp.flush_tlb_func_common
      0.38 ±  7%      +0.1        0.52 ± 13%  perf-profile.self.cycles-pp.get_swap_page
      0.74 ±  3%      +0.2        0.91 ± 10%  perf-profile.self.cycles-pp.try_to_unmap_one
      0.57 ±  9%      +0.2        0.81 ± 16%  perf-profile.self.cycles-pp.page_swap_info
      0.72 ±  3%      +0.3        1.03 ± 11%  perf-profile.self.cycles-pp.__test_set_page_writeback
      1.68 ±  6%      +0.3        2.00 ±  9%  perf-profile.self.cycles-pp.end_page_writeback
      0.98 ±  3%      +0.4        1.35 ± 12%  perf-profile.self.cycles-pp._swap_info_get
      1.22 ±  5%      +0.5        1.75 ± 14%  perf-profile.self.cycles-pp.page_counter_cancel
     52345 ± 17%     -84.0%       8384 ± 41%  softirqs.CPU0.RCU
     19063 ±  7%     -20.7%      15125 ±  6%  softirqs.CPU0.SCHED
     36466 ± 32%     -86.4%       4972 ± 11%  softirqs.CPU1.RCU
     36608 ± 40%     -87.4%       4628 ± 19%  softirqs.CPU10.RCU
     26212 ± 19%     -86.4%       3553 ±  2%  softirqs.CPU100.RCU
     27448 ± 17%     -86.5%       3706 ±  6%  softirqs.CPU101.RCU
     27842 ± 29%     -87.2%       3562 ±  4%  softirqs.CPU102.RCU
     26457 ± 29%     -87.1%       3409 ±  4%  softirqs.CPU103.RCU
     27085 ± 19%     -86.7%       3602 ±  6%  softirqs.CPU104.RCU
     27395 ± 20%     -86.8%       3611 ±  8%  softirqs.CPU105.RCU
     25853 ± 23%     -86.6%       3457 ±  5%  softirqs.CPU106.RCU
     27375 ± 24%     -86.5%       3696 ±  6%  softirqs.CPU107.RCU
     41902 ± 27%     -90.1%       4168 ± 47%  softirqs.CPU108.RCU
     18997 ±  7%     -15.2%      16115 ±  3%  softirqs.CPU108.SCHED
     35065 ± 28%     -91.6%       2936 ± 11%  softirqs.CPU109.RCU
     35638 ± 34%     -85.2%       5271 ± 41%  softirqs.CPU11.RCU
     34068 ± 26%     -91.6%       2847 ± 12%  softirqs.CPU110.RCU
     19277 ±  6%     -18.1%      15790 ±  9%  softirqs.CPU110.SCHED
     34270 ± 24%     -89.9%       3470 ± 31%  softirqs.CPU111.RCU
     34839 ± 33%     -90.4%       3339 ±  3%  softirqs.CPU112.RCU
     38104 ± 27%     -91.3%       3308 ±  5%  softirqs.CPU113.RCU
     32552 ± 31%     -90.9%       2966 ±  8%  softirqs.CPU114.RCU
     33623 ± 26%     -90.2%       3303 ±  7%  softirqs.CPU115.RCU
     35550 ± 27%     -90.4%       3417 ±  8%  softirqs.CPU116.RCU
     33356 ± 27%     -89.8%       3415 ±  5%  softirqs.CPU117.RCU
     33993 ± 32%     -90.0%       3405 ± 12%  softirqs.CPU118.RCU
     36253 ± 26%     -90.3%       3519 ±  3%  softirqs.CPU119.RCU
     34824 ± 31%     -87.1%       4494 ±  9%  softirqs.CPU12.RCU
     32768 ± 39%     -88.8%       3661 ± 11%  softirqs.CPU120.RCU
     36094 ± 23%     -90.6%       3401 ±  3%  softirqs.CPU121.RCU
     34497 ± 31%     -89.0%       3807 ± 12%  softirqs.CPU122.RCU
     36262 ± 25%     -88.4%       4193 ± 32%  softirqs.CPU123.RCU
     34252 ± 25%     -90.1%       3394 ±  5%  softirqs.CPU124.RCU
     36674 ± 21%     -88.3%       4301 ± 25%  softirqs.CPU125.RCU
     27951 ±  7%     -88.9%       3101 ± 10%  softirqs.CPU126.RCU
     18871 ±  7%     -14.7%      16104 ±  3%  softirqs.CPU126.SCHED
     23136 ± 14%     -88.8%       2600 ±  4%  softirqs.CPU127.RCU
     19431 ±  5%     -22.5%      15053 ± 14%  softirqs.CPU127.SCHED
     27391 ±  5%     -88.3%       3205        softirqs.CPU128.RCU
     25272 ± 14%     -86.9%       3319 ±  4%  softirqs.CPU129.RCU
     35293 ± 39%     -88.5%       4050 ± 11%  softirqs.CPU13.RCU
     26272 ± 10%     -88.4%       3041 ± 10%  softirqs.CPU130.RCU
     25115 ±  5%     -86.9%       3291 ±  3%  softirqs.CPU131.RCU
     23529 ±  6%     -86.9%       3071 ± 10%  softirqs.CPU132.RCU
     27781 ± 13%     -86.5%       3755 ± 15%  softirqs.CPU133.RCU
     26065 ± 16%     -87.4%       3289 ± 10%  softirqs.CPU134.RCU
     23998 ± 10%     -86.3%       3277 ±  3%  softirqs.CPU135.RCU
     26249 ± 18%     -87.3%       3328 ±  5%  softirqs.CPU136.RCU
     21876 ± 13%     -82.2%       3887 ± 14%  softirqs.CPU137.RCU
     22414 ± 10%     -82.1%       4001 ± 18%  softirqs.CPU138.RCU
     24824 ±  5%     -85.5%       3595 ±  9%  softirqs.CPU139.RCU
     34692 ± 44%     -88.7%       3921 ±  6%  softirqs.CPU14.RCU
     22513 ± 12%     -84.8%       3418 ±  7%  softirqs.CPU140.RCU
     23522 ±  5%     -85.0%       3517 ±  7%  softirqs.CPU141.RCU
     22534 ± 11%     -84.6%       3469 ±  8%  softirqs.CPU142.RCU
     25471 ± 22%     -84.8%       3869 ± 18%  softirqs.CPU143.RCU
     33424 ± 43%     -87.6%       4133 ±  9%  softirqs.CPU15.RCU
     32536 ± 25%     -89.7%       3340 ±  4%  softirqs.CPU16.RCU
     31138 ± 29%     -88.7%       3526 ±  7%  softirqs.CPU17.RCU
     51275 ± 11%     -92.3%       3956 ± 11%  softirqs.CPU18.RCU
     16486 ±  6%     -21.1%      13010 ± 12%  softirqs.CPU18.SCHED
     37422 ± 27%     -86.3%       5144 ± 42%  softirqs.CPU19.RCU
     18953 ±  6%     -29.2%      13419 ± 20%  softirqs.CPU19.SCHED
     38774 ± 41%     -88.8%       4336 ± 11%  softirqs.CPU2.RCU
     19636 ±  2%     -19.1%      15894 ±  7%  softirqs.CPU2.SCHED
     37496 ± 28%     -90.0%       3733 ±  8%  softirqs.CPU20.RCU
     19096 ±  5%     -19.5%      15372 ± 19%  softirqs.CPU20.SCHED
     36543 ± 22%     -89.9%       3688 ±  5%  softirqs.CPU21.RCU
     33600 ± 22%     -88.2%       3950 ± 16%  softirqs.CPU22.RCU
     38695 ± 21%     -90.1%       3830 ±  4%  softirqs.CPU23.RCU
     35007 ± 22%     -89.8%       3559 ±  7%  softirqs.CPU24.RCU
     19064 ±  4%     -12.9%      16604 ±  8%  softirqs.CPU24.SCHED
     37485 ± 21%     -90.1%       3728 ±  6%  softirqs.CPU25.RCU
     33627 ± 28%     -88.8%       3763 ±  7%  softirqs.CPU26.RCU
     36026 ± 24%     -89.4%       3829 ±  3%  softirqs.CPU27.RCU
     34690 ± 29%     -88.4%       4028 ±  5%  softirqs.CPU28.RCU
     36943 ± 24%     -89.5%       3882 ±  3%  softirqs.CPU29.RCU
     38484 ± 41%     -85.7%       5499 ± 39%  softirqs.CPU3.RCU
     34943 ± 26%     -89.3%       3743 ±  3%  softirqs.CPU30.RCU
     34022 ± 31%     -88.5%       3927 ±  5%  softirqs.CPU31.RCU
     34577 ± 26%     -90.2%       3392 ±  7%  softirqs.CPU32.RCU
     34324 ± 39%     -90.2%       3379 ±  4%  softirqs.CPU33.RCU
     32528 ± 21%     -89.1%       3560 ± 11%  softirqs.CPU34.RCU
     35054 ± 38%     -86.3%       4802 ± 28%  softirqs.CPU35.RCU
     55830 ± 22%     -91.4%       4827 ± 19%  softirqs.CPU36.RCU
     18131 ±  5%     -32.8%      12178 ± 16%  softirqs.CPU36.SCHED
     44535 ± 25%     -91.5%       3800 ±  6%  softirqs.CPU37.RCU
     18780 ±  6%     -17.0%      15579 ± 12%  softirqs.CPU37.SCHED
     46422 ± 32%     -90.5%       4423 ± 17%  softirqs.CPU38.RCU
     18645 ±  6%     -27.3%      13557 ± 20%  softirqs.CPU38.SCHED
     43345 ± 22%     -88.7%       4918 ± 36%  softirqs.CPU39.RCU
     32960 ± 30%     -88.1%       3917 ±  4%  softirqs.CPU4.RCU
     44490 ± 29%     -91.7%       3685 ±  2%  softirqs.CPU40.RCU
     46120 ± 29%     -91.8%       3798 ±  2%  softirqs.CPU41.RCU
     42257 ± 32%     -91.6%       3551 ±  4%  softirqs.CPU42.RCU
     46039 ± 20%     -90.9%       4188 ± 21%  softirqs.CPU43.RCU
     44088 ± 29%     -91.8%       3604        softirqs.CPU44.RCU
     41672 ± 31%     -91.2%       3647        softirqs.CPU45.RCU
     44499 ± 25%     -91.9%       3597 ±  8%  softirqs.CPU46.RCU
     47826 ± 27%     -92.4%       3616        softirqs.CPU47.RCU
     40295 ± 41%     -91.3%       3518        softirqs.CPU48.RCU
     41600 ± 24%     -91.6%       3497        softirqs.CPU49.RCU
     36523 ± 32%     -88.8%       4102 ±  6%  softirqs.CPU5.RCU
     43678 ± 34%     -92.3%       3347 ±  4%  softirqs.CPU50.RCU
     44579 ± 28%     -90.7%       4125 ± 15%  softirqs.CPU51.RCU
     43503 ± 28%     -92.2%       3396 ±  3%  softirqs.CPU52.RCU
     44468 ± 18%     -90.5%       4219 ± 33%  softirqs.CPU53.RCU
     49302 ±  5%     -90.4%       4747 ± 22%  softirqs.CPU54.RCU
     16826 ±  6%     -23.7%      12844 ± 10%  softirqs.CPU54.SCHED
     34574 ± 13%     -87.8%       4220 ± 18%  softirqs.CPU55.RCU
     18223 ±  8%     -35.1%      11830 ± 25%  softirqs.CPU55.SCHED
     34444 ± 10%     -88.1%       4102 ± 21%  softirqs.CPU56.RCU
     29792 ± 13%     -87.5%       3728 ± 11%  softirqs.CPU57.RCU
     29979 ± 15%     -86.7%       3997 ± 16%  softirqs.CPU58.RCU
     29522 ± 13%     -87.5%       3687 ±  8%  softirqs.CPU59.RCU
     34227 ± 39%     -88.9%       3815 ±  6%  softirqs.CPU6.RCU
     19618 ±  4%     -10.2%      17615 ±  4%  softirqs.CPU6.SCHED
     29331 ± 13%     -88.2%       3454 ±  8%  softirqs.CPU60.RCU
     30844 ± 15%     -88.5%       3555 ±  4%  softirqs.CPU61.RCU
     30676 ± 16%     -87.1%       3947 ± 18%  softirqs.CPU62.RCU
     31209 ± 20%     -89.1%       3399 ±  8%  softirqs.CPU63.RCU
     32203 ± 16%     -89.0%       3550 ± 12%  softirqs.CPU64.RCU
     26618 ± 18%     -86.7%       3543 ±  4%  softirqs.CPU65.RCU
     28476 ± 20%     -86.5%       3856 ± 11%  softirqs.CPU66.RCU
     29713 ± 12%     -87.1%       3842 ± 14%  softirqs.CPU67.RCU
     28361 ± 25%     -87.2%       3628 ±  6%  softirqs.CPU68.RCU
     30363 ± 18%     -87.3%       3850 ±  9%  softirqs.CPU69.RCU
     40268 ± 49%     -90.1%       3997 ±  6%  softirqs.CPU7.RCU
     29067 ± 11%     -83.9%       4679 ± 38%  softirqs.CPU70.RCU
     30143 ± 14%     -87.5%       3758 ±  4%  softirqs.CPU71.RCU
     33102 ± 37%     -90.7%       3088 ±  6%  softirqs.CPU72.RCU
     29408 ± 29%     -88.2%       3474 ±  5%  softirqs.CPU73.RCU
     19427           -18.5%      15841 ± 10%  softirqs.CPU73.SCHED
     31208 ± 34%     -89.4%       3312        softirqs.CPU74.RCU
     19059 ±  2%     -12.2%      16733 ±  7%  softirqs.CPU74.SCHED
     29231 ± 42%     -88.2%       3446 ±  2%  softirqs.CPU75.RCU
     27628 ± 26%     -86.2%       3814 ± 21%  softirqs.CPU76.RCU
     26377 ± 27%     -87.0%       3428 ±  2%  softirqs.CPU77.RCU
     26349 ± 50%     -86.3%       3622 ± 18%  softirqs.CPU78.RCU
     30531 ± 48%     -88.9%       3389 ±  7%  softirqs.CPU79.RCU
     35512 ± 42%     -88.0%       4272 ±  4%  softirqs.CPU8.RCU
     25522 ± 36%     -86.9%       3339 ± 16%  softirqs.CPU80.RCU
     27427 ± 48%     -86.2%       3784 ± 16%  softirqs.CPU81.RCU
     25383 ± 40%     -82.8%       4368 ± 33%  softirqs.CPU82.RCU
     26528 ± 40%     -84.2%       4180 ± 15%  softirqs.CPU83.RCU
     26637 ± 33%     -85.5%       3849 ±  9%  softirqs.CPU84.RCU
     27814 ± 30%     -86.6%       3735 ±  8%  softirqs.CPU85.RCU
     27615 ± 39%     -82.7%       4764 ± 19%  softirqs.CPU86.RCU
     27935 ± 40%     -85.1%       4154 ± 14%  softirqs.CPU87.RCU
     27323 ± 26%     -84.0%       4380 ± 15%  softirqs.CPU88.RCU
     26788 ± 28%     -85.9%       3788 ±  2%  softirqs.CPU89.RCU
     34530 ± 43%     -88.9%       3845 ± 11%  softirqs.CPU9.RCU
     34234 ± 12%     -90.2%       3338        softirqs.CPU90.RCU
     19353 ±  3%     -24.2%      14671 ±  2%  softirqs.CPU90.SCHED
     27937 ± 28%     -85.9%       3926 ± 15%  softirqs.CPU91.RCU
     19542 ±  6%     -15.2%      16566 ±  3%  softirqs.CPU91.SCHED
     28497 ± 22%     -87.9%       3441 ± 14%  softirqs.CPU92.RCU
     28320 ± 18%     -89.1%       3074 ±  4%  softirqs.CPU93.RCU
     26306 ± 11%     -85.4%       3840 ± 27%  softirqs.CPU94.RCU
     19738 ±  4%     -12.9%      17184 ±  9%  softirqs.CPU94.SCHED
     26514 ±  8%     -87.0%       3439 ± 23%  softirqs.CPU95.RCU
     25298 ± 14%     -86.9%       3318 ± 10%  softirqs.CPU96.RCU
     19776 ±  3%     -13.9%      17031 ±  6%  softirqs.CPU96.SCHED
     27549 ± 12%     -87.5%       3438 ±  6%  softirqs.CPU97.RCU
     28550 ± 25%     -88.7%       3235 ±  9%  softirqs.CPU98.RCU
     26684 ± 19%     -87.0%       3473 ±  6%  softirqs.CPU99.RCU
   4756845 ±  5%     -88.5%     544700        softirqs.RCU
     30919 ±  3%     -13.8%      26641 ±  2%  softirqs.TIMER
    145.50 ±  2%     -56.0%      64.00 ±100%  interrupts.175:PCI-MSI.512000-edge.ahci[0000:00:1f.2]
    112.25 ±  3%      -8.7%     102.50 ±  4%  interrupts.45:PCI-MSI.1572864-edge.eth0-TxRx-0
  78296627            +5.0%   82208891        interrupts.CAL:Function_call_interrupts
    112.25 ±  3%      -8.7%     102.50 ±  4%  interrupts.CPU0.45:PCI-MSI.1572864-edge.eth0-TxRx-0
   1231976 ± 21%    +185.1%    3511906 ± 22%  interrupts.CPU0.CAL:Function_call_interrupts
   1343291 ± 20%    +181.1%    3775855 ± 22%  interrupts.CPU0.TLB:TLB_shootdowns
    608271 ± 75%     -99.3%       4379 ±170%  interrupts.CPU10.CAL:Function_call_interrupts
    457.50 ± 61%     -79.9%      91.75 ± 38%  interrupts.CPU10.NMI:Non-maskable_interrupts
    457.50 ± 61%     -79.9%      91.75 ± 38%  interrupts.CPU10.PMI:Performance_monitoring_interrupts
     89.00 ± 59%     -89.9%       9.00 ± 79%  interrupts.CPU10.RES:Rescheduling_interrupts
    657079 ± 75%     -99.3%       4336 ±173%  interrupts.CPU10.TLB:TLB_shootdowns
    411365 ± 29%     -99.7%       1267 ± 64%  interrupts.CPU100.CAL:Function_call_interrupts
    444274 ± 30%     -99.7%       1249 ± 69%  interrupts.CPU100.TLB:TLB_shootdowns
    659.50 ± 68%     -74.6%     167.50 ± 22%  interrupts.CPU102.NMI:Non-maskable_interrupts
    659.50 ± 68%     -74.6%     167.50 ± 22%  interrupts.CPU102.PMI:Performance_monitoring_interrupts
    302129 ± 60%     -99.9%     166.75 ± 59%  interrupts.CPU103.CAL:Function_call_interrupts
    323.75 ± 53%     -56.4%     141.00 ± 34%  interrupts.CPU103.NMI:Non-maskable_interrupts
    323.75 ± 53%     -56.4%     141.00 ± 34%  interrupts.CPU103.PMI:Performance_monitoring_interrupts
    327333 ± 60%    -100.0%      32.50 ±166%  interrupts.CPU103.TLB:TLB_shootdowns
    312284 ± 48%     -53.9%     144043 ± 50%  interrupts.CPU104.CAL:Function_call_interrupts
    338648 ± 48%     -54.3%     154674 ± 50%  interrupts.CPU104.TLB:TLB_shootdowns
    115.50 ± 72%     -81.0%      22.00 ± 19%  interrupts.CPU108.RES:Rescheduling_interrupts
    626496 ± 83%     -90.8%      57927 ±172%  interrupts.CPU11.CAL:Function_call_interrupts
     78.00 ± 60%     -73.1%      21.00 ± 86%  interrupts.CPU11.RES:Rescheduling_interrupts
    679997 ± 83%     -90.5%      64580 ±173%  interrupts.CPU11.TLB:TLB_shootdowns
     81.25 ± 51%     -80.3%      16.00 ± 11%  interrupts.CPU111.RES:Rescheduling_interrupts
     87.50 ± 71%     -89.7%       9.00 ± 40%  interrupts.CPU112.RES:Rescheduling_interrupts
    645406 ± 66%     -89.8%      66133 ±103%  interrupts.CPU113.CAL:Function_call_interrupts
    103.50 ± 48%     -89.6%      10.75 ± 47%  interrupts.CPU113.RES:Rescheduling_interrupts
    698561 ± 66%     -89.8%      71091 ±104%  interrupts.CPU113.TLB:TLB_shootdowns
     78.00 ± 68%     -86.2%      10.75 ± 41%  interrupts.CPU114.RES:Rescheduling_interrupts
      1141 ± 56%     -72.3%     316.50 ±106%  interrupts.CPU115.NMI:Non-maskable_interrupts
      1141 ± 56%     -72.3%     316.50 ±106%  interrupts.CPU115.PMI:Performance_monitoring_interrupts
     91.00 ± 62%     -90.1%       9.00 ± 26%  interrupts.CPU115.RES:Rescheduling_interrupts
    757124 ± 58%     -60.1%     302332 ±132%  interrupts.CPU116.CAL:Function_call_interrupts
     98.25 ± 46%     -88.0%      11.75 ± 63%  interrupts.CPU116.RES:Rescheduling_interrupts
    823491 ± 58%     -60.9%     322256 ±132%  interrupts.CPU116.TLB:TLB_shootdowns
    628006 ± 76%     -73.8%     164708 ±146%  interrupts.CPU117.CAL:Function_call_interrupts
    987.50 ± 49%     -85.9%     139.25 ± 30%  interrupts.CPU117.NMI:Non-maskable_interrupts
    987.50 ± 49%     -85.9%     139.25 ± 30%  interrupts.CPU117.PMI:Performance_monitoring_interrupts
    109.25 ± 69%     -93.1%       7.50 ± 52%  interrupts.CPU117.RES:Rescheduling_interrupts
    681610 ± 76%     -74.4%     174707 ±146%  interrupts.CPU117.TLB:TLB_shootdowns
    104.00 ± 55%     -87.7%      12.75 ± 78%  interrupts.CPU119.RES:Rescheduling_interrupts
    471.25 ± 54%     -80.9%      90.00 ± 38%  interrupts.CPU12.NMI:Non-maskable_interrupts
    471.25 ± 54%     -80.9%      90.00 ± 38%  interrupts.CPU12.PMI:Performance_monitoring_interrupts
     58.25 ± 79%     -76.4%      13.75 ± 88%  interrupts.CPU12.RES:Rescheduling_interrupts
    878.50 ± 72%     -88.3%     103.00 ± 43%  interrupts.CPU120.NMI:Non-maskable_interrupts
    878.50 ± 72%     -88.3%     103.00 ± 43%  interrupts.CPU120.PMI:Performance_monitoring_interrupts
     64.00 ± 77%     -85.5%       9.25 ± 68%  interrupts.CPU120.RES:Rescheduling_interrupts
    868.50 ± 54%     -84.1%     138.50 ± 45%  interrupts.CPU121.NMI:Non-maskable_interrupts
    868.50 ± 54%     -84.1%     138.50 ± 45%  interrupts.CPU121.PMI:Performance_monitoring_interrupts
    597802 ± 74%     -99.8%       1291 ±164%  interrupts.CPU122.CAL:Function_call_interrupts
    804.25 ± 83%     -85.9%     113.25 ± 65%  interrupts.CPU122.NMI:Non-maskable_interrupts
    804.25 ± 83%     -85.9%     113.25 ± 65%  interrupts.CPU122.PMI:Performance_monitoring_interrupts
     95.50 ± 48%     -92.7%       7.00 ± 56%  interrupts.CPU122.RES:Rescheduling_interrupts
    649164 ± 75%     -99.8%       1290 ±173%  interrupts.CPU122.TLB:TLB_shootdowns
    114.50 ± 54%     -93.7%       7.25 ± 48%  interrupts.CPU123.RES:Rescheduling_interrupts
    629610 ± 63%     -69.6%     191537 ±168%  interrupts.CPU124.CAL:Function_call_interrupts
      1011 ± 58%     -86.9%     132.50 ± 45%  interrupts.CPU124.NMI:Non-maskable_interrupts
      1011 ± 58%     -86.9%     132.50 ± 45%  interrupts.CPU124.PMI:Performance_monitoring_interrupts
     70.25 ± 70%     -85.4%      10.25 ± 44%  interrupts.CPU124.RES:Rescheduling_interrupts
    683705 ± 63%     -69.7%     206886 ±168%  interrupts.CPU124.TLB:TLB_shootdowns
    739662 ± 49%     -82.9%     126387 ±117%  interrupts.CPU125.CAL:Function_call_interrupts
    968.50 ± 63%     -84.4%     151.00 ± 27%  interrupts.CPU125.NMI:Non-maskable_interrupts
    968.50 ± 63%     -84.4%     151.00 ± 27%  interrupts.CPU125.PMI:Performance_monitoring_interrupts
     66.25 ± 66%     -72.8%      18.00 ± 97%  interrupts.CPU125.RES:Rescheduling_interrupts
    804430 ± 50%     -83.3%     134653 ±118%  interrupts.CPU125.TLB:TLB_shootdowns
    501165 ± 14%    +221.8%    1612759 ± 13%  interrupts.CPU126.CAL:Function_call_interrupts
    545091 ± 14%    +217.3%    1729798 ± 13%  interrupts.CPU126.TLB:TLB_shootdowns
    303.75 ± 35%   +1609.7%       5193 ± 40%  interrupts.CPU127.NMI:Non-maskable_interrupts
    303.75 ± 35%   +1609.7%       5193 ± 40%  interrupts.CPU127.PMI:Performance_monitoring_interrupts
     81.00 ± 59%     -78.1%      17.75 ± 68%  interrupts.CPU13.RES:Rescheduling_interrupts
    221283 ± 18%     -61.8%      84423 ± 90%  interrupts.CPU132.CAL:Function_call_interrupts
    240847 ± 18%     -62.5%      90210 ± 89%  interrupts.CPU132.TLB:TLB_shootdowns
    228853 ± 19%     -64.6%      81121 ±115%  interrupts.CPU135.CAL:Function_call_interrupts
    514.00 ± 87%     -74.7%     130.25 ± 15%  interrupts.CPU135.NMI:Non-maskable_interrupts
    514.00 ± 87%     -74.7%     130.25 ± 15%  interrupts.CPU135.PMI:Performance_monitoring_interrupts
    250600 ± 19%     -65.4%      86702 ±116%  interrupts.CPU135.TLB:TLB_shootdowns
    319279 ± 47%     -92.1%      25144 ±150%  interrupts.CPU136.CAL:Function_call_interrupts
    343.25 ± 76%     -63.6%     125.00 ± 25%  interrupts.CPU136.NMI:Non-maskable_interrupts
    343.25 ± 76%     -63.6%     125.00 ± 25%  interrupts.CPU136.PMI:Performance_monitoring_interrupts
    349192 ± 47%     -92.4%      26548 ±150%  interrupts.CPU136.TLB:TLB_shootdowns
    270185 ± 37%     -78.4%      58300 ±172%  interrupts.CPU139.CAL:Function_call_interrupts
    378.00 ± 56%     -71.6%     107.25 ± 27%  interrupts.CPU139.NMI:Non-maskable_interrupts
    378.00 ± 56%     -71.6%     107.25 ± 27%  interrupts.CPU139.PMI:Performance_monitoring_interrupts
    293913 ± 37%     -78.9%      62022 ±173%  interrupts.CPU139.TLB:TLB_shootdowns
     78.00 ± 63%     -76.3%      18.50 ±111%  interrupts.CPU14.RES:Rescheduling_interrupts
    162067 ± 56%     -81.2%      30462 ±165%  interrupts.CPU140.CAL:Function_call_interrupts
    175459 ± 56%     -82.0%      31590 ±166%  interrupts.CPU140.TLB:TLB_shootdowns
    253.25 ± 32%     -50.4%     125.50 ± 35%  interrupts.CPU143.NMI:Non-maskable_interrupts
    253.25 ± 32%     -50.4%     125.50 ± 35%  interrupts.CPU143.PMI:Performance_monitoring_interrupts
     66.75 ± 72%     -82.4%      11.75 ± 49%  interrupts.CPU15.RES:Rescheduling_interrupts
    508268 ± 63%    -100.0%      79.25 ± 22%  interrupts.CPU16.CAL:Function_call_interrupts
      1082 ± 98%     -93.1%      75.25 ± 11%  interrupts.CPU16.NMI:Non-maskable_interrupts
      1082 ± 98%     -93.1%      75.25 ± 11%  interrupts.CPU16.PMI:Performance_monitoring_interrupts
     58.50 ± 46%     -86.3%       8.00 ± 81%  interrupts.CPU16.RES:Rescheduling_interrupts
    549265 ± 63%    -100.0%       4.00 ±133%  interrupts.CPU16.TLB:TLB_shootdowns
     52.25 ± 65%     -66.5%      17.50 ± 57%  interrupts.CPU17.RES:Rescheduling_interrupts
   1258584 ± 15%    +208.1%    3878023 ± 32%  interrupts.CPU18.CAL:Function_call_interrupts
   1369478 ± 15%    +204.9%    4175783 ± 31%  interrupts.CPU18.TLB:TLB_shootdowns
    629511 ± 88%    +241.1%    2147207 ± 76%  interrupts.CPU2.CAL:Function_call_interrupts
    678734 ± 89%    +237.7%    2292112 ± 75%  interrupts.CPU2.TLB:TLB_shootdowns
    620.75 ± 72%     -82.7%     107.50 ± 33%  interrupts.CPU21.NMI:Non-maskable_interrupts
    620.75 ± 72%     -82.7%     107.50 ± 33%  interrupts.CPU21.PMI:Performance_monitoring_interrupts
     50.50 ± 71%     -71.8%      14.25 ± 84%  interrupts.CPU21.RES:Rescheduling_interrupts
     63.75 ± 43%     -78.8%      13.50 ± 99%  interrupts.CPU22.RES:Rescheduling_interrupts
     66.75 ± 50%     -80.5%      13.00 ± 88%  interrupts.CPU23.RES:Rescheduling_interrupts
     54.25 ± 62%     -77.0%      12.50 ± 17%  interrupts.CPU25.RES:Rescheduling_interrupts
    323.25 ± 93%     -74.2%      83.25 ±  8%  interrupts.CPU28.NMI:Non-maskable_interrupts
    323.25 ± 93%     -74.2%      83.25 ±  8%  interrupts.CPU28.PMI:Performance_monitoring_interrupts
     71.75 ± 37%     -83.6%      11.75 ±109%  interrupts.CPU29.RES:Rescheduling_interrupts
    657.25 ± 54%     -83.5%     108.50 ± 30%  interrupts.CPU3.NMI:Non-maskable_interrupts
    657.25 ± 54%     -83.5%     108.50 ± 30%  interrupts.CPU3.PMI:Performance_monitoring_interrupts
     81.75 ± 86%     -78.0%      18.00 ± 42%  interrupts.CPU3.RES:Rescheduling_interrupts
    846.25 ± 65%     -88.7%      96.00 ± 15%  interrupts.CPU30.NMI:Non-maskable_interrupts
    846.25 ± 65%     -88.7%      96.00 ± 15%  interrupts.CPU30.PMI:Performance_monitoring_interrupts
     54.50 ± 54%     -85.3%       8.00 ± 49%  interrupts.CPU30.RES:Rescheduling_interrupts
    545049 ± 65%     -98.4%       8859 ±171%  interrupts.CPU31.CAL:Function_call_interrupts
    612.00 ± 63%     -84.8%      92.75 ± 14%  interrupts.CPU31.NMI:Non-maskable_interrupts
    612.00 ± 63%     -84.8%      92.75 ± 14%  interrupts.CPU31.PMI:Performance_monitoring_interrupts
     49.75 ± 83%     -78.4%      10.75 ± 54%  interrupts.CPU31.RES:Rescheduling_interrupts
    593689 ± 65%     -98.4%       9350 ±173%  interrupts.CPU31.TLB:TLB_shootdowns
     67.75 ± 84%     -85.6%       9.75 ± 42%  interrupts.CPU33.RES:Rescheduling_interrupts
    138.75 ± 65%     -78.7%      29.50 ± 25%  interrupts.CPU36.RES:Rescheduling_interrupts
    133.75 ± 81%     -84.1%      21.25 ± 49%  interrupts.CPU37.RES:Rescheduling_interrupts
    108.75 ± 54%     -82.3%      19.25 ± 45%  interrupts.CPU38.RES:Rescheduling_interrupts
    112.50 ± 75%     -93.8%       7.00 ± 42%  interrupts.CPU40.RES:Rescheduling_interrupts
    117.75 ± 69%     -89.0%      13.00 ± 44%  interrupts.CPU41.RES:Rescheduling_interrupts
    889287 ± 63%     -69.2%     273776 ±142%  interrupts.CPU42.CAL:Function_call_interrupts
    917.25 ± 65%     -84.8%     139.25 ± 48%  interrupts.CPU42.NMI:Non-maskable_interrupts
    917.25 ± 65%     -84.8%     139.25 ± 48%  interrupts.CPU42.PMI:Performance_monitoring_interrupts
    110.75 ± 72%     -91.0%      10.00 ± 50%  interrupts.CPU42.RES:Rescheduling_interrupts
    970285 ± 63%     -69.8%     292582 ±142%  interrupts.CPU42.TLB:TLB_shootdowns
    945929 ± 47%     -54.3%     432248 ±109%  interrupts.CPU43.CAL:Function_call_interrupts
      1431 ± 84%     -74.5%     365.50 ±134%  interrupts.CPU43.NMI:Non-maskable_interrupts
      1431 ± 84%     -74.5%     365.50 ±134%  interrupts.CPU43.PMI:Performance_monitoring_interrupts
     91.25 ± 46%     -93.7%       5.75 ± 44%  interrupts.CPU43.RES:Rescheduling_interrupts
   1030879 ± 47%     -54.7%     467326 ±108%  interrupts.CPU43.TLB:TLB_shootdowns
    120.25 ± 47%     -93.6%       7.75 ± 23%  interrupts.CPU44.RES:Rescheduling_interrupts
    869076 ± 61%     -79.1%     182037 ±126%  interrupts.CPU45.CAL:Function_call_interrupts
      1007 ± 52%     -88.8%     113.00 ± 61%  interrupts.CPU45.NMI:Non-maskable_interrupts
      1007 ± 52%     -88.8%     113.00 ± 61%  interrupts.CPU45.PMI:Performance_monitoring_interrupts
    119.00 ± 65%     -88.4%      13.75 ± 37%  interrupts.CPU45.RES:Rescheduling_interrupts
    945574 ± 61%     -79.6%     192547 ±127%  interrupts.CPU45.TLB:TLB_shootdowns
    121.00 ± 36%     -87.6%      15.00 ± 48%  interrupts.CPU46.RES:Rescheduling_interrupts
     96.50 ± 69%     -85.8%      13.75 ±116%  interrupts.CPU47.RES:Rescheduling_interrupts
     91.25 ± 70%     -91.2%       8.00 ± 31%  interrupts.CPU48.RES:Rescheduling_interrupts
    113.00 ± 73%     -93.1%       7.75 ±108%  interrupts.CPU49.RES:Rescheduling_interrupts
    471.25 ± 60%     -81.3%      88.25 ± 22%  interrupts.CPU5.NMI:Non-maskable_interrupts
    471.25 ± 60%     -81.3%      88.25 ± 22%  interrupts.CPU5.PMI:Performance_monitoring_interrupts
     72.25 ± 54%     -60.2%      28.75 ± 87%  interrupts.CPU5.RES:Rescheduling_interrupts
    959091 ± 62%     -94.9%      48533 ±172%  interrupts.CPU50.CAL:Function_call_interrupts
    137.75 ± 80%     -95.5%       6.25 ±  6%  interrupts.CPU50.RES:Rescheduling_interrupts
   1039830 ± 62%     -95.0%      52192 ±173%  interrupts.CPU50.TLB:TLB_shootdowns
      1333 ± 57%     -88.1%     158.50 ± 30%  interrupts.CPU51.NMI:Non-maskable_interrupts
      1333 ± 57%     -88.1%     158.50 ± 30%  interrupts.CPU51.PMI:Performance_monitoring_interrupts
    121.25 ± 51%     -89.5%      12.75 ± 45%  interrupts.CPU51.RES:Rescheduling_interrupts
    939728 ± 55%     -87.8%     114957 ±173%  interrupts.CPU52.CAL:Function_call_interrupts
    103.25 ± 63%     -94.2%       6.00 ± 39%  interrupts.CPU52.RES:Rescheduling_interrupts
   1020727 ± 55%     -88.1%     121660 ±173%  interrupts.CPU52.TLB:TLB_shootdowns
   1039918 ± 36%     -89.8%     106181 ±122%  interrupts.CPU53.CAL:Function_call_interrupts
      1284 ± 78%     -91.1%     114.00 ± 52%  interrupts.CPU53.NMI:Non-maskable_interrupts
      1284 ± 78%     -91.1%     114.00 ± 52%  interrupts.CPU53.PMI:Performance_monitoring_interrupts
    104.00 ± 68%     -94.7%       5.50 ± 27%  interrupts.CPU53.RES:Rescheduling_interrupts
   1134197 ± 36%     -90.1%     112045 ±121%  interrupts.CPU53.TLB:TLB_shootdowns
   1168628 ±  8%    +182.2%    3297943 ± 17%  interrupts.CPU54.CAL:Function_call_interrupts
    963.25 ± 47%    +350.1%       4336 ± 34%  interrupts.CPU54.NMI:Non-maskable_interrupts
    963.25 ± 47%    +350.1%       4336 ± 34%  interrupts.CPU54.PMI:Performance_monitoring_interrupts
     80.75 ± 40%     -48.3%      41.75 ± 25%  interrupts.CPU54.RES:Rescheduling_interrupts
   1275853 ±  8%    +177.1%    3535200 ± 18%  interrupts.CPU54.TLB:TLB_shootdowns
    392.25 ± 34%   +1015.9%       4377 ± 29%  interrupts.CPU55.NMI:Non-maskable_interrupts
    392.25 ± 34%   +1015.9%       4377 ± 29%  interrupts.CPU55.PMI:Performance_monitoring_interrupts
     84.75 ± 68%     -72.6%      23.25 ± 94%  interrupts.CPU6.RES:Rescheduling_interrupts
    707.00 ± 42%     -77.1%     162.25 ± 30%  interrupts.CPU61.NMI:Non-maskable_interrupts
    707.00 ± 42%     -77.1%     162.25 ± 30%  interrupts.CPU61.PMI:Performance_monitoring_interrupts
    405740 ± 55%     -76.0%      97283 ±115%  interrupts.CPU63.CAL:Function_call_interrupts
    684.50 ± 70%     -77.1%     157.00 ± 37%  interrupts.CPU63.NMI:Non-maskable_interrupts
    684.50 ± 70%     -77.1%     157.00 ± 37%  interrupts.CPU63.PMI:Performance_monitoring_interrupts
    440306 ± 55%     -76.6%     102844 ±115%  interrupts.CPU63.TLB:TLB_shootdowns
    315353 ± 24%     -62.0%     119836 ±106%  interrupts.CPU65.CAL:Function_call_interrupts
    340082 ± 24%     -62.6%     127245 ±106%  interrupts.CPU65.TLB:TLB_shootdowns
    356445 ± 33%     -88.5%      41068 ± 70%  interrupts.CPU66.CAL:Function_call_interrupts
    387163 ± 33%     -88.6%      44312 ± 71%  interrupts.CPU66.TLB:TLB_shootdowns
    390653 ± 33%     -91.5%      33124 ±172%  interrupts.CPU67.CAL:Function_call_interrupts
    461.25 ± 50%     -72.1%     128.75 ± 43%  interrupts.CPU67.NMI:Non-maskable_interrupts
    461.25 ± 50%     -72.1%     128.75 ± 43%  interrupts.CPU67.PMI:Performance_monitoring_interrupts
    424248 ± 33%     -91.6%      35682 ±173%  interrupts.CPU67.TLB:TLB_shootdowns
    362929 ± 57%     -67.3%     118702 ±125%  interrupts.CPU68.CAL:Function_call_interrupts
    393487 ± 57%     -68.0%     125761 ±125%  interrupts.CPU68.TLB:TLB_shootdowns
    396812 ± 29%     -91.7%      32913 ±168%  interrupts.CPU71.CAL:Function_call_interrupts
    407.00 ± 48%     -71.3%     117.00 ± 35%  interrupts.CPU71.NMI:Non-maskable_interrupts
    407.00 ± 48%     -71.3%     117.00 ± 35%  interrupts.CPU71.PMI:Performance_monitoring_interrupts
    432737 ± 29%     -91.1%      38455 ±169%  interrupts.CPU71.TLB:TLB_shootdowns
    617468 ± 68%    +224.7%    2004642 ± 58%  interrupts.CPU72.CAL:Function_call_interrupts
    670964 ± 68%    +220.4%    2149931 ± 58%  interrupts.CPU72.TLB:TLB_shootdowns
     62.50 ± 63%     -72.0%      17.50 ± 37%  interrupts.CPU75.RES:Rescheduling_interrupts
     58.50 ± 67%     -68.4%      18.50 ± 84%  interrupts.CPU77.RES:Rescheduling_interrupts
    584390 ± 73%     -92.3%      45050 ±111%  interrupts.CPU8.CAL:Function_call_interrupts
    498.50 ± 43%     -82.6%      86.50 ± 18%  interrupts.CPU8.NMI:Non-maskable_interrupts
    498.50 ± 43%     -82.6%      86.50 ± 18%  interrupts.CPU8.PMI:Performance_monitoring_interrupts
     71.25 ± 62%     -94.0%       4.25 ± 67%  interrupts.CPU8.RES:Rescheduling_interrupts
    634390 ± 73%     -92.5%      47709 ±111%  interrupts.CPU8.TLB:TLB_shootdowns
    305172 ±104%     -98.9%       3263 ±136%  interrupts.CPU80.CAL:Function_call_interrupts
    332295 ±104%     -99.0%       3256 ±144%  interrupts.CPU80.TLB:TLB_shootdowns
    306939 ± 97%    -100.0%      77.25 ± 23%  interrupts.CPU82.CAL:Function_call_interrupts
     45.50 ± 79%     -80.2%       9.00 ± 53%  interrupts.CPU82.RES:Rescheduling_interrupts
    332644 ± 98%    -100.0%       0.75 ±110%  interrupts.CPU82.TLB:TLB_shootdowns
    626.25 ± 82%     -78.1%     137.25 ± 44%  interrupts.CPU84.NMI:Non-maskable_interrupts
    626.25 ± 82%     -78.1%     137.25 ± 44%  interrupts.CPU84.PMI:Performance_monitoring_interrupts
    145.50 ±  2%     -56.0%      64.00 ±100%  interrupts.CPU87.175:PCI-MSI.512000-edge.ahci[0000:00:1f.2]
    330258 ±115%    -100.0%     114.75 ± 59%  interrupts.CPU88.CAL:Function_call_interrupts
    936.25 ±104%     -85.1%     139.50 ± 58%  interrupts.CPU88.NMI:Non-maskable_interrupts
    936.25 ±104%     -85.1%     139.50 ± 58%  interrupts.CPU88.PMI:Performance_monitoring_interrupts
    360088 ±116%    -100.0%      41.50 ±164%  interrupts.CPU88.TLB:TLB_shootdowns
     84.75 ± 77%     -69.0%      26.25 ± 97%  interrupts.CPU9.RES:Rescheduling_interrupts
    673363 ± 17%    +226.0%    2195380 ± 32%  interrupts.CPU90.CAL:Function_call_interrupts
    732427 ± 17%    +223.3%    2367615 ± 32%  interrupts.CPU90.TLB:TLB_shootdowns
      8947 ± 18%     -71.3%       2570 ± 19%  interrupts.RES:Rescheduling_interrupts


                                                                                
                               vm-scalability.throughput                        
                                                                                
  4.5e+06 +-----------------------------------------------------------------+   
  4.4e+06 |-+                                           O    O       O      |   
          |                                           O   O        O        |   
  4.3e+06 |-O O                                                  O        O |   
  4.2e+06 |-+    O O     O    O O O    O O   O    O O          O            |   
  4.1e+06 |-+        O      O       O      O   O                        O   |   
    4e+06 |-+          O                                                    |   
          |                                                                 |   
  3.9e+06 |-+                                                               |   
  3.8e+06 |-+                                                               |   
  3.7e+06 |-+                                                          .+.  |   
  3.6e+06 |.+.+..   .+.               .+. .+     .+. .+     .+. .+. .+.   + |   
          | +     .+   +.  .+.+.+. .+.   +  + .+.   +  + .+.   +   +       +|   
  3.5e+06 |-+    +       +.       +          +          +                   |   
  3.4e+06 +-----------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample



Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Rong Chen


View attachment "config-5.8.0-12310-gaae466b0052e1" of type "text/plain" (170150 bytes)

View attachment "job-script" of type "text/plain" (8644 bytes)

View attachment "job.yaml" of type "text/plain" (6056 bytes)

View attachment "reproduce" of type "text/plain" (1081 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ