lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <20200306025455.GF9382@xsang-OptiPlex-9020>
Date:   Fri, 6 Mar 2020 10:54:55 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     Vincent Guittot <vincent.guittot@...aro.org>
Cc:     Ingo Molnar <mingo@...nel.org>,
        Peter Zijlstra <a.p.zijlstra@...llo.nl>,
        Mel Gorman <mgorman@...hsingularity.net>,
        LKML <linux-kernel@...r.kernel.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        lkp@...ts.01.org
Subject: [sched/fair]  289de35984:  reaim.jobs_per_min 3.9% improvement

Greeting,

FYI, we noticed a 3.9% improvement of reaim.jobs_per_min due to commit:


commit: 289de35984815576793f579ec27248609e75976e ("sched/fair: Fix statistics for find_idlest_group()")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

in testcase: reaim
on test machine: 72 threads Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz with 128G memory
with following parameters:

	runtime: 300s
	nr_task: 100%
	test: shared
	cpufreq_governor: performance
	ucode: 0x2000065

test-description: REAIM is an updated and improved version of AIM 7 benchmark.
test-url: https://sourceforge.net/projects/re-aim-7/





Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode:
  gcc-7/performance/x86_64-rhel-7.6/100%/debian-x86_64-20191114.cgz/300s/lkp-skl-2sp7/shared/reaim/0x2000065

commit: 
  c5f8689118 (" RISC-V Fixes for 5.6-rc4")
  289de35984 ("sched/fair: Fix statistics for find_idlest_group()")

c5f86891185c408b 289de35984815576793f579ec27 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
           :4           25%           1:4     dmesg.WARNING:at_ip_perf_output_begin/0x
          2:4           13%           3:4     perf-profile.calltrace.cycles-pp.error_entry
          3:4           15%           4:4     perf-profile.children.cycles-pp.error_entry
          1:4            9%           2:4     perf-profile.self.cycles-pp.error_entry
         %stddev     %change         %stddev
             \          |                \  
      7.38            +5.7%       7.81        reaim.child_systime
    416955            +3.9%     433031        reaim.jobs_per_min
      5791            +3.9%       6014        reaim.jobs_per_min_child
    430897            +3.5%     445787        reaim.max_jobs_per_min
      1.01            -3.7%       0.97        reaim.parent_time
      3.12           -20.6%       2.48        reaim.std_dev_percent
      0.03           -27.7%       0.02 ±  2%  reaim.std_dev_time
    499466           +52.5%     761595        reaim.time.involuntary_context_switches
      1992            +2.7%       2046        reaim.time.percent_of_cpu_this_job_got
    744.17            +6.8%     794.47        reaim.time.system_time
      5292            +1.9%       5391        reaim.time.user_time
   3512800            +3.3%    3628904        reaim.time.voluntary_context_switches
     40398           +10.2%      44531        meminfo.Shmem
     22929 ±  3%     +11.9%      25659 ±  4%  numa-vmstat.node0.nr_slab_unreclaimable
  17108878           +36.4%   23331136 ±  5%  cpuidle.C1.time
    380613           +31.2%     499228 ± 10%  cpuidle.C1.usage
     17477 ± 57%     -66.6%       5843 ±171%  numa-numastat.node0.other_node
      5900 ±170%    +197.2%      17536 ± 57%  numa-numastat.node1.other_node
     72.00            -1.4%      71.00        vmstat.cpu.id
     29984            +1.9%      30549        vmstat.system.cs
     91885 ±  3%     +11.5%     102440 ±  4%  numa-meminfo.node0.SUnreclaim
      5871 ± 13%     -23.2%       4512 ± 18%  numa-meminfo.node1.PageTables
     96878 ±  3%     -10.7%      86519 ±  5%  numa-meminfo.node1.SUnreclaim
    135487 ±  5%     -10.2%     121688 ±  7%  numa-meminfo.node1.Slab
    896.25            +2.3%     917.25        turbostat.Avg_MHz
    383050           +30.6%     500148 ±  9%  turbostat.C1
      0.08            +0.0        0.11 ±  4%  turbostat.C1%
      7.08 ± 10%     +11.1       18.18 ±  3%  turbostat.PKG_%
     76026            +1.3%      77019        proc-vmstat.nr_active_anon
      4499            +1.1%       4549        proc-vmstat.nr_inactive_anon
      2656            -5.1%       2520        proc-vmstat.nr_page_table_pages
     10096           +10.2%      11130        proc-vmstat.nr_shmem
     76026            +1.3%      77019        proc-vmstat.nr_zone_active_anon
      4499            +1.1%       4549        proc-vmstat.nr_zone_inactive_anon
     23889           +14.5%      27352        proc-vmstat.pgactivate
     98431 ±  4%      +9.1%     107371 ±  8%  softirqs.CPU27.TIMER
     98807 ±  4%      +8.3%     107005 ±  6%  softirqs.CPU29.TIMER
     12277 ±  3%     +14.2%      14017 ±  9%  softirqs.CPU45.NET_RX
     14258 ±  4%     -14.0%      12258 ±  7%  softirqs.CPU58.NET_RX
     98434 ±  4%      +6.9%     105206 ±  5%  softirqs.CPU60.TIMER
     98281 ±  4%      +7.2%     105382 ±  7%  softirqs.CPU65.TIMER
     13450 ±  5%     -11.9%      11846 ±  7%  softirqs.CPU69.NET_RX
  18178405 ±  3%     +10.1%   20016639 ±  2%  perf-stat.i.cache-misses
     30173            +1.9%      30737        perf-stat.i.context-switches
 6.391e+10            +1.9%  6.515e+10        perf-stat.i.cpu-cycles
      4838            -2.0%       4743        perf-stat.i.cpu-migrations
 7.018e+09            +1.0%  7.088e+09        perf-stat.i.dTLB-stores
    380522            +1.1%     384844        perf-stat.i.minor-faults
     80.67            +1.6       82.28        perf-stat.i.node-load-miss-rate%
   4777138           +16.4%    5560547        perf-stat.i.node-load-misses
     58.08 ±  2%      +4.5       62.56        perf-stat.i.node-store-miss-rate%
   1001067           +20.2%    1203372        perf-stat.i.node-store-misses
    765102            -2.6%     745114        perf-stat.i.node-stores
    380522            +1.1%     384844        perf-stat.i.page-faults
      3523 ±  2%      -7.4%       3263 ±  2%  perf-stat.overall.cycles-between-cache-misses
     83.93            +2.1       85.98        perf-stat.overall.node-load-miss-rate%
     56.66            +5.1       61.73        perf-stat.overall.node-store-miss-rate%
 5.814e+09            +0.9%  5.865e+09        perf-stat.ps.branch-instructions
  18134570 ±  3%     +10.1%   19964224 ±  2%  perf-stat.ps.cache-misses
     30086            +1.8%      30637        perf-stat.ps.context-switches
 6.384e+10            +2.0%  6.512e+10        perf-stat.ps.cpu-cycles
      4826            -2.1%       4726        perf-stat.ps.cpu-migrations
 1.161e+10            +1.0%  1.173e+10        perf-stat.ps.dTLB-loads
  7.01e+09            +1.0%  7.083e+09        perf-stat.ps.dTLB-stores
   4764698           +16.4%    5545210        perf-stat.ps.node-load-misses
    998056           +20.2%    1199476        perf-stat.ps.node-store-misses
    763473            -2.6%     743558        perf-stat.ps.node-stores
      1.02 ± 39%    -100.0%       0.00        sched_debug.cfs_rq:/.load_avg.min
      0.23 ±  7%     -65.9%       0.08 ±  6%  sched_debug.cfs_rq:/.nr_running.avg
      0.28 ±  2%     -15.3%       0.24 ±  4%  sched_debug.cfs_rq:/.nr_running.stddev
      3.77 ± 11%     +80.3%       6.79 ± 32%  sched_debug.cfs_rq:/.nr_spread_over.max
      1.00 ±  9%     +37.4%       1.38 ± 15%  sched_debug.cfs_rq:/.nr_spread_over.stddev
    356.86 ±  8%     -52.8%     168.33        sched_debug.cfs_rq:/.removed.load_avg.max
     61.03 ± 21%     -50.4%      30.25 ± 36%  sched_debug.cfs_rq:/.removed.load_avg.stddev
     16449 ±  8%     -52.7%       7785        sched_debug.cfs_rq:/.removed.runnable_sum.max
      2812 ± 21%     -50.4%       1396 ± 36%  sched_debug.cfs_rq:/.removed.runnable_sum.stddev
    149.83 ±  9%     -59.0%      61.50 ± 40%  sched_debug.cfs_rq:/.removed.util_avg.max
    253.05 ±  8%     -67.6%      82.02 ±  4%  sched_debug.cfs_rq:/.util_avg.avg
      1062 ±  5%     -31.6%     727.17 ±  5%  sched_debug.cfs_rq:/.util_avg.max
     40.62 ± 57%    -100.0%       0.00        sched_debug.cfs_rq:/.util_avg.min
    187.05 ±  4%     -27.5%     135.70        sched_debug.cfs_rq:/.util_avg.stddev
    106.07 ±  8%     -93.1%       7.34 ±  4%  sched_debug.cfs_rq:/.util_est_enqueued.avg
    403.31 ± 29%     -62.2%     152.58 ±  3%  sched_debug.cfs_rq:/.util_est_enqueued.max
    104.99 ± 13%     -72.8%      28.52 ±  3%  sched_debug.cfs_rq:/.util_est_enqueued.stddev
    182939 ± 10%     +35.4%     247733 ±  3%  sched_debug.cpu.avg_idle.stddev
      8952 ±  8%     -94.7%     473.45 ±  2%  sched_debug.cpu.curr->pid.avg
      7522 ±  3%     -54.4%       3431        sched_debug.cpu.curr->pid.stddev
      0.00 ±  3%     -21.2%       0.00 ±  2%  sched_debug.cpu.next_balance.stddev
      0.24 ±  6%     -75.1%       0.06 ±  8%  sched_debug.cpu.nr_running.avg
      1.35 ±  2%     -25.9%       1.00        sched_debug.cpu.nr_running.max
      0.29 ±  3%     -28.0%       0.21        sched_debug.cpu.nr_running.stddev
      6290 ±  8%     +16.6%       7333 ±  2%  sched_debug.cpu.ttwu_local.min
     59391           +41.9%      84251        interrupts.CAL:Function_call_interrupts
    704.50 ±  3%     +39.5%     982.50 ±  4%  interrupts.CPU0.CAL:Function_call_interrupts
      2572 ±  2%     +15.8%       2977 ±  5%  interrupts.CPU0.NMI:Non-maskable_interrupts
      2572 ±  2%     +15.8%       2977 ±  5%  interrupts.CPU0.PMI:Performance_monitoring_interrupts
      2345 ± 22%    +107.8%       4873 ± 11%  interrupts.CPU0.RES:Rescheduling_interrupts
    431.25 ±  7%     +68.3%     726.00 ±  6%  interrupts.CPU0.TLB:TLB_shootdowns
    832.25 ±  2%     +37.9%       1147 ±  3%  interrupts.CPU1.CAL:Function_call_interrupts
      2544 ±  2%     +16.1%       2954 ±  5%  interrupts.CPU1.NMI:Non-maskable_interrupts
      2544 ±  2%     +16.1%       2954 ±  5%  interrupts.CPU1.PMI:Performance_monitoring_interrupts
      2295 ± 19%    +123.5%       5131 ±  9%  interrupts.CPU1.RES:Rescheduling_interrupts
    411.75 ±  6%     +81.4%     746.75 ±  4%  interrupts.CPU1.TLB:TLB_shootdowns
    830.50 ±  2%     +39.3%       1157 ±  3%  interrupts.CPU10.CAL:Function_call_interrupts
      2539 ±  3%     +16.0%       2945 ±  4%  interrupts.CPU10.NMI:Non-maskable_interrupts
      2539 ±  3%     +16.0%       2945 ±  4%  interrupts.CPU10.PMI:Performance_monitoring_interrupts
      1844 ±  6%    +153.9%       4682 ±  4%  interrupts.CPU10.RES:Rescheduling_interrupts
    422.25 ±  5%     +81.7%     767.25 ±  3%  interrupts.CPU10.TLB:TLB_shootdowns
    845.75           +40.2%       1185        interrupts.CPU11.CAL:Function_call_interrupts
      2557 ±  3%     +15.7%       2960 ±  4%  interrupts.CPU11.NMI:Non-maskable_interrupts
      2557 ±  3%     +15.7%       2960 ±  4%  interrupts.CPU11.PMI:Performance_monitoring_interrupts
      1705 ±  4%    +169.8%       4599 ±  2%  interrupts.CPU11.RES:Rescheduling_interrupts
    444.00 ±  5%     +75.8%     780.50 ±  2%  interrupts.CPU11.TLB:TLB_shootdowns
    829.25 ±  2%     +40.2%       1162 ±  2%  interrupts.CPU12.CAL:Function_call_interrupts
      2559 ±  2%     +16.1%       2972 ±  5%  interrupts.CPU12.NMI:Non-maskable_interrupts
      2559 ±  2%     +16.1%       2972 ±  5%  interrupts.CPU12.PMI:Performance_monitoring_interrupts
      1870 ±  6%    +155.4%       4777 ±  2%  interrupts.CPU12.RES:Rescheduling_interrupts
    406.50 ±  4%     +84.9%     751.75 ±  4%  interrupts.CPU12.TLB:TLB_shootdowns
    849.00           +34.9%       1145 ±  3%  interrupts.CPU13.CAL:Function_call_interrupts
      1914 ± 34%     +56.6%       2997 ±  4%  interrupts.CPU13.NMI:Non-maskable_interrupts
      1914 ± 34%     +56.6%       2997 ±  4%  interrupts.CPU13.PMI:Performance_monitoring_interrupts
      1892 ±  7%    +151.3%       4757 ±  3%  interrupts.CPU13.RES:Rescheduling_interrupts
    452.00 ± 12%     +63.4%     738.50 ±  6%  interrupts.CPU13.TLB:TLB_shootdowns
    835.50 ±  3%     +39.9%       1169 ±  3%  interrupts.CPU14.CAL:Function_call_interrupts
      2215 ± 24%     +35.0%       2990 ±  4%  interrupts.CPU14.NMI:Non-maskable_interrupts
      2215 ± 24%     +35.0%       2990 ±  4%  interrupts.CPU14.PMI:Performance_monitoring_interrupts
      1809 ±  3%    +155.7%       4628 ±  4%  interrupts.CPU14.RES:Rescheduling_interrupts
    423.75 ±  8%     +79.6%     761.25 ±  4%  interrupts.CPU14.TLB:TLB_shootdowns
    831.75 ±  4%     +39.8%       1162        interrupts.CPU15.CAL:Function_call_interrupts
      1791 ±  3%    +160.6%       4667 ±  4%  interrupts.CPU15.RES:Rescheduling_interrupts
    410.50 ±  7%     +84.8%     758.50 ±  3%  interrupts.CPU15.TLB:TLB_shootdowns
    846.00 ±  3%     +37.0%       1158        interrupts.CPU16.CAL:Function_call_interrupts
      2221 ± 24%     +34.4%       2984 ±  5%  interrupts.CPU16.NMI:Non-maskable_interrupts
      2221 ± 24%     +34.4%       2984 ±  5%  interrupts.CPU16.PMI:Performance_monitoring_interrupts
      1865 ±  5%    +153.9%       4736 ±  7%  interrupts.CPU16.RES:Rescheduling_interrupts
    426.25 ±  6%     +77.2%     755.25        interrupts.CPU16.TLB:TLB_shootdowns
    836.00 ±  2%     +38.7%       1159 ±  4%  interrupts.CPU17.CAL:Function_call_interrupts
      1841 ±  3%    +149.5%       4593        interrupts.CPU17.RES:Rescheduling_interrupts
    412.25 ±  5%     +81.9%     749.75 ±  6%  interrupts.CPU17.TLB:TLB_shootdowns
    841.00 ±  2%     +39.9%       1176 ±  2%  interrupts.CPU18.CAL:Function_call_interrupts
      2526           +17.6%       2971 ±  5%  interrupts.CPU18.NMI:Non-maskable_interrupts
      2526           +17.6%       2971 ±  5%  interrupts.CPU18.PMI:Performance_monitoring_interrupts
      1892 ±  9%    +142.8%       4595        interrupts.CPU18.RES:Rescheduling_interrupts
    421.00 ±  6%     +82.4%     768.00 ±  3%  interrupts.CPU18.TLB:TLB_shootdowns
    790.00 ±  9%     +48.6%       1173 ±  3%  interrupts.CPU19.CAL:Function_call_interrupts
      2544 ±  3%     +15.9%       2947 ±  4%  interrupts.CPU19.NMI:Non-maskable_interrupts
      2544 ±  3%     +15.9%       2947 ±  4%  interrupts.CPU19.PMI:Performance_monitoring_interrupts
      1731 ±  5%    +171.5%       4701 ±  4%  interrupts.CPU19.RES:Rescheduling_interrupts
    416.50 ±  6%     +83.6%     764.75 ±  6%  interrupts.CPU19.TLB:TLB_shootdowns
    839.00           +37.5%       1154 ±  2%  interrupts.CPU2.CAL:Function_call_interrupts
      2242 ± 25%     +31.9%       2956 ±  4%  interrupts.CPU2.NMI:Non-maskable_interrupts
      2242 ± 25%     +31.9%       2956 ±  4%  interrupts.CPU2.PMI:Performance_monitoring_interrupts
      1996 ± 15%    +139.7%       4785 ± 10%  interrupts.CPU2.RES:Rescheduling_interrupts
    430.00 ±  2%     +73.1%     744.25 ±  2%  interrupts.CPU2.TLB:TLB_shootdowns
    787.50 ±  7%     +43.3%       1128 ±  5%  interrupts.CPU20.CAL:Function_call_interrupts
      1699 ±  5%    +165.0%       4502        interrupts.CPU20.RES:Rescheduling_interrupts
    438.25 ±  4%     +72.4%     755.50 ±  4%  interrupts.CPU20.TLB:TLB_shootdowns
    830.25 ±  3%     +40.7%       1168 ±  5%  interrupts.CPU21.CAL:Function_call_interrupts
      2208 ± 23%     +35.4%       2990 ±  4%  interrupts.CPU21.NMI:Non-maskable_interrupts
      2208 ± 23%     +35.4%       2990 ±  4%  interrupts.CPU21.PMI:Performance_monitoring_interrupts
      1730 ±  2%    +171.9%       4704 ±  5%  interrupts.CPU21.RES:Rescheduling_interrupts
    421.75 ± 11%     +79.4%     756.50 ±  7%  interrupts.CPU21.TLB:TLB_shootdowns
    854.50 ±  3%     +42.0%       1213        interrupts.CPU22.CAL:Function_call_interrupts
      2209 ± 23%     +35.2%       2986 ±  5%  interrupts.CPU22.NMI:Non-maskable_interrupts
      2209 ± 23%     +35.2%       2986 ±  5%  interrupts.CPU22.PMI:Performance_monitoring_interrupts
      1746 ±  7%    +172.9%       4765 ±  2%  interrupts.CPU22.RES:Rescheduling_interrupts
    447.50 ±  7%     +79.3%     802.50        interrupts.CPU22.TLB:TLB_shootdowns
    835.25           +40.3%       1172        interrupts.CPU23.CAL:Function_call_interrupts
      2188 ± 24%     +36.5%       2987 ±  5%  interrupts.CPU23.NMI:Non-maskable_interrupts
      2188 ± 24%     +36.5%       2987 ±  5%  interrupts.CPU23.PMI:Performance_monitoring_interrupts
      1760 ±  3%    +166.9%       4699 ±  3%  interrupts.CPU23.RES:Rescheduling_interrupts
    416.75 ±  4%     +82.8%     762.00 ±  3%  interrupts.CPU23.TLB:TLB_shootdowns
    778.50 ± 10%     +53.0%       1191        interrupts.CPU24.CAL:Function_call_interrupts
      1718 ±  2%    +185.2%       4901 ±  5%  interrupts.CPU24.RES:Rescheduling_interrupts
    385.00 ±  7%    +101.2%     774.50 ±  3%  interrupts.CPU24.TLB:TLB_shootdowns
    822.25 ±  3%     +40.9%       1158 ±  3%  interrupts.CPU25.CAL:Function_call_interrupts
      1913 ± 30%     +55.1%       2967 ±  4%  interrupts.CPU25.NMI:Non-maskable_interrupts
      1913 ± 30%     +55.1%       2967 ±  4%  interrupts.CPU25.PMI:Performance_monitoring_interrupts
      1731 ±  4%    +171.0%       4691 ±  2%  interrupts.CPU25.RES:Rescheduling_interrupts
    398.25 ±  7%     +89.4%     754.25 ±  5%  interrupts.CPU25.TLB:TLB_shootdowns
    814.00 ±  2%     +44.0%       1172        interrupts.CPU26.CAL:Function_call_interrupts
      1903 ± 34%     +55.2%       2954 ±  4%  interrupts.CPU26.NMI:Non-maskable_interrupts
      1903 ± 34%     +55.2%       2954 ±  4%  interrupts.CPU26.PMI:Performance_monitoring_interrupts
      1663 ±  5%    +178.2%       4629 ±  3%  interrupts.CPU26.RES:Rescheduling_interrupts
    397.50 ±  5%     +91.4%     761.00        interrupts.CPU26.TLB:TLB_shootdowns
    839.00 ±  2%     +38.4%       1161        interrupts.CPU27.CAL:Function_call_interrupts
      1886 ± 31%     +57.8%       2976 ±  5%  interrupts.CPU27.NMI:Non-maskable_interrupts
      1886 ± 31%     +57.8%       2976 ±  5%  interrupts.CPU27.PMI:Performance_monitoring_interrupts
      1725          +173.1%       4711 ±  4%  interrupts.CPU27.RES:Rescheduling_interrupts
    433.00 ±  7%     +72.6%     747.50 ±  2%  interrupts.CPU27.TLB:TLB_shootdowns
    839.00 ±  3%     +40.1%       1175 ±  3%  interrupts.CPU28.CAL:Function_call_interrupts
      2536 ±  2%     +18.6%       3007 ±  6%  interrupts.CPU28.NMI:Non-maskable_interrupts
      2536 ±  2%     +18.6%       3007 ±  6%  interrupts.CPU28.PMI:Performance_monitoring_interrupts
      1708 ±  5%    +182.8%       4832 ±  3%  interrupts.CPU28.RES:Rescheduling_interrupts
    415.75 ±  8%     +84.2%     766.00 ±  4%  interrupts.CPU28.TLB:TLB_shootdowns
    810.50           +46.5%       1187 ±  2%  interrupts.CPU29.CAL:Function_call_interrupts
      2523 ±  2%     +17.9%       2975 ±  5%  interrupts.CPU29.NMI:Non-maskable_interrupts
      2523 ±  2%     +17.9%       2975 ±  5%  interrupts.CPU29.PMI:Performance_monitoring_interrupts
      1685 ±  4%    +176.5%       4659 ±  3%  interrupts.CPU29.RES:Rescheduling_interrupts
    390.00 ±  5%     +99.6%     778.50 ±  3%  interrupts.CPU29.TLB:TLB_shootdowns
    837.00 ±  2%     +42.3%       1191        interrupts.CPU3.CAL:Function_call_interrupts
      2551 ±  2%     +15.8%       2955 ±  4%  interrupts.CPU3.NMI:Non-maskable_interrupts
      2551 ±  2%     +15.8%       2955 ±  4%  interrupts.CPU3.PMI:Performance_monitoring_interrupts
      2166 ± 10%    +111.6%       4583 ±  3%  interrupts.CPU3.RES:Rescheduling_interrupts
    432.25 ±  7%     +82.0%     786.50        interrupts.CPU3.TLB:TLB_shootdowns
    843.75 ±  2%     +41.0%       1189 ±  3%  interrupts.CPU30.CAL:Function_call_interrupts
      2631           +12.8%       2967 ±  4%  interrupts.CPU30.NMI:Non-maskable_interrupts
      2631           +12.8%       2967 ±  4%  interrupts.CPU30.PMI:Performance_monitoring_interrupts
      1731 ±  3%    +173.5%       4735 ±  2%  interrupts.CPU30.RES:Rescheduling_interrupts
    846.25 ±  2%     +40.1%       1186 ±  2%  interrupts.CPU31.CAL:Function_call_interrupts
      2541           +16.7%       2965 ±  5%  interrupts.CPU31.NMI:Non-maskable_interrupts
      2541           +16.7%       2965 ±  5%  interrupts.CPU31.PMI:Performance_monitoring_interrupts
      1721 ±  5%    +168.5%       4622 ±  3%  interrupts.CPU31.RES:Rescheduling_interrupts
    426.25 ±  5%     +81.8%     775.00 ±  5%  interrupts.CPU31.TLB:TLB_shootdowns
    821.00 ±  3%     +46.4%       1202 ±  2%  interrupts.CPU32.CAL:Function_call_interrupts
      2233 ± 24%     +32.6%       2961 ±  5%  interrupts.CPU32.NMI:Non-maskable_interrupts
      2233 ± 24%     +32.6%       2961 ±  5%  interrupts.CPU32.PMI:Performance_monitoring_interrupts
      1737 ±  3%    +168.1%       4658 ±  2%  interrupts.CPU32.RES:Rescheduling_interrupts
    393.75 ±  6%    +101.3%     792.75 ±  4%  interrupts.CPU32.TLB:TLB_shootdowns
    809.50           +44.7%       1171        interrupts.CPU33.CAL:Function_call_interrupts
      2185 ± 23%     +35.3%       2955 ±  5%  interrupts.CPU33.NMI:Non-maskable_interrupts
      2185 ± 23%     +35.3%       2955 ±  5%  interrupts.CPU33.PMI:Performance_monitoring_interrupts
      1551 ±  3%    +193.8%       4556 ±  6%  interrupts.CPU33.RES:Rescheduling_interrupts
    380.75 ±  3%    +100.1%     761.75 ±  2%  interrupts.CPU33.TLB:TLB_shootdowns
    829.25           +41.5%       1173        interrupts.CPU34.CAL:Function_call_interrupts
      2535 ±  2%     +15.7%       2932 ±  5%  interrupts.CPU34.NMI:Non-maskable_interrupts
      2535 ±  2%     +15.7%       2932 ±  5%  interrupts.CPU34.PMI:Performance_monitoring_interrupts
      1797 ±  7%    +163.7%       4738 ±  4%  interrupts.CPU34.RES:Rescheduling_interrupts
    410.00 ±  4%     +87.5%     768.75 ±  2%  interrupts.CPU34.TLB:TLB_shootdowns
    792.50 ±  2%     +40.5%       1113 ±  3%  interrupts.CPU35.CAL:Function_call_interrupts
      2194 ± 23%     +34.4%       2950 ±  4%  interrupts.CPU35.NMI:Non-maskable_interrupts
      2194 ± 23%     +34.4%       2950 ±  4%  interrupts.CPU35.PMI:Performance_monitoring_interrupts
      2481 ±  7%     +88.1%       4668 ±  5%  interrupts.CPU35.RES:Rescheduling_interrupts
    411.75 ±  8%     +81.3%     746.50 ±  4%  interrupts.CPU35.TLB:TLB_shootdowns
    800.50 ±  3%     +41.1%       1129 ±  2%  interrupts.CPU36.CAL:Function_call_interrupts
      2215 ± 24%     +34.4%       2976 ±  5%  interrupts.CPU36.NMI:Non-maskable_interrupts
      2215 ± 24%     +34.4%       2976 ±  5%  interrupts.CPU36.PMI:Performance_monitoring_interrupts
      1765 ±  2%    +160.4%       4598 ±  4%  interrupts.CPU36.RES:Rescheduling_interrupts
    373.75 ±  5%     +90.3%     711.25 ±  4%  interrupts.CPU36.TLB:TLB_shootdowns
    792.75 ±  2%     +48.4%       1176        interrupts.CPU37.CAL:Function_call_interrupts
      1768 ±  2%    +163.7%       4664 ±  4%  interrupts.CPU37.RES:Rescheduling_interrupts
    364.25 ±  6%    +108.9%     760.75 ±  2%  interrupts.CPU37.TLB:TLB_shootdowns
    808.25 ±  2%     +43.1%       1156 ±  3%  interrupts.CPU38.CAL:Function_call_interrupts
      2555           +15.3%       2947 ±  5%  interrupts.CPU38.NMI:Non-maskable_interrupts
      2555           +15.3%       2947 ±  5%  interrupts.CPU38.PMI:Performance_monitoring_interrupts
      1754 ±  2%    +157.1%       4510 ±  2%  interrupts.CPU38.RES:Rescheduling_interrupts
    381.50 ±  6%     +98.0%     755.25 ±  7%  interrupts.CPU38.TLB:TLB_shootdowns
    804.75 ±  2%     +44.5%       1163 ±  2%  interrupts.CPU39.CAL:Function_call_interrupts
      1798 ±  3%    +164.2%       4752 ±  5%  interrupts.CPU39.RES:Rescheduling_interrupts
    387.25 ±  6%     +93.4%     749.00 ±  3%  interrupts.CPU39.TLB:TLB_shootdowns
    824.75 ±  2%     +41.2%       1164 ±  2%  interrupts.CPU4.CAL:Function_call_interrupts
      1929 ± 17%    +131.9%       4474 ±  3%  interrupts.CPU4.RES:Rescheduling_interrupts
    406.50 ±  7%     +85.7%     754.75 ±  3%  interrupts.CPU4.TLB:TLB_shootdowns
    810.00           +44.4%       1170 ±  2%  interrupts.CPU40.CAL:Function_call_interrupts
      1750 ±  4%    +171.2%       4748 ±  6%  interrupts.CPU40.RES:Rescheduling_interrupts
    394.00 ±  4%     +92.3%     757.75 ±  3%  interrupts.CPU40.TLB:TLB_shootdowns
    808.25           +42.3%       1150 ±  2%  interrupts.CPU41.CAL:Function_call_interrupts
      1766 ±  4%    +165.8%       4693        interrupts.CPU41.RES:Rescheduling_interrupts
    380.00 ±  2%     +92.2%     730.50 ±  3%  interrupts.CPU41.TLB:TLB_shootdowns
    813.50           +42.5%       1159 ±  3%  interrupts.CPU42.CAL:Function_call_interrupts
      2566 ±  2%     +15.8%       2973 ±  4%  interrupts.CPU42.NMI:Non-maskable_interrupts
      2566 ±  2%     +15.8%       2973 ±  4%  interrupts.CPU42.PMI:Performance_monitoring_interrupts
      1789 ±  3%    +156.8%       4596 ±  2%  interrupts.CPU42.RES:Rescheduling_interrupts
    409.50 ±  7%     +82.1%     745.50 ±  6%  interrupts.CPU42.TLB:TLB_shootdowns
    818.25           +42.8%       1168 ±  2%  interrupts.CPU43.CAL:Function_call_interrupts
      1825 ±  5%    +158.1%       4711 ±  4%  interrupts.CPU43.RES:Rescheduling_interrupts
    399.50 ±  4%     +88.7%     753.75 ±  3%  interrupts.CPU43.TLB:TLB_shootdowns
    793.25           +46.2%       1159 ±  3%  interrupts.CPU44.CAL:Function_call_interrupts
      2512           +17.3%       2947 ±  5%  interrupts.CPU44.NMI:Non-maskable_interrupts
      2512           +17.3%       2947 ±  5%  interrupts.CPU44.PMI:Performance_monitoring_interrupts
      1761 ±  6%    +167.1%       4704 ±  5%  interrupts.CPU44.RES:Rescheduling_interrupts
    362.75          +104.7%     742.50 ±  4%  interrupts.CPU44.TLB:TLB_shootdowns
    815.25           +39.6%       1138 ±  2%  interrupts.CPU45.CAL:Function_call_interrupts
      2521 ±  2%     +16.9%       2946 ±  4%  interrupts.CPU45.NMI:Non-maskable_interrupts
      2521 ±  2%     +16.9%       2946 ±  4%  interrupts.CPU45.PMI:Performance_monitoring_interrupts
      1766 ±  8%    +165.2%       4684 ±  4%  interrupts.CPU45.RES:Rescheduling_interrupts
    387.50 ±  3%     +88.1%     728.75 ±  2%  interrupts.CPU45.TLB:TLB_shootdowns
    807.25 ±  2%     +42.4%       1149        interrupts.CPU46.CAL:Function_call_interrupts
      2549 ±  3%     +16.5%       2969 ±  4%  interrupts.CPU46.NMI:Non-maskable_interrupts
      2549 ±  3%     +16.5%       2969 ±  4%  interrupts.CPU46.PMI:Performance_monitoring_interrupts
      1811 ±  4%    +164.3%       4789 ±  7%  interrupts.CPU46.RES:Rescheduling_interrupts
    384.25 ±  3%     +92.9%     741.25 ±  3%  interrupts.CPU46.TLB:TLB_shootdowns
    816.25           +43.8%       1174 ±  2%  interrupts.CPU47.CAL:Function_call_interrupts
      2516 ±  2%     +16.7%       2937 ±  4%  interrupts.CPU47.NMI:Non-maskable_interrupts
      2516 ±  2%     +16.7%       2937 ±  4%  interrupts.CPU47.PMI:Performance_monitoring_interrupts
      1835 ±  5%    +159.4%       4762 ±  2%  interrupts.CPU47.RES:Rescheduling_interrupts
    374.25 ±  3%    +107.1%     775.00 ±  3%  interrupts.CPU47.TLB:TLB_shootdowns
    808.25           +49.2%       1205 ±  4%  interrupts.CPU48.CAL:Function_call_interrupts
      2555 ±  2%     +16.8%       2984 ±  5%  interrupts.CPU48.NMI:Non-maskable_interrupts
      2555 ±  2%     +16.8%       2984 ±  5%  interrupts.CPU48.PMI:Performance_monitoring_interrupts
      1750 ±  4%    +172.8%       4774 ±  5%  interrupts.CPU48.RES:Rescheduling_interrupts
    372.75 ±  6%    +118.6%     814.75 ±  5%  interrupts.CPU48.TLB:TLB_shootdowns
    812.75 ±  3%     +45.6%       1183 ±  3%  interrupts.CPU49.CAL:Function_call_interrupts
      2573 ±  3%     +16.1%       2987 ±  4%  interrupts.CPU49.NMI:Non-maskable_interrupts
      2573 ±  3%     +16.1%       2987 ±  4%  interrupts.CPU49.PMI:Performance_monitoring_interrupts
      1801 ±  2%    +156.4%       4619 ±  3%  interrupts.CPU49.RES:Rescheduling_interrupts
    380.50 ±  6%    +108.0%     791.50 ±  9%  interrupts.CPU49.TLB:TLB_shootdowns
    838.00 ±  3%     +37.8%       1154 ±  2%  interrupts.CPU5.CAL:Function_call_interrupts
      2561 ±  3%     +15.3%       2952 ±  5%  interrupts.CPU5.NMI:Non-maskable_interrupts
      2561 ±  3%     +15.3%       2952 ±  5%  interrupts.CPU5.PMI:Performance_monitoring_interrupts
      1771          +166.4%       4718 ±  4%  interrupts.CPU5.RES:Rescheduling_interrupts
    419.25 ±  5%     +79.5%     752.50 ±  3%  interrupts.CPU5.TLB:TLB_shootdowns
    803.25 ±  3%     +45.7%       1170        interrupts.CPU50.CAL:Function_call_interrupts
      1823 ±  5%    +149.7%       4554 ±  3%  interrupts.CPU50.RES:Rescheduling_interrupts
    377.00 ±  3%    +103.2%     766.25 ±  3%  interrupts.CPU50.TLB:TLB_shootdowns
    815.25 ±  2%     +45.6%       1187 ±  2%  interrupts.CPU51.CAL:Function_call_interrupts
      2540 ±  2%     +17.2%       2976 ±  5%  interrupts.CPU51.NMI:Non-maskable_interrupts
      2540 ±  2%     +17.2%       2976 ±  5%  interrupts.CPU51.PMI:Performance_monitoring_interrupts
      1804          +156.0%       4619 ±  2%  interrupts.CPU51.RES:Rescheduling_interrupts
    389.00 ±  3%    +100.2%     778.75 ±  2%  interrupts.CPU51.TLB:TLB_shootdowns
    808.75 ±  2%     +40.6%       1136 ±  2%  interrupts.CPU52.CAL:Function_call_interrupts
      2552 ±  2%     +17.1%       2988 ±  5%  interrupts.CPU52.NMI:Non-maskable_interrupts
      2552 ±  2%     +17.1%       2988 ±  5%  interrupts.CPU52.PMI:Performance_monitoring_interrupts
      1755 ±  4%    +165.2%       4655 ±  3%  interrupts.CPU52.RES:Rescheduling_interrupts
    370.50 ±  7%    +100.5%     743.00 ±  2%  interrupts.CPU52.TLB:TLB_shootdowns
    843.50 ±  3%     +37.0%       1155 ±  2%  interrupts.CPU53.CAL:Function_call_interrupts
      1807 ±  9%    +160.5%       4707 ±  3%  interrupts.CPU53.RES:Rescheduling_interrupts
    429.25 ±  8%     +75.6%     753.75 ±  4%  interrupts.CPU53.TLB:TLB_shootdowns
    826.75 ±  3%     +37.1%       1133 ±  8%  interrupts.CPU54.CAL:Function_call_interrupts
      2540           +17.0%       2972 ±  4%  interrupts.CPU54.NMI:Non-maskable_interrupts
      2540           +17.0%       2972 ±  4%  interrupts.CPU54.PMI:Performance_monitoring_interrupts
      1700 ±  6%    +172.4%       4631 ±  3%  interrupts.CPU54.RES:Rescheduling_interrupts
    392.50 ±  8%     +91.8%     752.75 ±  3%  interrupts.CPU54.TLB:TLB_shootdowns
    830.75 ±  2%     +45.3%       1206        interrupts.CPU55.CAL:Function_call_interrupts
      2239 ± 24%     +31.5%       2944 ±  5%  interrupts.CPU55.NMI:Non-maskable_interrupts
      2239 ± 24%     +31.5%       2944 ±  5%  interrupts.CPU55.PMI:Performance_monitoring_interrupts
      1728 ±  4%    +181.3%       4863        interrupts.CPU55.RES:Rescheduling_interrupts
    399.25 ±  4%     +96.6%     785.00 ±  2%  interrupts.CPU55.TLB:TLB_shootdowns
    832.00 ±  2%     +41.9%       1181 ±  3%  interrupts.CPU56.CAL:Function_call_interrupts
      2244 ± 25%     +32.9%       2981 ±  4%  interrupts.CPU56.NMI:Non-maskable_interrupts
      2244 ± 25%     +32.9%       2981 ±  4%  interrupts.CPU56.PMI:Performance_monitoring_interrupts
      1755 ±  6%    +169.6%       4732 ±  3%  interrupts.CPU56.RES:Rescheduling_interrupts
    409.25 ±  5%     +84.9%     756.50 ±  5%  interrupts.CPU56.TLB:TLB_shootdowns
    818.75 ±  6%     +39.1%       1138 ±  4%  interrupts.CPU57.CAL:Function_call_interrupts
      2530 ±  2%     +17.1%       2963 ±  4%  interrupts.CPU57.NMI:Non-maskable_interrupts
      2530 ±  2%     +17.1%       2963 ±  4%  interrupts.CPU57.PMI:Performance_monitoring_interrupts
      1838 ±  7%    +161.8%       4812 ±  5%  interrupts.CPU57.RES:Rescheduling_interrupts
    417.75 ±  5%     +83.9%     768.25 ±  2%  interrupts.CPU57.TLB:TLB_shootdowns
    847.00           +38.6%       1174 ±  2%  interrupts.CPU58.CAL:Function_call_interrupts
      1711          +172.4%       4663 ±  2%  interrupts.CPU58.RES:Rescheduling_interrupts
    418.00 ±  6%     +78.7%     747.00 ±  3%  interrupts.CPU58.TLB:TLB_shootdowns
    840.50 ±  2%     +40.4%       1180        interrupts.CPU59.CAL:Function_call_interrupts
      2523           +18.1%       2979 ±  5%  interrupts.CPU59.NMI:Non-maskable_interrupts
      2523           +18.1%       2979 ±  5%  interrupts.CPU59.PMI:Performance_monitoring_interrupts
      1763 ±  7%    +166.8%       4703        interrupts.CPU59.RES:Rescheduling_interrupts
    407.75 ±  6%     +85.7%     757.00 ±  3%  interrupts.CPU59.TLB:TLB_shootdowns
    829.50 ±  4%     +43.8%       1193 ±  2%  interrupts.CPU6.CAL:Function_call_interrupts
      2242 ± 24%     +33.1%       2984 ±  4%  interrupts.CPU6.NMI:Non-maskable_interrupts
      2242 ± 24%     +33.1%       2984 ±  4%  interrupts.CPU6.PMI:Performance_monitoring_interrupts
      1806          +157.5%       4651 ±  2%  interrupts.CPU6.RES:Rescheduling_interrupts
    423.25 ± 14%     +89.4%     801.50 ±  6%  interrupts.CPU6.TLB:TLB_shootdowns
    843.25 ±  2%     +43.3%       1208        interrupts.CPU60.CAL:Function_call_interrupts
      1751 ±  4%    +160.1%       4555 ±  2%  interrupts.CPU60.RES:Rescheduling_interrupts
    388.00 ±  5%     +98.6%     770.75        interrupts.CPU60.TLB:TLB_shootdowns
    841.75           +39.1%       1170 ±  3%  interrupts.CPU61.CAL:Function_call_interrupts
      1716 ±  7%    +169.6%       4626 ±  2%  interrupts.CPU61.RES:Rescheduling_interrupts
    419.00 ±  6%     +80.7%     757.25 ±  4%  interrupts.CPU61.TLB:TLB_shootdowns
    841.50 ±  2%     +42.3%       1197        interrupts.CPU62.CAL:Function_call_interrupts
      2549           +16.2%       2962 ±  4%  interrupts.CPU62.NMI:Non-maskable_interrupts
      2549           +16.2%       2962 ±  4%  interrupts.CPU62.PMI:Performance_monitoring_interrupts
      1623 ±  2%    +187.9%       4674        interrupts.CPU62.RES:Rescheduling_interrupts
    396.75 ±  4%     +92.0%     761.75 ±  3%  interrupts.CPU62.TLB:TLB_shootdowns
    867.25           +42.5%       1235 ±  2%  interrupts.CPU63.CAL:Function_call_interrupts
      1665 ±  7%    +181.6%       4690        interrupts.CPU63.RES:Rescheduling_interrupts
    413.25          +101.8%     833.75 ±  4%  interrupts.CPU63.TLB:TLB_shootdowns
    843.50 ±  4%     +44.0%       1214 ±  2%  interrupts.CPU64.CAL:Function_call_interrupts
      2166 ± 23%     +38.9%       3008 ±  6%  interrupts.CPU64.NMI:Non-maskable_interrupts
      2166 ± 23%     +38.9%       3008 ±  6%  interrupts.CPU64.PMI:Performance_monitoring_interrupts
      1667 ±  2%    +175.2%       4589 ±  2%  interrupts.CPU64.RES:Rescheduling_interrupts
    398.25 ± 12%     +94.4%     774.00 ±  3%  interrupts.CPU64.TLB:TLB_shootdowns
    844.75 ±  3%     +42.0%       1199        interrupts.CPU65.CAL:Function_call_interrupts
      2532 ±  3%     +18.3%       2996 ±  4%  interrupts.CPU65.NMI:Non-maskable_interrupts
      2532 ±  3%     +18.3%       2996 ±  4%  interrupts.CPU65.PMI:Performance_monitoring_interrupts
      1610 ±  2%    +192.1%       4705 ±  3%  interrupts.CPU65.RES:Rescheduling_interrupts
    401.75 ±  8%     +88.8%     758.50 ±  2%  interrupts.CPU65.TLB:TLB_shootdowns
    835.25 ±  2%     +47.9%       1235 ±  2%  interrupts.CPU66.CAL:Function_call_interrupts
      1658          +185.1%       4728 ±  3%  interrupts.CPU66.RES:Rescheduling_interrupts
    390.75 ±  8%    +105.1%     801.25 ±  3%  interrupts.CPU66.TLB:TLB_shootdowns
    837.25 ±  2%     +36.9%       1146 ±  8%  interrupts.CPU67.CAL:Function_call_interrupts
      2210 ± 24%     +33.3%       2946 ±  5%  interrupts.CPU67.NMI:Non-maskable_interrupts
      2210 ± 24%     +33.3%       2946 ±  5%  interrupts.CPU67.PMI:Performance_monitoring_interrupts
      1589 ±  5%    +193.1%       4658        interrupts.CPU67.RES:Rescheduling_interrupts
    393.75 ±  4%     +88.2%     741.00 ±  4%  interrupts.CPU67.TLB:TLB_shootdowns
    829.75           +45.6%       1207 ±  3%  interrupts.CPU68.CAL:Function_call_interrupts
      2246 ± 26%     +32.3%       2973 ±  4%  interrupts.CPU68.NMI:Non-maskable_interrupts
      2246 ± 26%     +32.3%       2973 ±  4%  interrupts.CPU68.PMI:Performance_monitoring_interrupts
      1622 ±  3%    +183.8%       4605 ±  3%  interrupts.CPU68.RES:Rescheduling_interrupts
    379.50 ±  4%    +102.5%     768.50 ±  4%  interrupts.CPU68.TLB:TLB_shootdowns
    845.25 ±  2%     +40.3%       1186        interrupts.CPU69.CAL:Function_call_interrupts
      2549 ±  2%     +15.9%       2954 ±  5%  interrupts.CPU69.NMI:Non-maskable_interrupts
      2549 ±  2%     +15.9%       2954 ±  5%  interrupts.CPU69.PMI:Performance_monitoring_interrupts
      1606 ±  4%    +187.2%       4613        interrupts.CPU69.RES:Rescheduling_interrupts
    411.50 ± 11%     +82.2%     749.75 ±  2%  interrupts.CPU69.TLB:TLB_shootdowns
    841.00           +38.3%       1163        interrupts.CPU7.CAL:Function_call_interrupts
      2247 ± 25%     +32.4%       2974 ±  6%  interrupts.CPU7.NMI:Non-maskable_interrupts
      2247 ± 25%     +32.4%       2974 ±  6%  interrupts.CPU7.PMI:Performance_monitoring_interrupts
      1849 ±  4%    +146.3%       4556 ±  4%  interrupts.CPU7.RES:Rescheduling_interrupts
    413.50 ±  2%     +85.6%     767.25 ±  5%  interrupts.CPU7.TLB:TLB_shootdowns
    842.00           +41.7%       1193 ±  3%  interrupts.CPU70.CAL:Function_call_interrupts
      2533           +16.3%       2945 ±  4%  interrupts.CPU70.NMI:Non-maskable_interrupts
      2533           +16.3%       2945 ±  4%  interrupts.CPU70.PMI:Performance_monitoring_interrupts
      1620 ±  5%    +188.6%       4676 ±  2%  interrupts.CPU70.RES:Rescheduling_interrupts
    389.75 ±  2%     +95.8%     763.00 ±  5%  interrupts.CPU70.TLB:TLB_shootdowns
    845.25 ±  2%     +38.2%       1168        interrupts.CPU71.CAL:Function_call_interrupts
      2543 ±  3%     +16.6%       2966 ±  2%  interrupts.CPU71.NMI:Non-maskable_interrupts
      2543 ±  3%     +16.6%       2966 ±  2%  interrupts.CPU71.PMI:Performance_monitoring_interrupts
      1706 ±  3%    +169.0%       4589 ±  3%  interrupts.CPU71.RES:Rescheduling_interrupts
    830.75           +44.5%       1200        interrupts.CPU8.CAL:Function_call_interrupts
      2526 ±  2%     +17.4%       2965 ±  5%  interrupts.CPU8.NMI:Non-maskable_interrupts
      2526 ±  2%     +17.4%       2965 ±  5%  interrupts.CPU8.PMI:Performance_monitoring_interrupts
      1724 ±  4%    +172.8%       4704 ±  2%  interrupts.CPU8.RES:Rescheduling_interrupts
    418.75 ±  6%     +90.4%     797.50 ±  2%  interrupts.CPU8.TLB:TLB_shootdowns
    834.50 ±  3%     +40.5%       1172 ±  3%  interrupts.CPU9.CAL:Function_call_interrupts
      2547 ±  2%     +15.4%       2938 ±  4%  interrupts.CPU9.NMI:Non-maskable_interrupts
      2547 ±  2%     +15.4%       2938 ±  4%  interrupts.CPU9.PMI:Performance_monitoring_interrupts
      1772 ±  9%    +171.3%       4808 ±  3%  interrupts.CPU9.RES:Rescheduling_interrupts
    411.25 ±  5%     +83.9%     756.25 ±  4%  interrupts.CPU9.TLB:TLB_shootdowns
    173226           +18.8%     205806 ±  4%  interrupts.NMI:Non-maskable_interrupts
    173226           +18.8%     205806 ±  4%  interrupts.PMI:Performance_monitoring_interrupts
    128562          +162.4%     337364        interrupts.RES:Rescheduling_interrupts
     33299 ± 21%     +69.2%      56326 ±  4%  interrupts.TLB:TLB_shootdowns
     35.37 ±  3%      -7.9       27.51 ±  4%  perf-profile.calltrace.cycles-pp.secondary_startup_64
     34.92 ±  3%      -7.8       27.13 ±  4%  perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
     34.93 ±  3%      -7.8       27.14 ±  4%  perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
     34.93 ±  3%      -7.8       27.14 ±  4%  perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
     33.12 ±  2%      -7.6       25.55 ±  4%  perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
     32.83 ±  2%      -7.5       25.32 ±  4%  perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
     29.02            -6.8       22.26 ±  3%  perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
      7.95 ±  3%      -1.3        6.65 ±  2%  perf-profile.calltrace.cycles-pp.__strncat_chk
      3.66 ± 13%      -0.7        2.94 ±  9%  perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
      4.31 ±  4%      -0.7        3.65 ±  3%  perf-profile.calltrace.cycles-pp.__strcat_chk
      3.33 ± 13%      -0.6        2.70 ±  9%  perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
      5.23 ±  2%      -0.5        4.75 ±  2%  perf-profile.calltrace.cycles-pp.brk
      3.95 ±  2%      -0.4        3.56        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.brk
      3.90 ±  2%      -0.4        3.51 ±  2%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk
      1.88 ± 16%      -0.3        1.54 ± 11%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      0.56 ±  4%      -0.3        0.27 ±100%  perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.brk
      0.54 ±  4%      -0.3        0.26 ±100%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
      2.82 ±  2%      -0.3        2.57 ±  3%  perf-profile.calltrace.cycles-pp.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk
      1.08 ±  4%      -0.2        0.90 ±  5%  perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      1.84 ±  3%      -0.1        1.69 ±  3%  perf-profile.calltrace.cycles-pp.do_brk_flags.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk
      0.83 ±  8%      -0.1        0.71 ±  3%  perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state
      0.92 ±  2%      -0.1        0.84 ±  7%  perf-profile.calltrace.cycles-pp.creat
      0.64            -0.1        0.57 ±  5%  perf-profile.calltrace.cycles-pp.search_binary_handler.load_script.search_binary_handler.__do_execve_file.__x64_sys_execve
      0.66            -0.1        0.59 ±  5%  perf-profile.calltrace.cycles-pp.load_script.search_binary_handler.__do_execve_file.__x64_sys_execve.do_syscall_64
      0.63 ±  2%      -0.1        0.56 ±  5%  perf-profile.calltrace.cycles-pp.load_elf_binary.search_binary_handler.load_script.search_binary_handler.__do_execve_file
      0.59 ±  4%      -0.1        0.54 ±  4%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.brk
      0.58 ±  4%      +0.0        0.62 ±  4%  perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.flush_old_exec
      0.62 ±  3%      +0.1        0.68 ±  5%  perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.flush_old_exec.load_elf_binary
      0.76 ±  4%      +0.1        0.84 ±  4%  perf-profile.calltrace.cycles-pp._dl_addr
      0.57 ±  8%      +0.1        0.67 ±  7%  perf-profile.calltrace.cycles-pp.__split_vma.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64
      0.74 ±  4%      +0.1        0.86 ±  6%  perf-profile.calltrace.cycles-pp.prepare_exit_to_usermode.swapgs_restore_regs_and_return_to_usermode
      0.86 ±  3%      +0.1        0.99 ±  3%  perf-profile.calltrace.cycles-pp.setlocale
      0.95 ±  5%      +0.1        1.09 ±  3%  perf-profile.calltrace.cycles-pp.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.88 ±  5%      +0.1        1.02 ±  5%  perf-profile.calltrace.cycles-pp.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.95 ±  5%      +0.1        1.10 ±  4%  perf-profile.calltrace.cycles-pp.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.85 ±  7%      +0.2        1.01 ±  5%  perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64
      0.85 ± 12%      +0.2        1.01 ±  7%  perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
      0.94 ±  7%      +0.2        1.10 ±  5%  perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.00 ±  6%      +0.2        1.17 ±  4%  perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.02 ±  6%      +0.2        1.19 ±  4%  perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.19 ±  2%      +0.2        1.37 ±  4%  perf-profile.calltrace.cycles-pp.swapgs_restore_regs_and_return_to_usermode
      0.79 ±  4%      +0.2        0.99 ±  5%  perf-profile.calltrace.cycles-pp.page_remove_rmap.unmap_page_range.unmap_vmas.exit_mmap.mmput
      0.97 ±  3%      +0.2        1.17        perf-profile.calltrace.cycles-pp.wp_page_copy.do_wp_page.handle_pte_fault.__handle_mm_fault.handle_mm_fault
      0.88 ± 11%      +0.2        1.09 ±  8%  perf-profile.calltrace.cycles-pp.ret_from_fork
      1.09 ±  3%      +0.2        1.34 ±  2%  perf-profile.calltrace.cycles-pp.do_wp_page.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault
      0.79            +0.3        1.05 ±  2%  perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput
      0.74 ± 13%      +0.3        1.02 ± 18%  perf-profile.calltrace.cycles-pp.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents.do_syscall_64
      0.94 ±  4%      +0.3        1.22 ± 11%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      0.92 ±  4%      +0.3        1.20 ±  5%  perf-profile.calltrace.cycles-pp.free_pgtables.exit_mmap.mmput.do_exit.do_group_exit
      1.70 ±  2%      +0.3        1.99 ±  4%  perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.68 ±  2%      +0.3        1.97 ±  4%  perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64
      0.93 ±  9%      +0.3        1.24 ± 16%  perf-profile.calltrace.cycles-pp.dcache_readdir.iterate_dir.__x64_sys_getdents.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.95 ±  9%      +0.3        1.27 ± 16%  perf-profile.calltrace.cycles-pp.__x64_sys_getdents.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.94 ±  9%      +0.3        1.26 ± 16%  perf-profile.calltrace.cycles-pp.iterate_dir.__x64_sys_getdents.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.03 ±  2%      +0.3        1.35 ±  2%  perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput.do_exit
      1.03 ±  2%      +0.3        1.37 ±  2%  perf-profile.calltrace.cycles-pp.tlb_finish_mmu.exit_mmap.mmput.do_exit.do_group_exit
      0.27 ±100%      +0.3        0.60 ±  7%  perf-profile.calltrace.cycles-pp.__vma_adjust.__split_vma.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect
      1.18 ±  2%      +0.4        1.56 ±  3%  perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.19 ±  2%      +0.4        1.57 ±  3%  perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.27 ±100%      +0.4        0.65 ± 13%  perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
      0.28 ±100%      +0.4        0.68 ± 12%  perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
      2.35 ±  2%      +0.5        2.81 ±  3%  perf-profile.calltrace.cycles-pp.filemap_map_pages.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault
      1.48 ±  5%      +0.5        1.93 ± 10%  perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt
      1.51 ±  4%      +0.5        1.97 ± 10%  perf-profile.calltrace.cycles-pp.apic_timer_interrupt
      0.13 ±173%      +0.5        0.61 ± 13%  perf-profile.calltrace.cycles-pp.rcu_core.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
      0.00            +0.5        0.54 ±  4%  perf-profile.calltrace.cycles-pp.link_path_walk.path_openat.do_filp_open.do_sys_openat2.do_sys_open
      0.12 ±173%      +0.5        0.67 ±  5%  perf-profile.calltrace.cycles-pp.unlink_anon_vmas.free_pgtables.exit_mmap.mmput.do_exit
      0.81 ±  4%      +0.5        1.35 ±  2%  perf-profile.calltrace.cycles-pp.anon_vma_fork.dup_mm.copy_process._do_fork.__x64_sys_clone
      0.00            +0.6        0.57 ±  3%  perf-profile.calltrace.cycles-pp.do_brk_flags.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +0.6        0.58 ±  5%  perf-profile.calltrace.cycles-pp.mmap64
      2.42 ±  2%      +0.6        3.03 ±  2%  perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.do_exit
      2.54 ±  2%      +0.6        3.16 ±  2%  perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.do_exit.do_group_exit
      1.09 ±  6%      +0.6        1.73 ±  4%  perf-profile.calltrace.cycles-pp.copy_p4d_range.copy_page_range.dup_mm.copy_process._do_fork
      1.14 ±  6%      +0.7        1.81 ±  3%  perf-profile.calltrace.cycles-pp.copy_page_range.dup_mm.copy_process._do_fork.__x64_sys_clone
      0.00            +0.7        0.71 ±  2%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64
      0.00            +0.8        0.83        perf-profile.calltrace.cycles-pp.__do_execve_file.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +0.8        0.85        perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
      3.59 ±  3%      +0.9        4.45 ±  3%  perf-profile.calltrace.cycles-pp.dup_mm.copy_process._do_fork.__x64_sys_clone.do_syscall_64
      0.00            +0.9        0.86 ±  3%  perf-profile.calltrace.cycles-pp.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe
      4.67            +0.9        5.61 ±  2%  perf-profile.calltrace.cycles-pp.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault
      4.97            +1.0        5.96 ±  2%  perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault
      5.15            +1.0        6.18 ±  3%  perf-profile.calltrace.cycles-pp.handle_mm_fault.do_page_fault.page_fault
      4.78 ±  3%      +1.1        5.87 ±  3%  perf-profile.calltrace.cycles-pp.copy_process._do_fork.__x64_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.83            +1.2        7.01 ±  2%  perf-profile.calltrace.cycles-pp.do_page_fault.page_fault
      6.10            +1.2        7.29 ±  2%  perf-profile.calltrace.cycles-pp.page_fault
      1.12 ±  4%      +1.2        2.37 ±  3%  perf-profile.calltrace.cycles-pp.__x64_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.11 ±  4%      +1.3        2.37 ±  3%  perf-profile.calltrace.cycles-pp._do_fork.__x64_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
      4.82            +1.4        6.19 ±  2%  perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group
      4.83            +1.4        6.21 ±  2%  perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      5.73            +1.6        7.32 ±  2%  perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.73            +1.6        7.32 ±  2%  perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.72            +1.6        7.31 ±  2%  perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
     12.82            +6.9       19.73        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
     12.86            +6.9       19.77        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
     35.38 ±  3%      -7.9       27.52 ±  4%  perf-profile.children.cycles-pp.do_idle
     35.37 ±  3%      -7.9       27.51 ±  4%  perf-profile.children.cycles-pp.secondary_startup_64
     35.37 ±  3%      -7.9       27.51 ±  4%  perf-profile.children.cycles-pp.cpu_startup_entry
     34.93 ±  3%      -7.8       27.14 ±  4%  perf-profile.children.cycles-pp.start_secondary
     33.53 ±  2%      -7.6       25.89 ±  4%  perf-profile.children.cycles-pp.cpuidle_enter
     33.52 ±  2%      -7.6       25.89 ±  4%  perf-profile.children.cycles-pp.cpuidle_enter_state
     29.38            -6.8       22.56 ±  3%  perf-profile.children.cycles-pp.intel_idle
      7.96 ±  3%      -1.3        6.67 ±  2%  perf-profile.children.cycles-pp.__strncat_chk
      4.31 ±  4%      -0.7        3.65 ±  3%  perf-profile.children.cycles-pp.__strcat_chk
      5.33 ±  2%      -0.5        4.83 ±  2%  perf-profile.children.cycles-pp.brk
      0.93 ±  2%      -0.1        0.84 ±  7%  perf-profile.children.cycles-pp.creat
      0.15 ±  2%      -0.1        0.08 ± 10%  perf-profile.children.cycles-pp.newidle_balance
      0.20 ± 10%      -0.1        0.14 ± 11%  perf-profile.children.cycles-pp.find_busiest_group
      0.28 ± 11%      -0.1        0.23 ±  8%  perf-profile.children.cycles-pp._find_next_bit
      0.50 ±  4%      -0.1        0.45        perf-profile.children.cycles-pp.kill
      0.22 ±  7%      -0.0        0.18 ± 19%  perf-profile.children.cycles-pp.lapic_next_deadline
      0.23 ±  3%      -0.0        0.20 ±  4%  perf-profile.children.cycles-pp.pick_next_task_fair
      0.06 ± 11%      +0.0        0.08 ±  6%  perf-profile.children.cycles-pp.prepend_name
      0.09 ±  4%      +0.0        0.11 ±  6%  perf-profile.children.cycles-pp.apparmor_file_open
      0.08 ±  8%      +0.0        0.10 ±  8%  perf-profile.children.cycles-pp.pipe_read
      0.06 ± 11%      +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.__x64_sys_rt_sigaction
      0.06 ±  7%      +0.0        0.08 ±  6%  perf-profile.children.cycles-pp.touch_atime
      0.10 ±  4%      +0.0        0.12 ±  6%  perf-profile.children.cycles-pp.security_file_open
      0.10 ±  8%      +0.0        0.12 ±  7%  perf-profile.children.cycles-pp.common_file_perm
      0.22 ±  4%      +0.0        0.24 ±  2%  perf-profile.children.cycles-pp.malloc
      0.15 ±  5%      +0.0        0.17 ±  2%  perf-profile.children.cycles-pp.change_p4d_range
      0.08 ± 10%      +0.0        0.10 ±  4%  perf-profile.children.cycles-pp.ima_file_check
      0.14 ±  3%      +0.0        0.16 ±  8%  perf-profile.children.cycles-pp.inode_permission
      0.08 ± 11%      +0.0        0.10 ± 11%  perf-profile.children.cycles-pp.__vm_enough_memory
      0.08 ±  5%      +0.0        0.10 ±  7%  perf-profile.children.cycles-pp.queued_read_lock_slowpath
      0.07 ± 10%      +0.0        0.09 ± 11%  perf-profile.children.cycles-pp.queued_write_lock_slowpath
      0.09 ± 13%      +0.0        0.11 ±  7%  perf-profile.children.cycles-pp.sock_read_iter
      0.07 ± 12%      +0.0        0.09 ± 11%  perf-profile.children.cycles-pp.mem_cgroup_throttle_swaprate
      0.07 ± 15%      +0.0        0.10 ±  8%  perf-profile.children.cycles-pp.fsnotify
      0.11 ± 11%      +0.0        0.14 ±  7%  perf-profile.children.cycles-pp.call_rcu
      0.21 ±  2%      +0.0        0.24 ±  6%  perf-profile.children.cycles-pp.arch_get_unmapped_area_topdown
      0.14 ± 13%      +0.0        0.17 ±  4%  perf-profile.children.cycles-pp._raw_spin_lock_irq
      0.13 ± 12%      +0.0        0.16 ±  4%  perf-profile.children.cycles-pp.__d_alloc
      0.14 ± 10%      +0.0        0.17 ±  2%  perf-profile.children.cycles-pp.terminate_walk
      0.09 ±  8%      +0.0        0.12 ±  8%  perf-profile.children.cycles-pp.copy_fpstate_to_sigframe
      0.06 ± 11%      +0.0        0.09 ±  7%  perf-profile.children.cycles-pp.__update_load_avg_se
      0.04 ± 59%      +0.0        0.08 ±  6%  perf-profile.children.cycles-pp.__legitimize_mnt
      0.11 ±  4%      +0.0        0.15 ±  7%  perf-profile.children.cycles-pp._IO_file_open
      0.14 ± 10%      +0.0        0.17 ±  9%  perf-profile.children.cycles-pp.__dentry_kill
      0.11 ± 14%      +0.0        0.14 ±  6%  perf-profile.children.cycles-pp.free_pcppages_bulk
      0.11 ±  6%      +0.0        0.14 ±  3%  perf-profile.children.cycles-pp.__list_add_valid
      0.24 ±  3%      +0.0        0.28 ±  4%  perf-profile.children.cycles-pp.change_protection
      0.21 ±  8%      +0.0        0.24 ±  6%  perf-profile.children.cycles-pp.mem_cgroup_try_charge
      0.04 ± 60%      +0.0        0.08 ± 14%  perf-profile.children.cycles-pp.kfree
      0.18 ±  2%      +0.0        0.22 ±  3%  perf-profile.children.cycles-pp.strlen
      0.14 ± 12%      +0.0        0.18 ±  4%  perf-profile.children.cycles-pp.memset_erms
      0.13 ± 11%      +0.0        0.17 ±  2%  perf-profile.children.cycles-pp.autoremove_wake_function
      0.13 ±  6%      +0.0        0.17 ±  7%  perf-profile.children.cycles-pp.sched_move_task
      0.12 ±  6%      +0.0        0.16 ±  9%  perf-profile.children.cycles-pp.simple_lookup
      0.07 ± 17%      +0.0        0.10 ±  7%  perf-profile.children.cycles-pp.__rb_erase_color
      0.07 ± 25%      +0.0        0.10 ± 12%  perf-profile.children.cycles-pp.apparmor_task_kill
      0.04 ± 58%      +0.0        0.08 ± 20%  perf-profile.children.cycles-pp.free_ldt_pgtables
      0.08 ± 15%      +0.0        0.12 ±  3%  perf-profile.children.cycles-pp.__put_anon_vma
      0.09 ± 13%      +0.0        0.13 ± 14%  perf-profile.children.cycles-pp.complete_walk
      0.15 ± 10%      +0.0        0.19 ±  9%  perf-profile.children.cycles-pp.lockref_get_not_dead
      0.16 ±  7%      +0.0        0.20 ±  3%  perf-profile.children.cycles-pp._copy_from_user
      0.13 ±  9%      +0.0        0.17 ±  4%  perf-profile.children.cycles-pp.__might_fault
      0.12 ±  4%      +0.0        0.16 ± 10%  perf-profile.children.cycles-pp.dequeue_entity
      0.10            +0.0        0.14 ±  9%  perf-profile.children.cycles-pp.free_pgd_range
      0.10 ± 14%      +0.0        0.14 ±  9%  perf-profile.children.cycles-pp.lock_page_memcg
      0.19 ±  8%      +0.0        0.22 ±  4%  perf-profile.children.cycles-pp.get_user_pages_remote
      0.19 ±  8%      +0.0        0.22 ±  4%  perf-profile.children.cycles-pp.__get_user_pages
      0.21 ±  2%      +0.0        0.24 ±  7%  perf-profile.children.cycles-pp.lockref_put_or_lock
      0.16 ±  9%      +0.0        0.20 ±  6%  perf-profile.children.cycles-pp.pipe_write
      0.07 ± 25%      +0.0        0.11 ±  8%  perf-profile.children.cycles-pp.security_task_kill
      0.36 ±  4%      +0.0        0.41 ±  2%  perf-profile.children.cycles-pp.page_add_file_rmap
      0.03 ±100%      +0.0        0.07 ±  6%  perf-profile.children.cycles-pp.get_signal
      0.18 ± 10%      +0.0        0.22 ± 13%  perf-profile.children.cycles-pp.__check_object_size
      0.18 ±  6%      +0.0        0.22 ± 11%  perf-profile.children.cycles-pp.finish_task_switch
      0.13 ±  9%      +0.0        0.17 ±  7%  perf-profile.children.cycles-pp.arch_stack_walk
      0.20 ±  5%      +0.0        0.24 ±  3%  perf-profile.children.cycles-pp.time
      0.15 ±  3%      +0.0        0.20 ±  9%  perf-profile.children.cycles-pp.dequeue_task_fair
      0.18 ±  4%      +0.0        0.23 ± 14%  perf-profile.children.cycles-pp.__pmd_alloc
      0.29 ±  3%      +0.0        0.33 ±  7%  perf-profile.children.cycles-pp.__x64_sys_sync
      0.29 ±  3%      +0.0        0.33 ±  7%  perf-profile.children.cycles-pp.ksys_sync
      0.07 ± 19%      +0.0        0.11 ± 17%  perf-profile.children.cycles-pp.osq_lock
      0.09 ±  9%      +0.0        0.14 ±  8%  perf-profile.children.cycles-pp.__anon_vma_prepare
      0.31 ±  3%      +0.0        0.36 ±  2%  perf-profile.children.cycles-pp.ptep_clear_flush
      0.23 ±  6%      +0.0        0.28 ±  6%  perf-profile.children.cycles-pp.d_alloc
      0.14 ± 11%      +0.0        0.19 ±  5%  perf-profile.children.cycles-pp.cap_vm_enough_memory
      0.10 ± 12%      +0.0        0.15 ± 14%  perf-profile.children.cycles-pp.dup_userfaultfd
      0.22 ±  6%      +0.1        0.27 ±  7%  perf-profile.children.cycles-pp.__rb_insert_augmented
      0.14 ±  9%      +0.1        0.19 ±  8%  perf-profile.children.cycles-pp.stack_trace_save_tsk
      0.22 ±  4%      +0.1        0.27 ±  8%  perf-profile.children.cycles-pp.schedule_tail
      0.05 ± 57%      +0.1        0.10 ± 10%  perf-profile.children.cycles-pp.cap_mmap_addr
      0.29 ±  2%      +0.1        0.34 ±  8%  perf-profile.children.cycles-pp.sync
      0.16 ±  7%      +0.1        0.21 ±  5%  perf-profile.children.cycles-pp.security_vm_enough_memory_mm
      0.03 ±100%      +0.1        0.08 ±  5%  perf-profile.children.cycles-pp.vma_gap_callbacks_rotate
      0.29 ± 11%      +0.1        0.34 ±  4%  perf-profile.children.cycles-pp.perf_event_mmap_output
      0.26 ±  5%      +0.1        0.31 ±  4%  perf-profile.children.cycles-pp.anon_vma_interval_tree_remove
      0.34 ±  6%      +0.1        0.40        perf-profile.children.cycles-pp.new_sync_read
      0.19 ±  3%      +0.1        0.25 ± 10%  perf-profile.children.cycles-pp.__put_user_4
      0.16 ±  9%      +0.1        0.22 ±  7%  perf-profile.children.cycles-pp.__account_scheduler_latency
      0.15 ± 16%      +0.1        0.21 ± 10%  perf-profile.children.cycles-pp.update_curr
      0.32            +0.1        0.38 ±  8%  perf-profile.children.cycles-pp.strnlen_user
      0.22 ±  6%      +0.1        0.28 ±  6%  perf-profile.children.cycles-pp.wait4
      0.20 ±  6%      +0.1        0.26 ±  9%  perf-profile.children.cycles-pp.release_task
      0.07 ± 26%      +0.1        0.13 ± 30%  perf-profile.children.cycles-pp.group_send_sig_info
      0.00            +0.1        0.06 ± 11%  perf-profile.children.cycles-pp.vmacache_update
      0.29 ±  8%      +0.1        0.35 ±  4%  perf-profile.children.cycles-pp.mem_cgroup_try_charge_delay
      0.26 ±  3%      +0.1        0.32 ±  4%  perf-profile.children.cycles-pp.__pagevec_lru_add_fn
      0.20 ±  9%      +0.1        0.27 ±  5%  perf-profile.children.cycles-pp.legitimize_path
      0.37 ±  7%      +0.1        0.43        perf-profile.children.cycles-pp.getname_flags
      0.25 ±  4%      +0.1        0.31 ±  7%  perf-profile.children.cycles-pp.vmacache_find
      0.22 ±  8%      +0.1        0.28 ±  7%  perf-profile.children.cycles-pp.unlazy_walk
      0.49 ±  5%      +0.1        0.56        perf-profile.children.cycles-pp.native_flush_tlb_one_user
      0.35 ±  6%      +0.1        0.42 ±  3%  perf-profile.children.cycles-pp.d_alloc_parallel
      0.25 ±  7%      +0.1        0.32 ±  3%  perf-profile.children.cycles-pp.do_signal
      0.14 ± 12%      +0.1        0.21 ± 24%  perf-profile.children.cycles-pp.kill_pid_info
      0.27 ± 12%      +0.1        0.34 ±  8%  perf-profile.children.cycles-pp.__get_free_pages
      0.00            +0.1        0.07 ± 14%  perf-profile.children.cycles-pp.sched_ttwu_pending
      0.22 ±  7%      +0.1        0.29 ±  6%  perf-profile.children.cycles-pp.free_unref_page_list
      0.29 ±  5%      +0.1        0.36 ±  8%  perf-profile.children.cycles-pp.vma_interval_tree_remove
      0.15 ± 12%      +0.1        0.22 ± 22%  perf-profile.children.cycles-pp.kill_something_info
      0.30 ±  4%      +0.1        0.38 ±  2%  perf-profile.children.cycles-pp.free_pages_and_swap_cache
      0.53 ±  3%      +0.1        0.60        perf-profile.children.cycles-pp.flush_tlb_func_common
      0.16 ± 14%      +0.1        0.24 ± 21%  perf-profile.children.cycles-pp.__x64_sys_kill
      0.24 ±  2%      +0.1        0.32 ±  6%  perf-profile.children.cycles-pp.wait_consider_task
      0.26 ±  5%      +0.1        0.34 ±  6%  perf-profile.children.cycles-pp.do_notify_parent
      0.77 ±  3%      +0.1        0.84 ±  4%  perf-profile.children.cycles-pp._dl_addr
      0.42 ±  4%      +0.1        0.50 ±  6%  perf-profile.children.cycles-pp.__fput
      0.41 ±  6%      +0.1        0.49 ±  2%  perf-profile.children.cycles-pp.__lookup_slow
      0.61 ±  6%      +0.1        0.69 ±  3%  perf-profile.children.cycles-pp.alloc_pages_vma
      0.58 ±  6%      +0.1        0.67 ±  6%  perf-profile.children.cycles-pp.__alloc_file
      0.31 ±  9%      +0.1        0.39 ±  3%  perf-profile.children.cycles-pp.activate_task
      0.57 ±  4%      +0.1        0.66 ±  3%  perf-profile.children.cycles-pp.lookup_fast
      0.52            +0.1        0.60 ±  4%  perf-profile.children.cycles-pp.anon_vma_interval_tree_insert
      0.28 ±  8%      +0.1        0.37 ±  3%  perf-profile.children.cycles-pp.enqueue_entity
      0.26 ± 11%      +0.1        0.34 ±  2%  perf-profile.children.cycles-pp.ttwu_do_activate
      0.59 ±  6%      +0.1        0.68 ±  6%  perf-profile.children.cycles-pp.alloc_empty_file
      0.32 ±  5%      +0.1        0.41 ±  7%  perf-profile.children.cycles-pp.remove_vma
      0.26 ±  4%      +0.1        0.35 ± 11%  perf-profile.children.cycles-pp.update_load_avg
      0.40 ±  6%      +0.1        0.48 ±  3%  perf-profile.children.cycles-pp.copy_user_highpage
      0.39 ±  8%      +0.1        0.48 ±  3%  perf-profile.children.cycles-pp.copy_page
      0.29 ±  4%      +0.1        0.38 ±  7%  perf-profile.children.cycles-pp.rwsem_optimistic_spin
      0.24 ±  6%      +0.1        0.33 ±  6%  perf-profile.children.cycles-pp.schedule_idle
      0.32 ±  8%      +0.1        0.41 ±  4%  perf-profile.children.cycles-pp.enqueue_task_fair
      0.30 ±  2%      +0.1        0.39        perf-profile.children.cycles-pp.lru_add_drain_cpu
      0.53 ±  3%      +0.1        0.62 ±  5%  perf-profile.children.cycles-pp.vma_merge
      0.30 ±  6%      +0.1        0.39 ±  6%  perf-profile.children.cycles-pp.__list_del_entry_valid
      0.30 ±  2%      +0.1        0.40 ±  2%  perf-profile.children.cycles-pp.lru_add_drain
      0.53 ±  3%      +0.1        0.63 ±  3%  perf-profile.children.cycles-pp.vma_interval_tree_insert
      0.74 ±  3%      +0.1        0.84 ±  4%  perf-profile.children.cycles-pp.alloc_set_pte
      0.44 ±  4%      +0.1        0.54 ±  2%  perf-profile.children.cycles-pp.pagevec_lru_move_fn
      0.40 ±  4%      +0.1        0.50 ±  3%  perf-profile.children.cycles-pp.try_to_wake_up
      0.56 ±  2%      +0.1        0.66 ±  4%  perf-profile.children.cycles-pp.task_work_run
      0.87 ±  4%      +0.1        0.97 ±  3%  perf-profile.children.cycles-pp.unmap_region
      0.59 ±  3%      +0.1        0.70 ±  3%  perf-profile.children.cycles-pp.dput
      0.92 ±  2%      +0.1        1.03 ±  5%  perf-profile.children.cycles-pp.prepare_exit_to_usermode
      0.67 ±  2%      +0.1        0.78 ±  3%  perf-profile.children.cycles-pp.load_script
      0.59 ±  4%      +0.1        0.70 ±  4%  perf-profile.children.cycles-pp.___might_sleep
      0.27 ±  8%      +0.1        0.38 ±  5%  perf-profile.children.cycles-pp.__wake_up_common
      0.33 ±  3%      +0.1        0.44 ±  7%  perf-profile.children.cycles-pp.rwsem_down_write_slowpath
      0.31 ±  9%      +0.1        0.42 ±  6%  perf-profile.children.cycles-pp.__wake_up_common_lock
      0.66 ±  3%      +0.1        0.77 ±  4%  perf-profile.children.cycles-pp.kmem_cache_alloc
      0.71 ±  4%      +0.1        0.83 ±  5%  perf-profile.children.cycles-pp.vfs_statx
      0.63            +0.1        0.75        perf-profile.children.cycles-pp.flush_tlb_mm_range
      0.70 ±  6%      +0.1        0.82 ±  4%  perf-profile.children.cycles-pp.__do_sys_newstat
      0.46 ±  2%      +0.1        0.59 ±  5%  perf-profile.children.cycles-pp.mmap64
      0.53            +0.1        0.65 ±  6%  perf-profile.children.cycles-pp.do_wait
      0.27 ± 12%      +0.1        0.39 ± 12%  perf-profile.children.cycles-pp.run_ksoftirqd
      0.68 ±  4%      +0.1        0.81 ±  5%  perf-profile.children.cycles-pp.select_task_rq_fair
      0.68 ±  2%      +0.1        0.81 ±  4%  perf-profile.children.cycles-pp.__sched_text_start
      0.61 ±  6%      +0.1        0.74 ±  2%  perf-profile.children.cycles-pp.perf_iterate_sb
      0.50 ±  5%      +0.1        0.63 ±  2%  perf-profile.children.cycles-pp.vma_link
      0.55 ±  2%      +0.1        0.68 ±  6%  perf-profile.children.cycles-pp.__do_sys_wait4
      0.54            +0.1        0.68 ±  6%  perf-profile.children.cycles-pp.kernel_wait4
      0.49 ±  8%      +0.1        0.62 ±  7%  perf-profile.children.cycles-pp.wake_up_new_task
      0.90 ±  3%      +0.1        1.04 ±  3%  perf-profile.children.cycles-pp.path_lookupat
      0.69 ±  6%      +0.1        0.82 ±  3%  perf-profile.children.cycles-pp.exit_to_usermode_loop
      0.41 ±  2%      +0.1        0.55 ±  3%  perf-profile.children.cycles-pp.__pte_alloc
      0.31 ±  9%      +0.1        0.45 ± 11%  perf-profile.children.cycles-pp.smpboot_thread_fn
      0.92 ±  4%      +0.1        1.07 ±  3%  perf-profile.children.cycles-pp.filename_lookup
      0.72            +0.1        0.86 ±  4%  perf-profile.children.cycles-pp.copy_strings
      0.97 ±  4%      +0.2        1.12 ±  5%  perf-profile.children.cycles-pp.mprotect_fixup
      0.87 ±  8%      +0.2        1.02 ±  8%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      0.84 ±  2%      +0.2        0.99 ±  7%  perf-profile.children.cycles-pp.__split_vma
      0.48 ±  3%      +0.2        0.64 ±  4%  perf-profile.children.cycles-pp.pte_alloc_one
      1.01 ±  5%      +0.2        1.16 ±  3%  perf-profile.children.cycles-pp.do_mprotect_pkey
      1.01 ±  4%      +0.2        1.17 ±  3%  perf-profile.children.cycles-pp.__x64_sys_mprotect
      0.85 ± 12%      +0.2        1.01 ±  7%  perf-profile.children.cycles-pp.kthread
      0.68 ±  3%      +0.2        0.84 ±  2%  perf-profile.children.cycles-pp.find_vma
      0.39 ±  5%      +0.2        0.56 ±  6%  perf-profile.children.cycles-pp.__slab_free
      1.47            +0.2        1.65 ±  4%  perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
      1.06 ±  2%      +0.2        1.23 ±  4%  perf-profile.children.cycles-pp.link_path_walk
      1.29 ±  3%      +0.2        1.48 ±  4%  perf-profile.children.cycles-pp.__do_munmap
      0.83 ±  6%      +0.2        1.02 ±  4%  perf-profile.children.cycles-pp.clear_page_erms
      0.89 ±  6%      +0.2        1.08 ±  4%  perf-profile.children.cycles-pp.prep_new_page
      0.50 ±  9%      +0.2        0.69 ± 12%  perf-profile.children.cycles-pp.rcu_do_batch
      1.20 ±  4%      +0.2        1.39 ±  2%  perf-profile.children.cycles-pp.setlocale
      1.84 ±  5%      +0.2        2.04 ±  6%  perf-profile.children.cycles-pp.__softirqentry_text_start
      1.11 ±  2%      +0.2        1.31 ±  6%  perf-profile.children.cycles-pp.__vma_adjust
      1.16            +0.2        1.37        perf-profile.children.cycles-pp.walk_component
      0.86 ±  4%      +0.2        1.09 ±  2%  perf-profile.children.cycles-pp.anon_vma_clone
      1.16 ±  3%      +0.2        1.38        perf-profile.children.cycles-pp.wp_page_copy
      1.12 ±  8%      +0.2        1.35 ±  7%  perf-profile.children.cycles-pp.ret_from_fork
      1.36 ±  2%      +0.2        1.59 ±  2%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      1.42 ±  3%      +0.2        1.65 ±  2%  perf-profile.children.cycles-pp.entry_SYSCALL_64
      0.59 ±  2%      +0.2        0.83 ±  6%  perf-profile.children.cycles-pp.kmem_cache_free
      1.18 ±  5%      +0.3        1.44 ±  2%  perf-profile.children.cycles-pp.perf_event_mmap
      0.79 ±  3%      +0.3        1.05 ±  4%  perf-profile.children.cycles-pp.unlink_anon_vmas
      1.09 ±  2%      +0.3        1.35 ±  4%  perf-profile.children.cycles-pp.page_remove_rmap
      1.31            +0.3        1.58 ±  2%  perf-profile.children.cycles-pp.do_wp_page
      1.22 ± 11%      +0.3        1.49 ± 12%  perf-profile.children.cycles-pp.rcu_core
      0.74 ± 13%      +0.3        1.02 ± 19%  perf-profile.children.cycles-pp.scan_positives
      1.36 ±  7%      +0.3        1.65 ±  2%  perf-profile.children.cycles-pp.mmap_region
      1.38 ±  5%      +0.3        1.66        perf-profile.children.cycles-pp.ksys_mmap_pgoff
      1.07 ±  3%      +0.3        1.35 ±  2%  perf-profile.children.cycles-pp.anon_vma_fork
      1.54 ±  6%      +0.3        1.84 ±  2%  perf-profile.children.cycles-pp.do_mmap
      1.17 ± 11%      +0.3        1.47 ± 10%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      1.32 ±  3%      +0.3        1.62 ±  3%  perf-profile.children.cycles-pp.get_page_from_freelist
      0.93 ±  9%      +0.3        1.24 ± 16%  perf-profile.children.cycles-pp.dcache_readdir
      0.95 ±  9%      +0.3        1.27 ± 16%  perf-profile.children.cycles-pp.__x64_sys_getdents
      0.94 ±  9%      +0.3        1.26 ± 16%  perf-profile.children.cycles-pp.iterate_dir
      1.66 ±  6%      +0.3        1.98 ±  2%  perf-profile.children.cycles-pp.vm_mmap_pgoff
      1.49 ±  2%      +0.3        1.81 ±  3%  perf-profile.children.cycles-pp.__alloc_pages_nodemask
      1.16            +0.3        1.49 ±  2%  perf-profile.children.cycles-pp.release_pages
      1.41 ±  5%      +0.3        1.74 ±  3%  perf-profile.children.cycles-pp.copy_p4d_range
      1.47 ±  5%      +0.3        1.81 ±  3%  perf-profile.children.cycles-pp.copy_page_range
      1.54 ±  4%      +0.4        1.91 ±  4%  perf-profile.children.cycles-pp.free_pgtables
      1.80 ±  3%      +0.4        2.21 ±  3%  perf-profile.children.cycles-pp.flush_old_exec
      1.89 ±  2%      +0.4        2.31 ±  2%  perf-profile.children.cycles-pp.do_brk_flags
      1.68            +0.4        2.11        perf-profile.children.cycles-pp.tlb_finish_mmu
      1.65            +0.4        2.08        perf-profile.children.cycles-pp.tlb_flush_mmu
      1.62 ±  9%      +0.5        2.08 ±  9%  perf-profile.children.cycles-pp._raw_spin_lock
      2.48 ±  2%      +0.5        2.94 ±  4%  perf-profile.children.cycles-pp.do_filp_open
      2.45            +0.5        2.92 ±  4%  perf-profile.children.cycles-pp.path_openat
      2.64 ±  2%      +0.5        3.12 ±  3%  perf-profile.children.cycles-pp.do_sys_openat2
      2.66            +0.5        3.13 ±  3%  perf-profile.children.cycles-pp.do_sys_open
      3.78 ±  2%      +0.5        4.32 ±  3%  perf-profile.children.cycles-pp.filemap_map_pages
      2.86 ±  2%      +0.6        3.43 ±  3%  perf-profile.children.cycles-pp.__x64_sys_brk
      3.29 ±  2%      +0.6        3.87 ±  3%  perf-profile.children.cycles-pp.load_elf_binary
      3.32 ±  2%      +0.6        3.91 ±  3%  perf-profile.children.cycles-pp.search_binary_handler
      3.39 ±  2%      +0.9        4.24 ±  2%  perf-profile.children.cycles-pp.unmap_page_range
      4.71            +0.9        5.56 ±  4%  perf-profile.children.cycles-pp.__do_execve_file
      4.76            +0.9        5.62 ±  3%  perf-profile.children.cycles-pp.__x64_sys_execve
      3.61 ±  3%      +0.9        4.47 ±  3%  perf-profile.children.cycles-pp.dup_mm
      3.56            +0.9        4.44 ±  2%  perf-profile.children.cycles-pp.unmap_vmas
      4.79 ±  3%      +1.1        5.88 ±  3%  perf-profile.children.cycles-pp.copy_process
      6.61            +1.1        7.73 ±  2%  perf-profile.children.cycles-pp.handle_pte_fault
      7.11            +1.2        8.27 ±  3%  perf-profile.children.cycles-pp.__handle_mm_fault
      7.38            +1.2        8.58 ±  3%  perf-profile.children.cycles-pp.handle_mm_fault
      5.31 ±  3%      +1.2        6.54 ±  3%  perf-profile.children.cycles-pp._do_fork
      5.31 ±  3%      +1.2        6.55 ±  3%  perf-profile.children.cycles-pp.__x64_sys_clone
      8.17            +1.4        9.54 ±  2%  perf-profile.children.cycles-pp.do_page_fault
      8.70            +1.4       10.11 ±  2%  perf-profile.children.cycles-pp.page_fault
      5.82            +1.6        7.41 ±  2%  perf-profile.children.cycles-pp.do_exit
      5.83            +1.6        7.43 ±  2%  perf-profile.children.cycles-pp.__x64_sys_exit_group
      5.83            +1.6        7.43 ±  2%  perf-profile.children.cycles-pp.do_group_exit
      6.53            +1.8        8.31        perf-profile.children.cycles-pp.exit_mmap
      6.56            +1.8        8.34        perf-profile.children.cycles-pp.mmput
     32.55            +6.8       39.34        perf-profile.children.cycles-pp.do_syscall_64
     32.67            +6.8       39.45        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
     29.37            -6.8       22.55 ±  3%  perf-profile.self.cycles-pp.intel_idle
      7.88 ±  3%      -1.3        6.59 ±  2%  perf-profile.self.cycles-pp.__strncat_chk
      4.27 ±  4%      -0.7        3.61 ±  3%  perf-profile.self.cycles-pp.__strcat_chk
      0.27 ± 12%      -0.1        0.22 ±  7%  perf-profile.self.cycles-pp._find_next_bit
      0.11 ± 13%      -0.0        0.08 ± 24%  perf-profile.self.cycles-pp.__next_timer_interrupt
      0.12 ± 12%      -0.0        0.10 ± 26%  perf-profile.self.cycles-pp.native_sched_clock
      0.14            -0.0        0.12 ±  4%  perf-profile.self.cycles-pp.brk
      0.05 ±  8%      +0.0        0.07 ± 10%  perf-profile.self.cycles-pp.unmap_vmas
      0.15 ±  5%      +0.0        0.17 ±  2%  perf-profile.self.cycles-pp.malloc
      0.17 ±  5%      +0.0        0.18 ±  6%  perf-profile.self.cycles-pp.do_page_fault
      0.09 ±  7%      +0.0        0.11 ±  6%  perf-profile.self.cycles-pp.apparmor_file_open
      0.06 ± 16%      +0.0        0.08        perf-profile.self.cycles-pp.call_rcu
      0.08 ±  5%      +0.0        0.11 ±  7%  perf-profile.self.cycles-pp.common_file_perm
      0.09 ±  7%      +0.0        0.11 ±  9%  perf-profile.self.cycles-pp.unmap_single_vma
      0.07 ± 17%      +0.0        0.09 ±  9%  perf-profile.self.cycles-pp.xas_load
      0.06 ± 17%      +0.0        0.09 ± 13%  perf-profile.self.cycles-pp.___slab_alloc
      0.06 ±  9%      +0.0        0.08 ±  5%  perf-profile.self.cycles-pp.iterate_supers
      0.12 ±  3%      +0.0        0.15 ± 10%  perf-profile.self.cycles-pp.filldir
      0.10 ±  5%      +0.0        0.12 ±  8%  perf-profile.self.cycles-pp.vma_merge
      0.10 ± 10%      +0.0        0.12 ±  4%  perf-profile.self.cycles-pp.copy_user_generic_unrolled
      0.07 ± 15%      +0.0        0.10 ±  8%  perf-profile.self.cycles-pp.fsnotify
      0.09 ±  8%      +0.0        0.12 ±  5%  perf-profile.self.cycles-pp.__sched_text_start
      0.06 ± 14%      +0.0        0.09 ±  5%  perf-profile.self.cycles-pp.__rb_erase_color
      0.29 ±  6%      +0.0        0.32        perf-profile.self.cycles-pp.page_add_file_rmap
      0.11 ±  4%      +0.0        0.14 ±  3%  perf-profile.self.cycles-pp.__list_add_valid
      0.07 ± 12%      +0.0        0.10 ±  8%  perf-profile.self.cycles-pp.__d_alloc
      0.08 ± 19%      +0.0        0.11 ± 13%  perf-profile.self.cycles-pp.cap_vm_enough_memory
      0.04 ± 59%      +0.0        0.08 ±  6%  perf-profile.self.cycles-pp.__legitimize_mnt
      0.13 ±  8%      +0.0        0.16 ±  4%  perf-profile.self.cycles-pp._raw_spin_lock_irq
      0.04 ± 58%      +0.0        0.07 ±  5%  perf-profile.self.cycles-pp.free_pcppages_bulk
      0.16 ±  9%      +0.0        0.19 ±  6%  perf-profile.self.cycles-pp.__pagevec_lru_add_fn
      0.14 ± 10%      +0.0        0.17 ±  4%  perf-profile.self.cycles-pp.memset_erms
      0.12 ±  7%      +0.0        0.15 ±  9%  perf-profile.self.cycles-pp.do_wp_page
      0.21 ±  5%      +0.0        0.26 ±  4%  perf-profile.self.cycles-pp.get_page_from_freelist
      0.13 ±  9%      +0.0        0.17 ± 14%  perf-profile.self.cycles-pp.update_load_avg
      0.03 ±100%      +0.0        0.07 ± 10%  perf-profile.self.cycles-pp.kfree
      0.05 ± 58%      +0.0        0.09 ± 11%  perf-profile.self.cycles-pp.cap_mmap_addr
      0.01 ±173%      +0.0        0.06 ± 14%  perf-profile.self.cycles-pp.copy_page_range
      0.21 ±  5%      +0.0        0.25 ±  5%  perf-profile.self.cycles-pp.__rb_insert_augmented
      0.07 ± 19%      +0.0        0.11 ± 17%  perf-profile.self.cycles-pp.osq_lock
      0.03 ±100%      +0.0        0.07 ± 11%  perf-profile.self.cycles-pp.select_task_rq_fair
      0.01 ±173%      +0.1        0.06 ± 17%  perf-profile.self.cycles-pp.unwind_next_frame
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.switch_mm_irqs_off
      0.10 ± 10%      +0.1        0.15 ± 15%  perf-profile.self.cycles-pp.dup_userfaultfd
      0.31 ±  4%      +0.1        0.36 ±  6%  perf-profile.self.cycles-pp.__vma_adjust
      0.24 ±  8%      +0.1        0.29 ±  5%  perf-profile.self.cycles-pp.anon_vma_interval_tree_remove
      0.17 ±  9%      +0.1        0.22 ±  3%  perf-profile.self.cycles-pp.anon_vma_clone
      0.14 ± 11%      +0.1        0.20 ±  8%  perf-profile.self.cycles-pp.unlink_anon_vmas
      0.14 ±  5%      +0.1        0.20 ±  6%  perf-profile.self.cycles-pp.handle_pte_fault
      0.00            +0.1        0.06 ±  7%  perf-profile.self.cycles-pp.rwsem_down_write_slowpath
      0.00            +0.1        0.06 ±  7%  perf-profile.self.cycles-pp.vmacache_update
      0.01 ±173%      +0.1        0.07        perf-profile.self.cycles-pp.vma_gap_callbacks_rotate
      0.49 ±  5%      +0.1        0.55        perf-profile.self.cycles-pp.native_flush_tlb_one_user
      0.26 ± 10%      +0.1        0.32 ±  7%  perf-profile.self.cycles-pp.perf_iterate_sb
      0.55 ±  4%      +0.1        0.61 ±  3%  perf-profile.self.cycles-pp.swapgs_restore_regs_and_return_to_usermode
      0.37 ±  5%      +0.1        0.43 ±  7%  perf-profile.self.cycles-pp.kmem_cache_alloc
      0.20 ±  7%      +0.1        0.26 ±  6%  perf-profile.self.cycles-pp.perf_event_mmap_output
      0.18 ±  6%      +0.1        0.24 ± 10%  perf-profile.self.cycles-pp.kmem_cache_free
      0.39 ±  4%      +0.1        0.46 ±  5%  perf-profile.self.cycles-pp.find_vma
      0.29 ±  5%      +0.1        0.36 ±  8%  perf-profile.self.cycles-pp.vma_interval_tree_remove
      0.34 ±  3%      +0.1        0.41 ± 11%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.24 ±  5%      +0.1        0.31 ±  8%  perf-profile.self.cycles-pp.vmacache_find
      0.49 ±  4%      +0.1        0.56 ±  6%  perf-profile.self.cycles-pp.find_idlest_group
      0.29 ±  4%      +0.1        0.36 ±  4%  perf-profile.self.cycles-pp.free_pages_and_swap_cache
      0.81 ±  5%      +0.1        0.89 ±  5%  perf-profile.self.cycles-pp.native_irq_return_iret
      0.51            +0.1        0.59 ±  4%  perf-profile.self.cycles-pp.anon_vma_interval_tree_insert
      0.28 ±  4%      +0.1        0.36 ±  7%  perf-profile.self.cycles-pp.perf_event_mmap
      0.38 ±  7%      +0.1        0.47 ±  3%  perf-profile.self.cycles-pp.copy_page
      0.29 ±  6%      +0.1        0.39 ±  6%  perf-profile.self.cycles-pp.__list_del_entry_valid
      0.52 ±  2%      +0.1        0.62 ±  3%  perf-profile.self.cycles-pp.vma_interval_tree_insert
      0.57 ±  4%      +0.1        0.67 ±  4%  perf-profile.self.cycles-pp.___might_sleep
      0.89 ±  2%      +0.1        0.99 ±  5%  perf-profile.self.cycles-pp.prepare_exit_to_usermode
      0.72 ±  6%      +0.2        0.88 ±  5%  perf-profile.self.cycles-pp.copy_p4d_range
      0.39 ±  5%      +0.2        0.55 ±  6%  perf-profile.self.cycles-pp.__slab_free
      0.83 ±  6%      +0.2        1.00 ±  5%  perf-profile.self.cycles-pp.clear_page_erms
      0.70 ±  2%      +0.2        0.88 ±  2%  perf-profile.self.cycles-pp.release_pages
      1.35 ±  3%      +0.2        1.59 ±  2%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      0.98 ±  2%      +0.2        1.22 ±  4%  perf-profile.self.cycles-pp.page_remove_rmap
      1.01 ±  6%      +0.2        1.25        perf-profile.self.cycles-pp._raw_spin_lock
      1.27 ±  2%      +0.3        1.52 ±  2%  perf-profile.self.cycles-pp.entry_SYSCALL_64
      1.16 ± 11%      +0.3        1.46 ± 10%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
      2.28 ±  2%      +0.4        2.65        perf-profile.self.cycles-pp.do_syscall_64
      2.64 ±  2%      +0.4        3.02 ±  4%  perf-profile.self.cycles-pp.filemap_map_pages
      1.91            +0.5        2.39 ±  2%  perf-profile.self.cycles-pp.unmap_page_range


                                                                                
                               reaim.time.user_time                             
                                                                                
  6000 +--------------------------------------------------------------------+   
       | O    O  O O    O    O  O O  O O  O O O  O O  O O  O O  O O         |   
  5000 |.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+.+..+.+..+.+..+.+..+.+..+.+..+.|   
       |                                                                    |   
       |                                                                    |   
  4000 |-+                                                                  |   
       |                                                                    |   
  3000 |-+                                                                  |   
       |                                                                    |   
  2000 |-+                                                                  |   
       |                                                                    |   
       |                                                                    |   
  1000 |-+                                                                  |   
       |                                                                    |   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                              reaim.time.system_time                            
                                                                                
  900 +---------------------------------------------------------------------+   
      |                                                                     |   
  800 |..+.+..O O  O    O    O O .+.+..+.O  O.+..+.+..O O  O.+..+.O .+.+..+.|   
  700 |-+     +.+..+.+..+.+..+.+.        +..+         +.+..+      +.        |   
      |                                                                     |   
  600 |-+                                                                   |   
  500 |-+                                                                   |   
      |                                                                     |   
  400 |-+                                                                   |   
  300 |-+                                                                   |   
      |                                                                     |   
  200 |-+                                                                   |   
  100 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                      reaim.time.percent_of_cpu_this_job_got                    
                                                                                
  2500 +--------------------------------------------------------------------+   
       |                                                                    |   
       |                                                                    |   
  2000 |.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+.+..+.+..+.+..+.+..+.+..+.+..+.|   
       |                                                                    |   
       |                                                                    |   
  1500 |-+                                                                  |   
       |                                                                    |   
  1000 |-+                                                                  |   
       |                                                                    |   
       |                                                                    |   
   500 |-+                                                                  |   
       |                                                                    |   
       |                                                                    |   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                             reaim.time.minor_page_faults                       
                                                                                
  1.2e+08 +-----------------------------------------------------------------+   
          |           +.+..+.+.+..+.+           +                           |   
    1e+08 |-+                                                               |   
          |                                                                 |   
          |                                                                 |   
    8e+07 |-+                                                               |   
          |                                                                 |   
    6e+07 |-+                                                               |   
          |                                                                 |   
    4e+07 |-+                                                               |   
          |                                                                 |   
          |                                                                 |   
    2e+07 |-+                                                               |   
          |                                                                 |   
        0 +-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                         reaim.time.voluntary_context_switches                  
                                                                                
    4e+06 +-----------------------------------------------------------------+   
          | O    O O  O    O   O  O O O  O O O  O O O  O O  O O O  O        |   
  3.5e+06 |.+..+.+.+..+.+..+.+.+..+.+.+..+.+.+..+.+.+..+.+..+.+.+..+.+.+..+.|   
    3e+06 |-+                                                               |   
          |                                                                 |   
  2.5e+06 |-+                                                               |   
          |                                                                 |   
    2e+06 |-+                                                               |   
          |                                                                 |   
  1.5e+06 |-+                                                               |   
    1e+06 |-+                                                               |   
          |                                                                 |   
   500000 |-+                                                               |   
          |                                                                 |   
        0 +-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                       reaim.time.involuntary_context_switches                  
                                                                                
  800000 +------------------------------------------------------------------+   
         | O    O  O O                            O O O  O O  O O O         |   
  700000 |-+              O    O O O  O O  O O O                            |   
         |                                                                  |   
  600000 |-+        .+.                                                     |   
  500000 |.+..+.+..+   +..+.+..+.+.+..+.+..+.+.+..+.+.+..+.+..+.+.+..+.+..+.|   
         |                                                                  |   
  400000 |-+                                                                |   
         |                                                                  |   
  300000 |-+                                                                |   
  200000 |-+                                                                |   
         |                                                                  |   
  100000 |-+                                                                |   
         |                                                                  |   
       0 +------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                 reaim.parent_time                              
                                                                                
  1.2 +---------------------------------------------------------------------+   
      |                                                                     |   
    1 |..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.|   
      |  O    O O  O    O    O O  O O  O O  O O  O O  O O  O O  O O         |   
      |                                                                     |   
  0.8 |-+                                                                   |   
      |                                                                     |   
  0.6 |-+                                                                   |   
      |                                                                     |   
  0.4 |-+                                                                   |   
      |                                                                     |   
      |                                                                     |   
  0.2 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                 reaim.jobs_per_min                             
                                                                                
  450000 +------------------------------------------------------------------+   
         |.+..+.+..+.+.+..+.+..+.+.+..+.+..+.+.+..+.+.+..+.+..+.+.+..+.+..+.|   
  400000 |-+                                                                |   
  350000 |-+                                                                |   
         |                                                                  |   
  300000 |-+                                                                |   
  250000 |-+                                                                |   
         |                                                                  |   
  200000 |-+                                                                |   
  150000 |-+                                                                |   
         |                                                                  |   
  100000 |-+                                                                |   
   50000 |-+                                                                |   
         |                                                                  |   
       0 +------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                             reaim.jobs_per_min_child                           
                                                                                
  7000 +--------------------------------------------------------------------+   
       |                                                                    |   
  6000 |.+..+.+..+.O    O    O  O.+..+.+..+.+.+..+.+..+.+..+.+..+.+..+.+..+.|   
       |           +..+.+..+.+..+                                           |   
  5000 |-+                                                                  |   
       |                                                                    |   
  4000 |-+                                                                  |   
       |                                                                    |   
  3000 |-+                                                                  |   
       |                                                                    |   
  2000 |-+                                                                  |   
       |                                                                    |   
  1000 |-+                                                                  |   
       |                                                                    |   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                 reaim.std_dev_time                             
                                                                                
  0.035 +-------------------------------------------------------------------+   
        |                                                                   |   
   0.03 |.+..+.  .+.+..+.+.+..+.+..+.+..+. .+..+.+..+.  .+.+.+..+.+..+.+..+.|   
        |      +.                         +           +.                    |   
  0.025 |-+                                                                 |   
        |      O              O O  O O    O O  O O  O    O O O    O         |   
   0.02 |-O       O O    O              O             O         O           |   
        |                                                                   |   
  0.015 |-+                                                                 |   
        |                                                                   |   
   0.01 |-+                                                                 |   
        |                                                                   |   
  0.005 |-+                                                                 |   
        |                                                                   |   
      0 +-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                               reaim.std_dev_percent                            
                                                                                
  3.5 +---------------------------------------------------------------------+   
      |   .+..                        .+.         .+..    .+.+..      .+.. .|   
    3 |..+    +.+..+.+..+.+..+.+..+.+.   +..+.+..+    +.+.      +.+..+    + |   
      |                                                                     |   
  2.5 |-+O    O O  O    O    O O  O O  O O  O O  O O  O O  O O  O O         |   
      |                                                                     |   
    2 |-+                                                                   |   
      |                                                                     |   
  1.5 |-+                                                                   |   
      |                                                                     |   
    1 |-+                                                                   |   
      |                                                                     |   
  0.5 |-+                                                                   |   
      |                                                                     |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                               reaim.max_jobs_per_min                           
                                                                                
  500000 +------------------------------------------------------------------+   
  450000 |-+                       O                                        |   
         |.+..+.+..+.+.+..+.+..+.+.+..+.+..+.+.+..+.+.+..+.+..+.+.+..+.+..+.|   
  400000 |-+                                                                |   
  350000 |-+                                                                |   
         |                                                                  |   
  300000 |-+                                                                |   
  250000 |-+                                                                |   
  200000 |-+                                                                |   
         |                                                                  |   
  150000 |-+                                                                |   
  100000 |-+                                                                |   
         |                                                                  |   
   50000 |-+                                                                |   
       0 +------------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample



Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Oliver Sang


View attachment "config-5.6.0-rc3-00043-g289de35984815" of type "text/plain" (203539 bytes)

View attachment "job-script" of type "text/plain" (7714 bytes)

View attachment "job.yaml" of type "text/plain" (5331 bytes)

View attachment "reproduce" of type "text/plain" (10343 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ