lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Date:   Mon, 27 Jul 2020 16:02:33 +0800
From:   kernel test robot <rong.a.chen@...el.com>
To:     Kan Liang <kan.liang@...ux.intel.com>
Cc:     Peter Zijlstra <peterz@...radead.org>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org
Subject: [perf/x86/intel/lbr] 47125db27e: will-it-scale.per_process_ops
 229.0% improvement

Greeting,

FYI, we noticed a 229.0% improvement of will-it-scale.per_process_ops due to commit:


commit: 47125db27e47e9d44c878bf8925aa057824bb0d5 ("perf/x86/intel/lbr: Support Architectural LBR")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master


in testcase: will-it-scale
on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
with following parameters:

	nr_task: 50%
	mode: process
	test: context_switch1
	cpufreq_governor: performance
	ucode: 0x5002f01

test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
test-url: https://github.com/antonblanchard/will-it-scale





Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
  gcc-9/performance/x86_64-rhel-8.3/process/50%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2ap3/context_switch1/will-it-scale/0x5002f01

commit: 
  631618a0dc ("perf/x86/intel/lbr: Factor out intel_pmu_store_lbr")
  47125db27e ("perf/x86/intel/lbr: Support Architectural LBR")

631618a0dca31dc2 47125db27e47e9d44c878bf8925 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    103559          +229.0%     340664 ±  2%  will-it-scale.per_process_ops
   9941798          +229.0%   32703865 ±  2%  will-it-scale.workload
     13473 ± 22%     -27.1%       9827 ± 17%  numa-meminfo.node0.Mapped
     70.70           -23.3       47.44 ±  3%  mpstat.cpu.all.idle%
      0.89 ±  2%      +0.6        1.45 ± 15%  mpstat.cpu.all.irq%
      0.23 ±  2%      +0.0        0.27 ±  5%  mpstat.cpu.all.soft%
     26.97 ±  4%     +21.1       48.04 ±  3%  mpstat.cpu.all.sys%
      1.22 ±  3%      +1.6        2.80 ±  8%  mpstat.cpu.all.usr%
     70.75           -33.6%      47.00 ±  3%  vmstat.cpu.id
     27.25 ±  3%     +80.7%      49.25 ±  2%  vmstat.cpu.sy
    112.00           +13.8%     127.50        vmstat.procs.r
  19604811          +229.9%   64683174 ±  2%  vmstat.system.cs
   1403115           +88.3%    2641647 ±  3%  vmstat.system.in
 7.613e+08 ±  2%     -53.2%  3.561e+08 ± 11%  cpuidle.C1.time
 2.694e+08           -29.1%  1.909e+08 ±  4%  cpuidle.C1.usage
 8.264e+08 ± 12%     -49.2%  4.197e+08 ± 57%  cpuidle.C1E.time
 1.459e+08 ±  2%     -45.3%   79761080 ± 26%  cpuidle.C1E.usage
  1.79e+09          +203.2%  5.427e+09        cpuidle.POLL.time
 7.735e+08 ±  2%    +687.9%  6.095e+09 ±  3%  cpuidle.POLL.usage
      7521 ±  2%      -6.3%       7046        proc-vmstat.nr_inactive_anon
     12957 ±  2%      -6.8%      12075        proc-vmstat.nr_mapped
      4256            -0.9%       4217        proc-vmstat.nr_page_table_pages
     34575 ±  2%      +3.8%      35892 ±  2%  proc-vmstat.nr_shmem
      7521 ±  2%      -6.3%       7046        proc-vmstat.nr_zone_inactive_anon
   1101626 ±  5%      -6.9%    1025209        proc-vmstat.pgfree
     27938 ±  7%     +20.0%      33516 ± 10%  softirqs.CPU0.RCU
     21997 ±  7%     +16.5%      25634 ±  9%  softirqs.CPU124.RCU
    118583 ±  3%      -9.8%     106938        softirqs.CPU151.TIMER
    119202 ±  3%     -10.0%     107260        softirqs.CPU155.TIMER
     22453 ±  7%     +17.7%      26436 ±  9%  softirqs.CPU167.RCU
     22652 ±  8%     +15.8%      26225 ± 11%  softirqs.CPU26.RCU
     22721 ±  8%     +16.6%      26495 ± 10%  softirqs.CPU27.RCU
     22628 ±  7%     +17.3%      26542 ± 10%  softirqs.CPU30.RCU
    119410 ±  2%      -9.7%     107865 ±  2%  softirqs.CPU4.TIMER
     22691 ±  7%     +16.1%      26337 ± 10%  softirqs.CPU45.RCU
   3027831 ±  7%     -30.0%    2119774 ±  6%  sched_debug.cfs_rq:/.MIN_vruntime.avg
   9358257           +19.0%   11132870        sched_debug.cfs_rq:/.MIN_vruntime.max
      6409 ±  3%     -13.4%       5549 ±  4%  sched_debug.cfs_rq:/.load.avg
    194819           -89.8%      19853 ± 10%  sched_debug.cfs_rq:/.load.max
     17232           -69.5%       5248 ±  5%  sched_debug.cfs_rq:/.load.stddev
   3027831 ±  7%     -30.0%    2119774 ±  6%  sched_debug.cfs_rq:/.max_vruntime.avg
   9358257           +19.0%   11132870        sched_debug.cfs_rq:/.max_vruntime.max
   8942799           +23.4%   11035155        sched_debug.cfs_rq:/.min_vruntime.avg
   9585519           +17.0%   11214799        sched_debug.cfs_rq:/.min_vruntime.max
   8495484           +23.9%   10525689        sched_debug.cfs_rq:/.min_vruntime.min
     94179 ±  2%     -18.5%      76743 ± 15%  sched_debug.cfs_rq:/.min_vruntime.stddev
      1.17           -14.3%       1.00        sched_debug.cfs_rq:/.nr_running.max
    472.40 ±  2%      +9.6%     517.90        sched_debug.cfs_rq:/.runnable_avg.avg
      0.01          -100.0%       0.00        sched_debug.cfs_rq:/.spread.avg
      2.00          -100.0%       0.00        sched_debug.cfs_rq:/.spread.max
      0.14          -100.0%       0.00        sched_debug.cfs_rq:/.spread.stddev
    621731 ±  9%     -72.1%     173293        sched_debug.cfs_rq:/.spread0.max
     94156 ±  2%     -18.4%      76835 ± 15%  sched_debug.cfs_rq:/.spread0.stddev
    466.74 ±  2%     +10.3%     514.85        sched_debug.cfs_rq:/.util_avg.avg
    309.62 ±  6%     +20.9%     374.42 ±  9%  sched_debug.cfs_rq:/.util_avg.min
    270.50 ±  2%     +21.4%     328.33 ±  2%  sched_debug.cfs_rq:/.util_est_enqueued.avg
     14.51 ±  6%     -14.3%      12.44 ±  5%  sched_debug.cpu.clock.stddev
      1081 ±  6%     +57.9%       1708 ±  3%  sched_debug.cpu.curr->pid.avg
      0.00           -82.7%       0.00 ±  4%  sched_debug.cpu.next_balance.stddev
      0.50 ±  3%     +12.0%       0.56        sched_debug.cpu.nr_running.avg
  15463170          +241.7%   52844217        sched_debug.cpu.nr_switches.avg
  17080536          +217.6%   54244274        sched_debug.cpu.nr_switches.max
  12161959          +303.1%   49022305        sched_debug.cpu.nr_switches.min
   1598075 ±  9%     -37.8%     993691 ± 12%  sched_debug.cpu.nr_switches.stddev
      0.00          -100.0%       0.00        sched_debug.rt_rq:/.rt_nr_migratory.avg
      0.17          -100.0%       0.00        sched_debug.rt_rq:/.rt_nr_migratory.max
      0.01          -100.0%       0.00        sched_debug.rt_rq:/.rt_nr_migratory.stddev
      0.00          -100.0%       0.00        sched_debug.rt_rq:/.rt_nr_running.avg
      0.17          -100.0%       0.00        sched_debug.rt_rq:/.rt_nr_running.max
      0.01          -100.0%       0.00        sched_debug.rt_rq:/.rt_nr_running.stddev
      0.79 ±  4%     -36.4%       0.51 ± 24%  sched_debug.rt_rq:/.rt_time.avg
    152.55 ±  4%     -36.4%      97.01 ± 24%  sched_debug.rt_rq:/.rt_time.max
     10.98 ±  4%     -36.4%       6.98 ± 24%  sched_debug.rt_rq:/.rt_time.stddev
      3.86           -77.0%       0.89 ± 13%  perf-stat.i.MPKI
 2.209e+10          +220.0%  7.069e+10 ±  2%  perf-stat.i.branch-instructions
      1.42            -0.1        1.28        perf-stat.i.branch-miss-rate%
 3.117e+08          +187.6%  8.962e+08 ±  2%  perf-stat.i.branch-misses
     35.55           -20.5       15.02 ±  5%  perf-stat.i.cache-miss-rate%
 1.434e+08           -70.8%   41866609 ±  8%  perf-stat.i.cache-misses
 4.043e+08           -30.4%  2.813e+08 ± 11%  perf-stat.i.cache-references
  19776636          +229.7%   65204675 ±  2%  perf-stat.i.context-switches
      5.59           -69.1%       1.73 ±  3%  perf-stat.i.cpi
 5.878e+11            -1.8%  5.774e+11        perf-stat.i.cpu-cycles
    892.79 ±  2%      -5.9%     840.03 ±  2%  perf-stat.i.cpu-migrations
      4092          +255.0%      14527 ±  5%  perf-stat.i.cycles-between-cache-misses
      0.00 ±  8%      -0.0        0.00 ±  9%  perf-stat.i.dTLB-load-miss-rate%
   1067812 ±  9%     +35.5%    1447062 ±  3%  perf-stat.i.dTLB-load-misses
 2.872e+10          +225.5%  9.346e+10 ±  2%  perf-stat.i.dTLB-loads
      0.00 ± 11%      -0.0        0.00 ± 16%  perf-stat.i.dTLB-store-miss-rate%
     66113 ±  4%     +88.8%     124844        perf-stat.i.dTLB-store-misses
  1.76e+10          +227.7%  5.768e+10 ±  2%  perf-stat.i.dTLB-stores
     93.95            +4.3       98.25        perf-stat.i.iTLB-load-miss-rate%
 1.765e+08          +185.6%  5.042e+08 ±  2%  perf-stat.i.iTLB-load-misses
  11208676           -27.7%    8104966 ± 12%  perf-stat.i.iTLB-loads
 1.051e+11          +220.6%  3.371e+11 ±  2%  perf-stat.i.instructions
    597.97           +12.3%     671.27        perf-stat.i.instructions-per-iTLB-miss
      0.18          +224.9%       0.58 ±  2%  perf-stat.i.ipc
      3.06            -1.8%       3.01        perf-stat.i.metric.GHz
      0.54 ±  2%     +86.9%       1.01 ±  7%  perf-stat.i.metric.K/sec
    358.59          +222.7%       1157 ±  2%  perf-stat.i.metric.M/sec
  15841380 ±  2%     -73.5%    4198978 ±  8%  perf-stat.i.node-load-misses
   1977654 ± 18%     -69.9%     594678 ± 12%  perf-stat.i.node-loads
    248198 ±  2%     +47.5%     366134        perf-stat.i.node-store-misses
     38931 ± 11%     +83.5%      71449 ± 18%  perf-stat.i.node-stores
      3.85           -78.3%       0.83 ± 13%  perf-stat.overall.MPKI
      1.41            -0.1        1.27        perf-stat.overall.branch-miss-rate%
     35.46           -20.5       14.98 ±  5%  perf-stat.overall.cache-miss-rate%
      5.59           -69.4%       1.71 ±  2%  perf-stat.overall.cpi
      4098          +239.8%      13927 ±  8%  perf-stat.overall.cycles-between-cache-misses
      0.00 ±  8%      -0.0        0.00 ±  5%  perf-stat.overall.dTLB-load-miss-rate%
      0.00 ±  5%      -0.0        0.00        perf-stat.overall.dTLB-store-miss-rate%
     94.02            +4.4       98.42        perf-stat.overall.iTLB-load-miss-rate%
    594.98           +12.3%     668.36        perf-stat.overall.instructions-per-iTLB-miss
      0.18          +226.6%       0.58 ±  2%  perf-stat.overall.ipc
   3191992            -2.4%    3114953        perf-stat.overall.path-length
 2.201e+10          +220.3%  7.051e+10 ±  2%  perf-stat.ps.branch-instructions
 3.108e+08          +187.7%  8.941e+08 ±  2%  perf-stat.ps.branch-misses
 1.429e+08           -70.9%   41585017 ±  7%  perf-stat.ps.cache-misses
 4.032e+08           -30.7%  2.794e+08 ± 11%  perf-stat.ps.cache-references
  19721217          +230.0%   65072915 ±  2%  perf-stat.ps.context-switches
 5.858e+11            -1.8%  5.754e+11        perf-stat.ps.cpu-cycles
    881.86 ±  2%      -5.7%     831.78 ±  2%  perf-stat.ps.cpu-migrations
   1051663 ±  9%     +36.6%    1436684 ±  3%  perf-stat.ps.dTLB-load-misses
 2.863e+10          +225.7%  9.326e+10 ±  2%  perf-stat.ps.dTLB-loads
     65793 ±  4%     +88.8%     124213        perf-stat.ps.dTLB-store-misses
 1.755e+10          +227.9%  5.756e+10 ±  2%  perf-stat.ps.dTLB-stores
 1.761e+08          +185.6%   5.03e+08 ±  2%  perf-stat.ps.iTLB-load-misses
  11193309           -28.1%    8045969 ± 12%  perf-stat.ps.iTLB-loads
 1.048e+11          +220.9%  3.362e+11 ±  2%  perf-stat.ps.instructions
  15792947 ±  2%     -73.6%    4170436 ±  8%  perf-stat.ps.node-load-misses
   1968472 ± 18%     -70.1%     589097 ± 12%  perf-stat.ps.node-loads
    246930 ±  2%     +48.0%     365478        perf-stat.ps.node-store-misses
     38759 ± 12%     +83.2%      71007 ± 18%  perf-stat.ps.node-stores
 3.173e+13          +221.0%  1.019e+14 ±  2%  perf-stat.total.instructions
   5184240 ±  7%   +1212.5%   68044223 ± 23%  interrupts.CAL:Function_call_interrupts
     23114 ± 30%    +783.7%     204271 ± 82%  interrupts.CPU0.CAL:Function_call_interrupts
   1609832 ±  2%    +101.6%    3246203 ±  3%  interrupts.CPU0.RES:Rescheduling_interrupts
     27988 ± 50%   +1403.5%     420821 ± 64%  interrupts.CPU1.CAL:Function_call_interrupts
   1611938 ±  2%    +101.6%    3248960 ±  2%  interrupts.CPU1.RES:Rescheduling_interrupts
     38674 ± 66%   +1766.3%     721764 ±118%  interrupts.CPU10.CAL:Function_call_interrupts
   1682994 ±  7%     +86.1%    3132265 ±  7%  interrupts.CPU10.RES:Rescheduling_interrupts
     18604 ± 36%   +2473.0%     478706 ±133%  interrupts.CPU100.CAL:Function_call_interrupts
   1632726 ±  2%     +99.7%    3261252 ±  4%  interrupts.CPU100.RES:Rescheduling_interrupts
     24142 ± 42%   +6650.5%    1629757 ±122%  interrupts.CPU101.CAL:Function_call_interrupts
   1606301 ±  2%     +92.4%    3090680 ±  7%  interrupts.CPU101.RES:Rescheduling_interrupts
     15550 ± 16%    +926.6%     159635 ± 48%  interrupts.CPU102.CAL:Function_call_interrupts
   1631983 ±  2%     +97.2%    3218567 ±  4%  interrupts.CPU102.RES:Rescheduling_interrupts
     15333 ± 17%   +3971.8%     624330 ±124%  interrupts.CPU103.CAL:Function_call_interrupts
   1631925 ±  3%     +98.6%    3241350 ±  6%  interrupts.CPU103.RES:Rescheduling_interrupts
     14201 ± 12%    +971.9%     152216 ± 36%  interrupts.CPU104.CAL:Function_call_interrupts
   1628330 ±  2%     +99.3%    3245198 ±  4%  interrupts.CPU104.RES:Rescheduling_interrupts
     14419 ± 11%    +624.3%     104436 ±  4%  interrupts.CPU105.CAL:Function_call_interrupts
   1625125 ±  2%    +101.1%    3268054 ±  3%  interrupts.CPU105.RES:Rescheduling_interrupts
     13606 ±  7%    +935.3%     140869 ± 35%  interrupts.CPU106.CAL:Function_call_interrupts
   1622271 ±  2%    +100.9%    3258906 ±  8%  interrupts.CPU106.RES:Rescheduling_interrupts
     13166 ±  9%    +871.7%     127935 ± 26%  interrupts.CPU107.CAL:Function_call_interrupts
   1633548 ±  2%     +95.7%    3196094 ±  6%  interrupts.CPU107.RES:Rescheduling_interrupts
     15630 ± 21%    +585.6%     107157 ± 11%  interrupts.CPU108.CAL:Function_call_interrupts
   1632390 ±  2%     +99.6%    3257695 ±  3%  interrupts.CPU108.RES:Rescheduling_interrupts
     24706 ± 76%    +355.1%     112437 ± 12%  interrupts.CPU109.CAL:Function_call_interrupts
   1620176 ±  4%    +101.1%    3258533 ±  3%  interrupts.CPU109.RES:Rescheduling_interrupts
     20988 ± 18%   +2271.1%     497643 ±102%  interrupts.CPU11.CAL:Function_call_interrupts
   1714431 ± 10%     +80.6%    3095645 ±  7%  interrupts.CPU11.RES:Rescheduling_interrupts
     12969 ±  8%   +1097.5%     155314 ± 58%  interrupts.CPU110.CAL:Function_call_interrupts
   1635212 ±  2%     +93.9%    3170787 ±  4%  interrupts.CPU110.RES:Rescheduling_interrupts
     16611 ± 34%   +1055.7%     191971 ± 47%  interrupts.CPU111.CAL:Function_call_interrupts
   1621839 ±  3%    +108.5%    3381167 ±  5%  interrupts.CPU111.RES:Rescheduling_interrupts
     17018 ± 31%    +586.1%     116761 ± 14%  interrupts.CPU112.CAL:Function_call_interrupts
   1621423 ±  3%    +107.9%    3371743 ±  4%  interrupts.CPU112.RES:Rescheduling_interrupts
     21900 ± 63%    +442.4%     118797 ± 22%  interrupts.CPU113.CAL:Function_call_interrupts
   1638945 ±  2%     +97.1%    3230870 ±  5%  interrupts.CPU113.RES:Rescheduling_interrupts
     13838 ±  6%   +2590.8%     372355 ±123%  interrupts.CPU114.CAL:Function_call_interrupts
   1628974 ±  3%     +95.4%    3182842        interrupts.CPU114.RES:Rescheduling_interrupts
     13836 ±  3%   +1150.1%     172976 ± 73%  interrupts.CPU115.CAL:Function_call_interrupts
   1633079 ±  2%    +100.3%    3270265 ±  7%  interrupts.CPU115.RES:Rescheduling_interrupts
     13690 ±  2%   +2874.4%     407223 ±127%  interrupts.CPU116.CAL:Function_call_interrupts
   1152895 ±  3%    +175.0%    3170306 ±  5%  interrupts.CPU116.RES:Rescheduling_interrupts
     14542 ± 10%   +1535.7%     237868 ± 84%  interrupts.CPU117.CAL:Function_call_interrupts
   1603464 ±  2%    +101.1%    3223982 ±  2%  interrupts.CPU117.RES:Rescheduling_interrupts
     14131 ± 12%    +782.7%     124741 ± 17%  interrupts.CPU118.CAL:Function_call_interrupts
   1628643 ±  3%     +99.5%    3249384 ±  2%  interrupts.CPU118.RES:Rescheduling_interrupts
     13301 ±  5%    +785.5%     117790 ± 16%  interrupts.CPU119.CAL:Function_call_interrupts
   1634627 ±  3%     +99.6%    3261915 ±  2%  interrupts.CPU119.RES:Rescheduling_interrupts
     24308 ± 33%    +460.5%     136251 ± 24%  interrupts.CPU12.CAL:Function_call_interrupts
   1628528 ±  3%    +102.8%    3303215 ±  2%  interrupts.CPU12.RES:Rescheduling_interrupts
     21042 ± 29%    +743.2%     177437 ± 47%  interrupts.CPU120.CAL:Function_call_interrupts
   1805707           +82.6%    3297063        interrupts.CPU120.RES:Rescheduling_interrupts
     31325 ± 32%   +2486.9%     810344 ±114%  interrupts.CPU121.CAL:Function_call_interrupts
   1767716 ±  2%     +77.1%    3130853 ±  6%  interrupts.CPU121.RES:Rescheduling_interrupts
     33178 ± 78%    +728.4%     274851 ±101%  interrupts.CPU122.CAL:Function_call_interrupts
   1792381           +80.0%    3226199 ±  6%  interrupts.CPU122.RES:Rescheduling_interrupts
     17441 ±  9%    +896.8%     173863 ± 60%  interrupts.CPU123.CAL:Function_call_interrupts
   1801194           +82.0%    3277755 ±  2%  interrupts.CPU123.RES:Rescheduling_interrupts
     58899 ± 50%     +95.1%     114937 ±  7%  interrupts.CPU124.CAL:Function_call_interrupts
   1646473          +102.9%    3341104 ±  4%  interrupts.CPU124.RES:Rescheduling_interrupts
     23063 ± 53%    +794.1%     206206 ± 71%  interrupts.CPU125.CAL:Function_call_interrupts
   1787557           +80.2%    3220507 ±  4%  interrupts.CPU125.RES:Rescheduling_interrupts
     21818 ± 20%    +568.6%     145878 ± 35%  interrupts.CPU126.CAL:Function_call_interrupts
   1784883           +74.1%    3108026 ±  9%  interrupts.CPU126.RES:Rescheduling_interrupts
     19281 ± 25%   +3426.9%     680044 ±132%  interrupts.CPU127.CAL:Function_call_interrupts
   1796825           +76.7%    3175580 ±  6%  interrupts.CPU127.RES:Rescheduling_interrupts
     25791 ± 34%    +467.3%     146320 ± 45%  interrupts.CPU128.CAL:Function_call_interrupts
   1793050           +83.3%    3286204 ±  2%  interrupts.CPU128.RES:Rescheduling_interrupts
     30218 ± 91%    +449.8%     166130 ± 56%  interrupts.CPU129.CAL:Function_call_interrupts
   1774076           +89.2%    3357041 ±  4%  interrupts.CPU129.RES:Rescheduling_interrupts
     41634 ± 48%    +444.1%     226524 ± 39%  interrupts.CPU13.CAL:Function_call_interrupts
   1614561 ±  3%    +102.2%    3264130 ±  2%  interrupts.CPU13.RES:Rescheduling_interrupts
     17190 ± 21%    +592.8%     119086 ± 13%  interrupts.CPU130.CAL:Function_call_interrupts
   1788144           +87.8%    3357977 ±  2%  interrupts.CPU130.RES:Rescheduling_interrupts
     24752 ± 57%    +377.3%     118134 ± 18%  interrupts.CPU131.CAL:Function_call_interrupts
   1781788           +77.7%    3165737 ±  5%  interrupts.CPU131.RES:Rescheduling_interrupts
     14835 ± 11%   +1366.5%     217559 ± 42%  interrupts.CPU132.CAL:Function_call_interrupts
   1796911           +84.4%    3313811        interrupts.CPU132.RES:Rescheduling_interrupts
     27294 ± 59%    +312.9%     112685 ± 12%  interrupts.CPU133.CAL:Function_call_interrupts
   1815805           +81.0%    3287246 ±  2%  interrupts.CPU133.RES:Rescheduling_interrupts
     19595 ± 25%    +834.3%     183074 ± 51%  interrupts.CPU134.CAL:Function_call_interrupts
   1798271           +81.2%    3258819 ±  7%  interrupts.CPU134.RES:Rescheduling_interrupts
     21551 ± 42%    +400.7%     107910 ± 12%  interrupts.CPU135.CAL:Function_call_interrupts
   1785713           +79.1%    3198759 ±  4%  interrupts.CPU135.RES:Rescheduling_interrupts
     21378 ± 44%    +439.1%     115245 ± 27%  interrupts.CPU136.CAL:Function_call_interrupts
   1786095           +76.9%    3160051 ±  8%  interrupts.CPU136.RES:Rescheduling_interrupts
     30622 ± 63%    +312.6%     126364 ± 20%  interrupts.CPU137.CAL:Function_call_interrupts
   1779976           +82.6%    3250486 ±  3%  interrupts.CPU137.RES:Rescheduling_interrupts
     39742 ± 66%    +220.9%     127536 ± 14%  interrupts.CPU138.CAL:Function_call_interrupts
   1786717           +86.0%    3324062 ±  4%  interrupts.CPU138.RES:Rescheduling_interrupts
     21634 ± 52%    +543.0%     139099 ± 24%  interrupts.CPU139.CAL:Function_call_interrupts
   1797282           +72.7%    3103860 ±  6%  interrupts.CPU139.RES:Rescheduling_interrupts
     20344 ± 32%   +3106.1%     652256 ± 95%  interrupts.CPU14.CAL:Function_call_interrupts
   1619213 ±  3%     +99.0%    3222255 ±  3%  interrupts.CPU14.RES:Rescheduling_interrupts
     23403 ± 34%    +375.3%     111235 ± 13%  interrupts.CPU140.CAL:Function_call_interrupts
   1793787           +82.8%    3278822 ±  3%  interrupts.CPU140.RES:Rescheduling_interrupts
     17434 ± 27%    +680.4%     136061 ± 35%  interrupts.CPU141.CAL:Function_call_interrupts
   1780477           +80.7%    3216717 ±  3%  interrupts.CPU141.RES:Rescheduling_interrupts
     15801 ± 11%    +660.8%     120211 ± 13%  interrupts.CPU142.CAL:Function_call_interrupts
   1797839           +82.9%    3288547 ±  5%  interrupts.CPU142.RES:Rescheduling_interrupts
     23675 ± 48%    +410.2%     120787 ± 16%  interrupts.CPU143.CAL:Function_call_interrupts
   1782023           +83.3%    3266023 ±  2%  interrupts.CPU143.RES:Rescheduling_interrupts
     20107 ± 22%    +567.9%     134297 ± 33%  interrupts.CPU144.CAL:Function_call_interrupts
   1680114 ±  3%     +99.6%    3353154        interrupts.CPU144.RES:Rescheduling_interrupts
     32484 ± 15%    +640.4%     240525 ± 73%  interrupts.CPU145.CAL:Function_call_interrupts
   1641443 ±  2%    +101.7%    3310491 ±  4%  interrupts.CPU145.RES:Rescheduling_interrupts
     37860 ± 55%    +713.4%     307951 ± 75%  interrupts.CPU146.CAL:Function_call_interrupts
   1666994 ±  3%    +100.6%    3344522 ±  2%  interrupts.CPU146.RES:Rescheduling_interrupts
     24014 ± 20%   +1767.2%     448397 ± 55%  interrupts.CPU147.CAL:Function_call_interrupts
   1673447 ±  3%    +100.5%    3354764 ±  4%  interrupts.CPU147.RES:Rescheduling_interrupts
   1526160 ±  3%    +116.1%    3297297 ±  3%  interrupts.CPU148.RES:Rescheduling_interrupts
     19691 ± 17%   +1189.0%     253816 ± 56%  interrupts.CPU149.CAL:Function_call_interrupts
   1676558 ±  3%     +96.4%    3292037 ±  4%  interrupts.CPU149.RES:Rescheduling_interrupts
     17654 ± 25%    +719.4%     144656 ± 28%  interrupts.CPU15.CAL:Function_call_interrupts
   1626806 ±  3%     +94.6%    3165938 ±  5%  interrupts.CPU15.RES:Rescheduling_interrupts
     21669 ± 25%   +1900.2%     433431 ± 77%  interrupts.CPU150.CAL:Function_call_interrupts
   1670579 ±  2%     +93.3%    3228957 ±  2%  interrupts.CPU150.RES:Rescheduling_interrupts
     28628 ± 59%   +1022.3%     321303 ± 72%  interrupts.CPU151.CAL:Function_call_interrupts
   1677719 ±  2%    +104.1%    3423587        interrupts.CPU151.RES:Rescheduling_interrupts
     22993 ± 38%   +2737.7%     652473 ±103%  interrupts.CPU152.CAL:Function_call_interrupts
   1681706 ±  3%     +98.6%    3339093 ±  7%  interrupts.CPU152.RES:Rescheduling_interrupts
     17506 ± 19%    +953.0%     184333 ± 31%  interrupts.CPU153.CAL:Function_call_interrupts
   1681430 ±  3%     +91.9%    3227305        interrupts.CPU153.RES:Rescheduling_interrupts
     25960 ± 53%    +663.7%     198264 ± 53%  interrupts.CPU154.CAL:Function_call_interrupts
   1676892 ±  3%     +94.5%    3262163 ±  5%  interrupts.CPU154.RES:Rescheduling_interrupts
     16956 ± 35%    +837.9%     159027 ± 21%  interrupts.CPU155.CAL:Function_call_interrupts
   1675147 ±  3%     +97.0%    3299361 ±  5%  interrupts.CPU155.RES:Rescheduling_interrupts
     16566 ± 21%    +803.7%     149705 ± 23%  interrupts.CPU156.CAL:Function_call_interrupts
   1679820 ±  3%    +100.7%    3371697 ±  7%  interrupts.CPU156.RES:Rescheduling_interrupts
     18434 ± 25%    +565.2%     122636 ± 11%  interrupts.CPU157.CAL:Function_call_interrupts
   1707515 ±  3%     +95.8%    3343901 ±  5%  interrupts.CPU157.RES:Rescheduling_interrupts
     18292 ± 40%   +1690.2%     327468 ±117%  interrupts.CPU158.CAL:Function_call_interrupts
   1675990 ±  3%     +96.7%    3296451        interrupts.CPU158.RES:Rescheduling_interrupts
     33291 ± 55%    +955.7%     351475 ±121%  interrupts.CPU159.CAL:Function_call_interrupts
   1657487 ±  3%     +96.9%    3263522 ±  4%  interrupts.CPU159.RES:Rescheduling_interrupts
     21694 ± 59%    +777.5%     190380 ± 66%  interrupts.CPU16.CAL:Function_call_interrupts
   1617430 ±  3%    +109.6%    3390015 ±  4%  interrupts.CPU16.RES:Rescheduling_interrupts
     17668 ± 21%    +554.8%     115691 ± 10%  interrupts.CPU160.CAL:Function_call_interrupts
   1682752 ±  3%     +93.4%    3254569        interrupts.CPU160.RES:Rescheduling_interrupts
     19485 ± 42%    +923.8%     199485 ± 74%  interrupts.CPU161.CAL:Function_call_interrupts
   1678557 ±  3%     +90.1%    3190503 ±  3%  interrupts.CPU161.RES:Rescheduling_interrupts
     15771 ± 21%    +978.9%     170163 ± 51%  interrupts.CPU162.CAL:Function_call_interrupts
   1678868 ±  3%     +88.7%    3168240 ±  2%  interrupts.CPU162.RES:Rescheduling_interrupts
     17679 ± 25%    +793.0%     157874 ± 37%  interrupts.CPU163.CAL:Function_call_interrupts
   1686799 ±  3%     +92.4%    3244800 ±  4%  interrupts.CPU163.RES:Rescheduling_interrupts
     24450 ± 39%    +365.5%     113815 ± 11%  interrupts.CPU164.CAL:Function_call_interrupts
   1678904 ±  3%     +96.9%    3305600 ±  4%  interrupts.CPU164.RES:Rescheduling_interrupts
     17862 ± 40%   +1265.2%     243856 ± 65%  interrupts.CPU165.CAL:Function_call_interrupts
   1678101 ±  3%     +91.3%    3209911 ±  3%  interrupts.CPU165.RES:Rescheduling_interrupts
     18826 ± 27%    +856.0%     179980 ± 70%  interrupts.CPU166.CAL:Function_call_interrupts
   1678671 ±  3%     +92.3%    3227714 ±  5%  interrupts.CPU166.RES:Rescheduling_interrupts
     16639 ± 32%    +736.1%     139127 ± 32%  interrupts.CPU167.CAL:Function_call_interrupts
   1683743 ±  3%     +95.5%    3291157 ±  3%  interrupts.CPU167.RES:Rescheduling_interrupts
     66457 ±  9%   +2660.7%    1834708 ± 41%  interrupts.CPU168.CAL:Function_call_interrupts
   1253445 ±  2%    +147.6%    3102989 ±  2%  interrupts.CPU168.RES:Rescheduling_interrupts
     58357 ± 20%    +758.8%     501166 ± 72%  interrupts.CPU169.CAL:Function_call_interrupts
   1245723 ±  3%    +150.1%    3116060 ±  4%  interrupts.CPU169.RES:Rescheduling_interrupts
     17122 ± 14%   +1409.6%     258480 ± 39%  interrupts.CPU17.CAL:Function_call_interrupts
   1588249 ±  2%    +109.0%    3319670 ±  3%  interrupts.CPU17.RES:Rescheduling_interrupts
     36393 ± 16%   +1818.4%     698160 ± 58%  interrupts.CPU170.CAL:Function_call_interrupts
   1269588 ±  3%    +146.6%    3131275 ±  5%  interrupts.CPU170.RES:Rescheduling_interrupts
     33142 ± 19%   +1864.5%     651075 ± 76%  interrupts.CPU171.CAL:Function_call_interrupts
   1274853 ±  3%    +150.8%    3197508 ±  5%  interrupts.CPU171.RES:Rescheduling_interrupts
     40934 ± 20%   +1804.3%     779530 ±102%  interrupts.CPU172.CAL:Function_call_interrupts
   1275632 ±  3%    +148.5%    3169657 ±  2%  interrupts.CPU172.RES:Rescheduling_interrupts
     40500 ± 39%   +2490.7%    1049244 ± 82%  interrupts.CPU173.CAL:Function_call_interrupts
   1267813 ±  3%    +144.6%    3101517 ±  4%  interrupts.CPU173.RES:Rescheduling_interrupts
     43807 ± 27%   +1655.2%     768915 ± 53%  interrupts.CPU174.CAL:Function_call_interrupts
   1255219 ±  3%    +152.3%    3167208 ±  3%  interrupts.CPU174.RES:Rescheduling_interrupts
     66899 ± 63%    +609.9%     474895 ± 71%  interrupts.CPU175.CAL:Function_call_interrupts
   1258878          +155.4%    3215474 ±  2%  interrupts.CPU175.RES:Rescheduling_interrupts
     29561 ± 78%   +1796.0%     560489 ±109%  interrupts.CPU176.CAL:Function_call_interrupts
   1277008 ±  2%    +156.4%    3274505        interrupts.CPU176.RES:Rescheduling_interrupts
     18856 ± 33%   +1709.4%     341197 ± 77%  interrupts.CPU177.CAL:Function_call_interrupts
   1282354 ±  3%    +155.6%    3277586 ±  3%  interrupts.CPU177.RES:Rescheduling_interrupts
     25781 ± 54%   +1237.0%     344707 ± 94%  interrupts.CPU178.CAL:Function_call_interrupts
   1270975 ±  2%    +148.8%    3162067 ±  3%  interrupts.CPU178.RES:Rescheduling_interrupts
     23591 ± 47%   +1797.7%     447706 ±122%  interrupts.CPU179.CAL:Function_call_interrupts
   1276919 ±  3%    +157.4%    3286440 ±  3%  interrupts.CPU179.RES:Rescheduling_interrupts
     18563 ± 28%   +1643.3%     323629 ± 47%  interrupts.CPU18.CAL:Function_call_interrupts
   1622149 ±  2%     +99.3%    3232145 ±  4%  interrupts.CPU18.RES:Rescheduling_interrupts
     31892 ± 54%   +1047.8%     366068 ± 90%  interrupts.CPU180.CAL:Function_call_interrupts
    859343 ±  3%    +281.7%    3279862        interrupts.CPU180.RES:Rescheduling_interrupts
     23251 ± 43%   +3183.1%     763373 ±138%  interrupts.CPU181.CAL:Function_call_interrupts
   1244244 ±  3%    +154.7%    3169266 ±  4%  interrupts.CPU181.RES:Rescheduling_interrupts
     30081 ± 70%    +839.6%     282647 ± 66%  interrupts.CPU182.CAL:Function_call_interrupts
   1256238 ±  2%    +151.8%    3163645 ±  3%  interrupts.CPU182.RES:Rescheduling_interrupts
     22808 ± 48%   +2576.2%     610414 ± 91%  interrupts.CPU183.CAL:Function_call_interrupts
   1278185 ±  3%    +146.2%    3146541 ±  5%  interrupts.CPU183.RES:Rescheduling_interrupts
     49641 ± 80%   +1302.1%     696011 ±114%  interrupts.CPU184.CAL:Function_call_interrupts
   1259614 ±  2%    +150.6%    3156869        interrupts.CPU184.RES:Rescheduling_interrupts
     29326 ± 38%   +1313.9%     414651 ± 90%  interrupts.CPU185.CAL:Function_call_interrupts
   1270932 ±  3%    +147.1%    3140826 ±  3%  interrupts.CPU185.RES:Rescheduling_interrupts
     27703 ± 52%   +2247.5%     650335 ±137%  interrupts.CPU186.CAL:Function_call_interrupts
   1273572 ±  3%    +145.7%    3129737 ±  2%  interrupts.CPU186.RES:Rescheduling_interrupts
     36949 ±101%    +411.0%     188808 ± 56%  interrupts.CPU187.CAL:Function_call_interrupts
   1274170 ±  2%    +146.1%    3135286 ±  3%  interrupts.CPU187.RES:Rescheduling_interrupts
     19933 ± 21%   +2906.3%     599252 ±102%  interrupts.CPU188.CAL:Function_call_interrupts
   1370174 ±  3%    +131.0%    3165262        interrupts.CPU188.RES:Rescheduling_interrupts
     19243 ± 29%   +2555.3%     510965 ±134%  interrupts.CPU189.CAL:Function_call_interrupts
   1287245 ±  3%    +146.6%    3174180        interrupts.CPU189.RES:Rescheduling_interrupts
     16769 ± 31%   +1403.6%     252135 ± 57%  interrupts.CPU19.CAL:Function_call_interrupts
   1626803 ±  2%     +98.2%    3223912        interrupts.CPU19.RES:Rescheduling_interrupts
     20916 ± 60%   +2910.1%     629583 ± 81%  interrupts.CPU190.CAL:Function_call_interrupts
   1279779 ±  2%    +130.0%    2943355 ±  6%  interrupts.CPU190.RES:Rescheduling_interrupts
     16578 ± 23%    +469.7%      94445 ± 12%  interrupts.CPU191.CAL:Function_call_interrupts
   1233937 ±  3%    +137.4%    2928901 ±  4%  interrupts.CPU191.RES:Rescheduling_interrupts
     22178 ± 44%   +1270.3%     303901 ±105%  interrupts.CPU2.CAL:Function_call_interrupts
   1620436 ±  3%     +99.5%    3233232 ±  5%  interrupts.CPU2.RES:Rescheduling_interrupts
     25861 ± 27%   +1800.1%     491391 ±127%  interrupts.CPU20.CAL:Function_call_interrupts
   1604597 ±  4%     +98.0%    3177822 ±  3%  interrupts.CPU20.RES:Rescheduling_interrupts
     17895 ± 14%   +1837.6%     346741 ± 44%  interrupts.CPU21.CAL:Function_call_interrupts
   1645660 ±  2%     +91.9%    3158293        interrupts.CPU21.RES:Rescheduling_interrupts
     24667 ± 45%    +612.2%     175681 ± 32%  interrupts.CPU22.CAL:Function_call_interrupts
   1614932 ±  3%     +99.8%    3226734        interrupts.CPU22.RES:Rescheduling_interrupts
     23943 ± 36%    +964.8%     254964 ± 52%  interrupts.CPU23.CAL:Function_call_interrupts
   1628510 ±  3%     +98.5%    3232182 ±  2%  interrupts.CPU23.RES:Rescheduling_interrupts
     18104 ± 23%   +1132.4%     223129 ± 83%  interrupts.CPU24.CAL:Function_call_interrupts
   1804738           +79.1%    3233171        interrupts.CPU24.RES:Rescheduling_interrupts
     20675 ± 53%   +3401.5%     723926 ±132%  interrupts.CPU25.CAL:Function_call_interrupts
   1797236           +68.1%    3020982 ±  9%  interrupts.CPU25.RES:Rescheduling_interrupts
     15852 ±  9%   +1241.7%     212695 ± 57%  interrupts.CPU26.CAL:Function_call_interrupts
   1801094           +78.8%    3221104 ±  6%  interrupts.CPU26.RES:Rescheduling_interrupts
     17880 ± 28%    +508.6%     108817 ±  5%  interrupts.CPU27.CAL:Function_call_interrupts
   1799409           +83.0%    3293672        interrupts.CPU27.RES:Rescheduling_interrupts
     17792 ± 10%    +940.9%     185203 ± 55%  interrupts.CPU28.CAL:Function_call_interrupts
   1852170           +79.1%    3316890 ±  3%  interrupts.CPU28.RES:Rescheduling_interrupts
     15763 ± 13%   +2084.5%     344353 ±107%  interrupts.CPU29.CAL:Function_call_interrupts
   1788269           +81.8%    3250270 ±  4%  interrupts.CPU29.RES:Rescheduling_interrupts
     15527 ± 20%   +1395.7%     232237 ± 56%  interrupts.CPU3.CAL:Function_call_interrupts
   1629660 ±  3%    +100.1%    3260422 ±  3%  interrupts.CPU3.RES:Rescheduling_interrupts
     16209 ± 13%   +4758.5%     787546 ± 90%  interrupts.CPU30.CAL:Function_call_interrupts
   1786443           +78.1%    3180978 ±  5%  interrupts.CPU30.RES:Rescheduling_interrupts
     17180 ± 32%    +878.7%     168149 ± 45%  interrupts.CPU31.CAL:Function_call_interrupts
   1795555           +77.0%    3177864 ±  6%  interrupts.CPU31.RES:Rescheduling_interrupts
     14860 ± 10%   +1245.8%     199990 ± 52%  interrupts.CPU32.CAL:Function_call_interrupts
   1803500           +82.3%    3287095        interrupts.CPU32.RES:Rescheduling_interrupts
     15370 ± 10%   +1752.0%     284666 ±104%  interrupts.CPU33.CAL:Function_call_interrupts
   1778798           +82.3%    3242114 ±  2%  interrupts.CPU33.RES:Rescheduling_interrupts
     17962 ± 24%    +712.2%     145894 ± 35%  interrupts.CPU34.CAL:Function_call_interrupts
   1791223           +88.3%    3373127 ±  4%  interrupts.CPU34.RES:Rescheduling_interrupts
     17091 ± 31%   +3624.7%     636620 ±143%  interrupts.CPU35.CAL:Function_call_interrupts
   1784681           +74.4%    3113325 ±  6%  interrupts.CPU35.RES:Rescheduling_interrupts
     14920 ±  9%   +1165.6%     188831 ± 38%  interrupts.CPU36.CAL:Function_call_interrupts
   1799175           +86.7%    3359879        interrupts.CPU36.RES:Rescheduling_interrupts
     17785 ± 25%    +865.6%     171724 ± 57%  interrupts.CPU37.CAL:Function_call_interrupts
   1778439           +85.6%    3300638        interrupts.CPU37.RES:Rescheduling_interrupts
     17296 ± 19%   +1157.1%     217431 ± 74%  interrupts.CPU38.CAL:Function_call_interrupts
   1789969           +86.4%    3337075 ±  4%  interrupts.CPU38.RES:Rescheduling_interrupts
     16573 ± 13%   +2524.2%     434914 ± 66%  interrupts.CPU39.CAL:Function_call_interrupts
   1780700           +79.9%    3203185 ±  4%  interrupts.CPU39.RES:Rescheduling_interrupts
     22659 ± 35%    +427.7%     119581 ± 16%  interrupts.CPU4.CAL:Function_call_interrupts
   1629788 ±  3%     +96.8%    3206822 ±  7%  interrupts.CPU4.RES:Rescheduling_interrupts
     29699 ± 40%   +3328.9%    1018360 ±146%  interrupts.CPU40.CAL:Function_call_interrupts
   1778756           +78.0%    3165462 ±  7%  interrupts.CPU40.RES:Rescheduling_interrupts
     16669 ± 18%   +1521.7%     270330 ± 86%  interrupts.CPU41.CAL:Function_call_interrupts
   1794332           +80.5%    3239331 ±  3%  interrupts.CPU41.RES:Rescheduling_interrupts
     19400 ± 19%    +733.4%     161686 ± 37%  interrupts.CPU42.CAL:Function_call_interrupts
   1799586           +83.2%    3296134 ±  6%  interrupts.CPU42.RES:Rescheduling_interrupts
     17206 ± 16%   +5907.6%    1033687 ±112%  interrupts.CPU43.CAL:Function_call_interrupts
   1804369           +80.0%    3247316 ±  8%  interrupts.CPU43.RES:Rescheduling_interrupts
     20028 ± 25%    +543.5%     128880 ± 13%  interrupts.CPU44.CAL:Function_call_interrupts
   1799090           +83.1%    3294314 ±  3%  interrupts.CPU44.RES:Rescheduling_interrupts
     40272 ± 74%    +328.2%     172430 ± 63%  interrupts.CPU45.CAL:Function_call_interrupts
   1772486           +83.8%    3257229 ±  4%  interrupts.CPU45.RES:Rescheduling_interrupts
     23832 ± 34%    +949.5%     250123 ± 63%  interrupts.CPU46.CAL:Function_call_interrupts
   1790091           +85.0%    3311811 ±  4%  interrupts.CPU46.RES:Rescheduling_interrupts
     29177 ± 73%    +500.7%     175265 ± 31%  interrupts.CPU47.CAL:Function_call_interrupts
   1778720           +84.7%    3284815 ±  3%  interrupts.CPU47.RES:Rescheduling_interrupts
     17622 ± 45%    +679.5%     137360 ± 26%  interrupts.CPU48.CAL:Function_call_interrupts
   1682898 ±  3%    +106.7%    3478810 ±  6%  interrupts.CPU48.RES:Rescheduling_interrupts
     15222 ± 15%   +1095.3%     181944 ± 69%  interrupts.CPU49.CAL:Function_call_interrupts
   1674150 ±  3%     +94.2%    3251807 ±  4%  interrupts.CPU49.RES:Rescheduling_interrupts
     14944 ± 14%   +1396.5%     223641 ± 80%  interrupts.CPU5.CAL:Function_call_interrupts
   1626984 ±  3%     +88.0%    3058970 ±  5%  interrupts.CPU5.RES:Rescheduling_interrupts
     17267 ± 40%    +514.5%     106101 ±  7%  interrupts.CPU50.CAL:Function_call_interrupts
   1679660 ±  3%     +92.9%    3240373 ±  4%  interrupts.CPU50.RES:Rescheduling_interrupts
     20663 ± 59%    +438.5%     111274 ± 10%  interrupts.CPU51.CAL:Function_call_interrupts
   1676150 ±  4%     +95.7%    3280723 ±  2%  interrupts.CPU51.RES:Rescheduling_interrupts
     16231 ± 17%    +932.0%     167498 ± 53%  interrupts.CPU52.CAL:Function_call_interrupts
   1756376 ±  3%     +87.2%    3287924 ±  3%  interrupts.CPU52.RES:Rescheduling_interrupts
     16358 ± 36%   +1594.9%     277248 ± 63%  interrupts.CPU53.CAL:Function_call_interrupts
      7610 ± 24%     -30.6%       5281 ± 34%  interrupts.CPU53.NMI:Non-maskable_interrupts
      7610 ± 24%     -30.6%       5281 ± 34%  interrupts.CPU53.PMI:Performance_monitoring_interrupts
   1676378 ±  3%     +93.3%    3240139 ±  3%  interrupts.CPU53.RES:Rescheduling_interrupts
     15342 ± 27%   +1186.0%     197306 ± 47%  interrupts.CPU54.CAL:Function_call_interrupts
      7611 ± 24%     -30.6%       5281 ± 34%  interrupts.CPU54.NMI:Non-maskable_interrupts
      7611 ± 24%     -30.6%       5281 ± 34%  interrupts.CPU54.PMI:Performance_monitoring_interrupts
   1681443 ±  3%     +87.3%    3148773 ±  2%  interrupts.CPU54.RES:Rescheduling_interrupts
     22506 ± 73%    +834.4%     210292 ± 84%  interrupts.CPU55.CAL:Function_call_interrupts
      7610 ± 24%     -30.6%       5281 ± 34%  interrupts.CPU55.NMI:Non-maskable_interrupts
      7610 ± 24%     -30.6%       5281 ± 34%  interrupts.CPU55.PMI:Performance_monitoring_interrupts
   1686450 ±  3%     +95.6%    3298311 ±  2%  interrupts.CPU55.RES:Rescheduling_interrupts
     17535 ± 37%    +980.2%     189422 ± 74%  interrupts.CPU56.CAL:Function_call_interrupts
      7608 ± 24%     -30.6%       5281 ± 34%  interrupts.CPU56.NMI:Non-maskable_interrupts
      7608 ± 24%     -30.6%       5281 ± 34%  interrupts.CPU56.PMI:Performance_monitoring_interrupts
   1686800 ±  3%     +85.7%    3132979 ±  5%  interrupts.CPU56.RES:Rescheduling_interrupts
     21073 ± 65%    +498.6%     126147 ± 26%  interrupts.CPU57.CAL:Function_call_interrupts
      7610 ± 24%     -30.6%       5281 ± 34%  interrupts.CPU57.NMI:Non-maskable_interrupts
      7610 ± 24%     -30.6%       5281 ± 34%  interrupts.CPU57.PMI:Performance_monitoring_interrupts
   1686310 ±  3%     +94.9%    3286418 ±  3%  interrupts.CPU57.RES:Rescheduling_interrupts
     15579 ± 24%   +2356.3%     382686 ±105%  interrupts.CPU58.CAL:Function_call_interrupts
      7609 ± 24%     -30.6%       5281 ± 34%  interrupts.CPU58.NMI:Non-maskable_interrupts
      7609 ± 24%     -30.6%       5281 ± 34%  interrupts.CPU58.PMI:Performance_monitoring_interrupts
   1679011 ±  3%     +94.0%    3256903 ±  3%  interrupts.CPU58.RES:Rescheduling_interrupts
     15808 ± 26%   +1850.9%     308401 ±101%  interrupts.CPU59.CAL:Function_call_interrupts
   1666555 ±  3%     +96.5%    3274828 ±  4%  interrupts.CPU59.RES:Rescheduling_interrupts
     23644 ± 29%    +821.0%     217765 ± 56%  interrupts.CPU6.CAL:Function_call_interrupts
   1620707 ±  2%     +95.5%    3168324 ±  3%  interrupts.CPU6.RES:Rescheduling_interrupts
     15687 ± 21%   +1444.8%     242337 ± 57%  interrupts.CPU60.CAL:Function_call_interrupts
   1680407 ±  3%     +97.0%    3311092 ±  5%  interrupts.CPU60.RES:Rescheduling_interrupts
     17912 ± 46%   +1128.7%     220101 ± 79%  interrupts.CPU61.CAL:Function_call_interrupts
   1655815 ±  3%    +108.4%    3451451 ±  3%  interrupts.CPU61.RES:Rescheduling_interrupts
     15466 ± 24%    +598.5%     108036 ±  9%  interrupts.CPU62.CAL:Function_call_interrupts
   1681237 ±  3%     +94.6%    3271680 ±  2%  interrupts.CPU62.RES:Rescheduling_interrupts
     16736 ± 34%   +1887.5%     332634 ±106%  interrupts.CPU63.CAL:Function_call_interrupts
   1669958 ±  3%     +91.2%    3192434 ±  2%  interrupts.CPU63.RES:Rescheduling_interrupts
     29685 ± 42%    +970.9%     317898 ± 73%  interrupts.CPU64.CAL:Function_call_interrupts
   1665969 ±  3%    +101.6%    3359201 ±  5%  interrupts.CPU64.RES:Rescheduling_interrupts
     22248 ± 22%    +540.4%     142474 ± 22%  interrupts.CPU65.CAL:Function_call_interrupts
   1673002 ±  3%     +89.8%    3175265 ±  3%  interrupts.CPU65.RES:Rescheduling_interrupts
     22161 ± 44%   +2027.5%     471479 ± 78%  interrupts.CPU66.CAL:Function_call_interrupts
   1678696 ±  4%     +87.9%    3154107 ±  3%  interrupts.CPU66.RES:Rescheduling_interrupts
     28715 ± 37%   +1304.7%     403357 ± 51%  interrupts.CPU67.CAL:Function_call_interrupts
   1675504 ±  3%     +94.4%    3256617 ±  4%  interrupts.CPU67.RES:Rescheduling_interrupts
     21691 ± 28%    +585.5%     148695 ± 42%  interrupts.CPU68.CAL:Function_call_interrupts
   1680719 ±  2%     +97.2%    3313717 ±  3%  interrupts.CPU68.RES:Rescheduling_interrupts
     29622 ± 16%   +1194.4%     383428 ± 82%  interrupts.CPU69.CAL:Function_call_interrupts
   1666818 ±  2%     +93.5%    3224486 ±  2%  interrupts.CPU69.RES:Rescheduling_interrupts
     17223 ± 16%   +1162.3%     217415 ± 56%  interrupts.CPU7.CAL:Function_call_interrupts
   1630200 ±  3%     +96.9%    3210192 ±  6%  interrupts.CPU7.RES:Rescheduling_interrupts
     18869 ± 20%   +1200.5%     245406 ± 97%  interrupts.CPU70.CAL:Function_call_interrupts
   1681155 ±  3%     +94.9%    3276274 ±  3%  interrupts.CPU70.RES:Rescheduling_interrupts
     23097 ± 38%    +714.0%     188009 ± 51%  interrupts.CPU71.CAL:Function_call_interrupts
   1680428 ±  3%     +97.0%    3309893 ±  4%  interrupts.CPU71.RES:Rescheduling_interrupts
     37063 ± 47%   +1181.4%     474933 ± 96%  interrupts.CPU72.CAL:Function_call_interrupts
   1268490 ±  3%    +138.7%    3027944 ±  3%  interrupts.CPU72.RES:Rescheduling_interrupts
     29627 ± 34%   +2282.5%     705869 ± 99%  interrupts.CPU73.CAL:Function_call_interrupts
   1271122 ±  3%    +147.8%    3149498 ±  2%  interrupts.CPU73.RES:Rescheduling_interrupts
     45302 ± 54%    +941.3%     471729 ± 76%  interrupts.CPU74.CAL:Function_call_interrupts
   1270311 ±  4%    +149.4%    3168301 ±  5%  interrupts.CPU74.RES:Rescheduling_interrupts
     66385 ± 42%    +641.2%     492066 ± 54%  interrupts.CPU75.CAL:Function_call_interrupts
   1262392 ±  3%    +165.3%    3348710 ±  9%  interrupts.CPU75.RES:Rescheduling_interrupts
     46848 ± 43%   +1044.6%     536209 ± 70%  interrupts.CPU76.CAL:Function_call_interrupts
   1270118 ±  4%    +147.9%    3148909 ±  2%  interrupts.CPU76.RES:Rescheduling_interrupts
     46390 ± 50%    +722.2%     381441 ± 92%  interrupts.CPU77.CAL:Function_call_interrupts
   1260708 ±  4%    +143.8%    3074192 ±  4%  interrupts.CPU77.RES:Rescheduling_interrupts
     36701 ± 41%    +522.3%     228390 ± 48%  interrupts.CPU78.CAL:Function_call_interrupts
   1271167 ±  4%    +147.6%    3147477 ±  4%  interrupts.CPU78.RES:Rescheduling_interrupts
     52449 ± 50%    +446.6%     286690 ± 37%  interrupts.CPU79.CAL:Function_call_interrupts
   1273496 ±  4%    +157.0%    3273041 ±  6%  interrupts.CPU79.RES:Rescheduling_interrupts
     28006 ± 31%    +471.1%     159956 ± 28%  interrupts.CPU8.CAL:Function_call_interrupts
   1610514          +104.3%    3290646 ±  5%  interrupts.CPU8.RES:Rescheduling_interrupts
     41862 ± 41%    +788.2%     371834 ± 74%  interrupts.CPU80.CAL:Function_call_interrupts
   1265909 ±  4%    +156.6%    3247903        interrupts.CPU80.RES:Rescheduling_interrupts
     50189 ± 47%    +631.8%     367294 ±110%  interrupts.CPU81.CAL:Function_call_interrupts
   1220977 ±  4%    +168.6%    3279073 ±  4%  interrupts.CPU81.RES:Rescheduling_interrupts
     39654 ± 45%   +1996.1%     831199 ± 90%  interrupts.CPU82.CAL:Function_call_interrupts
   1254760 ±  4%    +151.5%    3155760 ±  3%  interrupts.CPU82.RES:Rescheduling_interrupts
     49564 ± 44%    +320.3%     208306 ± 55%  interrupts.CPU83.CAL:Function_call_interrupts
   1260074 ±  4%    +162.1%    3303129 ±  3%  interrupts.CPU83.RES:Rescheduling_interrupts
     46077 ± 57%    +813.5%     420934 ± 65%  interrupts.CPU84.CAL:Function_call_interrupts
   1255030 ±  4%    +160.8%    3273315        interrupts.CPU84.RES:Rescheduling_interrupts
     43897 ± 32%    +869.2%     425472 ± 68%  interrupts.CPU85.CAL:Function_call_interrupts
   1303806 ±  3%    +145.6%    3202625 ±  2%  interrupts.CPU85.RES:Rescheduling_interrupts
     52258 ± 36%   +1682.6%     931566 ± 65%  interrupts.CPU86.CAL:Function_call_interrupts
   1252215 ±  3%    +146.3%    3084104 ±  3%  interrupts.CPU86.RES:Rescheduling_interrupts
     67561 ± 71%   +1193.8%     874077 ± 58%  interrupts.CPU87.CAL:Function_call_interrupts
   1257942 ±  4%    +151.8%    3167663 ±  4%  interrupts.CPU87.RES:Rescheduling_interrupts
     48527 ± 19%   +1057.7%     561784 ± 70%  interrupts.CPU88.CAL:Function_call_interrupts
   1259878 ±  3%    +152.3%    3178972        interrupts.CPU88.RES:Rescheduling_interrupts
     52745 ± 31%   +1517.5%     853127 ± 29%  interrupts.CPU89.CAL:Function_call_interrupts
   1256249 ±  3%    +148.8%    3125495 ±  4%  interrupts.CPU89.RES:Rescheduling_interrupts
     18338 ± 14%   +1681.7%     326726 ± 71%  interrupts.CPU9.CAL:Function_call_interrupts
   1626751 ±  3%     +94.9%    3170415 ±  3%  interrupts.CPU9.RES:Rescheduling_interrupts
     72139 ± 39%    +984.5%     782339 ± 84%  interrupts.CPU90.CAL:Function_call_interrupts
   1241205 ±  3%    +152.1%    3128993 ±  4%  interrupts.CPU90.RES:Rescheduling_interrupts
     54166 ± 31%   +2017.2%    1146810 ±103%  interrupts.CPU91.CAL:Function_call_interrupts
   1259962 ±  4%    +150.4%    3155440 ±  3%  interrupts.CPU91.RES:Rescheduling_interrupts
    136488 ± 39%    +469.1%     776698 ± 53%  interrupts.CPU92.CAL:Function_call_interrupts
   1126914 ±  3%    +177.5%    3127348 ±  3%  interrupts.CPU92.RES:Rescheduling_interrupts
     62946 ± 10%    +442.3%     341345 ± 39%  interrupts.CPU93.CAL:Function_call_interrupts
   1249600 ±  4%    +154.0%    3174238 ±  2%  interrupts.CPU93.RES:Rescheduling_interrupts
     51935 ± 40%   +3011.8%    1616142 ± 94%  interrupts.CPU94.CAL:Function_call_interrupts
   1247268 ±  3%    +141.3%    3009576 ±  4%  interrupts.CPU94.RES:Rescheduling_interrupts
     61135 ± 12%   +2339.6%    1491459 ± 53%  interrupts.CPU95.CAL:Function_call_interrupts
   1206574 ±  2%    +148.8%    3001861 ±  3%  interrupts.CPU95.RES:Rescheduling_interrupts
     20839 ± 44%    +588.1%     143406 ± 16%  interrupts.CPU96.CAL:Function_call_interrupts
   1605339 ±  2%    +112.1%    3404230 ±  4%  interrupts.CPU96.RES:Rescheduling_interrupts
     20038 ± 40%   +1251.2%     270757 ± 61%  interrupts.CPU97.CAL:Function_call_interrupts
   1623651 ±  2%     +96.9%    3197212 ±  3%  interrupts.CPU97.RES:Rescheduling_interrupts
     18639 ± 40%   +2408.7%     467610 ± 86%  interrupts.CPU98.CAL:Function_call_interrupts
   1625450 ±  3%    +100.8%    3263095 ±  4%  interrupts.CPU98.RES:Rescheduling_interrupts
     16400 ± 22%    +753.1%     139911 ± 20%  interrupts.CPU99.CAL:Function_call_interrupts
   1630276 ±  2%    +103.7%    3320203 ±  3%  interrupts.CPU99.RES:Rescheduling_interrupts
    169.75 ±  6%    +106.8%     351.00 ±  2%  interrupts.IWI:IRQ_work_interrupts
  3.04e+08 ±  2%    +104.0%  6.202e+08 ±  2%  interrupts.RES:Rescheduling_interrupts
     46.63           -27.8       18.86        perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
     65.88           -19.6       46.28        perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
     66.23           -19.5       46.74        perf-profile.calltrace.cycles-pp.secondary_startup_64
     65.94           -19.4       46.50        perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
     65.96           -19.4       46.55        perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
      9.22            -3.0        6.20 ±  2%  perf-profile.calltrace.cycles-pp.__switch_to
     14.56            -2.6       12.00        perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
     13.23            -1.4       11.78        perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
      0.98            -0.4        0.55 ±  2%  perf-profile.calltrace.cycles-pp.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
      1.30            -0.1        1.21        perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
      1.36            -0.1        1.28        perf-profile.calltrace.cycles-pp.pick_next_task_fair.__sched_text_start.schedule.pipe_read.new_sync_read
      0.56 ±  2%      +0.5        1.03        perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
      0.00            +0.5        0.52 ±  2%  perf-profile.calltrace.cycles-pp.__calc_delta.update_curr.dequeue_entity.dequeue_task_fair.__sched_text_start
      0.00            +0.5        0.53 ±  2%  perf-profile.calltrace.cycles-pp.update_rq_clock.__sched_text_start.schedule.pipe_read.new_sync_read
      0.00            +0.5        0.54 ±  2%  perf-profile.calltrace.cycles-pp.update_cfs_group.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
      0.00            +0.6        0.55        perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.prepare_to_wait_event.pipe_read.new_sync_read.vfs_read
      0.00            +0.6        0.61        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.read
      0.00            +0.6        0.63 ±  2%  perf-profile.calltrace.cycles-pp.finish_task_switch.__sched_text_start.schedule.pipe_read.new_sync_read
      0.00            +0.6        0.63 ±  4%  perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +0.7        0.65        perf-profile.calltrace.cycles-pp.put_prev_task_fair.__sched_text_start.schedule.pipe_read.new_sync_read
      0.00            +0.7        0.66 ±  2%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.write
      0.00            +0.7        0.70 ±  6%  perf-profile.calltrace.cycles-pp.hrtimer_next_event_without.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry
      1.11            +0.7        1.83        perf-profile.calltrace.cycles-pp.prepare_to_wait_event.pipe_read.new_sync_read.vfs_read.ksys_read
      0.00            +0.9        0.88        perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common
      0.00            +0.9        0.89 ±  2%  perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.new_sync_read.vfs_read.ksys_read
      0.25 ±100%      +0.9        1.17        perf-profile.calltrace.cycles-pp.__switch_to.read
      0.00            +1.0        0.96        perf-profile.calltrace.cycles-pp.__switch_to_asm
      0.00            +1.0        0.96        perf-profile.calltrace.cycles-pp.__switch_to_asm.read
      1.00 ±  2%      +1.0        1.97        perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__sched_text_start.schedule
      0.00            +1.0        0.97        perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +1.0        0.98 ±  2%  perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write
      0.00            +1.0        1.01        perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
      0.00            +1.1        1.08        perf-profile.calltrace.cycles-pp.update_load_avg.set_next_entity.pick_next_task_fair.__sched_text_start.schedule_idle
      0.00            +1.2        1.16        perf-profile.calltrace.cycles-pp._find_next_bit.__next_timer_interrupt.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length
      0.60 ±  5%      +1.2        1.77        perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__sched_text_start.schedule_idle.do_idle
      0.00            +1.2        1.19 ±  3%  perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__sched_text_start.schedule
      0.00            +1.2        1.21 ±  3%  perf-profile.calltrace.cycles-pp.timekeeping_max_deferment.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle
      0.00            +1.2        1.25 ±  2%  perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read
      2.31            +1.5        3.81        perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
      0.75            +1.7        2.43        perf-profile.calltrace.cycles-pp.__next_timer_interrupt.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select
      2.87            +1.7        4.57        perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
      2.97            +1.8        4.75        perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
      3.00            +1.8        4.84        perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
      0.87 ±  4%      +1.9        2.77        perf-profile.calltrace.cycles-pp.pick_next_task_fair.__sched_text_start.schedule_idle.do_idle.cpu_startup_entry
      0.55            +2.2        2.74 ±  2%  perf-profile.calltrace.cycles-pp.tick_nohz_idle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
      0.00            +2.3        2.26 ±  2%  perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_idle_enter.do_idle.cpu_startup_entry.start_secondary
      0.00            +2.3        2.31 ±  3%  perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_idle_exit.do_idle.cpu_startup_entry.start_secondary
      1.14            +2.5        3.60        perf-profile.calltrace.cycles-pp.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle
      6.22            +2.6        8.85        perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
      2.07 ±  2%      +2.7        4.79        perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__sched_text_start.schedule.pipe_read
      6.29            +2.8        9.09        perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
      6.46            +2.8        9.26        perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
      0.00            +2.9        2.89 ±  2%  perf-profile.calltrace.cycles-pp.tick_nohz_idle_exit.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
      6.77            +3.1        9.88        perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
      2.45 ±  2%      +3.3        5.79        perf-profile.calltrace.cycles-pp.dequeue_task_fair.__sched_text_start.schedule.pipe_read.new_sync_read
      1.98 ±  2%      +3.5        5.52        perf-profile.calltrace.cycles-pp.__sched_text_start.schedule_idle.do_idle.cpu_startup_entry.start_secondary
      2.07 ±  2%      +3.7        5.81        perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
      1.50            +3.8        5.28        perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry
      1.93            +4.5        6.41        perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary
      8.68            +4.7       13.37        perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
      8.80            +5.0       13.82        perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      3.74            +5.1        8.85        perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
      6.61            +5.4       11.98        perf-profile.calltrace.cycles-pp.__sched_text_start.schedule.pipe_read.new_sync_read.vfs_read
      7.02            +5.4       12.44        perf-profile.calltrace.cycles-pp.schedule.pipe_read.new_sync_read.vfs_read.ksys_read
      9.29            +5.6       14.90        perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
      9.48            +6.0       15.52        perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
      9.75            +6.2       15.99        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
      9.80            +6.4       16.16        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
     10.28            +7.2       17.47        perf-profile.calltrace.cycles-pp.write
     10.55            +9.0       19.59        perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64
     10.80            +9.5       20.34        perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
     11.60           +11.0       22.61        perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
     11.77           +11.5       23.22        perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
     12.16           +12.1       24.27        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
     12.29           +12.4       24.65        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read
     13.66           +14.5       28.17        perf-profile.calltrace.cycles-pp.read
     46.84           -27.9       18.98        perf-profile.children.cycles-pp.menu_select
     66.20           -19.6       46.61        perf-profile.children.cycles-pp.do_idle
     66.23           -19.5       46.74        perf-profile.children.cycles-pp.secondary_startup_64
     66.23           -19.5       46.74        perf-profile.children.cycles-pp.cpu_startup_entry
     65.96           -19.4       46.55        perf-profile.children.cycles-pp.start_secondary
     14.58            -2.6       11.99        perf-profile.children.cycles-pp.cpuidle_enter_state
     14.62            -2.6       12.06        perf-profile.children.cycles-pp.cpuidle_enter
     10.18            -1.9        8.29        perf-profile.children.cycles-pp.__switch_to
      1.91 ±  2%      -1.5        0.41 ± 13%  perf-profile.children.cycles-pp.intel_idle
      1.46            -0.9        0.53 ±  2%  perf-profile.children.cycles-pp.asm_sysvec_reschedule_ipi
      0.71            -0.2        0.48        perf-profile.children.cycles-pp.select_idle_sibling
      1.10            -0.2        0.92        perf-profile.children.cycles-pp.___perf_sw_event
      0.17 ±  2%      -0.1        0.03 ±100%  perf-profile.children.cycles-pp.cpuidle_get_cpu_driver
      0.20 ±  2%      -0.1        0.07        perf-profile.children.cycles-pp.perf_exclude_event
      1.30            -0.1        1.21        perf-profile.children.cycles-pp.select_task_rq_fair
      1.18            -0.1        1.12        perf-profile.children.cycles-pp.update_rq_clock
      0.13 ±  6%      -0.1        0.07        perf-profile.children.cycles-pp.perf_swevent_event
      0.15 ±  3%      -0.1        0.09 ±  7%  perf-profile.children.cycles-pp.__x64_sys_write
      0.15 ±  2%      -0.0        0.11 ±  4%  perf-profile.children.cycles-pp.rw_verify_area
      0.14 ±  5%      -0.0        0.10        perf-profile.children.cycles-pp.enter_lazy_tlb
      0.17 ±  2%      +0.0        0.19 ±  2%  perf-profile.children.cycles-pp.available_idle_cpu
      0.14 ±  3%      +0.0        0.15 ±  3%  perf-profile.children.cycles-pp.tick_sched_handle
      0.10 ±  7%      +0.0        0.12 ± 10%  perf-profile.children.cycles-pp.irqtime_account_irq
      0.11            +0.0        0.13 ±  9%  perf-profile.children.cycles-pp._raw_spin_trylock
      0.94            +0.0        0.96        perf-profile.children.cycles-pp.newidle_balance
      0.04 ± 57%      +0.0        0.06        perf-profile.children.cycles-pp.kill_fasync
      0.06 ± 11%      +0.0        0.08 ± 10%  perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
      0.11 ±  7%      +0.0        0.14 ±  6%  perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64
      0.13 ±  3%      +0.0        0.16 ±  8%  perf-profile.children.cycles-pp.rebalance_domains
      0.13            +0.0        0.15 ±  3%  perf-profile.children.cycles-pp.update_process_times
      0.08 ±  5%      +0.0        0.11 ±  4%  perf-profile.children.cycles-pp.cpus_share_cache
      0.30            +0.0        0.33        perf-profile.children.cycles-pp.update_min_vruntime
      0.23 ±  2%      +0.0        0.26 ±  9%  perf-profile.children.cycles-pp.do_softirq_own_stack
      0.17 ±  2%      +0.0        0.21 ±  7%  perf-profile.children.cycles-pp.__sb_start_write
      0.11 ±  6%      +0.0        0.14 ±  3%  perf-profile.children.cycles-pp.__enqueue_entity
      0.05            +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.rcu_needs_cpu
      0.06 ±  9%      +0.0        0.09        perf-profile.children.cycles-pp.arch_cpu_idle_enter
      0.22            +0.0        0.26 ±  9%  perf-profile.children.cycles-pp.__softirqentry_text_start
      0.27            +0.0        0.32 ±  9%  perf-profile.children.cycles-pp.irq_exit_rcu
      0.01 ±173%      +0.0        0.06        perf-profile.children.cycles-pp.scheduler_tick
      0.16 ±  2%      +0.0        0.21 ±  6%  perf-profile.children.cycles-pp.tick_sched_timer
      0.00            +0.1        0.05        perf-profile.children.cycles-pp.sched_clock_idle_wakeup_event
      0.00            +0.1        0.05        perf-profile.children.cycles-pp.__vfs_write
      0.03 ±100%      +0.1        0.08 ±  5%  perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
      0.05 ±  8%      +0.1        0.11 ±  8%  perf-profile.children.cycles-pp.get_cpu_device
      0.00            +0.1        0.05 ±  8%  perf-profile.children.cycles-pp.default_send_IPI_single_phys
      0.09 ±  4%      +0.1        0.14 ±  5%  perf-profile.children.cycles-pp.clockevents_program_event
      0.12 ±  4%      +0.1        0.18        perf-profile.children.cycles-pp.rb_insert_color
      0.15 ±  2%      +0.1        0.21 ±  3%  perf-profile.children.cycles-pp.testcase
      0.00            +0.1        0.06 ±  7%  perf-profile.children.cycles-pp.cpuidle_select
      0.01 ±173%      +0.1        0.07 ± 17%  perf-profile.children.cycles-pp.write@plt
      0.15 ±  3%      +0.1        0.22 ±  5%  perf-profile.children.cycles-pp.child
      0.10 ±  4%      +0.1        0.16        perf-profile.children.cycles-pp.rcu_dynticks_eqs_enter
      0.00            +0.1        0.06 ±  6%  perf-profile.children.cycles-pp.get_xsave_addr
      0.24            +0.1        0.31 ±  2%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.24 ±  3%      +0.1        0.31 ±  2%  perf-profile.children.cycles-pp.__sb_end_write
      0.07 ± 12%      +0.1        0.14        perf-profile.children.cycles-pp.nr_iowait_cpu
      0.01 ±173%      +0.1        0.09 ±  4%  perf-profile.children.cycles-pp.irq_work_needs_cpu
      0.00            +0.1        0.08 ±  5%  perf-profile.children.cycles-pp.flush_smp_call_function_from_idle
      0.11 ±  3%      +0.1        0.20 ±  2%  perf-profile.children.cycles-pp.__cgroup_account_cputime
      0.00            +0.1        0.09 ±  5%  perf-profile.children.cycles-pp.init_wait_entry
      0.05            +0.1        0.14 ±  6%  perf-profile.children.cycles-pp.timestamp_truncate
      0.00            +0.1        0.09 ± 29%  perf-profile.children.cycles-pp.sysvec_call_function_single
      0.06            +0.1        0.15 ±  2%  perf-profile.children.cycles-pp.rcu_dynticks_eqs_exit
      0.00            +0.1        0.09 ± 11%  perf-profile.children.cycles-pp.ktime_get_update_offsets_now
      0.07 ±  6%      +0.1        0.17 ±  3%  perf-profile.children.cycles-pp.aa_file_perm
      0.00            +0.1        0.10 ±  8%  perf-profile.children.cycles-pp.read@plt
      0.92            +0.1        1.02        perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
      0.16 ±  2%      +0.1        0.26        perf-profile.children.cycles-pp.place_entity
      0.05 ±  8%      +0.1        0.16 ±  2%  perf-profile.children.cycles-pp.__x64_sys_read
      0.00            +0.1        0.11 ±  4%  perf-profile.children.cycles-pp.cpuidle_reflect
      0.05 ±  8%      +0.1        0.16 ±  2%  perf-profile.children.cycles-pp.cpuidle_not_available
      0.01 ±173%      +0.1        0.13 ±  3%  perf-profile.children.cycles-pp.ttwu_queue_wakelist
      0.35 ±  3%      +0.1        0.47 ±  2%  perf-profile.children.cycles-pp.cpuidle_governor_latency_req
      0.03 ±100%      +0.1        0.15 ±  7%  perf-profile.children.cycles-pp.rb_erase
      0.36 ± 10%      +0.1        0.49        perf-profile.children.cycles-pp.call_cpuidle
      0.01 ±173%      +0.1        0.14 ±  3%  perf-profile.children.cycles-pp.__list_del_entry_valid
      0.14            +0.1        0.27        perf-profile.children.cycles-pp.__syscall_return_slowpath
      0.00            +0.1        0.13 ±  3%  perf-profile.children.cycles-pp.__list_add_valid
      0.00            +0.1        0.13 ± 29%  perf-profile.children.cycles-pp.asm_sysvec_call_function_single
      0.08            +0.1        0.21 ±  5%  perf-profile.children.cycles-pp.__update_idle_core
      0.08 ±  5%      +0.1        0.21 ±  2%  perf-profile.children.cycles-pp.deactivate_task
      0.05            +0.1        0.18        perf-profile.children.cycles-pp.update_ts_time_stats
      0.00            +0.1        0.13 ±  7%  perf-profile.children.cycles-pp.tick_check_broadcast_expired
      0.23 ±  3%      +0.1        0.37 ±  4%  perf-profile.children.cycles-pp.atime_needs_update
      0.00            +0.1        0.14 ±  3%  perf-profile.children.cycles-pp.put_prev_task_idle
      0.11 ±  4%      +0.1        0.25        perf-profile.children.cycles-pp.rcu_eqs_enter
      0.08 ±  6%      +0.1        0.22        perf-profile.children.cycles-pp.can_stop_idle_tick
      0.00            +0.1        0.15 ±  3%  perf-profile.children.cycles-pp.iov_iter_init
      0.08            +0.1        0.23 ±  4%  perf-profile.children.cycles-pp.rcu_all_qs
      0.00            +0.2        0.15 ±  4%  perf-profile.children.cycles-pp.apparmor_file_permission
      0.07            +0.2        0.23 ±  2%  perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      0.03 ±100%      +0.2        0.18 ± 26%  perf-profile.children.cycles-pp.rb_next
      0.12 ± 11%      +0.2        0.27 ± 13%  perf-profile.children.cycles-pp.__hrtimer_next_event_base
      0.09 ±  4%      +0.2        0.25 ±  4%  perf-profile.children.cycles-pp.set_next_task_idle
      0.00            +0.2        0.16 ±  5%  perf-profile.children.cycles-pp.menu_reflect
      0.05            +0.2        0.22 ±  3%  perf-profile.children.cycles-pp.tick_nohz_tick_stopped
      0.39            +0.2        0.57 ±  4%  perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
      0.29 ±  2%      +0.2        0.46 ±  3%  perf-profile.children.cycles-pp.touch_atime
      0.39            +0.2        0.56 ±  4%  perf-profile.children.cycles-pp.hrtimer_interrupt
      0.09 ±  4%      +0.2        0.27 ±  2%  perf-profile.children.cycles-pp.anon_pipe_buf_release
      0.77            +0.2        0.95 ±  2%  perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
      0.10            +0.2        0.28 ±  4%  perf-profile.children.cycles-pp.pick_next_task_idle
      0.10 ±  4%      +0.2        0.28        perf-profile.children.cycles-pp.rcu_eqs_exit
      0.25 ±  2%      +0.2        0.43 ±  4%  perf-profile.children.cycles-pp.current_time
      0.00            +0.2        0.19 ±  5%  perf-profile.children.cycles-pp.clear_buddies
      0.09            +0.2        0.29 ±  3%  perf-profile.children.cycles-pp.switch_fpu_return
      0.09 ±  5%      +0.2        0.29 ±  2%  perf-profile.children.cycles-pp.switch_mm_irqs_off
      0.25 ±  3%      +0.2        0.45 ±  4%  perf-profile.children.cycles-pp.cpuacct_charge
      0.08            +0.2        0.29        perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
      0.09            +0.2        0.30 ±  2%  perf-profile.children.cycles-pp.finish_wait
      0.26 ±  3%      +0.2        0.48 ±  2%  perf-profile.children.cycles-pp.fsnotify
      0.12 ±  8%      +0.2        0.36 ±  3%  perf-profile.children.cycles-pp.account_entity_dequeue
      0.19 ±  4%      +0.2        0.43        perf-profile.children.cycles-pp.rcu_note_context_switch
      0.06            +0.2        0.30        perf-profile.children.cycles-pp.rcu_qs
      0.62            +0.2        0.86        perf-profile.children.cycles-pp.asm_call_on_stack
      0.16 ±  2%      +0.2        0.41        perf-profile.children.cycles-pp.put_prev_entity
      0.17 ±  2%      +0.2        0.42 ±  2%  perf-profile.children.cycles-pp.resched_curr
      0.12            +0.2        0.37 ±  3%  perf-profile.children.cycles-pp._cond_resched
      0.12 ±  4%      +0.2        0.38 ±  2%  perf-profile.children.cycles-pp.copyin
      0.12            +0.3        0.38        perf-profile.children.cycles-pp._raw_spin_lock_irq
      0.10 ±  4%      +0.3        0.36        perf-profile.children.cycles-pp.find_next_bit
      0.39            +0.3        0.67        perf-profile.children.cycles-pp.put_prev_task_fair
      0.13 ±  3%      +0.3        0.43 ±  2%  perf-profile.children.cycles-pp.copyout
      0.13            +0.3        0.44        perf-profile.children.cycles-pp.fsnotify_parent
      0.16            +0.3        0.47 ±  3%  perf-profile.children.cycles-pp.__prepare_exit_to_usermode
      0.18 ±  3%      +0.3        0.49 ±  2%  perf-profile.children.cycles-pp.rcu_idle_exit
      0.17 ±  2%      +0.3        0.49        perf-profile.children.cycles-pp.hrtimer_get_next_event
      0.16            +0.3        0.48        perf-profile.children.cycles-pp.mutex_unlock
      0.14 ±  3%      +0.3        0.47        perf-profile.children.cycles-pp.copy_user_generic_unrolled
      0.15            +0.3        0.48        perf-profile.children.cycles-pp.__fget_light
      0.14 ±  8%      +0.4        0.49 ±  2%  perf-profile.children.cycles-pp.pick_next_entity
      0.16 ±  2%      +0.4        0.53        perf-profile.children.cycles-pp.__calc_delta
      0.10 ±  4%      +0.4        0.47 ±  2%  perf-profile.children.cycles-pp.__might_fault
      0.31 ±  2%      +0.4        0.70 ±  3%  perf-profile.children.cycles-pp.reweight_entity
      0.18 ±  4%      +0.4        0.57        perf-profile.children.cycles-pp.__fdget_pos
      0.15            +0.4        0.59 ±  2%  perf-profile.children.cycles-pp.__might_sleep
      0.22            +0.5        0.67        perf-profile.children.cycles-pp.read_tsc
      0.27 ±  4%      +0.5        0.73 ±  6%  perf-profile.children.cycles-pp.hrtimer_next_event_without
      0.37 ±  2%      +0.5        0.90        perf-profile.children.cycles-pp.check_preempt_curr
      0.27            +0.6        0.83        perf-profile.children.cycles-pp._raw_spin_lock
      0.48            +0.6        1.05 ±  2%  perf-profile.children.cycles-pp.finish_task_switch
      0.36 ±  2%      +0.6        0.93 ±  2%  perf-profile.children.cycles-pp.__update_load_avg_se
      0.68            +0.6        1.26        perf-profile.children.cycles-pp.native_sched_clock
      0.59 ±  2%      +0.6        1.17 ±  2%  perf-profile.children.cycles-pp.update_cfs_group
      1.35            +0.6        1.94        perf-profile.children.cycles-pp.prepare_to_wait_event
      0.29            +0.6        0.90 ±  2%  perf-profile.children.cycles-pp.common_file_perm
      0.41 ±  2%      +0.6        1.02        perf-profile.children.cycles-pp.ttwu_do_wakeup
      0.69            +0.6        1.31        perf-profile.children.cycles-pp.sched_clock
      0.24            +0.7        0.90 ±  3%  perf-profile.children.cycles-pp.___might_sleep
      0.38            +0.7        1.05        perf-profile.children.cycles-pp.syscall_return_via_sysret
      0.32            +0.7        1.01        perf-profile.children.cycles-pp.copy_page_from_iter
      0.45 ±  2%      +0.7        1.17        perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
      0.73            +0.7        1.47        perf-profile.children.cycles-pp.sched_clock_cpu
      0.86            +0.9        1.73        perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      0.50            +0.9        1.37        perf-profile.children.cycles-pp.mutex_lock
      0.39            +0.9        1.28        perf-profile.children.cycles-pp.entry_SYSCALL_64
      0.33            +1.0        1.28 ±  2%  perf-profile.children.cycles-pp.copy_page_to_iter
      0.18 ±  4%      +1.0        1.22 ±  2%  perf-profile.children.cycles-pp.timekeeping_max_deferment
      0.51            +1.1        1.58        perf-profile.children.cycles-pp.__lock_text_start
      1.09            +1.1        2.18        perf-profile.children.cycles-pp.update_curr
      0.50            +1.1        1.63        perf-profile.children.cycles-pp._find_next_bit
      0.49            +1.1        1.62 ±  2%  perf-profile.children.cycles-pp.security_file_permission
      0.62 ±  5%      +1.2        1.88        perf-profile.children.cycles-pp.set_next_entity
      0.64            +1.4        2.00        perf-profile.children.cycles-pp.__switch_to_asm
      2.35            +1.6        3.96        perf-profile.children.cycles-pp.enqueue_entity
      2.88            +1.7        4.62        perf-profile.children.cycles-pp.enqueue_task_fair
      2.97            +1.8        4.79        perf-profile.children.cycles-pp.activate_task
      0.81            +1.9        2.67        perf-profile.children.cycles-pp.__next_timer_interrupt
      3.01            +1.9        4.89        perf-profile.children.cycles-pp.ttwu_do_activate
      2.26            +1.9        4.16        perf-profile.children.cycles-pp.pick_next_task_fair
      1.38 ±  4%      +2.1        3.46        perf-profile.children.cycles-pp.update_load_avg
      0.56            +2.2        2.77 ±  2%  perf-profile.children.cycles-pp.tick_nohz_idle_enter
      1.15            +2.5        3.63        perf-profile.children.cycles-pp.get_next_timer_interrupt
      0.52            +2.6        3.10 ±  2%  perf-profile.children.cycles-pp.tick_nohz_idle_exit
      6.23            +2.6        8.88        perf-profile.children.cycles-pp.try_to_wake_up
      2.10 ±  2%      +2.8        4.89        perf-profile.children.cycles-pp.dequeue_entity
      6.46            +2.8        9.27        perf-profile.children.cycles-pp.__wake_up_common
      6.29            +2.8        9.11        perf-profile.children.cycles-pp.autoremove_wake_function
      6.77            +3.1        9.90        perf-profile.children.cycles-pp.__wake_up_common_lock
      2.46 ±  2%      +3.3        5.80        perf-profile.children.cycles-pp.dequeue_task_fair
      2.08 ±  2%      +3.8        5.86        perf-profile.children.cycles-pp.schedule_idle
      1.52            +3.8        5.33        perf-profile.children.cycles-pp.tick_nohz_next_event
      0.97            +3.9        4.90 ±  2%  perf-profile.children.cycles-pp.ktime_get
      1.94            +4.5        6.45        perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      8.71            +4.7       13.41        perf-profile.children.cycles-pp.pipe_write
      8.81            +5.0       13.84        perf-profile.children.cycles-pp.new_sync_write
      3.87            +5.1        8.97        perf-profile.children.cycles-pp.poll_idle
      7.03            +5.4       12.45        perf-profile.children.cycles-pp.schedule
      9.29            +5.6       14.94        perf-profile.children.cycles-pp.vfs_write
      9.49            +6.0       15.54        perf-profile.children.cycles-pp.ksys_write
     10.36            +7.4       17.78        perf-profile.children.cycles-pp.write
      8.81            +9.0       17.77        perf-profile.children.cycles-pp.__sched_text_start
     10.59            +9.1       19.70        perf-profile.children.cycles-pp.pipe_read
     10.81            +9.6       20.38        perf-profile.children.cycles-pp.new_sync_read
     11.61           +11.1       22.67        perf-profile.children.cycles-pp.vfs_read
     11.77           +11.5       23.23        perf-profile.children.cycles-pp.ksys_read
     13.74           +14.7       28.46        perf-profile.children.cycles-pp.read
     21.97           +18.4       40.36        perf-profile.children.cycles-pp.do_syscall_64
     22.12           +18.7       40.85        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
     44.48           -32.6       11.87 ±  2%  perf-profile.self.cycles-pp.menu_select
      8.39            -6.2        2.19 ±  4%  perf-profile.self.cycles-pp.cpuidle_enter_state
     10.17            -1.9        8.23        perf-profile.self.cycles-pp.__switch_to
      1.91 ±  2%      -1.5        0.41 ± 13%  perf-profile.self.cycles-pp.intel_idle
      0.47            -0.3        0.22        perf-profile.self.cycles-pp.select_idle_sibling
      0.63            -0.2        0.47        perf-profile.self.cycles-pp.update_rq_clock
      0.38            -0.2        0.22 ±  3%  perf-profile.self.cycles-pp.file_update_time
      0.20 ±  2%      -0.1        0.06        perf-profile.self.cycles-pp.perf_exclude_event
      0.14 ±  5%      -0.1        0.08 ±  5%  perf-profile.self.cycles-pp.__x64_sys_write
      0.14 ±  5%      -0.1        0.09        perf-profile.self.cycles-pp.enter_lazy_tlb
      0.15 ±  3%      -0.0        0.10 ±  4%  perf-profile.self.cycles-pp.rw_verify_area
      0.07            +0.0        0.08 ±  5%  perf-profile.self.cycles-pp.sched_clock
      0.10 ±  4%      +0.0        0.12 ±  3%  perf-profile.self.cycles-pp.__enqueue_entity
      0.07 ±  7%      +0.0        0.09 ±  5%  perf-profile.self.cycles-pp.touch_atime
      0.29            +0.0        0.32 ±  2%  perf-profile.self.cycles-pp.update_min_vruntime
      0.17 ±  4%      +0.0        0.20 ±  7%  perf-profile.self.cycles-pp.__sb_start_write
      0.07            +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.cpus_share_cache
      0.05            +0.0        0.08        perf-profile.self.cycles-pp.rcu_needs_cpu
      0.15 ±  2%      +0.0        0.19 ±  3%  perf-profile.self.cycles-pp.testcase
      0.23            +0.0        0.27        perf-profile.self.cycles-pp.put_prev_task_fair
      0.05            +0.0        0.10 ±  9%  perf-profile.self.cycles-pp.get_cpu_device
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.cpuidle_enter
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
      0.00            +0.1        0.05 ±  9%  perf-profile.self.cycles-pp.hrtimer_get_next_event
      0.00            +0.1        0.05 ±  9%  perf-profile.self.cycles-pp.cpuidle_select
      0.12 ±  3%      +0.1        0.18 ±  2%  perf-profile.self.cycles-pp.rb_insert_color
      0.15 ±  2%      +0.1        0.21 ±  5%  perf-profile.self.cycles-pp.child
      0.00            +0.1        0.06        perf-profile.self.cycles-pp.copyout
      0.00            +0.1        0.06        perf-profile.self.cycles-pp.get_xsave_addr
      0.08 ± 10%      +0.1        0.15 ±  2%  perf-profile.self.cycles-pp.activate_task
      0.07 ±  7%      +0.1        0.13 ±  3%  perf-profile.self.cycles-pp.nr_iowait_cpu
      0.00            +0.1        0.06 ±  6%  perf-profile.self.cycles-pp.copyin
      0.09            +0.1        0.16 ±  2%  perf-profile.self.cycles-pp.rcu_dynticks_eqs_enter
      0.24 ±  3%      +0.1        0.31 ±  2%  perf-profile.self.cycles-pp.__sb_end_write
      0.72 ±  2%      +0.1        0.79        perf-profile.self.cycles-pp.___perf_sw_event
      0.00            +0.1        0.07 ±  5%  perf-profile.self.cycles-pp.flush_smp_call_function_from_idle
      0.07 ± 10%      +0.1        0.15 ±  7%  perf-profile.self.cycles-pp.current_time
      0.00            +0.1        0.08 ±  6%  perf-profile.self.cycles-pp.hrtimer_next_event_without
      0.00            +0.1        0.08 ±  6%  perf-profile.self.cycles-pp.init_wait_entry
      0.00            +0.1        0.08 ±  5%  perf-profile.self.cycles-pp.irq_work_needs_cpu
      0.00            +0.1        0.08 ± 10%  perf-profile.self.cycles-pp.ktime_get_update_offsets_now
      0.06 ±  7%      +0.1        0.14        perf-profile.self.cycles-pp.rcu_dynticks_eqs_exit
      0.04 ± 57%      +0.1        0.12 ±  3%  perf-profile.self.cycles-pp.rcu_eqs_exit
      0.00            +0.1        0.08 ±  5%  perf-profile.self.cycles-pp.finish_wait
      0.15 ±  2%      +0.1        0.23 ±  2%  perf-profile.self.cycles-pp.place_entity
      0.23 ±  2%      +0.1        0.31        perf-profile.self.cycles-pp.schedule
      0.00            +0.1        0.09 ±  9%  perf-profile.self.cycles-pp.read@plt
      0.00            +0.1        0.09 ±  4%  perf-profile.self.cycles-pp.rcu_eqs_enter
      0.10            +0.1        0.19 ±  3%  perf-profile.self.cycles-pp.__hrtimer_next_event_base
      0.06 ±  6%      +0.1        0.15 ±  2%  perf-profile.self.cycles-pp.aa_file_perm
      0.08 ±  5%      +0.1        0.18 ±  2%  perf-profile.self.cycles-pp.cpuidle_governor_latency_req
      0.05            +0.1        0.14 ±  3%  perf-profile.self.cycles-pp.find_next_bit
      0.05 ±  8%      +0.1        0.15 ±  2%  perf-profile.self.cycles-pp.__x64_sys_read
      0.07 ±  5%      +0.1        0.17        perf-profile.self.cycles-pp.tick_nohz_get_sleep_length
      0.00            +0.1        0.10 ±  4%  perf-profile.self.cycles-pp.__fdget_pos
      0.00            +0.1        0.10 ±  4%  perf-profile.self.cycles-pp.cpuidle_reflect
      0.05 ±  8%      +0.1        0.15 ±  2%  perf-profile.self.cycles-pp.cpuidle_not_available
      0.00            +0.1        0.10        perf-profile.self.cycles-pp.ttwu_do_activate
      0.05            +0.1        0.15 ±  4%  perf-profile.self.cycles-pp.__cgroup_account_cputime
      0.00            +0.1        0.11 ±  4%  perf-profile.self.cycles-pp.tick_nohz_idle_enter
      0.00            +0.1        0.11        perf-profile.self.cycles-pp.ttwu_queue_wakelist
      0.10 ±  4%      +0.1        0.21 ±  2%  perf-profile.self.cycles-pp.__syscall_return_slowpath
      0.00            +0.1        0.11 ±  3%  perf-profile.self.cycles-pp.__list_add_valid
      0.08            +0.1        0.20 ±  2%  perf-profile.self.cycles-pp.deactivate_task
      0.58            +0.1        0.70 ±  2%  perf-profile.self.cycles-pp.select_task_rq_fair
      0.07            +0.1        0.19 ±  4%  perf-profile.self.cycles-pp.__prepare_exit_to_usermode
      0.01 ±173%      +0.1        0.13 ±  5%  perf-profile.self.cycles-pp.timestamp_truncate
      0.08 ±  6%      +0.1        0.19 ± 11%  perf-profile.self.cycles-pp.atime_needs_update
      0.00            +0.1        0.12 ±  5%  perf-profile.self.cycles-pp.ttwu_do_wakeup
      0.00            +0.1        0.12 ±  8%  perf-profile.self.cycles-pp.tick_check_broadcast_expired
      0.07 ±  6%      +0.1        0.19 ±  5%  perf-profile.self.cycles-pp.rcu_all_qs
      0.00            +0.1        0.12 ±  3%  perf-profile.self.cycles-pp.update_ts_time_stats
      0.07 ±  7%      +0.1        0.19 ±  6%  perf-profile.self.cycles-pp.__update_idle_core
      0.00            +0.1        0.13 ±  8%  perf-profile.self.cycles-pp.rb_erase
      0.00            +0.1        0.13 ±  3%  perf-profile.self.cycles-pp.menu_reflect
      0.00            +0.1        0.13 ±  3%  perf-profile.self.cycles-pp.__list_del_entry_valid
      0.00            +0.1        0.13 ±  3%  perf-profile.self.cycles-pp.put_prev_task_idle
      0.80            +0.1        0.93        perf-profile.self.cycles-pp.newidle_balance
      0.00            +0.1        0.13        perf-profile.self.cycles-pp.__might_fault
      0.08            +0.1        0.21 ±  2%  perf-profile.self.cycles-pp.rcu_idle_exit
      0.05            +0.1        0.18 ±  2%  perf-profile.self.cycles-pp.cpu_startup_entry
      0.00            +0.1        0.13 ±  3%  perf-profile.self.cycles-pp.iov_iter_init
      0.10            +0.1        0.23 ±  2%  perf-profile.self.cycles-pp.copy_page_from_iter
      0.00            +0.1        0.14 ±  3%  perf-profile.self.cycles-pp._cond_resched
      0.00            +0.1        0.14 ±  3%  perf-profile.self.cycles-pp.apparmor_file_permission
      0.03 ±100%      +0.1        0.16 ± 26%  perf-profile.self.cycles-pp.rb_next
      0.07 ±  5%      +0.1        0.21 ±  2%  perf-profile.self.cycles-pp.can_stop_idle_tick
      0.53            +0.1        0.67        perf-profile.self.cycles-pp.enqueue_task_fair
      0.00            +0.1        0.14 ±  5%  perf-profile.self.cycles-pp.clear_buddies
      0.00            +0.1        0.14 ±  3%  perf-profile.self.cycles-pp.autoremove_wake_function
      0.51 ±  2%      +0.1        0.65        perf-profile.self.cycles-pp.pick_next_task_fair
      0.00            +0.1        0.15 ±  3%  perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
      0.07 ±  6%      +0.1        0.21 ±  2%  perf-profile.self.cycles-pp.schedule_idle
      0.00            +0.2        0.15 ±  4%  perf-profile.self.cycles-pp.sched_clock_cpu
      0.07 ±  6%      +0.2        0.22        perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      0.08 ±  5%      +0.2        0.26 ±  4%  perf-profile.self.cycles-pp.ksys_write
      0.09 ±  4%      +0.2        0.26        perf-profile.self.cycles-pp.anon_pipe_buf_release
      0.10 ±  4%      +0.2        0.28 ±  2%  perf-profile.self.cycles-pp.get_next_timer_interrupt
      0.31            +0.2        0.50        perf-profile.self.cycles-pp.try_to_wake_up
      0.09 ±  4%      +0.2        0.28 ±  2%  perf-profile.self.cycles-pp.switch_fpu_return
      0.00            +0.2        0.20 ±  2%  perf-profile.self.cycles-pp.tick_nohz_tick_stopped
      0.12 ± 11%      +0.2        0.32 ±  3%  perf-profile.self.cycles-pp.account_entity_dequeue
      0.08            +0.2        0.28 ±  2%  perf-profile.self.cycles-pp.switch_mm_irqs_off
      0.25 ±  3%      +0.2        0.45 ±  4%  perf-profile.self.cycles-pp.cpuacct_charge
      0.19            +0.2        0.39 ±  3%  perf-profile.self.cycles-pp.write
      0.12 ±  4%      +0.2        0.33 ±  2%  perf-profile.self.cycles-pp.tick_nohz_next_event
      0.25            +0.2        0.47 ±  2%  perf-profile.self.cycles-pp.fsnotify
      0.43            +0.2        0.66        perf-profile.self.cycles-pp.pipe_write
      0.09 ±  5%      +0.2        0.31 ±  3%  perf-profile.self.cycles-pp.security_file_permission
      0.32            +0.2        0.54 ±  2%  perf-profile.self.cycles-pp.dequeue_entity
      0.06            +0.2        0.28        perf-profile.self.cycles-pp.rcu_qs
      0.16 ±  2%      +0.2        0.39        perf-profile.self.cycles-pp.call_cpuidle
      0.10            +0.2        0.33 ±  2%  perf-profile.self.cycles-pp.ksys_read
      0.11            +0.2        0.35        perf-profile.self.cycles-pp.put_prev_entity
      0.17 ±  2%      +0.2        0.41 ±  2%  perf-profile.self.cycles-pp.resched_curr
      0.10 ±  5%      +0.2        0.34 ±  2%  perf-profile.self.cycles-pp.copy_page_to_iter
      0.17 ±  2%      +0.2        0.41 ±  2%  perf-profile.self.cycles-pp.read
      0.11 ±  7%      +0.3        0.36        perf-profile.self.cycles-pp.pick_next_entity
      0.11 ±  4%      +0.3        0.37        perf-profile.self.cycles-pp._raw_spin_lock_irq
      0.10 ±  4%      +0.3        0.36        perf-profile.self.cycles-pp.vfs_write
      0.15 ±  4%      +0.3        0.41        perf-profile.self.cycles-pp.check_preempt_curr
      0.12 ±  3%      +0.3        0.40        perf-profile.self.cycles-pp.fsnotify_parent
      0.08            +0.3        0.37        perf-profile.self.cycles-pp.new_sync_write
      0.15 ±  3%      +0.3        0.46        perf-profile.self.cycles-pp.mutex_unlock
      0.08 ±  5%      +0.3        0.40        perf-profile.self.cycles-pp.tick_nohz_idle_exit
      0.14 ±  3%      +0.3        0.45        perf-profile.self.cycles-pp.copy_user_generic_unrolled
      0.14 ±  3%      +0.3        0.46        perf-profile.self.cycles-pp.__fget_light
      0.15 ±  2%      +0.3        0.49        perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.26 ±  2%      +0.3        0.61        perf-profile.self.cycles-pp.mutex_lock
      0.15 ±  5%      +0.4        0.51        perf-profile.self.cycles-pp.set_next_entity
      0.29            +0.4        0.65 ±  2%  perf-profile.self.cycles-pp.finish_task_switch
      0.16 ±  2%      +0.4        0.52        perf-profile.self.cycles-pp.__calc_delta
      0.21 ±  2%      +0.4        0.58        perf-profile.self.cycles-pp.do_syscall_64
      0.76            +0.4        1.14        perf-profile.self.cycles-pp.pipe_read
      0.20 ±  2%      +0.4        0.59        perf-profile.self.cycles-pp.new_sync_read
      0.31            +0.4        0.70 ±  3%  perf-profile.self.cycles-pp.reweight_entity
      0.12            +0.4        0.52 ±  3%  perf-profile.self.cycles-pp.__might_sleep
      0.17 ±  3%      +0.4        0.59        perf-profile.self.cycles-pp.vfs_read
      0.28            +0.4        0.72 ±  2%  perf-profile.self.cycles-pp.enqueue_entity
      0.22            +0.4        0.66        perf-profile.self.cycles-pp.read_tsc
      0.29 ±  8%      +0.5        0.74 ±  4%  perf-profile.self.cycles-pp.update_curr
      0.23            +0.5        0.69 ±  2%  perf-profile.self.cycles-pp.entry_SYSCALL_64
      0.33 ±  3%      +0.5        0.83        perf-profile.self.cycles-pp.dequeue_task_fair
      0.23 ±  2%      +0.5        0.73 ±  3%  perf-profile.self.cycles-pp.common_file_perm
      0.26            +0.5        0.80        perf-profile.self.cycles-pp._raw_spin_lock
      0.34 ±  2%      +0.5        0.89 ±  2%  perf-profile.self.cycles-pp.__update_load_avg_se
      0.59 ±  2%      +0.6        1.15 ±  2%  perf-profile.self.cycles-pp.update_cfs_group
      0.61            +0.6        1.21        perf-profile.self.cycles-pp.native_sched_clock
      0.23            +0.6        0.86 ±  2%  perf-profile.self.cycles-pp.___might_sleep
      0.27            +0.6        0.92        perf-profile.self.cycles-pp.__next_timer_interrupt
      0.38            +0.7        1.04        perf-profile.self.cycles-pp.syscall_return_via_sysret
      0.43 ±  3%      +0.7        1.10 ±  2%  perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
      0.59 ±  7%      +0.8        1.37        perf-profile.self.cycles-pp.update_load_avg
      0.62            +1.0        1.62        perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.42            +1.0        1.42        perf-profile.self.cycles-pp.__lock_text_start
      0.18 ±  2%      +1.0        1.21 ±  3%  perf-profile.self.cycles-pp.timekeeping_max_deferment
      0.52            +1.1        1.57        perf-profile.self.cycles-pp.do_idle
      0.49            +1.1        1.61        perf-profile.self.cycles-pp._find_next_bit
      0.89            +1.2        2.06        perf-profile.self.cycles-pp.__sched_text_start
      0.64            +1.4        1.99        perf-profile.self.cycles-pp.__switch_to_asm
      0.75            +3.5        4.22 ±  3%  perf-profile.self.cycles-pp.ktime_get
      3.62 ±  2%      +4.8        8.45        perf-profile.self.cycles-pp.poll_idle


                                                                                
                            will-it-scale.per_process_ops                       
                                                                                
  450000 +------------------------------------------------------------------+   
         |                 O                                                |   
  400000 |-+                  O    O                                        |   
         |                           O  O                 O  O              |   
  350000 |-+          O O       O                                 O    O O  |   
         |    O                            O    O              O    O       |   
  300000 |-+                                                                |   
         |                                           O O                    |   
  250000 |-+                                 O    O                         |   
         |       O O                                                        |   
  200000 |-+O                                                               |   
         |                                                                  |   
  150000 |-.+.                                                              |   
         |.   +..+.+..+.+..                                                 |   
  100000 +------------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample



Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Rong Chen


View attachment "config-5.8.0-rc3-00035-g47125db27e47e" of type "text/plain" (158415 bytes)

View attachment "job-script" of type "text/plain" (7513 bytes)

View attachment "job.yaml" of type "text/plain" (5193 bytes)

View attachment "reproduce" of type "text/plain" (347 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ