lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20210420064934.GE31773@xsang-OptiPlex-9020>
Date:   Tue, 20 Apr 2021 14:49:34 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     "Paul E. McKenney" <paulmck@...nel.org>
Cc:     Thomas Gleixner <tglx@...utronix.de>,
        John Stultz <john.stultz@...aro.org>,
        Stephen Boyd <sboyd@...nel.org>,
        Jonathan Corbet <corbet@....net>,
        Mark Rutland <Mark.Rutland@....com>,
        Marc Zyngier <maz@...nel.org>, Andi Kleen <ak@...ux.intel.com>,
        Chris Mason <clm@...com>, LKML <linux-kernel@...r.kernel.org>,
        lkp@...ts.01.org, lkp@...el.com, ying.huang@...el.com,
        feng.tang@...el.com, zhengjun.xing@...el.com
Subject: [clocksource]  6c52b5f3cf:  stress-ng.opcode.ops_per_sec -14.4%
 regression



Greeting,

FYI, we noticed a -14.4% regression of stress-ng.opcode.ops_per_sec due to commit:


commit: 6c52b5f3cfefd6e429efc4413fd25e3c394e959f ("clocksource: Reduce WATCHDOG_THRESHOLD")
https://git.kernel.org/cgit/linux/kernel/git/paulmck/linux-rcu.git dev.2021.04.13a


in testcase: stress-ng
on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 192G memory
with following parameters:

	nr_threads: 10%
	disk: 1HDD
	testtime: 60s
	fs: ext4
	class: os
	test: opcode
	cpufreq_governor: performance
	ucode: 0x5003006


In addition to that, the commit also has significant impact on the following tests:

+------------------+---------------------------------------------------------------------------+
| testcase: change | unixbench: boot-time.dhcp 2.0% regression                                 |
| test machine     | 96 threads Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory                |
| test parameters  | cpufreq_governor=performance                                              |
|                  | nr_task=1                                                                 |
|                  | runtime=300s                                                              |
|                  | test=execl                                                                |
|                  | ucode=0x4003006                                                           |
+------------------+---------------------------------------------------------------------------+
| testcase: change | fsmark: fsmark.files_per_sec -37.8% regression                            |
| test machine     | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| test parameters  | cpufreq_governor=performance                                              |
|                  | disk=1BRD_48G                                                             |
|                  | filesize=4M                                                               |
|                  | fs=btrfs                                                                  |
|                  | iterations=1x                                                             |
|                  | nr_threads=1t                                                             |
|                  | sync_method=NoSync                                                        |
|                  | test_size=24G                                                             |
|                  | ucode=0x5003006                                                           |
+------------------+---------------------------------------------------------------------------+


If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang@...el.com>


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install                job.yaml  # job file is attached in this email
        bin/lkp split-job --compatible job.yaml
        bin/lkp run                    compatible-job.yaml

=========================================================================================
class/compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode:
  os/gcc-9/performance/1HDD/ext4/x86_64-rhel-8.3/10%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp5/opcode/stress-ng/60s/0x5003006

commit: 
  d42c7a8d68 ("kcsan: Document "value changed" line")
  6c52b5f3cf ("clocksource: Reduce WATCHDOG_THRESHOLD")

d42c7a8d68038bc0 6c52b5f3cfefd6e429efc4413fd 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    570473           -14.4%     488107        stress-ng.opcode.ops
      9507           -14.4%       8135        stress-ng.opcode.ops_per_sec
    573594           -14.5%     490520        stress-ng.time.involuntary_context_switches
     18566 ± 13%     -33.6%      12331 ± 16%  stress-ng.time.major_page_faults
    149469 ± 44%     -47.1%      79036 ±  5%  stress-ng.time.maximum_resident_set_size
  38932182 ±  2%     -14.8%   33173409        stress-ng.time.minor_page_faults
    800.67            +1.6%     813.67        stress-ng.time.percent_of_cpu_this_job_got
    289.84           +12.8%     327.07        stress-ng.time.system_time
    207.53           -14.1%     178.27        stress-ng.time.user_time
   1045208           -11.5%     924508        stress-ng.time.voluntary_context_switches
     15.47            +1.3%      15.67        boot-time.dhcp
      2168 ±  2%      +4.3%       2262        boot-time.idle
      9.46           -11.9%       8.34        iostat.cpu.system
      1.91           +18.3%       2.26 ±  5%  iostat.cpu.user
     17319 ±  5%     +40.7%      24370 ±  2%  numa-meminfo.node1.Active
     16704 ±  5%     +42.8%      23852        numa-meminfo.node1.Active(anon)
      4179 ±  5%     +42.8%       5968        numa-vmstat.node1.nr_active_anon
      4179 ±  5%     +42.8%       5968        numa-vmstat.node1.nr_zone_active_anon
     19143 ±  5%     +37.3%      26281        meminfo.Active
     17924 ±  5%     +39.8%      25063        meminfo.Active(anon)
     40996 ±  2%     +22.7%      50289        meminfo.Shmem
     88.00            +1.1%      89.00        vmstat.cpu.id
     68791           -13.1%      59772        vmstat.system.cs
   2602342 ±  2%     -21.6%    2038976        vmstat.system.in
 4.034e+09           -26.6%   2.96e+09        cpuidle.C1.time
 1.249e+08 ±  2%     -30.6%   86713543 ±  2%  cpuidle.C1.usage
  10583871 ± 19%    +106.1%   21814475 ±  3%  cpuidle.POLL.time
    486292 ± 18%     +23.0%     598021 ±  2%  cpuidle.POLL.usage
      2.41            -2.4        0.00        mpstat.cpu.all.irq%
      0.15            -0.1        0.01 ± 49%  mpstat.cpu.all.soft%
      6.03            +1.4        7.47        mpstat.cpu.all.sys%
      1.78            +0.3        2.08 ±  5%  mpstat.cpu.all.usr%
      0.00 ± 42%    +133.3%       0.01 ± 25%  perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown]
      0.01 ± 14%     +68.0%       0.01 ± 18%  perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select
      0.01 ± 14%     +68.0%       0.01 ± 18%  perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select
      3.07 ±  5%     +35.4%       4.16 ± 23%  perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
      2.63 ± 14%     +27.0%       3.34 ± 17%  sched_debug.cpu.clock.stddev
    581.67 ±  4%     -98.6%       8.36 ± 16%  sched_debug.cpu.clock_task.stddev
     20072 ± 41%     -60.5%       7934 ±114%  sched_debug.cpu.nr_switches.avg
     34831 ± 29%     -52.3%      16613 ± 69%  sched_debug.cpu.nr_switches.max
      1.00          -100.0%       0.00        sched_debug.sched_clock_stable()
     24399 ±  6%     -13.3%      21148 ±  7%  slabinfo.cred_jar.active_objs
     24409 ±  6%     -13.3%      21161 ±  7%  slabinfo.cred_jar.num_objs
      8294 ±  2%     -11.1%       7375        slabinfo.pid.active_objs
      8294 ±  2%     -11.1%       7375        slabinfo.pid.num_objs
     29772 ±  3%     -10.7%      26590 ±  6%  slabinfo.proc_inode_cache.active_objs
     29862 ±  3%     -10.7%      26670 ±  6%  slabinfo.proc_inode_cache.num_objs
    511.67          +128.8%       1170        turbostat.Avg_MHz
     18.44           +23.7       42.13        turbostat.Busy%
 1.249e+08 ±  2%     -30.6%   86709326 ±  2%  turbostat.C1
     65.77           -17.4       48.42        turbostat.C1%
     80.65           -28.5%      57.65        turbostat.CPU%c1
   1.7e+08           -22.0%  1.325e+08        turbostat.IRQ
     55.67 ±  2%      +3.9%      57.83        turbostat.PkgTmp
    186.07            +9.8%     204.40        turbostat.PkgWatt
      4482 ±  5%     +39.9%       6270        proc-vmstat.nr_active_anon
     19665            -2.8%      19111        proc-vmstat.nr_kernel_stack
     11209            +9.6%      12282        proc-vmstat.nr_mapped
     10271 ±  2%     +22.5%      12583        proc-vmstat.nr_shmem
     24169            -3.0%      23450        proc-vmstat.nr_slab_reclaimable
     52904            -1.4%      52156        proc-vmstat.nr_slab_unreclaimable
      4482 ±  5%     +39.9%       6270        proc-vmstat.nr_zone_active_anon
  35096977           -13.7%   30280778 ±  2%  proc-vmstat.numa_hit
  35010436           -13.8%   30194248 ±  2%  proc-vmstat.numa_local
     10212 ±  2%     +35.3%      13813 ±  4%  proc-vmstat.pgactivate
  37511123           -14.1%   32225979 ±  2%  proc-vmstat.pgalloc_normal
  39225190 ±  2%     -14.7%   33455229        proc-vmstat.pgfault
  37366865           -14.2%   32075663 ±  2%  proc-vmstat.pgfree
      9.58 ±  3%      -7.8%       8.84 ±  2%  perf-stat.i.MPKI
 6.556e+09            -2.9%  6.366e+09        perf-stat.i.branch-instructions
      0.86 ±  2%      +0.1        0.96        perf-stat.i.branch-miss-rate%
  52874393           +10.8%   58588777        perf-stat.i.branch-misses
  46443118 ±  2%      -8.2%   42638901        perf-stat.i.cache-misses
 2.597e+08            -7.0%  2.415e+08        perf-stat.i.cache-references
     70898           -13.3%      61457        perf-stat.i.context-switches
      2.02          +113.8%       4.32        perf-stat.i.cpi
  5.52e+10          +111.9%   1.17e+11        perf-stat.i.cpu-cycles
      1881 ±  4%     -28.7%       1341 ±  2%  perf-stat.i.cpu-migrations
      1214 ±  2%    +131.8%       2815        perf-stat.i.cycles-between-cache-misses
  47257151 ±  3%     -11.2%   41949022 ±  7%  perf-stat.i.dTLB-load-misses
 6.448e+09            -7.1%  5.988e+09        perf-stat.i.dTLB-loads
      0.15 ±  4%      -0.0        0.14 ±  3%  perf-stat.i.dTLB-store-miss-rate%
   7070051 ±  4%     -18.9%    5731351 ±  3%  perf-stat.i.dTLB-store-misses
 4.621e+09           -14.0%  3.974e+09        perf-stat.i.dTLB-stores
     49.48            +5.4       54.88        perf-stat.i.iTLB-load-miss-rate%
  29623646            -9.3%   26860981        perf-stat.i.iTLB-load-misses
  30337180 ±  2%     -27.7%   21933274 ±  2%  perf-stat.i.iTLB-loads
    989.66            +7.8%       1066        perf-stat.i.instructions-per-iTLB-miss
      0.50           -53.4%       0.23        perf-stat.i.ipc
    298.29 ± 13%     -33.3%     198.98 ± 16%  perf-stat.i.major-faults
      0.57          +111.9%       1.22        perf-stat.i.metric.GHz
      0.79           -12.6%       0.69        perf-stat.i.metric.K/sec
    186.95            -7.4%     173.13        perf-stat.i.metric.M/sec
    619972           -14.5%     529911        perf-stat.i.minor-faults
  16950944           -12.8%   14775464        perf-stat.i.node-load-misses
   1158174            -8.3%    1062165        perf-stat.i.node-loads
     82.15            +1.0       83.16        perf-stat.i.node-store-miss-rate%
   7266499 ±  3%      -9.8%    6556262 ±  2%  perf-stat.i.node-store-misses
   1461472 ±  3%     -14.1%    1254949 ±  3%  perf-stat.i.node-stores
    624047           -14.5%     533390        perf-stat.i.page-faults
      9.52            -6.0%       8.95        perf-stat.overall.MPKI
      0.81            +0.1        0.92        perf-stat.overall.branch-miss-rate%
      2.02          +114.3%       4.33        perf-stat.overall.cpi
      1190 ±  2%    +130.8%       2747        perf-stat.overall.cycles-between-cache-misses
      0.15 ±  4%      -0.0        0.14 ±  3%  perf-stat.overall.dTLB-store-miss-rate%
     49.41            +5.6       55.05        perf-stat.overall.iTLB-load-miss-rate%
    921.55            +9.1%       1004        perf-stat.overall.instructions-per-iTLB-miss
      0.49           -53.3%       0.23        perf-stat.overall.ipc
 6.456e+09            -2.9%  6.268e+09        perf-stat.ps.branch-instructions
  52068658           +10.8%   57699339        perf-stat.ps.branch-misses
  45689361 ±  2%      -8.2%   41941220        perf-stat.ps.cache-misses
 2.559e+08            -7.0%  2.379e+08        perf-stat.ps.cache-references
     69875           -13.4%      60528        perf-stat.ps.context-switches
 5.437e+10          +111.9%  1.152e+11        perf-stat.ps.cpu-cycles
      1856 ±  4%     -28.7%       1322 ±  2%  perf-stat.ps.cpu-migrations
  46573389 ±  3%     -11.2%   41343641 ±  7%  perf-stat.ps.dTLB-load-misses
 6.354e+09            -7.1%    5.9e+09        perf-stat.ps.dTLB-loads
   6967873 ±  4%     -19.0%    5646326 ±  3%  perf-stat.ps.dTLB-store-misses
 4.553e+09           -14.0%  3.915e+09        perf-stat.ps.dTLB-stores
  29182781            -9.3%   26458801        perf-stat.ps.iTLB-load-misses
  29879883 ±  2%     -27.7%   21606494 ±  2%  perf-stat.ps.iTLB-loads
    291.21 ± 13%     -33.3%     194.27 ± 16%  perf-stat.ps.major-faults
    611056 ±  2%     -14.6%     521852        perf-stat.ps.minor-faults
  16664002           -12.8%   14524399        perf-stat.ps.node-load-misses
   1161220            -8.4%    1063348        perf-stat.ps.node-loads
   7141376 ±  3%      -9.8%    6441085 ±  2%  perf-stat.ps.node-store-misses
   1446833 ±  3%     -14.2%    1241728 ±  3%  perf-stat.ps.node-stores
    615068 ±  2%     -14.6%     525276        perf-stat.ps.page-faults
  1.71e+12            -2.0%  1.676e+12        perf-stat.total.instructions
     14099 ± 12%     -34.3%       9258 ±  4%  softirqs.CPU0.RCU
     12872 ±  6%     +10.3%      14202 ±  3%  softirqs.CPU0.SCHED
     10233 ±  7%     -23.9%       7788 ±  3%  softirqs.CPU10.RCU
      9723 ±  4%     -20.3%       7752 ±  2%  softirqs.CPU11.RCU
      9728 ±  4%     -17.5%       8022 ±  6%  softirqs.CPU12.RCU
      9552 ±  7%     -19.0%       7739 ±  2%  softirqs.CPU13.RCU
     10179 ± 10%     -22.5%       7891 ±  3%  softirqs.CPU14.RCU
      9655 ±  5%     -16.5%       8064 ±  6%  softirqs.CPU15.RCU
     10140 ±  6%     -24.1%       7694        softirqs.CPU16.RCU
     10613 ±  6%     -26.6%       7787 ±  2%  softirqs.CPU17.RCU
     10154 ±  4%     -24.7%       7647        softirqs.CPU18.RCU
     10153 ±  4%     -25.0%       7614 ±  3%  softirqs.CPU19.RCU
     10271 ±  4%     -23.3%       7875 ±  7%  softirqs.CPU20.RCU
     10452 ±  3%     -24.6%       7884 ±  7%  softirqs.CPU21.RCU
     10249 ±  5%     -25.0%       7682 ±  4%  softirqs.CPU22.RCU
     10037 ±  4%     -24.3%       7595 ±  2%  softirqs.CPU23.RCU
     10086 ±  2%     -22.8%       7791 ±  3%  softirqs.CPU24.RCU
     10789 ± 17%     -25.4%       8052 ±  2%  softirqs.CPU25.RCU
      9938 ±  5%     -21.0%       7847 ±  6%  softirqs.CPU26.RCU
     10044 ±  4%     -24.3%       7604 ±  5%  softirqs.CPU27.RCU
     10007 ±  4%     -24.6%       7544 ±  3%  softirqs.CPU28.RCU
     10093 ±  7%     -24.6%       7613 ±  4%  softirqs.CPU29.RCU
     10018 ±  6%     -24.5%       7560 ±  3%  softirqs.CPU30.RCU
      9857 ±  3%     -22.5%       7642 ±  3%  softirqs.CPU31.RCU
      9618 ±  5%     -20.6%       7633 ±  3%  softirqs.CPU32.RCU
      9621 ±  4%     -21.7%       7528 ±  2%  softirqs.CPU33.RCU
      9840 ±  3%     -22.3%       7641 ±  3%  softirqs.CPU34.RCU
     10038 ±  8%     -25.6%       7465 ±  2%  softirqs.CPU35.RCU
      9852 ±  3%     -22.1%       7678 ±  4%  softirqs.CPU36.RCU
      9936 ±  6%     -22.7%       7676 ±  3%  softirqs.CPU37.RCU
      9721 ±  4%     -22.7%       7516 ±  3%  softirqs.CPU38.RCU
      9628 ±  5%     -21.4%       7572 ±  5%  softirqs.CPU39.RCU
     10003 ±  3%     -22.2%       7787 ±  3%  softirqs.CPU4.RCU
      9694 ±  4%     -22.1%       7550        softirqs.CPU40.RCU
      9750 ±  4%     -23.0%       7508 ±  2%  softirqs.CPU41.RCU
      9724 ±  5%     -21.7%       7617 ±  5%  softirqs.CPU42.RCU
      9746 ±  3%     -23.5%       7454 ±  2%  softirqs.CPU43.RCU
      9607 ±  5%     -20.1%       7679 ±  5%  softirqs.CPU44.RCU
      9600 ±  5%     -20.3%       7646 ±  7%  softirqs.CPU45.RCU
     10089 ±  8%     -24.5%       7622 ±  5%  softirqs.CPU46.RCU
      9534 ±  5%     -21.3%       7501 ±  2%  softirqs.CPU47.RCU
     10060 ±  4%     -21.8%       7864 ±  5%  softirqs.CPU48.RCU
      9977 ±  5%     -19.8%       8000 ±  4%  softirqs.CPU49.RCU
     10552 ±  7%     -26.0%       7812 ±  3%  softirqs.CPU50.RCU
     10431 ±  5%     -24.2%       7905 ±  4%  softirqs.CPU51.RCU
     10847 ± 16%     -26.0%       8023 ±  5%  softirqs.CPU52.RCU
     10367 ±  6%     -23.8%       7897 ±  4%  softirqs.CPU53.RCU
     10330 ±  3%     -25.1%       7741 ±  3%  softirqs.CPU54.RCU
     10586 ±  6%     -25.2%       7914 ±  2%  softirqs.CPU55.RCU
     10056 ±  7%     -22.5%       7796        softirqs.CPU56.RCU
     10704 ±  8%     -25.8%       7942 ±  3%  softirqs.CPU57.RCU
     10371 ±  4%     -24.7%       7812 ±  2%  softirqs.CPU58.RCU
     10294 ±  3%     -24.1%       7812 ±  3%  softirqs.CPU59.RCU
      9981 ±  4%     -20.8%       7909 ±  6%  softirqs.CPU6.RCU
     10241 ±  3%     -22.3%       7954 ±  3%  softirqs.CPU60.RCU
     10376 ±  7%     -23.6%       7929 ±  2%  softirqs.CPU61.RCU
     10290 ±  3%     -22.7%       7958 ±  4%  softirqs.CPU62.RCU
     10234 ±  3%     -23.2%       7856 ±  2%  softirqs.CPU63.RCU
     10616 ±  2%     -23.3%       8143 ±  7%  softirqs.CPU64.RCU
     10725 ±  5%     -26.9%       7835        softirqs.CPU65.RCU
     10809 ±  3%     -27.5%       7835 ±  3%  softirqs.CPU66.RCU
     10817 ±  5%     -27.3%       7867 ±  3%  softirqs.CPU67.RCU
     10906 ±  3%     -27.1%       7949 ±  4%  softirqs.CPU68.RCU
     11060 ±  3%     -28.7%       7886 ±  5%  softirqs.CPU69.RCU
      9806 ±  4%     -19.5%       7894 ±  3%  softirqs.CPU7.RCU
     10645 ±  4%     -27.3%       7738 ±  2%  softirqs.CPU70.RCU
     10764 ±  8%     -26.8%       7881 ±  2%  softirqs.CPU71.RCU
     10310 ±  5%     -25.5%       7680 ±  3%  softirqs.CPU72.RCU
     10301 ±  3%     -25.0%       7725 ±  3%  softirqs.CPU73.RCU
     10715 ±  2%     -25.7%       7960 ±  4%  softirqs.CPU74.RCU
     10495 ±  4%     -26.8%       7687 ±  2%  softirqs.CPU75.RCU
     10598 ±  8%     -25.5%       7892 ±  4%  softirqs.CPU76.RCU
     10592 ±  9%     -26.5%       7788 ±  3%  softirqs.CPU77.RCU
     10346 ±  4%     -25.8%       7678 ±  5%  softirqs.CPU78.RCU
     10415 ±  5%     -25.3%       7783 ±  3%  softirqs.CPU79.RCU
      9678 ±  5%     -17.8%       7956 ±  4%  softirqs.CPU8.RCU
     10111 ±  5%     -24.6%       7628 ±  3%  softirqs.CPU80.RCU
     10097 ±  3%     -18.3%       8246 ±  4%  softirqs.CPU81.RCU
     10222 ±  4%     -23.1%       7862 ±  3%  softirqs.CPU82.RCU
     10198 ±  5%     -23.0%       7855 ±  2%  softirqs.CPU83.RCU
     10325 ±  4%     -24.6%       7781 ±  4%  softirqs.CPU84.RCU
     10255 ±  3%     -21.9%       8005 ±  3%  softirqs.CPU85.RCU
     10194 ±  4%     -22.0%       7951 ±  2%  softirqs.CPU86.RCU
     10184 ±  5%     -24.7%       7671 ±  2%  softirqs.CPU87.RCU
     10368 ±  3%     -25.3%       7749 ±  2%  softirqs.CPU88.RCU
     10430 ±  5%     -24.6%       7862 ±  4%  softirqs.CPU89.RCU
      9775 ±  5%     -21.5%       7676 ±  2%  softirqs.CPU9.RCU
      9910 ±  4%     -21.9%       7740 ±  6%  softirqs.CPU90.RCU
     10147 ±  4%     -23.6%       7751 ±  3%  softirqs.CPU91.RCU
     10315 ±  7%     -25.4%       7693 ±  3%  softirqs.CPU92.RCU
     10103 ±  5%     -24.8%       7594 ±  2%  softirqs.CPU93.RCU
     10657 ±  9%     -27.6%       7713 ±  3%  softirqs.CPU94.RCU
     10088 ±  4%     -22.8%       7783 ±  5%  softirqs.CPU95.RCU
    980712 ±  4%     -23.2%     753584        softirqs.RCU
      0.00       +6.5e+103%      65.00 ±123%  interrupts.192:PCI-MSI.31983644-edge.i40e-eth1-TxRx-27
 1.584e+08           -24.6%  1.194e+08        interrupts.CAL:Function_call_interrupts
   1553818           -21.9%    1212996        interrupts.CPU0.CAL:Function_call_interrupts
      1916 ± 20%     +99.9%       3829 ±  2%  interrupts.CPU0.NMI:Non-maskable_interrupts
      1916 ± 20%     +99.9%       3829 ±  2%  interrupts.CPU0.PMI:Performance_monitoring_interrupts
    451.50 ± 10%    +231.5%       1496 ±  9%  interrupts.CPU0.RES:Rescheduling_interrupts
     92.83 ± 27%     +34.5%     124.83 ± 12%  interrupts.CPU0.TLB:TLB_shootdowns
   1555201           -22.0%    1213457        interrupts.CPU1.CAL:Function_call_interrupts
      1693 ± 36%    +134.4%       3969 ±  3%  interrupts.CPU1.NMI:Non-maskable_interrupts
      1693 ± 36%    +134.4%       3969 ±  3%  interrupts.CPU1.PMI:Performance_monitoring_interrupts
    397.67 ±  6%    +280.2%       1512 ±  9%  interrupts.CPU1.RES:Rescheduling_interrupts
   1560119           -22.1%    1215683        interrupts.CPU10.CAL:Function_call_interrupts
    365.67 ± 10%    +290.6%       1428 ± 11%  interrupts.CPU10.RES:Rescheduling_interrupts
   1560624           -22.1%    1215999        interrupts.CPU11.CAL:Function_call_interrupts
    366.50 ± 14%    +287.1%       1418 ± 11%  interrupts.CPU11.RES:Rescheduling_interrupts
   1561507           -22.1%    1215913        interrupts.CPU12.CAL:Function_call_interrupts
    373.00 ±  8%    +289.9%       1454 ± 13%  interrupts.CPU12.RES:Rescheduling_interrupts
   1562400           -22.1%    1216393        interrupts.CPU13.CAL:Function_call_interrupts
    355.17 ± 11%    +303.6%       1433 ± 14%  interrupts.CPU13.RES:Rescheduling_interrupts
     80.67 ± 23%     +33.7%     107.83 ± 12%  interrupts.CPU13.TLB:TLB_shootdowns
   1562669           -22.2%    1216425        interrupts.CPU14.CAL:Function_call_interrupts
    398.50 ± 13%    +271.4%       1480 ± 11%  interrupts.CPU14.RES:Rescheduling_interrupts
   1564260           -22.2%    1217102        interrupts.CPU15.CAL:Function_call_interrupts
    346.50 ±  9%    +306.3%       1408 ±  8%  interrupts.CPU15.RES:Rescheduling_interrupts
   1564778           -22.2%    1217389        interrupts.CPU16.CAL:Function_call_interrupts
    344.67 ± 14%    +284.7%       1325 ± 11%  interrupts.CPU16.RES:Rescheduling_interrupts
     71.00 ± 10%     +54.5%     109.67 ± 19%  interrupts.CPU16.TLB:TLB_shootdowns
   1565548           -22.2%    1217621        interrupts.CPU17.CAL:Function_call_interrupts
    343.50 ± 14%    +307.0%       1398 ± 11%  interrupts.CPU17.RES:Rescheduling_interrupts
   1566249           -22.3%    1217608        interrupts.CPU18.CAL:Function_call_interrupts
    323.00 ±  8%    +334.4%       1403 ± 10%  interrupts.CPU18.RES:Rescheduling_interrupts
     74.50 ± 12%     +54.1%     114.83 ± 25%  interrupts.CPU18.TLB:TLB_shootdowns
   1566835           -22.2%    1218368        interrupts.CPU19.CAL:Function_call_interrupts
    357.50 ± 13%    +286.9%       1383 ± 15%  interrupts.CPU19.RES:Rescheduling_interrupts
     76.33 ± 15%     +41.7%     108.17 ± 21%  interrupts.CPU19.TLB:TLB_shootdowns
   1555036           -21.9%    1213727        interrupts.CPU2.CAL:Function_call_interrupts
      1916 ± 29%    +103.5%       3900 ±  3%  interrupts.CPU2.NMI:Non-maskable_interrupts
      1916 ± 29%    +103.5%       3900 ±  3%  interrupts.CPU2.PMI:Performance_monitoring_interrupts
    405.67 ±  3%    +255.6%       1442 ± 13%  interrupts.CPU2.RES:Rescheduling_interrupts
   1566774           -22.2%    1218796        interrupts.CPU20.CAL:Function_call_interrupts
    379.00 ±  9%    +267.4%       1392 ± 11%  interrupts.CPU20.RES:Rescheduling_interrupts
   1567370           -22.3%    1218379        interrupts.CPU21.CAL:Function_call_interrupts
    384.67 ±  7%    +276.9%       1450 ±  9%  interrupts.CPU21.RES:Rescheduling_interrupts
   1568808           -22.3%    1219084        interrupts.CPU22.CAL:Function_call_interrupts
    363.83 ±  8%    +288.5%       1413 ±  9%  interrupts.CPU22.RES:Rescheduling_interrupts
   1570464           -22.3%    1219491        interrupts.CPU23.CAL:Function_call_interrupts
    331.33 ± 16%    +309.1%       1355 ± 12%  interrupts.CPU23.RES:Rescheduling_interrupts
     69.00 ± 22%     +58.7%     109.50 ± 28%  interrupts.CPU23.TLB:TLB_shootdowns
   1570852           -22.4%    1218594        interrupts.CPU24.CAL:Function_call_interrupts
    360.67 ± 10%    +300.6%       1444 ± 15%  interrupts.CPU24.RES:Rescheduling_interrupts
   1572421           -22.5%    1218832        interrupts.CPU25.CAL:Function_call_interrupts
    361.50 ± 12%    +313.7%       1495 ± 18%  interrupts.CPU25.RES:Rescheduling_interrupts
   1573135           -22.5%    1219681        interrupts.CPU26.CAL:Function_call_interrupts
    346.50 ±  8%    +312.0%       1427 ± 16%  interrupts.CPU26.RES:Rescheduling_interrupts
     81.83 ± 22%     +50.1%     122.83 ± 20%  interrupts.CPU26.TLB:TLB_shootdowns
      0.00       +6.4e+103%      64.33 ±124%  interrupts.CPU27.192:PCI-MSI.31983644-edge.i40e-eth1-TxRx-27
   1573607           -22.5%    1219860        interrupts.CPU27.CAL:Function_call_interrupts
    364.33 ± 11%    +299.3%       1454 ± 18%  interrupts.CPU27.RES:Rescheduling_interrupts
   1574486           -22.5%    1220253        interrupts.CPU28.CAL:Function_call_interrupts
    366.17 ±  4%    +272.5%       1363 ± 16%  interrupts.CPU28.RES:Rescheduling_interrupts
     73.67 ± 22%     +57.2%     115.83 ± 15%  interrupts.CPU28.TLB:TLB_shootdowns
   1576181           -22.5%    1220758        interrupts.CPU29.CAL:Function_call_interrupts
    338.00 ± 12%    +308.9%       1382 ± 14%  interrupts.CPU29.RES:Rescheduling_interrupts
     72.17 ± 18%     +71.4%     123.67 ± 21%  interrupts.CPU29.TLB:TLB_shootdowns
   1555885           -22.0%    1213686        interrupts.CPU3.CAL:Function_call_interrupts
    408.33 ±  9%    +252.6%       1439 ±  8%  interrupts.CPU3.RES:Rescheduling_interrupts
   1577545           -22.6%    1220782        interrupts.CPU30.CAL:Function_call_interrupts
    340.50 ± 10%    +306.1%       1382 ± 12%  interrupts.CPU30.RES:Rescheduling_interrupts
   1579433           -22.7%    1221357        interrupts.CPU31.CAL:Function_call_interrupts
    340.83 ±  7%    +295.1%       1346 ± 18%  interrupts.CPU31.RES:Rescheduling_interrupts
   1580661           -22.7%    1221936        interrupts.CPU32.CAL:Function_call_interrupts
    342.17 ±  8%    +281.4%       1305 ± 12%  interrupts.CPU32.RES:Rescheduling_interrupts
     60.33 ± 23%     +76.5%     106.50 ± 22%  interrupts.CPU32.TLB:TLB_shootdowns
   1581769           -22.7%    1222330        interrupts.CPU33.CAL:Function_call_interrupts
      2003 ± 32%    +105.4%       4115 ±  4%  interrupts.CPU33.NMI:Non-maskable_interrupts
      2003 ± 32%    +105.4%       4115 ±  4%  interrupts.CPU33.PMI:Performance_monitoring_interrupts
    342.00 ± 12%    +301.3%       1372 ± 12%  interrupts.CPU33.RES:Rescheduling_interrupts
   1582830           -22.8%    1222702        interrupts.CPU34.CAL:Function_call_interrupts
    357.00 ±  7%    +288.3%       1386 ± 14%  interrupts.CPU34.RES:Rescheduling_interrupts
   1586009           -22.9%    1223200        interrupts.CPU35.CAL:Function_call_interrupts
    318.67 ± 13%    +291.8%       1248 ± 10%  interrupts.CPU35.RES:Rescheduling_interrupts
   1589398           -23.0%    1223144        interrupts.CPU36.CAL:Function_call_interrupts
      2551 ± 10%     +61.6%       4122 ±  5%  interrupts.CPU36.NMI:Non-maskable_interrupts
      2551 ± 10%     +61.6%       4122 ±  5%  interrupts.CPU36.PMI:Performance_monitoring_interrupts
    327.17 ±  9%    +332.6%       1415 ± 16%  interrupts.CPU36.RES:Rescheduling_interrupts
     79.33 ± 24%     +62.6%     129.00 ± 19%  interrupts.CPU36.TLB:TLB_shootdowns
   1592078           -23.1%    1223994        interrupts.CPU37.CAL:Function_call_interrupts
      2525 ±  5%     +62.1%       4095 ±  5%  interrupts.CPU37.NMI:Non-maskable_interrupts
      2525 ±  5%     +62.1%       4095 ±  5%  interrupts.CPU37.PMI:Performance_monitoring_interrupts
    327.67 ± 15%    +300.2%       1311 ± 11%  interrupts.CPU37.RES:Rescheduling_interrupts
   1593159           -23.1%    1224772        interrupts.CPU38.CAL:Function_call_interrupts
      2449 ±  5%     +71.6%       4203 ±  3%  interrupts.CPU38.NMI:Non-maskable_interrupts
      2449 ±  5%     +71.6%       4203 ±  3%  interrupts.CPU38.PMI:Performance_monitoring_interrupts
    329.17 ± 11%    +284.5%       1265 ±  9%  interrupts.CPU38.RES:Rescheduling_interrupts
     77.33 ± 15%     +53.0%     118.33 ± 17%  interrupts.CPU38.TLB:TLB_shootdowns
   1597729           -23.3%    1225532        interrupts.CPU39.CAL:Function_call_interrupts
      2375 ±  7%     +75.5%       4169 ±  4%  interrupts.CPU39.NMI:Non-maskable_interrupts
      2375 ±  7%     +75.5%       4169 ±  4%  interrupts.CPU39.PMI:Performance_monitoring_interrupts
    321.33 ± 15%    +294.8%       1268 ± 10%  interrupts.CPU39.RES:Rescheduling_interrupts
     68.33 ± 35%     +68.3%     115.00 ± 26%  interrupts.CPU39.TLB:TLB_shootdowns
   1556572           -22.0%    1213891        interrupts.CPU4.CAL:Function_call_interrupts
    370.17 ± 12%    +288.8%       1439 ± 10%  interrupts.CPU4.RES:Rescheduling_interrupts
     76.17 ± 15%     +49.9%     114.17 ± 12%  interrupts.CPU4.TLB:TLB_shootdowns
   1599710           -23.4%    1226042        interrupts.CPU40.CAL:Function_call_interrupts
      2477 ± 12%     +67.3%       4144 ±  5%  interrupts.CPU40.NMI:Non-maskable_interrupts
      2477 ± 12%     +67.3%       4144 ±  5%  interrupts.CPU40.PMI:Performance_monitoring_interrupts
    324.83 ± 15%    +304.5%       1314 ± 11%  interrupts.CPU40.RES:Rescheduling_interrupts
   1602951           -23.5%    1226709        interrupts.CPU41.CAL:Function_call_interrupts
      2321 ±  8%     +79.0%       4155 ±  3%  interrupts.CPU41.NMI:Non-maskable_interrupts
      2321 ±  8%     +79.0%       4155 ±  3%  interrupts.CPU41.PMI:Performance_monitoring_interrupts
    320.67 ±  8%    +285.3%       1235 ± 15%  interrupts.CPU41.RES:Rescheduling_interrupts
     73.00 ± 18%     +55.9%     113.83 ± 19%  interrupts.CPU41.TLB:TLB_shootdowns
   1610138           -23.8%    1227114        interrupts.CPU42.CAL:Function_call_interrupts
      2314 ±  5%     +78.4%       4128 ±  5%  interrupts.CPU42.NMI:Non-maskable_interrupts
      2314 ±  5%     +78.4%       4128 ±  5%  interrupts.CPU42.PMI:Performance_monitoring_interrupts
    310.33 ±  8%    +312.2%       1279 ± 22%  interrupts.CPU42.RES:Rescheduling_interrupts
     66.17 ± 20%     +80.4%     119.33 ± 30%  interrupts.CPU42.TLB:TLB_shootdowns
   1611973           -23.8%    1227608        interrupts.CPU43.CAL:Function_call_interrupts
      2347 ±  9%     +75.8%       4125 ±  5%  interrupts.CPU43.NMI:Non-maskable_interrupts
      2347 ±  9%     +75.8%       4125 ±  5%  interrupts.CPU43.PMI:Performance_monitoring_interrupts
    303.50 ±  8%    +310.8%       1246 ± 13%  interrupts.CPU43.RES:Rescheduling_interrupts
     64.50 ± 16%    +117.1%     140.00 ± 24%  interrupts.CPU43.TLB:TLB_shootdowns
   1614258           -23.9%    1228447        interrupts.CPU44.CAL:Function_call_interrupts
      2414 ±  8%     +70.8%       4123 ±  5%  interrupts.CPU44.NMI:Non-maskable_interrupts
      2414 ±  8%     +70.8%       4123 ±  5%  interrupts.CPU44.PMI:Performance_monitoring_interrupts
    305.33 ±  7%    +316.4%       1271 ± 12%  interrupts.CPU44.RES:Rescheduling_interrupts
   1618727           -24.0%    1229501        interrupts.CPU45.CAL:Function_call_interrupts
      2425 ±  8%     +71.1%       4150 ±  6%  interrupts.CPU45.NMI:Non-maskable_interrupts
      2425 ±  8%     +71.1%       4150 ±  6%  interrupts.CPU45.PMI:Performance_monitoring_interrupts
    292.83 ±  7%    +320.0%       1229 ± 10%  interrupts.CPU45.RES:Rescheduling_interrupts
   1620732           -24.1%    1229767        interrupts.CPU46.CAL:Function_call_interrupts
      2152 ± 20%     +91.7%       4126 ±  5%  interrupts.CPU46.NMI:Non-maskable_interrupts
      2152 ± 20%     +91.7%       4126 ±  5%  interrupts.CPU46.PMI:Performance_monitoring_interrupts
    307.33 ± 14%    +313.6%       1271 ± 17%  interrupts.CPU46.RES:Rescheduling_interrupts
     63.17 ± 22%     +91.6%     121.00 ± 22%  interrupts.CPU46.TLB:TLB_shootdowns
   1625711           -24.3%    1230630        interrupts.CPU47.CAL:Function_call_interrupts
      1913 ± 29%    +116.8%       4147 ±  3%  interrupts.CPU47.NMI:Non-maskable_interrupts
      1913 ± 29%    +116.8%       4147 ±  3%  interrupts.CPU47.PMI:Performance_monitoring_interrupts
    310.33 ±  9%    +331.0%       1337 ± 17%  interrupts.CPU47.RES:Rescheduling_interrupts
     63.67 ± 20%    +103.9%     129.83 ± 28%  interrupts.CPU47.TLB:TLB_shootdowns
   1635071           -24.6%    1233173        interrupts.CPU48.CAL:Function_call_interrupts
      1749 ± 33%    +117.6%       3806 ±  2%  interrupts.CPU48.NMI:Non-maskable_interrupts
      1749 ± 33%    +117.6%       3806 ±  2%  interrupts.CPU48.PMI:Performance_monitoring_interrupts
    440.50 ± 13%    +285.3%       1697 ± 15%  interrupts.CPU48.RES:Rescheduling_interrupts
   1636847           -24.6%    1234586        interrupts.CPU49.CAL:Function_call_interrupts
      1907 ± 32%    +107.9%       3965 ±  3%  interrupts.CPU49.NMI:Non-maskable_interrupts
      1907 ± 32%    +107.9%       3965 ±  3%  interrupts.CPU49.PMI:Performance_monitoring_interrupts
    528.83 ± 14%    +261.5%       1911 ±  8%  interrupts.CPU49.RES:Rescheduling_interrupts
     63.67 ±  9%     +74.3%     111.00 ± 19%  interrupts.CPU49.TLB:TLB_shootdowns
   1557508           -22.0%    1214275        interrupts.CPU5.CAL:Function_call_interrupts
    360.00 ± 11%    +304.7%       1456 ± 13%  interrupts.CPU5.RES:Rescheduling_interrupts
   1638985           -24.6%    1235069        interrupts.CPU50.CAL:Function_call_interrupts
      1916 ± 28%    +103.7%       3903 ±  4%  interrupts.CPU50.NMI:Non-maskable_interrupts
      1916 ± 28%    +103.7%       3903 ±  4%  interrupts.CPU50.PMI:Performance_monitoring_interrupts
    543.83 ±  8%    +252.8%       1918 ± 14%  interrupts.CPU50.RES:Rescheduling_interrupts
     81.67 ± 11%     +34.9%     110.17 ± 13%  interrupts.CPU50.TLB:TLB_shootdowns
   1644500           -24.8%    1236087        interrupts.CPU51.CAL:Function_call_interrupts
      1849 ± 35%    +116.5%       4003 ±  2%  interrupts.CPU51.NMI:Non-maskable_interrupts
      1849 ± 35%    +116.5%       4003 ±  2%  interrupts.CPU51.PMI:Performance_monitoring_interrupts
    521.17 ± 12%    +261.6%       1884 ±  7%  interrupts.CPU51.RES:Rescheduling_interrupts
     73.17 ± 22%     +47.4%     107.83 ± 28%  interrupts.CPU51.TLB:TLB_shootdowns
   1645993           -24.8%    1237456        interrupts.CPU52.CAL:Function_call_interrupts
    529.00 ±  5%    +265.5%       1933 ±  8%  interrupts.CPU52.RES:Rescheduling_interrupts
     73.00 ± 16%     +50.0%     109.50 ± 16%  interrupts.CPU52.TLB:TLB_shootdowns
   1649801           -25.0%    1238136        interrupts.CPU53.CAL:Function_call_interrupts
    543.17 ±  7%    +249.9%       1900 ± 12%  interrupts.CPU53.RES:Rescheduling_interrupts
     73.50 ± 24%     +54.4%     113.50 ± 18%  interrupts.CPU53.TLB:TLB_shootdowns
   1654935           -25.1%    1239258        interrupts.CPU54.CAL:Function_call_interrupts
    560.50 ±  8%    +239.8%       1904 ±  9%  interrupts.CPU54.RES:Rescheduling_interrupts
     72.33 ± 13%     +53.7%     111.17 ± 15%  interrupts.CPU54.TLB:TLB_shootdowns
   1667829           -25.6%    1240627        interrupts.CPU55.CAL:Function_call_interrupts
    553.50 ± 13%    +251.8%       1947 ±  7%  interrupts.CPU55.RES:Rescheduling_interrupts
   1673782           -25.8%    1241296        interrupts.CPU56.CAL:Function_call_interrupts
      2134 ± 26%     +85.3%       3955 ±  3%  interrupts.CPU56.NMI:Non-maskable_interrupts
      2134 ± 26%     +85.3%       3955 ±  3%  interrupts.CPU56.PMI:Performance_monitoring_interrupts
    558.50 ± 11%    +232.8%       1858 ± 10%  interrupts.CPU56.RES:Rescheduling_interrupts
   1673858           -25.7%    1242929        interrupts.CPU57.CAL:Function_call_interrupts
      2169 ± 23%     +84.0%       3990 ±  4%  interrupts.CPU57.NMI:Non-maskable_interrupts
      2169 ± 23%     +84.0%       3990 ±  4%  interrupts.CPU57.PMI:Performance_monitoring_interrupts
    558.17 ±  2%    +259.2%       2004 ± 11%  interrupts.CPU57.RES:Rescheduling_interrupts
   1675001           -25.8%    1243500        interrupts.CPU58.CAL:Function_call_interrupts
      2143 ± 25%     +82.1%       3903 ±  3%  interrupts.CPU58.NMI:Non-maskable_interrupts
      2143 ± 25%     +82.1%       3903 ±  3%  interrupts.CPU58.PMI:Performance_monitoring_interrupts
    586.00 ± 13%    +241.5%       2001 ±  9%  interrupts.CPU58.RES:Rescheduling_interrupts
     76.67 ±  9%     +33.5%     102.33 ± 11%  interrupts.CPU58.TLB:TLB_shootdowns
   1677140           -25.8%    1245091        interrupts.CPU59.CAL:Function_call_interrupts
    603.83 ± 12%    +219.1%       1927 ±  8%  interrupts.CPU59.RES:Rescheduling_interrupts
   1558142           -22.1%    1214507        interrupts.CPU6.CAL:Function_call_interrupts
    345.83 ±  6%    +304.8%       1399 ±  8%  interrupts.CPU6.RES:Rescheduling_interrupts
     78.17 ± 18%     +47.8%     115.50 ± 15%  interrupts.CPU6.TLB:TLB_shootdowns
   1682308           -25.9%    1246000        interrupts.CPU60.CAL:Function_call_interrupts
      2358 ±  8%     +69.8%       4005 ±  4%  interrupts.CPU60.NMI:Non-maskable_interrupts
      2358 ±  8%     +69.8%       4005 ±  4%  interrupts.CPU60.PMI:Performance_monitoring_interrupts
    556.00 ±  4%    +254.7%       1972 ± 11%  interrupts.CPU60.RES:Rescheduling_interrupts
   1687753           -26.1%    1247152        interrupts.CPU61.CAL:Function_call_interrupts
    571.67 ± 12%    +238.6%       1935 ± 14%  interrupts.CPU61.RES:Rescheduling_interrupts
   1691195           -26.2%    1248322        interrupts.CPU62.CAL:Function_call_interrupts
    554.00 ± 11%    +256.3%       1973 ±  4%  interrupts.CPU62.RES:Rescheduling_interrupts
   1693421           -26.2%    1249765        interrupts.CPU63.CAL:Function_call_interrupts
    578.00 ±  8%    +252.9%       2040 ± 12%  interrupts.CPU63.RES:Rescheduling_interrupts
   1693802           -26.2%    1250259        interrupts.CPU64.CAL:Function_call_interrupts
    576.50 ± 13%    +237.9%       1947 ±  4%  interrupts.CPU64.RES:Rescheduling_interrupts
     70.83 ± 14%     +48.2%     105.00 ± 18%  interrupts.CPU64.TLB:TLB_shootdowns
   1696044           -26.2%    1252071        interrupts.CPU65.CAL:Function_call_interrupts
      2000 ± 20%     +80.8%       3616 ± 17%  interrupts.CPU65.NMI:Non-maskable_interrupts
      2000 ± 20%     +80.8%       3616 ± 17%  interrupts.CPU65.PMI:Performance_monitoring_interrupts
    592.67 ± 17%    +238.7%       2007 ±  6%  interrupts.CPU65.RES:Rescheduling_interrupts
   1698659           -26.3%    1252352        interrupts.CPU66.CAL:Function_call_interrupts
    544.00 ± 11%    +261.3%       1965 ± 10%  interrupts.CPU66.RES:Rescheduling_interrupts
   1703027           -26.3%    1254746        interrupts.CPU67.CAL:Function_call_interrupts
    558.33 ± 16%    +264.0%       2032 ± 12%  interrupts.CPU67.RES:Rescheduling_interrupts
   1705223           -26.3%    1256151        interrupts.CPU68.CAL:Function_call_interrupts
    546.50 ± 13%    +271.8%       2031 ± 12%  interrupts.CPU68.RES:Rescheduling_interrupts
   1706279           -26.3%    1257075        interrupts.CPU69.CAL:Function_call_interrupts
      2204 ± 26%     +84.2%       4061 ±  2%  interrupts.CPU69.NMI:Non-maskable_interrupts
      2204 ± 26%     +84.2%       4061 ±  2%  interrupts.CPU69.PMI:Performance_monitoring_interrupts
    594.17 ±  8%    +233.8%       1983 ±  9%  interrupts.CPU69.RES:Rescheduling_interrupts
   1558831           -22.1%    1214955        interrupts.CPU7.CAL:Function_call_interrupts
    375.17 ± 12%    +281.9%       1432 ±  7%  interrupts.CPU7.RES:Rescheduling_interrupts
     80.33 ± 17%     +32.0%     106.00 ± 17%  interrupts.CPU7.TLB:TLB_shootdowns
   1708194           -26.3%    1259777        interrupts.CPU70.CAL:Function_call_interrupts
      2167 ± 19%     +88.7%       4090 ±  2%  interrupts.CPU70.NMI:Non-maskable_interrupts
      2167 ± 19%     +88.7%       4090 ±  2%  interrupts.CPU70.PMI:Performance_monitoring_interrupts
    609.17 ±  3%    +235.2%       2042 ±  6%  interrupts.CPU70.RES:Rescheduling_interrupts
   1713148           -26.4%    1261613        interrupts.CPU71.CAL:Function_call_interrupts
      2097 ± 27%     +96.9%       4130 ±  3%  interrupts.CPU71.NMI:Non-maskable_interrupts
      2097 ± 27%     +96.9%       4130 ±  3%  interrupts.CPU71.PMI:Performance_monitoring_interrupts
    578.50 ± 15%    +254.4%       2050 ± 12%  interrupts.CPU71.RES:Rescheduling_interrupts
     70.17 ± 16%     +54.6%     108.50 ± 21%  interrupts.CPU71.TLB:TLB_shootdowns
   1718192           -26.6%    1260936        interrupts.CPU72.CAL:Function_call_interrupts
      1990 ± 26%    +111.9%       4217 ±  6%  interrupts.CPU72.NMI:Non-maskable_interrupts
      1990 ± 26%    +111.9%       4217 ±  6%  interrupts.CPU72.PMI:Performance_monitoring_interrupts
    519.33 ±  9%    +267.9%       1910 ± 21%  interrupts.CPU72.RES:Rescheduling_interrupts
   1722067           -26.7%    1262027        interrupts.CPU73.CAL:Function_call_interrupts
      2142 ± 23%     +92.1%       4116 ±  3%  interrupts.CPU73.NMI:Non-maskable_interrupts
      2142 ± 23%     +92.1%       4116 ±  3%  interrupts.CPU73.PMI:Performance_monitoring_interrupts
    531.17 ± 13%    +271.4%       1972 ± 12%  interrupts.CPU73.RES:Rescheduling_interrupts
   1724351           -26.6%    1264962        interrupts.CPU74.CAL:Function_call_interrupts
      2222 ± 23%     +93.6%       4301 ±  6%  interrupts.CPU74.NMI:Non-maskable_interrupts
      2222 ± 23%     +93.6%       4301 ±  6%  interrupts.CPU74.PMI:Performance_monitoring_interrupts
    547.17 ± 10%    +287.1%       2118 ± 16%  interrupts.CPU74.RES:Rescheduling_interrupts
     73.50 ± 22%     +56.7%     115.17 ± 18%  interrupts.CPU74.TLB:TLB_shootdowns
   1724876           -26.6%    1265941        interrupts.CPU75.CAL:Function_call_interrupts
      2246 ± 24%     +88.3%       4230 ±  6%  interrupts.CPU75.NMI:Non-maskable_interrupts
      2246 ± 24%     +88.3%       4230 ±  6%  interrupts.CPU75.PMI:Performance_monitoring_interrupts
    530.83 ±  9%    +276.8%       2000 ± 12%  interrupts.CPU75.RES:Rescheduling_interrupts
   1726311           -26.5%    1268269        interrupts.CPU76.CAL:Function_call_interrupts
      2045 ± 25%    +105.6%       4206 ±  5%  interrupts.CPU76.NMI:Non-maskable_interrupts
      2045 ± 25%    +105.6%       4206 ±  5%  interrupts.CPU76.PMI:Performance_monitoring_interrupts
    547.17 ±  9%    +288.8%       2127 ± 14%  interrupts.CPU76.RES:Rescheduling_interrupts
     77.17 ± 19%     +51.4%     116.83 ± 12%  interrupts.CPU76.TLB:TLB_shootdowns
   1728950           -26.5%    1270182        interrupts.CPU77.CAL:Function_call_interrupts
      2285 ± 17%     +91.5%       4377 ±  4%  interrupts.CPU77.NMI:Non-maskable_interrupts
      2285 ± 17%     +91.5%       4377 ±  4%  interrupts.CPU77.PMI:Performance_monitoring_interrupts
    564.67 ±  6%    +269.6%       2087 ± 11%  interrupts.CPU77.RES:Rescheduling_interrupts
     74.17 ± 21%     +65.4%     122.67 ± 23%  interrupts.CPU77.TLB:TLB_shootdowns
   1733767           -26.7%    1271681        interrupts.CPU78.CAL:Function_call_interrupts
      1760 ± 33%    +135.2%       4140 ±  4%  interrupts.CPU78.NMI:Non-maskable_interrupts
      1760 ± 33%    +135.2%       4140 ±  4%  interrupts.CPU78.PMI:Performance_monitoring_interrupts
    536.67 ±  7%    +272.3%       1998 ± 13%  interrupts.CPU78.RES:Rescheduling_interrupts
   1739078           -26.8%    1273254        interrupts.CPU79.CAL:Function_call_interrupts
      2014 ± 24%    +108.6%       4200 ±  3%  interrupts.CPU79.NMI:Non-maskable_interrupts
      2014 ± 24%    +108.6%       4200 ±  3%  interrupts.CPU79.PMI:Performance_monitoring_interrupts
    550.83 ± 11%    +256.2%       1961 ± 12%  interrupts.CPU79.RES:Rescheduling_interrupts
     80.17 ± 22%     +62.4%     130.17 ± 20%  interrupts.CPU79.TLB:TLB_shootdowns
   1560071           -22.1%    1214673        interrupts.CPU8.CAL:Function_call_interrupts
    376.67 ±  7%    +307.9%       1536 ± 16%  interrupts.CPU8.RES:Rescheduling_interrupts
   1739255           -26.6%    1276667        interrupts.CPU80.CAL:Function_call_interrupts
      1776 ± 38%    +134.2%       4160 ±  3%  interrupts.CPU80.NMI:Non-maskable_interrupts
      1776 ± 38%    +134.2%       4160 ±  3%  interrupts.CPU80.PMI:Performance_monitoring_interrupts
    593.00 ±  8%    +247.9%       2063 ± 12%  interrupts.CPU80.RES:Rescheduling_interrupts
     65.17 ± 20%     +72.6%     112.50 ± 24%  interrupts.CPU80.TLB:TLB_shootdowns
   1744994           -26.7%    1279747        interrupts.CPU81.CAL:Function_call_interrupts
      2179 ± 18%     +72.9%       3768 ± 17%  interrupts.CPU81.NMI:Non-maskable_interrupts
      2179 ± 18%     +72.9%       3768 ± 17%  interrupts.CPU81.PMI:Performance_monitoring_interrupts
    556.33 ±  6%    +283.2%       2131 ± 15%  interrupts.CPU81.RES:Rescheduling_interrupts
     84.00 ± 19%     +42.3%     119.50 ± 30%  interrupts.CPU81.TLB:TLB_shootdowns
   1746355           -26.5%    1283194        interrupts.CPU82.CAL:Function_call_interrupts
    580.67 ± 13%    +275.6%       2181 ± 11%  interrupts.CPU82.RES:Rescheduling_interrupts
     82.33 ± 23%     +50.2%     123.67 ± 11%  interrupts.CPU82.TLB:TLB_shootdowns
   1752956           -26.6%    1286255        interrupts.CPU83.CAL:Function_call_interrupts
    581.17 ± 11%    +267.1%       2133 ± 12%  interrupts.CPU83.RES:Rescheduling_interrupts
   1764860           -26.9%    1289469        interrupts.CPU84.CAL:Function_call_interrupts
    586.50 ± 12%    +244.7%       2021 ± 12%  interrupts.CPU84.RES:Rescheduling_interrupts
   1771742           -27.0%    1293323        interrupts.CPU85.CAL:Function_call_interrupts
    588.50 ± 11%    +256.0%       2095 ± 10%  interrupts.CPU85.RES:Rescheduling_interrupts
   1771025           -26.8%    1296747        interrupts.CPU86.CAL:Function_call_interrupts
      2281 ± 18%     +85.3%       4229 ±  3%  interrupts.CPU86.NMI:Non-maskable_interrupts
      2281 ± 18%     +85.3%       4229 ±  3%  interrupts.CPU86.PMI:Performance_monitoring_interrupts
    580.50 ±  6%    +270.5%       2150 ± 11%  interrupts.CPU86.RES:Rescheduling_interrupts
   1781041           -27.0%    1300147        interrupts.CPU87.CAL:Function_call_interrupts
    580.17 ±  8%    +261.4%       2096 ±  8%  interrupts.CPU87.RES:Rescheduling_interrupts
   1784978           -26.9%    1304062        interrupts.CPU88.CAL:Function_call_interrupts
      2354 ± 26%     +78.2%       4195 ±  5%  interrupts.CPU88.NMI:Non-maskable_interrupts
      2354 ± 26%     +78.2%       4195 ±  5%  interrupts.CPU88.PMI:Performance_monitoring_interrupts
    594.33 ± 11%    +265.4%       2171 ± 11%  interrupts.CPU88.RES:Rescheduling_interrupts
   1790968           -26.9%    1308585        interrupts.CPU89.CAL:Function_call_interrupts
    629.67 ±  8%    +241.2%       2148 ±  9%  interrupts.CPU89.RES:Rescheduling_interrupts
   1559678           -22.1%    1215175        interrupts.CPU9.CAL:Function_call_interrupts
    371.00 ± 17%    +297.6%       1475 ± 11%  interrupts.CPU9.RES:Rescheduling_interrupts
   1819853           -28.0%    1310330        interrupts.CPU90.CAL:Function_call_interrupts
    545.67 ±  6%    +283.6%       2093 ± 15%  interrupts.CPU90.RES:Rescheduling_interrupts
     67.17 ± 20%     +66.3%     111.67 ± 29%  interrupts.CPU90.TLB:TLB_shootdowns
   1817004           -27.7%    1313826        interrupts.CPU91.CAL:Function_call_interrupts
      1953 ± 25%    +114.4%       4189 ±  5%  interrupts.CPU91.NMI:Non-maskable_interrupts
      1953 ± 25%    +114.4%       4189 ±  5%  interrupts.CPU91.PMI:Performance_monitoring_interrupts
    580.67 ± 14%    +263.2%       2108 ± 13%  interrupts.CPU91.RES:Rescheduling_interrupts
     65.33 ± 17%    +109.4%     136.83 ± 22%  interrupts.CPU91.TLB:TLB_shootdowns
   1820669           -27.7%    1316902        interrupts.CPU92.CAL:Function_call_interrupts
      2018 ± 23%     +88.0%       3794 ± 18%  interrupts.CPU92.NMI:Non-maskable_interrupts
      2018 ± 23%     +88.0%       3794 ± 18%  interrupts.CPU92.PMI:Performance_monitoring_interrupts
    565.67 ±  8%    +269.1%       2087 ± 15%  interrupts.CPU92.RES:Rescheduling_interrupts
   1828689           -27.7%    1321787        interrupts.CPU93.CAL:Function_call_interrupts
      2272 ± 21%     +66.8%       3789 ± 16%  interrupts.CPU93.NMI:Non-maskable_interrupts
      2272 ± 21%     +66.8%       3789 ± 16%  interrupts.CPU93.PMI:Performance_monitoring_interrupts
    584.17 ±  5%    +269.9%       2160 ±  6%  interrupts.CPU93.RES:Rescheduling_interrupts
     70.33 ± 18%     +51.4%     106.50 ± 24%  interrupts.CPU93.TLB:TLB_shootdowns
   1825869           -27.5%    1323766        interrupts.CPU94.CAL:Function_call_interrupts
    594.50 ±  6%    +245.0%       2051 ± 10%  interrupts.CPU94.RES:Rescheduling_interrupts
   1835732           -27.0%    1340021        interrupts.CPU95.CAL:Function_call_interrupts
      2131 ± 18%     +81.2%       3862 ± 20%  interrupts.CPU95.NMI:Non-maskable_interrupts
      2131 ± 18%     +81.2%       3862 ± 20%  interrupts.CPU95.PMI:Performance_monitoring_interrupts
    530.83 ±  6%    +304.6%       2147 ± 13%  interrupts.CPU95.RES:Rescheduling_interrupts
     80.33 ±  7%     +79.3%     144.00 ± 23%  interrupts.CPU95.TLB:TLB_shootdowns
    205031 ±  6%     +77.8%     364501        interrupts.NMI:Non-maskable_interrupts
    205031 ±  6%     +77.8%     364501        interrupts.PMI:Performance_monitoring_interrupts
     43840 ±  2%    +272.5%     163324 ±  2%  interrupts.RES:Rescheduling_interrupts
      7385           +49.5%      11042 ±  3%  interrupts.TLB:TLB_shootdowns
     52.22           -29.6       22.66        perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
     19.19 ±  3%      -8.4       10.84        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
     19.26 ±  3%      -8.4       10.91        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
      7.11 ±  5%      -3.5        3.64        perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
      7.11 ±  5%      -3.5        3.64        perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
      7.10 ±  5%      -3.5        3.64        perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
      6.56 ±  5%      -3.2        3.34        perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      6.53 ±  5%      -3.2        3.32        perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group
      5.78 ±  2%      -3.1        2.67        perf-profile.calltrace.cycles-pp.__sysvec_call_function.sysvec_call_function.asm_sysvec_call_function.cpuidle_enter_state.cpuidle_enter
      5.59 ±  2%      -3.0        2.58        perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.__sysvec_call_function.sysvec_call_function.asm_sysvec_call_function.cpuidle_enter_state
      6.29            -2.9        3.41 ±  2%  perf-profile.calltrace.cycles-pp.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
      6.29            -2.9        3.41 ±  2%  perf-profile.calltrace.cycles-pp.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.93            -2.7        3.21 ±  2%  perf-profile.calltrace.cycles-pp.copy_process.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.27            -2.5        2.73        perf-profile.calltrace.cycles-pp.dup_mm.copy_process.kernel_clone.__do_sys_clone.do_syscall_64
      5.16            -2.5        2.67 ±  2%  perf-profile.calltrace.cycles-pp.dup_mmap.dup_mm.copy_process.kernel_clone.__do_sys_clone
      3.44 ±  2%      -2.3        1.19 ±  2%  perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
      3.88 ±  5%      -1.8        2.04        perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault
      3.85 ±  5%      -1.8        2.02        perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
      3.95 ±  5%      -1.8        2.13 ±  6%  perf-profile.calltrace.cycles-pp.asm_exc_page_fault
      3.35 ±  5%      -1.6        1.76        perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
      3.23 ±  5%      -1.5        1.69        perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
      2.35 ±  5%      -1.2        1.18 ±  4%  perf-profile.calltrace.cycles-pp.free_pgtables.exit_mmap.mmput.do_exit.do_group_exit
      3.86 ±  4%      -1.1        2.73 ±  3%  perf-profile.calltrace.cycles-pp.do_seccomp.do_syscall_64.entry_SYSCALL_64_after_hwframe
      3.85 ±  4%      -1.1        2.73 ±  3%  perf-profile.calltrace.cycles-pp.seccomp_set_mode_filter.do_seccomp.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.25 ±  6%      -1.1        1.16 ±  3%  perf-profile.calltrace.cycles-pp.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
      2.24 ±  6%      -1.1        1.18 ±  3%  perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.do_exit.do_group_exit
      2.17 ±  5%      -1.1        1.12 ±  3%  perf-profile.calltrace.cycles-pp.filemap_map_pages.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
      2.11 ±  7%      -1.0        1.10 ±  3%  perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.do_exit
      1.99 ±  2%      -1.0        0.99 ±  4%  perf-profile.calltrace.cycles-pp.anon_vma_fork.dup_mmap.dup_mm.copy_process.kernel_clone
      1.96 ±  7%      -0.9        1.03 ±  4%  perf-profile.calltrace.cycles-pp.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap.mmput
      1.86 ±  2%      -0.9        0.96        perf-profile.calltrace.cycles-pp.__flush_tlb_all.flush_smp_call_function_queue.__sysvec_call_function.sysvec_call_function.asm_sysvec_call_function
      3.38 ±  4%      -0.9        2.50 ±  3%  perf-profile.calltrace.cycles-pp.bpf_prog_create_from_user.seccomp_set_mode_filter.do_seccomp.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.83 ±  2%      -0.9        0.95        perf-profile.calltrace.cycles-pp.native_flush_tlb_global.__flush_tlb_all.flush_smp_call_function_queue.__sysvec_call_function.sysvec_call_function
      1.11 ±  2%      -0.8        0.26 ±100%  perf-profile.calltrace.cycles-pp.llist_reverse_order.flush_smp_call_function_queue.__sysvec_call_function.sysvec_call_function.asm_sysvec_call_function
      1.63 ±  5%      -0.8        0.82 ±  3%  perf-profile.calltrace.cycles-pp.unlink_anon_vmas.free_pgtables.exit_mmap.mmput.do_exit
      1.47 ±  2%      -0.7        0.74 ±  3%  perf-profile.calltrace.cycles-pp.anon_vma_clone.anon_vma_fork.dup_mmap.dup_mm.copy_process
      1.51 ±  5%      -0.7        0.84 ±  6%  perf-profile.calltrace.cycles-pp.copy_page_range.dup_mmap.dup_mm.copy_process.kernel_clone
      2.07            -0.7        1.41 ±  2%  perf-profile.calltrace.cycles-pp.ret_from_fork
      2.78 ±  4%      -0.6        2.15 ±  3%  perf-profile.calltrace.cycles-pp.bpf_prepare_filter.bpf_prog_create_from_user.seccomp_set_mode_filter.do_seccomp.do_syscall_64
      1.31 ±  8%      -0.6        0.67 ±  3%  perf-profile.calltrace.cycles-pp.tlb_finish_mmu.exit_mmap.mmput.do_exit.do_group_exit
      1.30 ±  8%      -0.6        0.67 ±  3%  perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput.do_exit
      2.74 ±  4%      -0.6        2.13 ±  3%  perf-profile.calltrace.cycles-pp.bpf_prog_select_runtime.bpf_prepare_filter.bpf_prog_create_from_user.seccomp_set_mode_filter.do_seccomp
      2.73 ±  4%      -0.6        2.12 ±  3%  perf-profile.calltrace.cycles-pp.bpf_int_jit_compile.bpf_prog_select_runtime.bpf_prepare_filter.bpf_prog_create_from_user.seccomp_set_mode_filter
      1.27 ±  6%      -0.6        0.67 ±  4%  perf-profile.calltrace.cycles-pp.next_uptodate_page.filemap_map_pages.do_fault.__handle_mm_fault.handle_mm_fault
      1.02 ±  9%      -0.6        0.44 ± 44%  perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput
      1.88            -0.6        1.31 ±  2%  perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
      1.85            -0.6        1.30        perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
      1.22 ±  7%      -0.5        0.68 ±  8%  perf-profile.calltrace.cycles-pp.copy_pte_range.copy_page_range.dup_mmap.dup_mm.copy_process
      1.76            -0.5        1.25 ±  2%  perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
      1.52            -0.4        1.12 ±  2%  perf-profile.calltrace.cycles-pp.bpf_jit_free.process_one_work.worker_thread.kthread.ret_from_fork
      1.51            -0.4        1.11 ±  2%  perf-profile.calltrace.cycles-pp.__vunmap.bpf_jit_binary_free.bpf_jit_free.process_one_work.worker_thread
      1.51            -0.4        1.12 ±  2%  perf-profile.calltrace.cycles-pp.bpf_jit_binary_free.bpf_jit_free.process_one_work.worker_thread.kthread
      1.42            -0.3        1.07        perf-profile.calltrace.cycles-pp._vm_unmap_aliases.__vunmap.bpf_jit_binary_free.bpf_jit_free.process_one_work
      0.68 ±  5%      -0.3        0.35 ± 70%  perf-profile.calltrace.cycles-pp.smp_call_function_many_cond.on_each_cpu.cpa_flush.change_page_attr_set_clr.set_memory_ro
      1.06 ±  3%      -0.3        0.78 ±  3%  perf-profile.calltrace.cycles-pp.change_page_attr_set_clr.set_memory_x.bpf_int_jit_compile.bpf_prog_select_runtime.bpf_prepare_filter
      1.06 ±  3%      -0.3        0.78 ±  3%  perf-profile.calltrace.cycles-pp.set_memory_x.bpf_int_jit_compile.bpf_prog_select_runtime.bpf_prepare_filter.bpf_prog_create_from_user
      1.47 ±  4%      -0.2        1.23 ±  3%  perf-profile.calltrace.cycles-pp.change_page_attr_set_clr.set_memory_ro.bpf_int_jit_compile.bpf_prog_select_runtime.bpf_prepare_filter
      1.47 ±  4%      -0.2        1.23 ±  3%  perf-profile.calltrace.cycles-pp.set_memory_ro.bpf_int_jit_compile.bpf_prog_select_runtime.bpf_prepare_filter.bpf_prog_create_from_user
      0.69 ±  5%      -0.2        0.53 ±  4%  perf-profile.calltrace.cycles-pp.cpa_flush.change_page_attr_set_clr.set_memory_ro.bpf_int_jit_compile.bpf_prog_select_runtime
      0.69 ±  5%      -0.2        0.53 ±  4%  perf-profile.calltrace.cycles-pp.on_each_cpu.cpa_flush.change_page_attr_set_clr.set_memory_ro.bpf_int_jit_compile
      0.67 ± 46%      +0.3        0.94        perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_kernel.secondary_startup_64_no_verify
      0.67 ± 46%      +0.3        0.94        perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64_no_verify
      0.67 ± 46%      +0.3        0.94        perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64_no_verify
      0.62 ± 46%      +0.3        0.92 ±  2%  perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64_no_verify
      0.60 ± 46%      +0.3        0.91 ±  2%  perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel
      0.00            +0.7        0.66        perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
      0.00            +0.7        0.69        perf-profile.calltrace.cycles-pp.ktime_get.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      0.00            +0.7        0.69        perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.sched_clock_tick.scheduler_tick.update_process_times
      0.00            +0.8        0.76        perf-profile.calltrace.cycles-pp.ktime_get.sched_clock_tick.scheduler_tick.update_process_times.tick_sched_handle
      0.00            +0.8        0.76        perf-profile.calltrace.cycles-pp.sched_clock_tick.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer
      0.00            +0.9        0.85 ±  2%  perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      0.00            +0.9        0.90 ±  2%  perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
      0.00            +0.9        0.90        perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      0.00            +1.1        1.06        perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +1.1        1.08        perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      0.00            +1.1        1.09        perf-profile.calltrace.cycles-pp.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      0.00            +1.2        1.17        perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt
      0.00            +1.2        1.18        perf-profile.calltrace.cycles-pp.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +1.2        1.18        perf-profile.calltrace.cycles-pp.read_hpet.ktime_get_update_offsets_now.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      0.00            +1.2        1.20        perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      0.00            +1.2        1.20        perf-profile.calltrace.cycles-pp.sched_clock_tick.sched_clock_idle_wakeup_event.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt
      0.00            +1.2        1.20        perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +1.2        1.22 ±  2%  perf-profile.calltrace.cycles-pp.sched_clock_idle_wakeup_event.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +1.3        1.29        perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      0.00            +1.3        1.33        perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.56 ±  4%      +1.6        2.16        perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +1.6        1.61        perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      0.00            +2.4        2.41        perf-profile.calltrace.cycles-pp.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      0.00            +2.4        2.41        perf-profile.calltrace.cycles-pp.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      1.06 ± 14%      +3.7        4.75        perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      1.04 ± 14%      +3.7        4.74        perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      2.02 ± 11%      +6.7        8.68        perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
      1.87 ± 11%      +6.7        8.59        perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
      0.00            +9.5        9.51        perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.tick_nohz_irq_exit.sysvec_call_function.asm_sysvec_call_function
      0.00            +9.6        9.60        perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_irq_exit.sysvec_call_function.asm_sysvec_call_function.cpuidle_enter_state
      0.00            +9.6        9.62        perf-profile.calltrace.cycles-pp.tick_nohz_irq_exit.sysvec_call_function.asm_sysvec_call_function.cpuidle_enter_state.cpuidle_enter
      0.00           +10.6       10.61        perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.cpuidle_enter_state
      0.00           +10.8       10.85        perf-profile.calltrace.cycles-pp.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.cpuidle_enter_state.cpuidle_enter
      0.00           +10.9       10.93        perf-profile.calltrace.cycles-pp.sched_clock_tick.sched_clock_idle_wakeup_event.cpuidle_enter_state.cpuidle_enter.do_idle
      0.00           +11.0       11.04        perf-profile.calltrace.cycles-pp.sched_clock_tick.sched_clock_idle_wakeup_event.tick_irq_enter.irq_enter_rcu.sysvec_call_function
      0.00           +11.1       11.09        perf-profile.calltrace.cycles-pp.sched_clock_idle_wakeup_event.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
     72.76           +11.1       83.89        perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
     72.76           +11.1       83.89        perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     72.73           +11.2       83.88        perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
      0.00           +11.2       11.22        perf-profile.calltrace.cycles-pp.sched_clock_idle_wakeup_event.tick_irq_enter.irq_enter_rcu.sysvec_call_function.asm_sysvec_call_function
     73.49           +11.3       84.83        perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
      0.62 ±  4%     +11.9       12.51        perf-profile.calltrace.cycles-pp.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_call_function.asm_sysvec_call_function
      0.00           +12.0       12.05        perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.tick_irq_enter
      0.00           +12.2       12.16        perf-profile.calltrace.cycles-pp.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.tick_irq_enter.irq_enter_rcu
      0.00           +12.4       12.36        perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_call_function
     68.20           +13.7       81.95        perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     66.98           +14.1       81.05        perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
      0.82 ±  3%     +23.0       23.86        perf-profile.calltrace.cycles-pp.irq_enter_rcu.sysvec_call_function.asm_sysvec_call_function.cpuidle_enter_state.cpuidle_enter
      0.79 ±  3%     +23.1       23.84        perf-profile.calltrace.cycles-pp.tick_irq_enter.irq_enter_rcu.sysvec_call_function.asm_sysvec_call_function.cpuidle_enter_state
     12.46           +25.8       38.27        perf-profile.calltrace.cycles-pp.asm_sysvec_call_function.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
      8.02           +28.5       36.53        perf-profile.calltrace.cycles-pp.sysvec_call_function.asm_sysvec_call_function.cpuidle_enter_state.cpuidle_enter.do_idle
     52.41           -29.5       22.89        perf-profile.children.cycles-pp.intel_idle
     19.27 ±  3%      -8.4       10.89        perf-profile.children.cycles-pp.do_syscall_64
     19.33 ±  3%      -8.4       10.96        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      7.14 ±  5%      -3.5        3.65        perf-profile.children.cycles-pp.do_group_exit
      7.14 ±  5%      -3.5        3.65        perf-profile.children.cycles-pp.do_exit
      7.11 ±  5%      -3.5        3.64        perf-profile.children.cycles-pp.__x64_sys_exit_group
      6.45 ±  2%      -3.5        3.00        perf-profile.children.cycles-pp.__sysvec_call_function
      6.24 ±  2%      -3.3        2.90        perf-profile.children.cycles-pp.flush_smp_call_function_queue
      6.59 ±  5%      -3.2        3.35        perf-profile.children.cycles-pp.mmput
      6.57 ±  5%      -3.2        3.34        perf-profile.children.cycles-pp.exit_mmap
      6.29            -2.9        3.42 ±  2%  perf-profile.children.cycles-pp.__do_sys_clone
      6.29            -2.9        3.42 ±  2%  perf-profile.children.cycles-pp.kernel_clone
      5.94            -2.7        3.21 ±  2%  perf-profile.children.cycles-pp.copy_process
      5.28            -2.5        2.74 ±  2%  perf-profile.children.cycles-pp.dup_mm
      5.17            -2.5        2.68 ±  2%  perf-profile.children.cycles-pp.dup_mmap
      3.50 ±  2%      -2.3        1.21 ±  2%  perf-profile.children.cycles-pp.menu_select
      4.19 ±  4%      -1.9        2.25 ±  5%  perf-profile.children.cycles-pp.asm_exc_page_fault
      4.06 ±  4%      -1.9        2.13        perf-profile.children.cycles-pp.exc_page_fault
      4.04 ±  4%      -1.9        2.12        perf-profile.children.cycles-pp.do_user_addr_fault
      3.47 ±  4%      -1.7        1.82        perf-profile.children.cycles-pp.handle_mm_fault
      3.34 ±  4%      -1.6        1.74        perf-profile.children.cycles-pp.__handle_mm_fault
      2.40 ±  4%      -1.2        1.20 ±  4%  perf-profile.children.cycles-pp.free_pgtables
      2.42 ±  6%      -1.2        1.27 ±  3%  perf-profile.children.cycles-pp.unmap_vmas
      3.86 ±  4%      -1.1        2.73 ±  3%  perf-profile.children.cycles-pp.seccomp_set_mode_filter
      3.86 ±  4%      -1.1        2.73 ±  3%  perf-profile.children.cycles-pp.do_seccomp
      2.28 ±  5%      -1.1        1.16 ±  3%  perf-profile.children.cycles-pp.do_fault
      2.28 ±  6%      -1.1        1.19 ±  3%  perf-profile.children.cycles-pp.unmap_page_range
      2.21 ±  5%      -1.1        1.13 ±  3%  perf-profile.children.cycles-pp.filemap_map_pages
      2.14 ±  7%      -1.0        1.12 ±  3%  perf-profile.children.cycles-pp.zap_pte_range
      2.08 ±  2%      -1.0        1.07        perf-profile.children.cycles-pp.__flush_tlb_all
      1.99 ±  2%      -1.0        0.99 ±  4%  perf-profile.children.cycles-pp.anon_vma_fork
      2.05 ±  2%      -1.0        1.06        perf-profile.children.cycles-pp.native_flush_tlb_global
      3.38 ±  4%      -0.9        2.50 ±  3%  perf-profile.children.cycles-pp.bpf_prog_create_from_user
      1.65 ±  5%      -0.8        0.83 ±  2%  perf-profile.children.cycles-pp.unlink_anon_vmas
      1.49            -0.8        0.69 ±  3%  perf-profile.children.cycles-pp.native_irq_return_iret
      1.47 ±  5%      -0.7        0.73 ±  2%  perf-profile.children.cycles-pp.kmem_cache_free
      1.50 ±  2%      -0.7        0.76 ±  3%  perf-profile.children.cycles-pp.anon_vma_clone
      1.52 ±  5%      -0.7        0.84 ±  6%  perf-profile.children.cycles-pp.copy_page_range
      2.07            -0.7        1.41 ±  2%  perf-profile.children.cycles-pp.ret_from_fork
      1.26            -0.7        0.60 ±  3%  perf-profile.children.cycles-pp.irq_exit_rcu
      1.36 ±  8%      -0.7        0.70 ±  4%  perf-profile.children.cycles-pp.tlb_finish_mmu
      1.35 ±  8%      -0.7        0.69 ±  4%  perf-profile.children.cycles-pp.tlb_flush_mmu
      1.23 ±  2%      -0.7        0.57        perf-profile.children.cycles-pp.llist_reverse_order
      2.79 ±  4%      -0.6        2.15 ±  3%  perf-profile.children.cycles-pp.bpf_prepare_filter
      1.30 ±  6%      -0.6        0.68 ±  5%  perf-profile.children.cycles-pp.next_uptodate_page
      2.74 ±  4%      -0.6        2.13 ±  3%  perf-profile.children.cycles-pp.bpf_prog_select_runtime
      2.73 ±  4%      -0.6        2.12 ±  3%  perf-profile.children.cycles-pp.bpf_int_jit_compile
      2.37 ±  2%      -0.6        1.80 ±  2%  perf-profile.children.cycles-pp.on_each_cpu
      1.88            -0.6        1.31 ±  2%  perf-profile.children.cycles-pp.kthread
      1.86            -0.6        1.30        perf-profile.children.cycles-pp.worker_thread
      2.61 ±  2%      -0.6        2.06 ±  2%  perf-profile.children.cycles-pp._vm_unmap_aliases
      2.34 ±  2%      -0.5        1.79 ±  2%  perf-profile.children.cycles-pp.smp_call_function_many_cond
      1.22 ±  7%      -0.5        0.69 ±  9%  perf-profile.children.cycles-pp.copy_pte_range
      0.93 ±  2%      -0.5        0.39        perf-profile.children.cycles-pp.__cpa_flush_tlb
      1.11 ±  2%      -0.5        0.58        perf-profile.children.cycles-pp.kmem_cache_alloc
      1.08 ±  8%      -0.5        0.55 ±  4%  perf-profile.children.cycles-pp.release_pages
      2.53 ±  3%      -0.5        2.00 ±  3%  perf-profile.children.cycles-pp.change_page_attr_set_clr
      1.76            -0.5        1.25 ±  2%  perf-profile.children.cycles-pp.process_one_work
      0.98 ± 11%      -0.5        0.47 ±  8%  perf-profile.children.cycles-pp.native_flush_tlb_one_user
      1.03 ±  3%      -0.5        0.53 ±  2%  perf-profile.children.cycles-pp.llist_add_batch
      0.95 ± 12%      -0.5        0.46 ±  8%  perf-profile.children.cycles-pp.flush_tlb_one_kernel
      1.65            -0.5        1.18        perf-profile.children.cycles-pp.__vunmap
      0.51 ±  3%      -0.5        0.06        perf-profile.children.cycles-pp.irqtime_account_irq
      1.07 ±  3%      -0.4        0.63 ±  2%  perf-profile.children.cycles-pp.__alloc_pages_nodemask
      0.73 ± 18%      -0.4        0.30 ± 14%  perf-profile.children.cycles-pp.do_kernel_range_flush
      0.71 ±  4%      -0.4        0.28 ±  3%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      1.52            -0.4        1.12 ±  2%  perf-profile.children.cycles-pp.bpf_jit_free
      1.51            -0.4        1.12 ±  2%  perf-profile.children.cycles-pp.bpf_jit_binary_free
      0.74 ±  6%      -0.4        0.39 ±  2%  perf-profile.children.cycles-pp.page_remove_rmap
      0.73 ±  2%      -0.4        0.37 ±  4%  perf-profile.children.cycles-pp.vm_area_dup
      0.62 ±  4%      -0.3        0.29 ±  3%  perf-profile.children.cycles-pp.remove_vma
      0.68 ±  3%      -0.3        0.35 ±  5%  perf-profile.children.cycles-pp.anon_vma_interval_tree_insert
      1.12 ±  2%      -0.3        0.81 ±  2%  perf-profile.children.cycles-pp.__purge_vmap_area_lazy
      0.76 ±  4%      -0.3        0.45 ±  3%  perf-profile.children.cycles-pp.get_page_from_freelist
      0.47 ±  8%      -0.3        0.17 ±  6%  perf-profile.children.cycles-pp.tick_nohz_next_event
      1.30 ±  3%      -0.3        1.01 ±  3%  perf-profile.children.cycles-pp.cpa_flush
      0.57 ±  6%      -0.3        0.28 ± 10%  perf-profile.children.cycles-pp.unlink_file_vma
      0.59 ±  2%      -0.3        0.31 ±  2%  perf-profile.children.cycles-pp.__do_sys_wait4
      1.06 ±  3%      -0.3        0.78 ±  3%  perf-profile.children.cycles-pp.set_memory_x
      0.59 ±  2%      -0.3        0.31 ±  2%  perf-profile.children.cycles-pp.kernel_wait4
      0.57 ±  2%      -0.3        0.29 ±  4%  perf-profile.children.cycles-pp.do_wait
      0.60 ±  3%      -0.3        0.33 ±  3%  perf-profile.children.cycles-pp.__schedule
      0.51 ±  3%      -0.3        0.24 ±  4%  perf-profile.children.cycles-pp.down_write
      0.50 ±  5%      -0.3        0.24 ±  3%  perf-profile.children.cycles-pp._raw_spin_lock
      0.65 ±  2%      -0.3        0.39 ±  3%  perf-profile.children.cycles-pp.__softirqentry_text_start
      0.58 ± 12%      -0.2        0.34 ±  3%  perf-profile.children.cycles-pp.bpf_prog_alloc
      1.47 ±  4%      -0.2        1.23 ±  3%  perf-profile.children.cycles-pp.set_memory_ro
      0.47 ±  6%      -0.2        0.24 ±  3%  perf-profile.children.cycles-pp.__x64_sys_munmap
      0.47 ±  7%      -0.2        0.24 ±  3%  perf-profile.children.cycles-pp.__vm_munmap
      0.47 ±  6%      -0.2        0.24 ±  3%  perf-profile.children.cycles-pp.__do_munmap
      0.43 ±  6%      -0.2        0.20 ±  6%  perf-profile.children.cycles-pp.do_set_pte
      0.52 ±  2%      -0.2        0.30 ±  5%  perf-profile.children.cycles-pp.pte_alloc_one
      0.42 ±  6%      -0.2        0.20 ±  7%  perf-profile.children.cycles-pp.seccomp_attach_filter
      0.42 ±  6%      -0.2        0.20 ±  7%  perf-profile.children.cycles-pp.seccomp_cache_prepare_bitmap
      0.35 ±  8%      -0.2        0.14 ± 10%  perf-profile.children.cycles-pp.rwsem_down_write_slowpath
      0.49 ±  4%      -0.2        0.28 ±  5%  perf-profile.children.cycles-pp.__pte_alloc
      0.53 ±  4%      -0.2        0.34 ±  4%  perf-profile.children.cycles-pp.prep_new_page
      0.52 ±  5%      -0.2        0.32 ±  4%  perf-profile.children.cycles-pp.clear_page_erms
      0.39 ±  3%      -0.2        0.19 ±  8%  perf-profile.children.cycles-pp.up_write
      0.44 ±  7%      -0.2        0.24 ±  7%  perf-profile.children.cycles-pp.do_anonymous_page
      0.38 ±  7%      -0.2        0.19 ±  5%  perf-profile.children.cycles-pp.__slab_free
      0.37 ± 12%      -0.2        0.18 ±  5%  perf-profile.children.cycles-pp.do_sys_open
      0.37 ± 12%      -0.2        0.18 ±  5%  perf-profile.children.cycles-pp.do_sys_openat2
      0.36 ±  7%      -0.2        0.18 ±  5%  perf-profile.children.cycles-pp.refill_obj_stock
      0.38 ±  3%      -0.2        0.19 ±  6%  perf-profile.children.cycles-pp.anon_vma_interval_tree_remove
      0.40 ±  8%      -0.2        0.22 ±  6%  perf-profile.children.cycles-pp.rcu_core
      0.36 ±  4%      -0.2        0.18 ±  5%  perf-profile.children.cycles-pp.__memcg_kmem_charge
      0.33 ±  4%      -0.2        0.15 ±  5%  perf-profile.children.cycles-pp.page_counter_try_charge
      0.66 ±  3%      -0.2        0.48        perf-profile.children.cycles-pp.sched_clock_cpu
      0.27 ±  7%      -0.2        0.09 ±  4%  perf-profile.children.cycles-pp.default_send_IPI_mask_sequence_phys
      0.34 ±  3%      -0.2        0.16 ±  5%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      0.34 ±  5%      -0.2        0.17 ±  3%  perf-profile.children.cycles-pp.cpumask_next
      0.40 ±  7%      -0.2        0.23 ±  3%  perf-profile.children.cycles-pp.__vmalloc_node_range
      0.31 ±  6%      -0.2        0.14 ±  6%  perf-profile.children.cycles-pp.page_add_file_rmap
      0.41 ± 12%      -0.2        0.24 ±  3%  perf-profile.children.cycles-pp.pcpu_alloc
      0.38 ±  8%      -0.2        0.21 ±  3%  perf-profile.children.cycles-pp.bpf_prog_alloc_no_stats
      0.31 ±  6%      -0.2        0.15 ± 10%  perf-profile.children.cycles-pp.vma_interval_tree_remove
      0.36 ± 11%      -0.2        0.20 ±  7%  perf-profile.children.cycles-pp.rcu_do_batch
      0.27 ±  4%      -0.2        0.12 ±  6%  perf-profile.children.cycles-pp.rcu_idle_exit
      0.23 ±  7%      -0.2        0.08 ±  6%  perf-profile.children.cycles-pp.__default_send_IPI_dest_field
      0.32 ±  4%      -0.2        0.17 ±  6%  perf-profile.children.cycles-pp.schedule
      0.34 ±  5%      -0.1        0.19 ±  4%  perf-profile.children.cycles-pp.pick_next_task_fair
      0.33 ±  4%      -0.1        0.18 ±  4%  perf-profile.children.cycles-pp.wp_page_copy
      0.32 ±  3%      -0.1        0.17 ±  4%  perf-profile.children.cycles-pp.wait_task_zombie
      0.30 ±  4%      -0.1        0.16 ±  4%  perf-profile.children.cycles-pp.release_task
      0.28 ±  7%      -0.1        0.14 ±  4%  perf-profile.children.cycles-pp.find_vma
      0.30 ±  3%      -0.1        0.16 ±  4%  perf-profile.children.cycles-pp.do_filp_open
      0.30 ±  3%      -0.1        0.16 ±  4%  perf-profile.children.cycles-pp.path_openat
      0.19 ±  4%      -0.1        0.05 ±  7%  perf-profile.children.cycles-pp.__cpa_addr
      0.33 ±  4%      -0.1        0.18 ±  4%  perf-profile.children.cycles-pp.load_balance
      0.35 ±  3%      -0.1        0.21 ±  3%  perf-profile.children.cycles-pp.wake_up_new_task
      0.28 ± 11%      -0.1        0.14 ±  6%  perf-profile.children.cycles-pp.unmap_region
      0.33 ±  5%      -0.1        0.19 ±  2%  perf-profile.children.cycles-pp.dup_task_struct
      0.28 ±  6%      -0.1        0.15 ±  4%  perf-profile.children.cycles-pp.__memcg_kmem_charge_page
      0.27 ±  6%      -0.1        0.14 ±  4%  perf-profile.children.cycles-pp.free_pages_and_swap_cache
      0.27 ±  5%      -0.1        0.14 ±  4%  perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
      0.28 ±  3%      -0.1        0.16 ±  7%  perf-profile.children.cycles-pp.newidle_balance
      0.21 ±  7%      -0.1        0.09 ± 12%  perf-profile.children.cycles-pp.rwsem_spin_on_owner
      0.23 ±  5%      -0.1        0.11 ±  5%  perf-profile.children.cycles-pp.sync_regs
      0.29 ±  3%      -0.1        0.16 ±  4%  perf-profile.children.cycles-pp.find_busiest_group
      0.28 ±  3%      -0.1        0.16 ±  4%  perf-profile.children.cycles-pp.update_sd_lb_stats
      0.30 ±  4%      -0.1        0.18 ±  4%  perf-profile.children.cycles-pp.select_task_rq_fair
      0.23 ± 10%      -0.1        0.11 ±  4%  perf-profile.children.cycles-pp.__put_anon_vma
      0.58 ±  2%      -0.1        0.46        perf-profile.children.cycles-pp.sched_clock
      0.23 ±  4%      -0.1        0.12 ±  4%  perf-profile.children.cycles-pp.__list_del_entry_valid
      0.23 ±  4%      -0.1        0.12 ±  7%  perf-profile.children.cycles-pp.get_obj_cgroup_from_current
      0.19 ±  5%      -0.1        0.08 ±  8%  perf-profile.children.cycles-pp.rcu_eqs_exit
      0.25 ± 10%      -0.1        0.14 ±  4%  perf-profile.children.cycles-pp.__get_vm_area_node
      0.23 ±  4%      -0.1        0.12 ±  6%  perf-profile.children.cycles-pp.idle_cpu
      0.21 ±  7%      -0.1        0.10 ±  5%  perf-profile.children.cycles-pp.drain_obj_stock
      0.54 ±  2%      -0.1        0.44        perf-profile.children.cycles-pp.native_sched_clock
      0.21 ±  4%      -0.1        0.11 ±  6%  perf-profile.children.cycles-pp._find_next_bit
      0.26 ±  5%      -0.1        0.15 ±  3%  perf-profile.children.cycles-pp.find_idlest_group
      0.23 ± 12%      -0.1        0.12 ±  6%  perf-profile.children.cycles-pp.alloc_vmap_area
      0.24 ± 15%      -0.1        0.14 ± 12%  perf-profile.children.cycles-pp.exit_to_user_mode_prepare
      0.21 ±  3%      -0.1        0.10 ±  4%  perf-profile.children.cycles-pp.dup_userfaultfd
      0.20 ±  8%      -0.1        0.10 ±  5%  perf-profile.children.cycles-pp.unlock_page
      0.18 ± 15%      -0.1        0.08 ±  6%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      0.19 ± 11%      -0.1        0.10 ± 13%  perf-profile.children.cycles-pp.cpuidle_governor_latency_req
      0.21 ±  6%      -0.1        0.12 ±  8%  perf-profile.children.cycles-pp.alloc_pages_vma
      0.19 ±  6%      -0.1        0.09 ±  5%  perf-profile.children.cycles-pp.___might_sleep
      0.38 ±  5%      -0.1        0.29 ±  5%  perf-profile.children.cycles-pp.flush_tlb_kernel_range
      0.20 ±  4%      -0.1        0.10 ±  7%  perf-profile.children.cycles-pp.obj_cgroup_charge
      0.20 ±  4%      -0.1        0.11 ±  6%  perf-profile.children.cycles-pp.__x64_sys_mprotect
      0.20 ±  4%      -0.1        0.11 ±  6%  perf-profile.children.cycles-pp.do_mprotect_pkey
      1.13 ±  3%      -0.1        1.04 ±  4%  perf-profile.children.cycles-pp.__mutex_lock
      0.19 ±  5%      -0.1        0.10 ±  4%  perf-profile.children.cycles-pp.rmqueue
      0.17 ±  8%      -0.1        0.08 ±  5%  perf-profile.children.cycles-pp.page_counter_uncharge
      0.13 ±  8%      -0.1        0.04 ± 71%  perf-profile.children.cycles-pp.tick_check_broadcast_expired
      0.15 ±  4%      -0.1        0.06 ±  6%  perf-profile.children.cycles-pp.call_cpuidle
      0.17 ±  9%      -0.1        0.08 ±  5%  perf-profile.children.cycles-pp.page_counter_cancel
      0.13 ± 10%      -0.1        0.04 ± 73%  perf-profile.children.cycles-pp.__rb_erase_color
      0.65 ±  4%      -0.1        0.57 ±  4%  perf-profile.children.cycles-pp.mutex_spin_on_owner
      0.18 ±  2%      -0.1        0.09 ±  6%  perf-profile.children.cycles-pp.link_path_walk
      0.18 ±  2%      -0.1        0.09 ±  5%  perf-profile.children.cycles-pp.schedule_tail
      0.16 ±  6%      -0.1        0.08 ±  8%  perf-profile.children.cycles-pp.get_next_timer_interrupt
      0.18 ±  4%      -0.1        0.10 ±  6%  perf-profile.children.cycles-pp.schedule_idle
      0.24 ± 11%      -0.1        0.16 ±  7%  perf-profile.children.cycles-pp.memset_erms
      0.17 ±  2%      -0.1        0.09 ±  5%  perf-profile.children.cycles-pp.mprotect_fixup
      0.17 ±  8%      -0.1        0.09 ±  7%  perf-profile.children.cycles-pp.finish_task_switch
      0.16 ±  3%      -0.1        0.08 ±  4%  perf-profile.children.cycles-pp.walk_component
      0.15 ±  4%      -0.1        0.07 ±  9%  perf-profile.children.cycles-pp.native_apic_mem_write
      0.17 ±  7%      -0.1        0.10 ±  6%  perf-profile.children.cycles-pp.lock_page_memcg
      0.17 ± 18%      -0.1        0.10 ± 15%  perf-profile.children.cycles-pp.arch_do_signal_or_restart
      0.16 ± 10%      -0.1        0.09 ± 12%  perf-profile.children.cycles-pp.__get_free_pages
      0.15 ±  9%      -0.1        0.07 ±  6%  perf-profile.children.cycles-pp.__pagevec_lru_add
      0.14 ± 11%      -0.1        0.07 ±  5%  perf-profile.children.cycles-pp.hrtimer_next_event_without
      0.17 ± 11%      -0.1        0.09 ± 10%  perf-profile.children.cycles-pp.mem_cgroup_charge
      0.12 ±  8%      -0.1        0.05 ±  7%  perf-profile.children.cycles-pp.rcu_eqs_enter
      0.14 ±  8%      -0.1        0.07 ±  5%  perf-profile.children.cycles-pp.free_unref_page_list
      0.13 ±  8%      -0.1        0.06 ±  6%  perf-profile.children.cycles-pp.__memcg_kmem_uncharge
      0.14 ±  7%      -0.1        0.07        perf-profile.children.cycles-pp.__cond_resched
      0.11 ±  4%      -0.1        0.05        perf-profile.children.cycles-pp.update_rq_clock
      0.15 ±  7%      -0.1        0.09 ±  5%  perf-profile.children.cycles-pp.__slab_alloc
      0.13 ±  8%      -0.1        0.07 ± 11%  perf-profile.children.cycles-pp.purge_fragmented_blocks_allcpus
      0.14 ±  6%      -0.1        0.08 ±  7%  perf-profile.children.cycles-pp.___slab_alloc
      0.09 ±  5%      -0.1        0.03 ± 99%  perf-profile.children.cycles-pp.__might_sleep
      0.09 ± 15%      -0.1        0.03 ±100%  perf-profile.children.cycles-pp.__pud_alloc
      0.12 ±  8%      -0.1        0.06 ±  7%  perf-profile.children.cycles-pp.__rb_insert_augmented
      0.11 ±  9%      -0.1        0.06 ±  9%  perf-profile.children.cycles-pp.lock_page_lruvec_irqsave
      0.13 ± 14%      -0.1        0.07 ±  6%  perf-profile.children.cycles-pp.__vmalloc_node
      0.11 ±  9%      -0.1        0.05 ±  7%  perf-profile.children.cycles-pp.free_pcppages_bulk
      0.08 ±  5%      -0.1        0.03 ± 99%  perf-profile.children.cycles-pp.perf_iterate_sb
      0.13 ±  8%      -0.1        0.07 ±  9%  perf-profile.children.cycles-pp.proc_invalidate_siblings_dcache
      0.11 ±  6%      -0.1        0.06 ±  6%  perf-profile.children.cycles-pp.unmap_single_vma
      0.13 ±  5%      -0.1        0.07 ±  9%  perf-profile.children.cycles-pp.bpf_jit_binary_alloc
      0.10 ±  7%      -0.1        0.04 ± 44%  perf-profile.children.cycles-pp.fput_many
      0.10 ±  4%      -0.1        0.04 ± 44%  perf-profile.children.cycles-pp.flush_tlb_mm_range
      0.12 ±  5%      -0.1        0.06 ±  7%  perf-profile.children.cycles-pp.rebalance_domains
      0.11 ±  3%      -0.1        0.06 ±  8%  perf-profile.children.cycles-pp.__split_vma
      0.09 ±  6%      -0.1        0.04 ± 71%  perf-profile.children.cycles-pp.mm_init
      0.11 ±  9%      -0.1        0.06 ±  8%  perf-profile.children.cycles-pp.kfree
      0.09 ± 10%      -0.1        0.04 ± 71%  perf-profile.children.cycles-pp.do_task_dead
      0.10 ±  7%      -0.0        0.06 ±  9%  perf-profile.children.cycles-pp.vma_interval_tree_insert_after
      0.09 ±  7%      -0.0        0.04 ± 44%  perf-profile.children.cycles-pp.lru_cache_add
      0.11 ±  6%      -0.0        0.06 ±  7%  perf-profile.children.cycles-pp.__pmd_alloc
      0.11 ±  6%      -0.0        0.06 ±  6%  perf-profile.children.cycles-pp.__mod_memcg_state
      0.11 ±  6%      -0.0        0.06 ±  9%  perf-profile.children.cycles-pp.__put_user_nocheck_4
      0.12 ±  9%      -0.0        0.07 ±  8%  perf-profile.children.cycles-pp.__setup_rt_frame
      0.11 ±  6%      -0.0        0.07 ± 11%  perf-profile.children.cycles-pp.copy_fpstate_to_sigframe
      0.11 ±  8%      -0.0        0.06 ±  7%  perf-profile.children.cycles-pp.acct_collect
      0.11 ±  6%      -0.0        0.06 ±  7%  perf-profile.children.cycles-pp.nr_iowait_cpu
      0.11 ±  4%      -0.0        0.06        perf-profile.children.cycles-pp.__perf_sw_event
      0.10 ±  7%      -0.0        0.06 ±  6%  perf-profile.children.cycles-pp.try_to_wake_up
      0.08 ± 19%      -0.0        0.03 ± 70%  perf-profile.children.cycles-pp.put_cred_rcu
      0.09 ±  4%      -0.0        0.04 ± 44%  perf-profile.children.cycles-pp.rcu_read_unlock_strict
      0.09 ±  5%      -0.0        0.04 ± 44%  perf-profile.children.cycles-pp.sched_move_task
      0.07 ±  8%      -0.0        0.03 ± 99%  perf-profile.children.cycles-pp.run_rebalance_domains
      0.09 ±  5%      -0.0        0.04 ± 44%  perf-profile.children.cycles-pp.d_invalidate
      0.10 ±  6%      -0.0        0.05 ±  9%  perf-profile.children.cycles-pp.module_alloc
      0.08 ±  6%      -0.0        0.03 ± 70%  perf-profile.children.cycles-pp.do_notify_parent
      0.09 ±  5%      -0.0        0.04 ± 45%  perf-profile.children.cycles-pp.update_load_avg
      0.07 ± 10%      -0.0        0.03 ± 70%  perf-profile.children.cycles-pp.__hrtimer_next_event_base
      0.09 ±  7%      -0.0        0.06 ±  9%  perf-profile.children.cycles-pp.allocate_slab
      0.11 ±  4%      -0.0        0.07 ±  8%  perf-profile.children.cycles-pp.copy_page
      0.06 ±  6%      -0.0        0.03 ± 99%  perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
      0.11 ±  4%      -0.0        0.08 ± 10%  perf-profile.children.cycles-pp.update_ts_time_stats
      0.10 ±  6%      -0.0        0.07 ± 10%  perf-profile.children.cycles-pp.try_charge
      0.00            +0.1        0.05 ±  7%  perf-profile.children.cycles-pp.do_setitimer
      0.00            +0.1        0.06 ±  9%  perf-profile.children.cycles-pp.__x64_sys_setitimer
      0.00            +0.1        0.10 ±  3%  perf-profile.children.cycles-pp.tick_nohz_idle_enter
      0.00            +0.1        0.14 ±  5%  perf-profile.children.cycles-pp.tick_nohz_idle_exit
      0.73 ± 23%      +0.2        0.94        perf-profile.children.cycles-pp.start_kernel
      0.08 ±  7%      +0.5        0.60 ±  3%  perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
      0.46 ±  3%      +0.6        1.09 ±  2%  perf-profile.children.cycles-pp.tick_sched_handle
      0.45 ±  4%      +0.6        1.09        perf-profile.children.cycles-pp.update_process_times
      0.29 ±  3%      +0.7        1.02        perf-profile.children.cycles-pp.scheduler_tick
      0.00            +0.8        0.77        perf-profile.children.cycles-pp.sched_clock_local
      0.41 ± 34%      +1.2        1.57        perf-profile.children.cycles-pp.clockevents_program_event
      0.09 ± 10%      +1.3        1.41        perf-profile.children.cycles-pp.ktime_get_update_offsets_now
      0.55 ±  2%      +1.4        1.92        perf-profile.children.cycles-pp.tick_sched_timer
      0.70 ±  3%      +1.9        2.58        perf-profile.children.cycles-pp.__hrtimer_run_queues
      1.30 ± 13%      +4.3        5.63        perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
      1.28 ± 13%      +4.3        5.62        perf-profile.children.cycles-pp.hrtimer_interrupt
      2.46 ± 10%      +7.4        9.91        perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
      2.27 ±  9%      +7.5        9.80        perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
      0.58 ±  7%     +10.4       10.96        perf-profile.children.cycles-pp.tick_nohz_irq_exit
     72.76           +11.1       83.89        perf-profile.children.cycles-pp.start_secondary
     73.49           +11.3       84.83        perf-profile.children.cycles-pp.secondary_startup_64_no_verify
     73.49           +11.3       84.83        perf-profile.children.cycles-pp.cpu_startup_entry
     73.49           +11.3       84.83        perf-profile.children.cycles-pp.do_idle
     68.89           +14.0       82.87        perf-profile.children.cycles-pp.cpuidle_enter
     68.86           +14.0       82.85        perf-profile.children.cycles-pp.cpuidle_enter_state
      0.00           +24.0       23.97        perf-profile.children.cycles-pp.sched_clock_idle_wakeup_event
      0.00           +24.5       24.49        perf-profile.children.cycles-pp.sched_clock_tick
      0.97 ±  2%     +25.8       26.75        perf-profile.children.cycles-pp.irq_enter_rcu
      0.95 ±  2%     +25.8       26.74        perf-profile.children.cycles-pp.tick_irq_enter
     11.42           +27.0       38.45        perf-profile.children.cycles-pp.asm_sysvec_call_function
      8.87 ±  2%     +28.5       37.41        perf-profile.children.cycles-pp.sysvec_call_function
      1.72 ±  8%     +50.6       52.31        perf-profile.children.cycles-pp.ktime_get
      0.00           +53.0       53.01        perf-profile.children.cycles-pp.read_hpet
     52.41           -29.5       22.89        perf-profile.self.cycles-pp.intel_idle
      3.81 ±  2%      -2.5        1.35        perf-profile.self.cycles-pp.cpuidle_enter_state
      2.53 ±  2%      -1.7        0.80 ±  2%  perf-profile.self.cycles-pp.menu_select
      2.05 ±  2%      -1.0        1.05        perf-profile.self.cycles-pp.native_flush_tlb_global
      1.49            -0.8        0.69 ±  2%  perf-profile.self.cycles-pp.native_irq_return_iret
      1.27 ±  2%      -0.7        0.56 ±  2%  perf-profile.self.cycles-pp.flush_smp_call_function_queue
      1.22 ±  2%      -0.7        0.57        perf-profile.self.cycles-pp.llist_reverse_order
      1.37 ± 10%      -0.6        0.72 ±  3%  perf-profile.self.cycles-pp.ktime_get
      1.26 ±  6%      -0.6        0.64 ±  4%  perf-profile.self.cycles-pp.next_uptodate_page
      1.09 ±  8%      -0.5        0.57 ±  4%  perf-profile.self.cycles-pp.zap_pte_range
      1.00 ±  3%      -0.5        0.50 ±  2%  perf-profile.self.cycles-pp.llist_add_batch
      0.97 ± 11%      -0.5        0.47 ±  8%  perf-profile.self.cycles-pp.native_flush_tlb_one_user
      0.71 ±  9%      -0.4        0.36 ±  5%  perf-profile.self.cycles-pp.release_pages
      0.43 ± 22%      -0.3        0.09 ±  5%  perf-profile.self.cycles-pp.poll_idle
      0.63 ±  6%      -0.3        0.30 ±  3%  perf-profile.self.cycles-pp.kmem_cache_free
      0.65 ±  3%      -0.3        0.33 ±  5%  perf-profile.self.cycles-pp.anon_vma_interval_tree_insert
      0.64 ±  6%      -0.3        0.35 ± 10%  perf-profile.self.cycles-pp.copy_pte_range
      0.56 ±  7%      -0.3        0.28 ±  2%  perf-profile.self.cycles-pp.page_remove_rmap
      0.36 ±  3%      -0.2        0.13 ±  5%  perf-profile.self.cycles-pp.__cpa_flush_tlb
      0.44 ±  3%      -0.2        0.21 ±  2%  perf-profile.self.cycles-pp.kmem_cache_alloc
      0.42 ±  6%      -0.2        0.20 ±  7%  perf-profile.self.cycles-pp.seccomp_cache_prepare_bitmap
      0.41 ±  4%      -0.2        0.20 ±  2%  perf-profile.self.cycles-pp.down_write
      0.43 ±  4%      -0.2        0.21 ±  4%  perf-profile.self.cycles-pp._raw_spin_lock
      0.37 ±  3%      -0.2        0.18 ±  7%  perf-profile.self.cycles-pp.up_write
      0.49 ±  6%      -0.2        0.30 ±  4%  perf-profile.self.cycles-pp.clear_page_erms
      0.36 ±  7%      -0.2        0.18 ±  5%  perf-profile.self.cycles-pp.__slab_free
      0.36 ±  2%      -0.2        0.18 ±  4%  perf-profile.self.cycles-pp.anon_vma_interval_tree_remove
      0.33 ±  2%      -0.2        0.16 ±  6%  perf-profile.self.cycles-pp.do_idle
      0.29 ±  5%      -0.2        0.13 ±  5%  perf-profile.self.cycles-pp.page_counter_try_charge
      0.30 ±  3%      -0.2        0.15 ±  6%  perf-profile.self.cycles-pp.anon_vma_clone
      0.23 ±  7%      -0.2        0.08 ±  6%  perf-profile.self.cycles-pp.__default_send_IPI_dest_field
      0.30 ±  6%      -0.2        0.15 ± 11%  perf-profile.self.cycles-pp.vma_interval_tree_remove
      0.19 ±  4%      -0.1        0.05        perf-profile.self.cycles-pp.__cpa_addr
      0.29 ±  4%      -0.1        0.15 ±  6%  perf-profile.self.cycles-pp.vm_area_dup
      0.26 ±  7%      -0.1        0.11 ± 10%  perf-profile.self.cycles-pp.page_add_file_rmap
      0.19 ±  8%      -0.1        0.06 ±  6%  perf-profile.self.cycles-pp.irqtime_account_irq
      0.22 ±  6%      -0.1        0.09 ±  7%  perf-profile.self.cycles-pp.sysvec_call_function
      0.26 ±  5%      -0.1        0.13 ±  3%  perf-profile.self.cycles-pp.free_pages_and_swap_cache
      0.20 ±  7%      -0.1        0.08 ± 14%  perf-profile.self.cycles-pp.rwsem_spin_on_owner
      0.23 ±  4%      -0.1        0.11 ±  5%  perf-profile.self.cycles-pp.sync_regs
      0.23 ±  3%      -0.1        0.11 ±  6%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.16 ±  8%      -0.1        0.05 ± 45%  perf-profile.self.cycles-pp.do_kernel_range_flush
      0.23 ±  8%      -0.1        0.12 ±  3%  perf-profile.self.cycles-pp.dup_mmap
      0.21 ±  6%      -0.1        0.10 ±  4%  perf-profile.self.cycles-pp.filemap_map_pages
      0.23 ±  5%      -0.1        0.12 ±  3%  perf-profile.self.cycles-pp.__list_del_entry_valid
      0.23 ±  4%      -0.1        0.12 ±  6%  perf-profile.self.cycles-pp.idle_cpu
      0.20 ±  4%      -0.1        0.10 ±  3%  perf-profile.self.cycles-pp.asm_sysvec_call_function
      0.52 ±  3%      -0.1        0.42        perf-profile.self.cycles-pp.native_sched_clock
      0.18 ± 13%      -0.1        0.07 ±  6%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
      0.19 ±  3%      -0.1        0.09 ±  5%  perf-profile.self.cycles-pp._find_next_bit
      0.19 ±  6%      -0.1        0.09 ±  5%  perf-profile.self.cycles-pp.find_vma
      0.20 ±  6%      -0.1        0.10 ±  4%  perf-profile.self.cycles-pp.get_obj_cgroup_from_current
      0.19 ±  3%      -0.1        0.10 ±  3%  perf-profile.self.cycles-pp.dup_userfaultfd
      0.21 ±  4%      -0.1        0.12 ±  5%  perf-profile.self.cycles-pp.update_sd_lb_stats
      0.20 ±  5%      -0.1        0.11 ±  4%  perf-profile.self.cycles-pp.__handle_mm_fault
      0.19 ±  9%      -0.1        0.09 ±  5%  perf-profile.self.cycles-pp.unlock_page
      0.18 ±  5%      -0.1        0.09 ±  5%  perf-profile.self.cycles-pp.___might_sleep
      0.12 ±  7%      -0.1        0.04 ± 71%  perf-profile.self.cycles-pp.tick_check_broadcast_expired
      0.62 ±  4%      -0.1        0.54 ±  4%  perf-profile.self.cycles-pp.mutex_spin_on_owner
      0.15 ±  3%      -0.1        0.06 ±  6%  perf-profile.self.cycles-pp.call_cpuidle
      0.22 ±  7%      -0.1        0.14 ±  4%  perf-profile.self.cycles-pp.find_idlest_group
      0.16 ± 10%      -0.1        0.08 ±  8%  perf-profile.self.cycles-pp.__mod_memcg_lruvec_state
      0.15 ±  9%      -0.1        0.07 ±  8%  perf-profile.self.cycles-pp.page_counter_cancel
      0.22 ± 11%      -0.1        0.15 ±  7%  perf-profile.self.cycles-pp.memset_erms
      0.15 ± 10%      -0.1        0.07 ± 10%  perf-profile.self.cycles-pp.unlink_anon_vmas
      0.17 ±  7%      -0.1        0.09 ±  5%  perf-profile.self.cycles-pp.lock_page_memcg
      0.14 ±  9%      -0.1        0.07 ±  7%  perf-profile.self.cycles-pp.refill_obj_stock
      0.10 ±  8%      -0.1        0.03 ± 99%  perf-profile.self.cycles-pp.__purge_vmap_area_lazy
      0.14 ±  6%      -0.1        0.07 ±  9%  perf-profile.self.cycles-pp.native_apic_mem_write
      0.13 ± 14%      -0.1        0.06        perf-profile.self.cycles-pp.cpumask_next
      0.09 ±  5%      -0.1        0.03 ± 99%  perf-profile.self.cycles-pp.unmap_page_range
      0.10 ±  8%      -0.1        0.03 ± 70%  perf-profile.self.cycles-pp.rcu_eqs_exit
      0.09 ±  7%      -0.1        0.03 ± 99%  perf-profile.self.cycles-pp.kfree
      0.09 ± 11%      -0.1        0.03 ± 70%  perf-profile.self.cycles-pp.irq_exit_rcu
      0.12 ±  8%      -0.1        0.06 ± 13%  perf-profile.self.cycles-pp.__rb_insert_augmented
      0.11 ±  6%      -0.1        0.05 ±  9%  perf-profile.self.cycles-pp.unmap_single_vma
      0.11 ±  5%      -0.1        0.06 ±  6%  perf-profile.self.cycles-pp.nr_iowait_cpu
      0.12 ±  6%      -0.1        0.07 ±  5%  perf-profile.self.cycles-pp.alloc_vmap_area
      0.11 ±  4%      -0.0        0.06 ±  6%  perf-profile.self.cycles-pp.acct_collect
      0.10 ±  7%      -0.0        0.06 ±  8%  perf-profile.self.cycles-pp.__mod_memcg_state
      0.10 ±  9%      -0.0        0.05        perf-profile.self.cycles-pp.vma_interval_tree_insert_after
      0.09 ±  8%      -0.0        0.04 ± 44%  perf-profile.self.cycles-pp.purge_fragmented_blocks_allcpus
      0.10 ±  4%      -0.0        0.06 ±  7%  perf-profile.self.cycles-pp.copy_page
      0.10 ±  4%      -0.0        0.06 ±  7%  perf-profile.self.cycles-pp._vm_unmap_aliases
      0.07 ± 12%      -0.0        0.05 ±  7%  perf-profile.self.cycles-pp.sched_clock_cpu
      0.93 ±  2%      +0.1        1.07 ±  3%  perf-profile.self.cycles-pp.smp_call_function_many_cond
      0.00            +0.5        0.45 ±  2%  perf-profile.self.cycles-pp.sched_clock_local
      0.00           +53.0       52.99        perf-profile.self.cycles-pp.read_hpet


                                                                                
                             stress-ng.time.user_time                           
                                                                                
  215 +---------------------------------------------------------------------+   
      |                  +                                        +       .+|   
  210 |+. +. +.+++. +.+ + ++.   +    +.  .++      +.+  +++.+    .+ +.++. + :|   
  205 |-++  +      +   +     + + +.++  ++   +  +. :  :+     +. +        +   |   
      |                       +              ++  +   +        +             |   
  200 |-+                                                                   |   
  195 |-+                                                                   |   
      |                                                                     |   
  190 |-+                                                                   |   
  185 |-+    O                                                              |   
      |   O                   O    O                                        |   
  180 |-+O     OO     OO  OO        O                                       |   
  175 |O+   O      O     O   O       O                                      |   
      |          O  O           OO                                          |   
  170 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                            stress-ng.time.system_time                          
                                                                                
  335 +---------------------------------------------------------------------+   
  330 |-+        O  O           OO                                          |   
      |O    O      O  OO OOO O       O                                      |   
  325 |-+OO    OO             O    OO                                       |   
  320 |-+    O                                                              |   
  315 |-+                                                                   |   
  310 |-+                                                                   |   
      |                                                                     |   
  305 |-+                                                                   |   
  300 |-+                                                                   |   
  295 |-+                     +              +  .+           .+             |   
  290 |-.+ .+   + .+   +.   .+ + +.++ .++.  + ++  :  +.+    +  +.      .+   |   
      |+  +  +.+ +  +.+  +++    +    +    ++      +.+   ++.+     +++.++  +.+|   
  285 |-+                                                                   |   
  280 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                    stress-ng.time.percent_of_cpu_this_job_got                  
                                                                                
  820 +---------------------------------------------------------------------+   
      |                                                                     |   
      |                       O                                             |   
  815 |-+                        O                                          |   
      |  O          O OO OOO O  O  OOO                                      |   
      |O  O O    O O                                                        |   
  810 |-+    O OO                                                           |   
      |                                                                     |   
  805 |-+                                                                   |   
      |                                                                     |   
      |      +. + .+                           +.           +.+ .+          |   
  800 |+.++.+  + +  +. +.+++.++  +.+++.++.++.++  ++.++.+++.+   +  ++.++.++.+|   
      |               +        : :                                          |   
      |                        ::                                           |   
  795 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                           stress-ng.time.minor_page_faults                     
                                                                                
    4e+07 +-----------------------------------------------------------------+   
          |+         + +.+ + ++.     ::  ::  +     + :          + .+ :+ ++. |   
  3.9e+07 |:+  +. + + +   +     +++.+ +.+ +.+ :.+++  +  ++  ++.+ +  +  +   +|   
  3.8e+07 |-+++  + +                          +       :+  :+               :|   
          |                                           +   +                 |   
  3.7e+07 |-+                                                               |   
          |                                                                 |   
  3.6e+07 |-+                                                               |   
          |                                                                 |   
  3.5e+07 |O+    OOO                                                        |   
  3.4e+07 |-+  O      O                                                     |   
          |  OO      O O  O   O     O   O                                   |   
  3.3e+07 |-+            O  OO  O O  OO                                     |   
          |                                                                 |   
  3.2e+07 +-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                     stress-ng.time.involuntary_context_switches                
                                                                                
  590000 +------------------------------------------------------------------+   
  580000 |-+ +                                                       +.   .+|   
         |+. :+.++. + .++ .++.++ .+++.+++.++.    + +.+ +.++.+  .+++.+  + +  |   
  570000 |-++      + +   +      +            +  + +   ::     ++         +   |   
  560000 |-+                                  ++      +                     |   
  550000 |-+                                                                |   
  540000 |-+                                                                |   
         |                                                                  |   
  530000 |-+                                                                |   
  520000 |-+                                                                |   
  510000 |-+ O  OO O                                                        |   
  500000 |O+O O     OO O                                                    |   
         |              OO  O O     O O                                     |   
  490000 |-+               O   O  OO   O                                    |   
  480000 +------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                stress-ng.opcode.ops                            
                                                                                
  580000 +------------------------------------------------------------------+   
  570000 |+. ++.++. + .++ .++.+  .+ +.+++.++.    + +.+ +.++.+    ++.+  + +  |   
         |  +      + +   +     ++  +         +  + +   ::     ++.+       +   |   
  560000 |-+                                  ++      +                     |   
  550000 |-+                                                                |   
  540000 |-+                                                                |   
  530000 |-+                                                                |   
         |                                                                  |   
  520000 |-+                                                                |   
  510000 |O+ O  OO O O                                                      |   
  500000 |-+O O     O  O                                                    |   
  490000 |-+            OO  O O     O O                                     |   
         |                 O   O  OO   O                                    |   
  480000 |-+                    O                                           |   
  470000 +------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                           stress-ng.opcode.ops_per_sec                         
                                                                                
  9800 +--------------------------------------------------------------------+   
  9600 |-+ +.                                                        +    .+|   
       |+. : +++. +. ++. +.++.+ +.++.+++.++.    + .++  ++.++.+ +.++.+ +. +  |   
  9400 |-++      +  +   +      +            +  + +   :+       +         +   |   
  9200 |-+                                   ++      +                      |   
       |                                                                    |   
  9000 |-+                                                                  |   
  8800 |-+                                                                  |   
  8600 |-+                                                                  |   
       |   O  OO O                                                          |   
  8400 |O+O  O    O OO                                                      |   
  8200 |-+            O O  OO      O O                                      |   
       |                 O    O O O   O                                     |   
  8000 |-+                     O                                            |   
  7800 +--------------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-csl-2sp4: 96 threads Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode:
  gcc-9/performance/x86_64-rhel-8.3/1/debian-10.4-x86_64-20200603.cgz/300s/lkp-csl-2sp4/execl/unixbench/0x4003006

commit: 
  d42c7a8d68 ("kcsan: Document "value changed" line")
  6c52b5f3cf ("clocksource: Reduce WATCHDOG_THRESHOLD")

d42c7a8d68038bc0 6c52b5f3cfefd6e429efc4413fd 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
     41269 ± 24%     -24.8%      31015 ±  4%  unixbench.time.involuntary_context_switches
     87.00            +2.3%      89.00        unixbench.time.percent_of_cpu_this_job_got
      9346            -6.5%       8736 ±  2%  proc-vmstat.pgactivate
      1.37 ± 16%      -1.4        0.00        mpstat.cpu.all.irq%
      0.10 ± 12%      -0.1        0.00 ± 59%  mpstat.cpu.all.soft%
      1600 ±  4%      -8.3%       1467 ±  6%  slabinfo.pool_workqueue.active_objs
      1609 ±  4%      -8.8%       1467 ±  6%  slabinfo.pool_workqueue.num_objs
     96.50            +2.6%      99.00        vmstat.cpu.id
      1918 ±  2%     -19.7%       1539        vmstat.system.cs
     27.42            +1.0%      27.69        boot-time.boot
     14.59            +2.0%      14.89        boot-time.dhcp
      2119            +7.3%       2273        boot-time.idle
  55342935 ±146%     -97.1%    1614545 ±  9%  cpuidle.C1.time
    369013 ±113%     -87.0%      47814 ± 10%  cpuidle.POLL.time
     80151 ± 24%     -94.7%       4258 ± 10%  cpuidle.POLL.usage
     72.00 ±  5%    +139.4%     172.33        turbostat.Avg_MHz
      3.76 ± 11%      +4.7        8.50        turbostat.Busy%
    113.08            +8.9%     123.12        turbostat.PkgWatt
     15.27 ±100%    +582.6%     104.25 ± 58%  sched_debug.cfs_rq:/.MIN_vruntime.avg
    144.93 ±100%    +338.3%     635.26 ± 54%  sched_debug.cfs_rq:/.MIN_vruntime.stddev
     15.27 ±100%    +582.6%     104.25 ± 58%  sched_debug.cfs_rq:/.max_vruntime.avg
    144.93 ±100%    +338.3%     635.26 ± 54%  sched_debug.cfs_rq:/.max_vruntime.stddev
    584.77 ±  3%     -98.3%       9.72 ± 18%  sched_debug.cpu.clock_task.stddev
      4639 ±  7%     -11.6%       4102        sched_debug.cpu.nr_switches.avg
     36958 ± 11%     -35.6%      23798 ± 17%  sched_debug.cpu.nr_switches.max
      4896 ±  6%     -34.3%       3216 ±  9%  sched_debug.cpu.nr_switches.stddev
      1.00          -100.0%       0.00        sched_debug.sched_clock_stable()
 9.417e+08 ±  2%      +7.3%   1.01e+09 ±  2%  perf-stat.i.branch-instructions
      5.82 ± 28%      +3.6        9.39 ±  2%  perf-stat.i.cache-miss-rate%
   2168912 ± 14%     +20.7%    2618168 ±  3%  perf-stat.i.cache-misses
      1868 ±  2%     -19.3%       1508        perf-stat.i.context-switches
      1.66 ± 15%    +114.4%       3.57 ±  2%  perf-stat.i.cpi
 6.219e+09 ±  6%    +155.1%  1.587e+10        perf-stat.i.cpu-cycles
      2937 ±  9%    +109.3%       6147 ±  2%  perf-stat.i.cycles-between-cache-misses
      0.12 ± 75%      -0.1        0.03 ±  4%  perf-stat.i.dTLB-load-miss-rate%
    993756 ± 62%     -63.4%     363963 ±  2%  perf-stat.i.dTLB-load-misses
      0.08 ± 30%      -0.0        0.05        perf-stat.i.dTLB-store-miss-rate%
    426111 ± 18%     -22.7%     329275 ±  3%  perf-stat.i.dTLB-store-misses
   2650130 ±  3%      +8.8%    2882167        perf-stat.i.iTLB-load-misses
 4.549e+09 ±  2%      +6.7%  4.853e+09 ±  2%  perf-stat.i.instructions
      0.72 ±  8%     -58.5%       0.30 ±  2%  perf-stat.i.ipc
      0.06 ±  6%    +155.1%       0.17        perf-stat.i.metric.GHz
      0.57 ± 59%     -63.9%       0.21 ±  6%  perf-stat.i.metric.K/sec
     83.19            +1.0       84.23        perf-stat.i.node-load-miss-rate%
    542248 ±  9%     +37.4%     745113        perf-stat.i.node-load-misses
    112232 ± 16%     +27.4%     143037 ±  3%  perf-stat.i.node-loads
     73.15 ± 29%     +23.3       96.45        perf-stat.i.node-store-miss-rate%
    142423 ± 19%     +81.4%     258350        perf-stat.i.node-store-misses
      5.57 ± 26%      +3.5        9.07 ±  3%  perf-stat.overall.cache-miss-rate%
      1.37 ±  9%    +138.8%       3.27 ±  2%  perf-stat.overall.cpi
      2902 ±  9%    +109.0%       6065 ±  2%  perf-stat.overall.cycles-between-cache-misses
      0.08 ± 64%      -0.1        0.03 ±  4%  perf-stat.overall.dTLB-load-miss-rate%
      0.07 ± 20%      -0.0        0.05        perf-stat.overall.dTLB-store-miss-rate%
      0.74 ±  8%     -58.4%       0.31 ±  2%  perf-stat.overall.ipc
     72.83 ± 28%     +23.1       95.94        perf-stat.overall.node-store-miss-rate%
   1242941           +11.5%    1386048        perf-stat.overall.path-length
 9.388e+08 ±  2%      +7.3%  1.007e+09 ±  2%  perf-stat.ps.branch-instructions
   2162600 ± 14%     +20.7%    2610503 ±  3%  perf-stat.ps.cache-misses
      1863 ±  2%     -19.3%       1504        perf-stat.ps.context-switches
   6.2e+09 ±  6%    +155.2%  1.582e+10        perf-stat.ps.cpu-cycles
    990761 ± 62%     -63.4%     362845 ±  2%  perf-stat.ps.dTLB-load-misses
    424764 ± 18%     -22.7%     328252 ±  3%  perf-stat.ps.dTLB-store-misses
   2641809 ±  3%      +8.8%    2873343        perf-stat.ps.iTLB-load-misses
 4.536e+09 ±  2%      +6.7%  4.839e+09 ±  2%  perf-stat.ps.instructions
    540562 ±  9%     +37.4%     742841        perf-stat.ps.node-load-misses
    111898 ± 16%     +27.5%     142630 ±  3%  perf-stat.ps.node-loads
    141992 ± 19%     +81.4%     257582        perf-stat.ps.node-store-misses
   1.5e+12 ±  2%      +6.6%    1.6e+12 ±  2%  perf-stat.total.instructions
      0.01 ± 23%   +5667.9%       0.51 ±109%  perf-sched.sch_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      0.01 ± 33%  +11121.1%       0.71 ± 81%  perf-sched.sch_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read
      0.00 ± 40%    +114.3%       0.01        perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.01 ± 47%    +106.7%       0.02 ±  3%  perf-sched.sch_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
      0.00 ± 49%   +5141.7%       0.10 ±215%  perf-sched.sch_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.01 ± 24%     +95.8%       0.02 ± 15%  perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion_interruptible_timeout.usb_stor_msg_common.usb_stor_bulk_transfer_buf
      0.00 ± 27%    +268.8%       0.01 ±  6%  perf-sched.sch_delay.avg.ms.wait_for_partner.fifo_open.do_dentry_open.do_open.isra
      0.01 ± 13%     +26.1%       0.01 ±  3%  perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
      0.01 ± 27%   +9866.7%       1.00 ±112%  perf-sched.sch_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      0.01 ± 42%  +19611.6%       1.41 ± 81%  perf-sched.sch_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read
      0.01 ± 43%    +110.0%       0.01 ± 17%  perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown]
      0.03 ± 72%    +517.5%       0.16 ± 72%  perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      0.02 ± 29%     +73.4%       0.03 ± 11%  perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion_interruptible_timeout.usb_stor_msg_common.usb_stor_bulk_transfer_buf
      2.54           +79.1%       4.55 ±  2%  perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
    129.77 ±  3%     +17.4%     152.41 ±  4%  perf-sched.total_wait_and_delay.average.ms
     11677 ±  4%     -12.9%      10170 ±  5%  perf-sched.total_wait_and_delay.count.ms
    129.77 ±  3%     +17.4%     152.39 ±  4%  perf-sched.total_wait_time.average.ms
    188.49 ± 10%     +27.2%     239.84 ±  9%  perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.11 ±  7%     +13.5%       0.12 ±  5%  perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
     71.26 ±  3%     +26.4%      90.08 ±  2%  perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
    305.63 ±  6%     +21.0%     369.75 ± 10%  perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
      0.00 ± 27%    +287.5%       0.01 ± 12%  perf-sched.wait_and_delay.avg.ms.wait_for_partner.fifo_open.do_dentry_open.do_open.isra
    209.83 ± 16%     -27.3%     152.50 ± 16%  perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
    105.33 ± 14%     -32.6%      71.00 ± 14%  perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown]
      2120 ±  3%     -21.5%       1665 ±  2%  perf-sched.wait_and_delay.count.pipe_read.new_sync_read.vfs_read.ksys_read
    144.17 ± 20%     -33.1%      96.50 ± 21%  perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
     89.00 ± 11%     -41.0%      52.50 ± 45%  perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.zap_pte_range.unmap_page_range.unmap_vmas
      3080 ±  8%     -16.7%       2566 ±  9%  perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
      4587 ± 50%     -59.8%       1845 ±103%  perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
    188.49 ± 10%     +27.2%     239.83 ±  9%  perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.11 ±  7%     +13.5%       0.12 ±  5%  perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
     71.26 ±  3%     +26.3%      89.98 ±  2%  perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.12 ± 31%     -75.3%       0.03 ±142%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc.security_prepare_creds.prepare_creds
      0.10 ±  7%     +20.5%       0.12 ±  7%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.copy_strings.isra.0
      0.10 ± 31%     +92.3%       0.20 ± 74%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.down_read.walk_component.path_lookupat
      0.11 ±  8%     +18.3%       0.13 ±  5%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.filemap_read.__kernel_read.exec_binprm
      0.12 ± 37%     -66.7%       0.04 ±104%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.getname_kernel.open_exec
      0.10 ± 18%     +34.6%       0.14 ± 12%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.unmap_page_range.unmap_vmas.exit_mmap
      3.58 ± 34%     -50.4%       1.78 ± 46%  perf-sched.wait_time.avg.ms.schedule_timeout.__skb_wait_for_more_packets.unix_dgram_recvmsg.__sys_recvfrom
    305.62 ±  6%     +21.0%     369.74 ± 10%  perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
      0.15 ± 37%     -80.4%       0.03 ±142%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__kmalloc.security_prepare_creds.prepare_creds
      0.13 ± 36%    +437.7%       0.72 ±155%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.down_read.walk_component.path_lookupat
      0.16 ± 28%     +42.2%       0.23 ±  5%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.down_write_killable.setup_arg_pages.load_elf_binary
      0.14 ± 37%     -69.1%       0.04 ±103%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.getname_kernel.open_exec
      4587 ± 50%     -59.8%       1844 ±103%  perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      3.58 ± 34%     -50.4%       1.78 ± 46%  perf-sched.wait_time.max.ms.schedule_timeout.__skb_wait_for_more_packets.unix_dgram_recvmsg.__sys_recvfrom
      0.09 ±  3%     +13.4%       0.11 ±  5%  perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion_interruptible.usb_stor_control_thread.kthread
     83958 ± 22%     -39.2%      51071 ±  7%  softirqs.CPU0.RCU
     52781 ± 24%     -42.2%      30516 ±  4%  softirqs.CPU10.RCU
     57678 ± 27%     -45.6%      31359 ±  6%  softirqs.CPU11.RCU
     49173 ± 17%     -38.0%      30501 ±  5%  softirqs.CPU12.RCU
     52583 ± 26%     -38.5%      32315 ± 11%  softirqs.CPU13.RCU
     53806 ± 32%     -42.7%      30806 ±  8%  softirqs.CPU14.RCU
     49603 ± 21%     -36.6%      31471 ± 11%  softirqs.CPU15.RCU
     48176 ± 17%     -37.4%      30155 ±  8%  softirqs.CPU16.RCU
     44644 ± 13%     -30.2%      31144 ±  4%  softirqs.CPU17.RCU
     41356 ± 12%     -26.0%      30596 ±  9%  softirqs.CPU18.RCU
     46651 ± 14%     -34.9%      30377 ±  8%  softirqs.CPU19.RCU
     58926 ± 31%     -41.1%      34685 ± 11%  softirqs.CPU2.RCU
     55066 ± 41%     -46.6%      29403 ±  4%  softirqs.CPU20.RCU
     46142 ± 19%     -34.0%      30462 ±  8%  softirqs.CPU21.RCU
     46125 ± 18%     -36.6%      29254 ±  5%  softirqs.CPU22.RCU
     43186 ± 12%     -27.6%      31271 ±  7%  softirqs.CPU27.RCU
     43298 ± 11%     -30.5%      30080 ±  4%  softirqs.CPU28.RCU
     45410 ± 17%     -33.3%      30308 ± 11%  softirqs.CPU29.RCU
     59941 ± 35%     -47.8%      31271 ±  9%  softirqs.CPU3.RCU
     42403 ±  9%     -32.1%      28793 ±  6%  softirqs.CPU30.RCU
     43296 ± 13%     -30.7%      29985 ±  6%  softirqs.CPU31.RCU
     50475 ± 15%     -39.1%      30719 ±  5%  softirqs.CPU32.RCU
     49326 ± 15%     -35.2%      31981 ± 17%  softirqs.CPU33.RCU
     51237 ± 12%     -39.6%      30970 ± 10%  softirqs.CPU34.RCU
     51711 ± 11%     -45.2%      28360 ±  3%  softirqs.CPU35.RCU
     49924 ± 17%     -43.2%      28373 ±  6%  softirqs.CPU36.RCU
     49802 ± 16%     -40.2%      29775 ± 11%  softirqs.CPU37.RCU
     52954 ± 17%     -46.1%      28545 ±  8%  softirqs.CPU38.RCU
     49661 ± 14%     -41.5%      29067 ±  8%  softirqs.CPU39.RCU
     60707 ± 28%     -43.7%      34155 ± 11%  softirqs.CPU4.RCU
     51390 ± 16%     -45.8%      27866 ±  8%  softirqs.CPU40.RCU
     58695 ± 20%     -51.3%      28563 ±  4%  softirqs.CPU41.RCU
     51871 ± 23%     -38.8%      31769 ± 15%  softirqs.CPU42.RCU
     50183 ± 12%     -42.7%      28735 ±  9%  softirqs.CPU43.RCU
     56295 ± 15%     -46.9%      29880 ± 14%  softirqs.CPU44.RCU
     54323 ± 17%     -45.3%      29698 ±  7%  softirqs.CPU45.RCU
     51361 ± 16%     -46.3%      27564 ±  2%  softirqs.CPU46.RCU
    163543 ± 17%     -43.8%      91839 ± 13%  softirqs.CPU47.RCU
     60813 ± 28%     -45.2%      33329 ±  7%  softirqs.CPU5.RCU
     60658 ± 27%     -44.3%      33808 ± 13%  softirqs.CPU50.RCU
     55217 ± 24%     -39.3%      33499 ± 13%  softirqs.CPU51.RCU
     58356 ± 28%     -43.8%      32790 ±  8%  softirqs.CPU52.RCU
     62221 ± 23%     -45.9%      33655 ±  5%  softirqs.CPU53.RCU
     55815 ± 27%     -44.1%      31202 ±  9%  softirqs.CPU54.RCU
     58726 ± 28%     -43.6%      33117 ± 10%  softirqs.CPU55.RCU
     52856 ± 26%     -42.7%      30294 ±  8%  softirqs.CPU56.RCU
     53839 ± 22%     -44.0%      30174 ±  8%  softirqs.CPU57.RCU
     55030 ± 28%     -45.0%      30274 ±  6%  softirqs.CPU58.RCU
     54756 ± 22%     -42.8%      31304 ±  5%  softirqs.CPU59.RCU
     54918 ± 31%     -47.1%      29032 ±  3%  softirqs.CPU6.RCU
     53535 ± 24%     -41.9%      31090 ±  6%  softirqs.CPU60.RCU
     52174 ± 23%     -41.2%      30704 ±  5%  softirqs.CPU61.RCU
     48552 ± 21%     -36.8%      30663 ±  8%  softirqs.CPU62.RCU
     56162 ± 29%     -43.7%      31643 ±  9%  softirqs.CPU63.RCU
     47055 ± 27%     -39.2%      28624 ±  3%  softirqs.CPU64.RCU
     45531 ± 16%     -36.9%      28741 ±  4%  softirqs.CPU65.RCU
     43623 ± 17%     -29.5%      30756 ±  7%  softirqs.CPU66.RCU
     43174 ± 20%     -30.6%      29945 ±  9%  softirqs.CPU67.RCU
     43982 ± 18%     -31.1%      30302 ± 10%  softirqs.CPU68.RCU
     45056 ± 21%     -37.4%      28219        softirqs.CPU69.RCU
     56189 ± 26%     -42.9%      32065 ±  9%  softirqs.CPU7.RCU
     47003 ± 20%     -37.3%      29489 ±  4%  softirqs.CPU70.RCU
     51052 ± 20%     -37.4%      31979 ± 10%  softirqs.CPU74.RCU
     42083 ± 10%     -24.0%      31964 ± 20%  softirqs.CPU75.RCU
     48436 ± 20%     -40.1%      29025 ±  7%  softirqs.CPU76.RCU
     47672 ± 22%     -37.1%      29972 ±  8%  softirqs.CPU77.RCU
     45766 ± 12%     -35.2%      29661 ±  8%  softirqs.CPU79.RCU
     55322 ± 30%     -41.8%      32216 ± 11%  softirqs.CPU8.RCU
     54363 ± 11%     -43.3%      30850 ±  6%  softirqs.CPU80.RCU
     53303 ± 15%     -37.7%      33196 ± 13%  softirqs.CPU81.RCU
     51853 ± 13%     -40.1%      31073 ±  5%  softirqs.CPU82.RCU
     51969 ± 13%     -41.2%      30553 ±  7%  softirqs.CPU83.RCU
     50169 ± 16%     -43.0%      28586 ±  4%  softirqs.CPU84.RCU
     51590 ± 19%     -40.9%      30502 ±  7%  softirqs.CPU85.RCU
     52550 ± 14%     -42.5%      30201 ±  8%  softirqs.CPU86.RCU
     51864 ± 16%     -42.4%      29871 ±  6%  softirqs.CPU87.RCU
     51409 ± 13%     -44.2%      28669 ±  9%  softirqs.CPU88.RCU
     53518 ± 15%     -45.4%      29224 ±  7%  softirqs.CPU89.RCU
     53986 ± 26%     -39.9%      32454 ± 12%  softirqs.CPU9.RCU
     48578 ± 14%     -39.4%      29454 ±  4%  softirqs.CPU90.RCU
     51131 ± 16%     -44.9%      28148 ±  4%  softirqs.CPU91.RCU
     51216 ± 13%     -43.8%      28775 ±  7%  softirqs.CPU92.RCU
     53664 ± 23%     -42.2%      31002 ± 12%  softirqs.CPU93.RCU
     53551 ± 14%     -44.8%      29573 ±  6%  softirqs.CPU94.RCU
     74490 ± 20%     -39.1%      45358 ± 11%  softirqs.CPU95.RCU
   5514177 ± 17%     -37.5%    3445171        softirqs.RCU
     93120 ± 31%     +64.2%     152920 ± 10%  interrupts.CAL:Function_call_interrupts
     20.67 ± 14%    +408.9%     105.17 ± 65%  interrupts.CPU10.NMI:Non-maskable_interrupts
     20.67 ± 14%    +408.9%     105.17 ± 65%  interrupts.CPU10.PMI:Performance_monitoring_interrupts
     20.33 ± 11%    +397.5%     101.17 ± 79%  interrupts.CPU11.NMI:Non-maskable_interrupts
     20.33 ± 11%    +397.5%     101.17 ± 79%  interrupts.CPU11.PMI:Performance_monitoring_interrupts
     20.50 ± 33%    +221.1%      65.83 ± 40%  interrupts.CPU13.NMI:Non-maskable_interrupts
     20.50 ± 33%    +221.1%      65.83 ± 40%  interrupts.CPU13.PMI:Performance_monitoring_interrupts
     11.50 ± 43%    +318.8%      48.17 ± 92%  interrupts.CPU13.RES:Rescheduling_interrupts
     16.00 ± 32%   +1041.7%     182.67 ±144%  interrupts.CPU14.NMI:Non-maskable_interrupts
     16.00 ± 32%   +1041.7%     182.67 ±144%  interrupts.CPU14.PMI:Performance_monitoring_interrupts
     18.67 ± 28%    +361.6%      86.17 ± 36%  interrupts.CPU15.NMI:Non-maskable_interrupts
     18.67 ± 28%    +361.6%      86.17 ± 36%  interrupts.CPU15.PMI:Performance_monitoring_interrupts
     18.50 ± 23%    +581.1%     126.00 ± 77%  interrupts.CPU16.NMI:Non-maskable_interrupts
     18.50 ± 23%    +581.1%     126.00 ± 77%  interrupts.CPU16.PMI:Performance_monitoring_interrupts
     20.33 ± 13%    +287.7%      78.83 ± 24%  interrupts.CPU17.NMI:Non-maskable_interrupts
     20.33 ± 13%    +287.7%      78.83 ± 24%  interrupts.CPU17.PMI:Performance_monitoring_interrupts
     20.33 ± 17%    +304.1%      82.17 ± 22%  interrupts.CPU18.NMI:Non-maskable_interrupts
     20.33 ± 17%    +304.1%      82.17 ± 22%  interrupts.CPU18.PMI:Performance_monitoring_interrupts
     19.33 ± 10%    +309.5%      79.17 ± 36%  interrupts.CPU19.NMI:Non-maskable_interrupts
     19.33 ± 10%    +309.5%      79.17 ± 36%  interrupts.CPU19.PMI:Performance_monitoring_interrupts
     20.00 ± 10%    +261.7%      72.33 ± 31%  interrupts.CPU20.NMI:Non-maskable_interrupts
     20.00 ± 10%    +261.7%      72.33 ± 31%  interrupts.CPU20.PMI:Performance_monitoring_interrupts
     20.00 ± 10%    +292.5%      78.50 ± 24%  interrupts.CPU21.NMI:Non-maskable_interrupts
     20.00 ± 10%    +292.5%      78.50 ± 24%  interrupts.CPU21.PMI:Performance_monitoring_interrupts
     20.33 ± 15%    +223.8%      65.83 ± 39%  interrupts.CPU22.NMI:Non-maskable_interrupts
     20.33 ± 15%    +223.8%      65.83 ± 39%  interrupts.CPU22.PMI:Performance_monitoring_interrupts
     41.67 ± 26%    +102.8%      84.50 ± 26%  interrupts.CPU23.TLB:TLB_shootdowns
     17.50 ± 24%    +473.3%     100.33 ± 65%  interrupts.CPU25.NMI:Non-maskable_interrupts
     17.50 ± 24%    +473.3%     100.33 ± 65%  interrupts.CPU25.PMI:Performance_monitoring_interrupts
    551.83 ±  3%    +108.4%       1150 ±110%  interrupts.CPU26.CAL:Function_call_interrupts
     25.50 ± 66%    +359.5%     117.17 ± 59%  interrupts.CPU26.NMI:Non-maskable_interrupts
     25.50 ± 66%    +359.5%     117.17 ± 59%  interrupts.CPU26.PMI:Performance_monitoring_interrupts
     22.50 ± 29%    +343.7%      99.83 ± 84%  interrupts.CPU28.NMI:Non-maskable_interrupts
     22.50 ± 29%    +343.7%      99.83 ± 84%  interrupts.CPU28.PMI:Performance_monitoring_interrupts
     18.83 ± 23%    +448.7%     103.33 ± 64%  interrupts.CPU31.NMI:Non-maskable_interrupts
     18.83 ± 23%    +448.7%     103.33 ± 64%  interrupts.CPU31.PMI:Performance_monitoring_interrupts
     22.33 ± 43%    +264.9%      81.50 ± 42%  interrupts.CPU32.NMI:Non-maskable_interrupts
     22.33 ± 43%    +264.9%      81.50 ± 42%  interrupts.CPU32.PMI:Performance_monitoring_interrupts
     18.00 ± 26%    +402.8%      90.50 ± 51%  interrupts.CPU34.NMI:Non-maskable_interrupts
     18.00 ± 26%    +402.8%      90.50 ± 51%  interrupts.CPU34.PMI:Performance_monitoring_interrupts
      6.00 ± 84%    +522.2%      37.33 ±119%  interrupts.CPU34.RES:Rescheduling_interrupts
     23.83 ± 62%    +325.9%     101.50 ± 54%  interrupts.CPU36.NMI:Non-maskable_interrupts
     23.83 ± 62%    +325.9%     101.50 ± 54%  interrupts.CPU36.PMI:Performance_monitoring_interrupts
     25.17 ± 79%    +437.7%     135.33 ± 75%  interrupts.CPU37.NMI:Non-maskable_interrupts
     25.17 ± 79%    +437.7%     135.33 ± 75%  interrupts.CPU37.PMI:Performance_monitoring_interrupts
     18.17 ± 23%    +337.6%      79.50 ± 41%  interrupts.CPU38.NMI:Non-maskable_interrupts
     18.17 ± 23%    +337.6%      79.50 ± 41%  interrupts.CPU38.PMI:Performance_monitoring_interrupts
     13.83 ± 32%    +636.1%     101.83 ± 78%  interrupts.CPU39.NMI:Non-maskable_interrupts
     13.83 ± 32%    +636.1%     101.83 ± 78%  interrupts.CPU39.PMI:Performance_monitoring_interrupts
     20.83 ± 45%    +346.4%      93.00 ± 57%  interrupts.CPU45.NMI:Non-maskable_interrupts
     20.83 ± 45%    +346.4%      93.00 ± 57%  interrupts.CPU45.PMI:Performance_monitoring_interrupts
    677.83 ±  6%     -12.9%     590.17 ±  3%  interrupts.CPU47.CAL:Function_call_interrupts
     20.67 ± 99%    +317.7%      86.33 ± 33%  interrupts.CPU48.TLB:TLB_shootdowns
     21.17 ± 30%    +353.5%      96.00 ± 18%  interrupts.CPU5.NMI:Non-maskable_interrupts
     21.17 ± 30%    +353.5%      96.00 ± 18%  interrupts.CPU5.PMI:Performance_monitoring_interrupts
      9.50 ± 37%    +649.1%      71.17 ± 84%  interrupts.CPU52.RES:Rescheduling_interrupts
     21.00 ± 26%    +342.1%      92.83 ± 13%  interrupts.CPU53.NMI:Non-maskable_interrupts
     21.00 ± 26%    +342.1%      92.83 ± 13%  interrupts.CPU53.PMI:Performance_monitoring_interrupts
     20.17 ± 18%    +330.6%      86.83 ± 16%  interrupts.CPU54.NMI:Non-maskable_interrupts
     20.17 ± 18%    +330.6%      86.83 ± 16%  interrupts.CPU54.PMI:Performance_monitoring_interrupts
     21.50 ± 22%    +276.7%      81.00 ± 26%  interrupts.CPU55.NMI:Non-maskable_interrupts
     21.50 ± 22%    +276.7%      81.00 ± 26%  interrupts.CPU55.PMI:Performance_monitoring_interrupts
     12.33 ± 75%    +297.3%      49.00 ± 66%  interrupts.CPU55.RES:Rescheduling_interrupts
     18.50 ± 29%    +418.9%      96.00 ± 23%  interrupts.CPU56.NMI:Non-maskable_interrupts
     18.50 ± 29%    +418.9%      96.00 ± 23%  interrupts.CPU56.PMI:Performance_monitoring_interrupts
     19.67 ± 13%    +444.1%     107.00 ± 51%  interrupts.CPU57.NMI:Non-maskable_interrupts
     19.67 ± 13%    +444.1%     107.00 ± 51%  interrupts.CPU57.PMI:Performance_monitoring_interrupts
     21.17 ± 14%    +336.2%      92.33 ± 25%  interrupts.CPU58.NMI:Non-maskable_interrupts
     21.17 ± 14%    +336.2%      92.33 ± 25%  interrupts.CPU58.PMI:Performance_monitoring_interrupts
     21.17 ± 15%    +475.6%     121.83 ± 73%  interrupts.CPU59.NMI:Non-maskable_interrupts
     21.17 ± 15%    +475.6%     121.83 ± 73%  interrupts.CPU59.PMI:Performance_monitoring_interrupts
     18.17 ± 29%    +379.8%      87.17 ± 16%  interrupts.CPU6.NMI:Non-maskable_interrupts
     18.17 ± 29%    +379.8%      87.17 ± 16%  interrupts.CPU6.PMI:Performance_monitoring_interrupts
      6.33 ± 81%    +473.7%      36.33 ± 87%  interrupts.CPU6.RES:Rescheduling_interrupts
     19.83 ± 15%    +337.0%      86.67 ± 16%  interrupts.CPU61.NMI:Non-maskable_interrupts
     19.83 ± 15%    +337.0%      86.67 ± 16%  interrupts.CPU61.PMI:Performance_monitoring_interrupts
     19.50 ± 14%    +731.6%     162.17 ±109%  interrupts.CPU62.NMI:Non-maskable_interrupts
     19.50 ± 14%    +731.6%     162.17 ±109%  interrupts.CPU62.PMI:Performance_monitoring_interrupts
     19.67 ± 33%    +383.1%      95.00 ± 31%  interrupts.CPU63.NMI:Non-maskable_interrupts
     19.67 ± 33%    +383.1%      95.00 ± 31%  interrupts.CPU63.PMI:Performance_monitoring_interrupts
     20.33 ± 11%    +452.5%     112.33 ± 58%  interrupts.CPU64.NMI:Non-maskable_interrupts
     20.33 ± 11%    +452.5%     112.33 ± 58%  interrupts.CPU64.PMI:Performance_monitoring_interrupts
     21.00 ± 16%    +312.7%      86.67 ± 15%  interrupts.CPU65.NMI:Non-maskable_interrupts
     21.00 ± 16%    +312.7%      86.67 ± 15%  interrupts.CPU65.PMI:Performance_monitoring_interrupts
     21.33 ± 20%    +318.0%      89.17 ± 12%  interrupts.CPU66.NMI:Non-maskable_interrupts
     21.33 ± 20%    +318.0%      89.17 ± 12%  interrupts.CPU66.PMI:Performance_monitoring_interrupts
     11.50 ± 41%    +523.2%      71.67 ±143%  interrupts.CPU66.RES:Rescheduling_interrupts
     17.83 ± 13%    +546.7%     115.33 ± 50%  interrupts.CPU67.NMI:Non-maskable_interrupts
     17.83 ± 13%    +546.7%     115.33 ± 50%  interrupts.CPU67.PMI:Performance_monitoring_interrupts
     18.17 ± 14%    +384.4%      88.00 ± 15%  interrupts.CPU68.NMI:Non-maskable_interrupts
     18.17 ± 14%    +384.4%      88.00 ± 15%  interrupts.CPU68.PMI:Performance_monitoring_interrupts
     19.00 ± 16%    +320.2%      79.83 ± 28%  interrupts.CPU69.NMI:Non-maskable_interrupts
     19.00 ± 16%    +320.2%      79.83 ± 28%  interrupts.CPU69.PMI:Performance_monitoring_interrupts
     17.17 ± 37%    +378.6%      82.17 ± 29%  interrupts.CPU7.NMI:Non-maskable_interrupts
     17.17 ± 37%    +378.6%      82.17 ± 29%  interrupts.CPU7.PMI:Performance_monitoring_interrupts
     17.67 ± 16%    +401.9%      88.67 ± 17%  interrupts.CPU70.NMI:Non-maskable_interrupts
     17.67 ± 16%    +401.9%      88.67 ± 17%  interrupts.CPU70.PMI:Performance_monitoring_interrupts
     17.17 ± 21%    +599.0%     120.00 ± 49%  interrupts.CPU73.NMI:Non-maskable_interrupts
     17.17 ± 21%    +599.0%     120.00 ± 49%  interrupts.CPU73.PMI:Performance_monitoring_interrupts
     18.33 ± 17%    +664.5%     140.17 ± 60%  interrupts.CPU74.NMI:Non-maskable_interrupts
     18.33 ± 17%    +664.5%     140.17 ± 60%  interrupts.CPU74.PMI:Performance_monitoring_interrupts
     24.33 ± 75%    +266.4%      89.17 ± 76%  interrupts.CPU75.NMI:Non-maskable_interrupts
     24.33 ± 75%    +266.4%      89.17 ± 76%  interrupts.CPU75.PMI:Performance_monitoring_interrupts
     18.67 ± 24%    +688.4%     147.17 ± 81%  interrupts.CPU76.NMI:Non-maskable_interrupts
     18.67 ± 24%    +688.4%     147.17 ± 81%  interrupts.CPU76.PMI:Performance_monitoring_interrupts
     12.17 ± 44%    +265.8%      44.50 ± 76%  interrupts.CPU77.RES:Rescheduling_interrupts
     18.83 ± 25%    +410.6%      96.17 ± 36%  interrupts.CPU79.NMI:Non-maskable_interrupts
     18.83 ± 25%    +410.6%      96.17 ± 36%  interrupts.CPU79.PMI:Performance_monitoring_interrupts
     19.50 ± 12%    +582.9%     133.17 ± 89%  interrupts.CPU8.NMI:Non-maskable_interrupts
     19.50 ± 12%    +582.9%     133.17 ± 89%  interrupts.CPU8.PMI:Performance_monitoring_interrupts
      9.67 ± 34%    +286.2%      37.33 ± 80%  interrupts.CPU80.RES:Rescheduling_interrupts
     18.00 ± 26%    +412.0%      92.17 ± 56%  interrupts.CPU82.NMI:Non-maskable_interrupts
     18.00 ± 26%    +412.0%      92.17 ± 56%  interrupts.CPU82.PMI:Performance_monitoring_interrupts
      9.67 ± 50%    +700.0%      77.33 ±125%  interrupts.CPU82.RES:Rescheduling_interrupts
     19.17 ± 36%    +463.5%     108.00 ± 62%  interrupts.CPU84.NMI:Non-maskable_interrupts
     19.17 ± 36%    +463.5%     108.00 ± 62%  interrupts.CPU84.PMI:Performance_monitoring_interrupts
     12.83 ± 42%    +351.9%      58.00 ± 64%  interrupts.CPU85.RES:Rescheduling_interrupts
     17.83 ± 24%    +379.4%      85.50 ± 48%  interrupts.CPU86.NMI:Non-maskable_interrupts
     17.83 ± 24%    +379.4%      85.50 ± 48%  interrupts.CPU86.PMI:Performance_monitoring_interrupts
     17.00 ± 25%    +597.1%     118.50 ± 96%  interrupts.CPU87.NMI:Non-maskable_interrupts
     17.00 ± 25%    +597.1%     118.50 ± 96%  interrupts.CPU87.PMI:Performance_monitoring_interrupts
     18.33 ± 27%    +469.1%     104.33 ± 61%  interrupts.CPU9.NMI:Non-maskable_interrupts
     18.33 ± 27%    +469.1%     104.33 ± 61%  interrupts.CPU9.PMI:Performance_monitoring_interrupts
      8.00 ± 61%    +472.9%      45.83 ± 66%  interrupts.CPU9.RES:Rescheduling_interrupts
     21.50 ± 43%    +427.1%     113.33 ± 68%  interrupts.CPU93.NMI:Non-maskable_interrupts
     21.50 ± 43%    +427.1%     113.33 ± 68%  interrupts.CPU93.PMI:Performance_monitoring_interrupts
     39.83 ± 13%    +853.6%     379.83 ± 24%  interrupts.CPU95.TLB:TLB_shootdowns
      4076 ±  9%    +145.6%      10013 ±  3%  interrupts.NMI:Non-maskable_interrupts
      4076 ±  9%    +145.6%      10013 ±  3%  interrupts.PMI:Performance_monitoring_interrupts
      1164 ± 10%    +192.5%       3404 ±  3%  interrupts.RES:Rescheduling_interrupts
    844.83 ± 36%    +122.8%       1882 ± 17%  interrupts.TLB:TLB_shootdowns
     37.96 ± 13%     -22.4       15.54 ± 12%  perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
     14.65 ± 12%      -9.1        5.59 ±  7%  perf-profile.calltrace.cycles-pp.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
     12.64 ± 10%      -7.7        4.95 ±  6%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
     11.71 ± 12%      -7.3        4.36 ± 11%  perf-profile.calltrace.cycles-pp.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
     11.27 ± 12%      -6.8        4.46 ±  8%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
     10.12 ± 14%      -6.6        3.49 ± 11%  perf-profile.calltrace.cycles-pp.exec_binprm.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64
      9.74 ± 15%      -6.3        3.40 ± 12%  perf-profile.calltrace.cycles-pp.load_elf_binary.exec_binprm.bprm_execve.do_execveat_common.__x64_sys_execve
      9.39 ± 11%      -6.0        3.43 ± 12%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.execve
      9.39 ± 11%      -6.0        3.43 ± 12%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
      9.39 ± 11%      -6.0        3.43 ± 12%  perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
      9.39 ± 11%      -6.0        3.43 ± 12%  perf-profile.calltrace.cycles-pp.execve
      5.38 ± 18%      -3.3        2.13 ± 12%  perf-profile.calltrace.cycles-pp.asm_exc_page_fault
      5.32 ± 16%      -3.1        2.18 ±  5%  perf-profile.calltrace.cycles-pp.main
      5.29 ± 16%      -3.1        2.18 ±  4%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.execve.main
      5.29 ± 16%      -3.1        2.18 ±  4%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve.main
      5.29 ± 16%      -3.1        2.18 ±  4%  perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve.main
      5.29 ± 16%      -3.1        2.18 ±  4%  perf-profile.calltrace.cycles-pp.execve.main
      4.19 ± 17%      -3.1        1.12 ± 17%  perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
      4.65 ± 15%      -3.0        1.65 ± 11%  perf-profile.calltrace.cycles-pp.begin_new_exec.load_elf_binary.exec_binprm.bprm_execve.do_execveat_common
      4.84 ± 17%      -2.9        1.95 ± 15%  perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault
      4.80 ± 17%      -2.9        1.93 ± 15%  perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
      4.36 ± 15%      -2.8        1.51 ± 11%  perf-profile.calltrace.cycles-pp.mmput.begin_new_exec.load_elf_binary.exec_binprm.bprm_execve
      4.32 ± 15%      -2.8        1.51 ± 11%  perf-profile.calltrace.cycles-pp.exit_mmap.mmput.begin_new_exec.load_elf_binary.exec_binprm
      4.82 ± 12%      -2.7        2.08 ±  6%  perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
      4.80 ± 13%      -2.7        2.07 ±  7%  perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
      4.15 ± 33%      -2.7        1.42 ± 12%  perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
      4.07 ± 18%      -2.5        1.53 ± 11%  perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
      3.94 ± 13%      -2.2        1.70 ±  4%  perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
      3.91 ± 13%      -2.2        1.68 ±  4%  perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64
      2.63 ± 24%      -1.7        0.90 ± 17%  perf-profile.calltrace.cycles-pp.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
      2.23 ± 17%      -1.4        0.81 ± 11%  perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      2.15 ±  7%      -1.4        0.74 ± 14%  perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.begin_new_exec.load_elf_binary
      2.09 ±  6%      -1.4        0.73 ± 14%  perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.begin_new_exec
      2.40 ± 19%      -1.3        1.05 ± 47%  perf-profile.calltrace.cycles-pp.irq_work_run_list.irq_work_run.__sysvec_irq_work.sysvec_irq_work.asm_sysvec_irq_work
      2.40 ± 19%      -1.3        1.05 ± 47%  perf-profile.calltrace.cycles-pp.irq_work_single.irq_work_run_list.irq_work_run.__sysvec_irq_work.sysvec_irq_work
      2.40 ± 19%      -1.3        1.05 ± 47%  perf-profile.calltrace.cycles-pp.printk.irq_work_single.irq_work_run_list.irq_work_run.__sysvec_irq_work
      2.40 ± 19%      -1.3        1.05 ± 47%  perf-profile.calltrace.cycles-pp.vprintk_emit.printk.irq_work_single.irq_work_run_list.irq_work_run
      2.40 ± 19%      -1.3        1.05 ± 47%  perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.printk.irq_work_single.irq_work_run_list
      2.03 ± 13%      -1.3        0.69 ±  8%  perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.00 ± 11%      -1.3        0.68 ±  7%  perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.31 ± 19%      -1.3        1.00 ± 47%  perf-profile.calltrace.cycles-pp.serial8250_console_write.console_unlock.vprintk_emit.printk.irq_work_single
      2.02 ± 13%      -1.3        0.75 ± 18%  perf-profile.calltrace.cycles-pp.filemap_map_pages.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
      2.25 ± 19%      -1.3        0.98 ± 47%  perf-profile.calltrace.cycles-pp.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit.printk
      1.90 ±  7%      -1.2        0.66 ± 16%  perf-profile.calltrace.cycles-pp.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap.mmput
      1.87 ± 10%      -1.2        0.64 ±  7%  perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.22 ± 13%      -1.2        1.02 ± 15%  perf-profile.calltrace.cycles-pp.link_path_walk.path_openat.do_filp_open.do_sys_openat2.do_sys_open
      2.13 ± 19%      -1.2        0.94 ± 47%  perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock
      2.13 ± 19%      -1.2        0.94 ± 47%  perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit
      1.68 ± 11%      -1.2        0.49 ± 44%  perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64
      1.97 ± 10%      -1.1        0.91 ± 16%  perf-profile.calltrace.cycles-pp.walk_component.link_path_walk.path_openat.do_filp_open.do_sys_openat2
      1.40 ± 15%      -1.0        0.36 ± 71%  perf-profile.calltrace.cycles-pp.ret_from_fork
      1.40 ± 15%      -1.0        0.36 ± 71%  perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
      1.85 ±  9%      -1.0        0.82 ± 23%  perf-profile.calltrace.cycles-pp.__do_sys_newstat.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.84 ± 10%      -1.0        0.81 ± 22%  perf-profile.calltrace.cycles-pp.vfs_statx.__do_sys_newstat.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.28 ± 29%      -1.0        0.26 ±100%  perf-profile.calltrace.cycles-pp.sched_exec.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64
      1.65 ± 15%      -1.0        0.64 ± 11%  perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      1.34 ± 31%      -1.0        0.35 ±107%  perf-profile.calltrace.cycles-pp.load_elf_interp.load_elf_binary.exec_binprm.bprm_execve.do_execveat_common
      1.71 ± 28%      -1.0        0.73 ± 47%  perf-profile.calltrace.cycles-pp.io_serial_in.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write
      1.80 ± 13%      -1.0        0.84 ± 12%  perf-profile.calltrace.cycles-pp.copy_strings.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.55 ± 10%      -0.9        0.62 ± 50%  perf-profile.calltrace.cycles-pp.filename_lookup.vfs_statx.__do_sys_newstat.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.51 ± 12%      -0.9        0.60 ± 50%  perf-profile.calltrace.cycles-pp.path_lookupat.filename_lookup.vfs_statx.__do_sys_newstat.do_syscall_64
      1.22 ± 18%      -0.8        0.43 ± 75%  perf-profile.calltrace.cycles-pp.link_path_walk.path_lookupat.filename_lookup.vfs_statx.__do_sys_newstat
      0.73 ± 21%      +2.3        3.07 ±  7%  perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      0.00            +2.8        2.80 ±  6%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt
      0.00            +2.8        2.84 ±  6%  perf-profile.calltrace.cycles-pp.ktime_get.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      2.52 ± 14%      +3.5        6.06 ±  7%  perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      2.35 ± 11%      +3.7        6.05 ±  7%  perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
      0.00            +4.5        4.51 ±  3%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
      1.23 ± 11%      +4.5        5.77 ±  6%  perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      0.00            +4.6        4.64 ±  3%  perf-profile.calltrace.cycles-pp.ktime_get.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      0.00            +4.9        4.94 ±  7%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.sched_clock_tick.scheduler_tick.update_process_times
      0.00            +5.3        5.30 ±  6%  perf-profile.calltrace.cycles-pp.ktime_get.sched_clock_tick.scheduler_tick.update_process_times.tick_sched_handle
      0.00            +5.3        5.33 ±  6%  perf-profile.calltrace.cycles-pp.sched_clock_tick.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer
      0.00            +6.2        6.22 ±  4%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.cpuidle_enter_state
      0.00            +6.3        6.29 ±  4%  perf-profile.calltrace.cycles-pp.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.cpuidle_enter_state.cpuidle_enter
      0.00            +6.4        6.35 ±  4%  perf-profile.calltrace.cycles-pp.sched_clock_tick.sched_clock_idle_wakeup_event.cpuidle_enter_state.cpuidle_enter.do_idle
      0.00            +6.5        6.48 ±  4%  perf-profile.calltrace.cycles-pp.sched_clock_idle_wakeup_event.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
      0.59 ± 80%      +6.8        7.38 ±  4%  perf-profile.calltrace.cycles-pp.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      0.49 ±103%      +6.9        7.37 ±  4%  perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      1.78 ± 19%      +7.1        8.90 ±  2%  perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +7.1        7.13 ±  4%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.61 ± 51%      +7.4        8.06 ±  3%  perf-profile.calltrace.cycles-pp.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.98 ± 49%      +7.6        8.54 ±  2%  perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      2.93 ± 12%      +7.9       10.80 ±  4%  perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      0.00            +8.0        7.97 ±  3%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt
      0.00            +8.0        8.02 ±  2%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      0.00            +8.3        8.31 ±  3%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.tick_irq_enter
      0.00            +8.4        8.39 ±  3%  perf-profile.calltrace.cycles-pp.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.tick_irq_enter.irq_enter_rcu
      0.00            +8.4        8.44 ±  2%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get_update_offsets_now.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      0.09 ±223%      +8.5        8.56 ±  2%  perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +8.5        8.47 ±  3%  perf-profile.calltrace.cycles-pp.sched_clock_tick.sched_clock_idle_wakeup_event.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt
      0.00            +8.6        8.61 ±  3%  perf-profile.calltrace.cycles-pp.sched_clock_idle_wakeup_event.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      4.41 ± 17%      +9.8       14.19 ±  4%  perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      1.23 ± 22%     +15.5       16.78 ±  3%  perf-profile.calltrace.cycles-pp.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      1.20 ± 24%     +15.6       16.77 ±  3%  perf-profile.calltrace.cycles-pp.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
     59.83 ±  5%     +23.6       83.45        perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     59.83 ±  5%     +23.6       83.45        perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
     59.80 ±  5%     +23.6       83.43        perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     60.74 ±  5%     +23.7       84.46        perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
      7.40 ± 11%     +24.7       32.10 ±  2%  perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      7.27 ± 11%     +24.8       32.06 ±  2%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
     54.60 ±  6%     +27.4       81.97        perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     53.26 ±  7%     +28.0       81.27        perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
     15.89 ±  8%     +43.1       59.00        perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
     12.01 ±  6%     +45.5       57.50        perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
     38.06 ± 13%     -22.4       15.66 ± 11%  perf-profile.children.cycles-pp.intel_idle
     27.72 ±  8%     -17.0       10.74 ±  5%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
     26.32 ± 10%     -16.1       10.24 ±  6%  perf-profile.children.cycles-pp.do_syscall_64
     14.71 ± 12%      -9.1        5.61 ±  7%  perf-profile.children.cycles-pp.__x64_sys_execve
     14.68 ± 12%      -9.1        5.60 ±  7%  perf-profile.children.cycles-pp.execve
     14.67 ± 12%      -9.1        5.60 ±  7%  perf-profile.children.cycles-pp.do_execveat_common
     11.72 ± 12%      -7.3        4.37 ± 11%  perf-profile.children.cycles-pp.bprm_execve
     10.12 ± 14%      -6.4        3.71 ± 10%  perf-profile.children.cycles-pp.exec_binprm
     10.05 ± 14%      -6.4        3.67 ± 11%  perf-profile.children.cycles-pp.load_elf_binary
      8.40 ± 19%      -5.1        3.32 ±  9%  perf-profile.children.cycles-pp.asm_exc_page_fault
      7.42 ± 19%      -4.4        2.99 ± 11%  perf-profile.children.cycles-pp.exc_page_fault
      7.38 ± 19%      -4.4        2.96 ± 11%  perf-profile.children.cycles-pp.do_user_addr_fault
      6.70 ± 20%      -4.1        2.55 ±  6%  perf-profile.children.cycles-pp.handle_mm_fault
      6.34 ± 21%      -4.0        2.39 ±  8%  perf-profile.children.cycles-pp.__handle_mm_fault
      5.50 ± 14%      -3.3        2.21 ±  4%  perf-profile.children.cycles-pp.main
      4.25 ± 17%      -3.1        1.14 ± 16%  perf-profile.children.cycles-pp.menu_select
      4.78 ± 15%      -3.1        1.69 ± 12%  perf-profile.children.cycles-pp.begin_new_exec
      4.44 ± 16%      -2.9        1.54 ± 11%  perf-profile.children.cycles-pp.mmput
      4.40 ± 16%      -2.9        1.53 ± 11%  perf-profile.children.cycles-pp.exit_mmap
      4.86 ± 12%      -2.8        2.09 ±  6%  perf-profile.children.cycles-pp.do_sys_open
      4.84 ± 13%      -2.8        2.08 ±  6%  perf-profile.children.cycles-pp.do_sys_openat2
      4.38 ± 15%      -2.7        1.63 ± 10%  perf-profile.children.cycles-pp.do_fault
      4.29 ± 14%      -2.4        1.92 ±  5%  perf-profile.children.cycles-pp.do_filp_open
      4.26 ± 13%      -2.4        1.91 ±  5%  perf-profile.children.cycles-pp.path_openat
      3.39 ±  8%      -2.1        1.30 ± 11%  perf-profile.children.cycles-pp.filemap_map_pages
      3.46 ± 16%      -1.9        1.52 ± 11%  perf-profile.children.cycles-pp.walk_component
      3.58 ± 15%      -1.9        1.67 ± 13%  perf-profile.children.cycles-pp.link_path_walk
      2.95 ± 12%      -1.7        1.21 ± 13%  perf-profile.children.cycles-pp.vm_mmap_pgoff
      2.63 ±  9%      -1.7        0.98 ±  8%  perf-profile.children.cycles-pp.__softirqentry_text_start
      2.72 ± 12%      -1.7        1.07 ±  9%  perf-profile.children.cycles-pp.do_mmap
      2.42 ±  7%      -1.6        0.83 ± 13%  perf-profile.children.cycles-pp.unmap_vmas
      2.58 ± 16%      -1.5        1.05 ± 10%  perf-profile.children.cycles-pp.irq_exit_rcu
      2.33 ±  6%      -1.5        0.81 ± 13%  perf-profile.children.cycles-pp.unmap_page_range
      1.91 ± 19%      -1.5        0.43 ± 18%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      2.43 ± 13%      -1.5        0.96 ± 10%  perf-profile.children.cycles-pp.mmap_region
      2.12 ±  5%      -1.4        0.73 ± 14%  perf-profile.children.cycles-pp.zap_pte_range
      2.18 ± 13%      -1.4        0.80 ±  9%  perf-profile.children.cycles-pp.kmem_cache_alloc
      2.04 ± 12%      -1.3        0.70 ±  7%  perf-profile.children.cycles-pp.ksys_mmap_pgoff
      2.04 ± 34%      -1.2        0.79 ± 19%  perf-profile.children.cycles-pp.filename_lookup
      1.53 ± 27%      -1.2        0.29 ± 17%  perf-profile.children.cycles-pp.tick_nohz_next_event
      1.78 ± 18%      -1.1        0.68 ± 11%  perf-profile.children.cycles-pp.__lookup_slow
      1.50 ± 20%      -1.1        0.41 ± 25%  perf-profile.children.cycles-pp.kmem_cache_free
      1.84 ± 21%      -1.1        0.77 ± 19%  perf-profile.children.cycles-pp.path_lookupat
      1.56 ± 24%      -1.0        0.51 ± 13%  perf-profile.children.cycles-pp.__do_munmap
      1.86 ±  9%      -1.0        0.82 ± 23%  perf-profile.children.cycles-pp.__do_sys_newstat
      1.84 ±  9%      -1.0        0.81 ± 22%  perf-profile.children.cycles-pp.vfs_statx
      1.42 ± 59%      -1.0        0.41 ± 15%  perf-profile.children.cycles-pp.tlb_finish_mmu
      1.40 ± 60%      -1.0        0.40 ± 16%  perf-profile.children.cycles-pp.tlb_flush_mmu
      1.81 ± 14%      -0.9        0.86 ± 12%  perf-profile.children.cycles-pp.copy_strings
      1.45 ±  2%      -0.9        0.52 ± 14%  perf-profile.children.cycles-pp.next_uptodate_page
      1.40 ± 15%      -0.9        0.50 ± 14%  perf-profile.children.cycles-pp.kthread
      1.40 ± 15%      -0.9        0.50 ± 15%  perf-profile.children.cycles-pp.ret_from_fork
      1.49 ± 14%      -0.9        0.59 ± 15%  perf-profile.children.cycles-pp.native_irq_return_iret
      1.19 ± 70%      -0.9        0.32 ± 17%  perf-profile.children.cycles-pp.release_pages
      1.93 ± 20%      -0.9        1.08 ±  8%  perf-profile.children.cycles-pp.io_serial_in
      2.31 ± 19%      -0.8        1.47 ±  6%  perf-profile.children.cycles-pp.serial8250_console_write
      1.22 ±  3%      -0.8        0.39 ± 10%  perf-profile.children.cycles-pp.rcu_core
      1.26 ± 11%      -0.8        0.43 ± 16%  perf-profile.children.cycles-pp.create_elf_tables
      1.28 ± 30%      -0.8        0.49 ±  7%  perf-profile.children.cycles-pp.sched_exec
      1.23 ± 10%      -0.8        0.44 ± 10%  perf-profile.children.cycles-pp._raw_spin_lock
      1.22 ± 25%      -0.8        0.44 ± 17%  perf-profile.children.cycles-pp.d_alloc_parallel
      1.42 ± 29%      -0.8        0.64 ± 26%  perf-profile.children.cycles-pp.elf_map
      1.34 ± 31%      -0.8        0.57 ± 33%  perf-profile.children.cycles-pp.load_elf_interp
      1.15 ± 15%      -0.7        0.42 ±  5%  perf-profile.children.cycles-pp.select_task_rq_fair
      1.11 ± 13%      -0.7        0.41 ± 20%  perf-profile.children.cycles-pp.strnlen_user
      1.10 ± 17%      -0.7        0.41 ±  6%  perf-profile.children.cycles-pp.find_idlest_group
      1.27 ± 19%      -0.7        0.58 ± 36%  perf-profile.children.cycles-pp.dput
      1.03 ± 35%      -0.7        0.36 ± 15%  perf-profile.children.cycles-pp.d_alloc
      1.06 ± 47%      -0.7        0.40 ± 11%  perf-profile.children.cycles-pp.__alloc_pages_nodemask
      0.97 ±  7%      -0.7        0.32 ± 16%  perf-profile.children.cycles-pp.rcu_do_batch
      0.78 ± 94%      -0.7        0.13 ± 25%  perf-profile.children.cycles-pp.do_faccessat
      0.99 ± 34%      -0.6        0.35 ±  8%  perf-profile.children.cycles-pp.__split_vma
      0.95 ± 56%      -0.6        0.34 ± 21%  perf-profile.children.cycles-pp.__clear_user
      0.99 ± 19%      -0.6        0.38 ± 10%  perf-profile.children.cycles-pp.perf_event_mmap
      0.73 ± 17%      -0.6        0.13 ± 31%  perf-profile.children.cycles-pp.smpboot_thread_fn
      0.98 ± 20%      -0.6        0.38 ± 10%  perf-profile.children.cycles-pp.exit_to_user_mode_prepare
      0.92 ± 18%      -0.6        0.34 ± 14%  perf-profile.children.cycles-pp.___might_sleep
      0.85 ± 35%      -0.6        0.28 ± 17%  perf-profile.children.cycles-pp.__d_alloc
      0.69 ± 24%      -0.6        0.12 ± 17%  perf-profile.children.cycles-pp.irqtime_account_irq
      0.99 ± 32%      -0.6        0.44 ± 14%  perf-profile.children.cycles-pp.__x64_sys_mprotect
      0.93 ± 19%      -0.6        0.38 ± 19%  perf-profile.children.cycles-pp.alloc_empty_file
      0.99 ± 32%      -0.6        0.44 ± 14%  perf-profile.children.cycles-pp.do_mprotect_pkey
      0.79 ± 41%      -0.5        0.24 ± 26%  perf-profile.children.cycles-pp.__vm_munmap
      0.90 ± 21%      -0.5        0.36 ± 17%  perf-profile.children.cycles-pp.__alloc_file
      0.79 ± 37%      -0.5        0.26 ± 13%  perf-profile.children.cycles-pp.__vma_adjust
      0.65 ± 10%      -0.5        0.12 ± 32%  perf-profile.children.cycles-pp.run_ksoftirqd
      0.91 ± 45%      -0.5        0.38 ± 23%  perf-profile.children.cycles-pp.do_set_pte
      0.86 ± 16%      -0.5        0.35 ± 24%  perf-profile.children.cycles-pp.getname_flags
      0.90 ± 33%      -0.5        0.40 ± 13%  perf-profile.children.cycles-pp.mprotect_fixup
      1.02 ± 11%      -0.5        0.54 ± 33%  perf-profile.children.cycles-pp.step_into
      0.70 ± 56%      -0.5        0.23 ± 19%  perf-profile.children.cycles-pp.wp_page_copy
      0.73 ± 33%      -0.5        0.26 ± 18%  perf-profile.children.cycles-pp.task_work_run
      0.83 ± 23%      -0.5        0.37 ± 14%  perf-profile.children.cycles-pp.rebalance_domains
      0.71 ± 21%      -0.4        0.28 ± 21%  perf-profile.children.cycles-pp.unmap_region
      0.71 ± 16%      -0.4        0.28 ± 17%  perf-profile.children.cycles-pp.do_anonymous_page
      0.61 ± 13%      -0.4        0.19 ± 30%  perf-profile.children.cycles-pp.get_user_arg_ptr
      0.67 ± 15%      -0.4        0.25 ± 22%  perf-profile.children.cycles-pp.strncpy_from_user
      0.67 ±  8%      -0.4        0.26 ± 14%  perf-profile.children.cycles-pp.free_pgtables
      0.51 ± 80%      -0.4        0.11 ± 13%  perf-profile.children.cycles-pp.flush_tlb_mm_range
      0.53 ± 70%      -0.4        0.16 ± 15%  perf-profile.children.cycles-pp.__cond_resched
      0.57 ± 58%      -0.4        0.20 ± 30%  perf-profile.children.cycles-pp.page_add_file_rmap
      0.51 ± 53%      -0.4        0.13 ± 37%  perf-profile.children.cycles-pp.terminate_walk
      0.58 ± 23%      -0.4        0.23 ± 30%  perf-profile.children.cycles-pp.page_remove_rmap
      0.55 ±  9%      -0.4        0.20 ± 17%  perf-profile.children.cycles-pp.time
      0.59 ± 17%      -0.3        0.24 ± 15%  perf-profile.children.cycles-pp.simple_lookup
      0.48 ± 57%      -0.3        0.13 ± 26%  perf-profile.children.cycles-pp.__fput
      0.55 ± 17%      -0.3        0.21 ± 14%  perf-profile.children.cycles-pp.d_add
      0.70 ± 13%      -0.3        0.37 ±  9%  perf-profile.children.cycles-pp.lapic_next_deadline
      0.59 ± 23%      -0.3        0.26 ± 21%  perf-profile.children.cycles-pp.lookup_fast
      0.44 ± 63%      -0.3        0.11 ± 45%  perf-profile.children.cycles-pp.vm_area_alloc
      0.45 ± 78%      -0.3        0.12 ± 17%  perf-profile.children.cycles-pp.vma_interval_tree_insert
      0.51 ± 25%      -0.3        0.19 ± 29%  perf-profile.children.cycles-pp.find_vma
      0.53 ± 23%      -0.3        0.21 ± 26%  perf-profile.children.cycles-pp.sync_regs
      0.40 ± 96%      -0.3        0.09 ± 45%  perf-profile.children.cycles-pp.__memcg_kmem_charge
      0.49 ± 25%      -0.3        0.17 ± 20%  perf-profile.children.cycles-pp._IO_vfscanf
      0.42 ± 90%      -0.3        0.10 ± 10%  perf-profile.children.cycles-pp.__do_fault
      0.47 ± 25%      -0.3        0.16 ± 23%  perf-profile.children.cycles-pp.count
      0.48 ± 22%      -0.3        0.18 ± 34%  perf-profile.children.cycles-pp.rcu_sched_clock_irq
      0.40 ± 27%      -0.3        0.10 ± 45%  perf-profile.children.cycles-pp.rcu_idle_exit
      0.41 ± 12%      -0.3        0.11 ± 18%  perf-profile.children.cycles-pp.__get_user_8
      0.50 ± 27%      -0.3        0.20 ± 12%  perf-profile.children.cycles-pp.vma_link
      0.45 ± 18%      -0.3        0.15 ± 22%  perf-profile.children.cycles-pp.unlock_page
      0.51 ± 19%      -0.3        0.23 ± 16%  perf-profile.children.cycles-pp.load_balance
      0.44 ± 23%      -0.3        0.17 ± 36%  perf-profile.children.cycles-pp.__might_fault
      0.41 ± 93%      -0.3        0.14 ± 33%  perf-profile.children.cycles-pp.pte_alloc_one
      0.46 ± 20%      -0.3        0.19 ± 21%  perf-profile.children.cycles-pp.perf_iterate_sb
      0.47 ± 19%      -0.3        0.20 ± 39%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      0.41 ± 31%      -0.3        0.14 ± 48%  perf-profile.children.cycles-pp.setup_arg_pages
      0.53 ± 24%      -0.3        0.26 ± 49%  perf-profile.children.cycles-pp.__dentry_kill
      0.33 ±104%      -0.3        0.06 ± 48%  perf-profile.children.cycles-pp.rcu_all_qs
      0.37 ± 12%      -0.3        0.10 ± 61%  perf-profile.children.cycles-pp.__perf_sw_event
      0.49 ± 29%      -0.3        0.23 ± 11%  perf-profile.children.cycles-pp.get_arg_page
      0.51 ± 27%      -0.3        0.25 ± 10%  perf-profile.children.cycles-pp.get_page_from_freelist
      0.48 ± 27%      -0.3        0.22 ± 11%  perf-profile.children.cycles-pp.__get_user_pages
      0.42 ± 23%      -0.3        0.15 ± 32%  perf-profile.children.cycles-pp.__might_sleep
      0.49 ± 28%      -0.3        0.22 ±  9%  perf-profile.children.cycles-pp.__get_user_pages_remote
      0.32 ±107%      -0.3        0.06 ± 45%  perf-profile.children.cycles-pp.filemap_read
      0.40 ± 30%      -0.3        0.15 ± 17%  perf-profile.children.cycles-pp.get_obj_cgroup_from_current
      0.39 ± 30%      -0.2        0.14 ± 29%  perf-profile.children.cycles-pp.alloc_pages_vma
      0.30 ± 41%      -0.2        0.06 ± 80%  perf-profile.children.cycles-pp.update_ts_time_stats
      0.30 ±114%      -0.2        0.06 ± 50%  perf-profile.children.cycles-pp.__kernel_read
      0.31 ± 22%      -0.2        0.07 ± 48%  perf-profile.children.cycles-pp.rcu_eqs_exit
      0.33 ± 75%      -0.2        0.09 ± 31%  perf-profile.children.cycles-pp.alloc_bprm
      0.36 ± 22%      -0.2        0.12 ± 26%  perf-profile.children.cycles-pp.call_rcu
      0.57 ± 21%      -0.2        0.33 ± 11%  perf-profile.children.cycles-pp.worker_thread
      0.47 ± 28%      -0.2        0.23 ± 18%  perf-profile.children.cycles-pp._copy_from_user
      0.39 ± 20%      -0.2        0.15 ± 23%  perf-profile.children.cycles-pp.try_to_unlazy
      0.56 ± 19%      -0.2        0.33 ± 10%  perf-profile.children.cycles-pp.process_one_work
      0.48 ± 22%      -0.2        0.26 ± 44%  perf-profile.children.cycles-pp.__check_object_size
      0.28 ± 22%      -0.2        0.07 ± 54%  perf-profile.children.cycles-pp.nr_iowait_cpu
      0.34 ± 27%      -0.2        0.12 ± 25%  perf-profile.children.cycles-pp.execl
      0.54 ± 17%      -0.2        0.33 ± 10%  perf-profile.children.cycles-pp.drm_fb_helper_damage_work
      0.54 ± 18%      -0.2        0.33 ± 10%  perf-profile.children.cycles-pp.memcpy_toio
      0.26 ± 26%      -0.2        0.05 ± 78%  perf-profile.children.cycles-pp.flush_tlb_func_common
      0.31 ± 29%      -0.2        0.10 ± 69%  perf-profile.children.cycles-pp.shift_arg_pages
      0.34 ± 24%      -0.2        0.14 ± 37%  perf-profile.children.cycles-pp.__x64_sys_munmap
      0.32 ± 22%      -0.2        0.12 ± 23%  perf-profile.children.cycles-pp.__legitimize_path
      0.33 ± 35%      -0.2        0.12 ± 31%  perf-profile.children.cycles-pp.__hrtimer_next_event_base
      0.34 ± 21%      -0.2        0.14 ± 32%  perf-profile.children.cycles-pp.update_rq_clock
      0.26 ± 17%      -0.2        0.06 ± 80%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      0.32 ± 26%      -0.2        0.12 ± 10%  perf-profile.children.cycles-pp.copy_string_kernel
      0.30 ± 26%      -0.2        0.11 ± 32%  perf-profile.children.cycles-pp.hrtimer_next_event_without
      0.32 ± 33%      -0.2        0.13 ± 22%  perf-profile.children.cycles-pp.user_path_at_empty
      0.32 ± 22%      -0.2        0.13 ± 26%  perf-profile.children.cycles-pp.unlink_anon_vmas
      0.26 ± 33%      -0.2        0.08 ± 50%  perf-profile.children.cycles-pp.__isoc99_sscanf
      0.34 ± 14%      -0.2        0.15 ± 22%  perf-profile.children.cycles-pp.find_busiest_group
      0.30 ± 32%      -0.2        0.12 ± 15%  perf-profile.children.cycles-pp.__anon_vma_prepare
      0.24 ± 23%      -0.2        0.06 ± 52%  perf-profile.children.cycles-pp.__put_user_nocheck_8
      0.34 ± 14%      -0.2        0.17 ± 11%  perf-profile.children.cycles-pp.copy_user_generic_unrolled
      0.25 ± 26%      -0.2        0.08 ± 24%  perf-profile.children.cycles-pp.__cxa_atexit
      0.22 ± 37%      -0.2        0.05 ± 79%  perf-profile.children.cycles-pp.map_vdso
      0.25 ± 27%      -0.2        0.08 ± 17%  perf-profile.children.cycles-pp.get_unmapped_area
      0.30 ± 27%      -0.2        0.13 ± 18%  perf-profile.children.cycles-pp.run_rebalance_domains
      0.30 ± 28%      -0.2        0.13 ± 21%  perf-profile.children.cycles-pp.lru_add_drain
      0.29 ± 26%      -0.2        0.12 ± 15%  perf-profile.children.cycles-pp.update_blocked_averages
      0.28 ± 33%      -0.2        0.12 ± 53%  perf-profile.children.cycles-pp.memmove
      0.23 ± 23%      -0.2        0.06 ± 78%  perf-profile.children.cycles-pp.___perf_sw_event
      0.25 ± 14%      -0.2        0.09 ± 33%  perf-profile.children.cycles-pp.perf_event_mmap_output
      0.21 ± 10%      -0.2        0.05 ± 76%  perf-profile.children.cycles-pp.memcpy_erms
      0.28 ± 33%      -0.2        0.12 ± 24%  perf-profile.children.cycles-pp.__pagevec_lru_add
      0.28 ± 25%      -0.2        0.13 ± 17%  perf-profile.children.cycles-pp.get_next_timer_interrupt
      0.29 ± 18%      -0.2        0.14 ± 22%  perf-profile.children.cycles-pp.update_sd_lb_stats
      0.22 ± 46%      -0.2        0.07 ± 47%  perf-profile.children.cycles-pp.__isoc99_vsscanf
      0.25 ± 51%      -0.2        0.10 ± 57%  perf-profile.children.cycles-pp.ksys_read
      0.29 ± 28%      -0.2        0.14 ± 21%  perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
      0.22 ± 37%      -0.2        0.07 ± 53%  perf-profile.children.cycles-pp._start
      0.20 ± 31%      -0.1        0.05 ± 78%  perf-profile.children.cycles-pp.note_gp_changes
      0.24 ± 45%      -0.1        0.09 ± 57%  perf-profile.children.cycles-pp.vfs_read
      0.18 ± 16%      -0.1        0.04 ± 75%  perf-profile.children.cycles-pp.do_brk_flags
      0.19 ± 22%      -0.1        0.06 ± 50%  perf-profile.children.cycles-pp.vm_brk_flags
      0.24 ± 15%      -0.1        0.11 ± 22%  perf-profile.children.cycles-pp.__d_lookup_done
      0.16 ± 41%      -0.1        0.03 ±100%  perf-profile.children.cycles-pp.arch_get_unmapped_area_topdown
      0.19 ± 24%      -0.1        0.06 ± 50%  perf-profile.children.cycles-pp.page_counter_try_charge
      0.19 ± 35%      -0.1        0.06 ± 52%  perf-profile.children.cycles-pp.fput_many
      0.21 ± 50%      -0.1        0.08 ± 26%  perf-profile.children.cycles-pp.__rb_insert_augmented
      0.18 ± 19%      -0.1        0.06 ± 78%  perf-profile.children.cycles-pp.__d_lookup_rcu
      0.15 ± 42%      -0.1        0.03 ±102%  perf-profile.children.cycles-pp.up_read
      0.21 ± 29%      -0.1        0.09 ± 23%  perf-profile.children.cycles-pp.unlink_file_vma
      0.18 ± 26%      -0.1        0.06 ± 47%  perf-profile.children.cycles-pp.mark_page_accessed
      0.19 ± 43%      -0.1        0.07 ± 11%  perf-profile.children.cycles-pp.__vma_link_rb
      0.17 ± 22%      -0.1        0.05 ± 46%  perf-profile.children.cycles-pp.__check_heap_object
      0.18 ± 36%      -0.1        0.06 ± 32%  perf-profile.children.cycles-pp.vm_normal_page
      0.26 ± 35%      -0.1        0.14 ± 30%  perf-profile.children.cycles-pp.__intel_pmu_enable_all
      0.19 ± 31%      -0.1        0.07 ± 52%  perf-profile.children.cycles-pp.up_write
      0.17 ± 67%      -0.1        0.06 ± 73%  perf-profile.children.cycles-pp.enqueue_hrtimer
      0.24 ± 15%      -0.1        0.12 ± 19%  perf-profile.children.cycles-pp.arch_scale_freq_tick
      0.14 ± 38%      -0.1        0.03 ±100%  perf-profile.children.cycles-pp.__list_del_entry_valid
      0.17 ± 53%      -0.1        0.05 ± 53%  perf-profile.children.cycles-pp.generic_permission
      0.14 ± 36%      -0.1        0.03 ±103%  perf-profile.children.cycles-pp.xas_load
      0.20 ± 41%      -0.1        0.09 ± 33%  perf-profile.children.cycles-pp.__get_free_pages
      0.14 ± 23%      -0.1        0.03 ±104%  perf-profile.children.cycles-pp.lockref_get_not_dead
      0.18 ± 24%      -0.1        0.08 ± 25%  perf-profile.children.cycles-pp.vm_area_dup
      0.15 ± 41%      -0.1        0.05 ± 46%  perf-profile.children.cycles-pp.refill_obj_stock
      0.15 ± 29%      -0.1        0.05 ± 46%  perf-profile.children.cycles-pp.__legitimize_mnt
      0.14 ± 26%      -0.1        0.05 ± 48%  perf-profile.children.cycles-pp.call_cpuidle
      0.16 ± 33%      -0.1        0.07 ± 20%  perf-profile.children.cycles-pp.__hrtimer_get_next_event
      0.18 ± 14%      -0.1        0.09 ± 50%  perf-profile.children.cycles-pp.__pte_alloc
      0.16 ± 26%      -0.1        0.07 ± 30%  perf-profile.children.cycles-pp.__mod_memcg_state
      0.17 ± 23%      -0.1        0.08 ± 54%  perf-profile.children.cycles-pp.d_path
      0.15 ± 18%      -0.1        0.07 ± 54%  perf-profile.children.cycles-pp.finish_fault
      0.11 ± 27%      -0.1        0.03 ±100%  perf-profile.children.cycles-pp.down_read
      0.11 ± 27%      -0.1        0.03 ±101%  perf-profile.children.cycles-pp.__slab_alloc
      0.10 ± 27%      -0.1        0.03 ±102%  perf-profile.children.cycles-pp.___slab_alloc
      0.10 ± 15%      -0.1        0.03 ±103%  perf-profile.children.cycles-pp.expand_downwards
      0.12 ± 19%      -0.1        0.05 ± 48%  perf-profile.children.cycles-pp.io_serial_out
      0.16 ±  9%      -0.1        0.10 ± 28%  perf-profile.children.cycles-pp.do_open
      0.02 ±142%      +0.2        0.18 ± 30%  perf-profile.children.cycles-pp.update_load_avg
      0.01 ±223%      +0.2        0.18 ± 22%  perf-profile.children.cycles-pp.update_curr
      0.05 ± 74%      +0.5        0.53 ± 20%  perf-profile.children.cycles-pp.task_tick_fair
      0.00            +0.7        0.73 ±  9%  perf-profile.children.cycles-pp.sched_clock_local
      0.74 ± 21%      +2.4        3.14 ±  7%  perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
      2.67 ± 12%      +4.7        7.33 ±  8%  perf-profile.children.cycles-pp.tick_sched_handle
      2.50 ±  9%      +4.8        7.29 ±  8%  perf-profile.children.cycles-pp.update_process_times
      1.32 ±  8%      +5.5        6.79 ±  8%  perf-profile.children.cycles-pp.scheduler_tick
      0.70 ± 54%      +6.8        7.46 ±  4%  perf-profile.children.cycles-pp.tick_nohz_irq_exit
      1.84 ± 19%      +8.0        9.79 ±  3%  perf-profile.children.cycles-pp.clockevents_program_event
      0.40 ± 27%      +8.3        8.72 ±  2%  perf-profile.children.cycles-pp.ktime_get_update_offsets_now
      3.09 ± 10%      +9.1       12.20 ±  5%  perf-profile.children.cycles-pp.tick_sched_timer
      4.60 ± 16%     +11.1       15.70 ±  5%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.01 ±223%     +15.3       15.26 ±  3%  perf-profile.children.cycles-pp.sched_clock_idle_wakeup_event
      1.24 ± 22%     +15.8       17.00 ±  2%  perf-profile.children.cycles-pp.irq_enter_rcu
      1.22 ± 24%     +15.8       16.99 ±  2%  perf-profile.children.cycles-pp.tick_irq_enter
      0.00           +20.7       20.69 ±  3%  perf-profile.children.cycles-pp.sched_clock_tick
     59.83 ±  5%     +23.6       83.45        perf-profile.children.cycles-pp.start_secondary
     60.74 ±  5%     +23.7       84.46        perf-profile.children.cycles-pp.secondary_startup_64_no_verify
     60.74 ±  5%     +23.7       84.46        perf-profile.children.cycles-pp.cpu_startup_entry
     60.74 ±  5%     +23.7       84.46        perf-profile.children.cycles-pp.do_idle
      7.66 ± 11%     +27.0       34.62 ±  2%  perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
      7.53 ± 11%     +27.0       34.58 ±  2%  perf-profile.children.cycles-pp.hrtimer_interrupt
     55.45 ±  6%     +27.5       82.93        perf-profile.children.cycles-pp.cpuidle_enter_state
     55.45 ±  6%     +27.5       82.95        perf-profile.children.cycles-pp.cpuidle_enter
     14.73 ±  7%     +46.3       61.01 ±  2%  perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
     12.64 ±  6%     +47.6       60.23 ±  2%  perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
      2.77 ± 29%     +50.5       53.26 ±  2%  perf-profile.children.cycles-pp.ktime_get
      0.00           +60.5       60.51        perf-profile.children.cycles-pp.read_hpet
     38.06 ± 13%     -22.4       15.66 ± 11%  perf-profile.self.cycles-pp.intel_idle
      3.26 ± 35%      -2.2        1.02 ± 10%  perf-profile.self.cycles-pp.cpuidle_enter_state
      2.08 ± 27%      -1.4        0.65 ± 25%  perf-profile.self.cycles-pp.menu_select
      1.45 ±  2%      -1.0        0.44 ± 14%  perf-profile.self.cycles-pp.next_uptodate_page
      1.49 ± 14%      -0.9        0.59 ± 15%  perf-profile.self.cycles-pp.native_irq_return_iret
      1.93 ± 20%      -0.9        1.08 ±  8%  perf-profile.self.cycles-pp.io_serial_in
      1.18 ± 10%      -0.8        0.40 ± 14%  perf-profile.self.cycles-pp._raw_spin_lock
      1.08 ± 14%      -0.7        0.35 ± 17%  perf-profile.self.cycles-pp.strnlen_user
      1.03 ±  8%      -0.7        0.31 ± 16%  perf-profile.self.cycles-pp.zap_pte_range
      0.85 ± 15%      -0.7        0.19 ± 18%  perf-profile.self.cycles-pp.kmem_cache_free
      1.01 ± 15%      -0.6        0.36 ±  7%  perf-profile.self.cycles-pp.find_idlest_group
      0.86 ± 18%      -0.6        0.26 ± 26%  perf-profile.self.cycles-pp.kmem_cache_alloc
      0.88 ± 20%      -0.6        0.29 ± 11%  perf-profile.self.cycles-pp.___might_sleep
      0.49 ± 35%      -0.4        0.12 ± 17%  perf-profile.self.cycles-pp.irqtime_account_irq
      0.51 ± 22%      -0.4        0.16 ± 26%  perf-profile.self.cycles-pp.release_pages
      0.70 ± 14%      -0.3        0.37 ±  9%  perf-profile.self.cycles-pp.lapic_next_deadline
      0.52 ± 25%      -0.3        0.21 ± 25%  perf-profile.self.cycles-pp.sync_regs
      0.42 ± 19%      -0.3        0.12 ± 31%  perf-profile.self.cycles-pp.unlock_page
      0.39 ±  9%      -0.3        0.09 ± 23%  perf-profile.self.cycles-pp.__get_user_8
      0.36 ± 32%      -0.3        0.09 ± 19%  perf-profile.self.cycles-pp.tick_nohz_next_event
      0.46 ± 20%      -0.3        0.19 ± 39%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.42 ± 26%      -0.3        0.15 ± 11%  perf-profile.self.cycles-pp.filemap_map_pages
      0.32 ± 26%      -0.3        0.06 ± 50%  perf-profile.self.cycles-pp.update_process_times
      0.39 ± 32%      -0.2        0.14 ± 39%  perf-profile.self.cycles-pp.page_remove_rmap
      0.37 ± 16%      -0.2        0.13 ± 34%  perf-profile.self.cycles-pp.__might_sleep
      0.39 ± 33%      -0.2        0.16 ± 34%  perf-profile.self.cycles-pp.rcu_sched_clock_irq
      0.30 ± 24%      -0.2        0.08 ± 22%  perf-profile.self.cycles-pp.find_vma
      0.54 ± 18%      -0.2        0.32 ±  9%  perf-profile.self.cycles-pp.memcpy_toio
      0.28 ± 24%      -0.2        0.06 ± 58%  perf-profile.self.cycles-pp.nr_iowait_cpu
      0.32 ± 45%      -0.2        0.10 ± 15%  perf-profile.self.cycles-pp.vma_interval_tree_insert
      0.32 ± 33%      -0.2        0.11 ± 35%  perf-profile.self.cycles-pp.page_add_file_rmap
      0.32 ± 27%      -0.2        0.11 ± 23%  perf-profile.self.cycles-pp.ktime_get_update_offsets_now
      0.26 ± 17%      -0.2        0.06 ± 80%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      0.30 ± 31%      -0.2        0.11 ± 16%  perf-profile.self.cycles-pp.get_obj_cgroup_from_current
      0.25 ± 18%      -0.2        0.06 ± 57%  perf-profile.self.cycles-pp.call_rcu
      0.26 ± 28%      -0.2        0.08 ± 50%  perf-profile.self.cycles-pp.strncpy_from_user
      0.21 ±  7%      -0.2        0.03 ±100%  perf-profile.self.cycles-pp.memcpy_erms
      0.32 ± 37%      -0.2        0.15 ± 25%  perf-profile.self.cycles-pp.do_idle
      0.21 ± 26%      -0.2        0.05 ± 54%  perf-profile.self.cycles-pp.__put_user_nocheck_8
      0.20 ± 18%      -0.2        0.04 ± 73%  perf-profile.self.cycles-pp.perf_event_mmap
      0.20 ± 31%      -0.2        0.05 ± 79%  perf-profile.self.cycles-pp.copy_strings
      0.19 ± 27%      -0.2        0.04 ± 72%  perf-profile.self.cycles-pp.rcu_eqs_exit
      0.26 ± 41%      -0.1        0.11 ± 29%  perf-profile.self.cycles-pp.__hrtimer_next_event_base
      0.26 ± 15%      -0.1        0.11 ±  8%  perf-profile.self.cycles-pp.copy_user_generic_unrolled
      0.19 ± 42%      -0.1        0.05 ±101%  perf-profile.self.cycles-pp.copy_page
      0.22 ± 33%      -0.1        0.08 ± 61%  perf-profile.self.cycles-pp.__hrtimer_run_queues
      0.25 ± 27%      -0.1        0.11 ± 27%  perf-profile.self.cycles-pp.update_sd_lb_stats
      0.18 ± 19%      -0.1        0.05 ±102%  perf-profile.self.cycles-pp.__d_lookup_rcu
      0.19 ± 39%      -0.1        0.05 ± 48%  perf-profile.self.cycles-pp.__vma_link_rb
      0.16 ± 31%      -0.1        0.03 ± 99%  perf-profile.self.cycles-pp._IO_vfscanf
      0.17 ± 23%      -0.1        0.03 ±108%  perf-profile.self.cycles-pp.___perf_sw_event
      0.20 ± 29%      -0.1        0.07 ± 18%  perf-profile.self.cycles-pp.__cond_resched
      0.17 ± 22%      -0.1        0.04 ± 71%  perf-profile.self.cycles-pp.__check_heap_object
      0.18 ± 30%      -0.1        0.05 ± 52%  perf-profile.self.cycles-pp.page_counter_try_charge
      0.16 ± 33%      -0.1        0.03 ±105%  perf-profile.self.cycles-pp.vm_normal_page
      0.15 ± 29%      -0.1        0.03 ±100%  perf-profile.self.cycles-pp.__legitimize_mnt
      0.15 ± 26%      -0.1        0.03 ±100%  perf-profile.self.cycles-pp.mark_page_accessed
      0.26 ± 35%      -0.1        0.14 ± 30%  perf-profile.self.cycles-pp.__intel_pmu_enable_all
      0.24 ± 15%      -0.1        0.12 ± 19%  perf-profile.self.cycles-pp.arch_scale_freq_tick
      0.18 ± 44%      -0.1        0.06 ± 60%  perf-profile.self.cycles-pp.rcu_read_unlock_strict
      0.17 ± 42%      -0.1        0.06 ± 48%  perf-profile.self.cycles-pp.__rb_insert_augmented
      0.14 ± 23%      -0.1        0.03 ±102%  perf-profile.self.cycles-pp.lockref_get_not_dead
      0.17 ± 32%      -0.1        0.06 ± 54%  perf-profile.self.cycles-pp.up_write
      0.14 ± 41%      -0.1        0.03 ±108%  perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
      0.13 ± 39%      -0.1        0.03 ± 99%  perf-profile.self.cycles-pp.__list_del_entry_valid
      0.16 ± 26%      -0.1        0.06 ± 87%  perf-profile.self.cycles-pp.vmacache_find
      0.14 ± 27%      -0.1        0.04 ± 75%  perf-profile.self.cycles-pp.call_cpuidle
      0.20 ± 30%      -0.1        0.11 ± 24%  perf-profile.self.cycles-pp.do_set_pte
      0.13 ± 36%      -0.1        0.04 ± 71%  perf-profile.self.cycles-pp.rcu_all_qs
      0.15 ± 30%      -0.1        0.06 ± 54%  perf-profile.self.cycles-pp.__mod_memcg_state
      0.13 ± 44%      -0.1        0.04 ± 75%  perf-profile.self.cycles-pp.update_rq_clock
      0.15 ± 30%      -0.1        0.06 ± 55%  perf-profile.self.cycles-pp.__check_object_size
      0.12 ± 38%      -0.1        0.05 ± 83%  perf-profile.self.cycles-pp.asm_sysvec_apic_timer_interrupt
      0.11 ± 36%      -0.1        0.03 ±103%  perf-profile.self.cycles-pp.__hrtimer_get_next_event
      0.14 ± 29%      -0.1        0.07 ± 41%  perf-profile.self.cycles-pp.handle_mm_fault
      0.12 ± 19%      -0.1        0.05 ± 48%  perf-profile.self.cycles-pp.io_serial_out
      0.00            +0.1        0.11 ± 22%  perf-profile.self.cycles-pp.update_curr
      0.00            +0.5        0.49 ± 11%  perf-profile.self.cycles-pp.sched_clock_local
      0.00           +60.5       60.51        perf-profile.self.cycles-pp.read_hpet



***************************************************************************************************
lkp-csl-2ap2: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
=========================================================================================
compiler/cpufreq_governor/disk/filesize/fs/iterations/kconfig/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase/ucode:
  gcc-9/performance/1BRD_48G/4M/btrfs/1x/x86_64-rhel-8.3/1t/debian-10.4-x86_64-20200603.cgz/NoSync/lkp-csl-2ap2/24G/fsmark/0x5003006

commit: 
  d42c7a8d68 ("kcsan: Document "value changed" line")
  6c52b5f3cf ("clocksource: Reduce WATCHDOG_THRESHOLD")

d42c7a8d68038bc0 6c52b5f3cfefd6e429efc4413fd 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    264577         +1732.3%    4847771        fsmark.app_overhead
    399.03           -37.8%     248.35 ±  2%  fsmark.files_per_sec
     21.70           +34.5%      29.19 ±  3%  fsmark.time.elapsed_time
     21.70           +34.5%      29.19 ±  3%  fsmark.time.elapsed_time.max
     79.67           +15.1%      91.67        fsmark.time.percent_of_cpu_this_job_got
     17.21           +53.9%      26.49 ±  2%  fsmark.time.system_time
    169429 ±  2%     -86.9%      22149 ±  8%  fsmark.time.voluntary_context_switches
    126301 ± 75%     -98.4%       2017 ± 85%  cpuidle.POLL.usage
      5510 ±  6%     +17.1%       6452 ±  3%  boot-time.idle
      1.72            +2.4%       1.77        boot-time.smp_boot
     96.92            +2.0%      98.86        iostat.cpu.idle
      2.86 ±  8%     -65.9%       0.97 ±  5%  iostat.cpu.system
     48.83 ± 14%    +161.8%     127.83 ± 34%  turbostat.Avg_MHz
      4.70 ±  6%      +8.9       13.58 ± 14%  turbostat.Busy%
     62.27 ±  2%     +16.4%      72.51 ±  2%  uptime.boot
     10421 ±  2%     +24.8%      13004 ±  3%  uptime.idle
      0.05 ±  5%      -0.0        0.02 ± 23%  mpstat.cpu.all.iowait%
      1.26 ± 17%      -1.3        0.00        mpstat.cpu.all.irq%
      0.09 ± 11%      -0.1        0.00 ± 48%  mpstat.cpu.all.soft%
      7332 ± 62%    +122.5%      16311 ± 39%  numa-meminfo.node3.Active
      4212 ± 20%    +198.0%      12550 ± 24%  numa-meminfo.node3.Active(anon)
      7027 ± 48%    +159.9%      18263 ± 33%  numa-meminfo.node3.Shmem
      1052 ± 20%    +196.1%       3117 ± 24%  numa-vmstat.node3.nr_active_anon
      1756 ± 48%    +159.5%       4557 ± 34%  numa-vmstat.node3.nr_shmem
      1052 ± 20%    +196.1%       3117 ± 24%  numa-vmstat.node3.nr_zone_active_anon
      7855 ± 11%     +39.8%      10981 ±  9%  softirqs.CPU0.SCHED
      5753 ± 13%     +37.2%       7895 ± 27%  softirqs.CPU1.SCHED
    970575 ±  5%     +20.3%    1167382 ±  3%  softirqs.SCHED
     96.67            +1.4%      98.00        vmstat.cpu.id
   1008185           -23.6%     770151 ±  5%  vmstat.io.bo
     18059 ±  2%     -74.6%       4595        vmstat.system.cs
      0.01 ± 14%     +51.7%       0.02 ± 11%  perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select
      0.01 ± 14%     +51.7%       0.02 ± 11%  perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select
      3.00 ± 12%     +54.8%       4.64 ± 16%  perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
      2.53 ± 47%     +83.5%       4.64 ± 16%  perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork
     16335 ±  6%     +44.7%      23644 ± 12%  meminfo.Active
      7042 ± 15%    +103.1%      14304 ± 20%  meminfo.Active(anon)
  13816046           -10.7%   12342039 ±  3%  meminfo.Inactive(file)
     26979 ±  4%     +31.6%      35497 ±  9%  meminfo.Shmem
   2269754 ±  3%     -73.2%     607860 ±  4%  meminfo.Writeback
      1940 ± 26%     -45.2%       1063 ± 41%  sched_debug.cpu.avg_idle.min
     32619 ±  3%     +28.6%      41932 ±  2%  sched_debug.cpu.clock_task.min
    645.89 ± 20%     -97.9%      13.74 ± 19%  sched_debug.cpu.clock_task.stddev
    165.42 ± 15%     -24.1%     125.63 ± 13%  sched_debug.cpu.curr->pid.avg
    720.17 ±  7%     -12.9%     627.50 ±  6%  sched_debug.cpu.curr->pid.stddev
      1.00          -100.0%       0.00        sched_debug.sched_clock_stable()
    657.83 ±  3%     -58.6%     272.50 ±  7%  slabinfo.biovec-max.active_objs
    684.00 ±  3%     -54.9%     308.50 ±  8%  slabinfo.biovec-max.num_objs
      4867           -13.7%       4202 ±  5%  slabinfo.btrfs_extent_map.active_objs
      4982           -13.7%       4301 ±  4%  slabinfo.btrfs_extent_map.num_objs
      3531           -10.2%       3173 ±  2%  slabinfo.btrfs_inode.active_objs
      3546           -10.0%       3191 ±  2%  slabinfo.btrfs_inode.num_objs
      6938 ±  5%     -11.2%       6158 ±  7%  slabinfo.files_cache.active_objs
      6938 ±  5%     -11.2%       6158 ±  7%  slabinfo.files_cache.num_objs
      2146           -18.1%       1757        slabinfo.kmalloc-8k.active_objs
      2164           -17.8%       1779        slabinfo.kmalloc-8k.num_objs
      1753 ± 15%    +104.0%       3575 ± 20%  proc-vmstat.nr_active_anon
   3742994           -10.4%    3351949 ±  2%  proc-vmstat.nr_file_pages
  42736503            +1.0%   43175533        proc-vmstat.nr_free_pages
   3474846           -11.3%    3081667 ±  3%  proc-vmstat.nr_inactive_file
      6738 ±  4%     +31.7%       8874 ±  9%  proc-vmstat.nr_shmem
     44643            -3.1%      43267        proc-vmstat.nr_slab_reclaimable
     77603            -3.7%      74734        proc-vmstat.nr_slab_unreclaimable
    570405 ±  3%     -73.4%     151983 ±  4%  proc-vmstat.nr_writeback
      1753 ± 15%    +104.0%       3575 ± 20%  proc-vmstat.nr_zone_active_anon
   3474846           -11.3%    3081667 ±  3%  proc-vmstat.nr_zone_inactive_file
   1652814           -22.3%    1284857 ±  4%  proc-vmstat.nr_zone_write_pending
   1359793           -61.0%     530557 ± 22%  proc-vmstat.numa_foreign
  11732111            +7.3%   12586943        proc-vmstat.numa_hit
  11472411            +7.5%   12327232        proc-vmstat.numa_local
   1359793           -61.0%     530557 ± 22%  proc-vmstat.numa_miss
   1619493           -51.2%     790267 ± 14%  proc-vmstat.numa_other
      3192 ±  5%     +87.0%       5969 ± 13%  proc-vmstat.pgactivate
    126035 ±  2%     +27.4%     160516 ±  4%  proc-vmstat.pgfault
      6459 ±  3%     +27.7%       8247 ±  6%  proc-vmstat.pgreuse
 1.513e+09           -11.7%  1.337e+09 ±  3%  perf-stat.i.branch-instructions
  48492068 ±  3%     -23.9%   36917714 ±  5%  perf-stat.i.cache-misses
     19647 ±  3%     -77.1%       4504        perf-stat.i.context-switches
      2.03 ±  8%    +237.1%       6.83 ± 18%  perf-stat.i.cpi
 1.402e+10 ±  8%    +200.4%  4.212e+10 ± 18%  perf-stat.i.cpu-cycles
    259.23 ±  2%      -8.6%     237.01        perf-stat.i.cpu-migrations
    824.65 ± 15%    +329.0%       3537 ± 24%  perf-stat.i.cycles-between-cache-misses
 1.929e+09 ±  2%     -14.1%  1.658e+09 ±  4%  perf-stat.i.dTLB-loads
 8.938e+08           -19.1%  7.228e+08 ±  3%  perf-stat.i.dTLB-stores
  7.72e+09           -13.0%  6.719e+09 ±  3%  perf-stat.i.instructions
      0.54 ±  8%     -69.7%       0.16 ± 25%  perf-stat.i.ipc
      9.44 ±  6%     -26.6%       6.93 ± 10%  perf-stat.i.major-faults
      0.07 ±  8%    +200.1%       0.22 ± 18%  perf-stat.i.metric.GHz
     23.21           -14.5%      19.84 ±  2%  perf-stat.i.metric.M/sec
      4560            -2.9%       4426        perf-stat.i.minor-faults
  11667068 ±  4%     -16.9%    9696044 ± 16%  perf-stat.i.node-loads
   2789878 ±  8%     -13.9%    2401011 ±  5%  perf-stat.i.node-stores
      4569            -3.0%       4433        perf-stat.i.page-faults
      1.82 ±  8%    +244.9%       6.26 ± 17%  perf-stat.overall.cpi
    289.37 ±  6%    +301.9%       1162 ± 21%  perf-stat.overall.cycles-between-cache-misses
      0.55 ±  8%     -69.9%       0.17 ± 25%  perf-stat.overall.ipc
 1.447e+09           -10.8%  1.291e+09 ±  3%  perf-stat.ps.branch-instructions
  46269702 ±  3%     -23.4%   35462108 ±  6%  perf-stat.ps.cache-misses
     18723 ±  3%     -76.8%       4334        perf-stat.ps.context-switches
    183348            +1.3%     185661        perf-stat.ps.cpu-clock
  1.34e+10 ±  8%    +204.0%  4.075e+10 ± 18%  perf-stat.ps.cpu-cycles
    247.92 ±  2%      -7.2%     230.01        perf-stat.ps.cpu-migrations
 1.844e+09 ±  2%     -13.2%    1.6e+09 ±  4%  perf-stat.ps.dTLB-loads
 8.542e+08           -18.3%  6.977e+08 ±  3%  perf-stat.ps.dTLB-stores
 7.381e+09           -12.1%   6.49e+09 ±  2%  perf-stat.ps.instructions
      9.01 ±  6%     -26.3%       6.64 ± 10%  perf-stat.ps.major-faults
  11128894 ±  4%     -16.4%    9301546 ± 16%  perf-stat.ps.node-loads
    183348            +1.3%     185661        perf-stat.ps.task-clock
 1.666e+11           +20.4%  2.006e+11 ±  6%  perf-stat.total.instructions
     36.10 ±  5%     -23.5       12.58 ± 30%  perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
     19.48 ± 10%     -15.3        4.21 ± 15%  perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
     23.57 ±  6%     -15.2        8.42 ± 11%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
     23.46 ±  6%     -15.1        8.38 ± 11%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
     18.86 ± 10%     -14.7        4.12 ±  9%  perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
     18.79 ± 10%     -14.7        4.11 ±  9%  perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
     18.66 ± 10%     -14.6        4.08 ±  9%  perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
     18.61 ± 10%     -14.5        4.07 ±  9%  perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write
     10.22 ±  5%      -8.0        2.24 ± 16%  perf-profile.calltrace.cycles-pp.ret_from_fork
     10.22 ±  5%      -8.0        2.24 ± 16%  perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
     10.13 ±  5%      -7.9        2.20 ± 15%  perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
     10.11 ±  5%      -7.9        2.19 ± 16%  perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
      6.84 ± 10%      -5.4        1.47 ±  7%  perf-profile.calltrace.cycles-pp.btrfs_copy_from_user.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write
      6.62 ± 10%      -5.2        1.43 ±  7%  perf-profile.calltrace.cycles-pp.iov_iter_copy_from_user_atomic.btrfs_copy_from_user.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write
      6.46 ± 10%      -5.1        1.38 ±  7%  perf-profile.calltrace.cycles-pp.copyin.iov_iter_copy_from_user_atomic.btrfs_copy_from_user.btrfs_buffered_write.btrfs_file_write_iter
      6.40 ± 10%      -5.0        1.37 ±  7%  perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.iov_iter_copy_from_user_atomic.btrfs_copy_from_user.btrfs_buffered_write
      6.03 ±  7%      -4.7        1.32 ± 20%  perf-profile.calltrace.cycles-pp.btrfs_work_helper.process_one_work.worker_thread.kthread.ret_from_fork
      5.84 ±  7%      -4.6        1.29 ± 21%  perf-profile.calltrace.cycles-pp.brd_submit_bio.submit_bio_noacct.submit_bio.btrfs_map_bio.run_one_async_done
      5.84 ±  7%      -4.6        1.29 ± 21%  perf-profile.calltrace.cycles-pp.submit_bio.btrfs_map_bio.run_one_async_done.btrfs_work_helper.process_one_work
      5.84 ±  7%      -4.6        1.29 ± 21%  perf-profile.calltrace.cycles-pp.submit_bio_noacct.submit_bio.btrfs_map_bio.run_one_async_done.btrfs_work_helper
      5.84 ±  7%      -4.5        1.29 ± 21%  perf-profile.calltrace.cycles-pp.run_one_async_done.btrfs_work_helper.process_one_work.worker_thread.kthread
      5.84 ±  7%      -4.5        1.29 ± 21%  perf-profile.calltrace.cycles-pp.btrfs_map_bio.run_one_async_done.btrfs_work_helper.process_one_work.worker_thread
      5.81 ± 11%      -4.5        1.27 ± 15%  perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write
      5.65 ± 11%      -4.4        1.24 ± 15%  perf-profile.calltrace.cycles-pp.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write
      5.64 ± 11%      -4.4        1.24 ± 15%  perf-profile.calltrace.cycles-pp.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter
      5.52 ± 11%      -4.3        1.21 ± 15%  perf-profile.calltrace.cycles-pp.need_preemptive_reclaim.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write
      5.42 ± 11%      -4.2        1.19 ± 15%  perf-profile.calltrace.cycles-pp.__percpu_counter_sum.need_preemptive_reclaim.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata
      5.20 ±  8%      -4.1        1.15 ± 21%  perf-profile.calltrace.cycles-pp.brd_do_bvec.brd_submit_bio.submit_bio_noacct.submit_bio.btrfs_map_bio
      4.59 ± 19%      -3.6        0.97 ± 26%  perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
      3.20 ±  8%      -2.6        0.62 ± 44%  perf-profile.calltrace.cycles-pp.brd_insert_page.brd_do_bvec.brd_submit_bio.submit_bio_noacct.submit_bio
      3.01 ± 10%      -2.3        0.69 ±  8%  perf-profile.calltrace.cycles-pp.prepare_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write
      2.79 ±  9%      -2.1        0.64 ±  8%  perf-profile.calltrace.cycles-pp.pagecache_get_page.prepare_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write
      2.23 ± 11%      -1.6        0.65 ± 37%  perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      0.00            +2.3        2.28 ±  9%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get_real_ts64.__x64_sys_gettimeofday.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +2.6        2.55 ±  6%  perf-profile.calltrace.cycles-pp.ktime_get_real_ts64.__x64_sys_gettimeofday.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +2.6        2.60 ±  6%  perf-profile.calltrace.cycles-pp.__x64_sys_gettimeofday.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.82 ± 34%      +4.0        4.84 ± 10%  perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      2.15 ± 18%      +4.4        6.55 ±  8%  perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      2.04 ± 15%      +4.5        6.50 ±  7%  perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
      0.00            +4.5        4.52 ±  7%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt
      0.00            +4.6        4.56 ±  8%  perf-profile.calltrace.cycles-pp.ktime_get.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      0.99 ± 15%      +5.2        6.22 ±  5%  perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      0.00            +5.4        5.40 ±  3%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
      0.10 ±223%      +5.5        5.55 ±  4%  perf-profile.calltrace.cycles-pp.ktime_get.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      0.00            +5.6        5.60 ±  3%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.sched_clock_tick.scheduler_tick.update_process_times
      0.00            +5.8        5.82 ±  3%  perf-profile.calltrace.cycles-pp.ktime_get.sched_clock_tick.scheduler_tick.update_process_times.tick_sched_handle
      0.00            +5.8        5.84 ±  3%  perf-profile.calltrace.cycles-pp.sched_clock_tick.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer
      0.00            +7.2        7.23 ± 16%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.cpuidle_enter_state
      0.45 ± 72%      +7.3        7.73 ± 14%  perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      0.46 ± 72%      +7.3        7.75 ± 14%  perf-profile.calltrace.cycles-pp.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      0.00            +7.3        7.33 ± 15%  perf-profile.calltrace.cycles-pp.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.cpuidle_enter_state.cpuidle_enter
      0.00            +7.4        7.37 ± 15%  perf-profile.calltrace.cycles-pp.sched_clock_tick.sched_clock_idle_wakeup_event.cpuidle_enter_state.cpuidle_enter.do_idle
      0.00            +7.5        7.49 ± 15%  perf-profile.calltrace.cycles-pp.sched_clock_idle_wakeup_event.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
      0.00            +7.6        7.58 ± 14%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      1.96 ± 28%      +7.6        9.59 ± 11%  perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      1.07 ± 56%      +8.2        9.25 ± 11%  perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      1.00 ± 29%      +8.3        9.26 ± 17%  perf-profile.calltrace.cycles-pp.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +9.0        8.98 ± 13%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.tick_irq_enter
      0.00            +9.0        9.02 ± 11%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      0.00            +9.1        9.10 ± 12%  perf-profile.calltrace.cycles-pp.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.tick_irq_enter.irq_enter_rcu
      0.41 ±106%      +9.1        9.53 ±  9%  perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +9.1        9.14 ± 12%  perf-profile.calltrace.cycles-pp.sched_clock_tick.sched_clock_idle_wakeup_event.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt
      0.00            +9.2        9.17 ± 17%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt
      0.00            +9.3        9.26 ± 12%  perf-profile.calltrace.cycles-pp.sched_clock_idle_wakeup_event.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.00            +9.4        9.38 ±  9%  perf-profile.calltrace.cycles-pp.read_hpet.ktime_get_update_offsets_now.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      2.58 ± 14%      +9.6       12.19 ±  6%  perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
      4.12 ± 18%     +13.3       17.42 ±  7%  perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      1.36 ± 23%     +17.3       18.64 ± 14%  perf-profile.calltrace.cycles-pp.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      1.33 ± 23%     +17.3       18.63 ± 14%  perf-profile.calltrace.cycles-pp.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
     60.63 ±  2%     +26.7       87.34        perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
     60.63 ±  2%     +26.7       87.34        perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     60.59 ±  2%     +26.7       87.33        perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     60.85 ±  2%     +27.0       87.88        perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
      7.59 ± 11%     +29.3       36.93        perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      7.43 ± 10%     +29.4       36.85        perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
     54.95 ±  2%     +31.0       85.98        perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     53.41 ±  3%     +32.0       85.40        perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
     17.19 ± 12%     +48.3       65.47 ±  5%  perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
     12.37 ± 10%     +51.9       64.29 ±  6%  perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
     36.20 ±  5%     -23.6       12.64 ± 30%  perf-profile.children.cycles-pp.intel_idle
     24.67 ±  5%     -15.7        8.99 ± 10%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
     24.56 ±  5%     -15.6        8.95 ± 10%  perf-profile.children.cycles-pp.do_syscall_64
     19.66 ± 10%     -15.2        4.49 ± 12%  perf-profile.children.cycles-pp.ksys_write
     19.60 ± 10%     -15.1        4.47 ± 12%  perf-profile.children.cycles-pp.vfs_write
     19.48 ± 10%     -15.0        4.44 ± 12%  perf-profile.children.cycles-pp.new_sync_write
     18.67 ± 10%     -14.6        4.08 ±  9%  perf-profile.children.cycles-pp.btrfs_file_write_iter
     18.62 ± 10%     -14.6        4.07 ±  9%  perf-profile.children.cycles-pp.btrfs_buffered_write
     10.22 ±  5%      -8.0        2.24 ± 16%  perf-profile.children.cycles-pp.ret_from_fork
     10.22 ±  5%      -8.0        2.24 ± 16%  perf-profile.children.cycles-pp.kthread
     10.13 ±  5%      -7.9        2.20 ± 15%  perf-profile.children.cycles-pp.worker_thread
     10.11 ±  5%      -7.9        2.19 ± 16%  perf-profile.children.cycles-pp.process_one_work
      6.84 ± 10%      -5.4        1.47 ±  7%  perf-profile.children.cycles-pp.btrfs_copy_from_user
      6.63 ± 10%      -5.2        1.43 ±  7%  perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic
      6.49 ± 10%      -5.1        1.39 ±  7%  perf-profile.children.cycles-pp.copyin
      6.44 ± 10%      -5.1        1.38 ±  7%  perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      6.03 ±  7%      -4.7        1.32 ± 20%  perf-profile.children.cycles-pp.btrfs_work_helper
      5.84 ±  7%      -4.6        1.29 ± 21%  perf-profile.children.cycles-pp.submit_bio
      5.84 ±  7%      -4.6        1.29 ± 21%  perf-profile.children.cycles-pp.submit_bio_noacct
      5.84 ±  7%      -4.6        1.29 ± 21%  perf-profile.children.cycles-pp.brd_submit_bio
      5.84 ±  7%      -4.5        1.29 ± 21%  perf-profile.children.cycles-pp.run_one_async_done
      5.84 ±  7%      -4.5        1.29 ± 21%  perf-profile.children.cycles-pp.btrfs_map_bio
      5.81 ± 11%      -4.5        1.27 ± 15%  perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata
      5.71 ± 11%      -4.5        1.25 ± 15%  perf-profile.children.cycles-pp.__reserve_bytes
      5.67 ± 11%      -4.4        1.24 ± 15%  perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes
      5.54 ± 11%      -4.3        1.22 ± 15%  perf-profile.children.cycles-pp.need_preemptive_reclaim
      5.46 ± 11%      -4.3        1.20 ± 15%  perf-profile.children.cycles-pp.__percpu_counter_sum
      5.21 ±  7%      -4.1        1.15 ± 21%  perf-profile.children.cycles-pp.brd_do_bvec
      4.63 ± 19%      -3.6        0.99 ± 26%  perf-profile.children.cycles-pp.menu_select
      3.09 ± 17%      -2.7        0.42 ±  6%  perf-profile.children.cycles-pp.__writeback_single_inode
      3.09 ± 17%      -2.7        0.42 ±  6%  perf-profile.children.cycles-pp.do_writepages
      3.09 ± 17%      -2.7        0.42 ±  6%  perf-profile.children.cycles-pp.extent_writepages
      3.07 ± 17%      -2.7        0.42 ±  6%  perf-profile.children.cycles-pp.extent_write_cache_pages
      3.22 ±  8%      -2.5        0.69 ± 19%  perf-profile.children.cycles-pp.brd_insert_page
      3.02 ± 10%      -2.3        0.70 ±  8%  perf-profile.children.cycles-pp.prepare_pages
      2.96 ±  7%      -2.3        0.68 ± 13%  perf-profile.children.cycles-pp._raw_spin_lock
      2.92 ±  9%      -2.3        0.66 ±  8%  perf-profile.children.cycles-pp.pagecache_get_page
      2.48 ± 17%      -2.1        0.34 ±  5%  perf-profile.children.cycles-pp.__extent_writepage
      2.31 ± 11%      -1.8        0.47 ± 22%  perf-profile.children.cycles-pp.cpumask_next
      2.36 ± 14%      -1.7        0.69 ± 35%  perf-profile.children.cycles-pp.irq_exit_rcu
      1.94 ± 11%      -1.5        0.40 ± 12%  perf-profile.children.cycles-pp.btrfs_dirty_pages
      1.78 ± 11%      -1.4        0.35 ± 30%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      1.60 ± 17%      -1.4        0.21 ±  7%  perf-profile.children.cycles-pp.__extent_writepage_io
      1.72 ±  9%      -1.3        0.40 ±  8%  perf-profile.children.cycles-pp.add_to_page_cache_lru
      1.45 ± 16%      -1.2        0.23 ± 28%  perf-profile.children.cycles-pp.tick_nohz_next_event
      1.74 ± 15%      -1.2        0.56 ± 41%  perf-profile.children.cycles-pp.__softirqentry_text_start
      1.55 ± 10%      -1.2        0.38 ± 14%  perf-profile.children.cycles-pp.__alloc_pages_nodemask
      1.37 ± 11%      -1.1        0.30 ± 24%  perf-profile.children.cycles-pp._find_next_bit
      1.34 ±  8%      -1.0        0.32 ± 15%  perf-profile.children.cycles-pp.get_page_from_freelist
      1.10 ± 10%      -0.8        0.26 ±  8%  perf-profile.children.cycles-pp.__add_to_page_cache_locked
      0.96 ± 28%      -0.8        0.14 ± 16%  perf-profile.children.cycles-pp.irqtime_account_irq
      1.11 ± 13%      -0.8        0.35 ± 34%  perf-profile.children.cycles-pp.rebalance_domains
      0.88 ± 11%      -0.7        0.18 ± 11%  perf-profile.children.cycles-pp.__set_page_dirty_nobuffers
      0.74 ± 17%      -0.6        0.10 ± 12%  perf-profile.children.cycles-pp.btrfs_set_range_writeback
      0.85 ± 25%      -0.6        0.24 ±  6%  perf-profile.children.cycles-pp.native_irq_return_iret
      0.69 ± 17%      -0.6        0.10 ±  6%  perf-profile.children.cycles-pp.writepage_delalloc
      0.69 ±  7%      -0.5        0.16 ± 14%  perf-profile.children.cycles-pp.rmqueue
      0.96 ± 16%      -0.5        0.44 ± 35%  perf-profile.children.cycles-pp.memcpy_toio
      0.96 ± 16%      -0.5        0.44 ± 34%  perf-profile.children.cycles-pp.drm_fb_helper_damage_work
      0.62 ±  8%      -0.5        0.14 ± 24%  perf-profile.children.cycles-pp.end_bio_extent_writepage
      0.62 ±  8%      -0.5        0.14 ± 21%  perf-profile.children.cycles-pp.btrfs_end_bio
      0.54 ± 17%      -0.5        0.07 ± 14%  perf-profile.children.cycles-pp.__test_set_page_writeback
      0.59 ±  9%      -0.5        0.14 ± 12%  perf-profile.children.cycles-pp.lru_cache_add
      0.77 ± 19%      -0.5        0.32 ± 10%  perf-profile.children.cycles-pp.lapic_next_deadline
      0.54 ± 17%      -0.5        0.09 ± 26%  perf-profile.children.cycles-pp.crc_128
      0.80 ± 33%      -0.4        0.36 ± 49%  perf-profile.children.cycles-pp.write
      0.79 ± 34%      -0.4        0.35 ± 50%  perf-profile.children.cycles-pp.devkmsg_write.cold
      0.79 ± 34%      -0.4        0.35 ± 50%  perf-profile.children.cycles-pp.devkmsg_emit
      0.49 ± 17%      -0.4        0.07 ±  8%  perf-profile.children.cycles-pp.__process_pages_contig
      0.54 ± 34%      -0.4        0.12 ± 10%  perf-profile.children.cycles-pp._raw_spin_trylock
      0.50 ± 13%      -0.4        0.08 ± 17%  perf-profile.children.cycles-pp.clear_extent_bit
      0.49 ± 12%      -0.4        0.08 ± 17%  perf-profile.children.cycles-pp.__clear_extent_bit
      0.53 ±  8%      -0.4        0.12 ± 12%  perf-profile.children.cycles-pp.__pagevec_lru_add
      0.45 ± 22%      -0.4        0.05 ± 48%  perf-profile.children.cycles-pp.submit_extent_page
      0.45 ± 14%      -0.4        0.06 ± 57%  perf-profile.children.cycles-pp.crc_42
      0.44 ± 15%      -0.4        0.06 ±  6%  perf-profile.children.cycles-pp.find_lock_delalloc_range
      0.45 ±  5%      -0.4        0.09 ± 23%  perf-profile.children.cycles-pp.rmqueue_bulk
      0.43 ±  9%      -0.3        0.10 ± 23%  perf-profile.children.cycles-pp.end_page_writeback
      0.38 ± 10%      -0.3        0.06 ± 48%  perf-profile.children.cycles-pp.alloc_extent_state
      0.39 ± 10%      -0.3        0.09 ± 22%  perf-profile.children.cycles-pp.test_clear_page_writeback
      0.41 ± 16%      -0.3        0.11 ± 25%  perf-profile.children.cycles-pp.prep_new_page
      0.37 ± 24%      -0.3        0.09 ± 63%  perf-profile.children.cycles-pp.update_rq_clock
      0.35 ±  9%      -0.3        0.07 ± 16%  perf-profile.children.cycles-pp.kmem_cache_alloc
      0.33 ± 15%      -0.3        0.06 ± 17%  perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
      0.49 ± 18%      -0.3        0.21 ± 45%  perf-profile.children.cycles-pp.load_balance
      0.33 ± 10%      -0.3        0.06 ± 11%  perf-profile.children.cycles-pp.xas_load
      0.34 ± 14%      -0.3        0.08 ± 17%  perf-profile.children.cycles-pp.mem_cgroup_charge
      0.28 ± 15%      -0.2        0.04 ± 75%  perf-profile.children.cycles-pp.find_next_bit
      0.29 ± 11%      -0.2        0.05 ±  7%  perf-profile.children.cycles-pp.__mod_lruvec_state
      0.29 ± 19%      -0.2        0.06 ± 16%  perf-profile.children.cycles-pp.account_page_dirtied
      0.33 ± 18%      -0.2        0.09 ± 27%  perf-profile.children.cycles-pp.clear_page_erms
      0.61 ± 20%      -0.2        0.39 ±  8%  perf-profile.children.cycles-pp.sched_clock_cpu
      0.34 ± 29%      -0.2        0.13 ± 43%  perf-profile.children.cycles-pp.hrtimer_update_next_event
      0.37 ± 13%      -0.2        0.16 ± 44%  perf-profile.children.cycles-pp.find_busiest_group
      0.27 ±  7%      -0.2        0.07 ± 16%  perf-profile.children.cycles-pp.set_extent_bit
      0.27 ±  7%      -0.2        0.07 ± 17%  perf-profile.children.cycles-pp.btrfs_set_extent_delalloc
      0.31 ± 34%      -0.2        0.11 ± 48%  perf-profile.children.cycles-pp.__hrtimer_next_event_base
      0.34 ± 14%      -0.2        0.15 ± 46%  perf-profile.children.cycles-pp.update_sd_lb_stats
      0.52 ± 15%      -0.2        0.33 ± 16%  perf-profile.children.cycles-pp.sched_clock
      0.29 ± 28%      -0.2        0.11 ± 64%  perf-profile.children.cycles-pp.rcu_idle_exit
      0.29 ± 41%      -0.2        0.10 ± 66%  perf-profile.children.cycles-pp.update_blocked_averages
      0.49 ± 16%      -0.2        0.31 ± 16%  perf-profile.children.cycles-pp.native_sched_clock
      0.27 ± 18%      -0.2        0.10 ± 18%  perf-profile.children.cycles-pp.get_next_timer_interrupt
      0.20 ± 12%      -0.2        0.04 ± 71%  perf-profile.children.cycles-pp.iov_iter_fault_in_readable
      0.30 ± 36%      -0.2        0.14 ± 35%  perf-profile.children.cycles-pp.__intel_pmu_enable_all
      0.25 ± 30%      -0.2        0.09 ± 35%  perf-profile.children.cycles-pp.hrtimer_next_event_without
      0.21 ± 27%      -0.1        0.07 ± 26%  perf-profile.children.cycles-pp.do_sys_open
      0.21 ± 27%      -0.1        0.07 ± 26%  perf-profile.children.cycles-pp.do_sys_openat2
      0.21 ± 29%      -0.1        0.07 ± 23%  perf-profile.children.cycles-pp.do_filp_open
      0.21 ± 29%      -0.1        0.07 ± 23%  perf-profile.children.cycles-pp.path_openat
      0.24 ± 12%      -0.1        0.12 ± 33%  perf-profile.children.cycles-pp.arch_scale_freq_tick
      0.20 ± 27%      -0.1        0.08 ± 60%  perf-profile.children.cycles-pp.rcu_eqs_exit
      0.14 ± 35%      -0.1        0.03 ±102%  perf-profile.children.cycles-pp.idle_cpu
      0.17 ± 27%      -0.1        0.06 ± 59%  perf-profile.children.cycles-pp.vt_console_print
      0.17 ± 29%      -0.1        0.06 ± 56%  perf-profile.children.cycles-pp.fbcon_redraw
      0.17 ± 29%      -0.1        0.06 ± 59%  perf-profile.children.cycles-pp.lf
      0.17 ± 29%      -0.1        0.06 ± 59%  perf-profile.children.cycles-pp.con_scroll
      0.17 ± 29%      -0.1        0.06 ± 59%  perf-profile.children.cycles-pp.fbcon_scroll
      0.16 ± 28%      -0.1        0.06 ± 59%  perf-profile.children.cycles-pp.fbcon_putcs
      0.16 ± 27%      -0.1        0.06 ± 59%  perf-profile.children.cycles-pp.bit_putcs
      0.13 ± 16%      -0.1        0.03 ±102%  perf-profile.children.cycles-pp.hrtimer_get_next_event
      0.15 ± 27%      -0.1        0.05 ± 59%  perf-profile.children.cycles-pp.drm_fb_helper_sys_imageblit
      0.12 ± 39%      -0.1        0.03 ± 99%  perf-profile.children.cycles-pp.btrfs_create
      0.15 ± 26%      -0.1        0.05 ± 59%  perf-profile.children.cycles-pp.sys_imageblit
      0.11 ± 18%      -0.1        0.03 ±100%  perf-profile.children.cycles-pp.__schedule
      0.16 ± 10%      -0.1        0.08 ± 36%  perf-profile.children.cycles-pp._raw_spin_lock_irq
      0.09 ± 19%      -0.1        0.03 ±108%  perf-profile.children.cycles-pp.cpumask_next_and
      0.02 ±142%      +0.1        0.08 ± 57%  perf-profile.children.cycles-pp.seq_read_iter
      0.21 ±  7%      +0.3        0.54 ± 34%  perf-profile.children.cycles-pp.start_kernel
      0.00            +0.6        0.63 ±  6%  perf-profile.children.cycles-pp.sched_clock_local
      0.31 ± 17%      +2.3        2.60 ±  6%  perf-profile.children.cycles-pp.__x64_sys_gettimeofday
      0.12 ± 20%      +2.5        2.58 ±  6%  perf-profile.children.cycles-pp.ktime_get_real_ts64
      0.85 ± 33%      +4.1        4.98 ±  9%  perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
      2.25 ± 17%      +4.6        6.87 ±  6%  perf-profile.children.cycles-pp.tick_sched_handle
      2.14 ± 14%      +4.7        6.82 ±  6%  perf-profile.children.cycles-pp.update_process_times
      1.05 ± 15%      +5.4        6.50 ±  3%  perf-profile.children.cycles-pp.scheduler_tick
      0.60 ± 24%      +7.2        7.81 ± 14%  perf-profile.children.cycles-pp.tick_nohz_irq_exit
      2.02 ± 27%      +8.0        9.99 ± 11%  perf-profile.children.cycles-pp.clockevents_program_event
      0.68 ± 31%      +9.1        9.75 ±  9%  perf-profile.children.cycles-pp.ktime_get_update_offsets_now
      2.69 ± 14%     +10.0       12.69 ±  5%  perf-profile.children.cycles-pp.tick_sched_timer
      4.28 ± 18%     +13.8       18.08 ±  6%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.00           +16.9       16.89 ± 13%  perf-profile.children.cycles-pp.sched_clock_idle_wakeup_event
      1.37 ± 23%     +17.4       18.78 ± 14%  perf-profile.children.cycles-pp.irq_enter_rcu
      1.35 ± 23%     +17.4       18.78 ± 14%  perf-profile.children.cycles-pp.tick_irq_enter
      0.00           +22.7       22.69 ± 10%  perf-profile.children.cycles-pp.sched_clock_tick
     60.63 ±  2%     +26.7       87.34        perf-profile.children.cycles-pp.start_secondary
     60.85 ±  2%     +27.0       87.88        perf-profile.children.cycles-pp.secondary_startup_64_no_verify
     60.85 ±  2%     +27.0       87.88        perf-profile.children.cycles-pp.cpu_startup_entry
     60.85 ±  2%     +27.0       87.88        perf-profile.children.cycles-pp.do_idle
      7.82 ± 11%     +30.4       38.22        perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
      7.67 ± 10%     +30.5       38.13 ±  2%  perf-profile.children.cycles-pp.hrtimer_interrupt
     55.14 ±  2%     +31.4       86.51        perf-profile.children.cycles-pp.cpuidle_enter
     55.10 ±  2%     +31.4       86.49        perf-profile.children.cycles-pp.cpuidle_enter_state
     15.30 ± 10%     +51.0       66.31 ±  6%  perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
     12.68 ± 10%     +53.0       65.66 ±  6%  perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
      3.22 ± 28%     +56.7       59.87 ±  9%  perf-profile.children.cycles-pp.ktime_get
      0.00           +70.8       70.85 ±  9%  perf-profile.children.cycles-pp.read_hpet
     36.19 ±  5%     -23.6       12.63 ± 30%  perf-profile.self.cycles-pp.intel_idle
      6.39 ± 10%      -5.2        1.21 ± 11%  perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      4.33 ± 36%      -3.4        0.90 ± 11%  perf-profile.self.cycles-pp.cpuidle_enter_state
      3.06 ± 11%      -2.4        0.61 ± 19%  perf-profile.self.cycles-pp.__percpu_counter_sum
      2.90 ±  6%      -2.3        0.64 ± 16%  perf-profile.self.cycles-pp._raw_spin_lock
      2.60 ± 27%      -2.0        0.55 ± 24%  perf-profile.self.cycles-pp.menu_select
      2.72 ± 32%      -1.7        0.97 ± 11%  perf-profile.self.cycles-pp.ktime_get
      1.87 ±  7%      -1.4        0.42 ± 28%  perf-profile.self.cycles-pp.brd_do_bvec
      1.36 ± 11%      -1.1        0.30 ± 24%  perf-profile.self.cycles-pp._find_next_bit
      0.85 ± 25%      -0.6        0.24 ±  6%  perf-profile.self.cycles-pp.native_irq_return_iret
      0.74 ± 35%      -0.6        0.14 ± 16%  perf-profile.self.cycles-pp.irqtime_account_irq
      0.73 ± 14%      -0.6        0.15 ± 23%  perf-profile.self.cycles-pp.cpumask_next
      0.96 ± 16%      -0.5        0.41 ± 35%  perf-profile.self.cycles-pp.memcpy_toio
      0.54 ± 17%      -0.5        0.08 ± 32%  perf-profile.self.cycles-pp.crc_128
      0.77 ± 19%      -0.5        0.32 ± 11%  perf-profile.self.cycles-pp.lapic_next_deadline
      0.57 ± 36%      -0.4        0.14 ± 38%  perf-profile.self.cycles-pp.ktime_get_update_offsets_now
      0.65 ± 10%      -0.4        0.22 ± 10%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.53 ± 35%      -0.4        0.12 ± 10%  perf-profile.self.cycles-pp._raw_spin_trylock
      0.44 ± 14%      -0.4        0.06 ± 59%  perf-profile.self.cycles-pp.crc_42
      0.36 ± 15%      -0.3        0.05 ± 93%  perf-profile.self.cycles-pp.tick_nohz_next_event
      0.38 ±  5%      -0.3        0.08 ± 24%  perf-profile.self.cycles-pp.rmqueue_bulk
      0.28 ± 15%      -0.2        0.04 ± 75%  perf-profile.self.cycles-pp.find_next_bit
      0.32 ± 18%      -0.2        0.09 ± 25%  perf-profile.self.cycles-pp.clear_page_erms
      0.26 ±  9%      -0.2        0.03 ± 70%  perf-profile.self.cycles-pp.xas_load
      0.25 ± 12%      -0.2        0.04 ± 71%  perf-profile.self.cycles-pp.__pagevec_lru_add
      0.31 ± 22%      -0.2        0.12 ± 46%  perf-profile.self.cycles-pp.do_idle
      0.47 ± 14%      -0.2        0.30 ± 16%  perf-profile.self.cycles-pp.native_sched_clock
      0.30 ± 36%      -0.2        0.14 ± 35%  perf-profile.self.cycles-pp.__intel_pmu_enable_all
      0.26 ± 34%      -0.2        0.10 ± 49%  perf-profile.self.cycles-pp.__hrtimer_next_event_base
      0.18 ± 19%      -0.1        0.04 ± 71%  perf-profile.self.cycles-pp.asm_sysvec_apic_timer_interrupt
      0.24 ± 12%      -0.1        0.12 ± 33%  perf-profile.self.cycles-pp.arch_scale_freq_tick
      0.22 ± 10%      -0.1        0.10 ± 45%  perf-profile.self.cycles-pp.update_sd_lb_stats
      0.13 ± 36%      -0.1        0.03 ±102%  perf-profile.self.cycles-pp.idle_cpu
      0.15 ± 26%      -0.1        0.05 ± 59%  perf-profile.self.cycles-pp.sys_imageblit
      0.18 ± 27%      -0.1        0.09 ± 50%  perf-profile.self.cycles-pp.hrtimer_interrupt
      0.16 ± 10%      -0.1        0.08 ± 38%  perf-profile.self.cycles-pp._raw_spin_lock_irq
      0.00            +0.4        0.40 ±  2%  perf-profile.self.cycles-pp.sched_clock_local
      0.00           +70.8       70.79 ±  9%  perf-profile.self.cycles-pp.read_hpet
     18.50 ± 33%   +1172.1%     235.33 ±116%  interrupts.34:PCI-MSI.524292-edge.eth0-TxRx-3
    252658 ±  2%     -19.6%     203234 ±  3%  interrupts.CAL:Function_call_interrupts
    953.00 ± 38%    +165.4%       2529 ± 48%  interrupts.CPU0.CAL:Function_call_interrupts
    110.33 ± 17%    +559.5%     727.67 ± 45%  interrupts.CPU0.NMI:Non-maskable_interrupts
    110.33 ± 17%    +559.5%     727.67 ± 45%  interrupts.CPU0.PMI:Performance_monitoring_interrupts
    320.17 ±149%    +406.0%       1620 ± 56%  interrupts.CPU1.NMI:Non-maskable_interrupts
    320.17 ±149%    +406.0%       1620 ± 56%  interrupts.CPU1.PMI:Performance_monitoring_interrupts
    109.67 ± 35%    +391.5%     539.00 ± 21%  interrupts.CPU10.NMI:Non-maskable_interrupts
    109.67 ± 35%    +391.5%     539.00 ± 21%  interrupts.CPU10.PMI:Performance_monitoring_interrupts
     93.83 ± 27%   +1274.6%       1289 ±155%  interrupts.CPU101.NMI:Non-maskable_interrupts
     93.83 ± 27%   +1274.6%       1289 ±155%  interrupts.CPU101.PMI:Performance_monitoring_interrupts
     93.67 ± 27%    +445.9%     511.33 ± 55%  interrupts.CPU103.NMI:Non-maskable_interrupts
     93.67 ± 27%    +445.9%     511.33 ± 55%  interrupts.CPU103.PMI:Performance_monitoring_interrupts
     94.67 ± 27%    +444.0%     515.00 ± 54%  interrupts.CPU104.NMI:Non-maskable_interrupts
     94.67 ± 27%    +444.0%     515.00 ± 54%  interrupts.CPU104.PMI:Performance_monitoring_interrupts
     95.33 ± 26%    +482.3%     555.17 ± 66%  interrupts.CPU105.NMI:Non-maskable_interrupts
     95.33 ± 26%    +482.3%     555.17 ± 66%  interrupts.CPU105.PMI:Performance_monitoring_interrupts
     92.17 ± 27%    +477.0%     531.83 ± 60%  interrupts.CPU106.NMI:Non-maskable_interrupts
     92.17 ± 27%    +477.0%     531.83 ± 60%  interrupts.CPU106.PMI:Performance_monitoring_interrupts
     91.50 ± 30%    +463.8%     515.83 ± 54%  interrupts.CPU107.NMI:Non-maskable_interrupts
     91.50 ± 30%    +463.8%     515.83 ± 54%  interrupts.CPU107.PMI:Performance_monitoring_interrupts
    103.00 ± 28%    +411.5%     526.83 ± 21%  interrupts.CPU11.NMI:Non-maskable_interrupts
    103.00 ± 28%    +411.5%     526.83 ± 21%  interrupts.CPU11.PMI:Performance_monitoring_interrupts
    157.67 ± 61%    +257.0%     562.83 ± 47%  interrupts.CPU110.NMI:Non-maskable_interrupts
    157.67 ± 61%    +257.0%     562.83 ± 47%  interrupts.CPU110.PMI:Performance_monitoring_interrupts
    116.50 ± 21%    +437.9%     626.67 ± 48%  interrupts.CPU111.NMI:Non-maskable_interrupts
    116.50 ± 21%    +437.9%     626.67 ± 48%  interrupts.CPU111.PMI:Performance_monitoring_interrupts
     99.83 ± 39%    +462.3%     561.33 ± 48%  interrupts.CPU112.NMI:Non-maskable_interrupts
     99.83 ± 39%    +462.3%     561.33 ± 48%  interrupts.CPU112.PMI:Performance_monitoring_interrupts
    102.50 ± 40%    +565.4%     682.00 ± 56%  interrupts.CPU113.NMI:Non-maskable_interrupts
    102.50 ± 40%    +565.4%     682.00 ± 56%  interrupts.CPU113.PMI:Performance_monitoring_interrupts
    132.17 ± 66%    +373.6%     626.00 ± 46%  interrupts.CPU114.NMI:Non-maskable_interrupts
    132.17 ± 66%    +373.6%     626.00 ± 46%  interrupts.CPU114.PMI:Performance_monitoring_interrupts
    100.33 ± 38%    +486.5%     588.50 ± 47%  interrupts.CPU116.NMI:Non-maskable_interrupts
    100.33 ± 38%    +486.5%     588.50 ± 47%  interrupts.CPU116.PMI:Performance_monitoring_interrupts
    101.67 ± 37%    +469.3%     578.83 ± 48%  interrupts.CPU117.NMI:Non-maskable_interrupts
    101.67 ± 37%    +469.3%     578.83 ± 48%  interrupts.CPU117.PMI:Performance_monitoring_interrupts
    106.33 ± 38%    +441.1%     575.33 ± 48%  interrupts.CPU118.NMI:Non-maskable_interrupts
    106.33 ± 38%    +441.1%     575.33 ± 48%  interrupts.CPU118.PMI:Performance_monitoring_interrupts
    106.67 ± 43%    +440.6%     576.67 ± 48%  interrupts.CPU119.NMI:Non-maskable_interrupts
    106.67 ± 43%    +440.6%     576.67 ± 48%  interrupts.CPU119.PMI:Performance_monitoring_interrupts
    146.33 ± 86%    +261.6%     529.17 ± 52%  interrupts.CPU121.NMI:Non-maskable_interrupts
    146.33 ± 86%    +261.6%     529.17 ± 52%  interrupts.CPU121.PMI:Performance_monitoring_interrupts
    101.17 ± 32%    +421.9%     528.00 ± 52%  interrupts.CPU122.NMI:Non-maskable_interrupts
    101.17 ± 32%    +421.9%     528.00 ± 52%  interrupts.CPU122.PMI:Performance_monitoring_interrupts
    140.67 ± 51%    +274.8%     527.17 ± 49%  interrupts.CPU124.NMI:Non-maskable_interrupts
    140.67 ± 51%    +274.8%     527.17 ± 49%  interrupts.CPU124.PMI:Performance_monitoring_interrupts
    115.00 ± 32%    +431.4%     611.17 ± 54%  interrupts.CPU129.NMI:Non-maskable_interrupts
    115.00 ± 32%    +431.4%     611.17 ± 54%  interrupts.CPU129.PMI:Performance_monitoring_interrupts
     18.50 ± 33%   +1172.1%     235.33 ±116%  interrupts.CPU13.34:PCI-MSI.524292-edge.eth0-TxRx-3
    113.67 ± 33%    +438.3%     611.83 ± 54%  interrupts.CPU130.NMI:Non-maskable_interrupts
    113.67 ± 33%    +438.3%     611.83 ± 54%  interrupts.CPU130.PMI:Performance_monitoring_interrupts
    118.17 ± 33%   +1087.6%       1403 ±142%  interrupts.CPU131.NMI:Non-maskable_interrupts
    118.17 ± 33%   +1087.6%       1403 ±142%  interrupts.CPU131.PMI:Performance_monitoring_interrupts
    115.17 ± 32%    +446.3%     629.17 ± 57%  interrupts.CPU133.NMI:Non-maskable_interrupts
    115.17 ± 32%    +446.3%     629.17 ± 57%  interrupts.CPU133.PMI:Performance_monitoring_interrupts
    213.33 ± 68%    +184.6%     607.17 ± 55%  interrupts.CPU134.NMI:Non-maskable_interrupts
    213.33 ± 68%    +184.6%     607.17 ± 55%  interrupts.CPU134.PMI:Performance_monitoring_interrupts
    114.00 ± 33%    +438.6%     614.00 ± 55%  interrupts.CPU135.NMI:Non-maskable_interrupts
    114.00 ± 33%    +438.6%     614.00 ± 55%  interrupts.CPU135.PMI:Performance_monitoring_interrupts
    115.83 ± 33%    +349.2%     520.33 ± 51%  interrupts.CPU136.NMI:Non-maskable_interrupts
    115.83 ± 33%    +349.2%     520.33 ± 51%  interrupts.CPU136.PMI:Performance_monitoring_interrupts
    136.50 ± 43%    +280.3%     519.17 ± 51%  interrupts.CPU137.NMI:Non-maskable_interrupts
    136.50 ± 43%    +280.3%     519.17 ± 51%  interrupts.CPU137.PMI:Performance_monitoring_interrupts
    152.83 ± 78%    +208.6%     471.67 ± 27%  interrupts.CPU14.NMI:Non-maskable_interrupts
    152.83 ± 78%    +208.6%     471.67 ± 27%  interrupts.CPU14.PMI:Performance_monitoring_interrupts
    123.50 ± 35%    +515.4%     760.00 ±102%  interrupts.CPU143.NMI:Non-maskable_interrupts
    123.50 ± 35%    +515.4%     760.00 ±102%  interrupts.CPU143.PMI:Performance_monitoring_interrupts
    137.00 ± 57%    +190.0%     397.33 ± 26%  interrupts.CPU145.NMI:Non-maskable_interrupts
    137.00 ± 57%    +190.0%     397.33 ± 26%  interrupts.CPU145.PMI:Performance_monitoring_interrupts
    124.33 ± 31%    +678.0%     967.33 ±130%  interrupts.CPU146.NMI:Non-maskable_interrupts
    124.33 ± 31%    +678.0%     967.33 ±130%  interrupts.CPU146.PMI:Performance_monitoring_interrupts
    108.00 ± 31%    +273.5%     403.33 ± 25%  interrupts.CPU147.NMI:Non-maskable_interrupts
    108.00 ± 31%    +273.5%     403.33 ± 25%  interrupts.CPU147.PMI:Performance_monitoring_interrupts
    110.33 ± 30%    +327.2%     471.33 ± 36%  interrupts.CPU148.NMI:Non-maskable_interrupts
    110.33 ± 30%    +327.2%     471.33 ± 36%  interrupts.CPU148.PMI:Performance_monitoring_interrupts
    108.83 ± 31%    +296.6%     431.67 ± 27%  interrupts.CPU149.NMI:Non-maskable_interrupts
    108.83 ± 31%    +296.6%     431.67 ± 27%  interrupts.CPU149.PMI:Performance_monitoring_interrupts
    111.50 ± 28%    +383.3%     538.83 ± 42%  interrupts.CPU15.NMI:Non-maskable_interrupts
    111.50 ± 28%    +383.3%     538.83 ± 42%  interrupts.CPU15.PMI:Performance_monitoring_interrupts
    110.00 ± 31%    +302.6%     442.83 ± 11%  interrupts.CPU150.NMI:Non-maskable_interrupts
    110.00 ± 31%    +302.6%     442.83 ± 11%  interrupts.CPU150.PMI:Performance_monitoring_interrupts
    112.67 ± 30%    +282.0%     430.33 ± 14%  interrupts.CPU151.NMI:Non-maskable_interrupts
    112.67 ± 30%    +282.0%     430.33 ± 14%  interrupts.CPU151.PMI:Performance_monitoring_interrupts
    118.33 ± 26%    +264.6%     431.50 ± 14%  interrupts.CPU152.NMI:Non-maskable_interrupts
    118.33 ± 26%    +264.6%     431.50 ± 14%  interrupts.CPU152.PMI:Performance_monitoring_interrupts
    109.67 ± 31%    +290.3%     428.00 ± 15%  interrupts.CPU153.NMI:Non-maskable_interrupts
    109.67 ± 31%    +290.3%     428.00 ± 15%  interrupts.CPU153.PMI:Performance_monitoring_interrupts
    110.33 ± 30%    +289.3%     429.50 ± 14%  interrupts.CPU154.NMI:Non-maskable_interrupts
    110.33 ± 30%    +289.3%     429.50 ± 14%  interrupts.CPU154.PMI:Performance_monitoring_interrupts
    104.33 ± 43%    +312.8%     430.67 ± 14%  interrupts.CPU155.NMI:Non-maskable_interrupts
    104.33 ± 43%    +312.8%     430.67 ± 14%  interrupts.CPU155.PMI:Performance_monitoring_interrupts
    105.33 ± 40%    +418.8%     546.50 ± 58%  interrupts.CPU156.NMI:Non-maskable_interrupts
    105.33 ± 40%    +418.8%     546.50 ± 58%  interrupts.CPU156.PMI:Performance_monitoring_interrupts
    108.83 ± 38%    +348.2%     487.83 ± 37%  interrupts.CPU157.NMI:Non-maskable_interrupts
    108.83 ± 38%    +348.2%     487.83 ± 37%  interrupts.CPU157.PMI:Performance_monitoring_interrupts
    148.50 ± 60%    +189.5%     429.83 ± 14%  interrupts.CPU158.NMI:Non-maskable_interrupts
    148.50 ± 60%    +189.5%     429.83 ± 14%  interrupts.CPU158.PMI:Performance_monitoring_interrupts
    116.17 ± 34%    +271.2%     431.17 ± 14%  interrupts.CPU159.NMI:Non-maskable_interrupts
    116.17 ± 34%    +271.2%     431.17 ± 14%  interrupts.CPU159.PMI:Performance_monitoring_interrupts
    104.00 ± 20%    +354.3%     472.50 ± 27%  interrupts.CPU16.NMI:Non-maskable_interrupts
    104.00 ± 20%    +354.3%     472.50 ± 27%  interrupts.CPU16.PMI:Performance_monitoring_interrupts
    115.17 ± 35%    +275.3%     432.17 ± 13%  interrupts.CPU160.NMI:Non-maskable_interrupts
    115.17 ± 35%    +275.3%     432.17 ± 13%  interrupts.CPU160.PMI:Performance_monitoring_interrupts
    121.67 ± 42%    +235.1%     407.67 ± 25%  interrupts.CPU161.NMI:Non-maskable_interrupts
    121.67 ± 42%    +235.1%     407.67 ± 25%  interrupts.CPU161.PMI:Performance_monitoring_interrupts
    113.50 ± 35%    +260.4%     409.00 ± 32%  interrupts.CPU162.NMI:Non-maskable_interrupts
    113.50 ± 35%    +260.4%     409.00 ± 32%  interrupts.CPU162.PMI:Performance_monitoring_interrupts
    115.67 ± 34%    +239.2%     392.33 ± 25%  interrupts.CPU163.NMI:Non-maskable_interrupts
    115.67 ± 34%    +239.2%     392.33 ± 25%  interrupts.CPU163.PMI:Performance_monitoring_interrupts
     94.50 ± 49%    +361.2%     435.83 ± 41%  interrupts.CPU164.NMI:Non-maskable_interrupts
     94.50 ± 49%    +361.2%     435.83 ± 41%  interrupts.CPU164.PMI:Performance_monitoring_interrupts
    104.50 ± 41%    +295.1%     412.83 ± 24%  interrupts.CPU165.NMI:Non-maskable_interrupts
    104.50 ± 41%    +295.1%     412.83 ± 24%  interrupts.CPU165.PMI:Performance_monitoring_interrupts
    105.67 ± 42%    +275.6%     396.83 ± 25%  interrupts.CPU166.NMI:Non-maskable_interrupts
    105.67 ± 42%    +275.6%     396.83 ± 25%  interrupts.CPU166.PMI:Performance_monitoring_interrupts
    110.33 ± 32%    +257.4%     394.33 ± 26%  interrupts.CPU167.NMI:Non-maskable_interrupts
    110.33 ± 32%    +257.4%     394.33 ± 26%  interrupts.CPU167.PMI:Performance_monitoring_interrupts
    121.67 ± 28%    +376.0%     579.17 ± 47%  interrupts.CPU169.NMI:Non-maskable_interrupts
    121.67 ± 28%    +376.0%     579.17 ± 47%  interrupts.CPU169.PMI:Performance_monitoring_interrupts
     97.00 ± 24%    +414.1%     498.67 ± 31%  interrupts.CPU17.NMI:Non-maskable_interrupts
     97.00 ± 24%    +414.1%     498.67 ± 31%  interrupts.CPU17.PMI:Performance_monitoring_interrupts
    132.33 ± 29%   +1128.8%       1626 ±135%  interrupts.CPU170.NMI:Non-maskable_interrupts
    132.33 ± 29%   +1128.8%       1626 ±135%  interrupts.CPU170.PMI:Performance_monitoring_interrupts
    127.00 ± 28%    +357.5%     581.00 ± 47%  interrupts.CPU171.NMI:Non-maskable_interrupts
    127.00 ± 28%    +357.5%     581.00 ± 47%  interrupts.CPU171.PMI:Performance_monitoring_interrupts
    126.83 ± 26%    +390.3%     621.83 ± 46%  interrupts.CPU172.NMI:Non-maskable_interrupts
    126.83 ± 26%    +390.3%     621.83 ± 46%  interrupts.CPU172.PMI:Performance_monitoring_interrupts
    147.33 ± 45%    +376.5%     702.00 ± 56%  interrupts.CPU174.NMI:Non-maskable_interrupts
    147.33 ± 45%    +376.5%     702.00 ± 56%  interrupts.CPU174.PMI:Performance_monitoring_interrupts
    129.67 ± 15%    +421.3%     676.00 ± 50%  interrupts.CPU176.NMI:Non-maskable_interrupts
    129.67 ± 15%    +421.3%     676.00 ± 50%  interrupts.CPU176.PMI:Performance_monitoring_interrupts
    147.33 ± 23%    +362.7%     681.67 ± 52%  interrupts.CPU177.NMI:Non-maskable_interrupts
    147.33 ± 23%    +362.7%     681.67 ± 52%  interrupts.CPU177.PMI:Performance_monitoring_interrupts
    132.83 ± 75%    +368.0%     621.67 ± 44%  interrupts.CPU18.NMI:Non-maskable_interrupts
    132.83 ± 75%    +368.0%     621.67 ± 44%  interrupts.CPU18.PMI:Performance_monitoring_interrupts
    127.83 ± 31%    +430.0%     677.50 ± 51%  interrupts.CPU180.NMI:Non-maskable_interrupts
    127.83 ± 31%    +430.0%     677.50 ± 51%  interrupts.CPU180.PMI:Performance_monitoring_interrupts
    127.50 ± 31%    +605.1%     899.00 ± 85%  interrupts.CPU181.NMI:Non-maskable_interrupts
    127.50 ± 31%    +605.1%     899.00 ± 85%  interrupts.CPU181.PMI:Performance_monitoring_interrupts
    180.00 ± 49%    +715.6%       1468 ±131%  interrupts.CPU182.NMI:Non-maskable_interrupts
    180.00 ± 49%    +715.6%       1468 ±131%  interrupts.CPU182.PMI:Performance_monitoring_interrupts
    123.17 ± 28%    +451.6%     679.33 ± 62%  interrupts.CPU184.NMI:Non-maskable_interrupts
    123.17 ± 28%    +451.6%     679.33 ± 62%  interrupts.CPU184.PMI:Performance_monitoring_interrupts
    145.17 ± 16%    +306.0%     589.33 ± 57%  interrupts.CPU185.NMI:Non-maskable_interrupts
    145.17 ± 16%    +306.0%     589.33 ± 57%  interrupts.CPU185.PMI:Performance_monitoring_interrupts
    127.17 ± 30%    +355.3%     579.00 ± 56%  interrupts.CPU187.NMI:Non-maskable_interrupts
    127.17 ± 30%    +355.3%     579.00 ± 56%  interrupts.CPU187.PMI:Performance_monitoring_interrupts
    124.33 ± 27%    +383.4%     601.00 ± 57%  interrupts.CPU188.NMI:Non-maskable_interrupts
    124.33 ± 27%    +383.4%     601.00 ± 57%  interrupts.CPU188.PMI:Performance_monitoring_interrupts
    106.83 ± 27%    +493.1%     633.67 ± 38%  interrupts.CPU20.NMI:Non-maskable_interrupts
    106.83 ± 27%    +493.1%     633.67 ± 38%  interrupts.CPU20.PMI:Performance_monitoring_interrupts
    106.50 ± 27%    +478.2%     615.83 ± 40%  interrupts.CPU21.NMI:Non-maskable_interrupts
    106.50 ± 27%    +478.2%     615.83 ± 40%  interrupts.CPU21.PMI:Performance_monitoring_interrupts
    121.33 ± 19%    +403.8%     611.33 ± 39%  interrupts.CPU22.NMI:Non-maskable_interrupts
    121.33 ± 19%    +403.8%     611.33 ± 39%  interrupts.CPU22.PMI:Performance_monitoring_interrupts
    121.33 ± 22%    +403.3%     610.67 ± 39%  interrupts.CPU23.NMI:Non-maskable_interrupts
    121.33 ± 22%    +403.3%     610.67 ± 39%  interrupts.CPU23.PMI:Performance_monitoring_interrupts
    149.83 ± 52%    +307.7%     610.83 ± 55%  interrupts.CPU25.NMI:Non-maskable_interrupts
    149.83 ± 52%    +307.7%     610.83 ± 55%  interrupts.CPU25.PMI:Performance_monitoring_interrupts
    115.17 ± 29%    +409.3%     586.50 ± 65%  interrupts.CPU26.NMI:Non-maskable_interrupts
    115.17 ± 29%    +409.3%     586.50 ± 65%  interrupts.CPU26.PMI:Performance_monitoring_interrupts
    114.00 ± 22%    +353.7%     517.17 ± 55%  interrupts.CPU3.NMI:Non-maskable_interrupts
    114.00 ± 22%    +353.7%     517.17 ± 55%  interrupts.CPU3.PMI:Performance_monitoring_interrupts
    115.17 ± 33%    +402.6%     578.83 ± 63%  interrupts.CPU33.NMI:Non-maskable_interrupts
    115.17 ± 33%    +402.6%     578.83 ± 63%  interrupts.CPU33.PMI:Performance_monitoring_interrupts
    112.50 ± 34%    +411.4%     575.33 ± 63%  interrupts.CPU34.NMI:Non-maskable_interrupts
    112.50 ± 34%    +411.4%     575.33 ± 63%  interrupts.CPU34.PMI:Performance_monitoring_interrupts
    115.83 ± 33%   +1500.9%       1854 ±162%  interrupts.CPU35.NMI:Non-maskable_interrupts
    115.83 ± 33%   +1500.9%       1854 ±162%  interrupts.CPU35.PMI:Performance_monitoring_interrupts
    113.83 ± 34%    +458.1%     635.33 ± 59%  interrupts.CPU37.NMI:Non-maskable_interrupts
    113.83 ± 34%    +458.1%     635.33 ± 59%  interrupts.CPU37.PMI:Performance_monitoring_interrupts
    100.50 ± 33%    +384.9%     487.33 ± 60%  interrupts.CPU39.NMI:Non-maskable_interrupts
    100.50 ± 33%    +384.9%     487.33 ± 60%  interrupts.CPU39.PMI:Performance_monitoring_interrupts
    172.67 ± 95%    +306.5%     701.83 ± 83%  interrupts.CPU4.NMI:Non-maskable_interrupts
    172.67 ± 95%    +306.5%     701.83 ± 83%  interrupts.CPU4.PMI:Performance_monitoring_interrupts
    106.83 ± 21%    +353.4%     484.33 ± 60%  interrupts.CPU40.NMI:Non-maskable_interrupts
    106.83 ± 21%    +353.4%     484.33 ± 60%  interrupts.CPU40.PMI:Performance_monitoring_interrupts
    137.17 ± 51%    +253.6%     485.00 ± 59%  interrupts.CPU41.NMI:Non-maskable_interrupts
    137.17 ± 51%    +253.6%     485.00 ± 59%  interrupts.CPU41.PMI:Performance_monitoring_interrupts
    118.17 ± 33%    +479.1%     684.33 ± 88%  interrupts.CPU47.NMI:Non-maskable_interrupts
    118.17 ± 33%    +479.1%     684.33 ± 88%  interrupts.CPU47.PMI:Performance_monitoring_interrupts
     93.33 ± 25%   +1731.4%       1709 ±163%  interrupts.CPU5.NMI:Non-maskable_interrupts
     93.33 ± 25%   +1731.4%       1709 ±163%  interrupts.CPU5.PMI:Performance_monitoring_interrupts
    130.83 ± 21%    +624.3%     947.67 ±139%  interrupts.CPU50.NMI:Non-maskable_interrupts
    130.83 ± 21%    +624.3%     947.67 ±139%  interrupts.CPU50.PMI:Performance_monitoring_interrupts
    121.17 ± 27%    +205.6%     370.33 ± 32%  interrupts.CPU51.NMI:Non-maskable_interrupts
    121.17 ± 27%    +205.6%     370.33 ± 32%  interrupts.CPU51.PMI:Performance_monitoring_interrupts
    120.67 ± 26%    +302.2%     485.33 ± 42%  interrupts.CPU52.NMI:Non-maskable_interrupts
    120.67 ± 26%    +302.2%     485.33 ± 42%  interrupts.CPU52.PMI:Performance_monitoring_interrupts
    120.83 ± 27%    +247.6%     420.00 ± 26%  interrupts.CPU53.NMI:Non-maskable_interrupts
    120.83 ± 27%    +247.6%     420.00 ± 26%  interrupts.CPU53.PMI:Performance_monitoring_interrupts
     94.67 ± 27%    +298.9%     377.67 ± 29%  interrupts.CPU54.NMI:Non-maskable_interrupts
     94.67 ± 27%    +298.9%     377.67 ± 29%  interrupts.CPU54.PMI:Performance_monitoring_interrupts
     97.00 ± 27%    +272.0%     360.83 ± 36%  interrupts.CPU55.NMI:Non-maskable_interrupts
     97.00 ± 27%    +272.0%     360.83 ± 36%  interrupts.CPU55.PMI:Performance_monitoring_interrupts
    119.00 ± 29%    +256.9%     424.67 ± 15%  interrupts.CPU56.NMI:Non-maskable_interrupts
    119.00 ± 29%    +256.9%     424.67 ± 15%  interrupts.CPU56.PMI:Performance_monitoring_interrupts
    117.50 ± 24%    +260.9%     424.00 ± 15%  interrupts.CPU57.NMI:Non-maskable_interrupts
    117.50 ± 24%    +260.9%     424.00 ± 15%  interrupts.CPU57.PMI:Performance_monitoring_interrupts
    111.17 ± 36%    +281.7%     424.33 ± 15%  interrupts.CPU58.NMI:Non-maskable_interrupts
    111.17 ± 36%    +281.7%     424.33 ± 15%  interrupts.CPU58.PMI:Performance_monitoring_interrupts
    120.50 ± 27%    +231.8%     399.83 ± 26%  interrupts.CPU59.NMI:Non-maskable_interrupts
    120.50 ± 27%    +231.8%     399.83 ± 26%  interrupts.CPU59.PMI:Performance_monitoring_interrupts
    120.00 ± 24%    +410.4%     612.50 ± 77%  interrupts.CPU60.NMI:Non-maskable_interrupts
    120.00 ± 24%    +410.4%     612.50 ± 77%  interrupts.CPU60.PMI:Performance_monitoring_interrupts
    123.33 ± 20%    +302.6%     496.50 ± 44%  interrupts.CPU61.NMI:Non-maskable_interrupts
    123.33 ± 20%    +302.6%     496.50 ± 44%  interrupts.CPU61.PMI:Performance_monitoring_interrupts
    141.67 ± 43%    +202.5%     428.50 ± 14%  interrupts.CPU62.NMI:Non-maskable_interrupts
    141.67 ± 43%    +202.5%     428.50 ± 14%  interrupts.CPU62.PMI:Performance_monitoring_interrupts
    108.00 ± 31%    +297.1%     428.83 ± 14%  interrupts.CPU63.NMI:Non-maskable_interrupts
    108.00 ± 31%    +297.1%     428.83 ± 14%  interrupts.CPU63.PMI:Performance_monitoring_interrupts
     89.83 ± 47%    +376.3%     427.83 ± 15%  interrupts.CPU64.NMI:Non-maskable_interrupts
     89.83 ± 47%    +376.3%     427.83 ± 15%  interrupts.CPU64.PMI:Performance_monitoring_interrupts
    103.50 ± 45%    +347.7%     463.33 ± 14%  interrupts.CPU65.NMI:Non-maskable_interrupts
    103.50 ± 45%    +347.7%     463.33 ± 14%  interrupts.CPU65.PMI:Performance_monitoring_interrupts
    110.17 ± 36%    +310.9%     452.67 ± 21%  interrupts.CPU66.NMI:Non-maskable_interrupts
    110.17 ± 36%    +310.9%     452.67 ± 21%  interrupts.CPU66.PMI:Performance_monitoring_interrupts
    102.17 ± 45%    +321.2%     430.33 ± 13%  interrupts.CPU67.NMI:Non-maskable_interrupts
    102.17 ± 45%    +321.2%     430.33 ± 13%  interrupts.CPU67.PMI:Performance_monitoring_interrupts
    109.33 ± 32%    +332.6%     473.00 ± 33%  interrupts.CPU68.NMI:Non-maskable_interrupts
    109.33 ± 32%    +332.6%     473.00 ± 33%  interrupts.CPU68.PMI:Performance_monitoring_interrupts
    102.50 ± 38%    +325.4%     436.00 ± 28%  interrupts.CPU69.NMI:Non-maskable_interrupts
    102.50 ± 38%    +325.4%     436.00 ± 28%  interrupts.CPU69.PMI:Performance_monitoring_interrupts
     93.17 ± 26%    +504.7%     563.33 ± 47%  interrupts.CPU7.NMI:Non-maskable_interrupts
     93.17 ± 26%    +504.7%     563.33 ± 47%  interrupts.CPU7.PMI:Performance_monitoring_interrupts
    108.67 ± 31%    +260.1%     391.33 ± 26%  interrupts.CPU70.NMI:Non-maskable_interrupts
    108.67 ± 31%    +260.1%     391.33 ± 26%  interrupts.CPU70.PMI:Performance_monitoring_interrupts
    109.50 ± 31%    +258.0%     392.00 ± 26%  interrupts.CPU71.NMI:Non-maskable_interrupts
    109.50 ± 31%    +258.0%     392.00 ± 26%  interrupts.CPU71.PMI:Performance_monitoring_interrupts
    115.33 ± 40%    +374.6%     547.33 ± 55%  interrupts.CPU73.NMI:Non-maskable_interrupts
    115.33 ± 40%    +374.6%     547.33 ± 55%  interrupts.CPU73.PMI:Performance_monitoring_interrupts
    123.67 ± 41%   +1485.7%       1961 ±137%  interrupts.CPU74.NMI:Non-maskable_interrupts
    123.67 ± 41%   +1485.7%       1961 ±137%  interrupts.CPU74.PMI:Performance_monitoring_interrupts
    128.67 ± 30%    +402.5%     646.50 ± 60%  interrupts.CPU75.NMI:Non-maskable_interrupts
    128.67 ± 30%    +402.5%     646.50 ± 60%  interrupts.CPU75.PMI:Performance_monitoring_interrupts
    138.00 ± 17%    +426.1%     726.00 ± 71%  interrupts.CPU76.NMI:Non-maskable_interrupts
    138.00 ± 17%    +426.1%     726.00 ± 71%  interrupts.CPU76.PMI:Performance_monitoring_interrupts
    157.67 ± 71%    +346.3%     703.67 ± 61%  interrupts.CPU78.NMI:Non-maskable_interrupts
    157.67 ± 71%    +346.3%     703.67 ± 61%  interrupts.CPU78.PMI:Performance_monitoring_interrupts
     93.83 ± 27%    +559.0%     618.33 ± 38%  interrupts.CPU8.NMI:Non-maskable_interrupts
     93.83 ± 27%    +559.0%     618.33 ± 38%  interrupts.CPU8.PMI:Performance_monitoring_interrupts
    119.67 ± 27%    +463.2%     674.00 ± 50%  interrupts.CPU80.NMI:Non-maskable_interrupts
    119.67 ± 27%    +463.2%     674.00 ± 50%  interrupts.CPU80.PMI:Performance_monitoring_interrupts
    139.17 ± 42%    +392.8%     685.83 ± 52%  interrupts.CPU81.NMI:Non-maskable_interrupts
    139.17 ± 42%    +392.8%     685.83 ± 52%  interrupts.CPU81.PMI:Performance_monitoring_interrupts
    110.33 ± 29%    +482.6%     642.83 ± 58%  interrupts.CPU84.NMI:Non-maskable_interrupts
    110.33 ± 29%    +482.6%     642.83 ± 58%  interrupts.CPU84.PMI:Performance_monitoring_interrupts
    121.33 ± 28%    +540.9%     777.67 ± 80%  interrupts.CPU85.NMI:Non-maskable_interrupts
    121.33 ± 28%    +540.9%     777.67 ± 80%  interrupts.CPU85.PMI:Performance_monitoring_interrupts
    145.83 ± 40%   +1180.5%       1867 ±157%  interrupts.CPU86.NMI:Non-maskable_interrupts
    145.83 ± 40%   +1180.5%       1867 ±157%  interrupts.CPU86.PMI:Performance_monitoring_interrupts
    121.17 ± 25%    +417.9%     627.50 ± 68%  interrupts.CPU88.NMI:Non-maskable_interrupts
    121.17 ± 25%    +417.9%     627.50 ± 68%  interrupts.CPU88.PMI:Performance_monitoring_interrupts
    167.67 ± 62%    +229.6%     552.67 ± 56%  interrupts.CPU89.NMI:Non-maskable_interrupts
    167.67 ± 62%    +229.6%     552.67 ± 56%  interrupts.CPU89.PMI:Performance_monitoring_interrupts
    109.50 ± 27%    +391.8%     538.50 ± 21%  interrupts.CPU9.NMI:Non-maskable_interrupts
    109.50 ± 27%    +391.8%     538.50 ± 21%  interrupts.CPU9.PMI:Performance_monitoring_interrupts
    107.83 ± 30%    +418.7%     559.33 ± 53%  interrupts.CPU91.NMI:Non-maskable_interrupts
    107.83 ± 30%    +418.7%     559.33 ± 53%  interrupts.CPU91.PMI:Performance_monitoring_interrupts
     97.00 ± 36%    +503.8%     585.67 ± 57%  interrupts.CPU92.NMI:Non-maskable_interrupts
     97.00 ± 36%    +503.8%     585.67 ± 57%  interrupts.CPU92.PMI:Performance_monitoring_interrupts
    152.00 ± 44%    +377.4%     725.67 ± 53%  interrupts.CPU95.NMI:Non-maskable_interrupts
    152.00 ± 44%    +377.4%     725.67 ± 53%  interrupts.CPU95.PMI:Performance_monitoring_interrupts
     99.33 ± 25%    +625.5%     720.67 ± 45%  interrupts.CPU96.NMI:Non-maskable_interrupts
     99.33 ± 25%    +625.5%     720.67 ± 45%  interrupts.CPU96.PMI:Performance_monitoring_interrupts
    251.67 ±126%    +395.4%       1246 ± 45%  interrupts.CPU97.NMI:Non-maskable_interrupts
    251.67 ±126%    +395.4%       1246 ± 45%  interrupts.CPU97.PMI:Performance_monitoring_interrupts
    201.83 ± 79%    +293.3%     793.83 ± 90%  interrupts.CPU98.NMI:Non-maskable_interrupts
    201.83 ± 79%    +293.3%     793.83 ± 90%  interrupts.CPU98.PMI:Performance_monitoring_interrupts
    111.00 ± 23%    +362.2%     513.00 ± 55%  interrupts.CPU99.NMI:Non-maskable_interrupts
    111.00 ± 23%    +362.2%     513.00 ± 55%  interrupts.CPU99.PMI:Performance_monitoring_interrupts
     46152 ± 10%    +158.6%     119363 ± 19%  interrupts.NMI:Non-maskable_interrupts
     46152 ± 10%    +158.6%     119363 ± 19%  interrupts.PMI:Performance_monitoring_interrupts
    398.67 ±  8%    +240.2%       1356 ± 17%  interrupts.RES:Rescheduling_interrupts





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


---
0DAY/LKP+ Test Infrastructure                   Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org       Intel Corporation

Thanks,
Oliver Sang


View attachment "config-5.12.0-rc5-00207-g6c52b5f3cfef" of type "text/plain" (172905 bytes)

View attachment "job-script" of type "text/plain" (8388 bytes)

View attachment "job.yaml" of type "text/plain" (5826 bytes)

View attachment "reproduce" of type "text/plain" (534 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ