lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Date:   Mon, 22 Jun 2020 08:55:25 +0800
From:   kernel test robot <rong.a.chen@...el.com>
To:     John Johansen <john.johansen@...onical.com>
Cc:     Vincent Guittot <vincent.guittot@...aro.org>,
        Sebastian Andrzej Siewior <bigeasy@...utronix.de>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org
Subject: [apparmor] 20d4e80d25: hackbench.throughput 1065.8% improvement

Greeting,

FYI, we noticed a 1065.8% improvement of hackbench.throughput due to commit:


commit: 20d4e80d255dd7cfecb53743bc550ebcad04549d ("apparmor: only get a label reference if the fast path check fails")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

in testcase: hackbench
on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
with following parameters:

	nr_threads: 100%
	mode: process
	ipc: socket
	cpufreq_governor: performance
	ucode: 0x500002c

test-description: Hackbench is both a benchmark and a stress test for the Linux kernel scheduler.
test-url: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/sched/cfs-scheduler/hackbench.c





Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
  gcc-9/performance/socket/x86_64-rhel-7.6/process/100%/debian-x86_64-20191114.cgz/lkp-csl-2ap4/hackbench/0x500002c

commit: 
  9c95a278ba ("apparmor: fix bind mounts aborting with -ENOMEM")
  20d4e80d25 ("apparmor: only get a label reference if the fast path check fails")

9c95a278ba7ca3cc 20d4e80d255dd7cfecb53743bc5 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
          2:4          -50%            :4     dmesg.WARNING:at#for_ip_interrupt_entry/0x
         %stddev     %change         %stddev
             \          |                \  
     34878         +1065.8%     406595        hackbench.throughput
 5.244e+08          +461.5%  2.944e+09        hackbench.time.involuntary_context_switches
    456280          +667.2%    3500695 ±  4%  hackbench.time.minor_page_faults
     19080            -1.9%      18719        hackbench.time.percent_of_cpu_this_job_got
    121254            -7.0%     112767 ±  2%  hackbench.time.system_time
      2020          +166.4%       5381        hackbench.time.user_time
 1.229e+09          +455.3%  6.822e+09        hackbench.time.voluntary_context_switches
      0.58 ± 14%      +1.9        2.47 ±  2%  mpstat.cpu.all.idle%
      1.63            +2.7        4.33        mpstat.cpu.all.usr%
     97.00            -4.9%      92.25        vmstat.cpu.sy
      3647 ±  4%     -13.0%       3174        vmstat.procs.r
   2733153          +469.9%   15577129 ±  2%  vmstat.system.cs
    695185 ±  3%    +212.0%    2169047 ±  3%  vmstat.system.in
 2.303e+08 ± 31%    +205.6%  7.039e+08 ±  4%  cpuidle.C1.time
  10587297 ± 34%    +496.6%   63159988        cpuidle.C1.usage
 3.537e+08 ± 35%    +339.0%  1.553e+09 ± 37%  cpuidle.C1E.time
   3413850 ± 15%    +293.7%   13441955 ±  8%  cpuidle.C1E.usage
    874916 ± 27%   +5930.2%   52759317        cpuidle.POLL.time
     59038 ± 28%  +35353.0%   20930846        cpuidle.POLL.usage
     24701           +22.3%      30219 ±  3%  meminfo.Inactive
     23849           +23.2%      29374 ±  3%  meminfo.Inactive(anon)
    150804           +14.2%     172279        meminfo.KReclaimable
   3839936           +12.3%    4310891        meminfo.Memused
    150804           +14.2%     172279        meminfo.SReclaimable
    540531           +38.6%     749154        meminfo.SUnreclaim
    199787 ±  3%     +20.4%     240504 ±  4%  meminfo.Shmem
    691336           +33.3%     921434        meminfo.Slab
    303940 ± 57%    +607.7%    2150965 ± 38%  numa-numastat.node0.local_node
    330237 ± 51%    +560.7%    2181894 ± 37%  numa-numastat.node0.numa_hit
    464460 ± 73%    +504.3%    2806948 ± 33%  numa-numastat.node1.local_node
    487784 ± 69%    +479.0%    2824067 ± 33%  numa-numastat.node1.numa_hit
    931990 ± 47%    +105.5%    1915162 ±  2%  numa-numastat.node2.local_node
    953760 ± 45%    +103.1%    1936956 ±  2%  numa-numastat.node2.numa_hit
    450973 ± 11%    +406.1%    2282177 ± 39%  numa-numastat.node3.local_node
    472777 ± 13%    +387.7%    2305526 ± 38%  numa-numastat.node3.numa_hit
      1561 ± 16%    +313.7%       6459 ± 54%  numa-meminfo.node0.Inactive
      1561 ± 16%    +282.1%       5966 ± 57%  numa-meminfo.node0.Inactive(anon)
     76955 ±  7%    +150.4%     192666 ± 84%  numa-meminfo.node0.SUnreclaim
    103824 ± 16%    +117.3%     225620 ± 77%  numa-meminfo.node0.Slab
    136739 ± 76%    +108.4%     284997 ± 66%  numa-meminfo.node1.SUnreclaim
      9297 ± 80%     -72.4%       2565 ± 48%  numa-meminfo.node2.Inactive
    170952 ± 49%     -92.9%      12133 ± 38%  numa-meminfo.node2.PageTables
    410370 ± 13%     +20.0%     492407 ±  3%  numa-meminfo.node3.FilePages
      7168 ±  6%     +25.9%       9026 ±  8%  numa-meminfo.node3.Mapped
    842340 ±  7%     +43.5%    1208935 ± 42%  numa-meminfo.node3.MemUsed
     77017 ±  8%    +138.9%     184018 ± 91%  numa-meminfo.node3.SUnreclaim
    124255 ± 56%     +68.9%     209826 ±  6%  numa-meminfo.node3.Shmem
    390.00 ± 16%    +282.3%       1491 ± 57%  numa-vmstat.node0.nr_inactive_anon
      1523 ±  8%     +54.3%       2350 ± 30%  numa-vmstat.node0.nr_mapped
     19317 ±  7%    +149.2%      48142 ± 84%  numa-vmstat.node0.nr_slab_unreclaimable
    390.00 ± 16%    +282.3%       1491 ± 57%  numa-vmstat.node0.nr_zone_inactive_anon
    534595 ± 23%    +187.1%    1534639 ± 26%  numa-vmstat.node0.numa_hit
    507829 ± 25%    +196.0%    1503302 ± 26%  numa-vmstat.node0.numa_local
     33957 ± 76%    +109.4%      71121 ± 66%  numa-vmstat.node1.nr_slab_unreclaimable
    692414 ± 40%    +184.4%    1969175 ± 29%  numa-vmstat.node1.numa_hit
    578156 ± 48%    +221.9%    1861001 ± 31%  numa-vmstat.node1.numa_local
     40938 ± 49%     -92.6%       3028 ± 38%  numa-vmstat.node2.nr_page_table_pages
    103194 ± 14%     +19.3%     123129 ±  3%  numa-vmstat.node3.nr_file_pages
      1730 ±  8%     +28.1%       2217 ±  9%  numa-vmstat.node3.nr_mapped
     31665 ± 56%     +65.7%      52484 ±  6%  numa-vmstat.node3.nr_shmem
     19149 ±  7%    +139.8%      45912 ± 91%  numa-vmstat.node3.nr_slab_unreclaimable
    641996 ± 15%    +156.3%    1645560 ± 30%  numa-vmstat.node3.numa_hit
    528834 ± 17%    +189.5%    1530811 ± 33%  numa-vmstat.node3.numa_local
    181761 ±  2%      +6.8%     194138        proc-vmstat.nr_active_anon
    336379            +2.8%     345788        proc-vmstat.nr_file_pages
      5970           +22.5%       7314 ±  3%  proc-vmstat.nr_inactive_anon
      7877            +9.4%       8616 ±  5%  proc-vmstat.nr_mapped
     50521 ±  2%     +18.6%      59931 ±  4%  proc-vmstat.nr_shmem
     37747           +14.0%      43033        proc-vmstat.nr_slab_reclaimable
    134803           +38.8%     187164        proc-vmstat.nr_slab_unreclaimable
    181761 ±  2%      +6.8%     194138        proc-vmstat.nr_zone_active_anon
      5970           +22.5%       7314 ±  3%  proc-vmstat.nr_zone_inactive_anon
    173335 ±  8%    +174.5%     475836 ±  3%  proc-vmstat.numa_hint_faults
     68385 ± 11%     +22.5%      83801 ±  3%  proc-vmstat.numa_hint_faults_local
   2266214          +308.5%    9257936 ±  2%  proc-vmstat.numa_hit
   2172991          +321.8%    9164713 ±  2%  proc-vmstat.numa_local
     96166 ± 10%    +320.7%     404613 ±  4%  proc-vmstat.numa_pages_migrated
    259711 ± 14%    +148.5%     645288 ±  3%  proc-vmstat.numa_pte_updates
      4274         +1212.9%      56118 ±  3%  proc-vmstat.pgactivate
   5130484 ±  7%    +475.8%   29542175 ±  4%  proc-vmstat.pgalloc_normal
   1939216 ±  7%    +174.0%    5313810 ±  3%  proc-vmstat.pgfault
   5142696 ±  7%    +472.9%   29460485 ±  4%  proc-vmstat.pgfree
     96166 ± 10%    +320.7%     404613 ±  4%  proc-vmstat.pgmigrate_success
     95656 ± 19%    +557.1%     628570 ± 58%  sched_debug.cfs_rq:/.MIN_vruntime.avg
  16595905 ± 36%    +184.6%   47224471 ± 27%  sched_debug.cfs_rq:/.MIN_vruntime.max
   1245896 ± 28%    +302.5%    5014862 ± 39%  sched_debug.cfs_rq:/.MIN_vruntime.stddev
    493.70 ± 26%     -85.5%      71.54 ±104%  sched_debug.cfs_rq:/.load.min
     95656 ± 19%    +557.1%     628570 ± 58%  sched_debug.cfs_rq:/.max_vruntime.avg
  16595905 ± 36%    +184.6%   47224471 ± 27%  sched_debug.cfs_rq:/.max_vruntime.max
   1245896 ± 28%    +302.5%    5014862 ± 39%  sched_debug.cfs_rq:/.max_vruntime.stddev
  94836732 ±  2%     -20.4%   75528292        sched_debug.cfs_rq:/.min_vruntime.avg
 1.274e+08 ±  4%     -31.9%   86769041 ±  2%  sched_debug.cfs_rq:/.min_vruntime.max
  74120340 ±  4%     -10.5%   66337853        sched_debug.cfs_rq:/.min_vruntime.min
  15473759 ± 19%     -74.2%    3993479 ±  8%  sched_debug.cfs_rq:/.min_vruntime.stddev
      0.64 ± 10%     -80.8%       0.12 ± 89%  sched_debug.cfs_rq:/.nr_running.min
      0.05 ± 17%    +165.7%       0.13 ± 12%  sched_debug.cfs_rq:/.nr_running.stddev
    186.40 ± 22%     -84.4%      29.07 ±100%  sched_debug.cfs_rq:/.runnable_weight.min
  22916553 ± 37%     -88.3%    2690868 ± 89%  sched_debug.cfs_rq:/.spread0.max
 -27003548           -36.3%  -17191228        sched_debug.cfs_rq:/.spread0.min
  14764449 ± 20%     -73.8%    3873274 ±  7%  sched_debug.cfs_rq:/.spread0.stddev
    235.38 ±  9%     +41.1%     332.07 ± 12%  sched_debug.cfs_rq:/.util_avg.min
      1052 ±  5%     -50.1%     525.17 ±  9%  sched_debug.cfs_rq:/.util_est_enqueued.avg
      3249 ±  2%     -44.7%       1798 ±  8%  sched_debug.cfs_rq:/.util_est_enqueued.max
     21.23 ± 22%     -84.3%       3.33 ± 24%  sched_debug.cfs_rq:/.util_est_enqueued.min
    756.69 ±  3%     -52.3%     360.96 ±  9%  sched_debug.cfs_rq:/.util_est_enqueued.stddev
      7363 ±  5%     -61.1%       2867 ± 15%  sched_debug.cpu.avg_idle.min
      5611 ± 19%     -73.6%       1483 ± 27%  sched_debug.cpu.clock.stddev
      5611 ± 19%     -73.6%       1483 ± 27%  sched_debug.cpu.clock_task.stddev
      6127          +630.0%      44729 ±  5%  sched_debug.cpu.curr->pid.avg
     14302 ±  9%    +257.9%      51194 ±  2%  sched_debug.cpu.curr->pid.max
      2143 ±  3%    +232.5%       7128 ± 11%  sched_debug.cpu.curr->pid.stddev
    628355 ±  7%    +132.8%    1462915 ± 49%  sched_debug.cpu.max_idle_balance_cost.max
     12270 ± 33%   +1095.4%     146681 ± 86%  sched_debug.cpu.max_idle_balance_cost.stddev
      0.01 ± 19%     -73.5%       0.00 ± 27%  sched_debug.cpu.next_balance.stddev
     16.51 ±  6%     -47.0%       8.75 ± 15%  sched_debug.cpu.nr_running.avg
     48.57 ±  2%     -42.7%      27.84 ± 14%  sched_debug.cpu.nr_running.max
      0.73 ±  8%     -63.0%       0.27 ± 40%  sched_debug.cpu.nr_running.min
     10.77 ±  3%     -44.4%       5.99 ± 15%  sched_debug.cpu.nr_running.stddev
   4470873          +471.6%   25555286 ±  3%  sched_debug.cpu.nr_switches.avg
   5654624 ±  6%    +442.7%   30688299 ±  4%  sched_debug.cpu.nr_switches.max
   2974750 ± 10%    +585.4%   20390053 ±  2%  sched_debug.cpu.nr_switches.min
    697832 ± 30%    +155.8%    1784781 ±  4%  sched_debug.cpu.nr_switches.stddev
      0.10 ±132%    +748.5%       0.86 ± 37%  sched_debug.cpu.nr_uninterruptible.avg
    -68.39          +170.2%    -184.78        sched_debug.cpu.nr_uninterruptible.min
     23.30 ±  3%    +175.6%      64.21 ±  6%  sched_debug.cpu.nr_uninterruptible.stddev
     15.96           -16.6%      13.31        perf-stat.i.MPKI
 6.068e+09          +723.4%  4.996e+10 ±  3%  perf-stat.i.branch-instructions
      1.70            -0.3        1.36        perf-stat.i.branch-miss-rate%
 1.023e+08          +552.1%  6.668e+08 ±  3%  perf-stat.i.branch-misses
     18.57 ±  8%      -6.3       12.32 ±  8%  perf-stat.i.cache-miss-rate%
  90539510 ±  8%    +315.1%  3.758e+08 ±  3%  perf-stat.i.cache-misses
 4.874e+08          +581.2%   3.32e+09 ±  3%  perf-stat.i.cache-references
   2704064          +474.6%   15536455 ±  3%  perf-stat.i.context-switches
     19.40           -87.0%       2.52 ±  6%  perf-stat.i.cpi
    196694            -1.5%     193677        perf-stat.i.cpu-clock
 5.956e+11            -3.2%  5.762e+11        perf-stat.i.cpu-cycles
    273142 ±  9%    +248.3%     951341 ±  2%  perf-stat.i.cpu-migrations
      7889 ±  7%     -79.5%       1616 ±  4%  perf-stat.i.cycles-between-cache-misses
      0.13 ±  2%      -0.0        0.12 ±  3%  perf-stat.i.dTLB-load-miss-rate%
  11586395 ±  3%    +639.6%   85697103 ±  3%  perf-stat.i.dTLB-load-misses
 8.793e+09          +731.7%  7.312e+10 ±  3%  perf-stat.i.dTLB-loads
      0.01 ±  3%      +0.0        0.03 ±  6%  perf-stat.i.dTLB-store-miss-rate%
    642041 ±  2%   +2188.4%   14692439 ±  7%  perf-stat.i.dTLB-store-misses
 5.282e+09          +731.1%   4.39e+10 ±  3%  perf-stat.i.dTLB-stores
     75.55            +3.1       78.68        perf-stat.i.iTLB-load-miss-rate%
  62846664          +565.0%  4.179e+08 ±  3%  perf-stat.i.iTLB-load-misses
  20137711          +461.3%   1.13e+08 ±  3%  perf-stat.i.iTLB-loads
 3.063e+10          +729.9%  2.543e+11 ±  3%  perf-stat.i.instructions
    499.33           +22.7%     612.52        perf-stat.i.instructions-per-iTLB-miss
      0.06          +685.0%       0.44 ±  3%  perf-stat.i.ipc
      2.62 ±  2%     +11.5%       2.92        perf-stat.i.metric.GHz
      0.67 ± 14%     -85.7%       0.10 ±  2%  perf-stat.i.metric.K/sec
     91.29 ±  3%    +846.1%     863.66 ±  4%  perf-stat.i.metric.M/sec
      4898 ±  4%     +94.6%       9532 ±  3%  perf-stat.i.minor-faults
     91.23            -5.9       85.33        perf-stat.i.node-load-miss-rate%
  33898182 ± 11%    +381.7%  1.633e+08 ±  5%  perf-stat.i.node-load-misses
   2642927 ±  5%    +761.9%   22778706 ±  3%  perf-stat.i.node-loads
     93.82           -11.3       82.55        perf-stat.i.node-store-miss-rate%
  13827198 ±  9%    +294.3%   54521791 ±  5%  perf-stat.i.node-store-misses
    679579 ±  6%   +1408.5%   10251380 ±  5%  perf-stat.i.node-stores
      4898 ±  4%     +94.6%       9532 ±  3%  perf-stat.i.page-faults
    196694            -1.5%     193677        perf-stat.i.task-clock
     15.88           -18.6%      12.93        perf-stat.overall.MPKI
      1.67            -0.3        1.34        perf-stat.overall.branch-miss-rate%
     15.74 ±  8%      -4.2       11.54 ±  6%  perf-stat.overall.cache-miss-rate%
     19.37           -88.3%       2.27 ±  3%  perf-stat.overall.cpi
      7796 ±  7%     -80.5%       1523 ±  3%  perf-stat.overall.cycles-between-cache-misses
      0.13 ±  3%      -0.0        0.12 ±  2%  perf-stat.overall.dTLB-load-miss-rate%
      0.01 ±  3%      +0.0        0.03 ±  5%  perf-stat.overall.dTLB-store-miss-rate%
     75.70            +2.9       78.60        perf-stat.overall.iTLB-load-miss-rate%
    484.34           +25.6%     608.13        perf-stat.overall.instructions-per-iTLB-miss
      0.05          +755.1%       0.44 ±  3%  perf-stat.overall.ipc
     91.58            -3.6       87.96        perf-stat.overall.node-load-miss-rate%
     94.33            -9.6       84.78        perf-stat.overall.node-store-miss-rate%
 6.006e+09          +728.2%  4.974e+10 ±  3%  perf-stat.ps.branch-instructions
 1.004e+08          +562.0%  6.646e+08 ±  3%  perf-stat.ps.branch-misses
  75823504 ±  7%    +396.9%  3.768e+08 ±  3%  perf-stat.ps.cache-misses
 4.819e+08          +579.1%  3.273e+09 ±  3%  perf-stat.ps.cache-references
   2728526          +471.5%   15593301 ±  2%  perf-stat.ps.context-switches
 5.878e+11            -2.5%  5.734e+11        perf-stat.ps.cpu-cycles
    227418 ±  6%    +306.2%     923715 ±  2%  perf-stat.ps.cpu-migrations
  11700638 ±  3%    +629.9%   85401653 ±  3%  perf-stat.ps.dTLB-load-misses
 8.725e+09          +734.2%  7.279e+10 ±  3%  perf-stat.ps.dTLB-loads
    586061 ±  3%   +2350.7%   14362665 ±  7%  perf-stat.ps.dTLB-store-misses
 5.256e+09          +731.2%  4.369e+10 ±  3%  perf-stat.ps.dTLB-stores
  62653466          +564.2%  4.161e+08 ±  3%  perf-stat.ps.iTLB-load-misses
  20112237          +463.2%  1.133e+08 ±  2%  perf-stat.ps.iTLB-loads
 3.035e+10          +734.1%  2.531e+11 ±  3%  perf-stat.ps.instructions
      2909 ±  7%    +184.1%       8265 ±  3%  perf-stat.ps.minor-faults
  27245612 ± 10%    +505.7%   1.65e+08 ±  5%  perf-stat.ps.node-load-misses
   2484641 ±  5%    +805.5%   22497476 ±  3%  perf-stat.ps.node-loads
  11314815 ±  8%    +386.0%   54993875 ±  4%  perf-stat.ps.node-store-misses
    673045 ±  6%   +1362.1%    9840843 ±  5%  perf-stat.ps.node-stores
      2909 ±  7%    +184.1%       8266 ±  3%  perf-stat.ps.page-faults
 1.963e+13          +714.5%  1.599e+14 ±  3%  perf-stat.total.instructions
      9343 ±  2%    +190.2%      27114        slabinfo.UNIX.active_objs
    292.50 ±  2%    +190.3%     849.25        slabinfo.UNIX.active_slabs
      9367 ±  2%    +190.3%      27192        slabinfo.UNIX.num_objs
    292.50 ±  2%    +190.3%     849.25        slabinfo.UNIX.num_slabs
    111891           +94.3%     217451        slabinfo.anon_vma.active_objs
      2433           +94.7%       4739        slabinfo.anon_vma.active_slabs
    111977           +94.7%     218024        slabinfo.anon_vma.num_objs
      2433           +94.7%       4739        slabinfo.anon_vma.num_slabs
    272027           +46.1%     397335        slabinfo.anon_vma_chain.active_objs
      4259           +46.2%       6227        slabinfo.anon_vma_chain.active_slabs
    272620           +46.2%     398605        slabinfo.anon_vma_chain.num_objs
      4259           +46.2%       6227        slabinfo.anon_vma_chain.num_slabs
     17554          +191.5%      51166 ±  2%  slabinfo.cred_jar.active_objs
    417.50          +192.0%       1219 ±  2%  slabinfo.cred_jar.active_slabs
     17554          +191.7%      51213 ±  2%  slabinfo.cred_jar.num_objs
    417.50          +192.0%       1219 ±  2%  slabinfo.cred_jar.num_slabs
     15160 ±  6%    +221.1%      48677 ±  2%  slabinfo.files_cache.active_objs
    329.00 ±  6%    +221.8%       1058 ±  2%  slabinfo.files_cache.active_slabs
     15160 ±  6%    +221.3%      48715 ±  2%  slabinfo.files_cache.num_objs
    329.00 ±  6%    +221.8%       1058 ±  2%  slabinfo.files_cache.num_slabs
     58825 ±  3%     +51.1%      88855        slabinfo.filp.active_objs
    925.25 ±  3%     +51.0%       1397        slabinfo.filp.active_slabs
     59237 ±  3%     +51.0%      89450        slabinfo.filp.num_objs
    925.25 ±  3%     +51.0%       1397        slabinfo.filp.num_slabs
     69324           +51.8%     105227        slabinfo.kmalloc-16.active_objs
    270.00           +52.2%     411.00        slabinfo.kmalloc-16.active_slabs
     69342           +51.9%     105327        slabinfo.kmalloc-16.num_objs
    270.00           +52.2%     411.00        slabinfo.kmalloc-16.num_slabs
    114153           +60.5%     183233        slabinfo.kmalloc-32.active_objs
    894.50           +60.6%       1436        slabinfo.kmalloc-32.active_slabs
    114571           +60.6%     183975        slabinfo.kmalloc-32.num_objs
    894.50           +60.6%       1436        slabinfo.kmalloc-32.num_slabs
     89036           +46.0%     129949 ±  2%  slabinfo.kmalloc-512.active_objs
      1469           +58.5%       2329 ±  2%  slabinfo.kmalloc-512.active_slabs
     94084           +58.5%     149086 ±  2%  slabinfo.kmalloc-512.num_objs
      1469           +58.5%       2329 ±  2%  slabinfo.kmalloc-512.num_slabs
     95145           +37.3%     130610        slabinfo.kmalloc-64.active_objs
      1491           +37.1%       2044        slabinfo.kmalloc-64.active_slabs
     95454           +37.1%     130849        slabinfo.kmalloc-64.num_objs
      1491           +37.1%       2044        slabinfo.kmalloc-64.num_slabs
    112185           +29.3%     145019        slabinfo.kmalloc-8.active_objs
    112221           +29.4%     145181        slabinfo.kmalloc-8.num_objs
     41613           +84.9%      76941        slabinfo.lsm_file_cache.active_objs
    244.00           +85.3%     452.25        slabinfo.lsm_file_cache.active_slabs
     41613           +84.9%      76941        slabinfo.lsm_file_cache.num_objs
    244.00           +85.3%     452.25        slabinfo.lsm_file_cache.num_slabs
     13027 ±  4%    +129.9%      29947        slabinfo.mm_struct.active_objs
    434.50 ±  4%    +130.6%       1002        slabinfo.mm_struct.active_slabs
     13056 ±  4%    +130.3%      30067        slabinfo.mm_struct.num_objs
    434.50 ±  4%    +130.6%       1002        slabinfo.mm_struct.num_slabs
     21502          +157.9%      55457 ±  2%  slabinfo.pid.active_objs
    335.50          +158.3%     866.50 ±  2%  slabinfo.pid.active_slabs
     21502          +158.0%      55478 ±  2%  slabinfo.pid.num_objs
    335.50          +158.3%     866.50 ±  2%  slabinfo.pid.num_slabs
     64096           -25.5%      47754 ±  5%  slabinfo.proc_inode_cache.active_objs
      1373           -15.7%       1157 ±  4%  slabinfo.proc_inode_cache.active_slabs
     67314           -15.7%      56712 ±  4%  slabinfo.proc_inode_cache.num_objs
      1373           -15.7%       1157 ±  4%  slabinfo.proc_inode_cache.num_slabs
     11150 ±  2%     +68.8%      18817        slabinfo.sighand_cache.active_objs
    747.75 ±  2%     +69.1%       1264        slabinfo.sighand_cache.active_slabs
     11221 ±  2%     +69.1%      18977        slabinfo.sighand_cache.num_objs
    747.75 ±  2%     +69.1%       1264        slabinfo.sighand_cache.num_slabs
     13952 ±  3%     +93.6%      27012 ±  2%  slabinfo.signal_cache.active_objs
    467.50 ±  3%     +94.3%     908.25 ±  2%  slabinfo.signal_cache.active_slabs
     14045 ±  3%     +94.1%      27262        slabinfo.signal_cache.num_objs
    467.50 ±  3%     +94.3%     908.25 ±  2%  slabinfo.signal_cache.num_slabs
    633.75 ± 11%     +23.6%     783.00 ± 10%  slabinfo.skbuff_fclone_cache.active_objs
    633.75 ± 11%     +23.6%     783.00 ± 10%  slabinfo.skbuff_fclone_cache.num_objs
     80763           +49.3%     120601 ±  2%  slabinfo.skbuff_head_cache.active_objs
      1334           +63.2%       2177 ±  2%  slabinfo.skbuff_head_cache.active_slabs
     85418           +63.2%     139394 ±  2%  slabinfo.skbuff_head_cache.num_objs
      1334           +63.2%       2177 ±  2%  slabinfo.skbuff_head_cache.num_slabs
     11132 ±  2%    +299.6%      44480 ±  2%  slabinfo.sock_inode_cache.active_objs
    284.75 ±  2%    +300.5%       1140 ±  2%  slabinfo.sock_inode_cache.active_slabs
     11132 ±  2%    +299.8%      44506 ±  2%  slabinfo.sock_inode_cache.num_objs
    284.75 ±  2%    +300.5%       1140 ±  2%  slabinfo.sock_inode_cache.num_slabs
     17586 ±  5%    +196.5%      52151 ±  2%  slabinfo.task_delay_info.active_objs
    344.25 ±  5%    +197.1%       1022 ±  2%  slabinfo.task_delay_info.active_slabs
     17586 ±  5%    +196.7%      52180 ±  2%  slabinfo.task_delay_info.num_objs
    344.25 ±  5%    +197.1%       1022 ±  2%  slabinfo.task_delay_info.num_slabs
    250594           +22.5%     307004        slabinfo.vm_area_struct.active_objs
      6281           +22.5%       7695        slabinfo.vm_area_struct.active_slabs
    251276           +22.5%     307836        slabinfo.vm_area_struct.num_objs
      6281           +22.5%       7695        slabinfo.vm_area_struct.num_slabs
     34583 ±  2%     +28.0%      44258 ±  2%  slabinfo.vmap_area.active_objs
    541.50 ±  2%     +31.9%     714.25 ±  2%  slabinfo.vmap_area.active_slabs
     34688 ±  2%     +31.9%      45748 ±  2%  slabinfo.vmap_area.num_objs
    541.50 ±  2%     +31.9%     714.25 ±  2%  slabinfo.vmap_area.num_slabs
    151.50 ± 23%     +50.3%     227.75 ± 24%  slabinfo.xfrm_state.active_objs
    151.50 ± 23%     +50.3%     227.75 ± 24%  slabinfo.xfrm_state.num_objs
    505131 ±  3%   +1120.4%    6164414 ±  9%  interrupts.CPU0.RES:Rescheduling_interrupts
    469994 ±  7%   +1171.9%    5977744 ±  9%  interrupts.CPU1.RES:Rescheduling_interrupts
    486843         +1078.5%    5737620 ±  6%  interrupts.CPU10.RES:Rescheduling_interrupts
    480817 ±  2%   +1118.9%    5860805 ±  7%  interrupts.CPU100.RES:Rescheduling_interrupts
    500309 ±  3%   +1048.6%    5746799 ±  9%  interrupts.CPU101.RES:Rescheduling_interrupts
    493286 ±  2%   +1032.2%    5584876 ±  6%  interrupts.CPU102.RES:Rescheduling_interrupts
    505896 ±  4%   +1001.4%    5572040 ±  4%  interrupts.CPU103.RES:Rescheduling_interrupts
    493679 ±  4%   +1089.8%    5873983 ±  5%  interrupts.CPU104.RES:Rescheduling_interrupts
    501273 ±  3%   +1035.4%    5691448 ±  6%  interrupts.CPU105.RES:Rescheduling_interrupts
    487640         +1082.4%    5765686 ±  7%  interrupts.CPU106.RES:Rescheduling_interrupts
    522470 ±  3%    +984.0%    5663614 ±  8%  interrupts.CPU107.RES:Rescheduling_interrupts
    513753 ±  4%   +1030.0%    5805645 ±  8%  interrupts.CPU108.RES:Rescheduling_interrupts
    493623 ±  4%   +1125.0%    6046820 ± 12%  interrupts.CPU109.RES:Rescheduling_interrupts
    500854 ±  3%   +1019.4%    5606717 ±  6%  interrupts.CPU11.RES:Rescheduling_interrupts
    512785 ±  5%    +999.0%    5635660 ±  2%  interrupts.CPU110.RES:Rescheduling_interrupts
    503023 ±  6%   +1020.1%    5634509 ±  9%  interrupts.CPU111.RES:Rescheduling_interrupts
    494950 ±  7%   +1026.0%    5572974 ± 10%  interrupts.CPU112.RES:Rescheduling_interrupts
    427562 ±  4%   +1253.6%    5787382 ± 10%  interrupts.CPU113.RES:Rescheduling_interrupts
    488349 ±  3%   +1053.9%    5635193 ±  3%  interrupts.CPU114.RES:Rescheduling_interrupts
    518113 ±  7%    +980.0%    5595697 ±  7%  interrupts.CPU115.RES:Rescheduling_interrupts
    498182 ±  2%   +1093.3%    5945017 ± 10%  interrupts.CPU116.RES:Rescheduling_interrupts
    497835 ±  5%   +1040.9%    5680049 ± 10%  interrupts.CPU117.RES:Rescheduling_interrupts
    503067 ±  4%   +1038.3%    5726511 ± 13%  interrupts.CPU118.RES:Rescheduling_interrupts
    501166 ±  5%   +1080.4%    5915906 ± 12%  interrupts.CPU119.RES:Rescheduling_interrupts
    509994 ±  3%   +1075.5%    5994768 ±  9%  interrupts.CPU12.RES:Rescheduling_interrupts
   1215965 ± 44%    +378.6%    5819408 ± 16%  interrupts.CPU120.RES:Rescheduling_interrupts
   1117392 ± 46%    +391.6%    5493119 ± 11%  interrupts.CPU121.RES:Rescheduling_interrupts
   1153071 ± 56%    +399.4%    5758200 ± 14%  interrupts.CPU122.RES:Rescheduling_interrupts
   1094813 ± 48%    +423.4%    5730300 ± 14%  interrupts.CPU123.RES:Rescheduling_interrupts
   1084940 ± 51%    +421.2%    5654561 ± 16%  interrupts.CPU124.RES:Rescheduling_interrupts
   1081587 ± 51%    +414.9%    5569256 ±  8%  interrupts.CPU125.RES:Rescheduling_interrupts
   1133682 ± 54%    +379.9%    5440619 ± 12%  interrupts.CPU126.RES:Rescheduling_interrupts
   1051266 ± 45%    +459.8%    5885305 ± 12%  interrupts.CPU127.RES:Rescheduling_interrupts
   1024314 ± 46%    +460.3%    5739075 ± 16%  interrupts.CPU128.RES:Rescheduling_interrupts
   1019900 ± 44%    +460.8%    5719387 ± 12%  interrupts.CPU129.RES:Rescheduling_interrupts
    496122 ±  4%   +1098.2%    5944367 ±  9%  interrupts.CPU13.RES:Rescheduling_interrupts
   1150880 ± 53%    +410.3%    5873066 ± 15%  interrupts.CPU130.RES:Rescheduling_interrupts
   1055806 ± 45%    +460.3%    5915165 ± 10%  interrupts.CPU131.RES:Rescheduling_interrupts
   1082200 ± 50%    +458.4%    6043466 ± 13%  interrupts.CPU132.RES:Rescheduling_interrupts
   1078673 ± 47%    +416.8%    5574769 ± 17%  interrupts.CPU133.RES:Rescheduling_interrupts
   1069800 ± 52%    +390.7%    5249780 ±  9%  interrupts.CPU134.RES:Rescheduling_interrupts
   1085944 ± 49%    +436.7%    5828563 ± 11%  interrupts.CPU135.RES:Rescheduling_interrupts
   1055203 ± 50%    +432.1%    5615141 ± 15%  interrupts.CPU136.RES:Rescheduling_interrupts
    965327 ± 45%    +477.6%    5576084 ± 12%  interrupts.CPU137.RES:Rescheduling_interrupts
   1136292 ± 52%    +401.2%    5694780 ± 15%  interrupts.CPU138.RES:Rescheduling_interrupts
   1119613 ± 52%    +415.7%    5774341 ± 11%  interrupts.CPU139.RES:Rescheduling_interrupts
    496363 ±  5%   +1028.8%    5602931 ±  3%  interrupts.CPU14.RES:Rescheduling_interrupts
   1112956 ± 53%    +412.4%    5703180 ± 13%  interrupts.CPU140.RES:Rescheduling_interrupts
   1109543 ± 51%    +420.6%    5776832 ± 14%  interrupts.CPU141.RES:Rescheduling_interrupts
   1093785 ± 47%    +421.5%    5704190 ± 14%  interrupts.CPU142.RES:Rescheduling_interrupts
   1050429 ± 45%    +454.1%    5819920 ± 11%  interrupts.CPU143.RES:Rescheduling_interrupts
   1775189 ± 29%    +239.1%    6018968 ±  3%  interrupts.CPU144.RES:Rescheduling_interrupts
   1478421 ± 28%    +316.5%    6157727 ±  2%  interrupts.CPU145.RES:Rescheduling_interrupts
   1542226 ± 28%    +306.7%    6271601 ± 15%  interrupts.CPU146.RES:Rescheduling_interrupts
   1539453 ± 28%    +307.0%    6266147 ±  4%  interrupts.CPU147.RES:Rescheduling_interrupts
   1586589 ± 27%    +277.6%    5991560 ±  2%  interrupts.CPU148.RES:Rescheduling_interrupts
   1549433 ± 29%    +293.9%    6102506 ±  5%  interrupts.CPU149.RES:Rescheduling_interrupts
    496402 ±  6%   +1042.0%    5669154 ±  8%  interrupts.CPU15.RES:Rescheduling_interrupts
   1503925 ± 27%    +315.1%    6242268 ± 11%  interrupts.CPU150.RES:Rescheduling_interrupts
   1499292 ± 29%    +327.4%    6408349 ±  5%  interrupts.CPU151.RES:Rescheduling_interrupts
   1571022 ± 28%    +273.9%    5873654 ±  5%  interrupts.CPU152.RES:Rescheduling_interrupts
   1590502 ± 27%    +293.2%    6253282 ±  3%  interrupts.CPU153.RES:Rescheduling_interrupts
   1574639 ± 28%    +260.4%    5675293 ±  3%  interrupts.CPU154.RES:Rescheduling_interrupts
   1534933 ± 26%    +305.7%    6227803 ±  5%  interrupts.CPU155.RES:Rescheduling_interrupts
   1501992 ± 26%    +314.5%    6225827 ±  6%  interrupts.CPU156.RES:Rescheduling_interrupts
   1502863 ± 27%    +324.9%    6385604 ±  3%  interrupts.CPU157.RES:Rescheduling_interrupts
   1595597 ± 28%    +284.2%    6129614 ±  6%  interrupts.CPU158.RES:Rescheduling_interrupts
   1682267 ± 30%    +312.7%    6943064 ±  5%  interrupts.CPU159.RES:Rescheduling_interrupts
    498606 ±  4%   +1049.4%    5730963 ±  9%  interrupts.CPU16.RES:Rescheduling_interrupts
   1439609 ± 27%    +347.5%    6441803 ±  6%  interrupts.CPU160.RES:Rescheduling_interrupts
   1576884 ± 29%    +308.7%    6444704 ±  9%  interrupts.CPU161.RES:Rescheduling_interrupts
   1552381 ± 29%    +293.1%    6102692 ±  9%  interrupts.CPU162.RES:Rescheduling_interrupts
   1506869 ± 28%    +321.9%    6357681 ±  5%  interrupts.CPU163.RES:Rescheduling_interrupts
   1555265 ± 29%    +300.6%    6231059 ±  7%  interrupts.CPU164.RES:Rescheduling_interrupts
   1641322 ± 30%    +289.3%    6389987 ±  6%  interrupts.CPU165.RES:Rescheduling_interrupts
   1515168 ± 23%    +321.4%    6384899 ±  4%  interrupts.CPU166.RES:Rescheduling_interrupts
   1497544 ± 25%    +300.0%    5989889 ±  2%  interrupts.CPU167.RES:Rescheduling_interrupts
     11.25 ± 49%    +973.3%     120.75 ± 23%  interrupts.CPU167.TLB:TLB_shootdowns
   1120458 ± 41%    +419.6%    5821412 ±  2%  interrupts.CPU168.RES:Rescheduling_interrupts
   1003946 ± 29%    +481.1%    5833851 ±  3%  interrupts.CPU169.RES:Rescheduling_interrupts
    427096 ±  3%   +1241.0%    5727512 ± 12%  interrupts.CPU17.RES:Rescheduling_interrupts
    981381 ± 28%    +506.1%    5947913 ±  2%  interrupts.CPU170.RES:Rescheduling_interrupts
    963570 ± 34%    +494.8%    5731692 ±  2%  interrupts.CPU171.RES:Rescheduling_interrupts
   1010437 ± 33%    +505.9%    6122540 ±  6%  interrupts.CPU172.RES:Rescheduling_interrupts
   1003138 ± 30%    +483.3%    5851568 ±  3%  interrupts.CPU173.RES:Rescheduling_interrupts
    977998 ± 30%    +496.6%    5834680 ±  4%  interrupts.CPU174.RES:Rescheduling_interrupts
    987341 ± 28%    +475.7%    5684003        interrupts.CPU175.RES:Rescheduling_interrupts
   1019108 ± 28%    +500.5%    6119439 ±  3%  interrupts.CPU176.RES:Rescheduling_interrupts
   1013072 ± 29%    +499.8%    6076452 ±  4%  interrupts.CPU177.RES:Rescheduling_interrupts
   1021297 ± 29%    +468.8%    5809160 ±  4%  interrupts.CPU178.RES:Rescheduling_interrupts
    997269 ± 30%    +457.5%    5559862 ±  6%  interrupts.CPU179.RES:Rescheduling_interrupts
    490538 ±  4%   +1096.0%    5866967 ±  7%  interrupts.CPU18.RES:Rescheduling_interrupts
    996653 ± 30%    +513.9%    6118261 ±  5%  interrupts.CPU180.RES:Rescheduling_interrupts
    982251 ± 25%    +488.5%    5780455 ±  2%  interrupts.CPU181.RES:Rescheduling_interrupts
    993814 ± 29%    +482.3%    5787372 ±  3%  interrupts.CPU182.RES:Rescheduling_interrupts
   1042657 ± 36%    +470.9%    5952536 ±  6%  interrupts.CPU183.RES:Rescheduling_interrupts
    978442 ± 31%    +526.4%    6129300 ±  5%  interrupts.CPU184.RES:Rescheduling_interrupts
    949788 ± 32%    +503.2%    5729381 ±  4%  interrupts.CPU185.RES:Rescheduling_interrupts
    981110 ± 33%    +458.8%    5482694 ±  4%  interrupts.CPU186.RES:Rescheduling_interrupts
    950926 ± 25%    +529.1%    5982248 ±  3%  interrupts.CPU187.RES:Rescheduling_interrupts
   1050599 ± 29%    +484.3%    6138778 ±  5%  interrupts.CPU188.RES:Rescheduling_interrupts
      8526           -29.4%       6023 ± 29%  interrupts.CPU189.NMI:Non-maskable_interrupts
      8526           -29.4%       6023 ± 29%  interrupts.CPU189.PMI:Performance_monitoring_interrupts
   1011122 ± 31%    +481.2%    5876995 ±  4%  interrupts.CPU189.RES:Rescheduling_interrupts
    524294 ±  6%    +989.5%    5712231 ±  9%  interrupts.CPU19.RES:Rescheduling_interrupts
   1001149 ± 33%    +501.1%    6017755 ±  8%  interrupts.CPU190.RES:Rescheduling_interrupts
    997852 ± 32%    +473.6%    5723617 ±  6%  interrupts.CPU191.RES:Rescheduling_interrupts
     23.50 ± 26%    +721.3%     193.00 ± 22%  interrupts.CPU191.TLB:TLB_shootdowns
    485334 ±  4%   +1059.8%    5628756 ±  9%  interrupts.CPU2.RES:Rescheduling_interrupts
    504676 ±  3%   +1037.6%    5741284 ±  9%  interrupts.CPU20.RES:Rescheduling_interrupts
    483900 ±  2%   +1063.0%    5627827 ±  9%  interrupts.CPU21.RES:Rescheduling_interrupts
    505544 ±  6%   +1056.8%    5848352 ± 14%  interrupts.CPU22.RES:Rescheduling_interrupts
    488580 ±  6%   +1087.5%    5801860 ± 13%  interrupts.CPU23.RES:Rescheduling_interrupts
   1207670 ± 42%    +374.0%    5724426 ± 13%  interrupts.CPU24.RES:Rescheduling_interrupts
   1108858 ± 44%    +394.2%    5480399 ± 13%  interrupts.CPU25.RES:Rescheduling_interrupts
   1122785 ± 55%    +424.0%    5883922 ± 14%  interrupts.CPU26.RES:Rescheduling_interrupts
   1087924 ± 48%    +433.9%    5808409 ± 15%  interrupts.CPU27.RES:Rescheduling_interrupts
   1064952 ± 49%    +418.6%    5522841 ± 11%  interrupts.CPU28.RES:Rescheduling_interrupts
   1057033 ± 49%    +426.7%    5567509 ± 12%  interrupts.CPU29.RES:Rescheduling_interrupts
    487823 ±  3%   +1054.5%    5631823 ±  5%  interrupts.CPU3.RES:Rescheduling_interrupts
   1114148 ± 54%    +396.8%    5534740 ± 13%  interrupts.CPU30.RES:Rescheduling_interrupts
   1052604 ± 41%    +458.9%    5883257 ± 16%  interrupts.CPU31.RES:Rescheduling_interrupts
   1011236 ± 45%    +459.4%    5657287 ± 16%  interrupts.CPU32.RES:Rescheduling_interrupts
   1009005 ± 44%    +446.3%    5512295 ± 12%  interrupts.CPU33.RES:Rescheduling_interrupts
   1128764 ± 52%    +410.4%    5760763 ± 14%  interrupts.CPU34.RES:Rescheduling_interrupts
   1069103 ± 46%    +443.8%    5814111 ± 12%  interrupts.CPU35.RES:Rescheduling_interrupts
   1069030 ± 51%    +441.5%    5788747 ±  8%  interrupts.CPU36.RES:Rescheduling_interrupts
   1059294 ± 48%    +432.5%    5640801 ± 15%  interrupts.CPU37.RES:Rescheduling_interrupts
   1052117 ± 51%    +406.0%    5323686 ± 10%  interrupts.CPU38.RES:Rescheduling_interrupts
   1088084 ± 49%    +432.3%    5791924 ± 13%  interrupts.CPU39.RES:Rescheduling_interrupts
    480050 ±  3%   +1117.3%    5843607 ±  5%  interrupts.CPU4.RES:Rescheduling_interrupts
   1043976 ± 51%    +451.3%    5755743 ± 14%  interrupts.CPU40.RES:Rescheduling_interrupts
    956710 ± 46%    +487.0%    5615549 ± 14%  interrupts.CPU41.RES:Rescheduling_interrupts
   1120581 ± 52%    +398.0%    5580666 ± 14%  interrupts.CPU42.RES:Rescheduling_interrupts
   1083516 ± 51%    +448.3%    5940937 ± 13%  interrupts.CPU43.RES:Rescheduling_interrupts
   1078552 ± 52%    +429.0%    5705095 ± 13%  interrupts.CPU44.RES:Rescheduling_interrupts
   1077502 ± 51%    +427.8%    5687045 ± 14%  interrupts.CPU45.RES:Rescheduling_interrupts
   1072840 ± 45%    +461.1%    6019663 ± 14%  interrupts.CPU46.RES:Rescheduling_interrupts
   1056804 ± 47%    +444.0%    5749092 ± 10%  interrupts.CPU47.RES:Rescheduling_interrupts
   1753909 ± 29%    +248.3%    6108580 ±  3%  interrupts.CPU48.RES:Rescheduling_interrupts
   1480042 ± 27%    +308.7%    6049550 ±  4%  interrupts.CPU49.RES:Rescheduling_interrupts
    491736 ±  2%   +1038.4%    5598137 ±  7%  interrupts.CPU5.RES:Rescheduling_interrupts
   1509361 ± 27%    +293.7%    5941712 ± 11%  interrupts.CPU50.RES:Rescheduling_interrupts
   1521435 ± 29%    +307.8%    6204597 ±  4%  interrupts.CPU51.RES:Rescheduling_interrupts
   1580968 ± 26%    +289.1%    6152108 ±  4%  interrupts.CPU52.RES:Rescheduling_interrupts
   1510871 ± 29%    +302.2%    6076833 ±  6%  interrupts.CPU53.RES:Rescheduling_interrupts
   1509986 ± 28%    +300.6%    6049414 ±  9%  interrupts.CPU54.RES:Rescheduling_interrupts
   1499979 ± 28%    +306.6%    6098823 ±  7%  interrupts.CPU55.RES:Rescheduling_interrupts
   1528868 ± 27%    +278.9%    5792590 ±  6%  interrupts.CPU56.RES:Rescheduling_interrupts
   1578019 ± 27%    +289.4%    6144830 ±  4%  interrupts.CPU57.RES:Rescheduling_interrupts
   1582115 ± 28%    +259.2%    5682355 ±  6%  interrupts.CPU58.RES:Rescheduling_interrupts
      8518           -40.4%       5081 ± 31%  interrupts.CPU59.NMI:Non-maskable_interrupts
      8518           -40.4%       5081 ± 31%  interrupts.CPU59.PMI:Performance_monitoring_interrupts
   1503611 ± 26%    +315.9%    6254138 ±  4%  interrupts.CPU59.RES:Rescheduling_interrupts
    487438 ±  3%   +1056.6%    5637787 ± 10%  interrupts.CPU6.RES:Rescheduling_interrupts
   1495788 ± 26%    +317.1%    6238878 ±  3%  interrupts.CPU60.RES:Rescheduling_interrupts
   1466923 ± 27%    +317.9%    6129881 ±  6%  interrupts.CPU61.RES:Rescheduling_interrupts
   1540817 ± 27%    +292.3%    6044889 ±  4%  interrupts.CPU62.RES:Rescheduling_interrupts
   1652998 ± 29%    +299.9%    6610760 ±  9%  interrupts.CPU63.RES:Rescheduling_interrupts
   1449028 ± 26%    +326.2%    6175689 ±  3%  interrupts.CPU64.RES:Rescheduling_interrupts
   1550115 ± 28%    +306.8%    6305134 ±  6%  interrupts.CPU65.RES:Rescheduling_interrupts
   1542106 ± 29%    +301.1%    6184838 ±  9%  interrupts.CPU66.RES:Rescheduling_interrupts
   1515954 ± 27%    +306.9%    6168014 ±  6%  interrupts.CPU67.RES:Rescheduling_interrupts
   1561220 ± 28%    +277.2%    5889284 ±  4%  interrupts.CPU68.RES:Rescheduling_interrupts
   1650264 ± 31%    +274.6%    6181732 ±  5%  interrupts.CPU69.RES:Rescheduling_interrupts
    495229 ±  4%    +999.8%    5446705 ±  6%  interrupts.CPU7.RES:Rescheduling_interrupts
   1475045 ± 23%    +342.9%    6533575 ±  2%  interrupts.CPU70.RES:Rescheduling_interrupts
   1484414 ± 24%    +311.3%    6105109 ±  8%  interrupts.CPU71.RES:Rescheduling_interrupts
      8514           -40.7%       5051 ± 30%  interrupts.CPU72.NMI:Non-maskable_interrupts
      8514           -40.7%       5051 ± 30%  interrupts.CPU72.PMI:Performance_monitoring_interrupts
   1155454 ± 39%    +405.9%    5845199 ±  4%  interrupts.CPU72.RES:Rescheduling_interrupts
    987077 ± 29%    +478.8%    5713592 ±  4%  interrupts.CPU73.RES:Rescheduling_interrupts
    964366 ± 27%    +497.6%    5763400 ±  2%  interrupts.CPU74.RES:Rescheduling_interrupts
    961248 ± 32%    +486.7%    5639959 ±  2%  interrupts.CPU75.RES:Rescheduling_interrupts
    995966 ± 30%    +490.9%    5885631 ±  3%  interrupts.CPU76.RES:Rescheduling_interrupts
   1005233 ± 31%    +485.2%    5882400        interrupts.CPU77.RES:Rescheduling_interrupts
    959462 ± 29%    +505.5%    5809467 ±  2%  interrupts.CPU78.RES:Rescheduling_interrupts
    994166 ± 28%    +489.8%    5863403 ±  4%  interrupts.CPU79.RES:Rescheduling_interrupts
    500101 ±  4%   +1019.7%    5599517 ±  6%  interrupts.CPU8.RES:Rescheduling_interrupts
    989893 ± 27%    +507.9%    6017257 ±  4%  interrupts.CPU80.RES:Rescheduling_interrupts
   1004821 ± 28%    +494.3%    5972034 ±  4%  interrupts.CPU81.RES:Rescheduling_interrupts
    988622 ± 31%    +480.3%    5736955 ±  5%  interrupts.CPU82.RES:Rescheduling_interrupts
    988783 ± 31%    +471.5%    5650762 ±  5%  interrupts.CPU83.RES:Rescheduling_interrupts
    975478 ± 31%    +505.2%    5903116 ±  5%  interrupts.CPU84.RES:Rescheduling_interrupts
    940795 ± 28%    +525.9%    5888432 ±  5%  interrupts.CPU85.RES:Rescheduling_interrupts
    965441 ± 30%    +504.8%    5838619 ±  4%  interrupts.CPU86.RES:Rescheduling_interrupts
   1057671 ± 32%    +454.1%    5860303 ±  5%  interrupts.CPU87.RES:Rescheduling_interrupts
    976112 ± 32%    +506.8%    5923371 ±  4%  interrupts.CPU88.RES:Rescheduling_interrupts
    965234 ± 31%    +499.0%    5781665 ±  8%  interrupts.CPU89.RES:Rescheduling_interrupts
    492362 ±  2%   +1052.9%    5676635 ±  7%  interrupts.CPU9.RES:Rescheduling_interrupts
    993161 ± 34%    +463.3%    5594037 ±  4%  interrupts.CPU90.RES:Rescheduling_interrupts
    956890 ± 26%    +518.4%    5916938 ±  3%  interrupts.CPU91.RES:Rescheduling_interrupts
   1038002 ± 29%    +484.0%    6062325 ±  6%  interrupts.CPU92.RES:Rescheduling_interrupts
   1015625 ± 33%    +485.3%    5944864 ±  4%  interrupts.CPU93.RES:Rescheduling_interrupts
    971893 ± 35%    +529.0%    6113465 ±  8%  interrupts.CPU94.RES:Rescheduling_interrupts
   1009337 ± 33%    +479.7%    5850700 ±  2%  interrupts.CPU95.RES:Rescheduling_interrupts
    551752 ±  4%   +1005.6%    6100041 ± 10%  interrupts.CPU96.RES:Rescheduling_interrupts
    470128 ±  8%   +1119.2%    5731950 ± 11%  interrupts.CPU97.RES:Rescheduling_interrupts
    489000 ±  4%   +1058.5%    5665028 ±  9%  interrupts.CPU98.RES:Rescheduling_interrupts
    486080 ±  2%   +1054.9%    5613880 ±  6%  interrupts.CPU99.RES:Rescheduling_interrupts
 1.978e+08 ±  7%    +470.0%  1.128e+09 ±  5%  interrupts.RES:Rescheduling_interrupts
      1580 ± 28%     +79.0%       2828 ± 20%  interrupts.TLB:TLB_shootdowns
     50.53           -38.0       12.50 ±101%  perf-profile.calltrace.cycles-pp.__GI___libc_read
     49.31           -37.6       11.75 ±101%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
     49.36           -37.6       11.81 ±101%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read
     48.70           -37.3       11.45 ±101%  perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
     48.53           -37.3       11.28 ±101%  perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
     48.44           -32.9       15.56 ±101%  perf-profile.calltrace.cycles-pp.__GI___libc_write
     47.80           -32.8       15.04 ±101%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
     47.84           -32.7       15.10 ±101%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write
     46.09           -32.0       14.11 ±101%  perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
     45.83           -31.6       14.27 ±101%  perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
     28.21           -28.0        0.25 ±173%  perf-profile.calltrace.cycles-pp.common_file_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64
     28.29           -27.8        0.44 ±112%  perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
     27.87           -27.5        0.35 ±106%  perf-profile.calltrace.cycles-pp.common_file_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64
     27.96           -27.3        0.70 ± 76%  perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
     26.82           -26.8        0.00        perf-profile.calltrace.cycles-pp.aa_file_perm.common_file_perm.security_file_permission.vfs_write.ksys_write
     26.45           -26.4        0.00        perf-profile.calltrace.cycles-pp.aa_file_perm.common_file_perm.security_file_permission.vfs_read.ksys_read
      1.73 ±  2%      -1.1        0.61 ± 75%  perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__sched_text_start.schedule.schedule_timeout.unix_stream_read_generic
      1.61 ±  2%      -1.0        0.59 ±100%  perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
      1.59 ±  2%      -1.0        0.58 ±100%  perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
      2.19 ±  2%      -0.8        1.35 ± 23%  perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
      0.00            +1.1        1.09 ± 51%  perf-profile.calltrace.cycles-pp.__slab_free.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      0.00            +1.2        1.18 ± 32%  perf-profile.calltrace.cycles-pp._raw_spin_lock.get_partial_node.___slab_alloc.__slab_alloc.kmem_cache_alloc_node
      0.54 ±  3%      +1.3        1.88 ± 37%  perf-profile.calltrace.cycles-pp.__slab_free.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      0.00            +1.4        1.39 ± 32%  perf-profile.calltrace.cycles-pp.get_partial_node.___slab_alloc.__slab_alloc.kmem_cache_alloc_node.__alloc_skb
      1.28            +1.5        2.76 ± 31%  perf-profile.calltrace.cycles-pp.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb
      0.00            +1.6        1.57 ± 25%  perf-profile.calltrace.cycles-pp._raw_spin_lock.get_partial_node.___slab_alloc.__slab_alloc.__kmalloc_node_track_caller
      1.40            +1.6        3.03 ± 30%  perf-profile.calltrace.cycles-pp.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic
      1.00            +1.7        2.69 ± 27%  perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
      0.00            +1.7        1.71 ± 30%  perf-profile.calltrace.cycles-pp.___slab_alloc.__slab_alloc.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
      1.41            +1.7        3.13 ± 30%  perf-profile.calltrace.cycles-pp.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
      1.41            +1.7        3.16 ± 30%  perf-profile.calltrace.cycles-pp.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      0.00            +1.7        1.75 ± 30%  perf-profile.calltrace.cycles-pp.__slab_alloc.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
      0.57 ±  3%      +1.8        2.34 ± 61%  perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
      0.00            +1.8        1.78 ± 27%  perf-profile.calltrace.cycles-pp.get_partial_node.___slab_alloc.__slab_alloc.__kmalloc_node_track_caller.__kmalloc_reserve
      1.31            +1.8        3.14 ± 22%  perf-profile.calltrace.cycles-pp.skb_set_owner_w.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
      0.58 ±  2%      +1.9        2.45 ± 60%  perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
      0.00            +2.0        2.03 ± 43%  perf-profile.calltrace.cycles-pp.__check_object_size.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
      0.00            +2.1        2.10 ± 26%  perf-profile.calltrace.cycles-pp.___slab_alloc.__slab_alloc.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb
      0.00            +2.1        2.10 ± 43%  perf-profile.calltrace.cycles-pp.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic
      0.00            +2.1        2.14 ± 26%  perf-profile.calltrace.cycles-pp.__slab_alloc.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
      0.00            +2.3        2.33 ± 38%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg
      0.00            +2.3        2.34 ± 44%  perf-profile.calltrace.cycles-pp.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
      0.56 ±  2%      +2.4        2.95 ± 32%  perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg.sock_sendmsg
      0.71 ±  3%      +2.5        3.17 ± 50%  perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic
      0.65 ±  4%      +2.5        3.14 ± 13%  perf-profile.calltrace.cycles-pp.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
      0.00            +2.6        2.59 ± 30%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.get_partial_node.___slab_alloc.__slab_alloc
      0.00            +2.6        2.63 ± 16%  perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
      0.00            +3.1        3.15 ± 17%  perf-profile.calltrace.cycles-pp.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
      0.00            +3.3        3.29 ± 17%  perf-profile.calltrace.cycles-pp.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
      1.11 ±  2%      +4.4        5.52 ± 44%  perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg
      1.12 ±  2%      +4.5        5.59 ± 44%  perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      1.13 ±  2%      +4.5        5.63 ± 43%  perf-profile.calltrace.cycles-pp.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
      2.40            +4.6        7.04 ± 24%  perf-profile.calltrace.cycles-pp.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
      0.89 ±  4%      +6.4        7.25 ± 14%  perf-profile.calltrace.cycles-pp.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
      0.90 ±  4%      +6.5        7.39 ± 14%  perf-profile.calltrace.cycles-pp.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
      2.33 ±  2%     +10.3       12.63 ± 14%  perf-profile.calltrace.cycles-pp.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
     19.31           +15.9       35.19 ±  2%  perf-profile.calltrace.cycles-pp.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read
     19.35           +16.1       35.48 ±  2%  perf-profile.calltrace.cycles-pp.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read
     20.15           +16.7       36.84 ±  2%  perf-profile.calltrace.cycles-pp.sock_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64
     20.32           +17.1       37.41 ±  3%  perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00           +28.2       28.25 ± 46%  perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
     16.82           +28.4       45.22 ±  5%  perf-profile.calltrace.cycles-pp.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write.ksys_write
     15.88           +28.5       44.41 ±  6%  perf-profile.calltrace.cycles-pp.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write
      0.00           +28.8       28.76 ± 46%  perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
     16.88           +29.0       45.84 ±  5%  perf-profile.calltrace.cycles-pp.sock_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
     17.11           +29.2       46.34 ±  4%  perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00           +34.0       34.04 ± 38%  perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00           +34.6       34.56 ± 39%  perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00           +66.0       65.97 ± 42%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00           +66.3       66.31 ± 42%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
     56.46           -55.3        1.17 ± 38%  perf-profile.children.cycles-pp.common_file_perm
     56.61           -55.0        1.63 ± 40%  perf-profile.children.cycles-pp.security_file_permission
     54.56           -54.3        0.25 ± 47%  perf-profile.children.cycles-pp.aa_file_perm
     50.60           -38.0       12.58 ±101%  perf-profile.children.cycles-pp.__GI___libc_read
     48.98           -33.3       15.66 ±101%  perf-profile.children.cycles-pp.__GI___libc_write
     48.57            -9.0       39.59 ±  4%  perf-profile.children.cycles-pp.vfs_read
     48.73            -8.5       40.22 ±  4%  perf-profile.children.cycles-pp.ksys_read
     97.69            -4.8       92.90        perf-profile.children.cycles-pp.do_syscall_64
     97.77            -4.5       93.31        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      2.49 ±  2%      -1.1        1.42 ± 33%  perf-profile.children.cycles-pp.switch_mm_irqs_off
      0.76 ±  2%      -0.4        0.33 ± 22%  perf-profile.children.cycles-pp.mutex_unlock
      1.49 ±  6%      -0.3        1.23 ±  8%  perf-profile.children.cycles-pp.apic_timer_interrupt
      0.86 ±  2%      -0.2        0.63 ± 29%  perf-profile.children.cycles-pp.native_write_msr
      0.30 ±  2%      -0.2        0.08 ± 23%  perf-profile.children.cycles-pp.preempt_schedule_common
      0.18 ±  3%      -0.1        0.07 ± 58%  perf-profile.children.cycles-pp.resched_curr
      0.05            +0.0        0.09 ± 16%  perf-profile.children.cycles-pp.get_xsave_addr
      0.07 ±  6%      +0.0        0.11 ± 22%  perf-profile.children.cycles-pp.maybe_add_creds
      0.00            +0.1        0.07 ± 26%  perf-profile.children.cycles-pp.__vfs_read
      0.00            +0.1        0.07 ± 31%  perf-profile.children.cycles-pp.__x64_sys_read
      0.00            +0.1        0.07 ± 38%  perf-profile.children.cycles-pp.kmalloc_slab
      0.04 ± 58%      +0.1        0.11 ± 31%  perf-profile.children.cycles-pp.ktime_get_update_offsets_now
      0.00            +0.1        0.08 ± 16%  perf-profile.children.cycles-pp.unix_scm_to_skb
      0.00            +0.1        0.08 ± 23%  perf-profile.children.cycles-pp.__x64_sys_write
      0.00            +0.1        0.08 ± 54%  perf-profile.children.cycles-pp.iov_iter_init
      0.00            +0.1        0.09 ± 31%  perf-profile.children.cycles-pp.put_pid
      0.00            +0.1        0.09 ± 45%  perf-profile.children.cycles-pp.apparmor_socket_sendmsg
      0.04 ± 58%      +0.1        0.13 ± 30%  perf-profile.children.cycles-pp.ktime_get
      0.00            +0.1        0.09 ± 42%  perf-profile.children.cycles-pp.apparmor_socket_recvmsg
      0.00            +0.1        0.10 ± 40%  perf-profile.children.cycles-pp.apparmor_socket_getpeersec_dgram
      0.13 ±  3%      +0.1        0.24 ±  6%  perf-profile.children.cycles-pp.kfree
      0.35 ±  2%      +0.1        0.46 ± 22%  perf-profile.children.cycles-pp._cond_resched
      0.17 ±  2%      +0.1        0.28 ± 26%  perf-profile.children.cycles-pp.__list_del_entry_valid
      0.00            +0.1        0.12 ± 44%  perf-profile.children.cycles-pp.apparmor_file_permission
      0.00            +0.1        0.15 ± 32%  perf-profile.children.cycles-pp.fpregs_assert_state_consistent
      0.03 ±100%      +0.2        0.19 ± 39%  perf-profile.children.cycles-pp.native_irq_return_iret
      0.00            +0.2        0.17 ± 40%  perf-profile.children.cycles-pp.check_stack_object
      0.00            +0.2        0.19 ± 31%  perf-profile.children.cycles-pp.rcu_all_qs
      0.00            +0.2        0.20 ± 37%  perf-profile.children.cycles-pp.security_socket_getpeersec_dgram
      0.68 ±  2%      +0.2        0.89 ± 12%  perf-profile.children.cycles-pp.mutex_lock
      0.16 ± 10%      +0.2        0.38 ± 25%  perf-profile.children.cycles-pp.task_tick_fair
      0.19 ± 11%      +0.2        0.42 ± 22%  perf-profile.children.cycles-pp.scheduler_tick
      0.00            +0.2        0.23 ± 53%  perf-profile.children.cycles-pp.find_next_bit
      0.00            +0.2        0.24 ± 36%  perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
      0.23 ±  9%      +0.3        0.55 ± 25%  perf-profile.children.cycles-pp.update_process_times
      0.23 ±  2%      +0.3        0.55 ± 18%  perf-profile.children.cycles-pp.__ksize
      0.18 ±  2%      +0.3        0.52 ± 30%  perf-profile.children.cycles-pp.__fsnotify_parent
      0.01 ±173%      +0.3        0.36 ± 43%  perf-profile.children.cycles-pp.wait_for_unix_gc
      0.23 ±  9%      +0.3        0.58 ± 29%  perf-profile.children.cycles-pp.tick_sched_handle
      0.00            +0.4        0.37 ± 67%  perf-profile.children.cycles-pp.cpumask_next_wrap
      0.33 ± 10%      +0.4        0.70 ± 25%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.25 ±  9%      +0.4        0.63 ± 27%  perf-profile.children.cycles-pp.tick_sched_timer
      0.44 ±  2%      +0.4        0.87 ± 52%  perf-profile.children.cycles-pp.unix_write_space
      0.09 ±  4%      +0.4        0.53 ± 46%  perf-profile.children.cycles-pp.__might_fault
      0.46 ±  9%      +0.5        0.93 ± 14%  perf-profile.children.cycles-pp.hrtimer_interrupt
      0.68 ±  6%      +0.5        1.15 ±  9%  perf-profile.children.cycles-pp.smp_apic_timer_interrupt
      0.06 ±  7%      +0.5        0.58 ± 39%  perf-profile.children.cycles-pp.__virt_addr_valid
      0.07 ±  6%      +0.6        0.62 ± 42%  perf-profile.children.cycles-pp.__might_sleep
      0.20 ±  2%      +0.6        0.81 ± 31%  perf-profile.children.cycles-pp.fsnotify
      0.12 ±  4%      +0.6        0.75 ± 30%  perf-profile.children.cycles-pp.entry_SYSCALL_64
      0.06 ±  6%      +0.6        0.70 ± 42%  perf-profile.children.cycles-pp.__check_heap_object
      0.87 ±  2%      +0.6        1.51 ± 14%  perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
      0.23 ±  2%      +0.7        0.89 ± 38%  perf-profile.children.cycles-pp.__fget_light
      0.10 ±  4%      +0.7        0.81 ± 50%  perf-profile.children.cycles-pp.copyin
      0.25 ±  3%      +0.7        0.99 ± 39%  perf-profile.children.cycles-pp.__fdget_pos
      0.21 ±  6%      +0.7        0.95 ± 60%  perf-profile.children.cycles-pp.skb_release_data
      1.13 ±  2%      +0.8        1.94 ± 30%  perf-profile.children.cycles-pp.prepare_to_wait
      0.24            +0.9        1.17 ± 45%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      0.43 ±  2%      +1.0        1.39 ± 29%  perf-profile.children.cycles-pp.___might_sleep
      0.12 ±  4%      +1.2        1.34 ± 49%  perf-profile.children.cycles-pp._copy_from_iter
      1.29            +1.5        2.77 ± 31%  perf-profile.children.cycles-pp.sock_wfree
      1.40            +1.7        3.06 ± 30%  perf-profile.children.cycles-pp.unix_destruct_scm
      0.04 ± 57%      +1.7        1.71 ± 44%  perf-profile.children.cycles-pp.unfreeze_partials
      1.41            +1.7        3.13 ± 30%  perf-profile.children.cycles-pp.skb_release_head_state
      0.06 ± 13%      +1.7        1.80 ± 43%  perf-profile.children.cycles-pp.put_cpu_partial
      1.41            +1.7        3.16 ± 30%  perf-profile.children.cycles-pp.skb_release_all
      1.33            +1.8        3.15 ± 22%  perf-profile.children.cycles-pp.skb_set_owner_w
      0.39 ±  3%      +1.8        2.23 ± 36%  perf-profile.children.cycles-pp.simple_copy_to_iter
      0.58 ±  2%      +2.0        2.57 ± 55%  perf-profile.children.cycles-pp.copyout
      0.29            +2.1        2.37 ± 44%  perf-profile.children.cycles-pp.skb_copy_datagram_from_iter
      0.94 ±  3%      +2.3        3.28 ± 31%  perf-profile.children.cycles-pp.__slab_free
      0.29 ±  8%      +2.5        2.77 ± 14%  perf-profile.children.cycles-pp.kmem_cache_alloc_node
      0.71 ±  3%      +2.5        3.19 ± 49%  perf-profile.children.cycles-pp._copy_to_iter
      0.65 ±  4%      +2.5        3.16 ± 13%  perf-profile.children.cycles-pp.kmem_cache_free
     46.33            +2.5       48.86 ±  2%  perf-profile.children.cycles-pp.ksys_write
      0.67            +2.6        3.25 ± 54%  perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      0.47 ±  2%      +2.6        3.07 ± 37%  perf-profile.children.cycles-pp.__check_object_size
      0.30 ±  8%      +3.0        3.30 ± 14%  perf-profile.children.cycles-pp.__kmalloc_node_track_caller
      0.32 ±  8%      +3.1        3.42 ± 14%  perf-profile.children.cycles-pp.__kmalloc_reserve
      0.13 ± 17%      +3.3        3.43 ± 22%  perf-profile.children.cycles-pp.get_partial_node
      3.81            +3.6        7.45 ± 22%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      0.35 ± 11%      +3.8        4.14 ± 20%  perf-profile.children.cycles-pp.___slab_alloc
      0.40 ± 11%      +3.9        4.25 ± 20%  perf-profile.children.cycles-pp.__slab_alloc
      1.11 ±  3%      +4.4        5.53 ± 44%  perf-profile.children.cycles-pp.__skb_datagram_iter
      1.12 ±  3%      +4.5        5.60 ± 44%  perf-profile.children.cycles-pp.skb_copy_datagram_iter
      1.13 ±  2%      +4.5        5.64 ± 43%  perf-profile.children.cycles-pp.unix_stream_read_actor
      2.40            +4.7        7.05 ± 24%  perf-profile.children.cycles-pp.consume_skb
      0.64 ±  3%      +5.2        5.83 ± 75%  perf-profile.children.cycles-pp.update_cfs_group
      5.75            +6.4       12.12 ± 16%  perf-profile.children.cycles-pp._raw_spin_lock
      0.89 ±  4%      +6.4        7.28 ± 14%  perf-profile.children.cycles-pp.__alloc_skb
      0.90 ±  4%      +6.5        7.40 ± 14%  perf-profile.children.cycles-pp.alloc_skb_with_frags
      1.57 ±  3%      +9.4       11.02 ± 27%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      2.33 ±  2%     +10.3       12.66 ± 14%  perf-profile.children.cycles-pp.sock_alloc_send_pskb
     19.34           +15.9       35.27 ±  2%  perf-profile.children.cycles-pp.unix_stream_read_generic
     19.35           +16.1       35.49 ±  2%  perf-profile.children.cycles-pp.unix_stream_recvmsg
     20.15           +16.7       36.85 ±  2%  perf-profile.children.cycles-pp.sock_read_iter
     20.33           +17.1       37.44 ±  3%  perf-profile.children.cycles-pp.new_sync_read
     16.82           +28.4       45.25 ±  5%  perf-profile.children.cycles-pp.sock_sendmsg
     15.90           +28.6       44.48 ±  6%  perf-profile.children.cycles-pp.unix_stream_sendmsg
     16.88           +29.0       45.84 ±  5%  perf-profile.children.cycles-pp.sock_write_iter
     17.25           +29.1       46.37 ±  4%  perf-profile.children.cycles-pp.new_sync_write
     54.03           -53.8        0.23 ± 47%  perf-profile.self.cycles-pp.aa_file_perm
      2.15 ±  2%      -1.4        0.71 ± 21%  perf-profile.self.cycles-pp.switch_mm_irqs_off
      1.88            -1.0        0.90 ± 36%  perf-profile.self.cycles-pp.common_file_perm
      1.28            -0.7        0.59 ± 45%  perf-profile.self.cycles-pp.aa_sk_perm
      0.75 ±  3%      -0.4        0.32 ± 23%  perf-profile.self.cycles-pp.mutex_unlock
      0.85 ±  2%      -0.2        0.63 ± 29%  perf-profile.self.cycles-pp.native_write_msr
      0.18 ±  3%      -0.1        0.07 ± 58%  perf-profile.self.cycles-pp.resched_curr
      0.08            +0.1        0.14 ± 39%  perf-profile.self.cycles-pp.skb_copy_datagram_from_iter
      0.03 ±100%      +0.1        0.08 ± 19%  perf-profile.self.cycles-pp.get_xsave_addr
      0.00            +0.1        0.07 ± 21%  perf-profile.self.cycles-pp.__x64_sys_write
      0.00            +0.1        0.07 ± 28%  perf-profile.self.cycles-pp.skb_copy_datagram_iter
      0.00            +0.1        0.07 ± 10%  perf-profile.self.cycles-pp.skb_release_head_state
      0.00            +0.1        0.07 ± 17%  perf-profile.self.cycles-pp.unix_scm_to_skb
      0.06 ±  6%      +0.1        0.14 ± 26%  perf-profile.self.cycles-pp.activate_task
      0.00            +0.1        0.07 ± 42%  perf-profile.self.cycles-pp.copyin
      0.00            +0.1        0.08 ± 40%  perf-profile.self.cycles-pp.apparmor_socket_sendmsg
      0.00            +0.1        0.08 ± 46%  perf-profile.self.cycles-pp.iov_iter_init
      0.00            +0.1        0.08 ± 36%  perf-profile.self.cycles-pp.sock_recvmsg
      0.06 ±  7%      +0.1        0.14 ± 28%  perf-profile.self.cycles-pp.migrate_task_rq_fair
      0.03 ±100%      +0.1        0.11 ± 30%  perf-profile.self.cycles-pp.ktime_get_update_offsets_now
      0.00            +0.1        0.08 ± 48%  perf-profile.self.cycles-pp.sock_sendmsg
      0.00            +0.1        0.09 ± 37%  perf-profile.self.cycles-pp.security_socket_recvmsg
      0.00            +0.1        0.09 ± 37%  perf-profile.self.cycles-pp.apparmor_socket_getpeersec_dgram
      0.00            +0.1        0.09 ± 37%  perf-profile.self.cycles-pp.apparmor_socket_recvmsg
      0.09 ±  9%      +0.1        0.17 ±  6%  perf-profile.self.cycles-pp.schedule_timeout
      0.00            +0.1        0.10 ± 46%  perf-profile.self.cycles-pp.wait_for_unix_gc
      0.00            +0.1        0.10 ± 35%  perf-profile.self.cycles-pp.security_socket_getpeersec_dgram
      0.01 ±173%      +0.1        0.11 ± 36%  perf-profile.self.cycles-pp.ktime_get
      0.13            +0.1        0.23 ±  6%  perf-profile.self.cycles-pp.kfree
      0.00            +0.1        0.10 ± 34%  perf-profile.self.cycles-pp.schedule
      0.00            +0.1        0.11 ± 39%  perf-profile.self.cycles-pp.__skb_datagram_iter
      0.00            +0.1        0.11 ± 38%  perf-profile.self.cycles-pp.__kmalloc_reserve
      0.00            +0.1        0.11 ± 42%  perf-profile.self.cycles-pp.apparmor_file_permission
      0.00            +0.1        0.11 ± 45%  perf-profile.self.cycles-pp.__fdget_pos
      0.15 ±  2%      +0.1        0.27 ± 26%  perf-profile.self.cycles-pp.__list_del_entry_valid
      0.00            +0.1        0.12 ± 34%  perf-profile.self.cycles-pp.alloc_skb_with_frags
      0.00            +0.1        0.12 ± 55%  perf-profile.self.cycles-pp.skb_unlink
      0.06 ±  7%      +0.1        0.18 ± 47%  perf-profile.self.cycles-pp.__might_fault
      0.00            +0.1        0.13 ± 34%  perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
      0.00            +0.1        0.14 ± 39%  perf-profile.self.cycles-pp.ksys_write
      0.08 ±  5%      +0.1        0.21 ± 41%  perf-profile.self.cycles-pp.security_file_permission
      0.00            +0.1        0.14 ± 31%  perf-profile.self.cycles-pp.fpregs_assert_state_consistent
      0.22 ±  4%      +0.1        0.36 ± 24%  perf-profile.self.cycles-pp.consume_skb
      0.00            +0.1        0.14 ± 31%  perf-profile.self.cycles-pp.rcu_all_qs
      0.00            +0.2        0.15 ± 41%  perf-profile.self.cycles-pp.check_stack_object
      0.10 ±  7%      +0.2        0.26 ± 25%  perf-profile.self.cycles-pp.unix_destruct_scm
      0.03 ±100%      +0.2        0.19 ± 39%  perf-profile.self.cycles-pp.native_irq_return_iret
      0.00            +0.2        0.17 ± 21%  perf-profile.self.cycles-pp.ksys_read
      0.00            +0.2        0.18 ± 34%  perf-profile.self.cycles-pp._cond_resched
      0.00            +0.2        0.21 ± 53%  perf-profile.self.cycles-pp.find_next_bit
      0.05            +0.2        0.26 ± 48%  perf-profile.self.cycles-pp.finish_task_switch
      0.00            +0.2        0.21 ± 31%  perf-profile.self.cycles-pp.unix_stream_recvmsg
      0.23 ±  6%      +0.2        0.47 ± 45%  perf-profile.self.cycles-pp.new_sync_write
      0.00            +0.3        0.28 ± 42%  perf-profile.self.cycles-pp._copy_from_iter
      0.07 ± 12%      +0.3        0.35 ± 17%  perf-profile.self.cycles-pp._copy_to_iter
      0.04 ± 58%      +0.3        0.33 ± 25%  perf-profile.self.cycles-pp.get_partial_node
      0.00            +0.3        0.30 ± 28%  perf-profile.self.cycles-pp.unfreeze_partials
      0.26            +0.3        0.56 ± 30%  perf-profile.self.cycles-pp.try_to_wake_up
      0.17 ±  2%      +0.3        0.47 ± 32%  perf-profile.self.cycles-pp.__fsnotify_parent
      0.07            +0.3        0.38 ± 19%  perf-profile.self.cycles-pp.kmem_cache_free
      0.23 ±  2%      +0.3        0.54 ± 18%  perf-profile.self.cycles-pp.__ksize
      0.08 ±  5%      +0.3        0.41 ± 28%  perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.15 ±  5%      +0.3        0.49 ± 20%  perf-profile.self.cycles-pp.vfs_read
      0.17 ±  3%      +0.3        0.51 ± 22%  perf-profile.self.cycles-pp.new_sync_read
      0.08 ±  5%      +0.4        0.43 ± 22%  perf-profile.self.cycles-pp.sock_alloc_send_pskb
      0.06 ±  7%      +0.4        0.45 ± 32%  perf-profile.self.cycles-pp.kmem_cache_alloc_node
      0.12 ±  5%      +0.4        0.53 ± 21%  perf-profile.self.cycles-pp.do_syscall_64
      0.07 ±  5%      +0.5        0.53 ± 30%  perf-profile.self.cycles-pp.vfs_write
      0.06 ±  6%      +0.5        0.53 ± 35%  perf-profile.self.cycles-pp.__kmalloc_node_track_caller
      0.22 ±  7%      +0.5        0.70 ± 17%  perf-profile.self.cycles-pp.___slab_alloc
      0.06            +0.5        0.55 ± 37%  perf-profile.self.cycles-pp.sock_read_iter
      0.06            +0.5        0.55 ± 44%  perf-profile.self.cycles-pp.__might_sleep
      0.06 ±  7%      +0.5        0.55 ± 39%  perf-profile.self.cycles-pp.__virt_addr_valid
      0.04 ± 57%      +0.5        0.54 ± 41%  perf-profile.self.cycles-pp.__alloc_skb
      0.06 ±  6%      +0.5        0.59 ± 89%  perf-profile.self.cycles-pp.set_task_cpu
      0.06            +0.5        0.59 ± 43%  perf-profile.self.cycles-pp.sock_write_iter
      0.07 ±  6%      +0.5        0.60 ± 44%  perf-profile.self.cycles-pp.entry_SYSCALL_64
      0.20 ±  4%      +0.6        0.78 ± 32%  perf-profile.self.cycles-pp.fsnotify
      0.06            +0.6        0.68 ± 42%  perf-profile.self.cycles-pp.__check_heap_object
      0.22 ±  3%      +0.6        0.85 ± 39%  perf-profile.self.cycles-pp.__fget_light
      0.21 ±  5%      +0.7        0.93 ± 62%  perf-profile.self.cycles-pp.skb_release_data
      1.91            +0.8        2.67 ± 15%  perf-profile.self.cycles-pp.unix_stream_read_generic
      0.42            +0.9        1.33 ± 30%  perf-profile.self.cycles-pp.___might_sleep
      0.23 ±  2%      +0.9        1.16 ± 45%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      0.84            +1.0        1.79 ± 25%  perf-profile.self.cycles-pp.sock_wfree
      0.33 ±  3%      +1.3        1.59 ± 35%  perf-profile.self.cycles-pp.__check_object_size
      1.31            +1.8        3.08 ± 22%  perf-profile.self.cycles-pp.skb_set_owner_w
      0.49 ±  8%      +1.9        2.40 ± 42%  perf-profile.self.cycles-pp.unix_stream_sendmsg
      0.92 ±  3%      +2.2        3.16 ± 33%  perf-profile.self.cycles-pp.__slab_free
      0.66            +2.5        3.15 ± 55%  perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      0.64 ±  3%      +5.2        5.83 ± 75%  perf-profile.self.cycles-pp.update_cfs_group
      1.57 ±  3%      +9.4       10.99 ± 27%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
    234844 ±  2%     +18.2%     277606 ±  2%  softirqs.CPU0.RCU
     15397 ±  7%     +93.8%      29837 ±  7%  softirqs.CPU0.SCHED
    239346 ±  3%     +16.8%     279600 ±  2%  softirqs.CPU1.RCU
      7077 ±  8%    +208.8%      21853 ±  2%  softirqs.CPU1.SCHED
    236892 ±  2%     +17.0%     277260 ±  2%  softirqs.CPU10.RCU
      7341 ±  3%    +192.3%      21463 ±  3%  softirqs.CPU10.SCHED
    236061 ±  2%     +17.5%     277270 ±  2%  softirqs.CPU100.RCU
      7219          +206.3%      22116 ±  4%  softirqs.CPU100.SCHED
    236408 ±  2%     +16.9%     276411 ±  2%  softirqs.CPU101.RCU
      7241 ±  2%    +204.5%      22049        softirqs.CPU101.SCHED
    236496 ±  2%     +17.1%     276987        softirqs.CPU102.RCU
      7366 ±  2%    +203.5%      22360 ±  5%  softirqs.CPU102.SCHED
    236289 ±  2%     +17.1%     276637 ±  2%  softirqs.CPU103.RCU
      7353 ±  3%    +201.8%      22193 ±  3%  softirqs.CPU103.SCHED
    236158 ±  2%     +17.1%     276469 ±  2%  softirqs.CPU104.RCU
      7027          +205.0%      21435 ±  7%  softirqs.CPU104.SCHED
    236946 ±  2%     +16.6%     276271 ±  2%  softirqs.CPU105.RCU
      7308 ±  6%    +209.8%      22639 ±  5%  softirqs.CPU105.SCHED
    236216 ±  2%     +17.2%     276911 ±  2%  softirqs.CPU106.RCU
      7116 ±  3%    +199.4%      21302 ±  4%  softirqs.CPU106.SCHED
    236934 ±  2%     +17.0%     277182 ±  2%  softirqs.CPU107.RCU
      7096 ±  3%    +213.5%      22248 ±  6%  softirqs.CPU107.SCHED
    236294 ±  2%     +16.9%     276290 ±  2%  softirqs.CPU108.RCU
      7153 ±  3%    +196.5%      21211 ±  6%  softirqs.CPU108.SCHED
    237018 ±  2%     +16.6%     276372 ±  2%  softirqs.CPU109.RCU
      7286 ±  7%    +192.9%      21343 ±  5%  softirqs.CPU109.SCHED
    238863 ±  2%     +16.1%     277226 ±  2%  softirqs.CPU11.RCU
      7658 ± 10%    +190.1%      22213 ±  8%  softirqs.CPU11.SCHED
    236455 ±  2%     +17.0%     276557 ±  2%  softirqs.CPU110.RCU
      7127 ±  3%    +204.6%      21711 ±  5%  softirqs.CPU110.SCHED
    236646 ±  2%     +17.5%     278121 ±  2%  softirqs.CPU111.RCU
      7225 ±  6%    +197.7%      21511 ±  2%  softirqs.CPU111.SCHED
    236309 ±  2%     +17.5%     277672 ±  2%  softirqs.CPU112.RCU
      7146 ±  3%    +202.2%      21599 ±  4%  softirqs.CPU112.SCHED
    235846 ±  2%     +17.7%     277621 ±  2%  softirqs.CPU113.RCU
      6962          +208.4%      21470 ±  7%  softirqs.CPU113.SCHED
    236118 ±  2%     +17.3%     276903 ±  2%  softirqs.CPU114.RCU
      7228          +192.1%      21114 ±  2%  softirqs.CPU114.SCHED
    236822 ±  2%     +17.0%     276986 ±  2%  softirqs.CPU115.RCU
      7044 ±  2%    +214.0%      22116 ±  8%  softirqs.CPU115.SCHED
    237467 ±  2%     +16.7%     277108 ±  2%  softirqs.CPU116.RCU
      7292 ±  7%    +200.4%      21908 ±  3%  softirqs.CPU116.SCHED
    236802 ±  2%     +16.9%     276838 ±  2%  softirqs.CPU117.RCU
      7622 ±  6%    +183.7%      21624 ±  6%  softirqs.CPU117.SCHED
    236593 ±  2%     +17.2%     277285 ±  2%  softirqs.CPU118.RCU
      7246 ±  3%    +195.9%      21438 ±  5%  softirqs.CPU118.SCHED
    236113 ±  2%     +17.3%     276854 ±  2%  softirqs.CPU119.RCU
      7355          +195.2%      21715 ±  6%  softirqs.CPU119.SCHED
    237260 ±  2%     +16.9%     277299 ±  2%  softirqs.CPU12.RCU
      7279 ±  5%    +195.7%      21522 ±  6%  softirqs.CPU12.SCHED
    238003 ±  3%     +16.5%     277226 ±  2%  softirqs.CPU120.RCU
     11696 ±  8%     +71.0%      20003 ±  3%  softirqs.CPU120.SCHED
    239663 ±  3%     +15.7%     277171 ±  2%  softirqs.CPU121.RCU
     11411 ±  7%     +90.2%      21701 ±  3%  softirqs.CPU121.SCHED
    240253 ±  2%     +15.4%     277215 ±  2%  softirqs.CPU122.RCU
     11276 ±  8%     +87.3%      21124 ±  3%  softirqs.CPU122.SCHED
    239766 ±  2%     +15.6%     277121 ±  2%  softirqs.CPU123.RCU
     11374 ±  8%     +97.0%      22409 ±  3%  softirqs.CPU123.SCHED
    239890 ±  2%     +15.3%     276689 ±  2%  softirqs.CPU124.RCU
     11669 ± 11%     +81.8%      21217 ±  4%  softirqs.CPU124.SCHED
    239821 ±  2%     +15.5%     277057 ±  2%  softirqs.CPU125.RCU
     11107 ± 11%    +101.8%      22411 ±  3%  softirqs.CPU125.SCHED
    239887 ±  3%     +15.5%     277083 ±  2%  softirqs.CPU126.RCU
     11467 ±  9%     +88.5%      21618 ±  5%  softirqs.CPU126.SCHED
    239656 ±  2%     +16.1%     278134 ±  2%  softirqs.CPU127.RCU
     11297 ±  9%     +87.6%      21196 ±  7%  softirqs.CPU127.SCHED
    239590 ±  2%     +15.5%     276712 ±  2%  softirqs.CPU128.RCU
     11441 ± 11%     +84.5%      21115        softirqs.CPU128.SCHED
    239832 ±  2%     +15.3%     276544 ±  2%  softirqs.CPU129.RCU
     11849 ±  7%     +87.1%      22172 ±  6%  softirqs.CPU129.SCHED
    237400 ±  2%     +16.9%     277418 ±  2%  softirqs.CPU13.RCU
      7268 ±  2%    +195.9%      21508 ±  5%  softirqs.CPU13.SCHED
    239796 ±  3%     +15.2%     276364 ±  2%  softirqs.CPU130.RCU
     11467 ± 10%     +87.0%      21446 ±  6%  softirqs.CPU130.SCHED
    239375 ±  3%     +15.5%     276570 ±  2%  softirqs.CPU131.RCU
     11499 ± 10%     +88.0%      21622 ±  2%  softirqs.CPU131.SCHED
    239843 ±  3%     +15.2%     276258 ±  2%  softirqs.CPU132.RCU
     11345 ± 13%     +76.9%      20069 ±  7%  softirqs.CPU132.SCHED
    240419 ±  3%     +14.9%     276258 ±  2%  softirqs.CPU133.RCU
     11746 ±  9%     +85.5%      21786 ±  6%  softirqs.CPU133.SCHED
    240725 ±  3%     +14.9%     276508 ±  2%  softirqs.CPU134.RCU
     11673 ± 11%     +93.2%      22557        softirqs.CPU134.SCHED
    239741 ±  3%     +15.4%     276565 ±  2%  softirqs.CPU135.RCU
     11319 ± 10%     +86.1%      21069 ±  4%  softirqs.CPU135.SCHED
    239833 ±  3%     +15.0%     275884 ±  2%  softirqs.CPU136.RCU
     11282 ± 11%     +91.2%      21567 ±  3%  softirqs.CPU136.SCHED
    239378 ±  2%     +15.5%     276427 ±  2%  softirqs.CPU137.RCU
     11182 ± 13%     +96.9%      22022 ±  3%  softirqs.CPU137.SCHED
    239137 ±  2%     +15.6%     276561 ±  2%  softirqs.CPU138.RCU
     11060 ± 12%     +95.2%      21590 ±  2%  softirqs.CPU138.SCHED
    239509 ±  2%     +15.2%     275998 ±  2%  softirqs.CPU139.RCU
     11501 ± 10%     +82.6%      21002        softirqs.CPU139.SCHED
    237583 ±  2%     +16.6%     276950 ±  2%  softirqs.CPU14.RCU
      7052 ±  2%    +207.2%      21663 ±  4%  softirqs.CPU14.SCHED
    238755 ±  3%     +16.0%     276907 ±  2%  softirqs.CPU140.RCU
     11888 ± 11%     +86.5%      22167 ±  4%  softirqs.CPU140.SCHED
    239365 ±  3%     +15.4%     276230 ±  2%  softirqs.CPU141.RCU
     11652 ± 10%     +82.1%      21215 ±  4%  softirqs.CPU141.SCHED
    239565 ±  2%     +14.9%     275145 ±  2%  softirqs.CPU142.RCU
     10862 ±  9%     +86.7%      20278 ± 10%  softirqs.CPU142.SCHED
    238635 ±  3%     +15.7%     276099 ±  2%  softirqs.CPU143.RCU
     11276 ±  9%     +94.7%      21958 ±  9%  softirqs.CPU143.SCHED
    239070 ±  2%     +15.6%     276282 ±  2%  softirqs.CPU144.RCU
     12622 ± 10%     +77.3%      22375 ±  6%  softirqs.CPU144.SCHED
    239434 ±  2%     +15.4%     276243 ±  2%  softirqs.CPU145.RCU
     12576 ± 13%     +77.3%      22293 ±  3%  softirqs.CPU145.SCHED
    239944 ±  2%     +15.0%     275943 ±  2%  softirqs.CPU146.RCU
     12828 ± 14%     +74.0%      22316 ± 10%  softirqs.CPU146.SCHED
    240037 ±  2%     +15.1%     276166 ±  2%  softirqs.CPU147.RCU
     12743 ± 12%     +72.5%      21988 ±  8%  softirqs.CPU147.SCHED
    240069 ±  2%     +15.0%     275985 ±  2%  softirqs.CPU148.RCU
     13305 ± 14%     +66.9%      22208 ±  4%  softirqs.CPU148.SCHED
    239926 ±  2%     +15.0%     275803 ±  2%  softirqs.CPU149.RCU
     12615 ± 17%     +78.3%      22497 ±  7%  softirqs.CPU149.SCHED
    236602 ±  2%     +17.2%     277287 ±  2%  softirqs.CPU15.RCU
      7125 ±  3%    +201.3%      21470 ±  2%  softirqs.CPU15.SCHED
    239822 ±  2%     +15.1%     276036 ±  2%  softirqs.CPU150.RCU
     12651 ± 14%     +81.5%      22961 ±  4%  softirqs.CPU150.SCHED
    239854 ±  2%     +15.5%     277120 ±  2%  softirqs.CPU151.RCU
     12666 ± 11%     +66.9%      21133 ±  2%  softirqs.CPU151.SCHED
    239765 ±  2%     +15.0%     275782 ±  2%  softirqs.CPU152.RCU
     12727 ± 14%     +84.4%      23473 ±  5%  softirqs.CPU152.SCHED
    241136 ±  2%     +14.5%     276179 ±  2%  softirqs.CPU153.RCU
     12682 ± 14%     +74.3%      22112 ±  4%  softirqs.CPU153.SCHED
    240323 ±  2%     +15.0%     276346 ±  2%  softirqs.CPU154.RCU
     12651 ± 10%     +92.9%      24408 ±  5%  softirqs.CPU154.SCHED
    240505           +14.7%     275806 ±  2%  softirqs.CPU155.RCU
     13309 ± 16%     +57.0%      20894 ±  6%  softirqs.CPU155.SCHED
    239754 ±  2%     +15.3%     276385 ±  2%  softirqs.CPU156.RCU
     12923 ± 16%     +64.1%      21208 ±  6%  softirqs.CPU156.SCHED
    241082           +14.6%     276212 ±  2%  softirqs.CPU157.RCU
     13363 ± 14%     +62.6%      21733 ±  5%  softirqs.CPU157.SCHED
    239894 ±  2%     +15.4%     276769 ±  2%  softirqs.CPU158.RCU
     13022 ± 11%     +71.9%      22381 ±  3%  softirqs.CPU158.SCHED
    240423 ±  2%     +15.7%     278064 ±  2%  softirqs.CPU159.RCU
     12370 ± 11%     +76.0%      21770        softirqs.CPU159.SCHED
    237225 ±  2%     +17.1%     277698 ±  2%  softirqs.CPU16.RCU
      7101 ±  3%    +206.1%      21735 ±  4%  softirqs.CPU16.SCHED
    239449 ±  2%     +15.8%     277377        softirqs.CPU160.RCU
     12774 ±  8%     +75.6%      22427 ±  5%  softirqs.CPU160.SCHED
    240226 ±  2%     +15.1%     276543 ±  2%  softirqs.CPU161.RCU
     12908 ± 14%     +69.1%      21831 ±  8%  softirqs.CPU161.SCHED
    239672 ±  2%     +15.4%     276500 ±  2%  softirqs.CPU162.RCU
     12676 ± 16%     +69.1%      21430 ±  7%  softirqs.CPU162.SCHED
    240166 ±  2%     +15.1%     276400 ±  2%  softirqs.CPU163.RCU
     12776 ± 16%     +72.3%      22009 ±  3%  softirqs.CPU163.SCHED
    239182 ±  2%     +15.6%     276465 ±  2%  softirqs.CPU164.RCU
     13251 ± 11%     +72.5%      22853 ±  4%  softirqs.CPU164.SCHED
    240521 ±  2%     +15.0%     276490 ±  2%  softirqs.CPU165.RCU
     12976 ± 16%     +67.6%      21749 ±  2%  softirqs.CPU165.SCHED
    239398 ±  2%     +15.4%     276361 ±  2%  softirqs.CPU166.RCU
     13019 ± 14%     +64.4%      21403 ±  5%  softirqs.CPU166.SCHED
    239844 ±  2%     +15.0%     275828 ±  2%  softirqs.CPU167.RCU
     13097 ± 16%     +67.5%      21939 ±  6%  softirqs.CPU167.SCHED
    236169 ±  3%     +17.0%     276240 ±  2%  softirqs.CPU168.RCU
     12151 ± 17%     +83.0%      22234 ±  6%  softirqs.CPU168.SCHED
    238004 ±  2%     +16.1%     276296 ±  2%  softirqs.CPU169.RCU
     12402 ± 21%     +81.9%      22554 ±  5%  softirqs.CPU169.SCHED
    236203 ±  2%     +17.6%     277678 ±  2%  softirqs.CPU17.RCU
      7127 ±  2%    +196.2%      21114 ±  6%  softirqs.CPU17.SCHED
    237699 ±  2%     +16.4%     276760 ±  2%  softirqs.CPU170.RCU
     12355 ± 20%     +80.5%      22306 ±  2%  softirqs.CPU170.SCHED
    237915 ±  2%     +16.5%     277272 ±  2%  softirqs.CPU171.RCU
     12252 ± 14%     +86.0%      22786 ±  3%  softirqs.CPU171.SCHED
    238255 ±  2%     +16.1%     276673 ±  2%  softirqs.CPU172.RCU
     11901 ± 16%     +84.5%      21963 ±  5%  softirqs.CPU172.SCHED
    238295 ±  2%     +15.9%     276093 ±  2%  softirqs.CPU173.RCU
     12026 ± 18%     +87.1%      22495 ±  3%  softirqs.CPU173.SCHED
    238158 ±  2%     +16.1%     276393 ±  2%  softirqs.CPU174.RCU
     12177 ± 18%     +82.0%      22168 ±  3%  softirqs.CPU174.SCHED
    237929 ±  2%     +16.4%     277027 ±  2%  softirqs.CPU175.RCU
     12165 ± 20%     +89.4%      23035 ±  4%  softirqs.CPU175.SCHED
    238594 ±  2%     +15.9%     276437 ±  2%  softirqs.CPU176.RCU
     12390 ± 18%     +79.6%      22248 ±  8%  softirqs.CPU176.SCHED
    238061 ±  2%     +16.1%     276397 ±  2%  softirqs.CPU177.RCU
     12183 ± 17%     +85.0%      22534 ±  2%  softirqs.CPU177.SCHED
    238766 ±  2%     +15.6%     276004 ±  2%  softirqs.CPU178.RCU
     11806 ± 21%     +91.1%      22564 ±  7%  softirqs.CPU178.SCHED
    238716 ±  3%     +15.4%     275492 ±  2%  softirqs.CPU179.RCU
     12307 ± 14%     +85.1%      22777 ±  3%  softirqs.CPU179.SCHED
    236973 ±  2%     +17.5%     278441        softirqs.CPU18.RCU
      7525 ±  2%    +189.7%      21804 ±  5%  softirqs.CPU18.SCHED
    238252 ±  2%     +15.9%     276100 ±  2%  softirqs.CPU180.RCU
     12482 ± 17%     +83.4%      22892 ±  3%  softirqs.CPU180.SCHED
    238516 ±  2%     +15.6%     275773 ±  2%  softirqs.CPU181.RCU
     12011 ± 16%     +86.1%      22356 ±  3%  softirqs.CPU181.SCHED
    237696 ±  2%     +16.0%     275723 ±  2%  softirqs.CPU182.RCU
     12305 ± 20%     +89.1%      23273 ±  5%  softirqs.CPU182.SCHED
    237946 ±  2%     +15.9%     275843 ±  2%  softirqs.CPU183.RCU
     12353 ± 16%     +83.3%      22650 ±  7%  softirqs.CPU183.SCHED
    238583 ±  2%     +15.5%     275651 ±  2%  softirqs.CPU184.RCU
     12310 ± 18%     +73.3%      21329 ±  3%  softirqs.CPU184.SCHED
    237523 ±  2%     +16.1%     275761 ±  2%  softirqs.CPU185.RCU
     12428 ± 17%     +87.5%      23301 ±  8%  softirqs.CPU185.SCHED
    237252 ±  2%     +16.2%     275794 ±  2%  softirqs.CPU186.RCU
     12723 ± 21%     +80.7%      22991 ±  5%  softirqs.CPU186.SCHED
    237483 ±  2%     +16.2%     275925 ±  2%  softirqs.CPU187.RCU
     12306 ± 15%     +78.5%      21970 ±  4%  softirqs.CPU187.SCHED
    238171 ±  2%     +15.9%     276106 ±  2%  softirqs.CPU188.RCU
     12167 ± 19%     +79.4%      21831 ±  9%  softirqs.CPU188.SCHED
    237944 ±  2%     +15.8%     275644 ±  2%  softirqs.CPU189.RCU
     12061 ± 19%     +85.5%      22374 ±  5%  softirqs.CPU189.SCHED
    237626 ±  2%     +17.1%     278182 ±  2%  softirqs.CPU19.RCU
      6961 ±  2%    +216.7%      22047 ±  7%  softirqs.CPU19.SCHED
    238054 ±  2%     +15.8%     275593 ±  2%  softirqs.CPU190.RCU
     12282 ± 18%     +83.7%      22559 ±  8%  softirqs.CPU190.SCHED
    233445 ±  2%     +15.7%     270209 ±  2%  softirqs.CPU191.RCU
     11930 ± 19%     +81.8%      21686 ±  5%  softirqs.CPU191.SCHED
    237864 ±  2%     +16.5%     277190 ±  2%  softirqs.CPU2.RCU
      7752          +186.0%      22171 ±  7%  softirqs.CPU2.SCHED
    238078 ±  2%     +16.8%     278014 ±  2%  softirqs.CPU20.RCU
      7416 ±  4%    +195.3%      21899 ±  4%  softirqs.CPU20.SCHED
    236549 ±  2%     +17.4%     277605 ±  2%  softirqs.CPU21.RCU
      7303 ±  3%    +200.0%      21910 ±  5%  softirqs.CPU21.SCHED
    237148 ±  2%     +17.1%     277808 ±  2%  softirqs.CPU22.RCU
      7099 ±  2%    +202.8%      21500 ±  5%  softirqs.CPU22.SCHED
    236495 ±  2%     +17.5%     277968 ±  2%  softirqs.CPU23.RCU
      7234          +202.1%      21858 ±  7%  softirqs.CPU23.SCHED
    237903 ±  3%     +16.9%     278156 ±  2%  softirqs.CPU24.RCU
     12150 ± 12%     +71.7%      20857 ±  3%  softirqs.CPU24.SCHED
    239923 ±  3%     +15.8%     277817 ±  2%  softirqs.CPU25.RCU
     10992 ±  8%     +98.3%      21794 ±  3%  softirqs.CPU25.SCHED
    239466 ±  3%     +16.1%     277996 ±  2%  softirqs.CPU26.RCU
     11042 ± 11%     +94.1%      21428 ±  2%  softirqs.CPU26.SCHED
    240078 ±  3%     +15.8%     277926 ±  2%  softirqs.CPU27.RCU
     11173 ± 10%    +100.3%      22384 ±  2%  softirqs.CPU27.SCHED
    240433 ±  2%     +15.5%     277671 ±  2%  softirqs.CPU28.RCU
     11433 ± 13%     +84.5%      21094 ±  3%  softirqs.CPU28.SCHED
    240301 ±  3%     +15.4%     277321 ±  2%  softirqs.CPU29.RCU
     11129 ± 14%    +100.3%      22294 ±  2%  softirqs.CPU29.SCHED
    236746 ±  2%     +17.5%     278064 ±  2%  softirqs.CPU3.RCU
      7516 ±  7%    +197.0%      22322 ±  6%  softirqs.CPU3.SCHED
    240275 ±  3%     +15.5%     277606 ±  2%  softirqs.CPU30.RCU
     11363 ±  9%     +89.0%      21476 ±  4%  softirqs.CPU30.SCHED
    240559 ±  2%     +15.4%     277650 ±  2%  softirqs.CPU31.RCU
     10797 ± 10%     +96.7%      21241 ±  6%  softirqs.CPU31.SCHED
    239885 ±  2%     +15.5%     277092 ±  2%  softirqs.CPU32.RCU
     10747 ± 10%     +96.8%      21154 ±  2%  softirqs.CPU32.SCHED
    239056 ±  3%     +15.8%     276897 ±  2%  softirqs.CPU33.RCU
     11036 ± 10%    +101.0%      22184 ±  4%  softirqs.CPU33.SCHED
    240203 ±  3%     +15.4%     277213 ±  2%  softirqs.CPU34.RCU
     11265 ± 12%     +91.1%      21526 ±  5%  softirqs.CPU34.SCHED
    239788 ±  2%     +15.7%     277392 ±  2%  softirqs.CPU35.RCU
     11537 ±  9%     +87.3%      21611 ±  3%  softirqs.CPU35.SCHED
    239688 ±  3%     +15.4%     276628 ±  2%  softirqs.CPU36.RCU
     11220 ± 10%     +76.3%      19779 ±  6%  softirqs.CPU36.SCHED
    240710 ±  3%     +15.1%     277049 ±  2%  softirqs.CPU37.RCU
     11487 ± 10%     +88.9%      21700 ±  5%  softirqs.CPU37.SCHED
    240363 ±  2%     +15.4%     277434 ±  2%  softirqs.CPU38.RCU
     11775 ±  8%     +92.7%      22690        softirqs.CPU38.SCHED
    239926 ±  3%     +15.6%     277323 ±  2%  softirqs.CPU39.RCU
     11221 ± 11%     +88.2%      21114 ±  2%  softirqs.CPU39.SCHED
    237001 ±  2%     +17.8%     279138 ±  2%  softirqs.CPU4.RCU
      7526 ±  3%    +182.7%      21280 ±  4%  softirqs.CPU4.SCHED
    239452 ±  3%     +16.2%     278199 ±  2%  softirqs.CPU40.RCU
     11554 ±  9%     +85.5%      21437 ±  4%  softirqs.CPU40.SCHED
    239274 ±  2%     +15.8%     277116 ±  2%  softirqs.CPU41.RCU
     11050 ± 11%    +101.8%      22302 ±  2%  softirqs.CPU41.SCHED
    239717 ±  2%     +15.6%     277161 ±  2%  softirqs.CPU42.RCU
     11008 ± 10%     +93.7%      21327        softirqs.CPU42.SCHED
    239640 ±  2%     +15.6%     277119 ±  2%  softirqs.CPU43.RCU
     11362 ± 10%     +86.7%      21209 ±  2%  softirqs.CPU43.SCHED
    240790 ±  2%     +15.0%     277010 ±  2%  softirqs.CPU44.RCU
     11740 ±  8%     +89.2%      22216 ±  4%  softirqs.CPU44.SCHED
    239377 ±  3%     +15.7%     277007 ±  2%  softirqs.CPU45.RCU
     11024 ±  7%     +92.5%      21225 ±  4%  softirqs.CPU45.SCHED
    240082 ±  3%     +15.7%     277808 ±  2%  softirqs.CPU46.RCU
     10812 ± 11%     +97.3%      21334 ±  2%  softirqs.CPU46.SCHED
    239341 ±  3%     +15.7%     276827 ±  2%  softirqs.CPU47.RCU
     11586 ± 10%     +89.7%      21977 ±  7%  softirqs.CPU47.SCHED
    239435 ±  2%     +15.9%     277428 ±  2%  softirqs.CPU48.RCU
     12441 ± 13%     +80.3%      22431 ±  6%  softirqs.CPU48.SCHED
    240196 ±  2%     +15.3%     276990 ±  2%  softirqs.CPU49.RCU
     12824 ± 11%     +70.8%      21897 ±  2%  softirqs.CPU49.SCHED
    235705 ±  2%     +17.6%     277082 ±  2%  softirqs.CPU5.RCU
      7164 ±  2%    +204.9%      21842 ±  2%  softirqs.CPU5.SCHED
    240038 ±  2%     +15.8%     277875 ±  2%  softirqs.CPU50.RCU
     12585 ± 15%     +78.1%      22416 ±  8%  softirqs.CPU50.SCHED
    240511 ±  2%     +15.2%     277071 ±  2%  softirqs.CPU51.RCU
     12318 ± 10%     +79.5%      22112 ±  9%  softirqs.CPU51.SCHED
    240220 ±  2%     +15.2%     276794 ±  2%  softirqs.CPU52.RCU
     13100 ± 13%     +71.8%      22502 ±  5%  softirqs.CPU52.SCHED
    240827 ±  2%     +14.9%     276753 ±  2%  softirqs.CPU53.RCU
     12356 ± 16%     +81.8%      22465 ±  8%  softirqs.CPU53.SCHED
    240556 ±  2%     +15.1%     276862 ±  2%  softirqs.CPU54.RCU
     12776 ± 15%     +74.8%      22337 ±  3%  softirqs.CPU54.SCHED
    240625 ±  2%     +15.0%     276659 ±  2%  softirqs.CPU55.RCU
     12908 ± 14%     +64.6%      21250 ±  3%  softirqs.CPU55.SCHED
    240388 ±  2%     +15.3%     277156 ±  2%  softirqs.CPU56.RCU
     12981 ± 11%     +80.3%      23401 ±  6%  softirqs.CPU56.SCHED
    240715 ±  2%     +15.0%     276719 ±  2%  softirqs.CPU57.RCU
     12599 ± 14%     +76.0%      22175        softirqs.CPU57.SCHED
    241813 ±  2%     +14.3%     276503 ±  2%  softirqs.CPU58.RCU
     12766 ± 12%     +85.3%      23658 ±  2%  softirqs.CPU58.SCHED
    239885 ±  2%     +15.2%     276461 ±  2%  softirqs.CPU59.RCU
     12730 ± 15%     +71.3%      21800 ±  8%  softirqs.CPU59.SCHED
    237048 ±  2%     +16.9%     277014 ±  2%  softirqs.CPU6.RCU
      7446 ±  3%    +201.8%      22475 ±  6%  softirqs.CPU6.SCHED
    241389 ±  2%     +14.9%     277260 ±  2%  softirqs.CPU60.RCU
     12989 ± 16%     +63.3%      21215 ±  4%  softirqs.CPU60.SCHED
    240429 ±  2%     +15.1%     276683 ±  2%  softirqs.CPU61.RCU
     13031 ± 11%     +67.7%      21850 ±  9%  softirqs.CPU61.SCHED
    240845           +15.0%     276996 ±  2%  softirqs.CPU62.RCU
     13007 ± 10%     +74.5%      22703 ±  5%  softirqs.CPU62.SCHED
    241631 ±  2%     +14.8%     277487 ±  2%  softirqs.CPU63.RCU
     12515 ±  9%     +67.4%      20950 ±  4%  softirqs.CPU63.SCHED
    240590 ±  2%     +15.3%     277424 ±  2%  softirqs.CPU64.RCU
     12832 ±  8%     +74.1%      22342 ±  3%  softirqs.CPU64.SCHED
    240324 ±  2%     +15.6%     277853 ±  2%  softirqs.CPU65.RCU
     12976 ± 14%     +69.4%      21988 ±  5%  softirqs.CPU65.SCHED
    240874 ±  2%     +15.0%     277061 ±  2%  softirqs.CPU66.RCU
     12631 ± 15%     +75.2%      22127 ±  4%  softirqs.CPU66.SCHED
    240514 ±  2%     +15.2%     277102 ±  2%  softirqs.CPU67.RCU
     12723 ± 14%     +74.0%      22135 ±  2%  softirqs.CPU67.SCHED
    239804 ±  2%     +15.5%     276908 ±  2%  softirqs.CPU68.RCU
     12790 ± 10%     +78.4%      22812 ±  4%  softirqs.CPU68.SCHED
    241049 ±  2%     +14.9%     276953 ±  2%  softirqs.CPU69.RCU
     13005 ± 14%     +66.4%      21639        softirqs.CPU69.SCHED
    236068 ±  2%     +17.3%     277001 ±  2%  softirqs.CPU7.RCU
      7306 ±  5%    +207.4%      22459 ±  6%  softirqs.CPU7.SCHED
    240222 ±  2%     +15.4%     277107 ±  2%  softirqs.CPU70.RCU
     12741 ± 15%     +70.7%      21750 ±  3%  softirqs.CPU70.SCHED
    239775 ±  2%     +15.6%     277122 ±  2%  softirqs.CPU71.RCU
     13046 ± 15%     +67.3%      21829 ±  5%  softirqs.CPU71.SCHED
    236457 ±  3%     +17.2%     277159 ±  2%  softirqs.CPU72.RCU
     12369 ± 13%     +74.2%      21545 ±  4%  softirqs.CPU72.SCHED
    238085 ±  2%     +16.4%     277099 ±  2%  softirqs.CPU73.RCU
     12049 ± 21%     +88.7%      22742 ±  5%  softirqs.CPU73.SCHED
    238347 ±  2%     +16.2%     277060 ±  2%  softirqs.CPU74.RCU
     12513 ± 18%     +73.0%      21651 ±  2%  softirqs.CPU74.SCHED
    238313 ±  2%     +16.2%     276847 ±  2%  softirqs.CPU75.RCU
     12440 ± 14%     +79.9%      22377 ±  5%  softirqs.CPU75.SCHED
    239678 ±  2%     +15.6%     276983 ±  2%  softirqs.CPU76.RCU
     11779 ± 15%     +86.2%      21929 ±  7%  softirqs.CPU76.SCHED
    238986 ±  2%     +15.9%     276937 ±  2%  softirqs.CPU77.RCU
     12026 ± 15%     +87.6%      22566 ±  5%  softirqs.CPU77.SCHED
    238837 ±  2%     +16.2%     277607 ±  2%  softirqs.CPU78.RCU
     12108 ± 18%     +84.9%      22382 ±  3%  softirqs.CPU78.SCHED
    238796 ±  2%     +16.0%     277071 ±  2%  softirqs.CPU79.RCU
     12079 ± 18%     +89.0%      22827 ±  4%  softirqs.CPU79.SCHED
    236495 ±  2%     +16.8%     276341 ±  2%  softirqs.CPU8.RCU
      7313 ±  3%    +192.3%      21378 ±  8%  softirqs.CPU8.SCHED
    238602 ±  2%     +16.3%     277396 ±  2%  softirqs.CPU80.RCU
     12208 ± 16%     +85.4%      22639 ±  7%  softirqs.CPU80.SCHED
    239904 ±  2%     +15.2%     276363 ±  2%  softirqs.CPU81.RCU
     12024 ± 16%     +85.1%      22261 ±  4%  softirqs.CPU81.SCHED
    239387 ±  2%     +15.6%     276623 ±  2%  softirqs.CPU82.RCU
     11826 ± 22%     +90.9%      22574 ±  5%  softirqs.CPU82.SCHED
    238618 ±  3%     +15.9%     276518 ±  2%  softirqs.CPU83.RCU
     11895 ± 13%     +94.1%      23089 ±  4%  softirqs.CPU83.SCHED
    238682 ±  2%     +15.9%     276750 ±  2%  softirqs.CPU84.RCU
     12414 ± 16%     +82.3%      22628 ±  2%  softirqs.CPU84.SCHED
    238899 ±  2%     +15.8%     276542 ±  2%  softirqs.CPU85.RCU
     11702 ± 16%     +94.0%      22706 ±  2%  softirqs.CPU85.SCHED
    238222 ±  2%     +16.0%     276327 ±  2%  softirqs.CPU86.RCU
     11697 ± 19%     +96.4%      22975 ±  5%  softirqs.CPU86.SCHED
    238691 ±  2%     +15.9%     276603 ±  2%  softirqs.CPU87.RCU
     12189 ± 15%     +85.3%      22582 ±  4%  softirqs.CPU87.SCHED
    238808 ±  2%     +15.7%     276305 ±  2%  softirqs.CPU88.RCU
     11778 ± 18%     +86.2%      21937 ±  3%  softirqs.CPU88.SCHED
    238381 ±  2%     +16.1%     276699 ±  2%  softirqs.CPU89.RCU
     12290 ± 16%     +89.7%      23314 ±  8%  softirqs.CPU89.SCHED
    237015 ±  2%     +17.1%     277559 ±  2%  softirqs.CPU9.RCU
      7232 ±  4%    +206.1%      22136 ±  5%  softirqs.CPU9.SCHED
    238408 ±  2%     +16.1%     276834 ±  2%  softirqs.CPU90.RCU
     12856 ± 20%     +79.8%      23112 ±  5%  softirqs.CPU90.SCHED
    238352 ±  2%     +16.0%     276459 ±  2%  softirqs.CPU91.RCU
     12336 ± 17%     +80.7%      22288 ±  6%  softirqs.CPU91.SCHED
    238103 ±  2%     +16.4%     277102 ±  2%  softirqs.CPU92.RCU
     11709 ± 16%     +88.5%      22073 ±  7%  softirqs.CPU92.SCHED
    238652 ±  2%     +15.8%     276433 ±  2%  softirqs.CPU93.RCU
     12219 ± 19%     +89.1%      23110 ±  6%  softirqs.CPU93.SCHED
    237990 ±  2%     +16.1%     276267 ±  2%  softirqs.CPU94.RCU
     12180 ± 19%     +81.2%      22068 ±  6%  softirqs.CPU94.SCHED
    238952 ±  2%     +15.6%     276325 ±  2%  softirqs.CPU95.RCU
     11712 ± 20%     +92.8%      22585 ±  4%  softirqs.CPU95.SCHED
    235861 ±  2%     +17.3%     276610 ±  2%  softirqs.CPU96.RCU
      7496 ±  4%    +178.2%      20857 ±  7%  softirqs.CPU96.SCHED
    235263 ±  2%     +17.6%     276598 ±  2%  softirqs.CPU97.RCU
      6700 ±  7%    +212.3%      20923 ±  4%  softirqs.CPU97.SCHED
    237582 ±  2%     +17.7%     279562        softirqs.CPU98.RCU
      7106 ±  5%    +222.1%      22887 ±  3%  softirqs.CPU98.SCHED
    236037 ±  2%     +17.5%     277259 ±  2%  softirqs.CPU99.RCU
      7139 ±  2%    +208.7%      22036 ±  4%  softirqs.CPU99.SCHED
  45836274 ±  2%     +16.0%   53154287 ±  2%  softirqs.RCU
   2100099 ± 10%    +101.3%    4228061 ±  2%  softirqs.SCHED


                                                                                
                                hackbench.throughput                            
                                                                                
  450000 +------------------------------------------------------------------+   
         |                O                       O  O                O  O  |   
  400000 |-+O  O  O  O  O    O  O  O  O  O  O  O        O  O O  O  O        |   
  350000 |-+                                                                |   
         |                                                                  |   
  300000 |-.+..+..+..+..+                                                   |   
  250000 |.+            :                                                   |   
         |               :                                                  |   
  200000 |-+             :                                                  |   
  150000 |-+             :                                                  |   
         |               :                                                  |   
  100000 |-+              :                                                 |   
   50000 |-+              :                                                 |   
         |                +..+..+..+..+..+..+..+..+..+..+..+                |   
       0 +------------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample



Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Rong Chen


View attachment "config-5.5.0-rc4-00002-g20d4e80d255dd" of type "text/plain" (199695 bytes)

View attachment "job-script" of type "text/plain" (7511 bytes)

View attachment "job.yaml" of type "text/plain" (5237 bytes)

View attachment "reproduce" of type "text/plain" (944 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ