lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date:   Thu, 12 Nov 2020 22:06:25 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     Lee Jones <lee.jones@...aro.org>
Cc:     Daniel Vetter <daniel.vetter@...ll.ch>,
        Russell King <linux@...linux.org.uk>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Peilin Ye <yepeilin.cs@...il.com>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org,
        lkp@...el.com, ying.huang@...el.com, feng.tang@...el.com,
        zhengjun.xing@...el.com
Subject: [Fonts]  9522750c66:  fio.read_iops 7.5% improvement


Greeting,

FYI, we noticed a 7.5% improvement of fio.read_iops due to commit:


commit: 9522750c66c689b739e151fcdf895420dc81efc0 ("Fonts: Replace discarded const qualifier")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master


in testcase: fio-basic
on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory
with following parameters:

	disk: 2pmem
	fs: xfs
	mount_option: dax
	runtime: 200s
	nr_task: 50%
	time_based: tb
	rw: read
	bs: 2M
	ioengine: mmap
	test_size: 200G
	cpufreq_governor: performance
	ucode: 0x5002f01

test-description: Fio is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user.
test-url: https://github.com/axboe/fio

In addition to that, the commit also has significant impact on the following tests:

+------------------+---------------------------------------------------------------------------+
| testcase: change | netperf: netperf.Throughput_tps 8.5% improvement                          |
| test machine     | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| test parameters  | cluster=cs-localhost                                                      |
|                  | cpufreq_governor=performance                                              |
|                  | ip=ipv4                                                                   |
|                  | nr_threads=16                                                             |
|                  | runtime=300s                                                              |
|                  | test=TCP_CRR                                                              |
|                  | ucode=0x5002f01                                                           |
+------------------+---------------------------------------------------------------------------+




Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/mount_option/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
  2M/gcc-9/performance/2pmem/xfs/mmap/x86_64-rhel-8.3/dax/50%/debian-10.4-x86_64-20200603.cgz/200s/read/lkp-csl-2sp6/200G/fio-basic/tb/0x5002f01

commit: 
  077aa5c8dd ("drm/vc4: drv: Remove unused variable")
  9522750c66 ("Fonts: Replace discarded const qualifier")

077aa5c8dd6e2b9c 9522750c66c689b739e151fcdf8 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
          1:4            1%           1:4     perf-profile.children.cycles-pp.error_entry
          0:4            1%           0:4     perf-profile.self.cycles-pp.error_entry
         %stddev     %change         %stddev
             \          |                \  
      1.45 ± 47%     +94.6       96.00        fio.latency_10ms%
     98.53           -94.6        3.98 ±  4%  fio.latency_20ms%
      9346            +7.5%      10046        fio.read_bw_MBps
  10420224            -6.9%    9699328        fio.read_clat_90%_us
  10551296            -7.5%    9764864        fio.read_clat_95%_us
  11010048            -6.5%   10289152        fio.read_clat_99%_us
  10270104            -7.0%    9553975        fio.read_clat_mean_us
      4673            +7.5%       5023        fio.read_iops
 4.786e+08            +7.5%  5.144e+08        fio.time.minor_page_faults
    480.16 ±  2%      +7.2%     514.95        fio.time.user_time
    934634            +7.5%    1004686        fio.workload
      2.45 ±  3%      +7.7%       2.64        iostat.cpu.user
    898746 ±  2%     +11.2%     999515 ±  3%  numa-meminfo.node1.PageTables
    223463 ±  2%     +11.5%     249070 ±  3%  numa-vmstat.node1.nr_page_table_pages
   1285531 ± 17%     +24.2%    1596457 ±  6%  numa-numastat.node1.local_node
   1305895 ± 16%     +24.0%    1619835 ±  6%  numa-numastat.node1.numa_hit
     80574 ±  3%     -14.8%      68623 ±  8%  sched_debug.cfs_rq:/.min_vruntime.avg
    165667 ± 10%     -14.8%     141206 ± 10%  sched_debug.cfs_rq:/.min_vruntime.max
    187.26 ±  5%     -10.9%     166.80 ±  6%  sched_debug.cfs_rq:/.util_est_enqueued.stddev
      2911           -11.0%       2591 ±  8%  slabinfo.fsnotify_mark_connector.active_objs
      2911           -11.0%       2591 ±  8%  slabinfo.fsnotify_mark_connector.num_objs
    776.00 ± 10%     -29.9%     544.00 ± 21%  slabinfo.kmalloc-rcl-128.active_objs
    776.00 ± 10%     -29.9%     544.00 ± 21%  slabinfo.kmalloc-rcl-128.num_objs
      2477 ±  5%     -15.7%       2089 ±  6%  slabinfo.kmalloc-rcl-96.active_objs
      2477 ±  5%     -15.7%       2089 ±  6%  slabinfo.kmalloc-rcl-96.num_objs
      6962 ± 78%    +200.8%      20942 ± 44%  softirqs.CPU31.SCHED
      8822 ±105%     +73.5%      15307 ± 74%  softirqs.CPU50.SCHED
      6520 ± 79%    +139.8%      15637 ± 74%  softirqs.CPU6.SCHED
     23828 ± 19%     -61.1%       9262 ±103%  softirqs.CPU79.SCHED
     22347 ± 32%     -59.9%       8959 ±108%  softirqs.CPU80.SCHED
     18000 ± 50%     -48.4%       9286 ±103%  softirqs.CPU89.SCHED
    460722            +7.8%     496617        proc-vmstat.nr_page_table_pages
     50476            +1.4%      51178        proc-vmstat.nr_slab_unreclaimable
   2806863            +5.5%    2962315        proc-vmstat.numa_hit
   2775487            +5.6%    2931000        proc-vmstat.numa_local
    382649 ±  2%      +9.5%     419188 ±  2%  proc-vmstat.numa_pte_updates
   2988102            +5.2%    3144128        proc-vmstat.pgalloc_normal
 4.797e+08            +7.5%  5.156e+08        proc-vmstat.pgfault
   2769962            +5.3%    2917620        proc-vmstat.pgfree
    934634            +7.5%    1004686        proc-vmstat.thp_fault_fallback
    477.50 ±  8%      -9.3%     433.00        interrupts.CPU10.CAL:Function_call_interrupts
    115.25 ± 77%     -54.0%      53.00 ±161%  interrupts.CPU13.RES:Rescheduling_interrupts
     55.75 ±149%    +177.6%     154.75 ± 54%  interrupts.CPU19.RES:Rescheduling_interrupts
     62.25 ± 43%     -54.2%      28.50 ± 64%  interrupts.CPU20.TLB:TLB_shootdowns
     74.50 ± 27%     -46.3%      40.00 ± 19%  interrupts.CPU23.TLB:TLB_shootdowns
      6578 ± 32%     -43.0%       3747 ± 64%  interrupts.CPU30.NMI:Non-maskable_interrupts
      6578 ± 32%     -43.0%       3747 ± 64%  interrupts.CPU30.PMI:Performance_monitoring_interrupts
    176.75 ± 25%     -65.8%      60.50 ±134%  interrupts.CPU30.RES:Rescheduling_interrupts
     33.75 ±110%    +148.9%      84.00 ± 49%  interrupts.CPU30.TLB:TLB_shootdowns
    177.50 ± 24%     -69.2%      54.75 ±146%  interrupts.CPU31.RES:Rescheduling_interrupts
    442.50 ±  6%     +70.8%     755.75 ± 61%  interrupts.CPU43.CAL:Function_call_interrupts
    642.75 ± 29%     -31.3%     441.75        interrupts.CPU52.CAL:Function_call_interrupts
      4072 ± 52%     +60.9%       6550 ± 32%  interrupts.CPU52.NMI:Non-maskable_interrupts
      4072 ± 52%     +60.9%       6550 ± 32%  interrupts.CPU52.PMI:Performance_monitoring_interrupts
      2766 ± 31%    +137.4%       6566 ± 32%  interrupts.CPU7.NMI:Non-maskable_interrupts
      2766 ± 31%    +137.4%       6566 ± 32%  interrupts.CPU7.PMI:Performance_monitoring_interrupts
     52.00 ±168%    +200.5%     156.25 ± 53%  interrupts.CPU91.RES:Rescheduling_interrupts
     12.68            +4.6%      13.26        perf-stat.i.MPKI
 5.504e+09            +4.0%  5.722e+09        perf-stat.i.branch-instructions
 2.131e+08            +8.2%  2.306e+08        perf-stat.i.cache-misses
 2.944e+08            +8.8%  3.204e+08        perf-stat.i.cache-references
      5.75            -3.7%       5.54        perf-stat.i.cpi
    635.20            -7.1%     589.90        perf-stat.i.cycles-between-cache-misses
      0.91            +0.0        0.94        perf-stat.i.dTLB-load-miss-rate%
  52157498 ±  2%      +7.3%   55942966        perf-stat.i.dTLB-load-misses
 5.648e+09            +3.7%  5.854e+09        perf-stat.i.dTLB-loads
 1.734e+09            +7.1%  1.858e+09        perf-stat.i.dTLB-stores
     84.17            +0.8       84.99        perf-stat.i.iTLB-load-miss-rate%
  15821747            +6.1%   16780552        perf-stat.i.iTLB-load-misses
 2.317e+10            +4.1%  2.412e+10        perf-stat.i.instructions
      1483 ±  2%      -2.8%       1442        perf-stat.i.instructions-per-iTLB-miss
    138.73            +4.4%     144.83        perf-stat.i.metric.M/sec
   2366716            +7.7%    2549877        perf-stat.i.minor-faults
   6666646            +5.8%    7054355 ±  2%  perf-stat.i.node-store-misses
  24228043            +7.8%   26116619        perf-stat.i.node-stores
   2368098            +7.7%    2551259        perf-stat.i.page-faults
     12.71            +4.5%      13.28        perf-stat.overall.MPKI
      5.76            -3.7%       5.55        perf-stat.overall.cpi
    626.87            -7.4%     580.52        perf-stat.overall.cycles-between-cache-misses
      0.92            +0.0        0.95        perf-stat.overall.dTLB-load-miss-rate%
      1464            -1.8%       1437        perf-stat.overall.instructions-per-iTLB-miss
      0.17            +3.9%       0.18        perf-stat.overall.ipc
   5018804            -3.4%    4848844        perf-stat.overall.path-length
 5.476e+09            +4.0%  5.693e+09        perf-stat.ps.branch-instructions
  2.12e+08            +8.2%  2.294e+08        perf-stat.ps.cache-misses
 2.929e+08            +8.8%  3.188e+08        perf-stat.ps.cache-references
  51896278 ±  2%      +7.3%   55661047        perf-stat.ps.dTLB-load-misses
 5.619e+09            +3.7%  5.825e+09        perf-stat.ps.dTLB-loads
 1.725e+09            +7.1%  1.848e+09        perf-stat.ps.dTLB-stores
  15742547            +6.1%   16696045        perf-stat.ps.iTLB-load-misses
 2.305e+10            +4.1%    2.4e+10        perf-stat.ps.instructions
   2354943            +7.7%    2537094        perf-stat.ps.minor-faults
   6632982            +5.8%    7018377 ±  2%  perf-stat.ps.node-store-misses
  24109364            +7.8%   25987507        perf-stat.ps.node-stores
   2356327            +7.7%    2538482        perf-stat.ps.page-faults
 4.691e+12            +3.9%  4.872e+12        perf-stat.total.instructions


                                                                                
                                  fio.read_bw_MBps                              
                                                                                
  10100 +-------------------------------------------------------------------+   
        | OO OO O OO   O  O  O OO O  O OO   OO O                            |   
  10000 |-+          O  O  O                                                |   
   9900 |-+                                                                 |   
        |                                                                   |   
   9800 |-+                                                                 |   
   9700 |-+                                                                 |   
        |                                                                   |   
   9600 |-+                                                                 |   
   9500 |-+                                                                 |   
        |                                                                   |   
   9400 |.+ .+      .+.++. +. .++. .+ .+ .+.++.+ .+. +.+. +.    .+ .+.+     |   
   9300 |-++  +. .++      +  +    +  +  +       +   +    +  ++.+  +    +.++.|   
        |       +                                                           |   
   9200 +-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                   fio.read_iops                                
                                                                                
  5050 +--------------------------------------------------------------------+   
       | OO O OO O O  O  O  O O OO   OO O  O OO                             |   
  5000 |-+          O  O   O                                                |   
  4950 |-+                                                                  |   
       |                                                                    |   
  4900 |-+                                                                  |   
  4850 |-+                                                                  |   
       |                                                                    |   
  4800 |-+                                                                  |   
  4750 |-+                                                                  |   
       |                                                                    |   
  4700 |.+ .+.      +.++. .+ .+.+ .+. +. .++.++. .+ .+.+ .+.    .+ .++.     |   
  4650 |-++   + .+.+     +  +    +   +  +       +  +    +   ++.+  +    +.++.|   
       |       +                                                            |   
  4600 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                 fio.read_clat_mean_us                          
                                                                                
  1.04e+07 +----------------------------------------------------------------+   
           |      .++.                                                      |   
  1.03e+07 |-++. +    +    .+. +.  .+ .+ .+. +.  .+ .+   +. .++.++.++.++.++.|   
  1.02e+07 |.+  +      +.++   +  ++  +  +   +  ++  +  +.+  +                |   
           |                                                                |   
  1.01e+07 |-+                                                              |   
     1e+07 |-+                                                              |   
           |                                                                |   
   9.9e+06 |-+                                                              |   
   9.8e+06 |-+                                                              |   
           |                                                                |   
   9.7e+06 |-+                                                              |   
   9.6e+06 |-+         O  O   O                                             |   
           | OO OO OO O  O  O  O OO O  OO O  O OO                           |   
   9.5e+06 +----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                 fio.read_clat_90__us                           
                                                                                
  1.06e+07 +----------------------------------------------------------------+   
  1.05e+07 |-+     +                                                        |   
           |      + :                                                       |   
  1.04e+07 |.++.++  +.++.++.+.++.++.++.++.+.++.++.++.++.++.+.++.++.++.++.++.|   
  1.03e+07 |-+                                                              |   
           |                                                                |   
  1.02e+07 |-+                                                              |   
  1.01e+07 |-+                                                              |   
     1e+07 |-+                                                              |   
           |                                                                |   
   9.9e+06 |-+                                                              |   
   9.8e+06 |-+                                                              |   
           |  O    OO OO OO O OO OO O   O O    OO                           |   
   9.7e+06 |-+                                                              |   
   9.6e+06 +----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                 fio.read_clat_95__us                           
                                                                                
  1.06e+07 +----------------------------------------------------------------+   
           |.++.++.++.++.++.+.++.++.++.++.+.+  ++.++.++.++.+.++.++.++.++.++.|   
  1.05e+07 |-+                               :+                             |   
  1.04e+07 |-+                               +                              |   
           |                                                                |   
  1.03e+07 |-+                                                              |   
  1.02e+07 |-+                                                              |   
           |                                                                |   
  1.01e+07 |-+                                                              |   
     1e+07 |-+                                                              |   
           |                                                                |   
   9.9e+06 |-+O    O   O O  O  O OO O                                       |   
   9.8e+06 |-+                                                              |   
           | O  OO  O O   O   O      O OO O OO OO                           |   
   9.7e+06 +----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                 fio.read_clat_99__us                           
                                                                                
  1.11e+07 +----------------------------------------------------------------+   
           |   + :   +  +    + :     :+            :+     + + :+    :+ :+  +|   
   1.1e+07 |-+  +     ++      +      +             +       +  +     +  +    |   
  1.09e+07 |-+                                                              |   
           |                                                                |   
  1.08e+07 |-+                                                              |   
  1.07e+07 |-+                                                              |   
           |                                                                |   
  1.06e+07 |-+                                                              |   
  1.05e+07 |-+                                                              |   
           |                                                                |   
  1.04e+07 |-+O    OO     O   O                                             |   
  1.03e+07 |-+                                                              |   
           | O  OO    OO O  O  O OO OO OO O OO OO                           |   
  1.02e+07 +----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                 fio.latency_10ms_                              
                                                                                
  100 +---------------------------------------------------------------------+   
   90 |-OO O OO O OO O OO O OO O OO O OO O OO O                             |   
      |                                                                     |   
   80 |-+                                                                   |   
   70 |-+                                                                   |   
      |                                                                     |   
   60 |-+                                                                   |   
   50 |-+                                                                   |   
   40 |-+                                                                   |   
      |                                                                     |   
   30 |-+                                                                   |   
   20 |-+                                                                   |   
      |                                                                     |   
   10 |-+          +.                   .+.                                 |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                 fio.latency_20ms_                              
                                                                                
  100 +---------------------------------------------------------------------+   
   90 |-+          +                     +                                  |   
      |                                                                     |   
   80 |-+                                                                   |   
   70 |-+                                                                   |   
      |                                                                     |   
   60 |-+                                                                   |   
   50 |-+                                                                   |   
   40 |-+                                                                   |   
      |                                                                     |   
   30 |-+                                                                   |   
   20 |-+                                                                   |   
      |                                                                     |   
   10 |-OO O OO O OO O OO O OO O OO O OO O OO O                             |   
    0 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                     fio.workload                               
                                                                                
  1.01e+06 +----------------------------------------------------------------+   
           | OO OO OO O  O  O  O OO O  OO O  O OO                           |   
     1e+06 |-+         O  O   O                                             |   
    990000 |-+                                                              |   
           |                                                                |   
    980000 |-+                                                              |   
    970000 |-+                                                              |   
           |                                                                |   
    960000 |-+                                                              |   
    950000 |-+                                                              |   
           |                                                                |   
    940000 |.+ .+      +.++. .+ .++. +. +. .++.++. +. +.+ .+.    +. +.+     |   
    930000 |-++  +. +.+     +  +    +  +  +       +  +   +   ++.+  +   +.++.|   
           |       +                                                        |   
    920000 +----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                              fio.time.minor_page_faults                        
                                                                                
   5.2e+08 +----------------------------------------------------------------+   
           |    O                    O      O                               |   
  5.15e+08 |-OO  O OO OO O  O  O OO O  OO O  O OO                           |   
   5.1e+08 |-+            O   O                                             |   
           |                                                                |   
  5.05e+08 |-+                                                              |   
     5e+08 |-+                                                              |   
           |                                                                |   
  4.95e+08 |-+                                                              |   
   4.9e+08 |-+                                                              |   
           |                                                                |   
  4.85e+08 |-+                                                              |   
   4.8e+08 |.+ .+      +.++. .+ .++. +. +. .+ .++. +. +.+ .+.       +.      |   
           |  +  +. +.+     +  +    +  +  +  +    +  +   +   ++.++.+  ++.++.|   
  4.75e+08 +----------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-csl-2ap3: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
=========================================================================================
cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase/ucode:
  cs-localhost/gcc-9/performance/ipv4/x86_64-rhel-8.3/16/debian-10.4-x86_64-20200603.cgz/300s/lkp-csl-2ap3/TCP_CRR/netperf/0x5002f01

commit: 
  077aa5c8dd ("drm/vc4: drv: Remove unused variable")
  9522750c66 ("Fonts: Replace discarded const qualifier")

077aa5c8dd6e2b9c 9522750c66c689b739e151fcdf8 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    274656            +8.5%     297973        netperf.Throughput_total_tps
     17166            +8.5%      18623        netperf.Throughput_tps
     11553 ±  2%      -7.2%      10719        netperf.time.minor_page_faults
     55.49            +7.1%      59.43        netperf.time.user_time
 1.604e+08            +8.0%  1.732e+08        netperf.time.voluntary_context_switches
  82396863            +8.5%   89391996        netperf.workload
 1.466e+08 ± 11%     -16.4%  1.225e+08        cpuidle.C1.usage
      9218 ± 29%     -31.3%       6333 ±  2%  numa-meminfo.node2.Mapped
    197303            -9.7%     178070        meminfo.KReclaimable
    197303            -9.7%     178070        meminfo.SReclaimable
      1598 ±  2%     +45.9%       2332 ± 28%  numa-vmstat.node1.nr_mapped
    494202 ±  4%     +24.4%     614896 ±  3%  numa-vmstat.node3.numa_hit
    405472 ± 13%     +24.4%     504570 ±  4%  numa-vmstat.node3.numa_local
    101869           -22.7%      78699        slabinfo.sock_inode_cache.active_objs
      2617           -22.8%       2021        slabinfo.sock_inode_cache.active_slabs
    102094           -22.8%      78862        slabinfo.sock_inode_cache.num_objs
      2617           -22.8%       2021        slabinfo.sock_inode_cache.num_slabs
     49206            -9.7%      44424        proc-vmstat.nr_slab_reclaimable
   1178597           +36.7%    1611082 ±  2%  proc-vmstat.numa_hit
   1085294           +39.9%    1517823 ±  2%  proc-vmstat.numa_local
   2213256 ±  4%    +157.5%    5699311 ±  4%  proc-vmstat.pgalloc_normal
   2260584 ±  4%    +154.4%    5749806 ±  4%  proc-vmstat.pgfree
    270522 ±  5%     +31.5%     355864 ±  8%  numa-numastat.node0.local_node
    293773 ±  6%     +29.1%     379131 ±  4%  numa-numastat.node0.numa_hit
    248107 ±  7%     +46.4%     363313 ± 11%  numa-numastat.node1.local_node
    279185 ±  7%     +38.5%     386626 ±  7%  numa-numastat.node1.numa_hit
    268004 ± 19%     +50.5%     403409 ±  5%  numa-numastat.node2.local_node
    283609 ± 13%     +47.7%     419000 ±  3%  numa-numastat.node2.numa_hit
    257580 ± 12%     +39.5%     359219 ±  3%  numa-numastat.node3.local_node
    280906 ±  6%     +38.9%     390276 ±  3%  numa-numastat.node3.numa_hit
     15.42           -11.6%      13.63 ±  2%  perf-stat.i.MPKI
 7.256e+09            +6.1%  7.697e+09        perf-stat.i.branch-instructions
 1.447e+08            +6.0%  1.535e+08        perf-stat.i.branch-misses
     21.16 ±  3%      -3.7       17.49 ±  3%  perf-stat.i.cache-miss-rate%
 1.166e+08 ±  3%     -23.3%   89396587 ±  4%  perf-stat.i.cache-misses
 5.519e+08            -7.6%  5.101e+08        perf-stat.i.cache-references
      2.26            -7.1%       2.10        perf-stat.i.cpi
 8.074e+10            -1.9%  7.919e+10        perf-stat.i.cpu-cycles
    662.25 ±  2%     +21.6%     805.45 ±  3%  perf-stat.i.cpu-migrations
    702.70 ±  3%     +30.7%     918.71 ±  4%  perf-stat.i.cycles-between-cache-misses
   1092204 ±  6%     +12.8%    1231474 ±  4%  perf-stat.i.dTLB-load-misses
 1.086e+10            +6.6%  1.158e+10        perf-stat.i.dTLB-loads
 6.352e+09            +6.7%   6.78e+09        perf-stat.i.dTLB-stores
     89.00            +0.7       89.73        perf-stat.i.iTLB-load-miss-rate%
 1.112e+08            +6.3%  1.182e+08        perf-stat.i.iTLB-load-misses
 3.633e+10            +6.3%  3.863e+10        perf-stat.i.instructions
      0.45            +8.2%       0.49        perf-stat.i.ipc
      0.42            -1.9%       0.41        perf-stat.i.metric.GHz
    130.59            +6.2%     138.63        perf-stat.i.metric.M/sec
  22300531           -15.6%   18829054        perf-stat.i.node-load-misses
   2087047           -17.4%    1724892 ±  2%  perf-stat.i.node-loads
  13228147           +10.7%   14645396        perf-stat.i.node-store-misses
    771369 ±  4%      +9.3%     842929 ±  2%  perf-stat.i.node-stores
     15.19           -13.1%      13.20        perf-stat.overall.MPKI
     21.12 ±  2%      -3.6       17.52 ±  2%  perf-stat.overall.cache-miss-rate%
      2.22            -7.8%       2.05        perf-stat.overall.cpi
    693.41 ±  3%     +28.0%     887.60 ±  4%  perf-stat.overall.cycles-between-cache-misses
     89.42            +0.9       90.30        perf-stat.overall.iTLB-load-miss-rate%
      0.45            +8.4%       0.49        perf-stat.overall.ipc
    134626            -2.0%     131970        perf-stat.overall.path-length
 7.232e+09            +6.1%  7.671e+09        perf-stat.ps.branch-instructions
 1.443e+08            +6.0%   1.53e+08        perf-stat.ps.branch-misses
 1.162e+08 ±  3%     -23.3%   89088157 ±  4%  perf-stat.ps.cache-misses
 5.501e+08            -7.6%  5.083e+08        perf-stat.ps.cache-references
 8.048e+10            -1.9%  7.892e+10        perf-stat.ps.cpu-cycles
    660.26 ±  2%     +21.6%     802.91 ±  3%  perf-stat.ps.cpu-migrations
   1088913 ±  6%     +12.7%    1227669 ±  4%  perf-stat.ps.dTLB-load-misses
 1.082e+10            +6.6%  1.154e+10        perf-stat.ps.dTLB-loads
 6.331e+09            +6.7%  6.756e+09        perf-stat.ps.dTLB-stores
 1.108e+08            +6.3%  1.178e+08        perf-stat.ps.iTLB-load-misses
 3.621e+10            +6.3%   3.85e+10        perf-stat.ps.instructions
  22226320           -15.6%   18763586        perf-stat.ps.node-load-misses
   2080373           -17.4%    1719338 ±  2%  perf-stat.ps.node-loads
  13184223           +10.7%   14595513        perf-stat.ps.node-store-misses
    769402 ±  4%      +9.3%     840981 ±  2%  perf-stat.ps.node-stores
 1.109e+13            +6.4%   1.18e+13        perf-stat.total.instructions
   1306583 ± 11%     +18.8%    1552049 ±  5%  softirqs.CPU10.NET_RX
   1292564 ±  8%     +10.2%    1424295 ±  7%  softirqs.CPU102.NET_RX
   1229647 ±  6%     +13.9%    1400017 ±  4%  softirqs.CPU104.NET_RX
     92423 ±  5%     +10.6%     102205 ±  3%  softirqs.CPU104.RCU
   1285264 ±  4%      +9.7%    1409545 ±  4%  softirqs.CPU109.NET_RX
   1189860 ±  8%     +31.8%    1567739 ±  3%  softirqs.CPU11.NET_RX
     90135 ±  5%     +19.1%     107361        softirqs.CPU11.RCU
   1186186 ±  7%     +15.8%    1373788 ±  6%  softirqs.CPU115.NET_RX
   1188780 ± 12%     +18.6%    1410406 ±  2%  softirqs.CPU116.NET_RX
   1141673 ± 14%     +38.2%    1577574 ± 17%  softirqs.CPU117.NET_RX
     91680 ±  9%     +19.0%     109106 ±  8%  softirqs.CPU117.RCU
   1225860 ±  9%     +14.7%    1406567 ±  6%  softirqs.CPU118.NET_RX
   1263255 ±  8%     +22.7%    1549935 ±  4%  softirqs.CPU12.NET_RX
     94381 ±  5%     +12.2%     105911        softirqs.CPU12.RCU
   1269390 ±  7%     +23.6%    1568529 ±  5%  softirqs.CPU13.NET_RX
     94951 ±  3%     +13.6%     107872 ±  4%  softirqs.CPU13.RCU
   1197487 ±  6%     +17.7%    1409284 ±  3%  softirqs.CPU134.NET_RX
     91649 ±  5%     +13.2%     103702 ±  2%  softirqs.CPU134.RCU
   1255515 ±  6%     +36.7%    1716105 ± 14%  softirqs.CPU14.NET_RX
     95758 ±  3%     +18.8%     113725 ±  8%  softirqs.CPU14.RCU
   1235074 ± 10%     +30.5%    1611446        softirqs.CPU15.NET_RX
     93594 ±  5%     +17.6%     110103 ±  2%  softirqs.CPU15.RCU
   1361868 ±  2%      -9.1%    1237730        softirqs.CPU150.NET_RX
   1170185 ± 13%     +28.2%    1499755 ±  4%  softirqs.CPU16.NET_RX
     90739 ±  8%     +17.2%     106340 ±  2%  softirqs.CPU16.RCU
   1213096 ±  9%     +24.9%    1514571 ±  4%  softirqs.CPU17.NET_RX
     93856 ±  5%     +14.2%     107199 ±  2%  softirqs.CPU17.RCU
   1403169 ±  4%     -12.8%    1224036 ±  3%  softirqs.CPU174.NET_RX
   1199022 ±  6%     +12.7%    1351797 ±  7%  softirqs.CPU177.NET_RX
   1220030 ±  7%     +24.6%    1519965 ±  4%  softirqs.CPU18.NET_RX
     93369 ±  5%     +13.2%     105677        softirqs.CPU18.RCU
   1189811 ± 13%     +24.4%    1480013 ±  4%  softirqs.CPU19.NET_RX
   1183057 ± 10%     +34.4%    1590153 ±  4%  softirqs.CPU20.NET_RX
     92686 ±  6%     +20.4%     111550 ±  4%  softirqs.CPU20.RCU
   1192136 ±  8%     +30.7%    1557702 ±  9%  softirqs.CPU21.NET_RX
   1258961 ±  9%     +25.3%    1578043 ±  3%  softirqs.CPU22.NET_RX
     97072 ±  6%     +12.7%     109390 ±  2%  softirqs.CPU22.RCU
   1170907 ± 12%     +26.9%    1486325 ±  6%  softirqs.CPU23.NET_RX
   1268084 ±  7%     +18.0%    1496069 ±  5%  softirqs.CPU33.NET_RX
   1264347 ±  6%     +17.4%    1483759 ±  5%  softirqs.CPU34.NET_RX
     97231 ±  3%     +11.3%     108181 ±  3%  softirqs.CPU34.RCU
   1248956 ±  2%     +16.6%    1456068 ±  9%  softirqs.CPU35.NET_RX
     96953 ±  4%      +9.4%     106093 ±  5%  softirqs.CPU35.RCU
   1269193 ±  9%     +19.1%    1511685 ±  6%  softirqs.CPU36.NET_RX
     96673 ±  3%     +12.4%     108624 ±  4%  softirqs.CPU36.RCU
   1243372 ±  8%     +17.4%    1459156 ±  6%  softirqs.CPU38.NET_RX
   1212529 ± 10%     +24.1%    1505068 ±  4%  softirqs.CPU39.NET_RX
     93910 ±  4%     +15.3%     108268 ±  2%  softirqs.CPU39.RCU
   1430979 ±  9%     +19.4%    1708843 ±  5%  softirqs.CPU4.NET_RX
    103983 ±  5%      +9.8%     114212 ±  2%  softirqs.CPU4.RCU
     95398 ±  2%     +12.2%     107048 ±  4%  softirqs.CPU40.RCU
   1294327 ±  4%     +15.5%    1495116 ±  9%  softirqs.CPU41.NET_RX
     98339 ±  5%     +11.1%     109215 ±  5%  softirqs.CPU41.RCU
   1235850 ±  5%     +21.4%    1500085 ±  9%  softirqs.CPU42.NET_RX
     94836 ±  2%     +12.9%     107034 ±  5%  softirqs.CPU42.RCU
   1175258 ±  9%     +27.5%    1498154 ±  5%  softirqs.CPU44.NET_RX
     91454 ±  6%     +17.8%     107732 ±  3%  softirqs.CPU44.RCU
   1361354 ±  8%     +21.3%    1651646 ±  7%  softirqs.CPU5.NET_RX
   1371339 ±  8%     +11.7%    1531614 ±  4%  softirqs.CPU51.NET_RX
   1285815 ±  5%      +8.2%    1391437 ±  4%  softirqs.CPU55.NET_RX
   1225706 ±  5%     +13.0%    1385345 ±  4%  softirqs.CPU57.NET_RX
   1212558 ±  5%      +9.5%    1327251 ±  5%  softirqs.CPU60.NET_RX
   1242888 ±  5%      +9.1%    1355795 ±  5%  softirqs.CPU61.NET_RX
   1228211 ±  7%     +13.5%    1393743 ±  5%  softirqs.CPU62.NET_RX
   1168045 ±  2%     +19.2%    1392755 ±  9%  softirqs.CPU64.NET_RX
   1200843 ±  5%     +16.9%    1403604 ±  7%  softirqs.CPU65.NET_RX
   1216538 ±  7%     +12.9%    1373654 ±  6%  softirqs.CPU66.NET_RX
   1206151 ±  7%     +11.2%    1341252 ±  5%  softirqs.CPU67.NET_RX
   1273798 ± 12%     +25.3%    1596184 ±  5%  softirqs.CPU7.NET_RX
     96063 ±  5%     +12.3%     107866 ±  3%  softirqs.CPU7.RCU
   1154072 ±  5%     +19.1%    1374085 ±  4%  softirqs.CPU70.NET_RX
   1135403 ±  6%     +17.7%    1335829 ±  5%  softirqs.CPU71.NET_RX
   1551155 ±  2%      +9.4%    1696887 ±  5%  softirqs.CPU74.NET_RX
     90781 ±  5%     +12.4%     101992 ±  2%  softirqs.CPU80.RCU
   1212453 ±  9%     +26.1%    1528497        softirqs.CPU81.NET_RX
     91613 ±  3%     +14.8%     105214        softirqs.CPU81.RCU
   1214248 ±  9%     +13.5%    1377647 ± 11%  softirqs.CPU85.NET_RX
     91372 ±  2%      +8.6%      99218 ±  6%  softirqs.CPU85.RCU
   1164366 ±  9%     +21.8%    1418032 ±  2%  softirqs.CPU87.NET_RX
     88963 ±  4%     +13.5%     100970 ±  3%  softirqs.CPU87.RCU
   1182137 ±  4%     +14.0%    1347671 ±  2%  softirqs.CPU88.NET_RX
     89419           +10.7%      98994 ±  2%  softirqs.CPU88.RCU
   1235032 ±  8%     +26.2%    1558211 ±  3%  softirqs.CPU9.NET_RX
   1215103 ± 12%     +17.5%    1427543 ±  5%  softirqs.CPU91.NET_RX
   1248217 ±  7%     +13.9%    1421582 ±  6%  softirqs.CPU94.NET_RX
   1185577 ± 10%     +24.8%    1479972 ±  3%  softirqs.CPU95.NET_RX
     90686 ±  4%     +15.7%     104891 ±  2%  softirqs.CPU95.RCU
   1425220 ±  4%     +12.3%    1599830 ±  3%  softirqs.CPU96.NET_RX
   2503376 ±  3%     +18.6%    2968400 ±  4%  interrupts.CAL:Function_call_interrupts
     16703 ±  4%     +34.4%      22442 ± 11%  interrupts.CPU0.CAL:Function_call_interrupts
     18446 ±  3%     -26.3%      13597 ±  6%  interrupts.CPU0.RES:Rescheduling_interrupts
     16935 ±  5%     +20.6%      20426 ± 11%  interrupts.CPU1.CAL:Function_call_interrupts
     19023 ±  3%     -30.7%      13186 ±  4%  interrupts.CPU1.RES:Rescheduling_interrupts
     13452 ± 12%     +42.7%      19195 ±  5%  interrupts.CPU10.CAL:Function_call_interrupts
     11877 ± 11%     +36.7%      16231 ±  7%  interrupts.CPU100.CAL:Function_call_interrupts
     13168 ±  7%     -16.6%      10982 ±  8%  interrupts.CPU100.RES:Rescheduling_interrupts
     13176 ±  9%     -26.6%       9676 ± 11%  interrupts.CPU101.RES:Rescheduling_interrupts
     13131 ± 12%     +22.2%      16050 ±  5%  interrupts.CPU102.CAL:Function_call_interrupts
     12150 ±  4%     -22.2%       9452 ± 10%  interrupts.CPU102.RES:Rescheduling_interrupts
     14805 ±  8%     +20.6%      17857 ±  8%  interrupts.CPU103.CAL:Function_call_interrupts
     13030 ± 13%     -25.7%       9680 ±  8%  interrupts.CPU103.RES:Rescheduling_interrupts
      2038 ± 24%     -32.2%       1381 ±  8%  interrupts.CPU104.NMI:Non-maskable_interrupts
      2038 ± 24%     -32.2%       1381 ±  8%  interrupts.CPU104.PMI:Performance_monitoring_interrupts
     12003 ± 11%     -21.5%       9423 ±  6%  interrupts.CPU104.RES:Rescheduling_interrupts
     12564 ± 14%     +30.8%      16435 ±  6%  interrupts.CPU105.CAL:Function_call_interrupts
      2095 ± 22%     -40.0%       1257 ± 24%  interrupts.CPU105.NMI:Non-maskable_interrupts
      2095 ± 22%     -40.0%       1257 ± 24%  interrupts.CPU105.PMI:Performance_monitoring_interrupts
     13416 ±  4%     -23.4%      10283 ±  7%  interrupts.CPU105.RES:Rescheduling_interrupts
     12435 ±  8%     +34.4%      16709 ± 10%  interrupts.CPU106.CAL:Function_call_interrupts
     13297 ± 10%     -28.1%       9565 ±  5%  interrupts.CPU106.RES:Rescheduling_interrupts
     11240 ± 17%     +39.5%      15684 ±  7%  interrupts.CPU107.CAL:Function_call_interrupts
     12815 ±  5%     -31.2%       8820 ± 10%  interrupts.CPU107.RES:Rescheduling_interrupts
     12233 ±  5%     +31.3%      16067 ±  6%  interrupts.CPU108.CAL:Function_call_interrupts
      2419 ± 25%     -44.3%       1347 ± 29%  interrupts.CPU108.NMI:Non-maskable_interrupts
      2419 ± 25%     -44.3%       1347 ± 29%  interrupts.CPU108.PMI:Performance_monitoring_interrupts
     13029 ±  7%     -25.0%       9775 ±  7%  interrupts.CPU108.RES:Rescheduling_interrupts
     12995 ± 14%     +32.3%      17198 ±  5%  interrupts.CPU109.CAL:Function_call_interrupts
     12676 ±  5%     -20.7%      10053 ±  2%  interrupts.CPU109.RES:Rescheduling_interrupts
     11600 ±  9%     +37.0%      15896 ±  3%  interrupts.CPU11.CAL:Function_call_interrupts
     11663 ±  9%     -14.7%       9947 ±  3%  interrupts.CPU11.RES:Rescheduling_interrupts
     13049 ± 11%     +32.3%      17263 ±  8%  interrupts.CPU110.CAL:Function_call_interrupts
     11985 ± 16%     +30.7%      15660 ± 13%  interrupts.CPU112.CAL:Function_call_interrupts
     11687 ±  8%     -24.4%       8840 ±  4%  interrupts.CPU112.RES:Rescheduling_interrupts
     13092 ± 10%     +19.5%      15647 ±  4%  interrupts.CPU113.CAL:Function_call_interrupts
      2018 ± 12%     -20.3%       1608 ± 11%  interrupts.CPU113.NMI:Non-maskable_interrupts
      2018 ± 12%     -20.3%       1608 ± 11%  interrupts.CPU113.PMI:Performance_monitoring_interrupts
     11622 ± 11%     -26.0%       8604 ± 10%  interrupts.CPU113.RES:Rescheduling_interrupts
     12177 ± 13%     -23.9%       9268 ±  8%  interrupts.CPU114.RES:Rescheduling_interrupts
      1488 ± 13%     +25.4%       1866 ± 10%  interrupts.CPU115.NMI:Non-maskable_interrupts
      1488 ± 13%     +25.4%       1866 ± 10%  interrupts.CPU115.PMI:Performance_monitoring_interrupts
     11487 ± 16%     +24.0%      14240 ±  7%  interrupts.CPU116.CAL:Function_call_interrupts
     11482 ± 13%     +49.2%      17130 ± 14%  interrupts.CPU117.CAL:Function_call_interrupts
     11092 ±  9%     +31.8%      14617 ± 11%  interrupts.CPU118.CAL:Function_call_interrupts
     12247 ± 11%     -24.5%       9251 ±  2%  interrupts.CPU118.RES:Rescheduling_interrupts
     11847 ± 17%     -25.0%       8883 ±  4%  interrupts.CPU119.RES:Rescheduling_interrupts
     11474 ±  8%     +50.0%      17211 ±  2%  interrupts.CPU12.CAL:Function_call_interrupts
     12131 ±  7%     -16.5%      10127 ±  2%  interrupts.CPU12.RES:Rescheduling_interrupts
      1396 ± 29%     +57.0%       2192 ± 11%  interrupts.CPU120.NMI:Non-maskable_interrupts
      1396 ± 29%     +57.0%       2192 ± 11%  interrupts.CPU120.PMI:Performance_monitoring_interrupts
     15909 ±  8%     -32.5%      10742 ±  5%  interrupts.CPU120.RES:Rescheduling_interrupts
     14532 ±  9%     +17.5%      17076 ±  9%  interrupts.CPU121.CAL:Function_call_interrupts
     14191 ±  6%     -24.9%      10660 ±  7%  interrupts.CPU121.RES:Rescheduling_interrupts
     13796 ±  6%     -28.1%       9926 ±  4%  interrupts.CPU122.RES:Rescheduling_interrupts
     13143 ± 13%     -25.3%       9814 ± 11%  interrupts.CPU123.RES:Rescheduling_interrupts
     12332 ±  6%     -27.4%       8957 ±  3%  interrupts.CPU124.RES:Rescheduling_interrupts
     12853 ± 10%     -33.1%       8604 ±  9%  interrupts.CPU125.RES:Rescheduling_interrupts
     11966 ±  9%     -28.7%       8531 ±  8%  interrupts.CPU126.RES:Rescheduling_interrupts
     13278 ±  9%     -31.2%       9134 ± 10%  interrupts.CPU127.RES:Rescheduling_interrupts
     12628 ± 14%     -26.6%       9275 ±  7%  interrupts.CPU128.RES:Rescheduling_interrupts
     12222 ±  7%     +20.3%      14698 ±  6%  interrupts.CPU129.CAL:Function_call_interrupts
     12633 ±  5%     -23.6%       9646 ±  9%  interrupts.CPU129.RES:Rescheduling_interrupts
     12118 ± 13%     +43.3%      17361 ±  8%  interrupts.CPU13.CAL:Function_call_interrupts
     12397 ± 17%     -29.2%       8776 ±  6%  interrupts.CPU130.RES:Rescheduling_interrupts
     11133 ± 11%     +22.8%      13671 ± 12%  interrupts.CPU131.CAL:Function_call_interrupts
     12525 ± 11%     -26.4%       9222 ± 10%  interrupts.CPU131.RES:Rescheduling_interrupts
     11877 ±  5%     +27.4%      15134 ± 12%  interrupts.CPU132.CAL:Function_call_interrupts
     12862 ± 13%     -28.8%       9152 ±  9%  interrupts.CPU132.RES:Rescheduling_interrupts
     12192 ± 12%     -26.9%       8915 ± 12%  interrupts.CPU133.RES:Rescheduling_interrupts
     12129 ±  7%     +21.7%      14765 ±  9%  interrupts.CPU134.CAL:Function_call_interrupts
     11848 ± 10%     -23.7%       9037        interrupts.CPU135.RES:Rescheduling_interrupts
     12053 ± 13%     -26.4%       8869 ± 15%  interrupts.CPU136.RES:Rescheduling_interrupts
     13435 ±  2%     +23.9%      16646 ±  3%  interrupts.CPU137.CAL:Function_call_interrupts
     11961 ±  5%     -24.3%       9055 ±  6%  interrupts.CPU137.RES:Rescheduling_interrupts
     11887 ±  7%     -25.0%       8921 ± 10%  interrupts.CPU138.RES:Rescheduling_interrupts
     13318 ± 10%     -38.3%       8222 ±  9%  interrupts.CPU139.RES:Rescheduling_interrupts
     14140 ± 12%     +33.8%      18922 ±  8%  interrupts.CPU14.CAL:Function_call_interrupts
     13757 ± 28%     -36.9%       8685 ±  7%  interrupts.CPU140.RES:Rescheduling_interrupts
     12524 ±  8%     -30.7%       8679 ± 15%  interrupts.CPU141.RES:Rescheduling_interrupts
     11761 ±  5%     -28.6%       8398 ± 15%  interrupts.CPU142.RES:Rescheduling_interrupts
     11928 ±  8%     -33.3%       7961 ± 13%  interrupts.CPU143.RES:Rescheduling_interrupts
     16940 ± 13%     -41.1%       9971 ±  7%  interrupts.CPU144.RES:Rescheduling_interrupts
     14430 ± 11%     -35.8%       9267 ±  7%  interrupts.CPU145.RES:Rescheduling_interrupts
     14064 ± 13%     -31.5%       9627 ± 10%  interrupts.CPU146.RES:Rescheduling_interrupts
     15172 ± 25%     -41.5%       8872 ±  5%  interrupts.CPU147.RES:Rescheduling_interrupts
     13587 ± 10%     -37.0%       8556 ± 12%  interrupts.CPU148.RES:Rescheduling_interrupts
     13468 ±  8%     -37.6%       8400 ±  9%  interrupts.CPU149.RES:Rescheduling_interrupts
     11112 ± 13%     +35.5%      15056 ±  5%  interrupts.CPU15.CAL:Function_call_interrupts
     13356 ±  6%     -41.6%       7800 ±  8%  interrupts.CPU150.RES:Rescheduling_interrupts
     13150 ± 11%     -42.5%       7558 ±  4%  interrupts.CPU151.RES:Rescheduling_interrupts
     12377 ± 10%     -33.0%       8288 ±  7%  interrupts.CPU152.RES:Rescheduling_interrupts
     11941 ± 15%     -24.4%       9022 ±  8%  interrupts.CPU153.RES:Rescheduling_interrupts
     11426 ± 12%     -22.7%       8835 ± 14%  interrupts.CPU154.RES:Rescheduling_interrupts
     12467 ± 14%     -33.5%       8286 ± 12%  interrupts.CPU155.RES:Rescheduling_interrupts
     11526 ± 11%     -32.6%       7772 ± 13%  interrupts.CPU156.RES:Rescheduling_interrupts
     11235 ±  2%     -33.9%       7425 ±  6%  interrupts.CPU157.RES:Rescheduling_interrupts
     11375 ± 12%     -28.7%       8108 ± 13%  interrupts.CPU158.RES:Rescheduling_interrupts
     11901 ± 10%     -25.9%       8814 ±  8%  interrupts.CPU159.RES:Rescheduling_interrupts
     11732 ± 14%     +46.3%      17160 ± 10%  interrupts.CPU16.CAL:Function_call_interrupts
     11467 ±  9%     -31.7%       7834 ±  3%  interrupts.CPU160.RES:Rescheduling_interrupts
     11610 ±  9%     -31.7%       7930 ± 12%  interrupts.CPU161.RES:Rescheduling_interrupts
      1508 ± 11%     -28.1%       1084 ± 20%  interrupts.CPU162.NMI:Non-maskable_interrupts
      1508 ± 11%     -28.1%       1084 ± 20%  interrupts.CPU162.PMI:Performance_monitoring_interrupts
     10857 ± 14%     -30.2%       7577 ± 10%  interrupts.CPU162.RES:Rescheduling_interrupts
      1689 ± 12%     -27.4%       1226 ± 26%  interrupts.CPU163.NMI:Non-maskable_interrupts
      1689 ± 12%     -27.4%       1226 ± 26%  interrupts.CPU163.PMI:Performance_monitoring_interrupts
     11713 ± 13%     -32.2%       7938 ±  4%  interrupts.CPU163.RES:Rescheduling_interrupts
     10641 ± 16%     -25.2%       7964 ±  4%  interrupts.CPU164.RES:Rescheduling_interrupts
     11600 ± 11%     -33.4%       7728 ± 13%  interrupts.CPU165.RES:Rescheduling_interrupts
     11341 ±  5%     -38.7%       6950 ±  8%  interrupts.CPU166.RES:Rescheduling_interrupts
     10699 ± 12%     -30.0%       7494 ±  7%  interrupts.CPU167.RES:Rescheduling_interrupts
     15045 ±  6%     +37.7%      20715 ±  8%  interrupts.CPU168.CAL:Function_call_interrupts
     16275 ±  5%     -30.9%      11249 ±  7%  interrupts.CPU168.RES:Rescheduling_interrupts
     17021 ± 34%     -40.2%      10176 ± 11%  interrupts.CPU169.RES:Rescheduling_interrupts
     10706 ± 15%     +46.2%      15650 ± 11%  interrupts.CPU17.CAL:Function_call_interrupts
     11657 ± 10%     -13.8%      10054 ±  4%  interrupts.CPU17.RES:Rescheduling_interrupts
     13747 ±  7%     -25.3%      10264 ± 10%  interrupts.CPU170.RES:Rescheduling_interrupts
     14345 ±  3%     -34.4%       9405 ± 12%  interrupts.CPU171.RES:Rescheduling_interrupts
     13908 ±  7%     -29.5%       9800 ±  5%  interrupts.CPU172.RES:Rescheduling_interrupts
     13770 ±  5%     -36.8%       8706 ±  5%  interrupts.CPU173.RES:Rescheduling_interrupts
     13645 ±  5%     -42.7%       7822 ±  8%  interrupts.CPU174.RES:Rescheduling_interrupts
     11993 ±  7%     -26.6%       8803 ±  2%  interrupts.CPU175.RES:Rescheduling_interrupts
     12843 ±  8%     -28.4%       9199 ± 10%  interrupts.CPU176.RES:Rescheduling_interrupts
     11753 ±  3%     -19.7%       9433 ±  7%  interrupts.CPU177.RES:Rescheduling_interrupts
     12736 ±  6%     -38.7%       7803 ±  7%  interrupts.CPU178.RES:Rescheduling_interrupts
     12028 ±  7%     -32.1%       8170 ±  4%  interrupts.CPU179.RES:Rescheduling_interrupts
     13626 ± 11%     +42.5%      19421 ±  9%  interrupts.CPU18.CAL:Function_call_interrupts
     11110 ±  6%     +23.6%      13736 ± 12%  interrupts.CPU180.CAL:Function_call_interrupts
     12716 ±  8%     -26.0%       9415 ±  4%  interrupts.CPU180.RES:Rescheduling_interrupts
     12367 ±  4%     -32.0%       8409 ±  6%  interrupts.CPU181.RES:Rescheduling_interrupts
     11952 ± 12%     -23.0%       9203 ± 13%  interrupts.CPU182.RES:Rescheduling_interrupts
     12116 ±  8%     +28.2%      15530 ±  8%  interrupts.CPU183.CAL:Function_call_interrupts
     12344 ±  7%     -32.2%       8367 ±  6%  interrupts.CPU183.RES:Rescheduling_interrupts
     12643 ±  8%     -33.0%       8466 ±  4%  interrupts.CPU184.RES:Rescheduling_interrupts
     12430 ±  3%     -30.7%       8609 ±  3%  interrupts.CPU185.RES:Rescheduling_interrupts
     12697 ±  7%     -38.3%       7839 ± 11%  interrupts.CPU186.RES:Rescheduling_interrupts
     11914 ± 11%     -32.6%       8026 ±  8%  interrupts.CPU187.RES:Rescheduling_interrupts
     13174 ± 10%     -34.3%       8657 ± 12%  interrupts.CPU188.RES:Rescheduling_interrupts
     11225 ± 12%     +20.7%      13550 ±  6%  interrupts.CPU189.CAL:Function_call_interrupts
     12562 ± 11%     -31.6%       8597 ± 14%  interrupts.CPU189.RES:Rescheduling_interrupts
     11894 ±  9%     +32.0%      15696 ±  4%  interrupts.CPU19.CAL:Function_call_interrupts
     12.75 ±118%    +366.7%      59.50 ± 82%  interrupts.CPU19.TLB:TLB_shootdowns
     12760 ±  7%     -33.5%       8482 ±  9%  interrupts.CPU190.RES:Rescheduling_interrupts
      1350 ± 22%     +42.7%       1927 ± 18%  interrupts.CPU191.NMI:Non-maskable_interrupts
      1350 ± 22%     +42.7%       1927 ± 18%  interrupts.CPU191.PMI:Performance_monitoring_interrupts
     12420 ± 11%     -30.7%       8602 ±  4%  interrupts.CPU191.RES:Rescheduling_interrupts
     15151 ± 16%     +29.1%      19567 ±  5%  interrupts.CPU2.CAL:Function_call_interrupts
     16369 ±  8%     -24.3%      12385 ±  5%  interrupts.CPU2.RES:Rescheduling_interrupts
     11903 ± 14%     +33.2%      15853 ±  6%  interrupts.CPU20.CAL:Function_call_interrupts
     11553 ± 18%     +52.6%      17630 ± 11%  interrupts.CPU21.CAL:Function_call_interrupts
     13602 ± 13%     +47.7%      20087 ±  9%  interrupts.CPU22.CAL:Function_call_interrupts
     12685 ± 12%     -14.7%      10821 ±  2%  interrupts.CPU22.RES:Rescheduling_interrupts
     10842 ± 12%     +38.1%      14974 ±  7%  interrupts.CPU23.CAL:Function_call_interrupts
     17511 ± 18%     +24.8%      21855 ±  7%  interrupts.CPU24.CAL:Function_call_interrupts
     20276 ±  4%     -34.9%      13197 ±  3%  interrupts.CPU24.RES:Rescheduling_interrupts
     18211 ±  5%     -38.1%      11268 ±  4%  interrupts.CPU25.RES:Rescheduling_interrupts
     14796 ±  7%     -28.0%      10658 ±  6%  interrupts.CPU26.RES:Rescheduling_interrupts
     14619 ± 10%     -31.1%      10068 ±  4%  interrupts.CPU27.RES:Rescheduling_interrupts
     12400 ±  3%     +19.9%      14871 ±  8%  interrupts.CPU28.CAL:Function_call_interrupts
     13367 ±  8%     -28.6%       9541 ±  3%  interrupts.CPU28.RES:Rescheduling_interrupts
     11655 ±  5%     +31.5%      15331 ±  9%  interrupts.CPU29.CAL:Function_call_interrupts
     12512 ±  3%     -22.5%       9703 ±  2%  interrupts.CPU29.RES:Rescheduling_interrupts
     15067 ± 11%     +28.4%      19339 ±  5%  interrupts.CPU3.CAL:Function_call_interrupts
     14915 ±  4%     -23.5%      11413 ± 10%  interrupts.CPU3.RES:Rescheduling_interrupts
     11931 ±  4%     +25.9%      15022 ±  3%  interrupts.CPU30.CAL:Function_call_interrupts
    941.75 ± 19%     +57.4%       1482 ± 24%  interrupts.CPU30.NMI:Non-maskable_interrupts
    941.75 ± 19%     +57.4%       1482 ± 24%  interrupts.CPU30.PMI:Performance_monitoring_interrupts
     12795 ±  3%     -31.0%       8833 ±  6%  interrupts.CPU30.RES:Rescheduling_interrupts
     13061 ±  7%     -32.7%       8785 ±  6%  interrupts.CPU31.RES:Rescheduling_interrupts
     11860 ±  6%     +16.0%      13752 ±  7%  interrupts.CPU32.CAL:Function_call_interrupts
     12894 ± 12%     -31.0%       8898 ±  8%  interrupts.CPU32.RES:Rescheduling_interrupts
     11660 ± 11%     +28.0%      14922 ±  9%  interrupts.CPU33.CAL:Function_call_interrupts
     12762 ±  6%     -24.2%       9672 ±  6%  interrupts.CPU33.RES:Rescheduling_interrupts
     14192 ±  5%     +16.9%      16588 ± 13%  interrupts.CPU34.CAL:Function_call_interrupts
     12540 ±  8%     -21.6%       9829 ±  6%  interrupts.CPU34.RES:Rescheduling_interrupts
     12065 ±  4%     -22.5%       9349 ±  5%  interrupts.CPU35.RES:Rescheduling_interrupts
     12009 ±  4%     +26.3%      15169 ±  8%  interrupts.CPU36.CAL:Function_call_interrupts
     13326 ±  8%     +25.6%      16740 ± 11%  interrupts.CPU37.CAL:Function_call_interrupts
     12297 ± 11%     -18.4%      10033 ± 10%  interrupts.CPU37.RES:Rescheduling_interrupts
     13239 ± 12%     +19.6%      15830 ±  5%  interrupts.CPU38.CAL:Function_call_interrupts
     12195 ± 11%     -19.5%       9816 ±  9%  interrupts.CPU38.RES:Rescheduling_interrupts
     11668 ±  8%     +26.2%      14727 ±  5%  interrupts.CPU39.CAL:Function_call_interrupts
     13778 ± 10%     +43.2%      19736 ±  7%  interrupts.CPU4.CAL:Function_call_interrupts
     13412 ±  9%     -15.8%      11294 ±  8%  interrupts.CPU4.RES:Rescheduling_interrupts
     11821 ±  6%     +31.6%      15559 ±  9%  interrupts.CPU40.CAL:Function_call_interrupts
     12185 ±  5%     -19.3%       9830 ± 15%  interrupts.CPU40.RES:Rescheduling_interrupts
     12422 ±  5%     +19.2%      14804 ± 10%  interrupts.CPU41.CAL:Function_call_interrupts
     12261 ±  7%     -20.2%       9789 ± 11%  interrupts.CPU41.RES:Rescheduling_interrupts
     11983 ±  6%     +30.3%      15614 ±  8%  interrupts.CPU42.CAL:Function_call_interrupts
     12002 ±  5%     -16.0%      10080 ± 11%  interrupts.CPU42.RES:Rescheduling_interrupts
     12238 ±  7%     +30.2%      15928 ± 12%  interrupts.CPU43.CAL:Function_call_interrupts
      1938 ±  6%     -20.0%       1550 ± 12%  interrupts.CPU43.NMI:Non-maskable_interrupts
      1938 ±  6%     -20.0%       1550 ± 12%  interrupts.CPU43.PMI:Performance_monitoring_interrupts
     12539 ± 11%     -19.5%      10093 ±  9%  interrupts.CPU43.RES:Rescheduling_interrupts
     13016 ± 11%     +26.9%      16521 ± 11%  interrupts.CPU44.CAL:Function_call_interrupts
     11120 ±  5%     -11.9%       9800 ±  7%  interrupts.CPU44.RES:Rescheduling_interrupts
     13107 ±  8%     -30.1%       9156 ± 12%  interrupts.CPU46.RES:Rescheduling_interrupts
     12029 ±  7%     -24.1%       9125 ± 11%  interrupts.CPU47.RES:Rescheduling_interrupts
     21254 ±  8%     -35.9%      13629 ±  2%  interrupts.CPU48.RES:Rescheduling_interrupts
     17898 ± 10%     -31.3%      12287 ± 13%  interrupts.CPU49.RES:Rescheduling_interrupts
     11294 ± 12%     +51.5%      17106 ± 13%  interrupts.CPU5.CAL:Function_call_interrupts
     12656 ±  5%     -15.4%      10701 ±  9%  interrupts.CPU5.RES:Rescheduling_interrupts
     15636 ± 13%     -31.6%      10700 ±  7%  interrupts.CPU50.RES:Rescheduling_interrupts
     12713 ±  5%     +24.1%      15776 ±  7%  interrupts.CPU51.CAL:Function_call_interrupts
     13332 ± 12%     -26.1%       9855 ±  3%  interrupts.CPU51.RES:Rescheduling_interrupts
     13084 ± 10%     -26.0%       9684 ±  9%  interrupts.CPU52.RES:Rescheduling_interrupts
     12325 ±  4%     -26.3%       9078 ±  5%  interrupts.CPU53.RES:Rescheduling_interrupts
     13589 ± 10%     -35.7%       8741 ±  7%  interrupts.CPU54.RES:Rescheduling_interrupts
     12675 ± 10%     -31.0%       8749 ±  6%  interrupts.CPU55.RES:Rescheduling_interrupts
     12922 ± 11%     -31.3%       8883 ±  5%  interrupts.CPU56.RES:Rescheduling_interrupts
     11264 ±  6%     -23.7%       8600 ±  7%  interrupts.CPU57.RES:Rescheduling_interrupts
     12177 ±  6%     -27.0%       8888 ±  7%  interrupts.CPU58.RES:Rescheduling_interrupts
     12471 ±  8%     +21.9%      15197 ± 10%  interrupts.CPU59.CAL:Function_call_interrupts
     12125 ±  7%     -27.4%       8801 ±  7%  interrupts.CPU59.RES:Rescheduling_interrupts
     11415 ± 13%     +37.3%      15673 ± 13%  interrupts.CPU6.CAL:Function_call_interrupts
     12053 ±  8%     +15.1%      13874 ±  2%  interrupts.CPU60.CAL:Function_call_interrupts
     11432 ± 12%     -24.4%       8645 ±  4%  interrupts.CPU60.RES:Rescheduling_interrupts
     12279 ±  7%     -32.7%       8266 ±  3%  interrupts.CPU61.RES:Rescheduling_interrupts
     12337 ± 10%     -25.0%       9258 ±  5%  interrupts.CPU62.RES:Rescheduling_interrupts
      1739 ±  4%     -24.8%       1308 ± 26%  interrupts.CPU63.NMI:Non-maskable_interrupts
      1739 ±  4%     -24.8%       1308 ± 26%  interrupts.CPU63.PMI:Performance_monitoring_interrupts
     11284 ±  4%     -21.8%       8823 ± 14%  interrupts.CPU64.RES:Rescheduling_interrupts
     11142 ±  7%     +22.0%      13599 ±  9%  interrupts.CPU65.CAL:Function_call_interrupts
     11877 ±  7%     -25.0%       8911 ±  9%  interrupts.CPU65.RES:Rescheduling_interrupts
     11972 ±  9%     -27.1%       8722 ± 11%  interrupts.CPU66.RES:Rescheduling_interrupts
     11894 ± 14%     +22.8%      14610 ±  7%  interrupts.CPU67.CAL:Function_call_interrupts
     11604 ±  9%     -24.0%       8821 ±  6%  interrupts.CPU67.RES:Rescheduling_interrupts
     11102 ±  7%     -21.5%       8710 ±  8%  interrupts.CPU68.RES:Rescheduling_interrupts
     11841 ± 12%     -25.9%       8773 ±  8%  interrupts.CPU69.RES:Rescheduling_interrupts
     12201 ± 11%     +44.9%      17684 ±  8%  interrupts.CPU7.CAL:Function_call_interrupts
     10164 ± 10%     +21.1%      12304 ±  4%  interrupts.CPU70.CAL:Function_call_interrupts
     11103 ±  7%     -23.9%       8445 ±  5%  interrupts.CPU70.RES:Rescheduling_interrupts
     10753 ±  5%     +19.5%      12849 ±  7%  interrupts.CPU71.CAL:Function_call_interrupts
     11240 ±  5%     -24.3%       8506 ±  9%  interrupts.CPU71.RES:Rescheduling_interrupts
     19043 ±  9%     +19.7%      22789 ±  2%  interrupts.CPU72.CAL:Function_call_interrupts
     20936 ±  7%     -35.1%      13579 ±  9%  interrupts.CPU72.RES:Rescheduling_interrupts
     16593 ± 11%     -24.6%      12508 ±  8%  interrupts.CPU73.RES:Rescheduling_interrupts
     16385 ±  6%     +18.7%      19445 ±  5%  interrupts.CPU74.CAL:Function_call_interrupts
     15335 ±  3%     -28.1%      11025 ±  5%  interrupts.CPU74.RES:Rescheduling_interrupts
     14814 ±  4%     -26.4%      10910 ±  6%  interrupts.CPU75.RES:Rescheduling_interrupts
     13901 ±  6%     -23.1%      10692 ±  8%  interrupts.CPU76.RES:Rescheduling_interrupts
     13490 ±  3%     -24.1%      10235 ± 14%  interrupts.CPU77.RES:Rescheduling_interrupts
     12764 ±  9%     -27.2%       9296 ±  8%  interrupts.CPU78.RES:Rescheduling_interrupts
     12432 ±  6%     +26.7%      15756 ±  6%  interrupts.CPU79.CAL:Function_call_interrupts
     12326 ± 15%     -20.7%       9777 ±  4%  interrupts.CPU79.RES:Rescheduling_interrupts
     14231 ± 30%     -24.8%      10697 ±  4%  interrupts.CPU8.RES:Rescheduling_interrupts
     13727 ±  8%     +28.3%      17618 ±  7%  interrupts.CPU80.CAL:Function_call_interrupts
     12417 ± 13%     -25.3%       9273 ± 11%  interrupts.CPU80.RES:Rescheduling_interrupts
     12969 ± 10%     +29.3%      16769 ±  8%  interrupts.CPU81.CAL:Function_call_interrupts
     11898 ±  6%     -14.6%      10164 ±  3%  interrupts.CPU81.RES:Rescheduling_interrupts
     11630 ±  7%     -24.3%       8802 ±  8%  interrupts.CPU82.RES:Rescheduling_interrupts
     14005 ± 14%     +29.5%      18141 ± 13%  interrupts.CPU83.CAL:Function_call_interrupts
     11465 ± 11%     -19.0%       9286 ±  5%  interrupts.CPU83.RES:Rescheduling_interrupts
     12453 ± 15%     -20.5%       9895 ±  8%  interrupts.CPU84.RES:Rescheduling_interrupts
     12384 ±  8%     -24.2%       9389 ± 11%  interrupts.CPU85.RES:Rescheduling_interrupts
     12644 ± 13%     +18.5%      14982 ±  7%  interrupts.CPU86.CAL:Function_call_interrupts
     11803 ±  9%     -21.2%       9299 ±  7%  interrupts.CPU86.RES:Rescheduling_interrupts
     12446 ± 14%     +29.1%      16065 ±  9%  interrupts.CPU87.CAL:Function_call_interrupts
     11141 ±  7%     -18.8%       9049 ±  3%  interrupts.CPU87.RES:Rescheduling_interrupts
     14719 ±  7%     +26.1%      18565 ± 14%  interrupts.CPU88.CAL:Function_call_interrupts
     11846 ±  4%     -25.3%       8848 ±  4%  interrupts.CPU88.RES:Rescheduling_interrupts
     11951 ±  9%     -27.5%       8663 ± 14%  interrupts.CPU89.RES:Rescheduling_interrupts
     11886 ±  7%     +49.7%      17798 ±  9%  interrupts.CPU9.CAL:Function_call_interrupts
     11990 ±  4%     -12.5%      10497 ±  7%  interrupts.CPU9.RES:Rescheduling_interrupts
     12364 ±  8%     -28.2%       8882 ±  8%  interrupts.CPU90.RES:Rescheduling_interrupts
     11851 ±  8%     -21.0%       9362 ± 11%  interrupts.CPU92.RES:Rescheduling_interrupts
     12804 ± 10%     -29.4%       9037 ±  5%  interrupts.CPU93.RES:Rescheduling_interrupts
     12328 ±  4%     -28.2%       8845 ± 11%  interrupts.CPU94.RES:Rescheduling_interrupts
     11935 ±  7%     +32.5%      15810 ±  7%  interrupts.CPU95.CAL:Function_call_interrupts
     11505 ±  8%     -17.3%       9512 ±  6%  interrupts.CPU95.RES:Rescheduling_interrupts
     14772 ±  8%     +33.2%      19670 ±  6%  interrupts.CPU96.CAL:Function_call_interrupts
     14594 ±  5%     -23.4%      11174 ±  4%  interrupts.CPU96.RES:Rescheduling_interrupts
     17057 ± 11%     +29.8%      22136 ±  4%  interrupts.CPU97.CAL:Function_call_interrupts
     14688 ±  7%     -19.8%      11777 ±  7%  interrupts.CPU97.RES:Rescheduling_interrupts
     14428 ± 11%     -23.6%      11025 ±  6%  interrupts.CPU98.RES:Rescheduling_interrupts
     13953 ± 12%     -26.2%      10294 ±  2%  interrupts.CPU99.RES:Rescheduling_interrupts
   2469658 ±  2%     -26.2%    1823809 ±  2%  interrupts.RES:Rescheduling_interrupts
     46.92 ±  6%      -2.5       44.37        perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     46.94 ±  6%      -2.5       44.39        perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     46.95 ±  6%      -2.5       44.40        perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
     47.26 ±  6%      -2.5       44.72        perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
      2.54 ±  6%      -0.8        1.73 ±  6%  perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
      1.58 ±  4%      -0.2        1.35 ±  9%  perf-profile.calltrace.cycles-pp.tcp_conn_request.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu
      0.55 ±  5%      +0.2        0.71 ±  5%  perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt
      1.14 ±  5%      +0.2        1.30 ±  4%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      1.16 ±  5%      +0.2        1.32 ±  4%  perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      1.15 ±  5%      +0.2        1.32 ±  4%  perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      5.52 ±  4%      +0.3        5.85        perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_close
      4.04 ±  4%      +0.3        4.36        perf-profile.calltrace.cycles-pp.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
      4.01 ±  4%      +0.3        4.34        perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64
      5.75 ±  4%      +0.3        6.08        perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_close.inet_release
      0.28 ±100%      +0.3        0.61 ±  9%  perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary
      3.05 ±  7%      +0.4        3.44        perf-profile.calltrace.cycles-pp.__sys_accept4_file.__sys_accept4.__x64_sys_accept.do_syscall_64.entry_SYSCALL_64_after_hwframe
      4.52 ±  8%      +0.4        4.96 ±  2%  perf-profile.calltrace.cycles-pp.kmem_cache_free.rcu_do_batch.rcu_core.__softirqentry_text_start.run_ksoftirqd
      4.61 ±  8%      +0.4        5.05 ±  2%  perf-profile.calltrace.cycles-pp.smpboot_thread_fn.kthread.ret_from_fork
      4.37 ±  8%      +0.5        4.89 ±  3%  perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
      2.19 ± 21%      +0.5        2.73 ±  5%  perf-profile.calltrace.cycles-pp.__memcg_kmem_uncharge.drain_obj_stock.refill_obj_stock.kmem_cache_free.rcu_do_batch
      1.75 ± 11%      +0.7        2.45 ±  6%  perf-profile.calltrace.cycles-pp.kmem_cache_free.rcu_do_batch.rcu_core.__softirqentry_text_start.asm_call_sysvec_on_stack
      3.95 ± 11%      +0.7        4.66 ±  2%  perf-profile.calltrace.cycles-pp.drain_obj_stock.refill_obj_stock.kmem_cache_free.rcu_do_batch.rcu_core
      1.76 ± 11%      +0.7        2.48 ±  6%  perf-profile.calltrace.cycles-pp.rcu_do_batch.rcu_core.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack
      1.79 ± 11%      +0.7        2.51 ±  6%  perf-profile.calltrace.cycles-pp.rcu_core.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu
      2.06 ± 10%      +0.8        2.81 ±  5%  perf-profile.calltrace.cycles-pp.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt
      2.06 ± 10%      +0.8        2.81 ±  5%  perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      2.06 ± 10%      +0.8        2.81 ±  5%  perf-profile.calltrace.cycles-pp.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      2.09 ± 10%      +0.8        2.86 ±  5%  perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      4.37 ± 11%      +0.8        5.16        perf-profile.calltrace.cycles-pp.refill_obj_stock.kmem_cache_free.rcu_do_batch.rcu_core.__softirqentry_text_start
     11.54 ±  5%      +0.8       12.35        perf-profile.calltrace.cycles-pp.inet_release.__sock_release.sock_close.__fput.task_work_run
     11.68 ±  5%      +0.8       12.49        perf-profile.calltrace.cycles-pp.__sock_release.sock_close.__fput.task_work_run.exit_to_user_mode_prepare
     11.69 ±  5%      +0.8       12.52        perf-profile.calltrace.cycles-pp.sock_close.__fput.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode
      3.77 ±  7%      +0.9        4.67 ±  4%  perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
     13.85 ±  5%      +1.1       14.90        perf-profile.calltrace.cycles-pp.__fput.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
     47.25 ±  6%      -2.6       44.70        perf-profile.children.cycles-pp.do_idle
     46.95 ±  6%      -2.5       44.40        perf-profile.children.cycles-pp.start_secondary
     47.26 ±  6%      -2.5       44.72        perf-profile.children.cycles-pp.secondary_startup_64_no_verify
     47.26 ±  6%      -2.5       44.72        perf-profile.children.cycles-pp.cpu_startup_entry
      2.56 ±  6%      -0.8        1.74 ±  6%  perf-profile.children.cycles-pp.menu_select
      0.93 ±  5%      -0.6        0.30 ±  3%  perf-profile.children.cycles-pp.ipv4_dst_check
      0.81 ±  5%      -0.5        0.27 ±  3%  perf-profile.children.cycles-pp.__sk_dst_check
      0.96 ±  4%      -0.3        0.62        perf-profile.children.cycles-pp.ip_route_output_key_hash_rcu
      1.00 ±  4%      -0.3        0.68        perf-profile.children.cycles-pp.ip_route_output_key_hash
      0.69 ±  3%      -0.2        0.48 ±  2%  perf-profile.children.cycles-pp.ip_route_output_flow
      1.58 ±  4%      -0.2        1.42        perf-profile.children.cycles-pp.tcp_conn_request
      0.29 ±  4%      -0.2        0.13 ±  3%  perf-profile.children.cycles-pp.inet_sk_rebuild_header
      0.31 ±  7%      -0.1        0.20 ±  2%  perf-profile.children.cycles-pp.ip_rcv_finish_core
      0.29 ±  5%      -0.1        0.17 ±  7%  perf-profile.children.cycles-pp.tcp_ack_update_rtt
      0.33 ±  6%      -0.1        0.22 ±  3%  perf-profile.children.cycles-pp.ip_rcv_finish
      0.21 ±  6%      -0.1        0.10 ±  4%  perf-profile.children.cycles-pp.inode_init_always
      0.31 ±  4%      -0.1        0.21 ±  2%  perf-profile.children.cycles-pp.inet_csk_route_req
      0.18 ±  2%      -0.1        0.08 ±  5%  perf-profile.children.cycles-pp.tcp_select_initial_window
      0.17 ±  8%      -0.1        0.08 ±  5%  perf-profile.children.cycles-pp.__tcp_select_window
      0.28 ±  2%      -0.1        0.19 ±  2%  perf-profile.children.cycles-pp.inet_csk_route_child_sock
      0.12 ±  5%      -0.1        0.04 ± 57%  perf-profile.children.cycles-pp.tcp_assign_congestion_control
      0.27 ±  7%      -0.1        0.19 ±  4%  perf-profile.children.cycles-pp.tcp_connect_init
      0.69 ±  2%      -0.1        0.62        perf-profile.children.cycles-pp.tcp_init_transfer
      0.20 ±  7%      -0.1        0.14 ±  3%  perf-profile.children.cycles-pp.tcp_schedule_loss_probe
      0.18 ±  9%      -0.1        0.11 ±  4%  perf-profile.children.cycles-pp.tcp_openreq_init_rwin
      0.26 ±  6%      -0.0        0.21 ±  6%  perf-profile.children.cycles-pp.ip_rcv_core
      0.07 ± 10%      -0.0        0.03 ±100%  perf-profile.children.cycles-pp.tcp_stream_memory_free
      0.16 ±  5%      -0.0        0.11 ±  3%  perf-profile.children.cycles-pp.tcp_update_pacing_rate
      0.12 ±  9%      -0.0        0.08 ±  6%  perf-profile.children.cycles-pp.skb_entail
      0.11 ±  3%      -0.0        0.07 ± 10%  perf-profile.children.cycles-pp.inet_twsk_alloc
      0.27 ±  4%      -0.0        0.23 ±  5%  perf-profile.children.cycles-pp.tick_irq_enter
      0.16 ±  8%      -0.0        0.12 ±  4%  perf-profile.children.cycles-pp.tcp_parse_options
      0.09 ±  7%      -0.0        0.06        perf-profile.children.cycles-pp.tcp_synack_rtt_meas
      0.11 ± 11%      -0.0        0.08 ±  5%  perf-profile.children.cycles-pp.tcp_sync_mss
      0.09 ±  4%      -0.0        0.07 ± 13%  perf-profile.children.cycles-pp.secure_tcp_ts_off
      0.07 ±  7%      -0.0        0.05 ±  8%  perf-profile.children.cycles-pp.tcp_mtup_init
      0.08            +0.0        0.09 ±  4%  perf-profile.children.cycles-pp.apparmor_sk_alloc_security
      0.07 ±  5%      +0.0        0.09 ±  5%  perf-profile.children.cycles-pp.memcpy_erms
      0.05            +0.0        0.06 ±  6%  perf-profile.children.cycles-pp.security_sock_rcv_skb
      0.06            +0.0        0.07 ±  5%  perf-profile.children.cycles-pp.up_write
      0.10 ±  4%      +0.0        0.11 ±  3%  perf-profile.children.cycles-pp.__destroy_inode
      0.08 ±  5%      +0.0        0.09 ±  4%  perf-profile.children.cycles-pp.tcp_send_mss
      0.07 ± 10%      +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.memset_erms
      0.11            +0.0        0.12 ±  4%  perf-profile.children.cycles-pp.sk_stream_kill_queues
      0.14 ±  6%      +0.0        0.15 ±  2%  perf-profile.children.cycles-pp.call_rcu
      0.07 ±  6%      +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.tcp_queue_rcv
      0.11 ±  3%      +0.0        0.13        perf-profile.children.cycles-pp._cond_resched
      0.11 ±  4%      +0.0        0.12 ±  4%  perf-profile.children.cycles-pp.load_balance
      0.08 ±  5%      +0.0        0.10 ±  7%  perf-profile.children.cycles-pp.tcp_add_backlog
      0.09 ±  7%      +0.0        0.11 ±  4%  perf-profile.children.cycles-pp.find_busiest_group
      0.09 ±  4%      +0.0        0.10 ±  4%  perf-profile.children.cycles-pp.update_sd_lb_stats
      0.06            +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.inet_csk_complete_hashdance
      0.08 ± 10%      +0.0        0.10 ±  4%  perf-profile.children.cycles-pp.newidle_balance
      0.20 ±  5%      +0.0        0.21 ±  2%  perf-profile.children.cycles-pp.__kmalloc_reserve
      0.18 ±  6%      +0.0        0.20 ±  2%  perf-profile.children.cycles-pp.__kmalloc_node_track_caller
      0.14 ±  6%      +0.0        0.15 ±  3%  perf-profile.children.cycles-pp.kmem_cache_alloc_node
      0.10            +0.0        0.12 ±  5%  perf-profile.children.cycles-pp.tcp_rcv_space_adjust
      0.07 ±  5%      +0.0        0.09 ±  4%  perf-profile.children.cycles-pp.sock_put
      0.06 ±  6%      +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.inet_getname
      0.17 ±  4%      +0.0        0.19 ±  2%  perf-profile.children.cycles-pp.kfree
      0.21 ±  5%      +0.0        0.24        perf-profile.children.cycles-pp.sk_stop_timer
      0.17 ±  6%      +0.0        0.19 ±  3%  perf-profile.children.cycles-pp.__might_sleep
      0.10 ±  4%      +0.0        0.12 ±  8%  perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
      0.25            +0.0        0.27 ±  2%  perf-profile.children.cycles-pp.security_sk_clone
      0.24 ±  2%      +0.0        0.27        perf-profile.children.cycles-pp.apparmor_sk_clone_security
      0.18 ±  2%      +0.0        0.21 ±  2%  perf-profile.children.cycles-pp.inet_ehash_insert
      0.12 ±  9%      +0.0        0.14 ±  5%  perf-profile.children.cycles-pp.ip_finish_output
      0.15 ±  7%      +0.0        0.18 ±  2%  perf-profile.children.cycles-pp.del_timer
      0.20 ±  6%      +0.0        0.23 ±  2%  perf-profile.children.cycles-pp.enqueue_to_backlog
      0.19 ±  5%      +0.0        0.21 ±  2%  perf-profile.children.cycles-pp.inet_csk_clear_xmit_timers
      0.04 ± 57%      +0.0        0.06 ±  6%  perf-profile.children.cycles-pp.__slab_alloc
      0.09 ±  9%      +0.0        0.11 ±  7%  perf-profile.children.cycles-pp.__ip_dev_find
      0.17 ±  3%      +0.0        0.19 ±  2%  perf-profile.children.cycles-pp.__cgroup_bpf_run_filter_skb
      0.06 ±  9%      +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.arch_cpu_idle_enter
      0.10 ±  4%      +0.0        0.13 ±  3%  perf-profile.children.cycles-pp.ip_send_check
      0.31 ±  3%      +0.0        0.34 ±  3%  perf-profile.children.cycles-pp.sched_clock
      0.28 ±  4%      +0.0        0.31 ±  3%  perf-profile.children.cycles-pp.___might_sleep
      0.20 ±  3%      +0.0        0.23 ±  3%  perf-profile.children.cycles-pp.irqtime_account_irq
      0.23 ±  5%      +0.0        0.26 ±  4%  perf-profile.children.cycles-pp.inet_csk_reqsk_queue_hash_add
      0.34 ±  3%      +0.0        0.37        perf-profile.children.cycles-pp.sched_clock_cpu
      0.22 ±  7%      +0.0        0.25 ±  4%  perf-profile.children.cycles-pp.tcp_write_queue_purge
      0.19 ±  4%      +0.0        0.22 ±  4%  perf-profile.children.cycles-pp.sk_filter_trim_cap
      0.15 ±  5%      +0.0        0.19 ±  2%  perf-profile.children.cycles-pp.ip_local_out
      0.04 ± 58%      +0.0        0.08 ± 10%  perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
      0.30 ±  6%      +0.0        0.33 ±  3%  perf-profile.children.cycles-pp.skb_release_data
      0.03 ±100%      +0.0        0.06 ±  6%  perf-profile.children.cycles-pp.__sk_free
      0.14 ±  3%      +0.0        0.17 ±  2%  perf-profile.children.cycles-pp.__ip_local_out
      0.06 ±  7%      +0.0        0.10 ± 11%  perf-profile.children.cycles-pp.nf_hook_slow
      0.27 ±  4%      +0.0        0.31 ±  2%  perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
      0.18 ±  6%      +0.0        0.22        perf-profile.children.cycles-pp.__skb_clone
      0.26 ±  4%      +0.0        0.30 ±  2%  perf-profile.children.cycles-pp.tcp_mstamp_refresh
      0.01 ±173%      +0.0        0.05 ±  9%  perf-profile.children.cycles-pp.__cgroup_bpf_run_filter_sock_ops
      0.06 ± 11%      +0.0        0.10 ±  8%  perf-profile.children.cycles-pp.calc_global_load_tick
      0.01 ±173%      +0.0        0.06 ± 14%  perf-profile.children.cycles-pp.ipv4_default_advmss
      0.42 ±  7%      +0.0        0.47        perf-profile.children.cycles-pp.__mod_memcg_state
      0.38 ±  6%      +0.0        0.43 ±  4%  perf-profile.children.cycles-pp.apparmor_file_free_security
      0.18 ± 10%      +0.0        0.23 ±  6%  perf-profile.children.cycles-pp.inet_put_port
      0.39 ±  6%      +0.0        0.44 ±  4%  perf-profile.children.cycles-pp.security_file_free
      0.00            +0.1        0.05        perf-profile.children.cycles-pp.inet_ehash_nolisten
      0.47 ±  6%      +0.1        0.52 ±  2%  perf-profile.children.cycles-pp.read_tsc
      0.00            +0.1        0.05 ±  8%  perf-profile.children.cycles-pp.inet_csk_reqsk_queue_drop
      0.00            +0.1        0.05 ±  8%  perf-profile.children.cycles-pp.del_timer_sync
      0.70 ±  6%      +0.1        0.76        perf-profile.children.cycles-pp.loopback_xmit
      0.40 ±  6%      +0.1        0.46 ±  2%  perf-profile.children.cycles-pp.__x64_sys_getsockopt
      0.00            +0.1        0.06 ±  9%  perf-profile.children.cycles-pp.___slab_alloc
      0.23 ±  7%      +0.1        0.29 ±  5%  perf-profile.children.cycles-pp.__inet_check_established
      0.31 ±  6%      +0.1        0.37 ±  2%  perf-profile.children.cycles-pp.lock_timer_base
      0.67 ±  5%      +0.1        0.72        perf-profile.children.cycles-pp.__alloc_skb
      0.40 ±  6%      +0.1        0.46 ±  3%  perf-profile.children.cycles-pp.__sys_getsockopt
      0.78 ±  6%      +0.1        0.84        perf-profile.children.cycles-pp.dev_hard_start_xmit
      0.58 ±  4%      +0.1        0.64 ±  2%  perf-profile.children.cycles-pp.tcp_done
      0.46 ±  7%      +0.1        0.52        perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
      0.38 ±  7%      +0.1        0.43        perf-profile.children.cycles-pp.dst_release
      0.36 ±  6%      +0.1        0.42 ±  4%  perf-profile.children.cycles-pp.inet_csk_bind_conflict
      0.38 ±  7%      +0.1        0.44 ±  2%  perf-profile.children.cycles-pp.tcp_v4_destroy_sock
      0.60 ±  5%      +0.1        0.66 ±  2%  perf-profile.children.cycles-pp.security_file_alloc
      0.49 ±  4%      +0.1        0.55 ±  2%  perf-profile.children.cycles-pp.tcp_send_fin
      0.20 ±  7%      +0.1        0.26 ±  4%  perf-profile.children.cycles-pp.scheduler_tick
      0.54 ±  5%      +0.1        0.60        perf-profile.children.cycles-pp.sk_reset_timer
      0.01 ±173%      +0.1        0.08 ± 12%  perf-profile.children.cycles-pp.inet_twsk_hashdance
      0.76 ±  6%      +0.1        0.83        perf-profile.children.cycles-pp.__kfree_skb
      0.52 ±  6%      +0.1        0.59 ±  3%  perf-profile.children.cycles-pp.inet_csk_get_port
      0.89 ±  5%      +0.1        0.96        perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      0.47 ±  6%      +0.1        0.54        perf-profile.children.cycles-pp.mod_timer
      0.72            +0.1        0.79        perf-profile.children.cycles-pp.tcp_create_openreq_child
      0.42 ±  7%      +0.1        0.50        perf-profile.children.cycles-pp.sk_stream_alloc_skb
      0.60 ±  2%      +0.1        0.68        perf-profile.children.cycles-pp.sk_clone_lock
      0.26 ±  4%      +0.1        0.34        perf-profile.children.cycles-pp.tcp_init_metrics
      0.57 ±  7%      +0.1        0.66        perf-profile.children.cycles-pp.__sk_mem_reduce_allocated
      0.61 ±  2%      +0.1        0.69        perf-profile.children.cycles-pp.inet_csk_clone_lock
      2.25 ±  3%      +0.1        2.34        perf-profile.children.cycles-pp.sk_wait_data
      1.25 ±  5%      +0.1        1.34 ±  2%  perf-profile.children.cycles-pp.inet_create
      0.29 ±  8%      +0.1        0.38 ±  2%  perf-profile.children.cycles-pp.inet_ehashfn
      0.74 ±  4%      +0.1        0.83 ±  2%  perf-profile.children.cycles-pp.sock_def_wakeup
      1.26 ±  5%      +0.1        1.35        perf-profile.children.cycles-pp.__dev_queue_xmit
      0.79 ±  5%      +0.1        0.89 ±  4%  perf-profile.children.cycles-pp.sk_prot_alloc
      0.89 ±  8%      +0.1        0.99        perf-profile.children.cycles-pp.page_counter_try_charge
      0.38 ±  5%      +0.1        0.48 ±  5%  perf-profile.children.cycles-pp.update_process_times
      1.74 ±  5%      +0.1        1.85        perf-profile.children.cycles-pp.schedule
      0.44 ± 10%      +0.1        0.54 ± 10%  perf-profile.children.cycles-pp.tick_nohz_next_event
      0.92 ±  9%      +0.1        1.03        perf-profile.children.cycles-pp.__memcg_kmem_charge
      0.39 ±  5%      +0.1        0.49 ±  5%  perf-profile.children.cycles-pp.tick_sched_handle
      1.86 ±  5%      +0.1        1.97        perf-profile.children.cycles-pp.schedule_timeout
      0.88 ±  6%      +0.1        0.99 ±  4%  perf-profile.children.cycles-pp.__d_alloc
      0.50 ±  9%      +0.1        0.61 ±  9%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      0.87 ±  7%      +0.1        0.98        perf-profile.children.cycles-pp.inet_csk_accept
      0.88 ±  6%      +0.1        0.99 ±  4%  perf-profile.children.cycles-pp.d_alloc_pseudo
      0.91 ±  5%      +0.1        1.03 ±  3%  perf-profile.children.cycles-pp.sk_alloc
      0.75 ±  6%      +0.1        0.87 ±  2%  perf-profile.children.cycles-pp.inet_csk_destroy_sock
      0.53 ±  5%      +0.1        0.66 ±  5%  perf-profile.children.cycles-pp.tick_sched_timer
      1.59 ±  7%      +0.1        1.72 ±  2%  perf-profile.children.cycles-pp.sock_alloc
      0.46 ±  5%      +0.1        0.59        perf-profile.children.cycles-pp.sk_forced_mem_schedule
      1.07 ±  6%      +0.1        1.21        perf-profile.children.cycles-pp.inet_accept
      0.32 ±  7%      +0.1        0.46        perf-profile.children.cycles-pp.__tcp_get_metrics
      1.20 ±  8%      +0.1        1.35        perf-profile.children.cycles-pp.obj_cgroup_charge
      1.32 ±  8%      +0.2        1.47 ±  3%  perf-profile.children.cycles-pp.alloc_empty_file
      0.70 ±  4%      +0.2        0.85 ±  5%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      1.31 ±  8%      +0.2        1.46 ±  3%  perf-profile.children.cycles-pp.__alloc_file
      3.04 ±  3%      +0.2        3.20        perf-profile.children.cycles-pp.tcp_data_queue
      0.50 ±  5%      +0.2        0.66 ±  3%  perf-profile.children.cycles-pp.percpu_counter_add_batch
      1.38 ±  5%      +0.2        1.54 ±  3%  perf-profile.children.cycles-pp.hrtimer_interrupt
      1.02 ±  4%      +0.2        1.19        perf-profile.children.cycles-pp.__inet_lookup_established
      1.54 ±  6%      +0.2        1.70 ±  3%  perf-profile.children.cycles-pp.__dentry_kill
      2.59 ±  5%      +0.2        2.76 ±  2%  perf-profile.children.cycles-pp.__sock_create
      1.31 ± 10%      +0.2        1.48 ±  4%  perf-profile.children.cycles-pp.propagate_protected_usage
      2.21 ±  3%      +0.2        2.39        perf-profile.children.cycles-pp.tcp_fin
      0.40 ±  5%      +0.2        0.59        perf-profile.children.cycles-pp.tcp_get_metrics
      1.53 ±  4%      +0.2        1.73 ±  2%  perf-profile.children.cycles-pp._raw_spin_lock
      1.31 ±  7%      +0.2        1.51 ±  2%  perf-profile.children.cycles-pp.sock_alloc_inode
      0.81 ±  5%      +0.2        1.04        perf-profile.children.cycles-pp.lock_sock_nested
      1.08 ±  5%      +0.2        1.31        perf-profile.children.cycles-pp._raw_spin_lock_bh
      0.69 ±  5%      +0.2        0.92 ±  2%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      2.29 ±  7%      +0.3        2.57 ±  3%  perf-profile.children.cycles-pp.alloc_file_pseudo
      2.30 ±  7%      +0.3        2.58 ±  3%  perf-profile.children.cycles-pp.sock_alloc_file
      2.96 ±  6%      +0.3        3.25 ±  2%  perf-profile.children.cycles-pp.__sk_destruct
      3.92 ±  6%      +0.3        4.24 ±  3%  perf-profile.children.cycles-pp.__sys_socket
      3.92 ±  6%      +0.3        4.25 ±  3%  perf-profile.children.cycles-pp.__x64_sys_socket
      4.04 ±  4%      +0.4        4.40        perf-profile.children.cycles-pp.inet_recvmsg
      4.02 ±  4%      +0.4        4.39        perf-profile.children.cycles-pp.tcp_recvmsg
      3.10 ±  7%      +0.4        3.49        perf-profile.children.cycles-pp.__x64_sys_accept
      3.09 ±  7%      +0.4        3.48        perf-profile.children.cycles-pp.__sys_accept4
      3.05 ±  7%      +0.4        3.44        perf-profile.children.cycles-pp.__sys_accept4_file
      4.46 ±  4%      +0.4        4.85        perf-profile.children.cycles-pp.__x64_sys_recvfrom
      4.44 ±  4%      +0.4        4.84        perf-profile.children.cycles-pp.__sys_recvfrom
      4.61 ±  8%      +0.4        5.05 ±  2%  perf-profile.children.cycles-pp.smpboot_thread_fn
      3.50 ±  7%      +0.5        3.97 ±  4%  perf-profile.children.cycles-pp.kmem_cache_alloc
      4.15 ±  9%      +0.5        4.66 ±  4%  perf-profile.children.cycles-pp.__memcg_kmem_uncharge
      8.88 ±  4%      +0.6        9.43        perf-profile.children.cycles-pp.tcp_close
      5.60 ±  5%      +0.7        6.29 ±  3%  perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
      5.89 ±  9%      +0.7        6.62 ±  2%  perf-profile.children.cycles-pp.page_counter_cancel
      6.19 ±  9%      +0.8        6.96 ±  2%  perf-profile.children.cycles-pp.page_counter_uncharge
      3.17 ±  7%      +0.8        3.96 ±  4%  perf-profile.children.cycles-pp.irq_exit_rcu
     11.54 ±  5%      +0.8       12.35        perf-profile.children.cycles-pp.inet_release
     11.68 ±  5%      +0.8       12.50        perf-profile.children.cycles-pp.__sock_release
     11.69 ±  5%      +0.8       12.52        perf-profile.children.cycles-pp.sock_close
      7.04 ±  9%      +0.9        7.94        perf-profile.children.cycles-pp.drain_obj_stock
      5.10 ±  6%      +0.9        6.04 ±  4%  perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
      7.91 ±  8%      +1.0        8.94        perf-profile.children.cycles-pp.refill_obj_stock
     13.86 ±  5%      +1.1       14.91        perf-profile.children.cycles-pp.__fput
     19.22 ±  4%      +1.1       20.29        perf-profile.children.cycles-pp.do_softirq_own_stack
     13.99 ±  5%      +1.1       15.07        perf-profile.children.cycles-pp.task_work_run
     14.16 ±  5%      +1.1       15.25        perf-profile.children.cycles-pp.exit_to_user_mode_prepare
     14.23 ±  5%      +1.1       15.33        perf-profile.children.cycles-pp.syscall_exit_to_user_mode
      7.97 ±  8%      +1.2        9.17        perf-profile.children.cycles-pp.rcu_do_batch
      8.02 ±  7%      +1.2        9.22        perf-profile.children.cycles-pp.rcu_core
     20.50 ±  4%      +1.2       21.73        perf-profile.children.cycles-pp.asm_call_sysvec_on_stack
     11.29 ±  7%      +1.5       12.78        perf-profile.children.cycles-pp.kmem_cache_free
     23.69 ±  5%      +1.5       25.20        perf-profile.children.cycles-pp.__softirqentry_text_start
     47.12 ±  5%      +1.9       48.99        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      1.95 ±  9%      -0.9        1.00 ±  7%  perf-profile.self.cycles-pp.menu_select
      1.40 ± 14%      -0.8        0.58 ±  6%  perf-profile.self.cycles-pp.cpuidle_enter_state
      0.91 ±  4%      -0.6        0.29 ±  3%  perf-profile.self.cycles-pp.ipv4_dst_check
      0.70 ±  5%      -0.4        0.34 ±  4%  perf-profile.self.cycles-pp.ip_route_output_key_hash_rcu
      0.23 ±  4%      -0.1        0.11 ±  7%  perf-profile.self.cycles-pp.tcp_ack_update_rtt
      0.16 ±  9%      -0.1        0.05 ±  8%  perf-profile.self.cycles-pp.inode_init_always
      0.31 ±  5%      -0.1        0.20 ±  2%  perf-profile.self.cycles-pp.ip_rcv_finish_core
      0.18 ±  5%      -0.1        0.08 ±  5%  perf-profile.self.cycles-pp.tcp_select_initial_window
      0.17 ±  7%      -0.1        0.08 ±  5%  perf-profile.self.cycles-pp.__tcp_select_window
      0.11 ±  3%      -0.1        0.03 ±100%  perf-profile.self.cycles-pp.tcp_assign_congestion_control
      0.12 ±  6%      -0.1        0.05 ±  8%  perf-profile.self.cycles-pp.__inet_bind
      0.16 ±  6%      -0.1        0.09 ±  4%  perf-profile.self.cycles-pp.tcp_schedule_loss_probe
      0.26 ±  6%      -0.0        0.21 ±  5%  perf-profile.self.cycles-pp.ip_rcv_core
      0.23 ±  4%      -0.0        0.18 ±  2%  perf-profile.self.cycles-pp.ip_protocol_deliver_rcu
      0.10 ± 12%      -0.0        0.05 ±  9%  perf-profile.self.cycles-pp.__inet_hash_connect
      0.10 ±  8%      -0.0        0.06 ±  9%  perf-profile.self.cycles-pp.tcp_connect_init
      0.11            -0.0        0.07 ±  7%  perf-profile.self.cycles-pp.inet_twsk_alloc
      0.10 ±  4%      -0.0        0.05 ±  8%  perf-profile.self.cycles-pp.tcp_v4_syn_recv_sock
      0.10 ± 16%      -0.0        0.05 ±  8%  perf-profile.self.cycles-pp.tcp_update_metrics
      0.15 ±  4%      -0.0        0.11 ±  6%  perf-profile.self.cycles-pp.tcp_update_pacing_rate
      0.11 ± 10%      -0.0        0.07 ±  6%  perf-profile.self.cycles-pp.skb_entail
      0.15 ±  7%      -0.0        0.12 ±  3%  perf-profile.self.cycles-pp.tcp_parse_options
      0.10 ± 15%      -0.0        0.08 ±  5%  perf-profile.self.cycles-pp.tcp_sync_mss
      0.07 ±  7%      -0.0        0.05        perf-profile.self.cycles-pp.tcp_mtup_init
      0.07            -0.0        0.06 ±  7%  perf-profile.self.cycles-pp.tcp_create_openreq_child
      0.09 ±  4%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.sk_clone_lock
      0.07 ±  5%      +0.0        0.09 ±  5%  perf-profile.self.cycles-pp.memcpy_erms
      0.06            +0.0        0.07 ±  5%  perf-profile.self.cycles-pp.sk_filter_trim_cap
      0.18 ±  3%      +0.0        0.20 ±  2%  perf-profile.self.cycles-pp.tcp_sendmsg_locked
      0.13 ±  5%      +0.0        0.15 ±  3%  perf-profile.self.cycles-pp.tcp_rcv_state_process
      0.10 ±  4%      +0.0        0.11 ±  4%  perf-profile.self.cycles-pp.__kmalloc_node_track_caller
      0.09 ±  4%      +0.0        0.11 ±  6%  perf-profile.self.cycles-pp.kmem_cache_alloc_node
      0.10 ±  8%      +0.0        0.12 ±  3%  perf-profile.self.cycles-pp.mod_timer
      0.11 ±  4%      +0.0        0.13 ±  3%  perf-profile.self.cycles-pp.net_rx_action
      0.16 ±  5%      +0.0        0.18 ±  2%  perf-profile.self.cycles-pp.kfree
      0.14 ±  5%      +0.0        0.16 ±  4%  perf-profile.self.cycles-pp.select_task_rq_fair
      0.10 ±  4%      +0.0        0.12 ±  3%  perf-profile.self.cycles-pp.__ip_finish_output
      0.08 ± 10%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.inet_lookup_ifaddr_rcu
      0.07 ± 10%      +0.0        0.09 ±  4%  perf-profile.self.cycles-pp.sock_put
      0.06            +0.0        0.08 ±  5%  perf-profile.self.cycles-pp.tcp_queue_rcv
      0.24 ±  2%      +0.0        0.27        perf-profile.self.cycles-pp.apparmor_sk_clone_security
      0.15 ±  3%      +0.0        0.18 ±  4%  perf-profile.self.cycles-pp.__cgroup_bpf_run_filter_skb
      0.10 ±  4%      +0.0        0.13        perf-profile.self.cycles-pp.ip_send_check
      0.28 ±  5%      +0.0        0.31 ±  2%  perf-profile.self.cycles-pp.native_sched_clock
      0.06 ±  9%      +0.0        0.08 ±  5%  perf-profile.self.cycles-pp.ip_local_deliver
      0.07 ±  5%      +0.0        0.10 ±  7%  perf-profile.self.cycles-pp.update_process_times
      0.26 ±  5%      +0.0        0.29 ±  3%  perf-profile.self.cycles-pp.___might_sleep
      0.15 ±  9%      +0.0        0.18 ±  6%  perf-profile.self.cycles-pp.obj_cgroup_charge
      0.23 ±  6%      +0.0        0.26        perf-profile.self.cycles-pp.ip_finish_output2
      0.04 ± 58%      +0.0        0.07 ±  5%  perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
      0.23 ±  5%      +0.0        0.27        perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
      0.14 ±  5%      +0.0        0.18        perf-profile.self.cycles-pp.__skb_clone
      0.29 ±  6%      +0.0        0.33 ±  2%  perf-profile.self.cycles-pp.skb_release_data
      0.08 ±  8%      +0.0        0.12        perf-profile.self.cycles-pp.tcp_get_metrics
      0.41 ±  6%      +0.0        0.46        perf-profile.self.cycles-pp.tcp_recvmsg
      0.06 ± 11%      +0.0        0.10 ±  8%  perf-profile.self.cycles-pp.calc_global_load_tick
      0.30 ± 10%      +0.0        0.34 ±  2%  perf-profile.self.cycles-pp.page_counter_uncharge
      0.12 ±  8%      +0.0        0.17 ± 14%  perf-profile.self.cycles-pp.tick_nohz_next_event
      0.42 ±  7%      +0.0        0.47        perf-profile.self.cycles-pp.__mod_memcg_state
      0.20 ±  5%      +0.0        0.24 ±  4%  perf-profile.self.cycles-pp.tcp_clean_rtx_queue
      0.45 ±  4%      +0.0        0.49        perf-profile.self.cycles-pp.__sk_destruct
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.__cgroup_bpf_run_filter_sock_ops
      0.36 ±  7%      +0.1        0.42        perf-profile.self.cycles-pp.dst_release
      0.36 ±  7%      +0.1        0.41 ±  3%  perf-profile.self.cycles-pp.apparmor_file_free_security
      0.76 ±  5%      +0.1        0.81        perf-profile.self.cycles-pp._raw_spin_lock_bh
      0.36 ±  7%      +0.1        0.42 ±  4%  perf-profile.self.cycles-pp.inet_csk_bind_conflict
      0.88 ±  5%      +0.1        0.95        perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.55 ±  7%      +0.1        0.63        perf-profile.self.cycles-pp.__sk_mem_reduce_allocated
      0.29 ±  8%      +0.1        0.37        perf-profile.self.cycles-pp.inet_ehashfn
      0.73 ±  9%      +0.1        0.82        perf-profile.self.cycles-pp.page_counter_try_charge
      0.79 ±  6%      +0.1        0.88        perf-profile.self.cycles-pp.__inet_lookup_established
      0.82 ±  6%      +0.1        0.94 ±  5%  perf-profile.self.cycles-pp.refill_obj_stock
      0.44 ±  4%      +0.1        0.56        perf-profile.self.cycles-pp.sk_forced_mem_schedule
      0.84 ±  7%      +0.1        0.97 ±  6%  perf-profile.self.cycles-pp.drain_obj_stock
      0.31 ±  6%      +0.1        0.45        perf-profile.self.cycles-pp.__tcp_get_metrics
      1.06 ±  7%      +0.1        1.20 ±  5%  perf-profile.self.cycles-pp.kmem_cache_alloc
      1.13 ±  4%      +0.1        1.27        perf-profile.self.cycles-pp._raw_spin_lock
      0.49 ±  6%      +0.2        0.65 ±  3%  perf-profile.self.cycles-pp.percpu_counter_add_batch
      1.30 ± 10%      +0.2        1.48 ±  4%  perf-profile.self.cycles-pp.propagate_protected_usage
      0.68 ±  4%      +0.2        0.91 ±  2%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
      3.08 ±  5%      +0.4        3.51 ±  4%  perf-profile.self.cycles-pp.kmem_cache_free
      4.72 ±  9%      +0.6        5.29        perf-profile.self.cycles-pp.page_counter_cancel





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Oliver Sang


View attachment "config-5.10.0-rc2-00010-g9522750c66c6" of type "text/plain" (171269 bytes)

View attachment "job-script" of type "text/plain" (8309 bytes)

View attachment "job.yaml" of type "text/plain" (5834 bytes)

View attachment "reproduce" of type "text/plain" (946 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ