lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Date:   Wed, 17 Mar 2021 23:05:24 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     Aubrey Li <aubrey.li@...el.com>
Cc:     0day robot <lkp@...el.com>, Peter Zijlstra <peterz@...radead.org>,
        Mel Gorman <mgorman@...e.de>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        Qais Yousef <qais.yousef@....com>,
        Valentin Schneider <valentin.schneider@....com>,
        Jiang Biao <benbjiang@...il.com>,
        Tim Chen <tim.c.chen@...ux.intel.com>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org,
        ying.huang@...el.com, feng.tang@...el.com, zhengjun.xing@...el.com,
        mingo@...hat.com, mgorman@...hsingularity.net,
        juri.lelli@...hat.com, dietmar.eggemann@....com,
        rostedt@...dmis.org, bsegall@...gle.com,
        Aubrey Li <aubrey.li@...ux.intel.com>, yu.c.chen@...el.com
Subject: [sched/fair]  d619f7afd7:  stress-ng.sock.ops_per_sec 69.4%
 improvement



Greeting,

FYI, we noticed a 69.4% improvement of stress-ng.sock.ops_per_sec due to commit:


commit: d619f7afd7731b1c3d56367e4583fd421ab1779e ("[PATCH v10] sched/fair: select idle cpu from idle cpumask for task wakeup")
url: https://github.com/0day-ci/linux/commits/Aubrey-Li/sched-fair-select-idle-cpu-from-idle-cpumask-for-task-wakeup/20210316-132415
base: https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git 13c2235b2b2870675195f0b551275d1abdd81068

in testcase: stress-ng
on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 192G memory
with following parameters:

	nr_threads: 100%
	disk: 1HDD
	testtime: 60s
	class: network
	test: sock
	cpufreq_governor: performance
	ucode: 0x5003006






Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install                job.yaml  # job file is attached in this email
        bin/lkp split-job --compatible job.yaml
        bin/lkp run                    compatible-job.yaml

=========================================================================================
class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode:
  network/gcc-9/performance/1HDD/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp5/sock/stress-ng/60s/0x5003006

commit: 
  13c2235b2b ("sched: Remove unnecessary variable from schedule_tail()")
  d619f7afd7 ("sched/fair: select idle cpu from idle cpumask for task wakeup")

13c2235b2b287067 d619f7afd7731b1c3d56367e458 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    950326           +69.4%    1609719 ±  2%  stress-ng.sock.ops
     15838           +69.4%      26828 ±  2%  stress-ng.sock.ops_per_sec
 4.457e+08           -59.0%  1.828e+08 ±  9%  stress-ng.time.involuntary_context_switches
     18729           +16.1%      21750        stress-ng.time.minor_page_faults
      6423           +18.3%       7597        stress-ng.time.percent_of_cpu_this_job_got
      3812           +19.1%       4539        stress-ng.time.system_time
 4.461e+08           -56.1%   1.96e+08 ±  8%  stress-ng.time.voluntary_context_switches
      7.44            +7.6%       8.00        iostat.cpu.idle
      0.93            +0.2        1.10 ±  2%  mpstat.cpu.all.irq%
     22.86           -10.3       12.51 ±  4%  mpstat.cpu.all.soft%
     51.53           +10.7       62.24        mpstat.cpu.all.sys%
     89.00            -1.1%      88.00        vmstat.cpu.sy
  13737126           -56.8%    5929682 ±  8%  vmstat.system.cs
    210226            +7.7%     226386        vmstat.system.in
   2382129 ± 25%    +940.1%   24775963 ±  6%  cpuidle.C1.time
    183698 ± 73%   +2123.6%    4084720 ±  7%  cpuidle.C1.usage
    421297 ± 81%   +1721.9%    7675416 ±  6%  cpuidle.POLL.time
    120897 ± 82%   +1582.7%    2034334 ±  5%  cpuidle.POLL.usage
   1141585 ± 37%   +3269.7%   38467867 ±  5%  numa-numastat.node0.local_node
   1201302 ± 37%   +3105.8%   38511127 ±  5%  numa-numastat.node0.numa_hit
    907692 ± 25%   +4255.5%   39534228 ± 13%  numa-numastat.node1.local_node
    934562 ± 22%   +4134.9%   39577574 ± 13%  numa-numastat.node1.numa_hit
   1387215 ± 12%   +1308.4%   19537698 ±  4%  numa-vmstat.node0.numa_hit
   1288196 ± 12%   +1413.2%   19493602 ±  4%  numa-vmstat.node0.numa_local
   1154758 ± 16%   +1532.5%   18851309 ± 16%  numa-vmstat.node1.numa_hit
   1010907 ± 22%   +1745.1%   18652251 ± 16%  numa-vmstat.node1.numa_local
     72843            +1.4%      73890        proc-vmstat.nr_inactive_anon
     13858 ±  4%      +8.7%      15058 ±  3%  proc-vmstat.nr_mapped
     23239            +8.2%      25139        proc-vmstat.nr_slab_reclaimable
     64719            +4.9%      67881        proc-vmstat.nr_slab_unreclaimable
     72843            +1.4%      73890        proc-vmstat.nr_zone_inactive_anon
    432.33 ±106%    +501.7%       2601 ± 75%  proc-vmstat.numa_hint_faults_local
   2127259 ± 26%   +3531.9%   77260442 ±  6%  proc-vmstat.numa_hit
   2040648 ± 27%   +3681.8%   77173834 ±  6%  proc-vmstat.numa_local
  14326984 ± 32%   +4233.7%  6.209e+08 ±  6%  proc-vmstat.pgalloc_normal
    245168            +3.8%     254432        proc-vmstat.pgfault
  14128459 ± 33%   +4293.1%  6.207e+08 ±  6%  proc-vmstat.pgfree
     11746 ±  4%     +11.4%      13087 ±  2%  proc-vmstat.pgreuse
      5137 ±  7%     -50.9%       2522 ± 70%  sched_debug.cfs_rq:/.load.min
      3.92 ± 39%     -76.6%       0.92 ± 48%  sched_debug.cfs_rq:/.load_avg.min
   1776013           +17.9%    2094409        sched_debug.cfs_rq:/.min_vruntime.avg
   1883764 ±  3%     +16.9%    2202539        sched_debug.cfs_rq:/.min_vruntime.max
   1694658           +14.5%    1940737        sched_debug.cfs_rq:/.min_vruntime.min
      1616 ±  9%     +40.1%       2265 ± 17%  sched_debug.cfs_rq:/.runnable_avg.max
    199.99 ± 19%     +66.2%     332.41 ± 16%  sched_debug.cfs_rq:/.runnable_avg.stddev
      1180 ± 14%     +31.9%       1556 ± 10%  sched_debug.cfs_rq:/.util_avg.max
    455.00 ± 25%     -62.5%     170.83 ± 21%  sched_debug.cfs_rq:/.util_avg.min
    169.21 ± 19%     +47.8%     250.04 ±  9%  sched_debug.cfs_rq:/.util_avg.stddev
    448.20 ±  3%     -16.5%     374.33 ±  5%  sched_debug.cfs_rq:/.util_est_enqueued.avg
    194.00 ± 22%     -89.0%      21.42 ±219%  sched_debug.cfs_rq:/.util_est_enqueued.min
    162.52 ±  6%     +20.6%     196.01 ±  6%  sched_debug.cfs_rq:/.util_est_enqueued.stddev
     14.15 ±  5%     -50.6%       6.99 ± 38%  sched_debug.cpu.clock.stddev
    602.77 ±  4%     +28.1%     772.40 ± 12%  sched_debug.cpu.clock_task.stddev
     21469 ± 15%     +56.1%      33509 ± 31%  sched_debug.cpu.max_idle_balance_cost.stddev
      0.41 ±  3%     +12.8%       0.46 ±  9%  sched_debug.cpu.nr_running.stddev
   4424282           -56.5%    1922647 ±  9%  sched_debug.cpu.nr_switches.avg
   4564285           -51.3%    2224187 ±  8%  sched_debug.cpu.nr_switches.max
   4082983 ±  2%     -61.8%    1559208 ± 13%  sched_debug.cpu.nr_switches.min
     69357 ±  6%    +245.7%     239751 ± 38%  sched_debug.cpu.nr_switches.stddev
      1304 ±  4%     +17.2%       1527 ±  2%  slabinfo.TCP.active_objs
      1304 ±  4%     +17.2%       1527 ±  2%  slabinfo.TCP.num_objs
      6641 ±  2%     +48.0%       9833 ±  7%  slabinfo.dmaengine-unmap-16.active_objs
      6641 ±  2%     +48.0%       9833 ±  7%  slabinfo.dmaengine-unmap-16.num_objs
     26279 ±  5%     -12.7%      22947        slabinfo.filp.active_objs
    855.50 ±  5%     -14.6%     730.83        slabinfo.filp.active_slabs
     27392 ±  5%     -14.6%      23404        slabinfo.filp.num_objs
    855.50 ±  5%     -14.6%     730.83        slabinfo.filp.num_slabs
     11247 ±  2%     +30.8%      14710        slabinfo.kmalloc-1k.active_objs
    354.67 ±  2%     +30.5%     462.67        slabinfo.kmalloc-1k.active_slabs
     11366 ±  2%     +30.3%      14815        slabinfo.kmalloc-1k.num_objs
    354.67 ±  2%     +30.5%     462.67        slabinfo.kmalloc-1k.num_slabs
    613.33 ±  8%     -24.3%     464.00 ± 17%  slabinfo.kmalloc-rcl-128.active_objs
    613.33 ±  8%     -24.3%     464.00 ± 17%  slabinfo.kmalloc-rcl-128.num_objs
      2107 ±  7%     -11.6%       1862 ±  8%  slabinfo.kmalloc-rcl-96.active_objs
      2107 ±  7%     -11.6%       1862 ±  8%  slabinfo.kmalloc-rcl-96.num_objs
      9878 ± 17%     +89.8%      18752        slabinfo.skbuff_fclone_cache.active_objs
    308.17 ± 17%     +90.5%     587.00        slabinfo.skbuff_fclone_cache.active_slabs
      9882 ± 17%     +90.2%      18799        slabinfo.skbuff_fclone_cache.num_objs
    308.17 ± 17%     +90.5%     587.00        slabinfo.skbuff_fclone_cache.num_slabs
     10637 ±  5%     +41.6%      15066 ±  4%  slabinfo.sock_inode_cache.active_objs
    272.50 ±  5%     +43.5%     391.17 ±  4%  slabinfo.sock_inode_cache.active_slabs
     10645 ±  5%     +43.6%      15286 ±  4%  slabinfo.sock_inode_cache.num_objs
    272.50 ±  5%     +43.5%     391.17 ±  4%  slabinfo.sock_inode_cache.num_slabs
      4.45 ±  3%    +154.6%      11.33 ±  5%  perf-stat.i.MPKI
 3.343e+10           -24.5%  2.524e+10 ±  2%  perf-stat.i.branch-instructions
      1.57            -0.0        1.55        perf-stat.i.branch-miss-rate%
 4.986e+08           -25.5%  3.716e+08 ±  2%  perf-stat.i.branch-misses
      2.63 ±  4%     +15.2       17.87 ±  2%  perf-stat.i.cache-miss-rate%
  14369471 ±  4%   +1660.1%  2.529e+08 ±  5%  perf-stat.i.cache-misses
 7.472e+08 ±  3%     +88.0%  1.404e+09 ±  4%  perf-stat.i.cache-references
  14175402           -56.5%    6165131 ±  8%  perf-stat.i.context-switches
      1.42           +37.9%       1.96 ±  2%  perf-stat.i.cpi
  2.41e+11            +2.1%   2.46e+11        perf-stat.i.cpu-cycles
      2786 ± 85%   +7071.1%     199815 ±  6%  perf-stat.i.cpu-migrations
     16638 ±  4%     -93.1%       1148 ± 20%  perf-stat.i.cycles-between-cache-misses
      0.00 ±  9%      +0.0        0.02 ± 10%  perf-stat.i.dTLB-load-miss-rate%
    847687 ± 12%    +570.6%    5684921 ±  9%  perf-stat.i.dTLB-load-misses
 4.994e+10           -27.0%  3.645e+10 ±  2%  perf-stat.i.dTLB-loads
      0.00 ± 11%      +0.0        0.02 ± 18%  perf-stat.i.dTLB-store-miss-rate%
    131018 ± 17%   +3378.5%    4557442 ± 17%  perf-stat.i.dTLB-store-misses
 2.874e+10           -29.4%  2.029e+10 ±  2%  perf-stat.i.dTLB-stores
     68.35            +6.4       74.79        perf-stat.i.iTLB-load-miss-rate%
 3.051e+08           -31.3%  2.096e+08 ±  2%  perf-stat.i.iTLB-load-misses
 1.363e+08           -49.0%   69593955 ±  8%  perf-stat.i.iTLB-loads
 1.679e+11           -25.7%  1.248e+11 ±  2%  perf-stat.i.instructions
    702.32            +6.3%     746.70        perf-stat.i.instructions-per-iTLB-miss
      0.71           -25.8%       0.52 ±  2%  perf-stat.i.ipc
      2.51            +2.1%       2.56        perf-stat.i.metric.GHz
      0.72 ± 24%     -82.7%       0.13 ± 23%  perf-stat.i.metric.K/sec
      1177           -26.0%     870.71 ±  2%  perf-stat.i.metric.M/sec
      3590            +3.6%       3719        perf-stat.i.minor-faults
     84.06           -78.5        5.58 ± 20%  perf-stat.i.node-load-miss-rate%
   1437587 ±  3%    +115.6%    3099118 ±  5%  perf-stat.i.node-load-misses
    367232 ± 14%  +30953.2%   1.14e+08 ±  5%  perf-stat.i.node-loads
     90.98           -64.4       26.57 ±  5%  perf-stat.i.node-store-miss-rate%
    855437 ±  3%    +269.7%    3162161 ±  7%  perf-stat.i.node-store-misses
     90133 ± 22%   +9998.1%    9101716 ±  6%  perf-stat.i.node-stores
      3602            +3.6%       3731        perf-stat.i.page-faults
      4.45 ±  3%    +153.6%      11.29 ±  6%  perf-stat.overall.MPKI
      1.49            -0.0        1.47        perf-stat.overall.branch-miss-rate%
      1.93 ±  6%     +16.1       18.00        perf-stat.overall.cache-miss-rate%
      1.44           +37.6%       1.97 ±  2%  perf-stat.overall.cpi
     16783 ±  4%     -94.2%     974.85 ±  5%  perf-stat.overall.cycles-between-cache-misses
      0.00 ± 11%      +0.0        0.02 ± 10%  perf-stat.overall.dTLB-load-miss-rate%
      0.00 ± 17%      +0.0        0.02 ± 18%  perf-stat.overall.dTLB-store-miss-rate%
     69.11            +6.0       75.13        perf-stat.overall.iTLB-load-miss-rate%
    550.55            +8.1%     595.17        perf-stat.overall.instructions-per-iTLB-miss
      0.70           -27.3%       0.51 ±  2%  perf-stat.overall.ipc
     79.52 ±  2%     -76.9        2.65 ±  3%  perf-stat.overall.node-load-miss-rate%
     90.36           -64.6       25.79 ±  2%  perf-stat.overall.node-store-miss-rate%
  3.29e+10           -24.5%  2.482e+10 ±  2%  perf-stat.ps.branch-instructions
 4.906e+08           -25.5%  3.654e+08 ±  2%  perf-stat.ps.branch-misses
  14156133 ±  4%   +1659.4%  2.491e+08 ±  5%  perf-stat.ps.cache-misses
 7.355e+08 ±  3%     +88.0%  1.383e+09 ±  4%  perf-stat.ps.cache-references
  13947799           -56.6%    6057134 ±  8%  perf-stat.ps.context-switches
 2.372e+11            +2.1%  2.421e+11        perf-stat.ps.cpu-cycles
      2761 ± 85%   +7024.8%     196786 ±  6%  perf-stat.ps.cpu-migrations
    863926 ± 11%    +548.2%    5600307 ±  9%  perf-stat.ps.dTLB-load-misses
 4.914e+10           -27.0%  3.585e+10 ±  2%  perf-stat.ps.dTLB-loads
    130017 ± 17%   +3348.4%    4483540 ± 17%  perf-stat.ps.dTLB-store-misses
 2.828e+10           -29.4%  1.995e+10 ±  2%  perf-stat.ps.dTLB-stores
 3.002e+08           -31.3%  2.062e+08 ±  2%  perf-stat.ps.iTLB-load-misses
 1.341e+08           -49.0%   68383313 ±  7%  perf-stat.ps.iTLB-loads
 1.652e+11           -25.7%  1.227e+11 ±  2%  perf-stat.ps.instructions
      3535            +3.6%       3662        perf-stat.ps.minor-faults
   1414957 ±  3%    +115.7%    3052181 ±  5%  perf-stat.ps.node-load-misses
    366486 ± 14%  +30547.1%  1.123e+08 ±  5%  perf-stat.ps.node-loads
    842073 ±  3%    +270.1%    3116228 ±  6%  perf-stat.ps.node-store-misses
     90486 ± 22%   +9808.6%    8965949 ±  5%  perf-stat.ps.node-stores
      3547            +3.6%       3673        perf-stat.ps.page-faults
 1.037e+13           -25.6%  7.715e+12 ±  2%  perf-stat.total.instructions
   4560843           -50.2%    2272786 ±  6%  softirqs.CPU0.NET_RX
     21241 ±  9%     +23.3%      26193 ±  3%  softirqs.CPU0.RCU
   4498329 ±  3%     -49.5%    2270538 ±  6%  softirqs.CPU1.NET_RX
   4543550           -49.7%    2285392 ±  6%  softirqs.CPU10.NET_RX
     18235 ±  2%     +15.1%      20985 ±  3%  softirqs.CPU10.RCU
   4511949           -49.7%    2271032 ±  7%  softirqs.CPU11.NET_RX
     18035           +15.3%      20792        softirqs.CPU11.RCU
   4481757           -49.5%    2261062 ±  7%  softirqs.CPU12.NET_RX
     18073           +15.3%      20840 ±  2%  softirqs.CPU12.RCU
   4499673           -49.9%    2252168 ±  6%  softirqs.CPU13.NET_RX
     18301           +14.3%      20926 ±  3%  softirqs.CPU13.RCU
   4508786           -49.3%    2284034 ±  7%  softirqs.CPU14.NET_RX
     18182           +16.4%      21162        softirqs.CPU14.RCU
   4493226           -50.0%    2246240 ±  6%  softirqs.CPU15.NET_RX
     18292           +14.4%      20928        softirqs.CPU15.RCU
   4513084           -49.7%    2272269 ±  7%  softirqs.CPU16.NET_RX
     18695 ±  3%     +16.0%      21689        softirqs.CPU16.RCU
   4429321 ±  2%     -49.8%    2225171 ± 10%  softirqs.CPU17.NET_RX
     18950 ±  2%     +13.7%      21548 ±  3%  softirqs.CPU17.RCU
   4493697           -49.4%    2274327 ±  7%  softirqs.CPU18.NET_RX
     19177           +14.7%      21992        softirqs.CPU18.RCU
   4516955           -49.4%    2285842 ±  7%  softirqs.CPU19.NET_RX
     19178           +12.3%      21543        softirqs.CPU19.RCU
   4551109           -50.5%    2251698 ±  6%  softirqs.CPU2.NET_RX
     18411           +14.4%      21064        softirqs.CPU2.RCU
   4522378           -50.8%    2225906 ±  7%  softirqs.CPU20.NET_RX
     19110           +15.7%      22119 ±  2%  softirqs.CPU20.RCU
   4514623           -50.3%    2243136 ±  7%  softirqs.CPU21.NET_RX
     19218 ±  2%     +12.8%      21687        softirqs.CPU21.RCU
   4477606 ±  2%     -50.1%    2234593 ±  7%  softirqs.CPU22.NET_RX
     18882           +14.5%      21618        softirqs.CPU22.RCU
   4504922           -50.0%    2251753 ±  6%  softirqs.CPU23.NET_RX
     19164           +14.0%      21851 ±  2%  softirqs.CPU23.RCU
   4392559           -51.7%    2123133 ± 17%  softirqs.CPU24.NET_RX
     18593           +15.0%      21374        softirqs.CPU24.RCU
   4491895           -52.4%    2139804 ± 17%  softirqs.CPU25.NET_RX
   4492562           -52.7%    2124465 ± 17%  softirqs.CPU26.NET_RX
     18774           +14.6%      21516 ±  3%  softirqs.CPU26.RCU
   4524012           -52.7%    2142018 ± 16%  softirqs.CPU27.NET_RX
     18705           +12.7%      21088 ±  3%  softirqs.CPU27.RCU
   4487532           -51.7%    2166923 ± 16%  softirqs.CPU28.NET_RX
     18890 ±  2%     +13.0%      21337        softirqs.CPU28.RCU
   4484561           -52.2%    2143458 ± 16%  softirqs.CPU29.NET_RX
     19019 ±  4%     +12.3%      21355 ±  2%  softirqs.CPU29.RCU
   4551360           -49.9%    2281174 ±  7%  softirqs.CPU3.NET_RX
     18366           +15.4%      21197        softirqs.CPU3.RCU
   4480179           -52.3%    2139257 ± 16%  softirqs.CPU30.NET_RX
     18520 ±  6%     +14.7%      21245        softirqs.CPU30.RCU
   4476574           -51.9%    2153958 ± 17%  softirqs.CPU31.NET_RX
     18594           +15.0%      21383 ±  2%  softirqs.CPU31.RCU
   4485217           -52.8%    2115908 ± 17%  softirqs.CPU32.NET_RX
     17909           +17.8%      21103 ±  2%  softirqs.CPU32.RCU
   4476301           -52.3%    2136742 ± 17%  softirqs.CPU33.NET_RX
     18182 ±  3%     +16.6%      21207 ±  5%  softirqs.CPU33.RCU
   4470120           -52.4%    2126390 ± 17%  softirqs.CPU34.NET_RX
     17982           +16.8%      20996 ±  2%  softirqs.CPU34.RCU
   4443917           -52.1%    2128227 ± 16%  softirqs.CPU35.NET_RX
     18180           +15.9%      21075 ±  3%  softirqs.CPU35.RCU
   4473260           -52.6%    2122457 ± 16%  softirqs.CPU36.NET_RX
     18125 ±  2%     +18.6%      21502 ±  4%  softirqs.CPU36.RCU
   4472379           -52.7%    2114795 ± 17%  softirqs.CPU37.NET_RX
     18007           +16.4%      20957 ±  2%  softirqs.CPU37.RCU
   4461059           -52.1%    2137066 ± 17%  softirqs.CPU38.NET_RX
     17935           +17.7%      21113 ±  2%  softirqs.CPU38.RCU
   4455317           -52.2%    2129969 ± 17%  softirqs.CPU39.NET_RX
     17911           +17.3%      21016 ±  3%  softirqs.CPU39.RCU
   4553331           -49.4%    2302532 ±  6%  softirqs.CPU4.NET_RX
     18113           +15.7%      20961 ±  2%  softirqs.CPU4.RCU
   4473435           -52.3%    2134944 ± 15%  softirqs.CPU40.NET_RX
     18248 ±  3%     +14.5%      20893 ±  2%  softirqs.CPU40.RCU
   4465184           -51.6%    2163077 ± 16%  softirqs.CPU41.NET_RX
     18202           +14.4%      20817 ±  2%  softirqs.CPU41.RCU
   4489626           -52.6%    2126225 ± 17%  softirqs.CPU42.NET_RX
     18161 ±  2%     +14.8%      20855 ±  2%  softirqs.CPU42.RCU
   4443047           -51.4%    2159443 ± 16%  softirqs.CPU43.NET_RX
     18129 ±  2%     +15.3%      20908 ±  2%  softirqs.CPU43.RCU
   4476402           -52.0%    2146618 ± 16%  softirqs.CPU44.NET_RX
     17536 ±  4%     +18.3%      20737 ±  2%  softirqs.CPU44.RCU
   4482489           -52.5%    2130397 ± 16%  softirqs.CPU45.NET_RX
     17834           +18.0%      21046 ±  3%  softirqs.CPU45.RCU
   4477349           -52.5%    2124980 ± 17%  softirqs.CPU46.NET_RX
     18077           +15.9%      20949 ±  2%  softirqs.CPU46.RCU
   4475117           -52.4%    2130668 ± 16%  softirqs.CPU47.NET_RX
     18436 ±  4%     +13.5%      20933 ±  2%  softirqs.CPU47.RCU
   4373778 ±  3%     -49.0%    2232155 ±  7%  softirqs.CPU48.NET_RX
     17745 ±  2%     +15.6%      20514 ±  2%  softirqs.CPU48.RCU
   4496858           -50.7%    2217693 ±  6%  softirqs.CPU49.NET_RX
     17594 ±  3%     +17.8%      20719 ±  4%  softirqs.CPU49.RCU
   4526397           -49.5%    2284088 ±  6%  softirqs.CPU5.NET_RX
     18140           +14.9%      20843        softirqs.CPU5.RCU
   4517853           -49.6%    2274925 ±  6%  softirqs.CPU50.NET_RX
   4496903           -49.2%    2285523 ±  6%  softirqs.CPU51.NET_RX
     17902           +17.0%      20945        softirqs.CPU51.RCU
   4520473           -50.0%    2261934 ±  6%  softirqs.CPU52.NET_RX
   4497228           -49.5%    2269608 ±  6%  softirqs.CPU53.NET_RX
     18098           +15.2%      20855        softirqs.CPU53.RCU
   4511411           -49.9%    2261161 ±  6%  softirqs.CPU54.NET_RX
     18471 ±  2%     +12.3%      20750        softirqs.CPU54.RCU
   4520114           -49.6%    2280245 ±  6%  softirqs.CPU55.NET_RX
     18106 ±  2%     +15.4%      20890 ±  3%  softirqs.CPU55.RCU
   4525942           -50.1%    2258939 ±  7%  softirqs.CPU56.NET_RX
     18002           +15.6%      20806 ±  2%  softirqs.CPU56.RCU
   4504366           -49.2%    2286767 ±  6%  softirqs.CPU57.NET_RX
     18075           +17.6%      21253 ±  3%  softirqs.CPU57.RCU
   4494686           -49.1%    2288598 ±  6%  softirqs.CPU58.NET_RX
     18444 ±  3%     +13.3%      20888        softirqs.CPU58.RCU
   4518913           -49.7%    2272563 ±  5%  softirqs.CPU59.NET_RX
     18077           +16.9%      21127 ±  4%  softirqs.CPU59.RCU
   4514808           -49.5%    2278657 ±  6%  softirqs.CPU6.NET_RX
     17961           +17.4%      21089 ±  2%  softirqs.CPU6.RCU
   4508378           -49.5%    2275322 ±  6%  softirqs.CPU60.NET_RX
     18163           +14.9%      20875        softirqs.CPU60.RCU
   4507669           -49.8%    2262680 ±  6%  softirqs.CPU61.NET_RX
     17799           +16.1%      20659        softirqs.CPU61.RCU
   4524653           -49.5%    2286102 ±  6%  softirqs.CPU62.NET_RX
     18109           +13.8%      20607        softirqs.CPU62.RCU
   4519567           -50.0%    2257645 ±  7%  softirqs.CPU63.NET_RX
     17920           +15.1%      20624 ±  3%  softirqs.CPU63.RCU
   4515072           -49.9%    2260971 ±  6%  softirqs.CPU64.NET_RX
     19783 ±  4%     +11.5%      22066        softirqs.CPU64.RCU
   4507658           -49.6%    2271667 ±  5%  softirqs.CPU65.NET_RX
     19468 ±  3%     +12.4%      21883        softirqs.CPU65.RCU
   4519295           -49.4%    2286044 ±  7%  softirqs.CPU66.NET_RX
     19478           +11.6%      21739        softirqs.CPU66.RCU
   4560346           -49.8%    2287499 ±  6%  softirqs.CPU67.NET_RX
     19841 ±  2%     +10.3%      21880        softirqs.CPU67.RCU
   4598839           -50.5%    2276110 ±  7%  softirqs.CPU68.NET_RX
     19499           +14.1%      22248 ±  3%  softirqs.CPU68.RCU
   4595176           -50.4%    2279210 ±  6%  softirqs.CPU69.NET_RX
     19782 ±  3%     +11.4%      22033        softirqs.CPU69.RCU
   4523914           -49.3%    2292970 ±  7%  softirqs.CPU7.NET_RX
     18066           +15.8%      20918        softirqs.CPU7.RCU
   4596985           -50.5%    2273992 ±  8%  softirqs.CPU70.NET_RX
     19518           +12.3%      21912        softirqs.CPU70.RCU
   4597696           -50.9%    2257307 ±  5%  softirqs.CPU71.NET_RX
     19724           +11.5%      21987        softirqs.CPU71.RCU
   4398050           -52.0%    2110908 ± 16%  softirqs.CPU72.NET_RX
     19263           +11.0%      21389 ±  2%  softirqs.CPU72.RCU
   4482365           -52.1%    2145713 ± 16%  softirqs.CPU73.NET_RX
     19202 ±  2%     +13.1%      21709 ±  2%  softirqs.CPU73.RCU
   4453475           -52.1%    2131900 ± 17%  softirqs.CPU74.NET_RX
     18981           +13.3%      21508        softirqs.CPU74.RCU
   4407861 ±  3%     -51.5%    2137316 ± 16%  softirqs.CPU75.NET_RX
     18845 ±  2%     +13.6%      21400 ±  2%  softirqs.CPU75.RCU
   4455731           -52.1%    2133073 ± 16%  softirqs.CPU76.NET_RX
     18978           +13.0%      21450 ±  2%  softirqs.CPU76.RCU
   4473339           -52.4%    2127571 ± 16%  softirqs.CPU77.NET_RX
     19010           +12.6%      21407 ±  2%  softirqs.CPU77.RCU
   4451979           -52.0%    2136060 ± 16%  softirqs.CPU78.NET_RX
     19020           +12.6%      21420 ±  2%  softirqs.CPU78.RCU
   4450498           -51.7%    2148363 ± 17%  softirqs.CPU79.NET_RX
     18837 ±  5%     +13.8%      21430 ±  2%  softirqs.CPU79.RCU
   4566338           -50.2%    2275865 ±  6%  softirqs.CPU8.NET_RX
     18206           +17.9%      21462 ±  3%  softirqs.CPU8.RCU
   4483014           -52.4%    2132319 ± 16%  softirqs.CPU80.NET_RX
     17798           +16.6%      20756 ±  2%  softirqs.CPU80.RCU
   4457292           -52.2%    2131632 ± 16%  softirqs.CPU81.NET_RX
     17826           +17.7%      20984 ±  3%  softirqs.CPU81.RCU
   4457882           -52.3%    2124800 ± 16%  softirqs.CPU82.NET_RX
     17995 ±  2%     +15.5%      20790 ±  2%  softirqs.CPU82.RCU
   4468206           -52.6%    2117092 ± 17%  softirqs.CPU83.NET_RX
     17839           +16.4%      20766 ±  2%  softirqs.CPU83.RCU
   4466076           -52.3%    2131641 ± 17%  softirqs.CPU84.NET_RX
     17808           +19.9%      21347 ±  4%  softirqs.CPU84.RCU
   4460142 ±  2%     -52.2%    2133593 ± 18%  softirqs.CPU85.NET_RX
     17759           +17.3%      20824 ±  2%  softirqs.CPU85.RCU
   4455148           -51.8%    2145521 ± 16%  softirqs.CPU86.NET_RX
     17811           +17.8%      20986 ±  3%  softirqs.CPU86.RCU
   4484463           -52.3%    2136959 ± 16%  softirqs.CPU87.NET_RX
     17794           +17.2%      20861        softirqs.CPU87.RCU
   4493967           -52.5%    2135760 ± 16%  softirqs.CPU88.NET_RX
     17951 ±  2%     +15.7%      20765 ±  2%  softirqs.CPU88.RCU
   4490187           -52.3%    2140828 ± 16%  softirqs.CPU89.NET_RX
     17910           +17.3%      21003        softirqs.CPU89.RCU
   4530910           -49.6%    2283047 ±  7%  softirqs.CPU9.NET_RX
     18161           +15.4%      20964 ±  2%  softirqs.CPU9.RCU
   4488277           -52.4%    2136450 ± 16%  softirqs.CPU90.NET_RX
     17769           +17.0%      20797        softirqs.CPU90.RCU
   4470274           -52.2%    2135528 ± 16%  softirqs.CPU91.NET_RX
     18367 ±  4%     +12.7%      20706 ±  2%  softirqs.CPU91.RCU
   4481323           -52.2%    2140226 ± 17%  softirqs.CPU92.NET_RX
   4506115           -52.7%    2132735 ± 17%  softirqs.CPU93.NET_RX
     17943 ±  2%     +16.5%      20906 ±  2%  softirqs.CPU93.RCU
   4490969           -52.5%    2133008 ± 17%  softirqs.CPU94.NET_RX
     17878           +17.2%      20960 ±  2%  softirqs.CPU94.RCU
   4289458           -52.9%    2018653 ± 17%  softirqs.CPU95.NET_RX
     18053           +14.9%      20735        softirqs.CPU95.RCU
 4.313e+08           -51.0%  2.112e+08 ±  6%  softirqs.NET_RX
   1773319           +14.9%    2037601        softirqs.RCU
    250242 ±  3%    +118.6%     546975 ±  3%  softirqs.SCHED
    482254           +33.4%     643317        softirqs.TIMER
     74585 ± 23%    +888.7%     737428 ±  6%  interrupts.CAL:Function_call_interrupts
    949.50 ± 46%    +759.3%       8159 ±  5%  interrupts.CPU0.CAL:Function_call_interrupts
    311.83 ± 79%   +1239.8%       4177 ± 10%  interrupts.CPU0.RES:Rescheduling_interrupts
      1701 ± 32%    +348.6%       7631 ± 14%  interrupts.CPU1.CAL:Function_call_interrupts
    430.00 ±152%    +939.6%       4470 ± 21%  interrupts.CPU1.RES:Rescheduling_interrupts
    813.17 ± 36%    +812.0%       7415 ±  3%  interrupts.CPU10.CAL:Function_call_interrupts
    167.50 ±103%   +2386.9%       4165 ±  6%  interrupts.CPU10.RES:Rescheduling_interrupts
    915.83 ± 32%    +765.5%       7926 ±  5%  interrupts.CPU11.CAL:Function_call_interrupts
    761.17 ±160%    +549.6%       4944 ± 10%  interrupts.CPU11.RES:Rescheduling_interrupts
    849.33 ± 39%    +757.2%       7280 ±  6%  interrupts.CPU12.CAL:Function_call_interrupts
    195.67 ± 92%   +2177.6%       4456 ± 17%  interrupts.CPU12.RES:Rescheduling_interrupts
    841.00 ± 35%    +748.5%       7135 ±  5%  interrupts.CPU13.CAL:Function_call_interrupts
    154.50 ± 69%   +2810.5%       4496 ± 17%  interrupts.CPU13.RES:Rescheduling_interrupts
    988.83 ± 58%    +687.6%       7787 ±  7%  interrupts.CPU14.CAL:Function_call_interrupts
    744.67 ±171%    +483.4%       4344 ±  9%  interrupts.CPU14.RES:Rescheduling_interrupts
    875.83 ± 41%    +735.4%       7316 ±  7%  interrupts.CPU15.CAL:Function_call_interrupts
    480.00 ±157%    +769.2%       4172 ±  8%  interrupts.CPU15.RES:Rescheduling_interrupts
    831.67 ± 36%    +773.6%       7265 ±  5%  interrupts.CPU16.CAL:Function_call_interrupts
    117.67 ± 18%   +3542.1%       4285 ±  4%  interrupts.CPU16.RES:Rescheduling_interrupts
    791.00 ± 39%    +871.6%       7685 ±  6%  interrupts.CPU17.CAL:Function_call_interrupts
    116.17 ± 35%   +4163.7%       4953 ± 17%  interrupts.CPU17.RES:Rescheduling_interrupts
    858.50 ± 38%    +795.7%       7689 ±  5%  interrupts.CPU18.CAL:Function_call_interrupts
    122.50 ± 49%   +3339.9%       4213 ± 12%  interrupts.CPU18.RES:Rescheduling_interrupts
    845.67 ± 37%    +813.6%       7725 ±  5%  interrupts.CPU19.CAL:Function_call_interrupts
    108.17 ± 35%   +3729.7%       4142 ±  8%  interrupts.CPU19.RES:Rescheduling_interrupts
      1069 ± 42%    +626.1%       7764 ±  9%  interrupts.CPU2.CAL:Function_call_interrupts
    245.50 ±138%   +1761.0%       4568 ± 16%  interrupts.CPU2.RES:Rescheduling_interrupts
    831.67 ± 37%    +775.0%       7276 ±  5%  interrupts.CPU20.CAL:Function_call_interrupts
    180.17 ± 97%   +2586.4%       4840 ± 17%  interrupts.CPU20.RES:Rescheduling_interrupts
    868.67 ± 35%    +756.8%       7443 ±  7%  interrupts.CPU21.CAL:Function_call_interrupts
    334.00 ±102%   +1274.1%       4589 ± 11%  interrupts.CPU21.RES:Rescheduling_interrupts
    815.50 ± 38%    +792.7%       7279 ±  7%  interrupts.CPU22.CAL:Function_call_interrupts
    207.00 ±112%   +1974.5%       4294 ± 14%  interrupts.CPU22.RES:Rescheduling_interrupts
    877.00 ± 40%    +737.8%       7347 ±  3%  interrupts.CPU23.CAL:Function_call_interrupts
    431.50 ± 91%    +880.0%       4228 ±  6%  interrupts.CPU23.RES:Rescheduling_interrupts
    771.67 ± 24%    +914.2%       7826 ± 14%  interrupts.CPU24.CAL:Function_call_interrupts
    669.17 ±123%    +597.9%       4670 ± 12%  interrupts.CPU24.RES:Rescheduling_interrupts
    652.83 ± 21%   +1166.0%       8265 ± 15%  interrupts.CPU25.CAL:Function_call_interrupts
    187.17 ± 68%   +2520.1%       4904 ±  9%  interrupts.CPU25.RES:Rescheduling_interrupts
    710.67 ± 16%    +998.2%       7804 ± 14%  interrupts.CPU26.CAL:Function_call_interrupts
    795.50 ± 94%    +455.8%       4421 ±  9%  interrupts.CPU26.RES:Rescheduling_interrupts
    663.33 ± 19%   +1117.2%       8074 ± 15%  interrupts.CPU27.CAL:Function_call_interrupts
    321.50 ±103%   +1195.1%       4163 ± 10%  interrupts.CPU27.RES:Rescheduling_interrupts
    728.33 ± 24%   +1008.5%       8073 ± 13%  interrupts.CPU28.CAL:Function_call_interrupts
    600.67 ±146%    +628.2%       4374 ± 11%  interrupts.CPU28.RES:Rescheduling_interrupts
    646.67 ± 20%   +1205.0%       8438 ± 14%  interrupts.CPU29.CAL:Function_call_interrupts
    135.00 ± 69%   +3127.9%       4357 ± 10%  interrupts.CPU29.RES:Rescheduling_interrupts
      1045 ± 32%    +638.8%       7723 ±  8%  interrupts.CPU3.CAL:Function_call_interrupts
    227.50 ± 70%   +1781.5%       4280 ± 11%  interrupts.CPU3.RES:Rescheduling_interrupts
    668.83 ± 18%   +1095.6%       7996 ± 15%  interrupts.CPU30.CAL:Function_call_interrupts
    286.67 ±159%   +1636.2%       4977 ± 13%  interrupts.CPU30.RES:Rescheduling_interrupts
    657.50 ± 27%   +1154.6%       8249 ± 15%  interrupts.CPU31.CAL:Function_call_interrupts
    404.33 ±149%    +956.7%       4272 ± 12%  interrupts.CPU31.RES:Rescheduling_interrupts
    591.50 ± 17%   +1204.1%       7713 ± 14%  interrupts.CPU32.CAL:Function_call_interrupts
    103.17 ± 77%   +4475.8%       4720 ± 14%  interrupts.CPU32.RES:Rescheduling_interrupts
    716.17 ± 22%   +1013.3%       7973 ± 15%  interrupts.CPU33.CAL:Function_call_interrupts
    502.67 ±122%    +745.2%       4248 ± 13%  interrupts.CPU33.RES:Rescheduling_interrupts
    694.33 ± 27%   +1097.4%       8313 ± 14%  interrupts.CPU34.CAL:Function_call_interrupts
    602.00 ±157%    +642.4%       4469 ± 17%  interrupts.CPU34.RES:Rescheduling_interrupts
    668.17 ± 21%   +1041.4%       7626 ± 15%  interrupts.CPU35.CAL:Function_call_interrupts
    392.33 ± 77%   +1021.7%       4400 ± 14%  interrupts.CPU35.RES:Rescheduling_interrupts
    656.83 ± 24%   +1073.3%       7706 ± 14%  interrupts.CPU36.CAL:Function_call_interrupts
    395.50 ±129%   +1018.1%       4422 ±  8%  interrupts.CPU36.RES:Rescheduling_interrupts
    769.50 ± 36%    +915.4%       7813 ± 13%  interrupts.CPU37.CAL:Function_call_interrupts
    644.67 ±161%    +647.9%       4821 ± 12%  interrupts.CPU37.RES:Rescheduling_interrupts
    796.33 ± 28%    +899.7%       7960 ± 16%  interrupts.CPU38.CAL:Function_call_interrupts
      1208 ±113%    +283.0%       4628 ± 11%  interrupts.CPU38.RES:Rescheduling_interrupts
    676.00 ± 22%   +1015.6%       7541 ± 14%  interrupts.CPU39.CAL:Function_call_interrupts
    650.83 ±131%    +591.3%       4499 ± 15%  interrupts.CPU39.RES:Rescheduling_interrupts
    914.00 ± 29%    +715.2%       7451 ±  6%  interrupts.CPU4.CAL:Function_call_interrupts
    173.17 ± 59%   +2370.7%       4278 ± 11%  interrupts.CPU4.RES:Rescheduling_interrupts
    647.67 ± 29%   +1118.6%       7892 ± 12%  interrupts.CPU40.CAL:Function_call_interrupts
    238.67 ±173%   +2078.4%       5199 ± 15%  interrupts.CPU40.RES:Rescheduling_interrupts
    666.17 ± 17%   +1137.1%       8241 ± 13%  interrupts.CPU41.CAL:Function_call_interrupts
    520.50 ±129%    +778.7%       4573 ± 14%  interrupts.CPU41.RES:Rescheduling_interrupts
    608.83 ± 16%   +1181.2%       7800 ± 15%  interrupts.CPU42.CAL:Function_call_interrupts
    180.83 ±147%   +2343.0%       4417 ± 19%  interrupts.CPU42.RES:Rescheduling_interrupts
    656.17 ± 19%   +1156.6%       8245 ± 13%  interrupts.CPU43.CAL:Function_call_interrupts
    256.00 ±165%   +1744.3%       4721 ± 14%  interrupts.CPU43.RES:Rescheduling_interrupts
    718.67 ± 19%   +1041.6%       8204 ± 13%  interrupts.CPU44.CAL:Function_call_interrupts
    750.00 ± 66%    +586.9%       5151 ± 13%  interrupts.CPU44.RES:Rescheduling_interrupts
    614.17 ± 14%   +1160.4%       7740 ± 12%  interrupts.CPU45.CAL:Function_call_interrupts
     93.33 ± 37%   +4536.1%       4327 ±  9%  interrupts.CPU45.RES:Rescheduling_interrupts
    627.50 ± 23%   +1113.4%       7614 ± 15%  interrupts.CPU46.CAL:Function_call_interrupts
     63.17 ± 41%   +6762.3%       4334 ±  8%  interrupts.CPU46.RES:Rescheduling_interrupts
    673.50 ± 24%   +1047.3%       7727 ± 14%  interrupts.CPU47.CAL:Function_call_interrupts
    447.67 ±171%    +797.8%       4019 ± 10%  interrupts.CPU47.RES:Rescheduling_interrupts
    901.33 ± 40%    +707.9%       7281 ±  3%  interrupts.CPU48.CAL:Function_call_interrupts
    273.00 ± 77%   +1507.9%       4389 ± 22%  interrupts.CPU48.RES:Rescheduling_interrupts
    792.17 ± 34%    +811.7%       7221 ±  7%  interrupts.CPU49.CAL:Function_call_interrupts
    220.33 ±119%   +1938.2%       4490 ± 23%  interrupts.CPU49.RES:Rescheduling_interrupts
    885.17 ± 48%    +781.6%       7803 ±  6%  interrupts.CPU5.CAL:Function_call_interrupts
    111.17 ± 37%   +4122.3%       4693 ±  8%  interrupts.CPU5.RES:Rescheduling_interrupts
    854.50 ± 41%    +796.8%       7663 ±  5%  interrupts.CPU50.CAL:Function_call_interrupts
     84.00 ± 43%   +5481.5%       4688 ± 17%  interrupts.CPU50.RES:Rescheduling_interrupts
    812.17 ± 38%    +839.6%       7631 ±  6%  interrupts.CPU51.CAL:Function_call_interrupts
    136.00 ± 62%   +2912.5%       4097 ±  6%  interrupts.CPU51.RES:Rescheduling_interrupts
    793.00 ± 32%    +886.8%       7825 ±  5%  interrupts.CPU52.CAL:Function_call_interrupts
    103.50 ± 35%   +3944.0%       4185 ± 18%  interrupts.CPU52.RES:Rescheduling_interrupts
    814.50 ± 38%    +843.9%       7688 ±  4%  interrupts.CPU53.CAL:Function_call_interrupts
    106.00 ± 54%   +4367.1%       4735 ± 26%  interrupts.CPU53.RES:Rescheduling_interrupts
    816.67 ± 36%    +810.2%       7433 ±  4%  interrupts.CPU54.CAL:Function_call_interrupts
    367.50 ±155%   +1119.7%       4482 ± 14%  interrupts.CPU54.RES:Rescheduling_interrupts
    797.00 ± 38%    +846.4%       7543 ±  5%  interrupts.CPU55.CAL:Function_call_interrupts
    188.50 ± 83%   +2103.5%       4153 ± 17%  interrupts.CPU55.RES:Rescheduling_interrupts
    796.17 ± 33%    +799.0%       7157 ±  4%  interrupts.CPU56.CAL:Function_call_interrupts
    115.50 ± 45%   +3379.9%       4019 ±  8%  interrupts.CPU56.RES:Rescheduling_interrupts
    822.83 ± 38%    +817.1%       7546 ±  5%  interrupts.CPU57.CAL:Function_call_interrupts
    118.00 ± 57%   +3346.8%       4067 ±  7%  interrupts.CPU57.RES:Rescheduling_interrupts
    859.17 ± 39%    +766.0%       7440 ±  3%  interrupts.CPU58.CAL:Function_call_interrupts
    194.33 ±109%   +1974.0%       4030 ± 10%  interrupts.CPU58.RES:Rescheduling_interrupts
    937.33 ± 34%    +713.6%       7626 ±  4%  interrupts.CPU59.CAL:Function_call_interrupts
    980.00 ±121%    +380.7%       4711 ± 12%  interrupts.CPU59.RES:Rescheduling_interrupts
    803.33 ± 36%    +843.5%       7579 ±  3%  interrupts.CPU6.CAL:Function_call_interrupts
    219.83 ± 85%   +1885.5%       4364 ± 16%  interrupts.CPU6.RES:Rescheduling_interrupts
    821.67 ± 36%    +778.9%       7221 ±  5%  interrupts.CPU60.CAL:Function_call_interrupts
    120.33 ± 54%   +3372.9%       4179 ± 10%  interrupts.CPU60.RES:Rescheduling_interrupts
    822.67 ± 37%    +751.5%       7005 ±  6%  interrupts.CPU61.CAL:Function_call_interrupts
     92.33 ± 46%   +5290.6%       4977 ± 14%  interrupts.CPU61.RES:Rescheduling_interrupts
    818.67 ± 35%    +830.7%       7619 ±  5%  interrupts.CPU62.CAL:Function_call_interrupts
    258.33 ±148%   +1511.4%       4162 ±  7%  interrupts.CPU62.RES:Rescheduling_interrupts
    836.00 ± 39%    +768.5%       7260 ±  6%  interrupts.CPU63.CAL:Function_call_interrupts
    405.17 ±157%   +1005.6%       4479 ±  7%  interrupts.CPU63.RES:Rescheduling_interrupts
    847.17 ± 35%    +746.2%       7168 ±  4%  interrupts.CPU64.CAL:Function_call_interrupts
    142.67 ± 44%   +2921.0%       4310 ±  8%  interrupts.CPU64.RES:Rescheduling_interrupts
    813.67 ± 33%    +783.4%       7187 ±  5%  interrupts.CPU65.CAL:Function_call_interrupts
    156.17 ± 73%   +2880.9%       4655 ± 26%  interrupts.CPU65.RES:Rescheduling_interrupts
    871.00 ± 35%    +781.8%       7680 ±  5%  interrupts.CPU66.CAL:Function_call_interrupts
    133.00 ± 29%   +3291.4%       4510 ± 11%  interrupts.CPU66.RES:Rescheduling_interrupts
    807.17 ± 29%    +815.9%       7392 ±  7%  interrupts.CPU67.CAL:Function_call_interrupts
    154.00 ± 42%   +2559.7%       4096 ±  6%  interrupts.CPU67.RES:Rescheduling_interrupts
    799.00 ± 28%    +774.9%       6990 ±  5%  interrupts.CPU68.CAL:Function_call_interrupts
    131.17 ± 25%   +3205.7%       4336 ±  7%  interrupts.CPU68.RES:Rescheduling_interrupts
    775.83 ± 29%    +809.8%       7058 ±  5%  interrupts.CPU69.CAL:Function_call_interrupts
    322.33 ±133%   +1370.1%       4738 ± 22%  interrupts.CPU69.RES:Rescheduling_interrupts
    873.33 ± 44%    +755.5%       7471 ±  5%  interrupts.CPU7.CAL:Function_call_interrupts
    869.17 ±151%    +393.3%       4287 ± 10%  interrupts.CPU7.RES:Rescheduling_interrupts
    783.67 ± 30%    +814.5%       7166 ±  6%  interrupts.CPU70.CAL:Function_call_interrupts
    119.67 ± 44%   +3966.4%       4866 ± 22%  interrupts.CPU70.RES:Rescheduling_interrupts
    857.67 ± 39%    +714.6%       6986 ±  5%  interrupts.CPU71.CAL:Function_call_interrupts
      1016 ±122%    +310.8%       4177 ± 14%  interrupts.CPU71.RES:Rescheduling_interrupts
    771.67 ± 23%    +906.2%       7764 ± 13%  interrupts.CPU72.CAL:Function_call_interrupts
    569.33 ± 94%    +702.7%       4570 ± 14%  interrupts.CPU72.RES:Rescheduling_interrupts
    697.17 ± 23%   +1079.9%       8225 ± 13%  interrupts.CPU73.CAL:Function_call_interrupts
    447.50 ±115%    +918.2%       4556 ±  8%  interrupts.CPU73.RES:Rescheduling_interrupts
    712.33 ± 16%    +962.1%       7565 ± 15%  interrupts.CPU74.CAL:Function_call_interrupts
    424.67 ± 79%   +1011.3%       4719 ± 10%  interrupts.CPU74.RES:Rescheduling_interrupts
    656.00 ± 17%   +1142.5%       8151 ± 14%  interrupts.CPU75.CAL:Function_call_interrupts
    161.67 ± 93%   +2668.7%       4476 ± 11%  interrupts.CPU75.RES:Rescheduling_interrupts
    648.00 ± 23%   +1167.7%       8214 ± 12%  interrupts.CPU76.CAL:Function_call_interrupts
    165.83 ±107%   +2423.5%       4184 ± 11%  interrupts.CPU76.RES:Rescheduling_interrupts
    647.17 ± 17%   +1163.1%       8174 ± 14%  interrupts.CPU77.CAL:Function_call_interrupts
     89.17 ± 38%   +4762.1%       4335 ± 12%  interrupts.CPU77.RES:Rescheduling_interrupts
    702.00 ± 19%   +1078.0%       8269 ± 15%  interrupts.CPU78.CAL:Function_call_interrupts
    342.83 ±115%   +1267.9%       4689 ± 17%  interrupts.CPU78.RES:Rescheduling_interrupts
    725.00 ± 19%   +1027.3%       8173 ± 14%  interrupts.CPU79.CAL:Function_call_interrupts
    487.83 ±103%    +796.8%       4374 ± 14%  interrupts.CPU79.RES:Rescheduling_interrupts
    843.17 ± 42%    +745.9%       7132 ±  4%  interrupts.CPU8.CAL:Function_call_interrupts
    479.50 ±153%    +851.2%       4561 ± 16%  interrupts.CPU8.RES:Rescheduling_interrupts
    629.17 ± 17%   +1119.4%       7671 ± 16%  interrupts.CPU80.CAL:Function_call_interrupts
    134.00 ± 94%   +3176.1%       4390 ±  8%  interrupts.CPU80.RES:Rescheduling_interrupts
    739.17 ± 25%   +1020.7%       8284 ± 14%  interrupts.CPU81.CAL:Function_call_interrupts
    588.67 ±131%    +655.6%       4448 ±  9%  interrupts.CPU81.RES:Rescheduling_interrupts
    738.83 ± 28%    +991.0%       8060 ± 15%  interrupts.CPU82.CAL:Function_call_interrupts
    936.00 ±141%    +331.6%       4039 ± 12%  interrupts.CPU82.RES:Rescheduling_interrupts
    649.83 ± 26%   +1076.6%       7645 ± 14%  interrupts.CPU83.CAL:Function_call_interrupts
    389.00 ±191%    +925.0%       3987 ± 11%  interrupts.CPU83.RES:Rescheduling_interrupts
    647.67 ± 18%   +1086.0%       7681 ± 16%  interrupts.CPU84.CAL:Function_call_interrupts
    157.33 ±115%   +2630.4%       4295 ± 13%  interrupts.CPU84.RES:Rescheduling_interrupts
    674.50 ± 18%   +1020.2%       7555 ± 15%  interrupts.CPU85.CAL:Function_call_interrupts
    439.83 ±122%    +919.1%       4482 ± 11%  interrupts.CPU85.RES:Rescheduling_interrupts
    762.00 ± 26%    +931.9%       7863 ± 13%  interrupts.CPU86.CAL:Function_call_interrupts
      1235 ± 55%    +280.0%       4694 ± 10%  interrupts.CPU86.RES:Rescheduling_interrupts
    672.33 ± 17%   +1062.5%       7816 ± 14%  interrupts.CPU87.CAL:Function_call_interrupts
    419.67 ± 95%    +960.2%       4449 ±  4%  interrupts.CPU87.RES:Rescheduling_interrupts
    638.67 ± 24%   +1100.3%       7665 ± 15%  interrupts.CPU88.CAL:Function_call_interrupts
    139.33 ±127%   +3397.1%       4872 ± 13%  interrupts.CPU88.RES:Rescheduling_interrupts
    661.67 ± 17%   +1151.1%       8277 ± 14%  interrupts.CPU89.CAL:Function_call_interrupts
    204.33 ± 93%   +2116.4%       4528 ±  7%  interrupts.CPU89.RES:Rescheduling_interrupts
    800.33 ± 34%    +839.1%       7515 ±  5%  interrupts.CPU9.CAL:Function_call_interrupts
     93.00 ± 59%   +4509.1%       4286 ±  9%  interrupts.CPU9.RES:Rescheduling_interrupts
    605.17 ± 25%   +1174.4%       7712 ± 13%  interrupts.CPU90.CAL:Function_call_interrupts
    188.33 ±110%   +2269.0%       4461 ± 13%  interrupts.CPU90.RES:Rescheduling_interrupts
    677.83 ± 19%   +1112.1%       8216 ± 14%  interrupts.CPU91.CAL:Function_call_interrupts
    310.00 ±152%   +1462.7%       4844 ± 14%  interrupts.CPU91.RES:Rescheduling_interrupts
    909.33 ± 51%    +790.0%       8092 ± 14%  interrupts.CPU92.CAL:Function_call_interrupts
    628.33 ± 20%   +1085.4%       7448 ± 14%  interrupts.CPU93.CAL:Function_call_interrupts
    265.50 ±118%   +1547.7%       4374 ± 10%  interrupts.CPU93.RES:Rescheduling_interrupts
    615.67 ± 16%   +1133.3%       7593 ± 16%  interrupts.CPU94.CAL:Function_call_interrupts
     77.67 ± 43%   +5157.7%       4083 ±  9%  interrupts.CPU94.RES:Rescheduling_interrupts
    755.67 ± 16%    +807.7%       6859 ± 17%  interrupts.CPU95.CAL:Function_call_interrupts
    450.50 ±121%    +816.7%       4129 ± 13%  interrupts.CPU95.RES:Rescheduling_interrupts
     33556 ± 32%   +1173.3%     427283 ±  6%  interrupts.RES:Rescheduling_interrupts
     31.20           -17.8       13.41 ± 14%  perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
     31.04           -17.7       13.30 ± 14%  perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
     28.44           -17.1       11.34 ± 16%  perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg
     31.02           -15.8       15.19 ± 10%  perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb
     26.09           -15.7       10.43 ± 16%  perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked
     30.64           -15.6       15.03 ± 10%  perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit
     30.16           -15.4       14.79 ± 10%  perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output
     28.56           -14.5       14.09 ± 10%  perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2
     27.75           -14.0       13.74 ± 10%  perf-profile.calltrace.cycles-pp.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip
     27.59           -13.9       13.66 ± 10%  perf-profile.calltrace.cycles-pp.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq
     24.97           -13.9       11.10 ± 13%  perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames
     26.56           -13.6       12.98 ± 10%  perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start
     23.82           -13.2       10.64 ± 13%  perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit
     25.11           -12.9       12.25 ± 10%  perf-profile.calltrace.cycles-pp.ip_rcv.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action
     23.83           -12.2       11.59 ± 10%  perf-profile.calltrace.cycles-pp.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog.__napi_poll
     23.69           -12.2       11.51 ± 10%  perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog
     23.55           -12.1       11.44 ± 10%  perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core
     23.00           -11.8       11.20 ± 10%  perf-profile.calltrace.cycles-pp.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv
     18.52           -10.7        7.83 ± 12%  perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish
     18.87           -10.0        8.90 ± 10%  perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver
     16.78            -9.2        7.56 ± 15%  perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
     15.34            -8.2        7.16 ± 14%  perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg
     13.14            -7.7        5.41 ± 16%  perf-profile.calltrace.cycles-pp.sk_wait_data.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
     14.29            -7.5        6.79 ± 14%  perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg_locked.tcp_recvmsg
     11.53            -6.9        4.61 ± 17%  perf-profile.calltrace.cycles-pp.wait_woken.sk_wait_data.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg
     11.13            -6.9        4.23 ± 12%  perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu
     10.91            -6.9        4.04 ± 14%  perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv
     13.24            -6.8        6.42 ± 13%  perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg_locked
     11.10            -6.7        4.40 ± 17%  perf-profile.calltrace.cycles-pp.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg_locked.tcp_recvmsg
     10.53            -6.6        3.89 ± 14%  perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv
     10.89            -6.6        4.30 ± 17%  perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg_locked
     10.21            -6.5        3.68 ± 17%  perf-profile.calltrace.cycles-pp.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_rcv_established
     10.61            -6.4        4.16 ± 17%  perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.wait_woken.sk_wait_data
      8.48            -4.6        3.93 ± 15%  perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
      8.06            -4.5        3.57 ± 16%  perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
      5.41            -3.5        1.91 ± 22%  perf-profile.calltrace.cycles-pp.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
     37.10            -3.5       33.65        perf-profile.calltrace.cycles-pp.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
      5.18            -3.4        1.82 ± 22%  perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
      5.21            -3.1        2.15 ± 17%  perf-profile.calltrace.cycles-pp.__dev_queue_xmit.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb
      4.79            -2.9        1.88 ± 17%  perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.schedule_timeout.wait_woken
      4.52            -2.8        1.74 ± 19%  perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable
      4.46            -2.8        1.71 ± 19%  perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.__wake_up_common.__wake_up_common_lock
     39.81            -2.7       37.16        perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
     39.67            -2.6       37.06        perf-profile.calltrace.cycles-pp.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
      3.87            -2.4        1.47 ± 21%  perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu
     38.48            -2.4       36.11        perf-profile.calltrace.cycles-pp.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
     38.25            -2.3       35.96        perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64
      2.72            -2.0        0.69 ± 15%  perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable
      3.15            -1.9        1.25 ± 27%  perf-profile.calltrace.cycles-pp.dev_hard_start_xmit.__dev_queue_xmit.ip_finish_output2.ip_output.__ip_queue_xmit
      2.85            -1.8        1.04 ± 34%  perf-profile.calltrace.cycles-pp.loopback_xmit.dev_hard_start_xmit.__dev_queue_xmit.ip_finish_output2.ip_output
      2.38            -1.5        0.90 ± 19%  perf-profile.calltrace.cycles-pp.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv
      2.10            -1.5        0.64 ± 49%  perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode
      2.21            -1.4        0.85 ± 20%  perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.__wake_up_common
      2.14            -1.3        0.87 ± 17%  perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.schedule_timeout
      1.44            -1.0        0.40 ± 72%  perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.schedule_timeout.wait_woken
      1.78            -1.0        0.77 ± 16%  perf-profile.calltrace.cycles-pp.switch_fpu_return.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
      1.34            -0.9        0.40 ± 72%  perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.schedule_timeout.wait_woken
      1.37            -0.9        0.49 ± 46%  perf-profile.calltrace.cycles-pp.__tcp_send_ack.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
      1.24            -0.9        0.37 ± 72%  perf-profile.calltrace.cycles-pp.__alloc_skb.__tcp_send_ack.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg
      1.96            -0.7        1.31 ±  5%  perf-profile.calltrace.cycles-pp.sk_stream_alloc_skb.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
      1.71            -0.6        1.08 ±  7%  perf-profile.calltrace.cycles-pp.__alloc_skb.sk_stream_alloc_skb.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
      1.76            -0.3        1.49 ±  2%  perf-profile.calltrace.cycles-pp.do_vfs_ioctl.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.57            -0.2        1.33 ±  2%  perf-profile.calltrace.cycles-pp.sock_ioctl.do_vfs_ioctl.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.28            -0.2        1.07 ±  2%  perf-profile.calltrace.cycles-pp.inet_ioctl.sock_do_ioctl.sock_ioctl.do_vfs_ioctl.__x64_sys_ioctl
      1.35            -0.2        1.14 ±  2%  perf-profile.calltrace.cycles-pp.sock_do_ioctl.sock_ioctl.do_vfs_ioctl.__x64_sys_ioctl.do_syscall_64
      2.48            -0.2        2.30        perf-profile.calltrace.cycles-pp.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.57            -0.1        0.43 ± 44%  perf-profile.calltrace.cycles-pp.sock_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.99            -0.1        0.85 ±  2%  perf-profile.calltrace.cycles-pp.tcp_ioctl.inet_ioctl.sock_do_ioctl.sock_ioctl.do_vfs_ioctl
      0.72            -0.1        0.64 ±  2%  perf-profile.calltrace.cycles-pp.tcp_rcv_space_adjust.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
      0.52            +0.1        0.58 ±  2%  perf-profile.calltrace.cycles-pp.tcp_send_mss.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
     95.64            +0.6       96.22        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.00            +0.6        0.58 ±  4%  perf-profile.calltrace.cycles-pp.__check_object_size.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked
      0.00            +0.6        0.59 ±  4%  perf-profile.calltrace.cycles-pp.lock_sock_nested.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto
      0.00            +0.6        0.61 ±  6%  perf-profile.calltrace.cycles-pp.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect.do_syscall_64
      0.00            +0.6        0.61 ±  6%  perf-profile.calltrace.cycles-pp.inet_stream_connect.__sys_connect.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +0.6        0.62 ±  6%  perf-profile.calltrace.cycles-pp.__sys_connect.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +0.6        0.62 ±  6%  perf-profile.calltrace.cycles-pp.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +0.6        0.62 ±  2%  perf-profile.calltrace.cycles-pp.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg
      0.00            +0.9        0.94 ± 11%  perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
      0.00            +0.9        0.94 ± 11%  perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
      0.00            +0.9        0.95 ± 11%  perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
      0.00            +0.9        0.95 ± 10%  perf-profile.calltrace.cycles-pp.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish
      0.00            +1.0        0.96 ± 11%  perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
      0.00            +1.0        0.97 ± 13%  perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_recvmsg.inet_recvmsg
      0.00            +1.1        1.10 ± 11%  perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established
      0.00            +1.2        1.18 ± 11%  perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv
      0.00            +1.3        1.28 ± 13%  perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock.tcp_recvmsg
      0.00            +1.3        1.29 ± 11%  perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv.__release_sock
      0.00            +1.3        1.29 ± 11%  perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock
      0.00            +1.4        1.36 ± 13%  perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
      0.59 ±  2%      +1.5        2.06 ± 10%  perf-profile.calltrace.cycles-pp.__kfree_skb.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv
      0.00            +1.6        1.64 ± 10%  perf-profile.calltrace.cycles-pp.release_sock.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
      0.00            +1.8        1.83 ± 10%  perf-profile.calltrace.cycles-pp.free_one_page.__free_pages_ok.skb_release_data.__kfree_skb.tcp_clean_rtx_queue
      0.00            +1.9        1.87 ± 18%  perf-profile.calltrace.cycles-pp.__sk_mem_raise_allocated.__sk_mem_schedule.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
      0.00            +1.9        1.88 ± 10%  perf-profile.calltrace.cycles-pp.__free_pages_ok.skb_release_data.__kfree_skb.tcp_clean_rtx_queue.tcp_ack
      0.00            +1.9        1.89 ± 18%  perf-profile.calltrace.cycles-pp.__sk_mem_schedule.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
      0.00            +2.0        2.03 ± 10%  perf-profile.calltrace.cycles-pp.skb_release_data.__kfree_skb.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established
      0.00            +2.3        2.32 ± 10%  perf-profile.calltrace.cycles-pp.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock
      0.00            +2.5        2.53 ±  9%  perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock
      0.56 ± 10%      +3.0        3.53 ±  9%  perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg
      0.45 ± 46%      +3.0        3.50 ±  9%  perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg
      1.90            +3.1        5.01 ±  7%  perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
      2.31            +3.1        5.45 ±  6%  perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg
      1.93            +3.2        5.09 ±  7%  perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked
      3.11            +3.4        6.55 ±  6%  perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg
      1.10 ±  7%      +3.5        4.55 ±  8%  perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto
      3.16            +3.5        6.61 ±  6%  perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
     38.75            +3.5       42.22        perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto
      1.45 ±  6%      +3.5        4.98 ±  8%  perf-profile.calltrace.cycles-pp.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto
      1.92 ±  7%      +4.2        6.16 ±  8%  perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg
      2.02 ±  7%      +4.3        6.29 ±  8%  perf-profile.calltrace.cycles-pp.copyin._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
      2.35 ±  6%      +4.7        7.04 ±  7%  perf-profile.calltrace.cycles-pp._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
     86.42            +5.2       91.67        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
     42.78            +7.1       49.89 ±  2%  perf-profile.calltrace.cycles-pp.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
     42.58            +7.2       49.74 ±  2%  perf-profile.calltrace.cycles-pp.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
     41.63            +7.2       48.88 ±  2%  perf-profile.calltrace.cycles-pp.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
     40.90            +7.3       48.17 ±  2%  perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64
      0.00           +10.3       10.27 ± 11%  perf-profile.calltrace.cycles-pp.free_one_page.__free_pages_ok.skb_release_data.__kfree_skb.tcp_recvmsg_locked
      0.00           +10.5       10.54 ± 11%  perf-profile.calltrace.cycles-pp.__free_pages_ok.skb_release_data.__kfree_skb.tcp_recvmsg_locked.tcp_recvmsg
      0.59 ±  2%     +10.7       11.29 ± 10%  perf-profile.calltrace.cycles-pp.__kfree_skb.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
      0.00           +11.0       10.95 ± 11%  perf-profile.calltrace.cycles-pp.skb_release_data.__kfree_skb.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg
      0.00           +11.6       11.63 ± 11%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_one_page.__free_pages_ok.skb_release_data
      0.00           +11.8       11.81 ± 11%  perf-profile.calltrace.cycles-pp._raw_spin_lock.free_one_page.__free_pages_ok.skb_release_data.__kfree_skb
      0.00           +11.8       11.83 ± 11%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.rmqueue.get_page_from_freelist.__alloc_pages_nodemask
      0.00           +12.0       11.99 ± 11%  perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.rmqueue.get_page_from_freelist.__alloc_pages_nodemask.skb_page_frag_refill
      0.00           +12.7       12.72 ± 11%  perf-profile.calltrace.cycles-pp.rmqueue.get_page_from_freelist.__alloc_pages_nodemask.skb_page_frag_refill.sk_page_frag_refill
      0.00           +13.0       13.01 ± 11%  perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.skb_page_frag_refill.sk_page_frag_refill.tcp_sendmsg_locked
      0.00           +13.1       13.08 ± 11%  perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.skb_page_frag_refill.sk_page_frag_refill.tcp_sendmsg_locked.tcp_sendmsg
      0.00           +13.4       13.37 ± 11%  perf-profile.calltrace.cycles-pp.skb_page_frag_refill.sk_page_frag_refill.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
      0.00           +13.4       13.43 ± 11%  perf-profile.calltrace.cycles-pp.sk_page_frag_refill.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
     46.23           -23.5       22.75 ± 11%  perf-profile.children.cycles-pp.__tcp_transmit_skb
     42.30           -21.3       20.97 ± 11%  perf-profile.children.cycles-pp.__ip_queue_xmit
     40.05           -20.0       20.02 ± 11%  perf-profile.children.cycles-pp.ip_output
     37.76           -18.8       19.00 ± 11%  perf-profile.children.cycles-pp.ip_finish_output2
     31.55           -15.7       15.88 ± 10%  perf-profile.children.cycles-pp.__tcp_push_pending_frames
     31.90           -15.4       16.46 ± 10%  perf-profile.children.cycles-pp.__local_bh_enable_ip
     31.52           -15.3       16.20 ± 10%  perf-profile.children.cycles-pp.tcp_write_xmit
     31.13           -15.1       16.00 ± 10%  perf-profile.children.cycles-pp.do_softirq
     30.87           -14.6       16.22 ±  9%  perf-profile.children.cycles-pp.__softirqentry_text_start
     29.02           -14.0       14.99 ± 10%  perf-profile.children.cycles-pp.net_rx_action
     28.19           -13.6       14.62 ± 10%  perf-profile.children.cycles-pp.__napi_poll
     28.04           -13.5       14.54 ± 10%  perf-profile.children.cycles-pp.process_backlog
     27.00           -13.0       14.01 ± 10%  perf-profile.children.cycles-pp.__netif_receive_skb_one_core
     25.54           -12.2       13.33 ±  9%  perf-profile.children.cycles-pp.ip_rcv
     24.23           -11.5       12.71 ±  9%  perf-profile.children.cycles-pp.ip_local_deliver
     24.09           -11.5       12.62 ±  9%  perf-profile.children.cycles-pp.ip_local_deliver_finish
     23.95           -11.4       12.55 ±  9%  perf-profile.children.cycles-pp.ip_protocol_deliver_rcu
     23.42           -11.1       12.31 ±  9%  perf-profile.children.cycles-pp.tcp_v4_rcv
     16.32           -10.0        6.28 ± 18%  perf-profile.children.cycles-pp.schedule
     16.37            -9.7        6.65 ± 17%  perf-profile.children.cycles-pp.__schedule
     13.16            -7.7        5.41 ± 16%  perf-profile.children.cycles-pp.sk_wait_data
     11.53            -6.9        4.61 ± 17%  perf-profile.children.cycles-pp.wait_woken
     11.11            -6.6        4.46 ± 16%  perf-profile.children.cycles-pp.schedule_timeout
     11.24            -6.5        4.74 ± 13%  perf-profile.children.cycles-pp.sock_def_readable
     11.05            -6.4        4.62 ± 13%  perf-profile.children.cycles-pp.__wake_up_common_lock
     19.26            -6.3       12.99 ±  4%  perf-profile.children.cycles-pp.tcp_rcv_established
     10.65            -6.2        4.44 ± 13%  perf-profile.children.cycles-pp.__wake_up_common
     10.35            -6.0        4.30 ± 13%  perf-profile.children.cycles-pp.try_to_wake_up
     19.83            -5.2       14.65 ±  3%  perf-profile.children.cycles-pp.tcp_v4_do_rcv
      8.52            -4.5        3.97 ± 14%  perf-profile.children.cycles-pp.syscall_exit_to_user_mode
      8.11            -4.5        3.60 ± 16%  perf-profile.children.cycles-pp.exit_to_user_mode_prepare
     37.16            -3.5       33.68        perf-profile.children.cycles-pp.tcp_recvmsg_locked
      4.80            -2.9        1.90 ± 17%  perf-profile.children.cycles-pp.dequeue_task_fair
     39.84            -2.7       37.19        perf-profile.children.cycles-pp.__x64_sys_recvfrom
      5.42            -2.6        2.81 ± 11%  perf-profile.children.cycles-pp.__dev_queue_xmit
     39.68            -2.6       37.07        perf-profile.children.cycles-pp.__sys_recvfrom
      4.58            -2.6        1.98 ± 16%  perf-profile.children.cycles-pp.ttwu_do_activate
      4.53            -2.6        1.95 ± 16%  perf-profile.children.cycles-pp.enqueue_task_fair
     38.49            -2.4       36.11        perf-profile.children.cycles-pp.inet_recvmsg
      3.86            -2.4        1.49 ± 20%  perf-profile.children.cycles-pp.pick_next_task_fair
     38.30            -2.3       35.99        perf-profile.children.cycles-pp.tcp_recvmsg
      3.60            -2.1        1.48 ± 16%  perf-profile.children.cycles-pp.update_load_avg
      3.22            -2.0        1.22 ± 20%  perf-profile.children.cycles-pp.update_curr
      2.75            -1.9        0.80 ± 12%  perf-profile.children.cycles-pp.select_task_rq_fair
      3.28            -1.6        1.67 ± 12%  perf-profile.children.cycles-pp.dev_hard_start_xmit
      2.08            -1.6        0.50 ± 12%  perf-profile.children.cycles-pp.select_idle_cpu
      2.67            -1.6        1.11 ± 17%  perf-profile.children.cycles-pp.switch_mm_irqs_off
      2.98            -1.5        1.50 ± 11%  perf-profile.children.cycles-pp.loopback_xmit
      3.09            -1.3        1.76 ±  8%  perf-profile.children.cycles-pp.__alloc_skb
      2.19            -1.3        0.91 ± 17%  perf-profile.children.cycles-pp.dequeue_entity
      2.28            -1.3        1.00 ± 16%  perf-profile.children.cycles-pp.enqueue_entity
      1.62            -1.1        0.53 ± 20%  perf-profile.children.cycles-pp.reweight_entity
      1.77            -1.0        0.73 ± 18%  perf-profile.children.cycles-pp.load_new_mm_cr3
      1.78            -1.0        0.77 ± 16%  perf-profile.children.cycles-pp.switch_fpu_return
      1.00            -0.9        0.13 ± 10%  perf-profile.children.cycles-pp.available_idle_cpu
      1.64            -0.8        0.81 ± 12%  perf-profile.children.cycles-pp.__cgroup_bpf_run_filter_skb
      1.64 ±  2%      -0.8        0.81 ± 11%  perf-profile.children.cycles-pp.netif_rx
      1.60            -0.8        0.78 ± 11%  perf-profile.children.cycles-pp.netif_rx_internal
      1.43            -0.8        0.64 ± 12%  perf-profile.children.cycles-pp.sk_filter_trim_cap
      1.46            -0.8        0.70 ± 12%  perf-profile.children.cycles-pp.__tcp_send_ack
      1.24            -0.8        0.49 ± 15%  perf-profile.children.cycles-pp.__switch_to_asm
      1.07            -0.7        0.39 ± 19%  perf-profile.children.cycles-pp.ttwu_do_wakeup
      0.94            -0.7        0.28 ± 22%  perf-profile.children.cycles-pp.pick_next_entity
      1.98            -0.6        1.33 ±  5%  perf-profile.children.cycles-pp.sk_stream_alloc_skb
      1.12            -0.6        0.48 ± 17%  perf-profile.children.cycles-pp.set_next_entity
      0.99            -0.6        0.35 ± 20%  perf-profile.children.cycles-pp.check_preempt_curr
      1.14            -0.6        0.51 ± 12%  perf-profile.children.cycles-pp.__netif_receive_skb_core
      1.04            -0.6        0.46 ± 14%  perf-profile.children.cycles-pp.sched_clock_cpu
      0.87            -0.6        0.29 ± 21%  perf-profile.children.cycles-pp.check_preempt_wakeup
      0.90            -0.6        0.33 ± 11%  perf-profile.children.cycles-pp.update_rq_clock
      0.86            -0.6        0.30 ± 19%  perf-profile.children.cycles-pp.copy_fpregs_to_fpstate
      0.96            -0.6        0.41 ± 15%  perf-profile.children.cycles-pp.__switch_to
      0.88            -0.5        0.34 ± 18%  perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
      0.86            -0.5        0.32 ± 22%  perf-profile.children.cycles-pp.put_prev_entity
      1.03            -0.5        0.52 ± 10%  perf-profile.children.cycles-pp.kmalloc_reserve
      0.92            -0.5        0.41 ± 13%  perf-profile.children.cycles-pp.sched_clock
      0.93            -0.5        0.42 ± 12%  perf-profile.children.cycles-pp.irqtime_account_irq
      0.91 ±  2%      -0.5        0.41 ± 17%  perf-profile.children.cycles-pp.validate_xmit_skb
      0.79            -0.5        0.29 ± 15%  perf-profile.children.cycles-pp.__update_load_avg_se
      0.89            -0.5        0.40 ± 14%  perf-profile.children.cycles-pp.native_sched_clock
      0.96            -0.5        0.49 ± 12%  perf-profile.children.cycles-pp.ip_finish_output
      0.88            -0.4        0.46 ± 10%  perf-profile.children.cycles-pp.__kmalloc_node_track_caller
      0.58 ±  3%      -0.4        0.19 ± 26%  perf-profile.children.cycles-pp.set_next_buddy
      0.71            -0.4        0.33 ± 13%  perf-profile.children.cycles-pp.tcp_schedule_loss_probe
      0.74            -0.4        0.36 ± 11%  perf-profile.children.cycles-pp.enqueue_to_backlog
      0.63 ±  2%      -0.4        0.26 ± 13%  perf-profile.children.cycles-pp.finish_task_switch
      0.80            -0.4        0.44 ±  9%  perf-profile.children.cycles-pp.kmem_cache_alloc_node
      0.92            -0.3        0.59 ±  7%  perf-profile.children.cycles-pp.__inet_lookup_established
      0.64 ±  2%      -0.3        0.33 ± 12%  perf-profile.children.cycles-pp.ktime_get_with_offset
      0.59            -0.3        0.28 ± 15%  perf-profile.children.cycles-pp.ip_rcv_finish
      0.61            -0.3        0.30 ± 15%  perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
      0.94 ±  5%      -0.3        0.64 ±  7%  perf-profile.children.cycles-pp.update_cfs_group
      0.44            -0.3        0.14 ± 23%  perf-profile.children.cycles-pp.clear_buddies
      0.53            -0.3        0.24 ± 15%  perf-profile.children.cycles-pp.__sk_dst_check
      0.93 ±  2%      -0.3        0.65 ±  5%  perf-profile.children.cycles-pp.tcp_mstamp_refresh
      0.63 ±  2%      -0.3        0.34 ±  8%  perf-profile.children.cycles-pp.__ksize
      0.58            -0.3        0.29 ± 14%  perf-profile.children.cycles-pp.___perf_sw_event
      0.53 ±  2%      -0.3        0.25 ± 12%  perf-profile.children.cycles-pp.ip_rcv_core
      0.45 ±  4%      -0.3        0.18 ± 16%  perf-profile.children.cycles-pp.ipv4_dst_check
      1.77            -0.3        1.50 ±  2%  perf-profile.children.cycles-pp.do_vfs_ioctl
      0.65            -0.3        0.40 ±  9%  perf-profile.children.cycles-pp.skb_release_all
      0.55 ±  2%      -0.3        0.30 ±  8%  perf-profile.children.cycles-pp.kfree
      0.38 ±  2%      -0.2        0.14 ± 18%  perf-profile.children.cycles-pp.cpumask_next_wrap
      0.39 ±  3%      -0.2        0.15 ± 23%  perf-profile.children.cycles-pp.update_min_vruntime
      0.42            -0.2        0.19 ± 14%  perf-profile.children.cycles-pp.migrate_disable
      0.43 ±  6%      -0.2        0.19 ± 17%  perf-profile.children.cycles-pp.netif_skb_features
      0.60            -0.2        0.37 ±  9%  perf-profile.children.cycles-pp.skb_release_head_state
      0.38            -0.2        0.15 ± 14%  perf-profile.children.cycles-pp.security_sock_rcv_skb
      0.38            -0.2        0.16 ± 13%  perf-profile.children.cycles-pp.tcp_ack_update_rtt
      0.39 ±  2%      -0.2        0.17 ± 15%  perf-profile.children.cycles-pp.__calc_delta
      1.39 ±  3%      -0.2        1.17        perf-profile.children.cycles-pp.ktime_get
      1.29            -0.2        1.08 ±  2%  perf-profile.children.cycles-pp.inet_ioctl
      0.38            -0.2        0.18 ± 19%  perf-profile.children.cycles-pp.migrate_enable
      0.52            -0.2        0.32 ± 12%  perf-profile.children.cycles-pp.ip_local_out
      0.35 ±  7%      -0.2        0.16 ± 19%  perf-profile.children.cycles-pp.cpuacct_charge
      0.46            -0.2        0.27 ± 11%  perf-profile.children.cycles-pp.__ip_local_out
      0.37 ±  3%      -0.2        0.18 ± 15%  perf-profile.children.cycles-pp.nf_hook_slow
      0.38            -0.2        0.19 ± 15%  perf-profile.children.cycles-pp.__ip_finish_output
      1.01            -0.2        0.82 ±  3%  perf-profile.children.cycles-pp.read_tsc
      0.33            -0.2        0.14 ± 16%  perf-profile.children.cycles-pp.skb_entail
      2.49            -0.2        2.31        perf-profile.children.cycles-pp.__x64_sys_ioctl
      0.27 ±  3%      -0.2        0.09 ± 18%  perf-profile.children.cycles-pp.resched_curr
      1.39            -0.2        1.21        perf-profile.children.cycles-pp.sock_do_ioctl
      0.29            -0.2        0.11 ± 17%  perf-profile.children.cycles-pp.__wrgsbase_inactive
      0.53 ±  3%      -0.2        0.36 ±  6%  perf-profile.children.cycles-pp.tcp_event_new_data_sent
      0.79            -0.2        0.62 ±  6%  perf-profile.children.cycles-pp.__cond_resched
      0.32 ±  2%      -0.2        0.16 ± 15%  perf-profile.children.cycles-pp.perf_tp_event
      1.04            -0.2        0.88 ±  2%  perf-profile.children.cycles-pp.tcp_ioctl
      0.33 ±  2%      -0.2        0.17 ± 12%  perf-profile.children.cycles-pp.ip_rcv_finish_core
      0.22 ±  4%      -0.2        0.07 ± 10%  perf-profile.children.cycles-pp.tcp_v4_inbound_md5_hash
      0.24 ±  3%      -0.2        0.08 ± 17%  perf-profile.children.cycles-pp.__usecs_to_jiffies
      0.28 ±  3%      -0.2        0.12 ± 15%  perf-profile.children.cycles-pp.add_wait_queue
      0.47            -0.1        0.32 ±  8%  perf-profile.children.cycles-pp.preempt_schedule_common
      0.25 ±  2%      -0.1        0.10 ±  8%  perf-profile.children.cycles-pp.tcp_event_data_recv
      0.25 ±  9%      -0.1        0.10 ± 22%  perf-profile.children.cycles-pp.eth_type_trans
      0.63            -0.1        0.49 ±  3%  perf-profile.children.cycles-pp.__might_sleep
      0.43            -0.1        0.29 ±  9%  perf-profile.children.cycles-pp.__skb_clone
      0.29 ±  3%      -0.1        0.15 ± 12%  perf-profile.children.cycles-pp.inet_ehashfn
      0.25 ±  2%      -0.1        0.12 ± 14%  perf-profile.children.cycles-pp.__tcp_select_window
      0.22 ±  5%      -0.1        0.09 ± 16%  perf-profile.children.cycles-pp.skb_network_protocol
      0.22 ±  4%      -0.1        0.10 ± 11%  perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
      0.39 ±  4%      -0.1        0.27 ±  7%  perf-profile.children.cycles-pp.sk_reset_timer
      0.22 ±  2%      -0.1        0.10 ± 15%  perf-profile.children.cycles-pp.tcp_options_write
      0.29            -0.1        0.17 ±  9%  perf-profile.children.cycles-pp.tcp_wfree
      0.23            -0.1        0.11 ± 13%  perf-profile.children.cycles-pp.tcp_v4_fill_cb
      0.20 ±  2%      -0.1        0.09 ± 14%  perf-profile.children.cycles-pp.raw_local_deliver
      0.19 ±  2%      -0.1        0.08 ± 22%  perf-profile.children.cycles-pp._find_next_bit
      0.19            -0.1        0.08 ± 21%  perf-profile.children.cycles-pp.bpf_skops_write_hdr_opt
      0.15 ±  2%      -0.1        0.05 ± 45%  perf-profile.children.cycles-pp.tcp_grow_window
      0.19 ± 14%      -0.1        0.08 ± 25%  perf-profile.children.cycles-pp.perf_trace_sched_switch
      0.19 ±  4%      -0.1        0.09 ± 13%  perf-profile.children.cycles-pp.bictcp_acked
      0.21            -0.1        0.11 ± 13%  perf-profile.children.cycles-pp.tcp_update_pacing_rate
      0.24            -0.1        0.14 ± 11%  perf-profile.children.cycles-pp.remove_wait_queue
      0.15 ± 23%      -0.1        0.05 ± 46%  perf-profile.children.cycles-pp.__cgroup_account_cputime
      0.19 ±  3%      -0.1        0.09 ± 12%  perf-profile.children.cycles-pp.tcp_update_skb_after_send
      0.16 ±  4%      -0.1        0.06 ± 14%  perf-profile.children.cycles-pp.__rdgsbase_inactive
      0.17 ±  3%      -0.1        0.07 ± 12%  perf-profile.children.cycles-pp.bictcp_cwnd_event
      0.18 ±  2%      -0.1        0.09 ± 14%  perf-profile.children.cycles-pp.rb_next
      0.30 ±  2%      -0.1        0.21 ±  5%  perf-profile.children.cycles-pp.tcp_established_options
      0.25            -0.1        0.16 ±  9%  perf-profile.children.cycles-pp.ip_send_check
      0.16 ±  4%      -0.1        0.07 ± 18%  perf-profile.children.cycles-pp.tcp_rate_gen
      0.17 ±  2%      -0.1        0.08 ± 16%  perf-profile.children.cycles-pp.woken_wake_function
      0.15 ±  3%      -0.1        0.07 ± 18%  perf-profile.children.cycles-pp.__enqueue_entity
      0.14 ±  3%      -0.1        0.06 ± 46%  perf-profile.children.cycles-pp.perf_trace_buf_alloc
      0.17 ±  4%      -0.1        0.09 ± 20%  perf-profile.children.cycles-pp.ksoftirqd_running
      0.21 ±  8%      -0.1        0.13 ±  7%  perf-profile.children.cycles-pp.clockevents_program_event
      0.14 ±  3%      -0.1        0.07 ± 14%  perf-profile.children.cycles-pp.apparmor_socket_sock_rcv_skb
      0.13 ± 12%      -0.1        0.05 ± 48%  perf-profile.children.cycles-pp.switch_ldt
      0.15 ±  3%      -0.1        0.08 ± 16%  perf-profile.children.cycles-pp.tcp_rearm_rto
      0.72            -0.1        0.64        perf-profile.children.cycles-pp.tcp_rcv_space_adjust
      0.27 ±  3%      -0.1        0.20 ±  3%  perf-profile.children.cycles-pp.import_single_range
      0.17 ±  4%      -0.1        0.09 ± 13%  perf-profile.children.cycles-pp.__sock_wfree
      0.31 ±  6%      -0.1        0.24 ±  6%  perf-profile.children.cycles-pp.mod_timer
      0.34 ±  2%      -0.1        0.27 ±  4%  perf-profile.children.cycles-pp.dst_release
      0.16 ±  4%      -0.1        0.09 ± 11%  perf-profile.children.cycles-pp.rcu_read_unlock_strict
      0.12 ±  3%      -0.1        0.05 ± 46%  perf-profile.children.cycles-pp.perf_trace_buf_update
      0.12 ±  3%      -0.1        0.05 ± 49%  perf-profile.children.cycles-pp.skb_csum_hwoffload_help
      0.15 ±  7%      -0.1        0.08 ± 16%  perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
      0.17 ±  3%      -0.1        0.09 ± 20%  perf-profile.children.cycles-pp.__copy_skb_header
      0.22 ±  2%      -0.1        0.15 ±  4%  perf-profile.children.cycles-pp.sock_put
      0.14 ±  4%      -0.1        0.07 ± 12%  perf-profile.children.cycles-pp._raw_spin_unlock_bh
      0.58            -0.1        0.52 ±  2%  perf-profile.children.cycles-pp.sock_recvmsg
      0.15 ±  4%      -0.1        0.09 ± 20%  perf-profile.children.cycles-pp.kmalloc_slab
      0.11 ±  5%      -0.1        0.05 ± 48%  perf-profile.children.cycles-pp.check_cfs_rq_runtime
      0.12 ±  3%      -0.1        0.06 ± 17%  perf-profile.children.cycles-pp.tcp_send_delayed_ack
      1.76            -0.1        1.70        perf-profile.children.cycles-pp.sock_ioctl
      0.09 ±  4%      -0.1        0.04 ± 71%  perf-profile.children.cycles-pp.skb_clone_tx_timestamp
      0.12 ±  4%      -0.1        0.06 ± 14%  perf-profile.children.cycles-pp.update_irq_load_avg
      0.18 ±  2%      -0.1        0.12 ± 11%  perf-profile.children.cycles-pp.tcp_recv_timestamp
      0.10 ±  4%      -0.1        0.04 ± 45%  perf-profile.children.cycles-pp.apparmor_ipv4_postroute
      0.24 ±  2%      -0.1        0.19 ±  5%  perf-profile.children.cycles-pp.__build_skb_around
      0.23 ±  2%      -0.1        0.18 ±  7%  perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
      0.54            -0.1        0.49 ±  2%  perf-profile.children.cycles-pp.__might_fault
      0.15 ±  3%      -0.1        0.10 ± 15%  perf-profile.children.cycles-pp.rb_insert_color
      0.18 ±  2%      -0.1        0.13 ±  3%  perf-profile.children.cycles-pp.kfree_skbmem
      0.11 ±  4%      -0.1        0.06 ± 14%  perf-profile.children.cycles-pp.validate_xmit_xfrm
      0.10            -0.1        0.05 ± 46%  perf-profile.children.cycles-pp.tcp_chrono_stop
      0.12 ±  3%      -0.1        0.07 ± 21%  perf-profile.children.cycles-pp.bictcp_cong_avoid
      0.11 ±  3%      -0.1        0.06 ±  6%  perf-profile.children.cycles-pp.smpboot_thread_fn
      0.11 ±  4%      -0.1        0.06 ±  8%  perf-profile.children.cycles-pp.__fdget
      0.12 ±  5%      -0.0        0.07 ± 10%  perf-profile.children.cycles-pp.ret_from_fork
      0.12 ±  5%      -0.0        0.07 ± 10%  perf-profile.children.cycles-pp.kthread
      1.24            -0.0        1.19        perf-profile.children.cycles-pp.lock_sock_nested
      0.52            -0.0        0.47 ±  2%  perf-profile.children.cycles-pp.security_socket_recvmsg
      0.12 ±  3%      -0.0        0.08 ±  6%  perf-profile.children.cycles-pp.tcp_rate_check_app_limited
      0.11 ±  4%      -0.0        0.07 ± 10%  perf-profile.children.cycles-pp.tcp_check_space
      0.07 ±  8%      -0.0        0.03 ± 99%  perf-profile.children.cycles-pp.inet_send_prepare
      0.26            -0.0        0.22 ±  4%  perf-profile.children.cycles-pp.rcu_nocb_flush_deferred_wakeup
      0.08 ±  4%      -0.0        0.04 ± 71%  perf-profile.children.cycles-pp.place_entity
      0.20 ±  3%      -0.0        0.16 ±  5%  perf-profile.children.cycles-pp.tcp_release_cb
      0.15 ±  5%      -0.0        0.10 ±  9%  perf-profile.children.cycles-pp.inet_sendmsg
      0.13 ±  2%      -0.0        0.09 ± 10%  perf-profile.children.cycles-pp.rb_erase
      0.07 ±  6%      -0.0        0.03 ± 70%  perf-profile.children.cycles-pp.sk_free
      0.09 ±  4%      -0.0        0.05 ±  7%  perf-profile.children.cycles-pp.run_ksoftirqd
      0.08 ±  5%      -0.0        0.04 ± 45%  perf-profile.children.cycles-pp.ip_copy_addrs
      0.12 ±  4%      -0.0        0.09 ±  8%  perf-profile.children.cycles-pp.sock_rfree
      0.08 ±  5%      -0.0        0.05 ±  8%  perf-profile.children.cycles-pp.sk_forced_mem_schedule
      0.78            -0.0        0.75        perf-profile.children.cycles-pp.__fget_light
      0.11 ±  8%      -0.0        0.09 ±  7%  perf-profile.children.cycles-pp.__tcp_ack_snd_check
      0.10 ±  4%      -0.0        0.08 ±  5%  perf-profile.children.cycles-pp.skb_clone
      0.08 ±  6%      -0.0        0.06 ±  7%  perf-profile.children.cycles-pp.tcp_fin
      0.12 ±  3%      +0.0        0.14 ±  3%  perf-profile.children.cycles-pp.tcp_close
      0.16 ±  2%      +0.0        0.18 ±  3%  perf-profile.children.cycles-pp.__list_add_valid
      0.11 ±  3%      +0.0        0.14 ±  5%  perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
      0.11 ±  3%      +0.0        0.14 ±  4%  perf-profile.children.cycles-pp.syscall_enter_from_user_mode
      0.14 ±  3%      +0.0        0.18 ±  2%  perf-profile.children.cycles-pp.sock_close
      0.14 ±  4%      +0.0        0.18 ±  2%  perf-profile.children.cycles-pp.__sock_release
      0.07 ±  7%      +0.0        0.10 ±  9%  perf-profile.children.cycles-pp.drain_obj_stock
      0.14 ±  3%      +0.0        0.18 ±  2%  perf-profile.children.cycles-pp.inet_release
      0.05 ±  8%      +0.0        0.10 ±  5%  perf-profile.children.cycles-pp.ksys_read
      0.21 ±  3%      +0.1        0.27 ±  4%  perf-profile.children.cycles-pp.tcp_queue_rcv
      0.08 ±  4%      +0.1        0.13 ±  7%  perf-profile.children.cycles-pp.__sys_getsockopt
      0.01 ±223%      +0.1        0.06 ± 13%  perf-profile.children.cycles-pp.ktime_get_update_offsets_now
      0.00            +0.1        0.05 ±  7%  perf-profile.children.cycles-pp.ip_route_output_flow
      0.04 ± 45%      +0.1        0.10 ±  4%  perf-profile.children.cycles-pp.vfs_read
      0.34 ±  2%      +0.1        0.39 ±  3%  perf-profile.children.cycles-pp.__list_del_entry_valid
      0.00            +0.1        0.05 ±  8%  perf-profile.children.cycles-pp.tcp_validate_incoming
      0.08 ±  5%      +0.1        0.14 ±  6%  perf-profile.children.cycles-pp.__x64_sys_getsockopt
      0.00            +0.1        0.05 ±  9%  perf-profile.children.cycles-pp.inet_create
      0.03 ± 99%      +0.1        0.08 ±  4%  perf-profile.children.cycles-pp.tcp_conn_request
      0.05            +0.1        0.11 ±  8%  perf-profile.children.cycles-pp.__alloc_file
      0.05            +0.1        0.11 ±  6%  perf-profile.children.cycles-pp.alloc_empty_file
      0.03 ± 70%      +0.1        0.09 ± 10%  perf-profile.children.cycles-pp.tcp_check_req
      0.00            +0.1        0.06 ±  9%  perf-profile.children.cycles-pp.do_open
      0.00            +0.1        0.06 ±  9%  perf-profile.children.cycles-pp.walk_component
      0.09 ±  5%      +0.1        0.15 ±  5%  perf-profile.children.cycles-pp.__sys_setsockopt
      0.00            +0.1        0.06 ±  6%  perf-profile.children.cycles-pp.__sk_destruct
      0.09 ±  5%      +0.1        0.15 ±  7%  perf-profile.children.cycles-pp.__x64_sys_setsockopt
      0.02 ±141%      +0.1        0.08 ±  7%  perf-profile.children.cycles-pp.seq_read
      0.00            +0.1        0.06 ±  7%  perf-profile.children.cycles-pp.alloc_file_pseudo
      0.00            +0.1        0.06 ±  7%  perf-profile.children.cycles-pp.ip_route_output_key_hash_rcu
      0.03 ± 70%      +0.1        0.10 ±  3%  perf-profile.children.cycles-pp.__dentry_kill
      0.00            +0.1        0.07 ±  7%  perf-profile.children.cycles-pp.sock_alloc_file
      0.00            +0.1        0.07 ±  7%  perf-profile.children.cycles-pp.ip_route_output_key_hash
      0.00            +0.1        0.07 ± 13%  perf-profile.children.cycles-pp.autoremove_wake_function
      0.00            +0.1        0.07 ± 10%  perf-profile.children.cycles-pp.lookup_fast
      0.53            +0.1        0.60 ±  2%  perf-profile.children.cycles-pp.tcp_send_mss
      0.01 ±223%      +0.1        0.08 ±  4%  perf-profile.children.cycles-pp.seq_read_iter
      0.00            +0.1        0.07 ± 12%  perf-profile.children.cycles-pp.do_dentry_open
      0.43 ±  2%      +0.1        0.51 ±  2%  perf-profile.children.cycles-pp.tcp_current_mss
      0.06 ±  6%      +0.1        0.13 ±  9%  perf-profile.children.cycles-pp.__sys_socket
      0.24 ±  6%      +0.1        0.32 ±  7%  perf-profile.children.cycles-pp.update_process_times
      0.06 ±  6%      +0.1        0.13 ±  7%  perf-profile.children.cycles-pp.__x64_sys_socket
      0.00            +0.1        0.08 ± 10%  perf-profile.children.cycles-pp.do_tcp_setsockopt
      0.72 ±  2%      +0.1        0.80 ±  2%  perf-profile.children.cycles-pp.aa_sk_perm
      0.10 ±  6%      +0.1        0.18 ±  9%  perf-profile.children.cycles-pp.refill_obj_stock
      0.00            +0.1        0.08 ±  8%  perf-profile.children.cycles-pp.link_path_walk
      0.00            +0.1        0.08 ± 13%  perf-profile.children.cycles-pp.set_task_cpu
      0.24 ±  6%      +0.1        0.32 ±  6%  perf-profile.children.cycles-pp.tick_sched_handle
      0.14 ±  6%      +0.1        0.22 ±  9%  perf-profile.children.cycles-pp.scheduler_tick
      0.10 ±  7%      +0.1        0.18 ± 11%  perf-profile.children.cycles-pp.task_tick_fair
      0.00            +0.1        0.08 ±  7%  perf-profile.children.cycles-pp.tcp_v4_syn_recv_sock
      0.28 ±  3%      +0.1        0.36 ±  5%  perf-profile.children.cycles-pp.tick_sched_timer
      0.00            +0.1        0.08 ±  8%  perf-profile.children.cycles-pp.__x64_sys_accept
      0.00            +0.1        0.08 ±  8%  perf-profile.children.cycles-pp.dentry_open
      0.00            +0.1        0.08 ± 11%  perf-profile.children.cycles-pp.inet_csk_accept
      0.00            +0.1        0.08 ± 11%  perf-profile.children.cycles-pp.__x64_sys_accept4
      0.74            +0.1        0.82 ±  2%  perf-profile.children.cycles-pp.kmem_cache_free
      0.00            +0.1        0.09 ± 12%  perf-profile.children.cycles-pp.__zone_watermark_ok
      0.12 ±  3%      +0.1        0.21 ±  5%  perf-profile.children.cycles-pp.tcp_connect
      0.35 ±  3%      +0.1        0.44 ±  4%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.00            +0.1        0.09 ±  7%  perf-profile.children.cycles-pp.__sock_create
      0.00            +0.1        0.09 ±  9%  perf-profile.children.cycles-pp.inet_accept
      0.00            +0.1        0.10 ±  9%  perf-profile.children.cycles-pp.alloc_inode
      0.00            +0.1        0.10 ±  9%  perf-profile.children.cycles-pp.tcp_child_process
      0.01 ±223%      +0.1        0.11 ±  6%  perf-profile.children.cycles-pp.__ns_get_path
      0.37 ±  4%      +0.1        0.47 ±  6%  perf-profile.children.cycles-pp.security_socket_sendmsg
      0.00            +0.1        0.10 ±  9%  perf-profile.children.cycles-pp.new_inode_pseudo
      0.08            +0.1        0.19 ±  7%  perf-profile.children.cycles-pp.do_filp_open
      0.08            +0.1        0.19 ±  7%  perf-profile.children.cycles-pp.path_openat
      0.07 ±  5%      +0.1        0.19 ±  7%  perf-profile.children.cycles-pp.kmem_cache_alloc
      0.10            +0.1        0.21 ±  5%  perf-profile.children.cycles-pp.open_related_ns
      0.00            +0.1        0.12 ± 15%  perf-profile.children.cycles-pp.skb_try_coalesce
      0.09 ±  5%      +0.1        0.21 ±  8%  perf-profile.children.cycles-pp.do_sys_open
      0.09            +0.1        0.21 ±  8%  perf-profile.children.cycles-pp.do_sys_openat2
      0.00            +0.1        0.12 ± 11%  perf-profile.children.cycles-pp.prep_compound_page
      0.17 ±  5%      +0.1        0.29 ±  5%  perf-profile.children.cycles-pp.tcp_v4_connect
      0.00            +0.1        0.13 ± 12%  perf-profile.children.cycles-pp.prep_new_page
      0.00            +0.1        0.14 ± 10%  perf-profile.children.cycles-pp.poll_idle
      0.00            +0.1        0.14 ±  9%  perf-profile.children.cycles-pp.tcp_add_backlog
      0.00            +0.1        0.14 ± 13%  perf-profile.children.cycles-pp.tcp_try_coalesce
      0.26 ±  3%      +0.1        0.40 ±  3%  perf-profile.children.cycles-pp.__fput
      0.00            +0.1        0.14 ± 14%  perf-profile.children.cycles-pp.__slab_free
      0.02 ±141%      +0.1        0.16 ±  7%  perf-profile.children.cycles-pp.__sys_accept4
      0.27 ±  3%      +0.2        0.42 ±  3%  perf-profile.children.cycles-pp.task_work_run
      0.01 ±223%      +0.2        0.16 ±  8%  perf-profile.children.cycles-pp.__sys_accept4_file
      0.17 ±  2%      +0.2        0.33 ±  6%  perf-profile.children.cycles-pp.inet_shutdown
      0.17 ±  2%      +0.2        0.34 ±  5%  perf-profile.children.cycles-pp.__x64_sys_shutdown
      0.17 ±  2%      +0.2        0.34 ±  5%  perf-profile.children.cycles-pp.__sys_shutdown
      0.12 ±  3%      +0.2        0.30 ±  8%  perf-profile.children.cycles-pp.tcp_rcv_synsent_state_process
      0.45 ±  2%      +0.2        0.63 ±  3%  perf-profile.children.cycles-pp.simple_copy_to_iter
      0.79            +0.2        0.99        perf-profile.children.cycles-pp.syscall_return_via_sysret
      4.31            +0.2        4.53        perf-profile.children.cycles-pp.tcp_ack
      0.06 ±  8%      +0.2        0.28 ± 11%  perf-profile.children.cycles-pp.tcp_tx_timestamp
      0.27 ±  3%      +0.2        0.50 ±  7%  perf-profile.children.cycles-pp.rcu_core
      0.24 ±  4%      +0.2        0.48 ±  8%  perf-profile.children.cycles-pp.rcu_do_batch
      0.00            +0.3        0.26 ± 11%  perf-profile.children.cycles-pp.__free_one_page
      0.00            +0.3        0.27 ± 11%  perf-profile.children.cycles-pp.schedule_idle
      1.08            +0.3        1.35 ±  3%  perf-profile.children.cycles-pp._raw_spin_lock_bh
      0.16 ±  5%      +0.3        0.45 ±  9%  perf-profile.children.cycles-pp.irq_exit_rcu
      0.00            +0.3        0.30 ± 19%  perf-profile.children.cycles-pp.tcp_try_rmem_schedule
      0.30 ±  2%      +0.3        0.61 ±  6%  perf-profile.children.cycles-pp.__inet_stream_connect
      0.30 ±  3%      +0.3        0.61 ±  6%  perf-profile.children.cycles-pp.inet_stream_connect
      0.31 ±  2%      +0.3        0.62 ±  6%  perf-profile.children.cycles-pp.__sys_connect
      0.31 ±  2%      +0.3        0.62 ±  6%  perf-profile.children.cycles-pp.__x64_sys_connect
      0.80 ±  3%      +0.3        1.14 ±  5%  perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
      0.76 ±  2%      +0.3        1.10 ±  3%  perf-profile.children.cycles-pp.__check_object_size
      0.86 ±  3%      +0.3        1.21 ±  5%  perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
      0.10 ±  5%      +0.4        0.45 ± 10%  perf-profile.children.cycles-pp.iov_iter_advance
      0.11 ±  6%      +0.4        0.49 ±  8%  perf-profile.children.cycles-pp.tcp_push
      0.01 ±223%      +0.4        0.44 ± 10%  perf-profile.children.cycles-pp.intel_idle
      0.20 ±  4%      +0.6        0.78 ± 11%  perf-profile.children.cycles-pp.tcp_data_queue
     95.69            +0.6       96.27        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.01 ±223%      +0.6        0.60 ± 11%  perf-profile.children.cycles-pp.cpuidle_enter
      0.01 ±223%      +0.6        0.60 ± 11%  perf-profile.children.cycles-pp.cpuidle_enter_state
      0.02 ±223%      +0.9        0.95 ± 11%  perf-profile.children.cycles-pp.start_secondary
      0.02 ±223%      +0.9        0.96 ± 11%  perf-profile.children.cycles-pp.secondary_startup_64_no_verify
      0.02 ±223%      +0.9        0.96 ± 11%  perf-profile.children.cycles-pp.cpu_startup_entry
      0.02 ±223%      +0.9        0.96 ± 11%  perf-profile.children.cycles-pp.do_idle
      2.68            +1.0        3.65 ±  2%  perf-profile.children.cycles-pp.tcp_clean_rtx_queue
      0.26 ±  3%      +1.5        1.78 ±  9%  perf-profile.children.cycles-pp.tcp_rcv_state_process
      0.00            +2.2        2.25 ± 18%  perf-profile.children.cycles-pp.__sk_mem_raise_allocated
      0.00            +2.3        2.27 ± 18%  perf-profile.children.cycles-pp.__sk_mem_schedule
      2.33            +3.1        5.46 ±  6%  perf-profile.children.cycles-pp._copy_to_iter
      1.94            +3.2        5.10 ±  7%  perf-profile.children.cycles-pp.copyout
      3.13            +3.4        6.57 ±  6%  perf-profile.children.cycles-pp.__skb_datagram_iter
      3.16            +3.5        6.61 ±  6%  perf-profile.children.cycles-pp.skb_copy_datagram_iter
     38.80            +3.6       42.39        perf-profile.children.cycles-pp.tcp_sendmsg_locked
      2.04 ±  7%      +4.3        6.31 ±  8%  perf-profile.children.cycles-pp.copyin
      2.37 ±  6%      +4.7        7.12 ±  7%  perf-profile.children.cycles-pp._copy_from_iter_full
      2.11 ±  4%      +5.1        7.21 ±  8%  perf-profile.children.cycles-pp.release_sock
      1.25 ±  8%      +5.1        6.37 ±  9%  perf-profile.children.cycles-pp.__release_sock
     86.46            +5.2       91.71        perf-profile.children.cycles-pp.do_syscall_64
     42.80            +7.1       49.91 ±  2%  perf-profile.children.cycles-pp.__x64_sys_sendto
     42.60            +7.2       49.76 ±  2%  perf-profile.children.cycles-pp.__sys_sendto
     41.65            +7.2       48.89 ±  2%  perf-profile.children.cycles-pp.sock_sendmsg
     40.94            +7.3       48.19 ±  2%  perf-profile.children.cycles-pp.tcp_sendmsg
      3.87 ±  3%      +7.4       11.29 ±  7%  perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      1.89           +11.2       13.07 ± 10%  perf-profile.children.cycles-pp._raw_spin_lock
      0.68 ±  2%     +11.7       12.42 ± 10%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      0.01 ±223%     +12.3       12.31 ± 11%  perf-profile.children.cycles-pp.free_one_page
      1.76           +12.4       14.17 ±  9%  perf-profile.children.cycles-pp.__kfree_skb
      0.01 ±223%     +12.6       12.64 ± 11%  perf-profile.children.cycles-pp.__free_pages_ok
      0.01 ±223%     +12.7       12.74 ± 11%  perf-profile.children.cycles-pp.rmqueue
      0.53 ±  3%     +12.9       13.42 ± 10%  perf-profile.children.cycles-pp.skb_release_data
      0.01 ±223%     +13.0       13.02 ± 11%  perf-profile.children.cycles-pp.get_page_from_freelist
      0.01 ±223%     +13.1       13.09 ± 11%  perf-profile.children.cycles-pp.__alloc_pages_nodemask
      0.23 ± 10%     +13.1       13.37 ± 11%  perf-profile.children.cycles-pp.skb_page_frag_refill
      0.30 ±  8%     +13.2       13.48 ± 11%  perf-profile.children.cycles-pp.sk_page_frag_refill
      0.02 ±223%     +24.1       24.14 ± 11%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      1.78            -1.2        0.61 ± 22%  perf-profile.self.cycles-pp.update_curr
      1.91 ±  2%      -1.1        0.80 ± 16%  perf-profile.self.cycles-pp.update_load_avg
      1.76            -1.0        0.73 ± 18%  perf-profile.self.cycles-pp.load_new_mm_cr3
      1.76            -1.0        0.76 ± 17%  perf-profile.self.cycles-pp.switch_fpu_return
      2.06            -1.0        1.08 ± 10%  perf-profile.self.cycles-pp.__tcp_transmit_skb
      1.72 ±  2%      -1.0        0.75 ± 16%  perf-profile.self.cycles-pp.__schedule
      1.84            -0.9        0.95 ±  4%  perf-profile.self.cycles-pp._raw_spin_lock
      0.98            -0.9        0.13 ± 12%  perf-profile.self.cycles-pp.available_idle_cpu
      1.22            -0.7        0.49 ± 14%  perf-profile.self.cycles-pp.__switch_to_asm
      1.11            -0.6        0.46 ± 12%  perf-profile.self.cycles-pp.__ip_queue_xmit
      1.13            -0.6        0.51 ± 13%  perf-profile.self.cycles-pp.__netif_receive_skb_core
      0.84            -0.5        0.29 ± 19%  perf-profile.self.cycles-pp.copy_fpregs_to_fpstate
      0.84            -0.5        0.32 ± 19%  perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
      0.87            -0.5        0.36 ± 15%  perf-profile.self.cycles-pp.__switch_to
      0.88 ±  2%      -0.5        0.37 ± 16%  perf-profile.self.cycles-pp.switch_mm_irqs_off
      1.03 ±  2%      -0.5        0.52 ± 11%  perf-profile.self.cycles-pp.ip_finish_output2
      0.69            -0.5        0.20 ± 17%  perf-profile.self.cycles-pp.select_idle_cpu
      0.78            -0.5        0.29 ± 14%  perf-profile.self.cycles-pp.__update_load_avg_se
      0.85            -0.5        0.38 ± 14%  perf-profile.self.cycles-pp.native_sched_clock
      0.97            -0.5        0.51 ±  9%  perf-profile.self.cycles-pp.tcp_v4_rcv
      0.70 ±  3%      -0.4        0.28 ± 18%  perf-profile.self.cycles-pp.enqueue_task_fair
      0.85            -0.4        0.43 ±  9%  perf-profile.self.cycles-pp.__cgroup_bpf_run_filter_skb
      0.65            -0.4        0.28 ± 15%  perf-profile.self.cycles-pp.ip_output
      0.80 ±  2%      -0.4        0.42 ± 12%  perf-profile.self.cycles-pp.tcp_clean_rtx_queue
      0.65            -0.4        0.28 ± 21%  perf-profile.self.cycles-pp.pick_next_task_fair
      0.62 ±  2%      -0.4        0.25 ± 18%  perf-profile.self.cycles-pp.enqueue_entity
      0.84            -0.4        0.47 ± 11%  perf-profile.self.cycles-pp.tcp_ack
      0.65            -0.4        0.28 ± 12%  perf-profile.self.cycles-pp.__softirqentry_text_start
      0.54 ±  3%      -0.4        0.18 ± 27%  perf-profile.self.cycles-pp.set_next_buddy
      0.68            -0.4        0.32 ± 15%  perf-profile.self.cycles-pp.net_rx_action
      0.50            -0.3        0.15 ±  5%  perf-profile.self.cycles-pp.update_rq_clock
      0.50 ±  2%      -0.3        0.17 ± 14%  perf-profile.self.cycles-pp.reweight_entity
      0.47 ±  2%      -0.3        0.14 ± 20%  perf-profile.self.cycles-pp.pick_next_entity
      0.93            -0.3        0.61 ±  5%  perf-profile.self.cycles-pp.tcp_recvmsg_locked
      0.89 ±  4%      -0.3        0.58 ±  6%  perf-profile.self.cycles-pp.__dev_queue_xmit
      0.76            -0.3        0.46 ± 10%  perf-profile.self.cycles-pp.__local_bh_enable_ip
      0.54 ±  2%      -0.3        0.23 ± 16%  perf-profile.self.cycles-pp.select_task_rq_fair
      0.62            -0.3        0.32 ± 11%  perf-profile.self.cycles-pp.process_backlog
      0.93 ±  4%      -0.3        0.64 ±  6%  perf-profile.self.cycles-pp.update_cfs_group
      0.55            -0.3        0.28 ±  8%  perf-profile.self.cycles-pp.__kmalloc_node_track_caller
      0.61 ±  2%      -0.3        0.34 ±  9%  perf-profile.self.cycles-pp.__ksize
      0.59            -0.3        0.32 ± 11%  perf-profile.self.cycles-pp.kmem_cache_alloc_node
      0.52 ±  2%      -0.3        0.24 ± 13%  perf-profile.self.cycles-pp.ip_rcv_core
      0.46            -0.3        0.19 ± 15%  perf-profile.self.cycles-pp.tcp_schedule_loss_probe
      0.41 ±  2%      -0.3        0.14 ± 16%  perf-profile.self.cycles-pp.finish_task_switch
      0.44 ±  5%      -0.3        0.17 ± 18%  perf-profile.self.cycles-pp.ipv4_dst_check
      0.53            -0.3        0.28 ±  9%  perf-profile.self.cycles-pp.kfree
      0.38            -0.3        0.12 ± 24%  perf-profile.self.cycles-pp.clear_buddies
      0.52            -0.3        0.26 ± 16%  perf-profile.self.cycles-pp.___perf_sw_event
      0.38 ±  2%      -0.2        0.14 ± 23%  perf-profile.self.cycles-pp.update_min_vruntime
      0.47 ±  2%      -0.2        0.23 ± 11%  perf-profile.self.cycles-pp.loopback_xmit
      0.56            -0.2        0.33 ±  8%  perf-profile.self.cycles-pp.tcp_rcv_established
      0.38            -0.2        0.17 ± 16%  perf-profile.self.cycles-pp.dequeue_task_fair
      0.38 ±  3%      -0.2        0.17 ± 16%  perf-profile.self.cycles-pp.__calc_delta
      0.41            -0.2        0.20 ± 12%  perf-profile.self.cycles-pp.wait_woken
      0.35 ±  3%      -0.2        0.14 ± 24%  perf-profile.self.cycles-pp.dequeue_entity
      0.33            -0.2        0.12 ± 19%  perf-profile.self.cycles-pp.check_preempt_wakeup
      0.38            -0.2        0.18 ± 15%  perf-profile.self.cycles-pp.validate_xmit_skb
      0.40            -0.2        0.20 ± 18%  perf-profile.self.cycles-pp.schedule
      0.64            -0.2        0.44 ±  6%  perf-profile.self.cycles-pp.__inet_lookup_established
      0.33 ±  2%      -0.2        0.13 ± 14%  perf-profile.self.cycles-pp.tcp_ack_update_rtt
      0.37 ±  2%      -0.2        0.17 ±  9%  perf-profile.self.cycles-pp.enqueue_to_backlog
      0.35 ±  6%      -0.2        0.16 ± 19%  perf-profile.self.cycles-pp.cpuacct_charge
      0.98            -0.2        0.79 ±  3%  perf-profile.self.cycles-pp.read_tsc
      0.36 ±  2%      -0.2        0.17 ± 18%  perf-profile.self.cycles-pp.migrate_enable
      0.34 ± 11%      -0.2        0.16 ± 16%  perf-profile.self.cycles-pp.ip_protocol_deliver_rcu
      0.33 ±  2%      -0.2        0.15 ± 18%  perf-profile.self.cycles-pp.sk_wait_data
      0.26 ±  3%      -0.2        0.08 ± 16%  perf-profile.self.cycles-pp.resched_curr
      0.35            -0.2        0.17 ± 14%  perf-profile.self.cycles-pp.migrate_disable
      0.29            -0.2        0.11 ± 17%  perf-profile.self.cycles-pp.__wrgsbase_inactive
      0.69 ±  5%      -0.2        0.52 ±  4%  perf-profile.self.cycles-pp.ktime_get
      0.34            -0.2        0.17 ± 15%  perf-profile.self.cycles-pp.__netif_receive_skb_one_core
      0.53 ±  2%      -0.2        0.36 ±  7%  perf-profile.self.cycles-pp.__alloc_skb
      0.31 ±  3%      -0.2        0.15 ± 13%  perf-profile.self.cycles-pp.ktime_get_with_offset
      0.27            -0.2        0.11 ± 16%  perf-profile.self.cycles-pp.ip_rcv_finish
      0.26 ±  2%      -0.2        0.11 ± 11%  perf-profile.self.cycles-pp.set_next_entity
      0.33 ±  3%      -0.2        0.17 ± 14%  perf-profile.self.cycles-pp.do_softirq
      0.37 ±  2%      -0.1        0.22 ± 10%  perf-profile.self.cycles-pp.try_to_wake_up
      0.27            -0.1        0.12 ± 17%  perf-profile.self.cycles-pp.skb_entail
      0.28 ±  3%      -0.1        0.13 ± 14%  perf-profile.self.cycles-pp.irqtime_account_irq
      0.24            -0.1        0.10 ± 10%  perf-profile.self.cycles-pp.tcp_event_data_recv
      0.29 ±  3%      -0.1        0.15 ± 13%  perf-profile.self.cycles-pp.ip_rcv_finish_core
      0.21 ±  2%      -0.1        0.07 ± 16%  perf-profile.self.cycles-pp.__usecs_to_jiffies
      0.42 ±  3%      -0.1        0.28 ±  6%  perf-profile.self.cycles-pp.exit_to_user_mode_prepare
      0.24 ± 10%      -0.1        0.10 ± 22%  perf-profile.self.cycles-pp.eth_type_trans
      0.30 ±  2%      -0.1        0.17 ± 14%  perf-profile.self.cycles-pp.dev_hard_start_xmit
      0.28 ±  2%      -0.1        0.15 ± 15%  perf-profile.self.cycles-pp.__ip_finish_output
      0.25            -0.1        0.11 ± 16%  perf-profile.self.cycles-pp.__tcp_select_window
      0.24 ±  2%      -0.1        0.11 ±  8%  perf-profile.self.cycles-pp.sk_filter_trim_cap
      0.19 ±  5%      -0.1        0.05 ± 13%  perf-profile.self.cycles-pp.tcp_v4_inbound_md5_hash
      0.27 ±  3%      -0.1        0.14 ± 11%  perf-profile.self.cycles-pp.inet_ehashfn
      0.20 ±  2%      -0.1        0.08 ± 16%  perf-profile.self.cycles-pp.security_sock_rcv_skb
      0.22 ±  5%      -0.1        0.10 ± 15%  perf-profile.self.cycles-pp.netif_rx_internal
      0.23 ±  7%      -0.1        0.11 ± 15%  perf-profile.self.cycles-pp.netif_skb_features
      0.24 ±  7%      -0.1        0.12 ±  8%  perf-profile.self.cycles-pp.tcp_event_new_data_sent
      0.17 ±  3%      -0.1        0.05 ± 45%  perf-profile.self.cycles-pp.cpumask_next_wrap
      0.29            -0.1        0.17 ±  8%  perf-profile.self.cycles-pp.tcp_wfree
      0.72            -0.1        0.60 ±  3%  perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.21 ±  4%      -0.1        0.10 ± 16%  perf-profile.self.cycles-pp.nf_hook_slow
      0.34 ±  2%      -0.1        0.22 ±  6%  perf-profile.self.cycles-pp.__sys_recvfrom
      0.22 ±  2%      -0.1        0.11 ± 16%  perf-profile.self.cycles-pp.schedule_timeout
      0.22            -0.1        0.11 ± 13%  perf-profile.self.cycles-pp.tcp_v4_fill_cb
      0.19 ±  4%      -0.1        0.08 ± 19%  perf-profile.self.cycles-pp.skb_network_protocol
      0.18 ±  4%      -0.1        0.07 ± 17%  perf-profile.self.cycles-pp.put_prev_entity
      0.19 ±  4%      -0.1        0.08 ± 22%  perf-profile.self.cycles-pp._find_next_bit
      0.15 ±  2%      -0.1        0.05 ± 45%  perf-profile.self.cycles-pp.tcp_grow_window
      0.25 ±  7%      -0.1        0.14 ± 11%  perf-profile.self.cycles-pp.mod_timer
      0.18 ± 13%      -0.1        0.08 ± 22%  perf-profile.self.cycles-pp.perf_trace_sched_switch
      0.21 ±  2%      -0.1        0.11 ± 18%  perf-profile.self.cycles-pp.__sk_dst_check
      0.19            -0.1        0.09 ± 16%  perf-profile.self.cycles-pp.tcp_options_write
      0.18 ±  2%      -0.1        0.08 ± 14%  perf-profile.self.cycles-pp.raw_local_deliver
      0.19 ±  2%      -0.1        0.09 ± 16%  perf-profile.self.cycles-pp.bictcp_acked
      0.21 ±  2%      -0.1        0.11 ± 13%  perf-profile.self.cycles-pp.__ip_local_out
      0.20 ±  2%      -0.1        0.10 ± 15%  perf-profile.self.cycles-pp.perf_tp_event
      0.17 ±  2%      -0.1        0.07 ± 17%  perf-profile.self.cycles-pp.bpf_skops_write_hdr_opt
      0.54            -0.1        0.44 ±  3%  perf-profile.self.cycles-pp.__might_sleep
      0.21 ±  2%      -0.1        0.11 ± 13%  perf-profile.self.cycles-pp.tcp_update_pacing_rate
      0.15 ±  3%      -0.1        0.05 ± 48%  perf-profile.self.cycles-pp.kmalloc_reserve
      0.16 ±  2%      -0.1        0.07 ± 18%  perf-profile.self.cycles-pp.bictcp_cwnd_event
      0.20 ±  3%      -0.1        0.10 ± 15%  perf-profile.self.cycles-pp.ip_rcv
      0.16 ±  3%      -0.1        0.06 ± 14%  perf-profile.self.cycles-pp.__rdgsbase_inactive
      0.12 ±  3%      -0.1        0.03 ±102%  perf-profile.self.cycles-pp.check_preempt_curr
      0.16 ±  3%      -0.1        0.07 ± 17%  perf-profile.self.cycles-pp.tcp_rate_gen
      0.25            -0.1        0.16 ±  6%  perf-profile.self.cycles-pp.__might_fault
      0.24            -0.1        0.15 ±  9%  perf-profile.self.cycles-pp.ip_send_check
      0.21 ±  3%      -0.1        0.12 ±  7%  perf-profile.self.cycles-pp.tcp_queue_rcv
      0.29            -0.1        0.20 ±  5%  perf-profile.self.cycles-pp.tcp_established_options
      0.26 ±  2%      -0.1        0.18 ±  5%  perf-profile.self.cycles-pp.tcp_recvmsg
      0.16 ±  2%      -0.1        0.08 ± 16%  perf-profile.self.cycles-pp.woken_wake_function
      0.65            -0.1        0.57 ±  2%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.13 ±  4%      -0.1        0.05 ± 45%  perf-profile.self.cycles-pp.__enqueue_entity
      0.17 ±  2%      -0.1        0.09 ± 13%  perf-profile.self.cycles-pp.__sock_wfree
      0.16 ±  2%      -0.1        0.09 ± 15%  perf-profile.self.cycles-pp.skb_release_head_state
      0.13 ± 12%      -0.1        0.05 ± 48%  perf-profile.self.cycles-pp.switch_ldt
      0.21 ±  2%      -0.1        0.13 ±  4%  perf-profile.self.cycles-pp.lock_sock_nested
      0.15 ±  3%      -0.1        0.07 ± 12%  perf-profile.self.cycles-pp.rb_next
      0.14 ±  3%      -0.1        0.06 ± 16%  perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
      0.12 ±  4%      -0.1        0.05 ± 45%  perf-profile.self.cycles-pp.tcp_update_skb_after_send
      0.16 ±  5%      -0.1        0.09 ± 18%  perf-profile.self.cycles-pp.ksoftirqd_running
      0.11 ±  6%      -0.1        0.03 ± 70%  perf-profile.self.cycles-pp.sched_clock_cpu
      0.26 ±  4%      -0.1        0.19 ±  3%  perf-profile.self.cycles-pp.import_single_range
      0.21 ±  2%      -0.1        0.14 ±  8%  perf-profile.self.cycles-pp.sock_def_readable
      0.13 ±  3%      -0.1        0.06 ± 14%  perf-profile.self.cycles-pp.apparmor_socket_sock_rcv_skb
      0.19 ±  4%      -0.1        0.12 ± 10%  perf-profile.self.cycles-pp.inet_recvmsg
      0.37            -0.1        0.30 ±  4%  perf-profile.self.cycles-pp.tcp_rcv_space_adjust
      0.16 ±  2%      -0.1        0.09 ± 20%  perf-profile.self.cycles-pp.__copy_skb_header
      0.15 ±  9%      -0.1        0.08 ± 16%  perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
      0.14 ±  3%      -0.1        0.07 ± 14%  perf-profile.self.cycles-pp.__wake_up_common
      0.14 ±  5%      -0.1        0.07 ± 17%  perf-profile.self.cycles-pp.ip_finish_output
      0.12 ±  4%      -0.1        0.06 ± 13%  perf-profile.self.cycles-pp.__tcp_send_ack
      0.26 ±  2%      -0.1        0.20 ±  4%  perf-profile.self.cycles-pp.__skb_clone
      0.10 ±  3%      -0.1        0.04 ± 71%  perf-profile.self.cycles-pp.sk_reset_timer
      0.21 ±  2%      -0.1        0.15 ±  3%  perf-profile.self.cycles-pp.sock_put
      0.20            -0.1        0.13 ±  8%  perf-profile.self.cycles-pp.sock_sendmsg
      0.14 ±  3%      -0.1        0.07 ± 16%  perf-profile.self.cycles-pp.rcu_read_unlock_strict
      0.13 ±  2%      -0.1        0.07 ± 10%  perf-profile.self.cycles-pp.__napi_poll
      0.09            -0.1        0.03 ±100%  perf-profile.self.cycles-pp.apparmor_ipv4_postroute
      0.15 ±  3%      -0.1        0.09 ± 17%  perf-profile.self.cycles-pp.ip_local_deliver
      0.14 ±  4%      -0.1        0.08 ± 24%  perf-profile.self.cycles-pp.kmalloc_slab
      0.31 ±  2%      -0.1        0.25 ±  4%  perf-profile.self.cycles-pp.dst_release
      0.12            -0.1        0.06 ± 13%  perf-profile.self.cycles-pp.tcp_send_delayed_ack
      0.12 ±  3%      -0.1        0.07 ± 11%  perf-profile.self.cycles-pp._raw_spin_unlock_bh
      0.12            -0.1        0.07 ± 14%  perf-profile.self.cycles-pp.ip_local_deliver_finish
      0.12 ±  4%      -0.1        0.06 ± 14%  perf-profile.self.cycles-pp.update_irq_load_avg
      0.16 ±  3%      -0.1        0.10 ±  4%  perf-profile.self.cycles-pp.tcp_v4_do_rcv
      0.12 ±  3%      -0.1        0.07        perf-profile.self.cycles-pp.tcp_rate_check_app_limited
      0.08 ±  4%      -0.1        0.03 ±100%  perf-profile.self.cycles-pp.ip_copy_addrs
      0.09 ±  5%      -0.1        0.04 ± 71%  perf-profile.self.cycles-pp.tcp_chrono_stop
      0.11 ±  4%      -0.1        0.06 ± 14%  perf-profile.self.cycles-pp.validate_xmit_xfrm
      0.11 ±  4%      -0.1        0.06 ± 14%  perf-profile.self.cycles-pp.tcp_rearm_rto
      0.15 ±  4%      -0.1        0.10 ±  6%  perf-profile.self.cycles-pp.sk_stream_alloc_skb
      0.24 ±  5%      -0.1        0.19 ±  3%  perf-profile.self.cycles-pp.inet_ioctl
      0.12 ±  4%      -0.0        0.07 ± 18%  perf-profile.self.cycles-pp.bictcp_cong_avoid
      0.16 ±  6%      -0.0        0.11 ±  6%  perf-profile.self.cycles-pp.__x64_sys_recvfrom
      0.07 ±  6%      -0.0        0.03 ± 99%  perf-profile.self.cycles-pp.sk_free
      0.14 ±  3%      -0.0        0.10 ± 15%  perf-profile.self.cycles-pp.rb_insert_color
      0.20 ±  5%      -0.0        0.15 ±  4%  perf-profile.self.cycles-pp.__x64_sys_sendto
      0.16            -0.0        0.11 ± 11%  perf-profile.self.cycles-pp.tcp_recv_timestamp
      0.07 ±  6%      -0.0        0.03 ±100%  perf-profile.self.cycles-pp.place_entity
      0.26 ±  2%      -0.0        0.21 ±  5%  perf-profile.self.cycles-pp.do_syscall_64
      0.17 ±  2%      -0.0        0.12 ±  4%  perf-profile.self.cycles-pp.kfree_skbmem
      0.11            -0.0        0.06 ± 11%  perf-profile.self.cycles-pp.tcp_check_space
      0.08 ±  4%      -0.0        0.04 ± 71%  perf-profile.self.cycles-pp.lock_sock_fast
      0.12 ±  4%      -0.0        0.08 ±  6%  perf-profile.self.cycles-pp.tcp_sendmsg
      0.64 ±  2%      -0.0        0.60        perf-profile.self.cycles-pp.kmem_cache_free
      0.19 ±  3%      -0.0        0.15 ±  5%  perf-profile.self.cycles-pp.tcp_release_cb
      0.25            -0.0        0.21 ±  3%  perf-profile.self.cycles-pp.rcu_nocb_flush_deferred_wakeup
      0.34            -0.0        0.30 ±  3%  perf-profile.self.cycles-pp.__sys_sendto
      0.15 ±  3%      -0.0        0.12 ±  9%  perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
      0.12 ±  4%      -0.0        0.09 ±  9%  perf-profile.self.cycles-pp.__tcp_push_pending_frames
      0.77            -0.0        0.73        perf-profile.self.cycles-pp.__fget_light
      0.12 ±  3%      -0.0        0.09 ±  6%  perf-profile.self.cycles-pp.sock_rfree
      0.12 ±  4%      -0.0        0.08 ± 14%  perf-profile.self.cycles-pp.rb_erase
      0.18 ±  2%      -0.0        0.15 ±  7%  perf-profile.self.cycles-pp.do_vfs_ioctl
      0.18 ±  3%      -0.0        0.15 ±  3%  perf-profile.self.cycles-pp.tcp_ioctl
      0.10            -0.0        0.07 ± 10%  perf-profile.self.cycles-pp.perf_trace_sched_wakeup_template
      0.08            -0.0        0.05        perf-profile.self.cycles-pp.sk_forced_mem_schedule
      0.20 ±  2%      -0.0        0.18 ±  5%  perf-profile.self.cycles-pp.release_sock
      0.16 ±  4%      -0.0        0.13 ±  7%  perf-profile.self.cycles-pp.__x64_sys_ioctl
      0.10 ±  4%      -0.0        0.08 ±  7%  perf-profile.self.cycles-pp.tcp_mstamp_refresh
      0.10 ±  5%      -0.0        0.07 ±  6%  perf-profile.self.cycles-pp.tcp_update_recv_tstamps
      0.09 ±  5%      -0.0        0.07 ±  8%  perf-profile.self.cycles-pp.__tcp_ack_snd_check
      0.10 ±  4%      -0.0        0.08        perf-profile.self.cycles-pp.skb_clone
      0.24 ±  3%      -0.0        0.21 ±  3%  perf-profile.self.cycles-pp.sock_ioctl
      0.07 ±  6%      -0.0        0.06 ±  8%  perf-profile.self.cycles-pp.inet_sendmsg
      0.07 ±  5%      -0.0        0.05 ±  8%  perf-profile.self.cycles-pp.security_socket_recvmsg
      0.10 ±  3%      -0.0        0.09 ±  4%  perf-profile.self.cycles-pp.copyin
      0.07 ±  7%      -0.0        0.05 ±  8%  perf-profile.self.cycles-pp.apparmor_socket_recvmsg
      0.15 ±  2%      +0.0        0.17 ±  3%  perf-profile.self.cycles-pp.__list_add_valid
      0.09 ±  5%      +0.0        0.12 ±  8%  perf-profile.self.cycles-pp.syscall_enter_from_user_mode
      0.11 ±  3%      +0.0        0.14 ±  5%  perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
      0.21 ±  3%      +0.0        0.23 ±  4%  perf-profile.self.cycles-pp._copy_to_iter
      0.20 ±  2%      +0.0        0.24        perf-profile.self.cycles-pp.tcp_current_mss
      0.18 ±  2%      +0.0        0.22 ±  3%  perf-profile.self.cycles-pp.skb_page_frag_refill
      0.08 ±  5%      +0.1        0.14 ±  7%  perf-profile.self.cycles-pp.sk_page_frag_refill
      0.00            +0.1        0.06 ± 13%  perf-profile.self.cycles-pp.ktime_get_update_offsets_now
      0.70            +0.1        0.76        perf-profile.self.cycles-pp.tcp_write_xmit
      0.00            +0.1        0.07 ± 11%  perf-profile.self.cycles-pp.refill_obj_stock
      0.00            +0.1        0.07 ± 10%  perf-profile.self.cycles-pp.__release_sock
      0.30 ±  2%      +0.1        0.37 ±  2%  perf-profile.self.cycles-pp.__list_del_entry_valid
      1.03            +0.1        1.10 ±  2%  perf-profile.self.cycles-pp._raw_spin_lock_bh
      0.00            +0.1        0.07 ± 12%  perf-profile.self.cycles-pp.__alloc_pages_nodemask
      0.00            +0.1        0.08 ± 12%  perf-profile.self.cycles-pp.drain_obj_stock
      0.00            +0.1        0.09 ± 10%  perf-profile.self.cycles-pp.__zone_watermark_ok
      0.00            +0.1        0.10 ±  9%  perf-profile.self.cycles-pp.kmem_cache_alloc
      0.11 ±  5%      +0.1        0.23 ±  6%  perf-profile.self.cycles-pp._copy_from_iter_full
      0.00            +0.1        0.12 ± 11%  perf-profile.self.cycles-pp.prep_compound_page
      0.00            +0.1        0.12 ±  9%  perf-profile.self.cycles-pp.__free_one_page
      0.00            +0.1        0.12 ± 15%  perf-profile.self.cycles-pp.skb_try_coalesce
      0.24 ±  3%      +0.1        0.36 ±  4%  perf-profile.self.cycles-pp.__skb_datagram_iter
      0.00            +0.1        0.12 ± 10%  perf-profile.self.cycles-pp.poll_idle
      0.00            +0.1        0.14 ± 13%  perf-profile.self.cycles-pp.__slab_free
      0.00            +0.2        0.17 ± 10%  perf-profile.self.cycles-pp.__free_pages_ok
      0.05            +0.2        0.24 ± 12%  perf-profile.self.cycles-pp.tcp_tx_timestamp
      0.78 ±  2%      +0.2        0.98        perf-profile.self.cycles-pp.syscall_return_via_sysret
      0.50            +0.2        0.72 ±  4%  perf-profile.self.cycles-pp.skb_release_data
      0.09 ±  8%      +0.3        0.40 ± 10%  perf-profile.self.cycles-pp.iov_iter_advance
      0.10 ±  4%      +0.3        0.41 ±  8%  perf-profile.self.cycles-pp.tcp_push
      0.29 ±  4%      +0.3        0.60 ±  5%  perf-profile.self.cycles-pp.__check_object_size
      0.00            +0.3        0.33 ± 10%  perf-profile.self.cycles-pp.rmqueue
      0.01 ±223%      +0.4        0.44 ± 10%  perf-profile.self.cycles-pp.intel_idle
      1.28 ±  2%      +1.5        2.78 ±  6%  perf-profile.self.cycles-pp.tcp_sendmsg_locked
      0.00            +2.2        2.20 ± 18%  perf-profile.self.cycles-pp.__sk_mem_raise_allocated
      3.82 ±  3%      +7.3       11.12 ±  7%  perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      0.02 ±223%     +24.1       24.14 ± 11%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath


                                                                                
                            stress-ng.time.system_time                          
                                                                                
  4700 +--------------------------------------------------------------------+   
  4600 |-+ O O        O                         O       O                   |   
       |O O    O OO  O   O O  OOO  O OOO O   OO  O O O O  OO                |   
  4500 |-+    O     O   O   O     O       O O       O                       |   
  4400 |-+                                                                  |   
       |                                                                    |   
  4300 |-+                                                                  |   
  4200 |-+                                                                  |   
  4100 |-+                                                                  |   
       |                                                                    |   
  4000 |-+                                                                  |   
  3900 |-+                                                                  |   
       |                                            ++.+        .++.+++.+ .+|   
  3800 |+.++.+++.++.+++.++.++.+++.++.+++.++.+++.++.+    +.++.+++         +  |   
  3700 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                    stress-ng.time.percent_of_cpu_this_job_got                  
                                                                                
  7800 +--------------------------------------------------------------------+   
       |  OO O        O  O    OO              O O      OO                   |   
  7600 |O+    OO OO  O     OO   O OO OOO OO  O   O O O    OO                |   
  7400 |-+          O   O                   O       O                       |   
       |                                                                    |   
  7200 |-+                                                                  |   
       |                                                                    |   
  7000 |-+                                                                  |   
       |                                                                    |   
  6800 |-+                                                                  |   
  6600 |-+                            +            +                        |   
       |                              ::          + :                       |   
  6400 |+.++.+++.++.+++.++.++.+++.++.+ +.++.+++.++  ++.++.++.+++.++.+++.++.+|   
       |                                                                    |   
  6200 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                       stress-ng.time.voluntary_context_switches                
                                                                                
    5e+08 +-----------------------------------------------------------------+   
          |                                                                 |   
  4.5e+08 |+.+++.+++.+++.++.+++.+++.+++  ++.+++.+++  ++.+++.++.+++.+++.+++.+|   
          |                            + :         + :                      |   
    4e+08 |-+                           +           +                       |   
          |                                                                 |   
  3.5e+08 |-+                                                               |   
          |                                                                 |   
    3e+08 |-+                                                               |   
          |                                                                 |   
  2.5e+08 |-+                                                               |   
          |                                  O       O                      |   
    2e+08 |O+O   O O  O   O   O   O OOO OOO O O O O O O O O O               |   
          |   OO  O  O O    OO  OO               O       O                  |   
  1.5e+08 +-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                      stress-ng.time.involuntary_context_switches               
                                                                                
    5e+08 +-----------------------------------------------------------------+   
          |                                                                 |   
  4.5e+08 |+.+++.+++.+++.++.+++.+++.+++  ++.+++.+++  ++.+++.++.+++.+++.+++.+|   
    4e+08 |-+                          + :         + :                      |   
          |                             +           +                       |   
  3.5e+08 |-+                                                               |   
          |                                                                 |   
    3e+08 |-+                                                               |   
          |                                                                 |   
  2.5e+08 |-+                                                               |   
    2e+08 |-+                                O       O                      |   
          |O O   OOO OO   O   O   O OOO OOO O O O O O O O O O               |   
  1.5e+08 |-+ OO       O O  OO  OO               O       O                  |   
          |                                                                 |   
    1e+08 +-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                  stress-ng.sock.ops                            
                                                                                
  1.8e+06 +-----------------------------------------------------------------+   
  1.7e+06 |-+ O          O                                                  |   
          |  O O  O  O O    OO  OO   OO O       OO      OO                  |   
  1.6e+06 |O+    O O  O   O   O   O O    OO O O   O O O   O O               |   
  1.5e+06 |-+                                O       O                      |   
          |                                                                 |   
  1.4e+06 |-+                                                               |   
  1.3e+06 |-+                                                               |   
  1.2e+06 |-+                                                               |   
          |                                                                 |   
  1.1e+06 |-+                                                               |   
    1e+06 |-+  +.                                                           |   
          |+.++  +++.+++.++.+++.+++.+++. ++.+++.+++. ++.+++.++.+++.+++.+++.+|   
   900000 |-+                           +           +                       |   
   800000 +-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                             stress-ng.sock.ops_per_sec                         
                                                                                
  30000 +-------------------------------------------------------------------+   
        |              O                                                    |   
  28000 |-+OO O O  O  O   OO  O O   O           O        O                  |   
  26000 |O+    O  O O    O   O   OO  O OOO O O O  OO  OO  OO                |   
        |                                   O       O                       |   
  24000 |-+                                                                 |   
        |                                                                   |   
  22000 |-+                                                                 |   
        |                                                                   |   
  20000 |-+                                                                 |   
  18000 |-+                                                                 |   
        |                                                                   |   
  16000 |+.++.+++.+++.++.++   +. + .+     .+ +. +.      .+ +.++.+++.+++.++.+|   
        |                  +.+  + +  +. ++  +  +  + +.++  +                 |   
  14000 +-------------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample



Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


---
0DAY/LKP+ Test Infrastructure                   Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org       Intel Corporation

Thanks,
Oliver Sang


View attachment "config-5.12.0-rc2-00031-gd619f7afd773" of type "text/plain" (172899 bytes)

View attachment "job-script" of type "text/plain" (8153 bytes)

View attachment "job.yaml" of type "text/plain" (5575 bytes)

View attachment "reproduce" of type "text/plain" (337 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ