lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Date:   Mon, 2 Jul 2018 13:27:43 +0800
From:   kernel test robot <xiaolong.ye@...el.com>
To:     David Howells <dhowells@...hat.com>
Cc:     LKML <linux-kernel@...r.kernel.org>,
        Stephen Rothwell <sfr@...b.auug.org.au>, lkp@...org
Subject: [lkp-robot] [vfs]  8a2e54b8af:  netperf.Throughput_total_tps -26.1%
 regression


Greeting,

FYI, we noticed a -26.1% regression of netperf.Throughput_total_tps due to commit:


commit: 8a2e54b8af88639527b308878bcde31dfa2644f6 ("vfs: Implement a filesystem superblock creation/configuration context")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master

in testcase: netperf
on test machine: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory
with following parameters:

	ip: ipv4
	runtime: 300s
	nr_threads: 25%
	cluster: cs-localhost
	test: TCP_CRR
	cpufreq_governor: performance

test-description: Netperf is a benchmark that can be use to measure various aspect of networking performance.
test-url: http://www.netperf.org/netperf/


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase:
  cs-localhost/gcc-7/performance/ipv4/x86_64-rhel-7.2/25%/debian-x86_64-2018-04-03.cgz/300s/lkp-bdw-ep2/TCP_CRR/netperf

commit: 
  889bc23d98 ("vfs: Require specification of size of mount data for internal mounts")
  8a2e54b8af ("vfs: Implement a filesystem superblock creation/configuration context")

889bc23d98821043 8a2e54b8af88639527b308878b 
---------------- -------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    468684           -26.1%     346523        netperf.Throughput_total_tps
     21303           -26.1%      15751        netperf.Throughput_tps
    305.02            +0.0%     305.02        netperf.time.elapsed_time
    305.02            +0.0%     305.02        netperf.time.elapsed_time.max
   1240905            +4.0%    1290438        netperf.time.involuntary_context_switches
      2340            +0.7%       2357        netperf.time.maximum_resident_set_size
      6722            +1.4%       6813 ±  2%  netperf.time.minor_page_faults
      4096            +0.0%       4096        netperf.time.page_size
      1879            +4.4%       1962        netperf.time.percent_of_cpu_this_job_got
      5453            +6.1%       5786        netperf.time.system_time
    279.41           -28.7%     199.20        netperf.time.user_time
 2.724e+08           -25.9%  2.018e+08        netperf.time.voluntary_context_switches
 1.406e+08           -26.1%   1.04e+08        netperf.workload
    336286            -0.2%     335615        interrupts.CAL:Function_call_interrupts
    438.30            -2.1%     429.02        pmeter.Average_Active_Power
     48.61           -24.5%      36.71        pmeter.performance_per_watt
     27.42            -0.5%      27.29 ±  4%  boot-time.boot
     15.78            -3.0%      15.30 ±  5%  boot-time.dhcp
      2226            +2.3%       2277        boot-time.idle
     18.86 ±  2%      -1.3%      18.61 ±  5%  boot-time.kernel_boot
 4.304e+08           -24.7%  3.243e+08        softirqs.NET_RX
  39646822           -18.7%   32219712        softirqs.RCU
   5590041            +7.3%    5996673 ±  2%  softirqs.SCHED
  12330529 ± 10%      -1.3%   12166738 ± 12%  softirqs.TIMER
     69.16            -5.2       63.92        mpstat.cpu.idle%
      0.00 ±100%      -0.0        0.00 ±173%  mpstat.cpu.iowait%
     11.34            -3.6        7.71        mpstat.cpu.soft%
     18.02            +9.3       27.32        mpstat.cpu.sys%
      1.47            -0.4        1.05        mpstat.cpu.usr%
      4.00            +0.0%       4.00        vmstat.memory.buff
   1005802            +0.5%    1010518        vmstat.memory.cache
   1.3e+08            -0.0%    1.3e+08        vmstat.memory.free
     27.75           +14.4%      31.75        vmstat.procs.r
   3249087           -33.5%    2159914        vmstat.system.cs
     93331            -0.1%      93238        vmstat.system.in
      0.00          -100.0%       0.00        numa-numastat.node0.interleave_hit
    478419 ± 29%      -5.1%     454087 ± 22%  numa-numastat.node0.local_node
    492847 ± 28%      -5.5%     465546 ± 22%  numa-numastat.node0.numa_hit
     14432 ± 33%     -20.6%      11460 ± 61%  numa-numastat.node0.other_node
      0.00          -100.0%       0.00        numa-numastat.node1.interleave_hit
    431892 ± 34%     -14.4%     369692 ± 29%  numa-numastat.node1.local_node
    434779 ± 32%     -13.6%     375541 ± 29%  numa-numastat.node1.numa_hit
      2887 ±169%    +102.5%       5848 ±120%  numa-numastat.node1.other_node
 4.093e+09           -30.4%  2.848e+09        cpuidle.C1.time
 4.895e+08           -35.2%  3.173e+08        cpuidle.C1.usage
  81116892 ±  2%    +177.7%  2.253e+08 ±  3%  cpuidle.C1E.time
   3590244 ±  2%    +173.5%    9819699 ±  3%  cpuidle.C1E.usage
  85009589           -21.6%   66665506        cpuidle.C3.time
    209102           -14.3%     179301        cpuidle.C3.usage
 1.333e+10            +0.4%  1.338e+10        cpuidle.C6.time
  13846804            -0.0%   13839975        cpuidle.C6.usage
  43536088           -33.9%   28763801        cpuidle.POLL.time
   6632569           -29.1%    4700058        cpuidle.POLL.usage
    983.75           +10.9%       1090        turbostat.Avg_MHz
     35.27            +3.8       39.08        turbostat.Busy%
      2797            +0.0%       2797        turbostat.Bzy_MHz
 4.895e+08           -35.2%  3.173e+08        turbostat.C1
     15.19            -4.6       10.57        turbostat.C1%
   3590143 ±  2%    +173.5%    9819585 ±  3%  turbostat.C1E
      0.30 ±  3%      +0.5        0.83 ±  3%  turbostat.C1E%
    208933           -14.3%     179136        turbostat.C3
      0.32            -0.1        0.25        turbostat.C3%
  13844451            -0.0%   13837698        turbostat.C6
     49.44            +0.2       49.64        turbostat.C6%
     53.73            -5.7%      50.69        turbostat.CPU%c1
      0.23           -26.1%       0.17        turbostat.CPU%c3
     10.78            -6.6%      10.07 ±  2%  turbostat.CPU%c6
     75.00 ±  5%      -7.7%      69.25        turbostat.CoreTmp
  28660427            -0.1%   28629630        turbostat.IRQ
      0.55 ±  9%      -2.3%       0.54 ± 10%  turbostat.Pkg%pc2
      0.55 ±  6%      +5.5%       0.58 ±  6%  turbostat.Pkg%pc6
     77.00 ±  2%      -5.8%      72.50        turbostat.PkgTmp
    221.95            -2.6%     216.17        turbostat.PkgWatt
     16.44            +3.5%      17.01        turbostat.RAMWatt
      2195            +0.0%       2195        turbostat.TSC_MHz
    286110            +1.6%     290795        meminfo.Active
    285878            +1.6%     290563        meminfo.Active(anon)
    177310            +0.2%     177735        meminfo.AnonHugePages
    255935            -0.0%     255847        meminfo.AnonPages
    939803            +0.5%     944631        meminfo.Cached
    200608            +0.0%     200608        meminfo.CmaFree
    204800            +0.0%     204800        meminfo.CmaTotal
  65946628            +0.0%   65946628        meminfo.CommitLimit
    422156            -1.6%     415405        meminfo.Committed_AS
 1.292e+08            +0.2%  1.295e+08        meminfo.DirectMap1G
   6697984 ±  6%      -3.9%    6434304 ±  7%  meminfo.DirectMap2M
    261424 ± 14%      +0.6%     262960 ±  4%  meminfo.DirectMap4k
      2048            +0.0%       2048        meminfo.Hugepagesize
     18398            +0.7%      18518        meminfo.Inactive
     18051            +0.7%      18171        meminfo.Inactive(anon)
     15429            -0.3%      15389        meminfo.KernelStack
     24964            +1.0%      25218        meminfo.Mapped
 1.294e+08            -0.0%  1.294e+08        meminfo.MemAvailable
   1.3e+08            -0.0%    1.3e+08        meminfo.MemFree
 1.319e+08            -0.0%  1.319e+08        meminfo.MemTotal
    809.25 ±100%     -50.1%     403.50 ±173%  meminfo.Mlocked
      5903            -0.7%       5860        meminfo.PageTables
     65803            -0.2%      65678        meminfo.SReclaimable
    219346            -0.3%     218728        meminfo.SUnreclaim
     48660 ±  6%      +9.9%      53484        meminfo.Shmem
    285150            -0.3%     284407        meminfo.Slab
    890696            -0.0%     890634        meminfo.Unevictable
 3.436e+10            +0.0%  3.436e+10        meminfo.VmallocTotal
  3.39e+12            -4.9%  3.225e+12        perf-stat.branch-instructions
      2.35            -0.5        1.81        perf-stat.branch-miss-rate%
 7.958e+10           -26.5%  5.853e+10        perf-stat.branch-misses
      1.28            +1.0        2.28        perf-stat.cache-miss-rate%
 5.826e+09           +35.8%  7.912e+09        perf-stat.cache-misses
 4.561e+11           -24.0%  3.465e+11        perf-stat.cache-references
 1.001e+09           -33.5%  6.652e+08        perf-stat.context-switches
      1.55           +21.2%       1.88        perf-stat.cpi
 2.662e+13           +10.1%  2.931e+13        perf-stat.cpu-cycles
    101022           -35.2%      65435        perf-stat.cpu-migrations
      0.08            -0.0        0.07        perf-stat.dTLB-load-miss-rate%
  4.13e+09           -22.5%  3.203e+09        perf-stat.dTLB-load-misses
 5.225e+12           -12.2%  4.587e+12        perf-stat.dTLB-loads
      0.05            -0.0        0.05        perf-stat.dTLB-store-miss-rate%
 1.794e+09           -35.9%  1.151e+09        perf-stat.dTLB-store-misses
 3.329e+12           -25.7%  2.474e+12        perf-stat.dTLB-stores
     31.10 ± 21%      -1.8       29.27 ±  4%  perf-stat.iTLB-load-miss-rate%
 7.682e+09 ± 32%     -38.5%  4.728e+09 ±  4%  perf-stat.iTLB-load-misses
  1.65e+10           -30.8%  1.142e+10        perf-stat.iTLB-loads
  1.72e+13            -9.1%  1.562e+13        perf-stat.instructions
      2429 ± 23%     +36.3%       3311 ±  4%  perf-stat.instructions-per-iTLB-miss
      0.65           -17.5%       0.53        perf-stat.ipc
    791192            +0.1%     791897        perf-stat.minor-faults
     89.78            +6.1       95.89        perf-stat.node-load-miss-rate%
 1.938e+09           +89.9%  3.681e+09        perf-stat.node-load-misses
 2.205e+08           -28.4%  1.578e+08 ±  2%  perf-stat.node-loads
     83.13           -22.4       60.74        perf-stat.node-store-miss-rate%
 1.868e+09            +0.3%  1.873e+09        perf-stat.node-store-misses
 3.791e+08          +219.3%  1.211e+09        perf-stat.node-stores
    791204            +0.1%     791913        perf-stat.page-faults
    122296           +22.9%     150294        perf-stat.path-length
     71487            +1.6%      72661        proc-vmstat.nr_active_anon
     63972            -0.0%      63948        proc-vmstat.nr_anon_pages
   3229476            -0.0%    3229370        proc-vmstat.nr_dirty_background_threshold
   6466849            -0.0%    6466637        proc-vmstat.nr_dirty_threshold
    234958            +0.5%     236166        proc-vmstat.nr_file_pages
     50152            +0.0%      50152        proc-vmstat.nr_free_cma
  32496765            -0.0%   32495704        proc-vmstat.nr_free_pages
      4489            +0.6%       4516        proc-vmstat.nr_inactive_anon
     15435            -0.3%      15392        proc-vmstat.nr_kernel_stack
      6301            +0.9%       6357        proc-vmstat.nr_mapped
    202.25 ±100%     -50.1%     101.00 ±173%  proc-vmstat.nr_mlock
      1476            -0.8%       1464        proc-vmstat.nr_page_table_pages
     12171 ±  6%      +9.9%      13379        proc-vmstat.nr_shmem
     16450            -0.2%      16419        proc-vmstat.nr_slab_reclaimable
     54829            -0.3%      54661        proc-vmstat.nr_slab_unreclaimable
    222673            -0.0%     222658        proc-vmstat.nr_unevictable
     71487            +1.6%      72661        proc-vmstat.nr_zone_active_anon
      4489            +0.6%       4516        proc-vmstat.nr_zone_inactive_anon
    222673            -0.0%     222658        proc-vmstat.nr_zone_unevictable
      3898 ± 74%     -26.1%       2880 ± 16%  proc-vmstat.numa_hint_faults
    958.75 ± 36%      +1.5%     973.50 ± 15%  proc-vmstat.numa_hint_faults_local
    956184            -9.2%     868423        proc-vmstat.numa_hit
    938852            -9.3%     851105        proc-vmstat.numa_local
     17332            -0.1%      17318        proc-vmstat.numa_other
      2474 ±111%     -44.0%       1384 ± 32%  proc-vmstat.numa_pages_migrated
     10570 ±132%      +7.1%      11325 ±130%  proc-vmstat.numa_pte_updates
     11386 ±  8%     +13.4%      12916        proc-vmstat.pgactivate
   1101289           -10.5%     985451        proc-vmstat.pgalloc_normal
    812475            +0.3%     815020        proc-vmstat.pgfault
   1043137           -11.0%     927908        proc-vmstat.pgfree
      2474 ±111%     -44.0%       1384 ± 32%  proc-vmstat.pgmigrate_success
    153400 ± 33%      +6.8%     163789 ± 12%  numa-meminfo.node0.Active
    153284 ± 33%      +6.8%     163673 ± 12%  numa-meminfo.node0.Active(anon)
    100577 ± 37%      +6.9%     107492 ± 18%  numa-meminfo.node0.AnonHugePages
    143856 ± 36%      +7.4%     154566 ± 10%  numa-meminfo.node0.AnonPages
    472071 ±  3%      -0.8%     468470 ±  3%  numa-meminfo.node0.FilePages
     16604 ±  4%     -19.6%      13355 ± 50%  numa-meminfo.node0.Inactive
     16433 ±  4%     -19.8%      13180 ± 51%  numa-meminfo.node0.Inactive(anon)
      8551 ±  6%      -7.4%       7915        numa-meminfo.node0.KernelStack
     14689            -6.6%      13723 ± 14%  numa-meminfo.node0.Mapped
  64814375            +0.1%   64857851        numa-meminfo.node0.MemFree
  65867452            +0.0%   65867452        numa-meminfo.node0.MemTotal
   1053075 ±  5%      -4.1%    1009599 ±  6%  numa-meminfo.node0.MemUsed
      3796 ± 35%     -41.0%       2238 ± 17%  numa-meminfo.node0.PageTables
     36487 ±  9%     -11.1%      32442 ± 19%  numa-meminfo.node0.SReclaimable
    117644 ±  3%     -10.2%     105665 ±  3%  numa-meminfo.node0.SUnreclaim
     26220 ± 53%     -13.7%      22631 ± 75%  numa-meminfo.node0.Shmem
    154132 ±  4%     -10.4%     138109 ±  7%  numa-meminfo.node0.Slab
    445627            -0.0%     445610        numa-meminfo.node0.Unevictable
    132774 ± 40%      -4.3%     127037 ± 16%  numa-meminfo.node1.Active
    132658 ± 40%      -4.3%     126921 ± 16%  numa-meminfo.node1.Active(anon)
     76742 ± 49%      -8.4%      70282 ± 30%  numa-meminfo.node1.AnonHugePages
    112031 ± 47%      -9.6%     101265 ± 16%  numa-meminfo.node1.AnonPages
    467751 ±  2%      +1.8%     476180 ±  3%  numa-meminfo.node1.FilePages
      1694 ± 41%    +202.8%       5130 ±132%  numa-meminfo.node1.Inactive
      1519 ± 51%    +226.4%       4958 ±139%  numa-meminfo.node1.Inactive(anon)
      6879 ±  7%      +8.6%       7473 ±  2%  numa-meminfo.node1.KernelStack
     10006           +14.1%      11417 ± 18%  numa-meminfo.node1.Mapped
  65172705            -0.1%   65124824        numa-meminfo.node1.MemFree
  66025808            -0.0%   66025804        numa-meminfo.node1.MemTotal
    853102 ±  6%      +5.6%     900978 ±  7%  numa-meminfo.node1.MemUsed
      2107 ± 62%     +71.8%       3620 ± 11%  numa-meminfo.node1.PageTables
     29316 ± 12%     +13.4%      33234 ± 18%  numa-meminfo.node1.SReclaimable
    101687 ±  5%     +11.1%     112982 ±  3%  numa-meminfo.node1.SUnreclaim
     22455 ± 55%     +37.5%      30868 ± 56%  numa-meminfo.node1.Shmem
    131004 ±  6%     +11.6%     146217 ±  6%  numa-meminfo.node1.Slab
    445068            -0.0%     445023        numa-meminfo.node1.Unevictable
    194846            +3.2%     201093 ±  4%  slabinfo.Acpi-Namespace.active_objs
    195003            +3.5%     201804 ±  4%  slabinfo.Acpi-Namespace.num_objs
    150215            -0.1%     150062        slabinfo.Acpi-Operand.active_objs
    150220            -0.1%     150066        slabinfo.Acpi-Operand.num_objs
     13071 ±  4%      -1.4%      12890        slabinfo.anon_vma.active_objs
     13071 ±  4%      -1.4%      12890        slabinfo.anon_vma.num_objs
      1952 ± 14%     +23.9%       2418 ±  2%  slabinfo.biovec-64.active_objs
      1952 ± 14%     +23.9%       2418 ±  2%  slabinfo.biovec-64.num_objs
      7860 ±  2%      -2.5%       7663 ±  2%  slabinfo.cred_jar.active_objs
      7860 ±  2%      -2.5%       7663 ±  2%  slabinfo.cred_jar.num_objs
     73763            -1.0%      73030        slabinfo.dentry.active_objs
     74616            -1.0%      73851        slabinfo.dentry.num_objs
    681.25 ± 11%     +14.5%     780.25 ±  8%  slabinfo.dmaengine-unmap-128.active_objs
    681.25 ± 11%     +14.5%     780.25 ±  8%  slabinfo.dmaengine-unmap-128.num_objs
      4603 ±  6%      -4.1%       4414 ± 10%  slabinfo.dmaengine-unmap-16.active_objs
      4603 ±  6%      -4.1%       4414 ± 10%  slabinfo.dmaengine-unmap-16.num_objs
      3439 ±  5%      -6.7%       3208 ±  6%  slabinfo.eventpoll_pwq.active_objs
      3439 ±  5%      -6.7%       3208 ±  6%  slabinfo.eventpoll_pwq.num_objs
     37128            -9.2%      33700        slabinfo.filp.active_objs
      1161            -9.3%       1054        slabinfo.filp.active_slabs
     37186            -9.2%      33748        slabinfo.filp.num_objs
      1161            -9.3%       1054        slabinfo.filp.num_slabs
     78122            -0.2%      77950        slabinfo.kernfs_node_cache.active_objs
     78122            -0.2%      77950        slabinfo.kernfs_node_cache.num_objs
      6797            -2.7%       6612 ±  3%  slabinfo.kmalloc-2048.active_objs
      6857 ±  2%      -2.8%       6666 ±  3%  slabinfo.kmalloc-2048.num_objs
     37114            +1.8%      37780 ±  3%  slabinfo.kmalloc-32.active_objs
     37408            +1.6%      37990 ±  2%  slabinfo.kmalloc-32.num_objs
      8206 ±  4%      -2.8%       7974 ±  3%  slabinfo.kmalloc-512.active_objs
    133449            +0.9%     134658        slabinfo.kmalloc-64.active_objs
    133782            +0.8%     134895        slabinfo.kmalloc-64.num_objs
     10650 ±  3%      +3.2%      10990 ±  5%  slabinfo.kmalloc-96.active_objs
     10664 ±  3%      +3.2%      11002 ±  5%  slabinfo.kmalloc-96.num_objs
    681.00 ±  4%      -9.4%     617.00 ±  8%  slabinfo.kmem_cache_node.active_objs
    736.00 ±  4%      -8.7%     672.00 ±  8%  slabinfo.kmem_cache_node.num_objs
     31500 ±  4%      -2.6%      30689        slabinfo.pid.active_objs
     31500 ±  4%      -2.6%      30689        slabinfo.pid.num_objs
    278948            +0.1%     279251        slabinfo.tw_sock_TCP.active_objs
    279098            +0.1%     279394        slabinfo.tw_sock_TCP.num_objs
     27069 ±  3%      -2.7%      26348 ±  2%  slabinfo.vm_area_struct.active_objs
     27069 ±  3%      -2.6%      26367 ±  2%  slabinfo.vm_area_struct.num_objs
     19563 ± 15%     -11.1%      17395 ± 20%  numa-vmstat.node0
     38314 ± 33%      +6.8%      40916 ± 12%  numa-vmstat.node0.nr_active_anon
     35963 ± 36%      +7.5%      38645 ± 10%  numa-vmstat.node0.nr_anon_pages
    118011 ±  3%      -0.8%     117119 ±  3%  numa-vmstat.node0.nr_file_pages
  16203578            +0.1%   16214439        numa-vmstat.node0.nr_free_pages
      4108 ±  4%     -19.6%       3302 ± 51%  numa-vmstat.node0.nr_inactive_anon
      8552 ±  6%      -7.4%       7915        numa-vmstat.node0.nr_kernel_stack
      3741 ±  2%      -6.4%       3501 ± 13%  numa-vmstat.node0.nr_mapped
    101.25 ±102%     -42.2%      58.50 ±173%  numa-vmstat.node0.nr_mlock
    948.25 ± 35%     -41.0%     559.00 ± 17%  numa-vmstat.node0.nr_page_table_pages
      6549 ± 53%     -13.6%       5659 ± 75%  numa-vmstat.node0.nr_shmem
      9121 ±  9%     -11.1%       8110 ± 19%  numa-vmstat.node0.nr_slab_reclaimable
     29408 ±  3%     -10.1%      26433 ±  3%  numa-vmstat.node0.nr_slab_unreclaimable
    111406            -0.0%     111402        numa-vmstat.node0.nr_unevictable
     38314 ± 33%      +6.8%      40916 ± 12%  numa-vmstat.node0.nr_zone_active_anon
      4108 ±  4%     -19.6%       3302 ± 51%  numa-vmstat.node0.nr_zone_inactive_anon
    111406            -0.0%     111402        numa-vmstat.node0.nr_zone_unevictable
    724926 ±  8%      -6.9%     675065 ± 11%  numa-vmstat.node0.numa_hit
    131009            -0.1%     130856        numa-vmstat.node0.numa_interleave
    710309 ±  9%      -6.7%     663035 ± 11%  numa-vmstat.node0.numa_local
     14616 ± 33%     -17.7%      12029 ± 60%  numa-vmstat.node0.numa_other
     10355 ± 30%     +21.0%      12528 ± 27%  numa-vmstat.node1
     33151 ± 40%      -4.3%      31734 ± 16%  numa-vmstat.node1.nr_active_anon
     28010 ± 47%      -9.6%      25317 ± 16%  numa-vmstat.node1.nr_anon_pages
    116927 ±  2%      +1.8%     119045 ±  3%  numa-vmstat.node1.nr_file_pages
     50152            +0.0%      50152        numa-vmstat.node1.nr_free_cma
  16293163            -0.1%   16281185        numa-vmstat.node1.nr_free_pages
    385.75 ± 50%    +220.5%       1236 ±139%  numa-vmstat.node1.nr_inactive_anon
      6881 ±  7%      +8.6%       7475        numa-vmstat.node1.nr_kernel_stack
      2576           +12.8%       2907 ± 16%  numa-vmstat.node1.nr_mapped
    100.50 ±102%     -58.2%      42.00 ±173%  numa-vmstat.node1.nr_mlock
    527.00 ± 62%     +71.6%     904.50 ± 11%  numa-vmstat.node1.nr_page_table_pages
      5603 ± 55%     +37.7%       7717 ± 56%  numa-vmstat.node1.nr_shmem
      7329 ± 12%     +13.4%       8309 ± 18%  numa-vmstat.node1.nr_slab_reclaimable
     25419 ±  5%     +11.2%      28263 ±  3%  numa-vmstat.node1.nr_slab_unreclaimable
    111266            -0.0%     111255        numa-vmstat.node1.nr_unevictable
     33151 ± 40%      -4.3%      31734 ± 16%  numa-vmstat.node1.nr_zone_active_anon
    385.75 ± 50%    +220.5%       1236 ±139%  numa-vmstat.node1.nr_zone_inactive_anon
    111266            -0.0%     111255        numa-vmstat.node1.nr_zone_unevictable
    545817 ± 12%      -0.3%     544018 ± 14%  numa-vmstat.node1.numa_hit
    130594            +0.1%     130718        numa-vmstat.node1.numa_interleave
    408964 ± 18%      -1.1%     404358 ± 19%  numa-vmstat.node1.numa_local
    136853 ±  3%      +2.1%     139659 ±  5%  numa-vmstat.node1.numa_other
     26133 ± 24%      -1.0%      25884 ± 41%  sched_debug.cfs_rq:/.MIN_vruntime.avg
    950650 ± 23%      +1.2%     962090 ± 31%  sched_debug.cfs_rq:/.MIN_vruntime.max
      0.00            +0.0%       0.00        sched_debug.cfs_rq:/.MIN_vruntime.min
    141020 ± 22%      +3.2%     145486 ± 36%  sched_debug.cfs_rq:/.MIN_vruntime.stddev
     21791 ±  4%      +2.0%      22223 ±  9%  sched_debug.cfs_rq:/.load.avg
     86656           -19.7%      69575        sched_debug.cfs_rq:/.load.max
     30245           -11.9%      26637 ±  2%  sched_debug.cfs_rq:/.load.stddev
     29.05 ±  6%      +5.4%      30.63 ±  5%  sched_debug.cfs_rq:/.load_avg.avg
    232.03 ±  6%      -5.4%     219.54 ±  7%  sched_debug.cfs_rq:/.load_avg.max
     53.60 ± 11%      -0.7%      53.25 ± 13%  sched_debug.cfs_rq:/.load_avg.stddev
     26133 ± 24%      -1.0%      25884 ± 41%  sched_debug.cfs_rq:/.max_vruntime.avg
    950650 ± 23%      +1.2%     962090 ± 31%  sched_debug.cfs_rq:/.max_vruntime.max
      0.00            +0.0%       0.00        sched_debug.cfs_rq:/.max_vruntime.min
    141020 ± 22%      +3.2%     145486 ± 36%  sched_debug.cfs_rq:/.max_vruntime.stddev
    757888 ± 11%     +28.0%     969742 ±  9%  sched_debug.cfs_rq:/.min_vruntime.avg
   1142337 ± 12%     +21.3%    1385641 ±  9%  sched_debug.cfs_rq:/.min_vruntime.max
    340383 ± 15%     +60.4%     545887 ±  9%  sched_debug.cfs_rq:/.min_vruntime.min
    210789 ± 13%      +2.2%     215407 ±  8%  sched_debug.cfs_rq:/.min_vruntime.stddev
      0.35 ±  4%      +9.5%       0.39 ±  8%  sched_debug.cfs_rq:/.nr_running.avg
      1.00            +0.0%       1.00        sched_debug.cfs_rq:/.nr_running.max
      0.47            -0.6%       0.46        sched_debug.cfs_rq:/.nr_running.stddev
      3.26 ± 65%     +11.9%       3.65 ± 76%  sched_debug.cfs_rq:/.removed.load_avg.avg
    185.72 ±  8%     -26.5%     136.45 ± 58%  sched_debug.cfs_rq:/.removed.load_avg.max
     23.47 ± 33%      -9.0%      21.35 ± 65%  sched_debug.cfs_rq:/.removed.load_avg.stddev
    149.99 ± 65%     +12.2%     168.31 ± 76%  sched_debug.cfs_rq:/.removed.runnable_sum.avg
      8552 ±  9%     -26.3%       6299 ± 58%  sched_debug.cfs_rq:/.removed.runnable_sum.max
      1079 ± 33%      -8.7%     984.95 ± 65%  sched_debug.cfs_rq:/.removed.runnable_sum.stddev
      1.33 ± 56%     +19.4%       1.58 ± 82%  sched_debug.cfs_rq:/.removed.util_avg.avg
     84.93 ± 21%     -27.0%      62.02 ± 64%  sched_debug.cfs_rq:/.removed.util_avg.max
      9.87 ± 30%      -6.8%       9.19 ± 72%  sched_debug.cfs_rq:/.removed.util_avg.stddev
     13.40 ±  5%     +18.5%      15.88 ± 11%  sched_debug.cfs_rq:/.runnable_load_avg.avg
     43.87            -2.1%      42.94        sched_debug.cfs_rq:/.runnable_load_avg.max
     17.85            +3.7%      18.51 ±  2%  sched_debug.cfs_rq:/.runnable_load_avg.stddev
     21442 ±  5%      +4.9%      22495 ± 10%  sched_debug.cfs_rq:/.runnable_weight.avg
     85912           -20.8%      68077        sched_debug.cfs_rq:/.runnable_weight.max
     30069 ±  2%     -11.8%      26527 ±  3%  sched_debug.cfs_rq:/.runnable_weight.stddev
    349537 ±  7%     -15.3%     296038 ± 12%  sched_debug.cfs_rq:/.spread0.avg
    733987 ±  9%      -3.0%     711942 ± 11%  sched_debug.cfs_rq:/.spread0.max
    -67975           +88.1%    -127834        sched_debug.cfs_rq:/.spread0.min
    210791 ± 13%      +2.2%     215411 ±  8%  sched_debug.cfs_rq:/.spread0.stddev
    376.57 ±  3%     +11.4%     419.65 ±  3%  sched_debug.cfs_rq:/.util_avg.avg
      1013 ±  3%      -3.2%     981.61        sched_debug.cfs_rq:/.util_avg.max
    349.07            +6.0%     369.94        sched_debug.cfs_rq:/.util_avg.stddev
    230.99 ±  2%     +27.2%     293.92 ± 10%  sched_debug.cfs_rq:/.util_est_enqueued.avg
    852.23            +2.5%     873.57        sched_debug.cfs_rq:/.util_est_enqueued.max
    328.76 ±  2%      +7.8%     354.36 ±  2%  sched_debug.cfs_rq:/.util_est_enqueued.stddev
    321875 ± 15%     +12.2%     361029 ±  7%  sched_debug.cpu.avg_idle.avg
   1000000            +1.7%    1017141 ±  2%  sched_debug.cpu.avg_idle.max
      6003 ± 56%     +10.3%       6624 ± 42%  sched_debug.cpu.avg_idle.min
    367737 ± 10%      +7.9%     396924 ±  2%  sched_debug.cpu.avg_idle.stddev
    164232 ±  9%      +4.4%     171536 ±  7%  sched_debug.cpu.clock.avg
    164237 ±  9%      +4.4%     171541 ±  7%  sched_debug.cpu.clock.max
    164227 ±  9%      +4.4%     171531 ±  7%  sched_debug.cpu.clock.min
      3.09 ±  4%      -1.8%       3.03 ±  2%  sched_debug.cpu.clock.stddev
    164232 ±  9%      +4.4%     171536 ±  7%  sched_debug.cpu.clock_task.avg
    164237 ±  9%      +4.4%     171541 ±  7%  sched_debug.cpu.clock_task.max
    164227 ±  9%      +4.4%     171531 ±  7%  sched_debug.cpu.clock_task.min
      3.09 ±  4%      -1.8%       3.03 ±  2%  sched_debug.cpu.clock_task.stddev
     12.70 ±  2%     +12.6%      14.30        sched_debug.cpu.cpu_load[0].avg
     43.85            -1.8%      43.07        sched_debug.cpu.cpu_load[0].max
     17.67            +3.7%      18.33        sched_debug.cpu.cpu_load[0].stddev
     12.76 ±  2%     +13.4%      14.48        sched_debug.cpu.cpu_load[1].avg
     44.38 ±  8%      -5.5%      41.92        sched_debug.cpu.cpu_load[1].max
     15.37            +5.6%      16.24 ±  2%  sched_debug.cpu.cpu_load[1].stddev
     12.81           +13.8%      14.58        sched_debug.cpu.cpu_load[2].avg
     45.12 ± 17%     -10.8%      40.23        sched_debug.cpu.cpu_load[2].max
     14.84 ±  4%      +4.6%      15.53        sched_debug.cpu.cpu_load[2].stddev
     12.81           +14.9%      14.72        sched_debug.cpu.cpu_load[3].avg
     45.01 ± 11%      -6.1%      42.24 ±  3%  sched_debug.cpu.cpu_load[3].max
     14.59 ±  3%      +5.9%      15.45        sched_debug.cpu.cpu_load[3].stddev
     13.10           +15.8%      15.16        sched_debug.cpu.cpu_load[4].avg
     63.50 ± 10%      -0.9%      62.96 ±  4%  sched_debug.cpu.cpu_load[4].max
     16.43 ±  4%      +7.0%      17.58        sched_debug.cpu.cpu_load[4].stddev
    617.86 ±  3%      +9.5%     676.70        sched_debug.cpu.curr->pid.avg
      5226 ±  7%      +3.5%       5406 ±  5%  sched_debug.cpu.curr->pid.max
    994.97 ±  2%      +1.2%       1007 ±  2%  sched_debug.cpu.curr->pid.stddev
     19393 ±  3%      -0.5%      19287        sched_debug.cpu.load.avg
    133909 ± 60%     -48.0%      69575        sched_debug.cpu.load.max
     32436 ± 19%     -20.2%      25887        sched_debug.cpu.load.stddev
    500000            +0.1%     500448        sched_debug.cpu.max_idle_balance_cost.avg
    500000            +2.7%     513396 ±  4%  sched_debug.cpu.max_idle_balance_cost.max
    500000            +0.0%     500000        sched_debug.cpu.max_idle_balance_cost.min
      4294            +0.0%       4294        sched_debug.cpu.next_balance.avg
      4294            +0.0%       4294        sched_debug.cpu.next_balance.max
      4294            +0.0%       4294        sched_debug.cpu.next_balance.min
      0.00 ±  7%      -0.6%       0.00 ±  7%  sched_debug.cpu.next_balance.stddev
    140354 ± 10%      +5.3%     147746 ±  8%  sched_debug.cpu.nr_load_updates.avg
    147133 ± 10%      +5.3%     154878 ±  8%  sched_debug.cpu.nr_load_updates.max
    138176 ± 11%      +5.3%     145481 ±  8%  sched_debug.cpu.nr_load_updates.min
      1184 ± 15%      -0.2%       1181 ±  4%  sched_debug.cpu.nr_load_updates.stddev
      0.31 ±  2%      +9.7%       0.34        sched_debug.cpu.nr_running.avg
      1.00            +0.0%       1.00        sched_debug.cpu.nr_running.max
      0.46            +0.4%       0.46        sched_debug.cpu.nr_running.stddev
   5058918 ± 11%     -29.7%    3554884 ±  8%  sched_debug.cpu.nr_switches.avg
   7455016 ± 11%     -32.1%    5065543 ±  8%  sched_debug.cpu.nr_switches.max
   2132318 ± 13%      -8.7%    1946837 ± 11%  sched_debug.cpu.nr_switches.min
   1418043 ± 14%     -43.1%     807437 ±  7%  sched_debug.cpu.nr_switches.stddev
      0.01 ± 10%     +28.2%       0.01 ± 41%  sched_debug.cpu.nr_uninterruptible.avg
      9.00 ± 32%      -1.9%       8.83 ± 17%  sched_debug.cpu.nr_uninterruptible.max
     -8.50           +11.3%      -9.46        sched_debug.cpu.nr_uninterruptible.min
      3.05 ± 14%      -2.5%       2.97 ± 15%  sched_debug.cpu.nr_uninterruptible.stddev
    164227 ±  9%      +4.4%     171531 ±  7%  sched_debug.cpu_clk
    996147            +0.0%     996147        sched_debug.dl_rq:.dl_bw->bw.avg
    996147            +0.0%     996147        sched_debug.dl_rq:.dl_bw->bw.max
    996147            +0.0%     996147        sched_debug.dl_rq:.dl_bw->bw.min
 4.295e+09            +0.0%  4.295e+09        sched_debug.jiffies
    164227 ±  9%      +4.4%     171531 ±  7%  sched_debug.ktime
    950.00            +0.0%     950.00        sched_debug.rt_rq:/.rt_runtime.avg
    950.00            +0.5%     955.00        sched_debug.rt_rq:/.rt_runtime.max
    950.00            -0.1%     948.92        sched_debug.rt_rq:/.rt_runtime.min
      0.00 ±120%     -69.0%       0.00 ±173%  sched_debug.rt_rq:/.rt_time.avg
      0.01 ±137%      -1.4%       0.01 ±173%  sched_debug.rt_rq:/.rt_time.max
      0.00 ±124%     -10.1%       0.00 ±173%  sched_debug.rt_rq:/.rt_time.stddev
    164984 ±  9%      +4.7%     172783 ±  7%  sched_debug.sched_clk
      1.00            +0.0%       1.00        sched_debug.sched_clock_stable()
   4118331            +0.0%    4118331        sched_debug.sysctl_sched.sysctl_sched_features
     24.00            +0.0%      24.00        sched_debug.sysctl_sched.sysctl_sched_latency
      3.00            +0.0%       3.00        sched_debug.sysctl_sched.sysctl_sched_min_granularity
      1.00            +0.0%       1.00        sched_debug.sysctl_sched.sysctl_sched_tunable_scaling
      4.00            +0.0%       4.00        sched_debug.sysctl_sched.sysctl_sched_wakeup_granularity
     18.26            -6.7       11.57        perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.ip_output.ip_queue_xmit.tcp_transmit_skb
     18.26            -6.7       11.58        perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output.ip_queue_xmit
     18.14            -6.6       11.49        perf-profile.calltrace.cycles-pp.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output
     18.13            -6.6       11.49        perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2
     17.88            -6.5       11.35        perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.do_softirq_own_stack.do_softirq.__local_bh_enable_ip
     17.62            -6.4       11.20        perf-profile.calltrace.cycles-pp.process_backlog.net_rx_action.__softirqentry_text_start.do_softirq_own_stack.do_softirq
     17.26            -6.3       10.97        perf-profile.calltrace.cycles-pp.__netif_receive_skb_core.process_backlog.net_rx_action.__softirqentry_text_start.do_softirq_own_stack
     16.98            -6.2       10.81        perf-profile.calltrace.cycles-pp.ip_rcv.__netif_receive_skb_core.process_backlog.net_rx_action.__softirqentry_text_start
     16.59            -6.0       10.59        perf-profile.calltrace.cycles-pp.ip_local_deliver.ip_rcv.__netif_receive_skb_core.process_backlog.net_rx_action
     16.47            -5.9       10.52        perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_core.process_backlog
     16.32            -5.9       10.43        perf-profile.calltrace.cycles-pp.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_core
     42.22            -5.7       36.53        perf-profile.calltrace.cycles-pp.secondary_startup_64
     41.67            -5.6       36.09        perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
     41.67            -5.6       36.09        perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
     41.67            -5.6       36.09        perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
     12.36            -4.7        7.63        perf-profile.calltrace.cycles-pp.ip_output.ip_queue_xmit.tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames
     12.20            -4.7        7.50        perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.ip_queue_xmit.tcp_transmit_skb.tcp_write_xmit
     12.04            -4.2        7.81        perf-profile.calltrace.cycles-pp.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
     10.02            -4.0        6.01        perf-profile.calltrace.cycles-pp.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
      9.98            -4.0        5.97        perf-profile.calltrace.cycles-pp.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
      9.82            -3.9        5.87        perf-profile.calltrace.cycles-pp.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
      9.68            -3.9        5.78        perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64
      9.78            -3.7        6.06        perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver.ip_rcv
     36.84            -3.7       33.18        perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
     10.39            -3.6        6.77        perf-profile.calltrace.cycles-pp.sock_close.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64
     10.37            -3.6        6.76        perf-profile.calltrace.cycles-pp.__sock_release.sock_close.__fput.task_work_run.exit_to_usermode_loop
     10.24            -3.6        6.68        perf-profile.calltrace.cycles-pp.inet_release.__sock_release.sock_close.__fput.task_work_run
      8.68            -3.5        5.16        perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto
      9.97            -3.5        6.50        perf-profile.calltrace.cycles-pp.tcp_close.inet_release.__sock_release.sock_close.__fput
     11.08            -3.5        7.62        perf-profile.calltrace.cycles-pp.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe
     11.06            -3.5        7.61        perf-profile.calltrace.cycles-pp.__sys_connect.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe
     35.81            -3.4       32.42        perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
     10.72            -3.4        7.35        perf-profile.calltrace.cycles-pp.inet_stream_connect.__sys_connect.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe
     10.65            -3.3        7.31        perf-profile.calltrace.cycles-pp.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect.do_syscall_64
      7.64            -3.1        4.50        perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
      7.63            -3.1        4.50        perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
      7.67            -2.7        4.98        perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_close.inet_release.__sock_release
      7.68            -2.7        4.98        perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_close.inet_release.__sock_release.sock_close
      6.13            -2.5        3.59        perf-profile.calltrace.cycles-pp.tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg
      5.89            -2.5        3.43        perf-profile.calltrace.cycles-pp.ip_queue_xmit.tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked
      6.92            -2.4        4.50        perf-profile.calltrace.cycles-pp.tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_close.inet_release
      6.64            -2.3        4.31        perf-profile.calltrace.cycles-pp.ip_queue_xmit.tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_close
      5.31            -2.2        3.08        perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver
      5.96            -2.1        3.90        perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.92            -2.0        3.88        perf-profile.calltrace.cycles-pp.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.69            -2.0        3.72        perf-profile.calltrace.cycles-pp.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.62            -1.9        3.68        perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64
      5.25            -1.8        3.48        perf-profile.calltrace.cycles-pp.release_sock.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect
      5.22            -1.8        3.47        perf-profile.calltrace.cycles-pp.__release_sock.release_sock.__inet_stream_connect.inet_stream_connect.__sys_connect
      5.16            -1.7        3.43        perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.__inet_stream_connect.inet_stream_connect
      4.69            -1.6        3.11        perf-profile.calltrace.cycles-pp.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock.release_sock.__inet_stream_connect
      5.20            -1.5        3.72        perf-profile.calltrace.cycles-pp.tcp_v4_connect.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect
      4.43            -1.5        2.95        perf-profile.calltrace.cycles-pp.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver
      4.09            -1.4        2.70        perf-profile.calltrace.cycles-pp.tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock.release_sock
      2.23            -1.4        0.86 ±  3%  perf-profile.calltrace.cycles-pp.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_local_deliver_finish
      3.99            -1.4        2.63        perf-profile.calltrace.cycles-pp.ip_queue_xmit.tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock
      3.91            -1.3        2.58        perf-profile.calltrace.cycles-pp.ip_output.ip_queue_xmit.tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv
      3.55            -1.3        2.25        perf-profile.calltrace.cycles-pp.sk_wait_data.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
      3.79            -1.3        2.50        perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.ip_queue_xmit.tcp_transmit_skb.tcp_rcv_state_process
      4.32            -1.3        3.03        perf-profile.calltrace.cycles-pp.tcp_connect.tcp_v4_connect.__inet_stream_connect.inet_stream_connect.__sys_connect
      2.53            -1.1        1.42        perf-profile.calltrace.cycles-pp.wait_woken.sk_wait_data.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
      3.66            -1.1        2.57        perf-profile.calltrace.cycles-pp.tcp_transmit_skb.tcp_connect.tcp_v4_connect.__inet_stream_connect.inet_stream_connect
      2.58 ±  2%      -1.0        1.55        perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
      2.55 ±  2%      -1.0        1.52        perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
      2.31            -1.0        1.30        perf-profile.calltrace.cycles-pp.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg.inet_recvmsg
      3.42 ±  2%      -1.0        2.42        perf-profile.calltrace.cycles-pp.ip_queue_xmit.tcp_transmit_skb.tcp_connect.tcp_v4_connect.__inet_stream_connect
      3.10            -1.0        2.10        perf-profile.calltrace.cycles-pp.__x64_sys_accept.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.29            -1.0        1.29        perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg
      3.09            -1.0        2.10        perf-profile.calltrace.cycles-pp.__sys_accept4.__x64_sys_accept.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.25            -1.0        1.28        perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.wait_woken.sk_wait_data
      3.03 ±  2%      -0.9        2.09        perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.ip_queue_xmit.tcp_transmit_skb.tcp_connect
      3.05 ±  2%      -0.9        2.13        perf-profile.calltrace.cycles-pp.ip_output.ip_queue_xmit.tcp_transmit_skb.tcp_connect.tcp_v4_connect
      0.78 ±  2%      -0.8        0.00        perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg
      0.75            -0.7        0.00        perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.schedule_timeout.wait_woken
      0.75 ±  2%      -0.7        0.00        perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg
      0.74 ±  2%      -0.7        0.00        perf-profile.calltrace.cycles-pp.security_socket_bind.__sys_bind.__x64_sys_bind.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.73 ±  2%      -0.7        0.00        perf-profile.calltrace.cycles-pp.ip_queue_xmit.tcp_transmit_skb.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv
      0.72 ±  2%      -0.7        0.00        perf-profile.calltrace.cycles-pp.selinux_socket_bind.security_socket_bind.__sys_bind.__x64_sys_bind.do_syscall_64
      0.71 ±  2%      -0.7        0.00        perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv
      1.87 ±  2%      -0.7        1.19        perf-profile.calltrace.cycles-pp.tcp_child_process.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver.ip_rcv
      0.67 ±  3%      -0.7        0.00        perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv
      0.65 ±  2%      -0.7        0.00        perf-profile.calltrace.cycles-pp.destroy_inode.__dentry_kill.dentry_kill.dput.__fput
      0.65            -0.6        0.00        perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_local_deliver_finish
      0.64 ±  2%      -0.6        0.00        perf-profile.calltrace.cycles-pp.__destroy_inode.destroy_inode.__dentry_kill.dentry_kill.dput
      0.64 ±  2%      -0.6        0.00        perf-profile.calltrace.cycles-pp.sock_alloc_file.__sys_accept4.__x64_sys_accept.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.63            -0.6        0.00        perf-profile.calltrace.cycles-pp.tcp_send_fin.tcp_close.inet_release.__sock_release.sock_close
      0.63 ±  2%      -0.6        0.00        perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_data_queue.tcp_rcv_established
      0.62 ±  2%      -0.6        0.00        perf-profile.calltrace.cycles-pp.tcp_create_openreq_child.tcp_v4_syn_recv_sock.tcp_check_req.tcp_v4_rcv.ip_local_deliver_finish
      0.61 ±  2%      -0.6        0.00        perf-profile.calltrace.cycles-pp.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_data_queue
      0.61 ±  2%      -0.6        0.00        perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
      0.60 ±  2%      -0.6        0.00        perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry
      1.81 ±  2%      -0.6        1.22        perf-profile.calltrace.cycles-pp.inet_accept.__sys_accept4.__x64_sys_accept.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.59            -0.6        0.00        perf-profile.calltrace.cycles-pp.inet_create.__sock_create.__sys_socket.__x64_sys_socket.do_syscall_64
      0.58 ±  2%      -0.6        0.00        perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
      1.71            -0.6        1.13        perf-profile.calltrace.cycles-pp.tcp_data_queue.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_local_deliver_finish
      1.67            -0.6        1.10        perf-profile.calltrace.cycles-pp.tcp_fin.tcp_data_queue.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv
      0.56 ±  2%      -0.6        0.00        perf-profile.calltrace.cycles-pp.tcp_v4_send_synack.tcp_conn_request.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv
      0.56 ±  4%      -0.6        0.00        perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule_idle.do_idle
      0.55 ±  2%      -0.6        0.00        perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock
      0.55 ±  2%      -0.6        0.00        perf-profile.calltrace.cycles-pp.__kfree_skb.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
      1.77            -0.5        1.23        perf-profile.calltrace.cycles-pp.__x64_sys_socket.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.54 ±  3%      -0.5        0.00        perf-profile.calltrace.cycles-pp.switch_mm.__schedule.schedule.schedule_timeout.wait_woken
      1.75            -0.5        1.22        perf-profile.calltrace.cycles-pp.__sys_socket.__x64_sys_socket.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.54 ±  2%      -0.5        0.00        perf-profile.calltrace.cycles-pp.__inet_bind.__sys_bind.__x64_sys_bind.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.37            -0.5        0.83        perf-profile.calltrace.cycles-pp.__x64_sys_bind.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.53 ±  3%      -0.5        0.00        perf-profile.calltrace.cycles-pp.security_inode_free.__destroy_inode.destroy_inode.__dentry_kill.dentry_kill
      1.35            -0.5        0.82        perf-profile.calltrace.cycles-pp.__sys_bind.__x64_sys_bind.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.53 ±  2%      -0.5        0.00        perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
      0.53 ±  2%      -0.5        0.00        perf-profile.calltrace.cycles-pp.ip_output.ip_queue_xmit.tcp_transmit_skb.tcp_rcv_established.tcp_v4_do_rcv
      1.70            -0.5        1.18        perf-profile.calltrace.cycles-pp.tcp_conn_request.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_local_deliver_finish
      1.35 ±  2%      -0.5        0.83        perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_child_process.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver
      0.52            -0.5        0.00        perf-profile.calltrace.cycles-pp.sel_netport_sid.selinux_socket_bind.security_socket_bind.__sys_bind.__x64_sys_bind
      0.52            -0.5        0.00        perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable
      1.27 ±  2%      -0.5        0.78 ±  2%  perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_child_process.tcp_v4_rcv.ip_local_deliver_finish
      1.42            -0.5        0.94        perf-profile.calltrace.cycles-pp.inet_csk_accept.inet_accept.__sys_accept4.__x64_sys_accept.do_syscall_64
      1.20 ±  2%      -0.5        0.74        perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_child_process.tcp_v4_rcv
      0.85 ±  3%      -0.5        0.39 ± 57%  perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule_idle.do_idle.cpu_startup_entry
      1.19 ±  2%      -0.5        0.73        perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_child_process
      1.36            -0.5        0.91        perf-profile.calltrace.cycles-pp.syscall_return_via_sysret
      1.17 ±  2%      -0.4        0.72        perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable
      1.27            -0.4        0.82        perf-profile.calltrace.cycles-pp.dput.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64
      1.32 ±  3%      -0.4        0.88        perf-profile.calltrace.cycles-pp.__entry_trampoline_start
      1.22            -0.4        0.79        perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_local_deliver_finish
      1.35            -0.4        0.94        perf-profile.calltrace.cycles-pp.tcp_check_req.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver.ip_rcv
      1.17            -0.4        0.76        perf-profile.calltrace.cycles-pp.dentry_kill.dput.__fput.task_work_run.exit_to_usermode_loop
      1.15            -0.4        0.74        perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv
      1.22            -0.4        0.81 ±  2%  perf-profile.calltrace.cycles-pp.__sock_create.__sys_socket.__x64_sys_socket.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.11            -0.4        0.71 ±  3%  perf-profile.calltrace.cycles-pp.tcp_fin.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv
      1.14            -0.4        0.74        perf-profile.calltrace.cycles-pp.__dentry_kill.dentry_kill.dput.__fput.task_work_run
      0.39 ± 57%      -0.4        0.00        perf-profile.calltrace.cycles-pp.__indirect_thunk_start
      1.06            -0.4        0.67 ±  2%  perf-profile.calltrace.cycles-pp.sock_def_wakeup.tcp_fin.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv
      1.08            -0.4        0.69        perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv
      1.13            -0.4        0.74 ±  2%  perf-profile.calltrace.cycles-pp.tcp_v4_syn_recv_sock.tcp_check_req.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver
      1.05 ±  2%      -0.4        0.67        perf-profile.calltrace.cycles-pp.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_rcv_established
      0.91            -0.4        0.53        perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule_idle.do_idle.cpu_startup_entry
      0.99            -0.4        0.63 ±  2%  perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_wakeup.tcp_fin.tcp_data_queue.tcp_rcv_established
      0.92            -0.4        0.56 ±  2%  perf-profile.calltrace.cycles-pp.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto
      0.91            -0.4        0.55 ±  2%  perf-profile.calltrace.cycles-pp.tcp_transmit_skb.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_local_deliver_finish
      0.87 ±  2%      -0.3        0.53 ±  2%  perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto
      0.93            -0.3        0.59 ±  2%  perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_wakeup.tcp_fin.tcp_data_queue
      0.89            -0.3        0.57 ±  2%  perf-profile.calltrace.cycles-pp.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_wakeup.tcp_fin
      0.45 ± 58%      -0.3        0.13 ±173%  perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64
      0.44 ± 58%      -0.3        0.13 ±173%  perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_kernel
      1.02            -0.3        0.71        perf-profile.calltrace.cycles-pp.release_sock.tcp_close.inet_release.__sock_release.sock_close
      0.82            -0.3        0.51        perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
      0.98            -0.3        0.68        perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_close.inet_release.__sock_release
      0.89 ±  2%      -0.3        0.61        perf-profile.calltrace.cycles-pp.schedule_timeout.inet_csk_accept.inet_accept.__sys_accept4.__x64_sys_accept
      0.88 ±  2%      -0.3        0.61        perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.inet_csk_accept.inet_accept.__sys_accept4
      0.93            -0.3        0.66        perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_close.inet_release
      0.93            -0.3        0.66        perf-profile.calltrace.cycles-pp.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock.release_sock.tcp_close
      0.87 ±  2%      -0.3        0.60        perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.inet_csk_accept.inet_accept
      0.26 ±100%      -0.3        0.00        perf-profile.calltrace.cycles-pp.inet_csk_clone_lock.tcp_create_openreq_child.tcp_v4_syn_recv_sock.tcp_check_req.tcp_v4_rcv
      0.48 ± 58%      -0.2        0.26 ±100%  perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64
      0.48 ± 58%      -0.2        0.26 ±100%  perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_kernel.secondary_startup_64
      0.48 ± 58%      -0.2        0.26 ±100%  perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64
      0.00            +0.1        0.12 ±173%  perf-profile.calltrace.cycles-pp.release_sock.sk_wait_data.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
      0.00            +1.5        1.51        perf-profile.calltrace.cycles-pp.mnt_get_count.mntput_no_expire.task_work_run.exit_to_usermode_loop.do_syscall_64
     47.12            +9.2       56.32        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
     46.93            +9.3       56.21        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
     12.24           +21.3       33.53        perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
     12.21           +21.3       33.51        perf-profile.calltrace.cycles-pp.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00           +23.5       23.48        perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.mntput_no_expire.task_work_run.exit_to_usermode_loop
      0.00           +23.8       23.77        perf-profile.calltrace.cycles-pp._raw_spin_lock.mntput_no_expire.task_work_run.exit_to_usermode_loop.do_syscall_64
      0.00           +25.5       25.47        perf-profile.calltrace.cycles-pp.mntput_no_expire.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
     22.48            -7.9       14.54        perf-profile.children.cycles-pp.tcp_transmit_skb
     21.39            -7.5       13.84        perf-profile.children.cycles-pp.ip_queue_xmit
     20.72            -7.3       13.39        perf-profile.children.cycles-pp.ip_output
     19.92            -7.1       12.82        perf-profile.children.cycles-pp.ip_finish_output2
     19.03            -6.8       12.26        perf-profile.children.cycles-pp.__softirqentry_text_start
     18.65            -6.7       11.92        perf-profile.children.cycles-pp.__local_bh_enable_ip
     18.48            -6.7       11.82        perf-profile.children.cycles-pp.do_softirq
     18.34            -6.6       11.72        perf-profile.children.cycles-pp.do_softirq_own_stack
     18.14            -6.5       11.61        perf-profile.children.cycles-pp.net_rx_action
     17.88            -6.4       11.46        perf-profile.children.cycles-pp.process_backlog
     17.46            -6.3       11.18        perf-profile.children.cycles-pp.__netif_receive_skb_core
     17.21            -6.2       11.06        perf-profile.children.cycles-pp.ip_rcv
     16.82            -6.0       10.84        perf-profile.children.cycles-pp.ip_local_deliver
     17.08            -6.0       11.11        perf-profile.children.cycles-pp.tcp_v4_do_rcv
     16.67            -5.9       10.73        perf-profile.children.cycles-pp.ip_local_deliver_finish
     16.58            -5.9       10.69        perf-profile.children.cycles-pp.tcp_v4_rcv
     15.33            -5.8        9.49        perf-profile.children.cycles-pp.tcp_write_xmit
     15.32            -5.8        9.49        perf-profile.children.cycles-pp.__tcp_push_pending_frames
     42.27            -5.7       36.56        perf-profile.children.cycles-pp.do_idle
     42.23            -5.7       36.53        perf-profile.children.cycles-pp.cpu_startup_entry
     42.22            -5.7       36.53        perf-profile.children.cycles-pp.secondary_startup_64
     41.67            -5.6       36.09        perf-profile.children.cycles-pp.start_secondary
     12.07            -4.2        7.83        perf-profile.children.cycles-pp.__fput
     10.00            -4.0        5.99        perf-profile.children.cycles-pp.__sys_sendto
     10.02            -4.0        6.01        perf-profile.children.cycles-pp.__x64_sys_sendto
      9.83            -4.0        5.88        perf-profile.children.cycles-pp.sock_sendmsg
      9.69            -3.9        5.78        perf-profile.children.cycles-pp.tcp_sendmsg
     37.39            -3.8       33.63        perf-profile.children.cycles-pp.cpuidle_enter_state
     10.39            -3.6        6.77        perf-profile.children.cycles-pp.sock_close
     10.38            -3.6        6.77        perf-profile.children.cycles-pp.__sock_release
     10.25            -3.6        6.69        perf-profile.children.cycles-pp.inet_release
      8.70            -3.5        5.17        perf-profile.children.cycles-pp.tcp_sendmsg_locked
     36.34            -3.5       32.85        perf-profile.children.cycles-pp.intel_idle
     10.00            -3.5        6.51        perf-profile.children.cycles-pp.tcp_close
     10.52            -3.5        7.04        perf-profile.children.cycles-pp.tcp_rcv_state_process
     11.08            -3.5        7.62        perf-profile.children.cycles-pp.__x64_sys_connect
     11.07            -3.4        7.62        perf-profile.children.cycles-pp.__sys_connect
     10.73            -3.4        7.36        perf-profile.children.cycles-pp.inet_stream_connect
     10.66            -3.3        7.32        perf-profile.children.cycles-pp.__inet_stream_connect
      8.02            -2.5        5.52        perf-profile.children.cycles-pp.release_sock
      6.45            -2.5        4.00        perf-profile.children.cycles-pp.tcp_rcv_established
      7.49            -2.3        5.15        perf-profile.children.cycles-pp.__release_sock
      5.75            -2.3        3.46        perf-profile.children.cycles-pp.__schedule
      5.97            -2.1        3.91        perf-profile.children.cycles-pp.__x64_sys_recvfrom
      5.94            -2.1        3.89        perf-profile.children.cycles-pp.__sys_recvfrom
      5.70            -2.0        3.73        perf-profile.children.cycles-pp.inet_recvmsg
      5.64            -1.9        3.69        perf-profile.children.cycles-pp.tcp_recvmsg
      4.27            -1.8        2.47        perf-profile.children.cycles-pp.__wake_up_common_lock
      4.16            -1.7        2.45        perf-profile.children.cycles-pp.tcp_data_queue
      3.97            -1.7        2.27        perf-profile.children.cycles-pp.__wake_up_common
      3.82            -1.6        2.20        perf-profile.children.cycles-pp.try_to_wake_up
      3.42            -1.5        1.91        perf-profile.children.cycles-pp.sock_def_readable
      5.21            -1.5        3.73        perf-profile.children.cycles-pp.tcp_v4_connect
      3.56            -1.3        2.25        perf-profile.children.cycles-pp.sk_wait_data
      4.33            -1.3        3.03        perf-profile.children.cycles-pp.tcp_connect
      3.21            -1.3        1.92        perf-profile.children.cycles-pp.schedule
      2.56            -1.1        1.44        perf-profile.children.cycles-pp.wait_woken
      2.60 ±  2%      -1.0        1.56        perf-profile.children.cycles-pp.schedule_idle
      3.11            -1.0        2.10        perf-profile.children.cycles-pp.__x64_sys_accept
      3.10            -1.0        2.10        perf-profile.children.cycles-pp.__sys_accept4
      2.79            -1.0        1.82        perf-profile.children.cycles-pp.tcp_fin
      1.76            -0.7        1.03        perf-profile.children.cycles-pp.ttwu_do_activate
      2.04            -0.7        1.35        perf-profile.children.cycles-pp.tcp_ack
      1.88 ±  2%      -0.7        1.19        perf-profile.children.cycles-pp.tcp_child_process
      1.70            -0.7        1.03        perf-profile.children.cycles-pp.switch_mm_irqs_off
      1.63            -0.7        0.97        perf-profile.children.cycles-pp.enqueue_task_fair
      1.82 ±  2%      -0.6        1.23        perf-profile.children.cycles-pp.inet_accept
      1.44            -0.6        0.86 ±  2%  perf-profile.children.cycles-pp.enqueue_entity
      1.78            -0.6        1.23        perf-profile.children.cycles-pp.__x64_sys_socket
      1.75            -0.5        1.22        perf-profile.children.cycles-pp.__sys_socket
      1.35            -0.5        0.82        perf-profile.children.cycles-pp.__sys_bind
      1.37            -0.5        0.83        perf-profile.children.cycles-pp.__x64_sys_bind
      1.70            -0.5        1.18        perf-profile.children.cycles-pp.tcp_conn_request
      1.69            -0.5        1.18        perf-profile.children.cycles-pp.__dev_queue_xmit
      1.14 ±  2%      -0.5        0.64        perf-profile.children.cycles-pp.pick_next_task_fair
      1.43            -0.5        0.95        perf-profile.children.cycles-pp.inet_csk_accept
      1.20 ±  2%      -0.5        0.74        perf-profile.children.cycles-pp.autoremove_wake_function
      1.37            -0.5        0.92        perf-profile.children.cycles-pp.syscall_return_via_sysret
      1.28            -0.4        0.83        perf-profile.children.cycles-pp.dput
      1.33 ±  3%      -0.4        0.89        perf-profile.children.cycles-pp.__entry_trampoline_start
      1.51 ±  2%      -0.4        1.08 ±  2%  perf-profile.children.cycles-pp._raw_spin_lock_bh
      1.21            -0.4        0.78 ±  2%  perf-profile.children.cycles-pp.sock_def_wakeup
      1.18            -0.4        0.76        perf-profile.children.cycles-pp.dentry_kill
      1.36            -0.4        0.95        perf-profile.children.cycles-pp.tcp_check_req
      1.09            -0.4        0.68 ±  2%  perf-profile.children.cycles-pp.__kfree_skb
      1.15            -0.4        0.74        perf-profile.children.cycles-pp.__dentry_kill
      1.08            -0.4        0.68 ±  2%  perf-profile.children.cycles-pp.dequeue_task_fair
      1.18            -0.4        0.77 ±  2%  perf-profile.children.cycles-pp.tcp_clean_rtx_queue
      1.20            -0.4        0.80        perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      1.22            -0.4        0.82        perf-profile.children.cycles-pp.__sock_create
      1.14            -0.4        0.75 ±  2%  perf-profile.children.cycles-pp.tcp_v4_syn_recv_sock
      0.99            -0.4        0.60 ±  2%  perf-profile.children.cycles-pp.dequeue_entity
      1.12            -0.4        0.76 ±  2%  perf-profile.children.cycles-pp.__alloc_skb
      1.20 ±  2%      -0.4        0.84        perf-profile.children.cycles-pp.dev_hard_start_xmit
      1.14            -0.3        0.79        perf-profile.children.cycles-pp.loopback_xmit
      0.70 ±  2%      -0.3        0.35 ±  3%  perf-profile.children.cycles-pp.select_task_rq_fair
      0.95 ±  2%      -0.3        0.62        perf-profile.children.cycles-pp.tcp_filter
      0.95 ±  2%      -0.3        0.62        perf-profile.children.cycles-pp.sk_filter_trim_cap
      0.85            -0.3        0.53        perf-profile.children.cycles-pp.menu_select
      0.80 ±  2%      -0.3        0.48        perf-profile.children.cycles-pp.mod_timer
      0.84 ±  2%      -0.3        0.53        perf-profile.children.cycles-pp.update_load_avg
      0.90 ±  2%      -0.3        0.59        perf-profile.children.cycles-pp.security_sock_rcv_skb
      0.92 ±  2%      -0.3        0.61        perf-profile.children.cycles-pp.selinux_socket_sock_rcv_skb
      1.06            -0.3        0.75        perf-profile.children.cycles-pp.sock_alloc_file
      0.78            -0.3        0.48 ±  2%  perf-profile.children.cycles-pp.sk_reset_timer
      0.81 ±  3%      -0.3        0.51        perf-profile.children.cycles-pp.switch_mm
      0.91            -0.3        0.61 ±  2%  perf-profile.children.cycles-pp.__inet_lookup_established
      0.73 ±  2%      -0.3        0.44 ±  3%  perf-profile.children.cycles-pp.selinux_socket_bind
      0.74 ±  2%      -0.3        0.45 ±  2%  perf-profile.children.cycles-pp.security_socket_bind
      0.68            -0.3        0.40 ±  2%  perf-profile.children.cycles-pp.inet_csk_destroy_sock
      1.07            -0.3        0.79 ±  2%  perf-profile.children.cycles-pp.lock_sock_nested
      0.88            -0.3        0.61        perf-profile.children.cycles-pp.nf_hook_slow
      0.63 ±  3%      -0.3        0.38 ±  2%  perf-profile.children.cycles-pp.__slab_free
      0.65 ±  2%      -0.2        0.41 ±  2%  perf-profile.children.cycles-pp.destroy_inode
      0.65 ±  2%      -0.2        0.41 ±  2%  perf-profile.children.cycles-pp.__destroy_inode
      0.55            -0.2        0.31 ±  5%  perf-profile.children.cycles-pp.update_curr
      0.55 ±  3%      -0.2        0.31        perf-profile.children.cycles-pp.__inet_bind
      0.72 ±  2%      -0.2        0.49 ±  2%  perf-profile.children.cycles-pp.__indirect_thunk_start
      0.57 ±  3%      -0.2        0.34        perf-profile.children.cycles-pp.load_new_mm_cr3
      0.53 ±  3%      -0.2        0.30 ±  3%  perf-profile.children.cycles-pp.security_inode_free
      0.64            -0.2        0.41        perf-profile.children.cycles-pp.tcp_send_fin
      0.70 ±  2%      -0.2        0.48 ±  2%  perf-profile.children.cycles-pp.sel_netport_sid
      0.70 ±  2%      -0.2        0.47        perf-profile.children.cycles-pp.selinux_ip_postroute
      0.59 ±  4%      -0.2        0.37        perf-profile.children.cycles-pp.set_next_entity
      0.56            -0.2        0.34 ±  3%  perf-profile.children.cycles-pp.skb_release_all
      0.55            -0.2        0.34 ±  2%  perf-profile.children.cycles-pp.skb_release_head_state
      0.53 ±  2%      -0.2        0.32        perf-profile.children.cycles-pp.ttwu_do_wakeup
      0.52 ±  2%      -0.2        0.30        perf-profile.children.cycles-pp.check_preempt_curr
      0.61            -0.2        0.40 ±  2%  perf-profile.children.cycles-pp.kmem_cache_alloc
      0.61            -0.2        0.40 ±  5%  perf-profile.children.cycles-pp.sched_clock_cpu
      0.61 ±  2%      -0.2        0.40        perf-profile.children.cycles-pp.kmem_cache_free
      0.63 ±  2%      -0.2        0.42 ±  3%  perf-profile.children.cycles-pp.tcp_create_openreq_child
      0.61 ±  2%      -0.2        0.40 ±  2%  perf-profile.children.cycles-pp.sk_stream_alloc_skb
      0.46            -0.2        0.26        perf-profile.children.cycles-pp.__sk_mem_reduce_allocated
      0.56            -0.2        0.36 ±  5%  perf-profile.children.cycles-pp.sched_clock
      0.59            -0.2        0.39 ±  2%  perf-profile.children.cycles-pp.sock_alloc
      0.55 ±  2%      -0.2        0.35 ±  2%  perf-profile.children.cycles-pp.d_instantiate
      0.59            -0.2        0.41        perf-profile.children.cycles-pp.inet_create
      0.48 ±  2%      -0.2        0.30 ±  2%  perf-profile.children.cycles-pp.sk_forced_mem_schedule
      0.54 ±  2%      -0.2        0.35 ±  4%  perf-profile.children.cycles-pp.netif_rx_internal
      0.47 ±  4%      -0.2        0.29 ±  3%  perf-profile.children.cycles-pp.tcp_event_new_data_sent
      0.47            -0.2        0.29        perf-profile.children.cycles-pp.percpu_counter_add_batch
      0.53 ±  2%      -0.2        0.34 ±  4%  perf-profile.children.cycles-pp.native_sched_clock
      0.54            -0.2        0.37 ±  2%  perf-profile.children.cycles-pp.new_inode_pseudo
      0.41 ±  2%      -0.2        0.23        perf-profile.children.cycles-pp.security_d_instantiate
      0.29            -0.2        0.11 ±  4%  perf-profile.children.cycles-pp.select_idle_sibling
      0.50            -0.2        0.33 ±  3%  perf-profile.children.cycles-pp.alloc_inode
      0.43            -0.2        0.26 ±  2%  perf-profile.children.cycles-pp.__x64_sys_close
      0.53 ±  3%      -0.2        0.37 ±  2%  perf-profile.children.cycles-pp.selinux_ip_postroute_compat
      0.42 ±  2%      -0.2        0.26        perf-profile.children.cycles-pp.resched_curr
      0.53 ±  2%      -0.2        0.36 ±  4%  perf-profile.children.cycles-pp.rcu_process_callbacks
      0.51 ±  2%      -0.2        0.35 ±  2%  perf-profile.children.cycles-pp.inet_csk_clone_lock
      0.44 ±  2%      -0.2        0.28 ±  5%  perf-profile.children.cycles-pp.ip_local_out
      0.35 ±  3%      -0.2        0.19 ±  3%  perf-profile.children.cycles-pp.selinux_inode_free_security
      0.48            -0.2        0.33 ±  3%  perf-profile.children.cycles-pp.tcp_init_transfer
      0.53 ±  2%      -0.2        0.37 ±  5%  perf-profile.children.cycles-pp.ip_route_output_key_hash
      0.57 ±  2%      -0.2        0.42 ±  2%  perf-profile.children.cycles-pp.tcp_v4_send_synack
      0.47 ±  4%      -0.2        0.32        perf-profile.children.cycles-pp.dst_release
      0.45            -0.2        0.30 ±  2%  perf-profile.children.cycles-pp.selinux_sock_rcv_skb_compat
      0.42 ±  3%      -0.1        0.27 ±  2%  perf-profile.children.cycles-pp.tcp_send_ack
      0.41 ±  3%      -0.1        0.26 ±  2%  perf-profile.children.cycles-pp.tcp_get_metrics
      0.38 ±  4%      -0.1        0.23 ±  2%  perf-profile.children.cycles-pp.inet_csk_get_port
      0.35 ±  3%      -0.1        0.21 ±  3%  perf-profile.children.cycles-pp.ret_from_fork
      0.35 ±  3%      -0.1        0.21 ±  3%  perf-profile.children.cycles-pp.kthread
      0.46 ±  2%      -0.1        0.32 ±  2%  perf-profile.children.cycles-pp.sk_clone_lock
      0.39            -0.1        0.25        perf-profile.children.cycles-pp.__call_rcu
      0.35 ±  2%      -0.1        0.21 ±  3%  perf-profile.children.cycles-pp.tcp_v4_destroy_sock
      0.36 ±  3%      -0.1        0.22 ±  3%  perf-profile.children.cycles-pp.__tcp_get_metrics
      0.50 ±  3%      -0.1        0.36 ±  5%  perf-profile.children.cycles-pp.ip_route_output_key_hash_rcu
      0.52            -0.1        0.38 ±  2%  perf-profile.children.cycles-pp.lock_timer_base
      0.41 ±  2%      -0.1        0.28 ±  6%  perf-profile.children.cycles-pp.__ip_local_out
      0.44            -0.1        0.31 ±  4%  perf-profile.children.cycles-pp.avc_has_perm
      0.33 ±  2%      -0.1        0.20 ±  4%  perf-profile.children.cycles-pp.sock_rfree
      0.31 ±  3%      -0.1        0.18 ±  5%  perf-profile.children.cycles-pp.smpboot_thread_fn
      0.31 ±  2%      -0.1        0.18 ±  3%  perf-profile.children.cycles-pp.filp_close
      0.33 ±  2%      -0.1        0.21 ±  2%  perf-profile.children.cycles-pp.inode_doinit_with_dentry
      0.36 ±  2%      -0.1        0.24 ±  3%  perf-profile.children.cycles-pp.tcp_finish_connect
      0.97 ±  3%      -0.1        0.84        perf-profile.children.cycles-pp.smp_apic_timer_interrupt
      0.28 ±  4%      -0.1        0.16 ±  8%  perf-profile.children.cycles-pp.run_ksoftirqd
      0.97 ±  2%      -0.1        0.85        perf-profile.children.cycles-pp.apic_timer_interrupt
      0.28            -0.1        0.16 ±  2%  perf-profile.children.cycles-pp.finish_task_switch
      0.38 ±  3%      -0.1        0.26 ±  3%  perf-profile.children.cycles-pp.__skb_clone
      0.36            -0.1        0.24 ±  7%  perf-profile.children.cycles-pp.__sk_dst_check
      0.34            -0.1        0.23 ±  4%  perf-profile.children.cycles-pp.__kmalloc_reserve
      0.34 ±  2%      -0.1        0.23 ±  6%  perf-profile.children.cycles-pp.ipv4_dst_check
      0.32 ±  2%      -0.1        0.20 ±  2%  perf-profile.children.cycles-pp.update_cfs_group
      0.31            -0.1        0.19 ±  4%  perf-profile.children.cycles-pp.tcp_time_wait
      0.35 ±  3%      -0.1        0.23 ±  3%  perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
      0.36 ±  3%      -0.1        0.24 ±  3%  perf-profile.children.cycles-pp.___might_sleep
      0.26 ±  3%      -0.1        0.15 ±  2%  perf-profile.children.cycles-pp.kfree
      0.33            -0.1        0.21 ±  5%  perf-profile.children.cycles-pp.__kmalloc_node_track_caller
      0.28            -0.1        0.17 ±  4%  perf-profile.children.cycles-pp.__list_del_entry_valid
      0.29 ±  5%      -0.1        0.17 ±  2%  perf-profile.children.cycles-pp.poll_idle
      0.33 ±  4%      -0.1        0.21 ±  2%  perf-profile.children.cycles-pp.read_tsc
      0.32 ±  3%      -0.1        0.21 ±  8%  perf-profile.children.cycles-pp.ip_route_output_flow
      0.35            -0.1        0.24 ±  5%  perf-profile.children.cycles-pp.__sk_destruct
      0.29 ±  2%      -0.1        0.18 ±  2%  perf-profile.children.cycles-pp.ktime_get_with_offset
      0.55 ± 28%      -0.1        0.45 ± 26%  perf-profile.children.cycles-pp.start_kernel
      0.26 ±  3%      -0.1        0.15 ±  2%  perf-profile.children.cycles-pp.tcp_update_metrics
      0.22 ±  3%      -0.1        0.12 ±  3%  perf-profile.children.cycles-pp.fput
      0.35 ±  2%      -0.1        0.24 ±  2%  perf-profile.children.cycles-pp.sockfd_lookup_light
      0.30 ±  2%      -0.1        0.20 ±  2%  perf-profile.children.cycles-pp.selinux_parse_skb
      0.24 ±  4%      -0.1        0.14 ±  5%  perf-profile.children.cycles-pp.__list_add_valid
      0.25 ±  4%      -0.1        0.15 ±  3%  perf-profile.children.cycles-pp.skb_copy_datagram_iter
      0.33 ±  2%      -0.1        0.23        perf-profile.children.cycles-pp.get_empty_filp
      0.28 ±  2%      -0.1        0.18 ±  2%  perf-profile.children.cycles-pp.inode_init_always
      0.23 ±  3%      -0.1        0.13 ±  3%  perf-profile.children.cycles-pp.___perf_sw_event
      0.66            -0.1        0.56        perf-profile.children.cycles-pp.tcp_done
      0.36            -0.1        0.26 ±  2%  perf-profile.children.cycles-pp.alloc_file
      0.32            -0.1        0.22 ±  4%  perf-profile.children.cycles-pp.account_entity_enqueue
      0.29 ±  3%      -0.1        0.19 ±  4%  perf-profile.children.cycles-pp.tick_nohz_idle_exit
      0.21 ±  3%      -0.1        0.11 ±  3%  perf-profile.children.cycles-pp.remove_wait_queue
      0.27            -0.1        0.17        perf-profile.children.cycles-pp.tcp_init_metrics
      0.27 ±  3%      -0.1        0.17        perf-profile.children.cycles-pp.reweight_entity
      0.24 ±  3%      -0.1        0.15 ±  4%  perf-profile.children.cycles-pp.update_rq_clock
      0.21 ±  5%      -0.1        0.11 ±  7%  perf-profile.children.cycles-pp.inet_csk_route_req
      0.19 ±  2%      -0.1        0.10 ±  5%  perf-profile.children.cycles-pp.available_idle_cpu
      0.19 ±  2%      -0.1        0.10 ±  5%  perf-profile.children.cycles-pp.task_work_add
      0.24            -0.1        0.15 ±  3%  perf-profile.children.cycles-pp.inet_csk_reqsk_queue_hash_add
      0.25 ±  2%      -0.1        0.16        perf-profile.children.cycles-pp.sk_prot_alloc
      0.29            -0.1        0.21 ±  2%  perf-profile.children.cycles-pp.skb_release_data
      0.23 ±  7%      -0.1        0.15 ±  5%  perf-profile.children.cycles-pp.inet_csk_bind_conflict
      0.21 ±  5%      -0.1        0.13 ±  3%  perf-profile.children.cycles-pp._copy_to_iter
      0.39 ±  2%      -0.1        0.30 ±  4%  perf-profile.children.cycles-pp.__x64_sys_getsockopt
      0.33            -0.1        0.24        perf-profile.children.cycles-pp.kmem_cache_alloc_node
      0.27 ±  4%      -0.1        0.18        perf-profile.children.cycles-pp.ipv4_mtu
      0.24 ±  2%      -0.1        0.15 ±  3%  perf-profile.children.cycles-pp.rcu_segcblist_enqueue
      0.22 ±  3%      -0.1        0.14 ±  3%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      0.19 ±  5%      -0.1        0.11 ±  4%  perf-profile.children.cycles-pp._copy_from_iter_full
      0.31 ±  2%      -0.1        0.23 ±  3%  perf-profile.children.cycles-pp.enqueue_to_backlog
      0.27 ±  3%      -0.1        0.18 ±  2%  perf-profile.children.cycles-pp.__inet_hash_connect
      0.08 ± 10%      -0.1        0.00        perf-profile.children.cycles-pp.security_req_classify_flow
      0.08 ±  5%      -0.1        0.00        perf-profile.children.cycles-pp.inet_csk_reqsk_queue_drop
      0.25            -0.1        0.17 ±  4%  perf-profile.children.cycles-pp.__switch_to
      0.08 ±  8%      -0.1        0.00        perf-profile.children.cycles-pp.arch_cpu_idle_enter
      0.08 ±  8%      -0.1        0.00        perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
      0.25 ±  3%      -0.1        0.17 ±  2%  perf-profile.children.cycles-pp.__fget_light
      0.41 ±  3%      -0.1        0.33 ±  5%  perf-profile.children.cycles-pp.tcp_set_state
      0.37 ±  2%      -0.1        0.29 ±  5%  perf-profile.children.cycles-pp.__sys_getsockopt
      0.28            -0.1        0.20 ±  7%  perf-profile.children.cycles-pp.fib_table_lookup
      0.28 ±  5%      -0.1        0.21 ±  2%  perf-profile.children.cycles-pp.ip_finish_output
      0.27            -0.1        0.19        perf-profile.children.cycles-pp.sk_alloc
      0.08 ± 10%      -0.1        0.00        perf-profile.children.cycles-pp.put_prev_entity
      0.24 ± 16%      -0.1        0.17 ± 10%  perf-profile.children.cycles-pp.ktime_get
      0.24 ±  3%      -0.1        0.17 ±  2%  perf-profile.children.cycles-pp.tcp_v4_init_sock
      0.23 ±  4%      -0.1        0.15 ±  3%  perf-profile.children.cycles-pp.__inet_check_established
      0.22 ±  3%      -0.1        0.15 ±  3%  perf-profile.children.cycles-pp.pick_next_task_idle
      0.18 ±  3%      -0.1        0.10 ±  4%  perf-profile.children.cycles-pp.tcp_write_queue_purge
      0.19 ±  4%      -0.1        0.11        perf-profile.children.cycles-pp.tcp_current_mss
      0.20 ±  2%      -0.1        0.12 ±  8%  perf-profile.children.cycles-pp.native_write_msr
      0.19            -0.1        0.11 ±  7%  perf-profile.children.cycles-pp.cpuidle_governor_latency_req
      0.22            -0.1        0.14 ±  5%  perf-profile.children.cycles-pp.ip_send_check
      0.21 ±  4%      -0.1        0.14 ±  5%  perf-profile.children.cycles-pp.__might_sleep
      0.20 ±  5%      -0.1        0.12 ±  3%  perf-profile.children.cycles-pp.tick_nohz_idle_enter
      0.07 ± 11%      -0.1        0.00        perf-profile.children.cycles-pp.inet_sock_destruct
      0.07 ±  5%      -0.1        0.00        perf-profile.children.cycles-pp.__tcp_v4_send_check
      0.07 ±  5%      -0.1        0.00        perf-profile.children.cycles-pp.down_write
      0.07 ±  5%      -0.1        0.00        perf-profile.children.cycles-pp.secure_tcp_ts_off
      0.07 ±  5%      -0.1        0.00        perf-profile.children.cycles-pp.raw_local_deliver
      0.11 ±  6%      -0.1        0.04 ± 57%  perf-profile.children.cycles-pp.inet_csk_complete_hashdance
      0.20 ±  3%      -0.1        0.13        perf-profile.children.cycles-pp.security_socket_create
      0.07 ± 10%      -0.1        0.00        perf-profile.children.cycles-pp.inet_sendmsg
      0.07 ± 10%      -0.1        0.00        perf-profile.children.cycles-pp.__ip_dev_find
      0.07 ± 10%      -0.1        0.00        perf-profile.children.cycles-pp.rcu_all_qs
      0.22 ±  5%      -0.1        0.15 ±  2%  perf-profile.children.cycles-pp.copy_user_generic_unrolled
      0.22            -0.1        0.15 ±  8%  perf-profile.children.cycles-pp.kmem_cache_alloc_trace
      0.17 ±  4%      -0.1        0.10 ±  7%  perf-profile.children.cycles-pp.inet_csk_route_child_sock
      0.18 ±  2%      -0.1        0.11 ±  6%  perf-profile.children.cycles-pp.selinux_socket_create
      0.21 ±  6%      -0.1        0.14 ±  7%  perf-profile.children.cycles-pp.sock_alloc_inode
      0.07 ±  6%      -0.1        0.00        perf-profile.children.cycles-pp.fsnotify
      0.07 ±  6%      -0.1        0.00        perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      0.07 ±  6%      -0.1        0.00        perf-profile.children.cycles-pp.del_timer_sync
      0.21 ±  2%      -0.1        0.15 ±  3%  perf-profile.children.cycles-pp.tcp_make_synack
      0.46 ±  8%      -0.1        0.40 ±  4%  perf-profile.children.cycles-pp.hrtimer_interrupt
      0.28 ±  2%      -0.1        0.22        perf-profile.children.cycles-pp.security_socket_connect
      0.15 ±  3%      -0.1        0.08        perf-profile.children.cycles-pp.sk_stream_kill_queues
      0.23            -0.1        0.17 ±  2%  perf-profile.children.cycles-pp.selinux_socket_connect_helper
      0.16 ±  5%      -0.1        0.10 ±  4%  perf-profile.children.cycles-pp.__update_load_avg_se
      0.16            -0.1        0.10 ±  5%  perf-profile.children.cycles-pp.tcp_update_pacing_rate
      0.11 ±  4%      -0.1        0.04 ± 58%  perf-profile.children.cycles-pp.iput
      0.07 ±  7%      -0.1        0.00        perf-profile.children.cycles-pp.__d_instantiate
      0.07 ±  7%      -0.1        0.00        perf-profile.children.cycles-pp.kfree_skb
      0.07 ±  7%      -0.1        0.00        perf-profile.children.cycles-pp.___slab_alloc
      0.22            -0.1        0.15 ±  5%  perf-profile.children.cycles-pp.tcp_init_sock
      0.20 ±  7%      -0.1        0.14 ±  6%  perf-profile.children.cycles-pp.__might_fault
      0.18 ±  2%      -0.1        0.11 ±  7%  perf-profile.children.cycles-pp.nr_iowait_cpu
      0.08 ±  6%      -0.1        0.01 ±173%  perf-profile.children.cycles-pp.selinux_netlbl_socket_post_create
      0.24 ±  6%      -0.1        0.18 ±  6%  perf-profile.children.cycles-pp.__x64_sys_setsockopt
      0.19 ±  3%      -0.1        0.13 ±  6%  perf-profile.children.cycles-pp.security_inode_alloc
      0.18 ±  4%      -0.1        0.12 ±  5%  perf-profile.children.cycles-pp.selinux_inode_alloc_security
      0.06 ±  6%      -0.1        0.00        perf-profile.children.cycles-pp.tcp_options_write
      0.06 ± 17%      -0.1        0.00        perf-profile.children.cycles-pp.netlink_has_listeners
      0.06 ±  6%      -0.1        0.00        perf-profile.children.cycles-pp.try_to_del_timer_sync
      0.06 ±  6%      -0.1        0.00        perf-profile.children.cycles-pp.security_sk_free
      0.06 ±  6%      -0.1        0.00        perf-profile.children.cycles-pp.cpus_share_cache
      0.15 ±  4%      -0.1        0.09        perf-profile.children.cycles-pp.tcp_rcv_space_adjust
      0.15 ±  4%      -0.1        0.09 ±  7%  perf-profile.children.cycles-pp.inet_ehash_insert
      0.14 ±  5%      -0.1        0.08        perf-profile.children.cycles-pp.pick_next_entity
      0.07 ±  5%      -0.1        0.01 ±173%  perf-profile.children.cycles-pp.security_socket_getsockopt
      0.24 ±  2%      -0.1        0.18        perf-profile.children.cycles-pp.selinux_socket_connect
      0.12 ±  7%      -0.1        0.06 ±  7%  perf-profile.children.cycles-pp.pm_qos_request
      0.06 ± 11%      -0.1        0.00        perf-profile.children.cycles-pp.tcp_select_initial_window
      0.06            -0.1        0.00        perf-profile.children.cycles-pp.tcp_cleanup_rbuf
      0.06 ± 11%      -0.1        0.00        perf-profile.children.cycles-pp.tick_check_broadcast_expired
      0.06 ± 11%      -0.1        0.00        perf-profile.children.cycles-pp.tcp_assign_congestion_control
      0.11 ±  7%      -0.1        0.05 ±  8%  perf-profile.children.cycles-pp.activate_task
      0.14 ±  7%      -0.1        0.08 ±  5%  perf-profile.children.cycles-pp.inet_ehashfn
      0.07 ± 10%      -0.1        0.01 ±173%  perf-profile.children.cycles-pp.account_entity_dequeue
      0.08 ± 10%      -0.1        0.03 ±100%  perf-profile.children.cycles-pp.__sk_free
      0.06 ±  7%      -0.1        0.00        perf-profile.children.cycles-pp.rcu_cblist_dequeue
      0.06 ±  7%      -0.1        0.00        perf-profile.children.cycles-pp.selinux_sock_graft
      0.06 ±  7%      -0.1        0.00        perf-profile.children.cycles-pp.tcp_validate_incoming
      0.15 ±  8%      -0.1        0.10 ±  5%  perf-profile.children.cycles-pp.tcp_wfree
      0.14 ±  7%      -0.1        0.09 ±  4%  perf-profile.children.cycles-pp.copyout
      0.06 ±  9%      -0.1        0.00        perf-profile.children.cycles-pp.sched_ttwu_pending
      0.06 ±  9%      -0.1        0.00        perf-profile.children.cycles-pp.sel_netnode_sid
      0.20 ±  2%      -0.1        0.14 ±  5%  perf-profile.children.cycles-pp.update_ts_time_stats
      0.16 ±  2%      -0.1        0.10 ±  8%  perf-profile.children.cycles-pp.entry_SYSCALL_64_stage2
      0.16 ±  6%      -0.1        0.10 ±  8%  perf-profile.children.cycles-pp.tcp_send_mss
      0.05 ±  9%      -0.1        0.00        perf-profile.children.cycles-pp.inet_addr_type_table
      0.22 ±  7%      -0.1        0.16 ±  5%  perf-profile.children.cycles-pp.__sys_setsockopt
      0.17 ± 11%      -0.1        0.12 ±  3%  perf-profile.children.cycles-pp.validate_xmit_skb
      0.05 ±  8%      -0.1        0.00        perf-profile.children.cycles-pp.import_single_range
      0.05 ±  8%      -0.1        0.00        perf-profile.children.cycles-pp.netlbl_req_setattr
      0.05 ±  8%      -0.1        0.00        perf-profile.children.cycles-pp.inet_lookup_ifaddr_rcu
      0.05 ±  8%      -0.1        0.00        perf-profile.children.cycles-pp.ip_mc_drop_socket
      0.05 ±  8%      -0.1        0.00        perf-profile.children.cycles-pp.hrtimer_try_to_cancel
      0.05 ±  8%      -0.1        0.00        perf-profile.children.cycles-pp.sk_free
      0.05 ±  8%      -0.1        0.00        perf-profile.children.cycles-pp.xfrm_lookup
      0.05 ± 57%      -0.1        0.00        perf-profile.children.cycles-pp.__usecs_to_jiffies
      0.17 ±  2%      -0.1        0.12 ±  3%  perf-profile.children.cycles-pp.inet_unhash
      0.08 ±  6%      -0.1        0.03 ±100%  perf-profile.children.cycles-pp.__switch_to_asm
      0.16 ±  4%      -0.1        0.11        perf-profile.children.cycles-pp.security_inet_conn_request
      0.05            -0.1        0.00        perf-profile.children.cycles-pp.skb_push
      0.05            -0.1        0.00        perf-profile.children.cycles-pp.security_socket_setsockopt
      0.05            -0.1        0.00        perf-profile.children.cycles-pp.tcp_rate_check_app_limited
      0.05            -0.1        0.00        perf-profile.children.cycles-pp.update_blocked_averages
      0.05            -0.1        0.00        perf-profile.children.cycles-pp.security_port_sid
      0.17 ±  2%      -0.0        0.12        perf-profile.children.cycles-pp.sock_has_perm
      0.13 ±  6%      -0.0        0.09 ±  5%  perf-profile.children.cycles-pp.rcu_eqs_enter
      0.06 ± 11%      -0.0        0.01 ±173%  perf-profile.children.cycles-pp.move_addr_to_kernel
      0.09 ±  5%      -0.0        0.04 ± 57%  perf-profile.children.cycles-pp.kfree_skbmem
      0.05 ± 58%      -0.0        0.00        perf-profile.children.cycles-pp.inet_ehash_nolisten
      0.36 ±  5%      -0.0        0.32 ±  4%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.15 ±  4%      -0.0        0.10 ±  8%  perf-profile.children.cycles-pp.inet_sk_rebuild_header
      0.14 ±  7%      -0.0        0.10 ±  4%  perf-profile.children.cycles-pp.ksize
      0.10 ±  4%      -0.0        0.06 ±  7%  perf-profile.children.cycles-pp.rcu_idle_exit
      0.07            -0.0        0.03 ±100%  perf-profile.children.cycles-pp.__calc_delta
      0.11 ±  4%      -0.0        0.06 ±  6%  perf-profile.children.cycles-pp.put_prev_task_fair
      0.12 ±  3%      -0.0        0.07 ± 15%  perf-profile.children.cycles-pp.security_socket_post_create
      0.12 ±  3%      -0.0        0.07 ± 15%  perf-profile.children.cycles-pp.selinux_socket_post_create
      0.08 ±  5%      -0.0        0.04 ± 57%  perf-profile.children.cycles-pp.deactivate_task
      0.07 ±  6%      -0.0        0.03 ±100%  perf-profile.children.cycles-pp.bictcp_cong_avoid
      0.15            -0.0        0.11 ±  4%  perf-profile.children.cycles-pp.selinux_inet_conn_request
      0.10 ±  4%      -0.0        0.06 ±  6%  perf-profile.children.cycles-pp.tcp_schedule_loss_probe
      0.06 ±  9%      -0.0        0.01 ±173%  perf-profile.children.cycles-pp.tcp_check_space
      0.05 ±  9%      -0.0        0.01 ±173%  perf-profile.children.cycles-pp.run_rebalance_domains
      0.04 ± 58%      -0.0        0.00        perf-profile.children.cycles-pp._copy_from_user
      0.16            -0.0        0.12 ±  5%  perf-profile.children.cycles-pp.evict
      0.14 ±  6%      -0.0        0.10 ±  4%  perf-profile.children.cycles-pp.tick_nohz_next_event
      0.12 ±  4%      -0.0        0.08 ±  5%  perf-profile.children.cycles-pp.tcp_release_cb
      0.10 ± 10%      -0.0        0.06 ±  6%  perf-profile.children.cycles-pp.ipv4_default_advmss
      0.10 ±  4%      -0.0        0.06 ±  7%  perf-profile.children.cycles-pp.rb_erase_cached
      0.10 ±  4%      -0.0        0.06 ±  6%  perf-profile.children.cycles-pp.selinux_sk_alloc_security
      0.08 ±  5%      -0.0        0.04 ± 57%  perf-profile.children.cycles-pp.inet_twsk_alloc
      0.07 ±  7%      -0.0        0.03 ±100%  perf-profile.children.cycles-pp.locks_remove_posix
      0.04 ± 58%      -0.0        0.00        perf-profile.children.cycles-pp.__internal_add_timer
      0.09            -0.0        0.05        perf-profile.children.cycles-pp.sock_recvmsg
      0.08 ±  5%      -0.0        0.04 ± 57%  perf-profile.children.cycles-pp.call_cpuidle
      0.12 ±  5%      -0.0        0.08 ±  8%  perf-profile.children.cycles-pp._cond_resched
      0.14 ±  6%      -0.0        0.10        perf-profile.children.cycles-pp._copy_to_user
      0.13            -0.0        0.09 ±  4%  perf-profile.children.cycles-pp.move_addr_to_user
      0.12 ±  3%      -0.0        0.08 ±  5%  perf-profile.children.cycles-pp.bictcp_acked
      0.10 ±  4%      -0.0        0.07 ±  7%  perf-profile.children.cycles-pp.inet_reqsk_alloc
      0.10 ±  9%      -0.0        0.06 ±  7%  perf-profile.children.cycles-pp.copyin
      0.08 ±  6%      -0.0        0.04 ± 57%  perf-profile.children.cycles-pp.tcp_v4_send_check
      0.08 ±  6%      -0.0        0.04 ± 57%  perf-profile.children.cycles-pp.security_socket_recvmsg
      0.04 ± 57%      -0.0        0.00        perf-profile.children.cycles-pp.file_free_rcu
      0.04 ± 57%      -0.0        0.00        perf-profile.children.cycles-pp._raw_write_lock_bh
      0.04 ± 57%      -0.0        0.00        perf-profile.children.cycles-pp.xfrm_lookup_route
      0.31 ±  3%      -0.0        0.27 ±  5%  perf-profile.children.cycles-pp.sk_stop_timer
      0.11 ± 13%      -0.0        0.08 ±  5%  perf-profile.children.cycles-pp.netif_skb_features
      0.14 ±  6%      -0.0        0.10 ±  8%  perf-profile.children.cycles-pp.memcpy_erms
      0.13 ±  6%      -0.0        0.10 ±  8%  perf-profile.children.cycles-pp.sock_setsockopt
      0.12 ±  5%      -0.0        0.08 ±  5%  perf-profile.children.cycles-pp.secure_tcp_seq
      0.10 ±  8%      -0.0        0.06 ±  6%  perf-profile.children.cycles-pp.__alloc_fd
      0.07 ±  5%      -0.0        0.04 ± 57%  perf-profile.children.cycles-pp.prepare_to_wait_exclusive
      0.24 ±  9%      -0.0        0.21 ±  2%  perf-profile.children.cycles-pp.tick_sched_timer
      0.43 ±  3%      -0.0        0.40 ±  2%  perf-profile.children.cycles-pp.irq_exit
      0.13 ±  3%      -0.0        0.10 ±  5%  perf-profile.children.cycles-pp.__inet_inherit_port
      0.13 ±  3%      -0.0        0.10        perf-profile.children.cycles-pp.memset_erms
      0.12 ±  5%      -0.0        0.09 ±  4%  perf-profile.children.cycles-pp.__d_alloc
      0.09 ±  4%      -0.0        0.06        perf-profile.children.cycles-pp.tcp_init_buffer_space
      0.10            -0.0        0.07 ±  6%  perf-profile.children.cycles-pp.security_sk_alloc
      0.06 ±  7%      -0.0        0.03 ±100%  perf-profile.children.cycles-pp.security_sock_graft
      0.11 ±  4%      -0.0        0.08 ±  5%  perf-profile.children.cycles-pp.tcp_queue_rcv
      0.21 ±  8%      -0.0        0.18 ±  3%  perf-profile.children.cycles-pp.tick_sched_handle
      0.09 ±  5%      -0.0        0.05 ±  9%  perf-profile.children.cycles-pp.menu_reflect
      0.23            -0.0        0.20 ±  2%  perf-profile.children.cycles-pp.sock_getsockopt
      0.08 ±  5%      -0.0        0.05 ±  9%  perf-profile.children.cycles-pp.skb_entail
      0.08 ±  5%      -0.0        0.05 ±  8%  perf-profile.children.cycles-pp.selinux_ipv4_output
      0.10 ±  8%      -0.0        0.08 ±  6%  perf-profile.children.cycles-pp.tcp_init_xmit_timers
      0.09 ±  5%      -0.0        0.06 ±  7%  perf-profile.children.cycles-pp.tcp_v4_inbound_md5_hash
      0.20 ±  8%      -0.0        0.17 ±  2%  perf-profile.children.cycles-pp.update_process_times
      0.10 ±  4%      -0.0        0.08 ±  6%  perf-profile.children.cycles-pp.fsnotify_destroy_marks
      0.11 ± 15%      -0.0        0.09 ± 13%  perf-profile.children.cycles-pp.__inet_lookup_listener
      0.10 ±  4%      -0.0        0.08 ±  6%  perf-profile.children.cycles-pp.fsnotify_grab_connector
      0.10 ±  4%      -0.0        0.07        perf-profile.children.cycles-pp.bictcp_cwnd_event
      0.09 ±  4%      -0.0        0.06 ± 11%  perf-profile.children.cycles-pp.tcp_v4_fill_cb
      0.08            -0.0        0.05 ±  8%  perf-profile.children.cycles-pp.security_file_alloc
      0.08 ±  5%      -0.0        0.05        perf-profile.children.cycles-pp.tcp_established_options
      0.28 ±  3%      -0.0        0.25 ±  6%  perf-profile.children.cycles-pp.inet_csk_clear_xmit_timers
      0.08 ±  6%      -0.0        0.05        perf-profile.children.cycles-pp.__slab_alloc
      0.08 ±  6%      -0.0        0.05        perf-profile.children.cycles-pp.add_wait_queue
      0.03 ±100%      -0.0        0.00        perf-profile.children.cycles-pp.note_gp_changes
      0.03 ±100%      -0.0        0.00        perf-profile.children.cycles-pp.selinux_d_instantiate
      0.03 ±100%      -0.0        0.00        perf-profile.children.cycles-pp.worker_thread
      0.03 ±100%      -0.0        0.00        perf-profile.children.cycles-pp.dentry_unlink_inode
      0.03 ±100%      -0.0        0.00        perf-profile.children.cycles-pp.rcu_check_callbacks
      0.03 ±100%      -0.0        0.00        perf-profile.children.cycles-pp.tcp_newly_delivered
      0.03 ±100%      -0.0        0.00        perf-profile.children.cycles-pp.sel_netnode_find
      0.03 ±100%      -0.0        0.00        perf-profile.children.cycles-pp.netlbl_sock_setattr
      0.10 ±  4%      -0.0        0.08        perf-profile.children.cycles-pp.sock_put
      0.08 ±  6%      -0.0        0.05 ±  8%  perf-profile.children.cycles-pp.selinux_file_alloc_security
      0.07 ± 11%      -0.0        0.05        perf-profile.children.cycles-pp.__sock_wfree
      0.10 ±  5%      -0.0        0.07 ±  5%  perf-profile.children.cycles-pp.__copy_skb_header
      0.19 ±  6%      -0.0        0.17 ±  7%  perf-profile.children.cycles-pp.inet_put_port
      0.08 ±  5%      -0.0        0.06 ±  7%  perf-profile.children.cycles-pp.tcp_parse_options
      0.08 ±  6%      -0.0        0.05 ±  9%  perf-profile.children.cycles-pp.selinux_socket_accept
      0.08 ± 16%      -0.0        0.06 ± 14%  perf-profile.children.cycles-pp.eth_type_trans
      0.07            -0.0        0.05        perf-profile.children.cycles-pp.selinux_netlbl_inet_conn_request
      0.07 ±  5%      -0.0        0.05 ±  8%  perf-profile.children.cycles-pp.ip_rcv_finish
      0.08 ±  5%      -0.0        0.07 ±  7%  perf-profile.children.cycles-pp.sock_destroy_inode
      0.08 ±  5%      -0.0        0.06        perf-profile.children.cycles-pp.security_socket_accept
      0.07 ±  5%      -0.0        0.05 ±  9%  perf-profile.children.cycles-pp.__srcu_read_lock
      0.04 ± 58%      -0.0        0.03 ±100%  perf-profile.children.cycles-pp.__update_idle_core
      0.08 ±  6%      -0.0        0.06 ± 11%  perf-profile.children.cycles-pp.__hrtimer_init
      0.07 ±  6%      -0.0        0.05 ±  8%  perf-profile.children.cycles-pp.__get_user_4
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.up_write
      0.04 ±100%      -0.0        0.03 ±100%  perf-profile.children.cycles-pp.clockevents_program_event
      0.11 ±  6%      -0.0        0.10 ±  5%  perf-profile.children.cycles-pp.scheduler_tick
      0.21 ±  5%      -0.0        0.19 ±  4%  perf-profile.children.cycles-pp.del_timer
      0.10 ±  4%      -0.0        0.08 ±  5%  perf-profile.children.cycles-pp.inet_twsk_deschedule_put
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.tcp_md5_do_lookup
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.update_min_vruntime
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.console_unlock
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.lockref_put_return
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.serial8250_console_write
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.wait_for_xmitr
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.uart_console_write
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.serial8250_console_putchar
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.tcp_ca_openreq_child
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.process_one_work
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.__tcp_select_window
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.woken_wake_function
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.rcu_eqs_exit
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp._atomic_dec_and_lock
      0.01 ±173%      -0.0        0.00        perf-profile.children.cycles-pp.security_inet_conn_established
      0.08 ± 10%      -0.0        0.07 ±  7%  perf-profile.children.cycles-pp.load_balance
      0.05            -0.0        0.04 ± 57%  perf-profile.children.cycles-pp.get_next_timer_interrupt
      0.07 ±  5%      -0.0        0.06 ± 13%  perf-profile.children.cycles-pp.tcp_openreq_init_rwin
      0.07            -0.0        0.06        perf-profile.children.cycles-pp.security_file_free
      0.09 ±  4%      -0.0        0.08        perf-profile.children.cycles-pp.selinux_ipv4_postroute
      0.07 ± 11%      -0.0        0.07 ± 12%  perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
      0.06 ± 14%      -0.0        0.05 ±  8%  perf-profile.children.cycles-pp.find_busiest_group
      0.08 ±  5%      -0.0        0.08 ±  5%  perf-profile.children.cycles-pp.inet_twsk_kill
      0.10 ± 10%      -0.0        0.10 ± 12%  perf-profile.children.cycles-pp.rebalance_domains
      0.06            -0.0        0.06 ±  7%  perf-profile.children.cycles-pp.security_socket_sendmsg
      0.06 ±  6%      -0.0        0.06 ± 11%  perf-profile.children.cycles-pp._raw_spin_trylock
      0.06 ±  9%      -0.0        0.05 ±  8%  perf-profile.children.cycles-pp.selinux_skb_peerlbl_sid
      0.05 ±  9%      -0.0        0.05 ±  8%  perf-profile.children.cycles-pp.selinux_netlbl_sock_rcv_skb
      0.06            +0.0        0.06 ±  6%  perf-profile.children.cycles-pp.__close_fd
      0.13 ±  3%      +0.0        0.15 ±  7%  perf-profile.children.cycles-pp.tcp_ack_update_rtt
      0.06 ±  7%      +0.0        0.11 ±  4%  perf-profile.children.cycles-pp.tcp_synack_rtt_meas
      0.06 ±  6%      +0.0        0.11 ±  3%  perf-profile.children.cycles-pp.minmax_running_min
      0.08 ±  6%      +0.1        0.14 ±  5%  perf-profile.children.cycles-pp.sk_setup_caps
      0.08 ±  8%      +0.1        0.18 ±  4%  perf-profile.children.cycles-pp._raw_spin_lock_irq
      0.00            +0.3        0.27        perf-profile.children.cycles-pp.find_next_bit
      0.00            +0.4        0.36        perf-profile.children.cycles-pp.cpumask_next
      0.00            +1.6        1.57        perf-profile.children.cycles-pp.mnt_get_count
     47.19            +9.2       56.39        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
     47.02            +9.3       56.30        perf-profile.children.cycles-pp.do_syscall_64
     12.25           +21.3       33.53        perf-profile.children.cycles-pp.exit_to_usermode_loop
     12.21           +21.3       33.51        perf-profile.children.cycles-pp.task_work_run
      2.04 ±  2%     +23.2       25.26        perf-profile.children.cycles-pp._raw_spin_lock
      0.73 ±  2%     +23.3       23.99        perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      0.00           +25.5       25.47        perf-profile.children.cycles-pp.mntput_no_expire
     36.34            -3.5       32.85        perf-profile.self.cycles-pp.intel_idle
      1.24 ±  2%      -0.5        0.78        perf-profile.self.cycles-pp._raw_spin_lock_bh
      1.17            -0.5        0.71        perf-profile.self.cycles-pp.switch_mm_irqs_off
      1.37            -0.5        0.92        perf-profile.self.cycles-pp.syscall_return_via_sysret
      1.33 ±  3%      -0.4        0.89        perf-profile.self.cycles-pp.__entry_trampoline_start
      1.19            -0.4        0.80        perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.92            -0.4        0.57 ±  2%  perf-profile.self.cycles-pp.__schedule
      0.71 ±  2%      -0.3        0.45        perf-profile.self.cycles-pp.update_load_avg
      0.82            -0.3        0.56        perf-profile.self.cycles-pp.__inet_lookup_established
      0.63 ±  2%      -0.3        0.38 ±  2%  perf-profile.self.cycles-pp.__slab_free
      0.64 ±  2%      -0.2        0.39        perf-profile.self.cycles-pp.tcp_transmit_skb
      0.72 ±  2%      -0.2        0.49        perf-profile.self.cycles-pp.__indirect_thunk_start
      0.57 ±  3%      -0.2        0.34        perf-profile.self.cycles-pp.load_new_mm_cr3
      0.46            -0.2        0.26        perf-profile.self.cycles-pp.__sk_mem_reduce_allocated
      0.66            -0.2        0.46 ±  3%  perf-profile.self.cycles-pp.tcp_v4_rcv
      0.59            -0.2        0.40        perf-profile.self.cycles-pp.tcp_ack
      0.58 ±  2%      -0.2        0.39        perf-profile.self.cycles-pp.tcp_recvmsg
      0.44            -0.2        0.25 ±  5%  perf-profile.self.cycles-pp.update_curr
      0.48 ±  2%      -0.2        0.29 ±  2%  perf-profile.self.cycles-pp.sk_forced_mem_schedule
      0.47            -0.2        0.29        perf-profile.self.cycles-pp.percpu_counter_add_batch
      0.52            -0.2        0.34 ±  4%  perf-profile.self.cycles-pp.native_sched_clock
      0.41 ±  2%      -0.2        0.23 ±  3%  perf-profile.self.cycles-pp.select_task_rq_fair
      0.51 ±  4%      -0.2        0.34 ±  2%  perf-profile.self.cycles-pp.selinux_socket_sock_rcv_skb
      0.42            -0.2        0.26 ±  3%  perf-profile.self.cycles-pp.menu_select
      0.39            -0.2        0.22        perf-profile.self.cycles-pp.ip_rcv
      0.42 ±  2%      -0.2        0.26        perf-profile.self.cycles-pp.resched_curr
      0.44 ±  2%      -0.2        0.28 ±  5%  perf-profile.self.cycles-pp.kmem_cache_alloc
      0.28            -0.2        0.12 ±  3%  perf-profile.self.cycles-pp.pick_next_task_fair
      0.38 ±  2%      -0.2        0.22        perf-profile.self.cycles-pp.do_idle
      0.46 ±  4%      -0.2        0.31        perf-profile.self.cycles-pp.dst_release
      0.51            -0.1        0.36 ±  3%  perf-profile.self.cycles-pp.sel_netport_sid
      0.38 ±  4%      -0.1        0.24        perf-profile.self.cycles-pp.set_next_entity
      0.41            -0.1        0.28 ±  4%  perf-profile.self.cycles-pp.tcp_clean_rtx_queue
      0.37            -0.1        0.24        perf-profile.self.cycles-pp.ip_queue_xmit
      0.29            -0.1        0.16 ±  6%  perf-profile.self.cycles-pp.__netif_receive_skb_core
      0.36 ±  4%      -0.1        0.22 ±  3%  perf-profile.self.cycles-pp.__tcp_get_metrics
      0.44            -0.1        0.31 ±  3%  perf-profile.self.cycles-pp.avc_has_perm
      0.37 ±  4%      -0.1        0.24 ±  7%  perf-profile.self.cycles-pp.ip_finish_output2
      0.33 ±  2%      -0.1        0.20 ±  4%  perf-profile.self.cycles-pp.sock_rfree
      0.34            -0.1        0.22        perf-profile.self.cycles-pp.__alloc_skb
      0.34 ±  2%      -0.1        0.22 ±  4%  perf-profile.self.cycles-pp.ipv4_dst_check
      0.33 ±  2%      -0.1        0.21 ±  4%  perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
      0.32 ±  2%      -0.1        0.20 ±  2%  perf-profile.self.cycles-pp.update_cfs_group
      0.30 ±  2%      -0.1        0.18 ±  2%  perf-profile.self.cycles-pp.tcp_write_xmit
      0.31 ±  3%      -0.1        0.19 ±  3%  perf-profile.self.cycles-pp.tcp_rcv_established
      0.26 ±  4%      -0.1        0.14 ±  5%  perf-profile.self.cycles-pp.try_to_wake_up
      0.26 ±  3%      -0.1        0.15 ±  2%  perf-profile.self.cycles-pp.kfree
      0.28            -0.1        0.17 ±  4%  perf-profile.self.cycles-pp.__list_del_entry_valid
      0.32 ±  3%      -0.1        0.21 ±  2%  perf-profile.self.cycles-pp.read_tsc
      0.26            -0.1        0.15 ±  3%  perf-profile.self.cycles-pp.wait_woken
      0.35 ±  2%      -0.1        0.24 ±  3%  perf-profile.self.cycles-pp.___might_sleep
      0.25 ±  2%      -0.1        0.14 ±  3%  perf-profile.self.cycles-pp.enqueue_entity
      0.23 ±  4%      -0.1        0.13 ±  5%  perf-profile.self.cycles-pp.mod_timer
      0.24            -0.1        0.14 ±  6%  perf-profile.self.cycles-pp.finish_task_switch
      0.24 ±  4%      -0.1        0.14 ±  5%  perf-profile.self.cycles-pp.__list_add_valid
      0.29            -0.1        0.19 ±  2%  perf-profile.self.cycles-pp.selinux_parse_skb
      0.24 ±  2%      -0.1        0.15 ±  5%  perf-profile.self.cycles-pp.process_backlog
      0.24 ±  4%      -0.1        0.14 ±  5%  perf-profile.self.cycles-pp.enqueue_task_fair
      0.33 ±  3%      -0.1        0.23 ±  5%  perf-profile.self.cycles-pp.__dev_queue_xmit
      0.27 ±  3%      -0.1        0.18 ±  2%  perf-profile.self.cycles-pp.loopback_xmit
      0.27 ±  3%      -0.1        0.17        perf-profile.self.cycles-pp.ip_output
      0.26            -0.1        0.17 ±  3%  perf-profile.self.cycles-pp.__kmalloc_node_track_caller
      0.23 ±  7%      -0.1        0.13        perf-profile.self.cycles-pp.__softirqentry_text_start
      0.20 ±  2%      -0.1        0.11 ±  6%  perf-profile.self.cycles-pp.___perf_sw_event
      0.30            -0.1        0.21 ±  3%  perf-profile.self.cycles-pp.do_syscall_64
      0.19 ±  2%      -0.1        0.10 ±  5%  perf-profile.self.cycles-pp.available_idle_cpu
      0.28 ±  5%      -0.1        0.19 ±  2%  perf-profile.self.cycles-pp.__skb_clone
      0.29            -0.1        0.21 ±  2%  perf-profile.self.cycles-pp.skb_release_data
      0.23 ±  2%      -0.1        0.15 ±  3%  perf-profile.self.cycles-pp.sock_def_readable
      0.23 ±  7%      -0.1        0.15 ±  5%  perf-profile.self.cycles-pp.inet_csk_bind_conflict
      0.26 ±  3%      -0.1        0.17 ±  2%  perf-profile.self.cycles-pp.reweight_entity
      0.24 ±  2%      -0.1        0.15 ±  5%  perf-profile.self.cycles-pp.net_rx_action
      0.18 ±  2%      -0.1        0.10 ±  5%  perf-profile.self.cycles-pp.task_work_add
      0.28            -0.1        0.20 ±  2%  perf-profile.self.cycles-pp.kmem_cache_free
      0.26 ±  2%      -0.1        0.18 ±  2%  perf-profile.self.cycles-pp.ipv4_mtu
      0.24 ±  2%      -0.1        0.15 ±  3%  perf-profile.self.cycles-pp.rcu_segcblist_enqueue
      0.19 ±  2%      -0.1        0.11 ±  6%  perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.08 ± 10%      -0.1        0.00        perf-profile.self.cycles-pp.selinux_socket_bind
      0.25            -0.1        0.17 ±  4%  perf-profile.self.cycles-pp.__switch_to
      0.23 ±  3%      -0.1        0.15 ±  3%  perf-profile.self.cycles-pp.poll_idle
      0.28            -0.1        0.20 ±  5%  perf-profile.self.cycles-pp.fib_table_lookup
      0.21 ±  5%      -0.1        0.12 ±  4%  perf-profile.self.cycles-pp.dequeue_entity
      0.08 ±  8%      -0.1        0.00        perf-profile.self.cycles-pp.netif_rx_internal
      0.21 ±  3%      -0.1        0.13 ±  3%  perf-profile.self.cycles-pp.__local_bh_enable_ip
      0.25 ±  4%      -0.1        0.17 ±  2%  perf-profile.self.cycles-pp.__fget_light
      0.20 ±  2%      -0.1        0.12 ±  8%  perf-profile.self.cycles-pp.native_write_msr
      0.19 ±  3%      -0.1        0.11 ±  7%  perf-profile.self.cycles-pp.tcp_sendmsg_locked
      0.22            -0.1        0.14 ±  5%  perf-profile.self.cycles-pp.ip_send_check
      0.21 ±  4%      -0.1        0.14 ±  5%  perf-profile.self.cycles-pp.__might_sleep
      0.07 ± 11%      -0.1        0.00        perf-profile.self.cycles-pp.put_prev_entity
      0.07 ±  5%      -0.1        0.00        perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
      0.07 ±  5%      -0.1        0.00        perf-profile.self.cycles-pp.__tcp_v4_send_check
      0.07 ±  5%      -0.1        0.00        perf-profile.self.cycles-pp.raw_local_deliver
      0.22 ±  5%      -0.1        0.15        perf-profile.self.cycles-pp.copy_user_generic_unrolled
      0.19 ±  3%      -0.1        0.12 ±  5%  perf-profile.self.cycles-pp.ip_local_deliver
      0.07 ± 10%      -0.1        0.00        perf-profile.self.cycles-pp.tcp_v4_syn_recv_sock
      0.07 ± 10%      -0.1        0.00        perf-profile.self.cycles-pp.inet_sendmsg
      0.07            -0.1        0.00        perf-profile.self.cycles-pp.__inet_stream_connect
      0.07            -0.1        0.00        perf-profile.self.cycles-pp.__x64_sys_close
      0.07 ± 10%      -0.1        0.00        perf-profile.self.cycles-pp.lock_sock_nested
      0.07 ± 10%      -0.1        0.00        perf-profile.self.cycles-pp.rcu_all_qs
      0.08            -0.1        0.01 ±173%  perf-profile.self.cycles-pp.__sys_sendto
      0.07 ±  6%      -0.1        0.00        perf-profile.self.cycles-pp.tick_nohz_get_sleep_length
      0.07 ± 12%      -0.1        0.00        perf-profile.self.cycles-pp.account_entity_dequeue
      0.07 ±  6%      -0.1        0.00        perf-profile.self.cycles-pp.fsnotify
      0.07 ±  6%      -0.1        0.00        perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      0.27 ±  7%      -0.1        0.20 ±  2%  perf-profile.self.cycles-pp.tcp_rcv_state_process
      0.27            -0.1        0.20 ±  3%  perf-profile.self.cycles-pp.kmem_cache_alloc_node
      0.16 ±  4%      -0.1        0.10 ±  5%  perf-profile.self.cycles-pp.update_rq_clock
      0.07 ±  7%      -0.1        0.00        perf-profile.self.cycles-pp.security_inode_free
      0.07 ±  7%      -0.1        0.00        perf-profile.self.cycles-pp.__inet_bind
      0.07 ±  7%      -0.1        0.00        perf-profile.self.cycles-pp.put_prev_task_fair
      0.07 ±  7%      -0.1        0.00        perf-profile.self.cycles-pp.iput
      0.07 ±  7%      -0.1        0.00        perf-profile.self.cycles-pp.inet_twsk_alloc
      0.18 ±  2%      -0.1        0.11 ±  7%  perf-profile.self.cycles-pp.nr_iowait_cpu
      0.18 ±  3%      -0.1        0.12 ±  3%  perf-profile.self.cycles-pp.selinux_ip_postroute
      0.21 ±  2%      -0.1        0.15 ±  4%  perf-profile.self.cycles-pp.enqueue_to_backlog
      0.18 ±  2%      -0.1        0.12 ±  5%  perf-profile.self.cycles-pp.tcp_close
      0.16 ±  2%      -0.1        0.10 ±  5%  perf-profile.self.cycles-pp.tcp_update_pacing_rate
      0.13 ±  3%      -0.1        0.07        perf-profile.self.cycles-pp.selinux_socket_create
      0.12 ±  3%      -0.1        0.06        perf-profile.self.cycles-pp.tcp_data_queue
      0.06 ±  6%      -0.1        0.00        perf-profile.self.cycles-pp.tcp_update_metrics
      0.06 ±  6%      -0.1        0.00        perf-profile.self.cycles-pp.tick_nohz_idle_enter
      0.06 ±  6%      -0.1        0.00        perf-profile.self.cycles-pp.skb_entail
      0.06 ±  6%      -0.1        0.00        perf-profile.self.cycles-pp.tcp_options_write
      0.06 ± 13%      -0.1        0.00        perf-profile.self.cycles-pp.tcp_v4_connect
      0.06 ±  6%      -0.1        0.00        perf-profile.self.cycles-pp.skb_release_head_state
      0.06 ±  6%      -0.1        0.00        perf-profile.self.cycles-pp.inode_init_always
      0.06 ±  6%      -0.1        0.00        perf-profile.self.cycles-pp.___slab_alloc
      0.06 ± 17%      -0.1        0.00        perf-profile.self.cycles-pp.netlink_has_listeners
      0.06 ±  6%      -0.1        0.00        perf-profile.self.cycles-pp.cpus_share_cache
      0.07 ±  5%      -0.1        0.01 ±173%  perf-profile.self.cycles-pp.tcp_ack_update_rtt
      0.07 ±  5%      -0.1        0.01 ±173%  perf-profile.self.cycles-pp.tcp_create_openreq_child
      0.07 ±  5%      -0.1        0.01 ±173%  perf-profile.self.cycles-pp.inet_reqsk_alloc
      0.15 ±  3%      -0.1        0.10 ±  5%  perf-profile.self.cycles-pp.__update_load_avg_se
      0.13 ±  5%      -0.1        0.07        perf-profile.self.cycles-pp.pick_next_entity
      0.12 ±  7%      -0.1        0.06 ±  7%  perf-profile.self.cycles-pp.pm_qos_request
      0.06            -0.1        0.00        perf-profile.self.cycles-pp.rcu_idle_exit
      0.06 ± 11%      -0.1        0.00        perf-profile.self.cycles-pp.tcp_schedule_loss_probe
      0.06            -0.1        0.00        perf-profile.self.cycles-pp.menu_reflect
      0.06 ± 11%      -0.1        0.00        perf-profile.self.cycles-pp.tcp_select_initial_window
      0.06 ± 11%      -0.1        0.00        perf-profile.self.cycles-pp.tick_check_broadcast_expired
      0.11 ±  7%      -0.1        0.05 ±  8%  perf-profile.self.cycles-pp.activate_task
      0.15 ±  7%      -0.1        0.09        perf-profile.self.cycles-pp.ip_local_deliver_finish
      0.07            -0.1        0.01 ±173%  perf-profile.self.cycles-pp.inet_csk_reqsk_queue_hash_add
      0.08 ±  5%      -0.1        0.03 ±100%  perf-profile.self.cycles-pp.__fput
      0.20 ±  5%      -0.1        0.14 ±  6%  perf-profile.self.cycles-pp.selinux_ip_postroute_compat
      0.06 ±  7%      -0.1        0.00        perf-profile.self.cycles-pp.inet_csk_route_req
      0.06 ±  7%      -0.1        0.00        perf-profile.self.cycles-pp.selinux_ipv4_output
      0.06 ±  7%      -0.1        0.00        perf-profile.self.cycles-pp.rcu_cblist_dequeue
      0.06 ±  7%      -0.1        0.00        perf-profile.self.cycles-pp.selinux_sock_graft
      0.15 ±  8%      -0.1        0.10 ±  5%  perf-profile.self.cycles-pp.tcp_wfree
      0.14 ±  5%      -0.1        0.08 ±  5%  perf-profile.self.cycles-pp.inet_ehashfn
      0.17 ±  3%      -0.1        0.11 ±  6%  perf-profile.self.cycles-pp.selinux_sock_rcv_skb_compat
      0.06 ±  9%      -0.1        0.00        perf-profile.self.cycles-pp.inet_csk_get_port
      0.06 ±  9%      -0.1        0.00        perf-profile.self.cycles-pp.inode_doinit_with_dentry
      0.06 ±  9%      -0.1        0.00        perf-profile.self.cycles-pp.sched_ttwu_pending
      0.06 ±  9%      -0.1        0.00        perf-profile.self.cycles-pp.tcp_validate_incoming
      0.16 ±  2%      -0.1        0.10 ±  8%  perf-profile.self.cycles-pp.entry_SYSCALL_64_stage2
      0.05 ±  9%      -0.1        0.00        perf-profile.self.cycles-pp.tcp_child_process
      0.05 ±  9%      -0.1        0.00        perf-profile.self.cycles-pp.sock_setsockopt
      0.05 ±  9%      -0.1        0.00        perf-profile.self.cycles-pp.inet_addr_type_table
      0.05 ±  9%      -0.1        0.00        perf-profile.self.cycles-pp.__sys_recvfrom
      0.05 ±  9%      -0.1        0.00        perf-profile.self.cycles-pp.tcp_assign_congestion_control
      0.15 ±  2%      -0.1        0.10 ±  5%  perf-profile.self.cycles-pp.__call_rcu
      0.11            -0.1        0.06 ±  7%  perf-profile.self.cycles-pp.__release_sock
      0.07 ±  7%      -0.1        0.01 ±173%  perf-profile.self.cycles-pp.inet_create
      0.07 ±  7%      -0.1        0.01 ±173%  perf-profile.self.cycles-pp.sched_clock_cpu
      0.05 ±  8%      -0.1        0.00        perf-profile.self.cycles-pp.ip_local_out
      0.05 ±  8%      -0.1        0.00        perf-profile.self.cycles-pp.import_single_range
      0.05 ±  8%      -0.1        0.00        perf-profile.self.cycles-pp.tcp_init_metrics
      0.05 ±  8%      -0.1        0.00        perf-profile.self.cycles-pp.inet_lookup_ifaddr_rcu
      0.05 ±  8%      -0.1        0.00        perf-profile.self.cycles-pp.ip_mc_drop_socket
      0.05 ±  8%      -0.1        0.00        perf-profile.self.cycles-pp.sk_free
      0.05 ± 57%      -0.1        0.00        perf-profile.self.cycles-pp.__usecs_to_jiffies
      0.09 ±  4%      -0.1        0.04 ± 58%  perf-profile.self.cycles-pp.check_preempt_curr
      0.18 ±  3%      -0.1        0.13 ±  3%  perf-profile.self.cycles-pp.account_entity_enqueue
      0.15 ±  2%      -0.1        0.10 ±  7%  perf-profile.self.cycles-pp.kmem_cache_alloc_trace
      0.12 ±  3%      -0.1        0.07 ±  7%  perf-profile.self.cycles-pp.tcp_rcv_space_adjust
      0.08 ±  6%      -0.1        0.03 ±100%  perf-profile.self.cycles-pp.__switch_to_asm
      0.08 ±  6%      -0.1        0.03 ±100%  perf-profile.self.cycles-pp.tcp_v4_send_check
      0.06 ± 13%      -0.1        0.01 ±173%  perf-profile.self.cycles-pp.sk_filter_trim_cap
      0.05            -0.1        0.00        perf-profile.self.cycles-pp.task_work_run
      0.05            -0.1        0.00        perf-profile.self.cycles-pp.tcp_event_new_data_sent
      0.05            -0.1        0.00        perf-profile.self.cycles-pp.tcp_cleanup_rbuf
      0.05            -0.1        0.00        perf-profile.self.cycles-pp.skb_push
      0.05            -0.1        0.00        perf-profile.self.cycles-pp.tcp_rate_check_app_limited
      0.17 ±  2%      -0.0        0.12        perf-profile.self.cycles-pp.sock_has_perm
      0.15 ±  4%      -0.0        0.10 ±  4%  perf-profile.self.cycles-pp.sk_clone_lock
      0.14 ±  7%      -0.0        0.10 ±  5%  perf-profile.self.cycles-pp.ksize
      0.13 ±  6%      -0.0        0.09 ±  5%  perf-profile.self.cycles-pp.rcu_eqs_enter
      0.06            -0.0        0.01 ±173%  perf-profile.self.cycles-pp.tcp_set_state
      0.09 ±  5%      -0.0        0.04 ± 57%  perf-profile.self.cycles-pp.kfree_skbmem
      0.14 ±  7%      -0.0        0.10 ±  5%  perf-profile.self.cycles-pp.do_softirq
      0.13 ±  6%      -0.0        0.08 ±  5%  perf-profile.self.cycles-pp.__inet_check_established
      0.07            -0.0        0.03 ±100%  perf-profile.self.cycles-pp.__calc_delta
      0.11 ±  3%      -0.0        0.07 ±  6%  perf-profile.self.cycles-pp.release_sock
      0.08 ± 13%      -0.0        0.04 ± 57%  perf-profile.self.cycles-pp.ipv4_default_advmss
      0.08 ±  5%      -0.0        0.04 ± 57%  perf-profile.self.cycles-pp.deactivate_task
      0.07 ±  6%      -0.0        0.03 ±100%  perf-profile.self.cycles-pp.bictcp_cong_avoid
      0.12 ±  4%      -0.0        0.08 ±  5%  perf-profile.self.cycles-pp.sk_reset_timer
      0.06 ±  9%      -0.0        0.01 ±173%  perf-profile.self.cycles-pp.tcp_check_space
      0.11 ±  9%      -0.0        0.07        perf-profile.self.cycles-pp.ktime_get_with_offset
      0.15 ±  7%      -0.0        0.11 ±  3%  perf-profile.self.cycles-pp.__sk_destruct
      0.14 ±  6%      -0.0        0.10 ±  5%  perf-profile.self.cycles-pp.tcp_connect
      0.12 ±  4%      -0.0        0.08 ±  5%  perf-profile.self.cycles-pp.tcp_release_cb
      0.10 ±  8%      -0.0        0.06 ±  7%  perf-profile.self.cycles-pp.__wake_up_common
      0.07 ±  7%      -0.0        0.03 ±100%  perf-profile.self.cycles-pp.__sk_dst_check
      0.07 ±  7%      -0.0        0.03 ±100%  perf-profile.self.cycles-pp.locks_remove_posix
      0.04 ± 58%      -0.0        0.00        perf-profile.self.cycles-pp.__kmalloc_reserve
      0.04 ± 58%      -0.0        0.00        perf-profile.self.cycles-pp.xfrm_lookup
      0.12 ±  8%      -0.0        0.08 ±  5%  perf-profile.self.cycles-pp.dev_hard_start_xmit
      0.08 ±  5%      -0.0        0.04 ± 57%  perf-profile.self.cycles-pp.call_cpuidle
      0.18 ±  2%      -0.0        0.14 ±  6%  perf-profile.self.cycles-pp.tcp_conn_request
      0.15 ±  4%      -0.0        0.11 ±  3%  perf-profile.self.cycles-pp.ip_finish_output
      0.12 ±  3%      -0.0        0.08 ±  5%  perf-profile.self.cycles-pp.bictcp_acked
      0.12 ±  5%      -0.0        0.08 ±  5%  perf-profile.self.cycles-pp.get_empty_filp
      0.10 ±  5%      -0.0        0.06 ±  7%  perf-profile.self.cycles-pp.tcp_v4_do_rcv
      0.04 ± 57%      -0.0        0.00        perf-profile.self.cycles-pp.tcp_get_metrics
      0.04 ± 57%      -0.0        0.00        perf-profile.self.cycles-pp.select_idle_sibling
      0.04 ± 57%      -0.0        0.00        perf-profile.self.cycles-pp._copy_from_iter_full
      0.04 ± 57%      -0.0        0.00        perf-profile.self.cycles-pp.selinux_inet_conn_request
      0.04 ± 57%      -0.0        0.00        perf-profile.self.cycles-pp._copy_to_user
      0.04 ± 57%      -0.0        0.00        perf-profile.self.cycles-pp.tcp_v4_inbound_md5_hash
      0.04 ± 57%      -0.0        0.00        perf-profile.self.cycles-pp.down_write
      0.04 ± 57%      -0.0        0.00        perf-profile.self.cycles-pp.file_free_rcu
      0.04 ± 57%      -0.0        0.00        perf-profile.self.cycles-pp.switch_mm
      0.04 ± 57%      -0.0        0.00        perf-profile.self.cycles-pp._raw_write_lock_bh
      0.12 ±  6%      -0.0        0.09 ±  4%  perf-profile.self.cycles-pp.sock_def_wakeup
      0.09            -0.0        0.05 ±  8%  perf-profile.self.cycles-pp.lock_timer_base
      0.14 ±  6%      -0.0        0.10 ±  8%  perf-profile.self.cycles-pp.memcpy_erms
      0.09 ±  4%      -0.0        0.05 ±  8%  perf-profile.self.cycles-pp.rb_erase_cached
      0.07 ± 11%      -0.0        0.04 ± 57%  perf-profile.self.cycles-pp.__sock_wfree
      0.12 ±  5%      -0.0        0.09 ±  5%  perf-profile.self.cycles-pp.nf_hook_slow
      0.10 ± 33%      -0.0        0.07 ± 30%  perf-profile.self.cycles-pp.ktime_get
      0.13 ±  3%      -0.0        0.10        perf-profile.self.cycles-pp.memset_erms
      0.10 ±  9%      -0.0        0.06 ±  6%  perf-profile.self.cycles-pp.tcp_send_ack
      0.10 ± 11%      -0.0        0.07 ±  7%  perf-profile.self.cycles-pp.netif_skb_features
      0.09 ± 13%      -0.0        0.05 ±  8%  perf-profile.self.cycles-pp.tcp_current_mss
      0.06 ±  7%      -0.0        0.03 ±100%  perf-profile.self.cycles-pp.tcp_check_req
      0.11 ±  4%      -0.0        0.08 ±  5%  perf-profile.self.cycles-pp.tcp_queue_rcv
      0.11 ±  4%      -0.0        0.08 ± 10%  perf-profile.self.cycles-pp.sk_stop_timer
      0.16 ±  6%      -0.0        0.13 ±  6%  perf-profile.self.cycles-pp.ip_route_output_key_hash_rcu
      0.13 ±  6%      -0.0        0.10 ±  4%  perf-profile.self.cycles-pp.dequeue_task_fair
      0.08            -0.0        0.05        perf-profile.self.cycles-pp.pick_next_task_idle
      0.07 ±  6%      -0.0        0.04 ± 57%  perf-profile.self.cycles-pp._cond_resched
      0.10 ± 10%      -0.0        0.07 ± 10%  perf-profile.self.cycles-pp.__might_fault
      0.09 ±  9%      -0.0        0.06 ± 11%  perf-profile.self.cycles-pp.inet_sk_rebuild_header
      0.05 ±  8%      -0.0        0.03 ±100%  perf-profile.self.cycles-pp.selinux_netlbl_sock_rcv_skb
      0.11 ± 15%      -0.0        0.09 ± 13%  perf-profile.self.cycles-pp.__inet_lookup_listener
      0.09 ±  4%      -0.0        0.07 ±  7%  perf-profile.self.cycles-pp.inet_csk_accept
      0.09            -0.0        0.06 ±  6%  perf-profile.self.cycles-pp.sockfd_lookup_light
      0.10 ±  4%      -0.0        0.07        perf-profile.self.cycles-pp.bictcp_cwnd_event
      0.09            -0.0        0.06 ±  6%  perf-profile.self.cycles-pp.rcu_process_callbacks
      0.09 ±  4%      -0.0        0.06 ± 11%  perf-profile.self.cycles-pp.tcp_v4_fill_cb
      0.08 ±  5%      -0.0        0.05        perf-profile.self.cycles-pp.tcp_established_options
      0.07 ±  7%      -0.0        0.04 ± 58%  perf-profile.self.cycles-pp.tcp_make_synack
      0.03 ±100%      -0.0        0.00        perf-profile.self.cycles-pp.__wake_up_common_lock
      0.03 ±100%      -0.0        0.00        perf-profile.self.cycles-pp.tick_nohz_next_event
      0.03 ±100%      -0.0        0.00        perf-profile.self.cycles-pp.selinux_d_instantiate
      0.03 ±100%      -0.0        0.00        perf-profile.self.cycles-pp.dentry_unlink_inode
      0.03 ±100%      -0.0        0.00        perf-profile.self.cycles-pp.security_d_instantiate
      0.03 ±100%      -0.0        0.00        perf-profile.self.cycles-pp.find_busiest_group
      0.03 ±100%      -0.0        0.00        perf-profile.self.cycles-pp.tcp_newly_delivered
      0.03 ±100%      -0.0        0.00        perf-profile.self.cycles-pp.sel_netnode_find
      0.10 ±  4%      -0.0        0.08        perf-profile.self.cycles-pp.sock_put
      0.07 ± 11%      -0.0        0.05        perf-profile.self.cycles-pp.validate_xmit_skb
      0.07 ±  5%      -0.0        0.05        perf-profile.self.cycles-pp.move_addr_to_user
      0.08 ± 16%      -0.0        0.06 ± 15%  perf-profile.self.cycles-pp.eth_type_trans
      0.11 ±  4%      -0.0        0.09 ±  5%  perf-profile.self.cycles-pp.sk_alloc
      0.11 ±  3%      -0.0        0.09 ±  7%  perf-profile.self.cycles-pp.selinux_inode_free_security
      0.10 ±  5%      -0.0        0.07 ±  5%  perf-profile.self.cycles-pp.__copy_skb_header
      0.08 ±  5%      -0.0        0.06 ±  7%  perf-profile.self.cycles-pp.tcp_parse_options
      0.07 ± 10%      -0.0        0.05        perf-profile.self.cycles-pp.inet_recvmsg
      0.07 ± 10%      -0.0        0.05        perf-profile.self.cycles-pp.selinux_inode_alloc_security
      0.07 ±  5%      -0.0        0.05 ±  8%  perf-profile.self.cycles-pp.__srcu_read_lock
      0.07 ±  5%      -0.0        0.05 ±  8%  perf-profile.self.cycles-pp.ip_rcv_finish
      0.07 ±  6%      -0.0        0.05        perf-profile.self.cycles-pp.tcp_v4_destroy_sock
      0.07 ±  6%      -0.0        0.05        perf-profile.self.cycles-pp.__get_user_4
      0.04 ± 58%      -0.0        0.03 ±100%  perf-profile.self.cycles-pp.__update_idle_core
      0.08 ±  6%      -0.0        0.06 ± 11%  perf-profile.self.cycles-pp.__hrtimer_init
      0.08 ±  5%      -0.0        0.07        perf-profile.self.cycles-pp.__ip_local_out
      0.01 ±173%      -0.0        0.00        perf-profile.self.cycles-pp.__internal_add_timer
      0.01 ±173%      -0.0        0.00        perf-profile.self.cycles-pp.tcp_md5_do_lookup
      0.01 ±173%      -0.0        0.00        perf-profile.self.cycles-pp.update_min_vruntime
      0.01 ±173%      -0.0        0.00        perf-profile.self.cycles-pp.sk_prot_alloc
      0.01 ±173%      -0.0        0.00        perf-profile.self.cycles-pp.lockref_put_return
      0.01 ±173%      -0.0        0.00        perf-profile.self.cycles-pp.__tcp_select_window
      0.01 ±173%      -0.0        0.00        perf-profile.self.cycles-pp.woken_wake_function
      0.01 ±173%      -0.0        0.00        perf-profile.self.cycles-pp.__x64_sys_recvfrom
      0.01 ±173%      -0.0        0.00        perf-profile.self.cycles-pp.tcp_fin
      0.01 ±173%      -0.0        0.00        perf-profile.self.cycles-pp.security_req_classify_flow
      0.01 ±173%      -0.0        0.00        perf-profile.self.cycles-pp.up_write
      0.01 ±173%      -0.0        0.00        perf-profile.self.cycles-pp.inet_release
      0.01 ±173%      -0.0        0.00        perf-profile.self.cycles-pp.schedule
      0.01 ±173%      -0.0        0.00        perf-profile.self.cycles-pp.tcp_init_buffer_space
      1.58 ±  2%      -0.0        1.57        perf-profile.self.cycles-pp._raw_spin_lock
      0.08 ±  6%      -0.0        0.07 ±  6%  perf-profile.self.cycles-pp.cpuidle_enter_state
      0.09 ±  4%      -0.0        0.08        perf-profile.self.cycles-pp.selinux_ipv4_postroute
      0.06 ±  6%      -0.0        0.06 ± 11%  perf-profile.self.cycles-pp._raw_spin_trylock
      0.07 ±  7%      +0.0        0.07 ±  7%  perf-profile.self.cycles-pp.sock_getsockopt
      0.00            +0.0        0.01 ±173%  perf-profile.self.cycles-pp.tcp_openreq_init_rwin
      0.06 ±  6%      +0.0        0.11 ±  3%  perf-profile.self.cycles-pp.minmax_running_min
      0.07            +0.1        0.14 ±  3%  perf-profile.self.cycles-pp.sk_setup_caps
      0.08 ±  8%      +0.1        0.18 ±  4%  perf-profile.self.cycles-pp._raw_spin_lock_irq
      0.00            +0.1        0.11 ±  3%  perf-profile.self.cycles-pp.cpumask_next
      0.00            +0.2        0.18 ±  3%  perf-profile.self.cycles-pp.mntput_no_expire
      0.00            +0.3        0.27        perf-profile.self.cycles-pp.find_next_bit
      0.00            +1.2        1.24        perf-profile.self.cycles-pp.mnt_get_count
      0.73           +23.1       23.84        perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath


                                                                                
                               netperf.Throughput_tps                           
                                                                                
  23000 +-+-----------------------------------------------------------------+   
        |                                                            +      |   
  22000 +-+++.+                                                     + +.    |   
  21000 +-+    +.++. ++.++.++.++.+++.++.++.++.++.+++.++.++.++.+++.++    ++.+|   
        |           +                                                       |   
  20000 +-+                                                                 |   
        |                                                                   |   
  19000 +-+                                                                 |   
        |                                                                   |   
  18000 +-+                                                                 |   
  17000 +-+                                                                 |   
        |                                                                   |   
  16000 +-+   O         O                         O                         |   
        OO OO  O OO OOO  O OO OO OOO OO OO OO OO O O OO OO                  |   
  15000 +-+-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                            netperf.Throughput_total_tps                        
                                                                                
  500000 +-+----------------------------------------------------------------+   
         |                                                            +.    |   
  480000 +-+++.+  +  .++.++.                                          : ++.+|   
  460000 +-+    :+ :+       +++.++.++.+++.++.+++.++.++.+++.++.++.+++.:      |   
         |      +  +                                                 +      |   
  440000 +-+                                                                |   
  420000 +-+                                                                |   
         |                                                                  |   
  400000 +-+                                                                |   
  380000 +-+                                                                |   
         |                                                                  |   
  360000 +-+                                                                |   
  340000 OO+OO  O OO  OO OO OOO OO OO OOO OO OOO OO OO OOO                  |   
         |     O    O                                                       |   
  320000 +-+----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                   netperf.workload                             
                                                                                
  1.45e+08 +-+--------------------------------------------------------------+   
           |+.++.+ +   +.++.+                                         : ++.+|   
   1.4e+08 +-+    ::+ +      ++.++.+++.+++.++.+++.+++.++.+++.++.+++.+ :     |   
  1.35e+08 +-+    +  +                                               :      |   
           |                                                         +      |   
   1.3e+08 +-+                                                              |   
  1.25e+08 +-+                                                              |   
           |                                                                |   
   1.2e+08 +-+                                                              |   
  1.15e+08 +-+                                                              |   
           |                                                                |   
   1.1e+08 +-+                                                              |   
  1.05e+08 +-+            O                        O                        |   
           OO OO  OO OOO O  OOO OO OOO OOO OO OOO O O OO OO                 |   
     1e+08 +-+---O----------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                              netperf.time.user_time                            
                                                                                
  300 +-+-------------------------------------------------------------------+   
  290 +-++       .++.                                                       |   
      |+  +.+   +    ++.                                              +.+ .+|   
  280 +-+   :  +        + .+   +. +.  .++.+ .+   +.    +.  .+ .+   +  :  +  |   
  270 +-+    :+          +  +.+  +  ++     +  +.+  ++.+  ++  +  +.+ +:      |   
  260 +-+    +                                                       +      |   
  250 +-+                                                                   |   
      |                                                                     |   
  240 +-+                                                                   |   
  230 +-+                                                                   |   
  220 +O+OO O                                                               |   
  210 O-+         OO O     O            O  O  O  O O                        |   
      |      O OO     O OO  O OO OO OO O  O  O  O      O                    |   
  200 +-+                                           O O  O                  |   
  190 +-+-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                             netperf.time.system_time                           
                                                                                
  5900 +-+------------------------------------------------------------------+   
       |                        O  O                                        |   
  5800 OO+OO  O OO OO OO OO OOO  O  O OO OO OO OO OO OOO O                  |   
  5700 +-+                                                                  |   
       |     O                                                              |   
  5600 +-+                                                                  |   
  5500 +-+                                                                  |   
       |+.++.+  + .++.++.++.+++.++.++.++.++.++.++.++.+++.++.++.++.++  +.++.+|   
  5400 +-+   : : +                                                 :  :     |   
  5300 +-+    ::                                                    : :     |   
       |      +                                                     : :     |   
  5200 +-+                                                          ::      |   
  5100 +-+                                                           :      |   
       |                                                             +      |   
  5000 +-+------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                     netperf.time.percent_of_cpu_this_job_got                   
                                                                                
  2000 +-+------------------------------------------------------------------+   
       |                  O     OO OO                                       |   
  1950 OO+OO  O OO OO OO O  OOO       OO OO OO OO OO OOO O                  |   
       |                                                                    |   
       |     O                                                              |   
  1900 +-+          +.++.+                                                  |   
       |+.++.+  + .+      +.+++.++.++.++.++.++.++.++.+++.++.++.++.++  +.++.+|   
  1850 +-+   : : +                                                 :  :     |   
       |      ::                                                    : :     |   
  1800 +-+    +                                                     : :     |   
       |                                                            ::      |   
       |                                                             :      |   
  1750 +-+                                                           +      |   
       |                                                                    |   
  1700 +-+------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                        netperf.time.voluntary_context_switches                 
                                                                                
  2.8e+08 +-+---------------------------------------------------------------+   
          |+    + +.  .+++.++.  +. +.  +. +.++.  +. +.++   +. +   +.  : ++.+|   
  2.7e+08 +-+    :: ++        ++  +  ++  +     ++  +    +.+  + +.+  +:      |   
  2.6e+08 +-+    +                                                   +      |   
          |                                                                 |   
  2.5e+08 +-+                                                               |   
  2.4e+08 +-+                                                               |   
          |                                                                 |   
  2.3e+08 +-+                                                               |   
  2.2e+08 +-+                                                               |   
          |                                                                 |   
  2.1e+08 +-+                                                               |   
    2e+08 +O+ O OOO    OOO OO O O OO OOO OO OO OOO OO OOO O                 |   
          O  O      OO         O                                            |   
  1.9e+08 +-+---------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample


Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Xiaolong

View attachment "config-4.18.0-rc1-00013-g8a2e54b" of type "text/plain" (166044 bytes)

View attachment "job-script" of type "text/plain" (7555 bytes)

View attachment "job.yaml" of type "text/plain" (5094 bytes)

View attachment "reproduce" of type "text/plain" (1399 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ