lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date:   Wed, 14 Nov 2018 17:22:42 +0800
From:   kernel test robot <rong.a.chen@...el.com>
To:     Matthew Wilcox <willy@...radead.org>
Cc:     LKML <linux-kernel@...r.kernel.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>, lkp@...org
Subject: [LKP] [page cache]  eb797a8ee0:  vm-scalability.throughput -16.5%
 regression

Greeting,

FYI, we noticed a -16.5% regression of vm-scalability.throughput due to commit:


commit: eb797a8ee0ab4cd03df556980ce7bf167cadaa50 ("page cache: Rearrange address_space")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

in testcase: vm-scalability
on test machine: 80 threads Skylake with 64G memory
with following parameters:

	runtime: 300s
	test: small-allocs
	cpufreq_governor: performance

test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us.
test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/

In addition to that, the commit also has significant impact on the following tests:

+------------------+----------------------------------------------------------------------+
| testcase: change | unixbench: unixbench.score 20.9% improvement                         |
| test machine     | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 64G memory |
| test parameters  | cpufreq_governor=performance                                         |
|                  | nr_task=100%                                                         |
|                  | runtime=300s                                                         |
|                  | test=shell8                                                          |
|                  | ucode=0xb00002e                                                      |
+------------------+----------------------------------------------------------------------+


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase:
  gcc-7/performance/x86_64-rhel-7.2/debian-x86_64-2018-04-03.cgz/300s/lkp-skl-2sp2/small-allocs/vm-scalability

commit: 
  f32f004cdd ("ida: Convert to XArray")
  eb797a8ee0 ("page cache: Rearrange address_space")

f32f004cddf86d63 eb797a8ee0ab4cd03df556980c 
---------------- -------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
          3:4          -13%           3:4     perf-profile.calltrace.cycles-pp.error_entry
          3:4          -12%           3:4     perf-profile.children.cycles-pp.error_entry
          1:4           -6%           1:4     perf-profile.self.cycles-pp.error_entry
         %stddev     %change         %stddev
             \          |                \  
    235891           -16.5%     197085        vm-scalability.median
  18881481           -16.5%   15769081        vm-scalability.throughput
    316.19           +14.4%     361.58        vm-scalability.time.elapsed_time
    316.19           +14.4%     361.58        vm-scalability.time.elapsed_time.max
     22924           +15.9%      26576        vm-scalability.time.system_time
   3254041 ±  9%     +36.4%    4437311 ±  3%  vm-scalability.time.voluntary_context_switches
    277831 ±  3%      +9.5%     304359        interrupts.CAL:Function_call_interrupts
    102367 ±  2%     +10.1%     112694 ±  2%  meminfo.Shmem
      6.67 ±  5%      -0.9        5.76 ±  2%  mpstat.cpu.usr%
      0.49            -5.0%       0.46        pmeter.Average_Active_Power
  38678749           -12.1%   34005251        pmeter.performance_per_watt
   2621420 ± 38%     +59.2%    4173292 ±  3%  turbostat.C1
  62964314 ± 10%     +18.7%   74735975        turbostat.IRQ
     20821 ± 10%     +20.6%      25103 ±  2%  vmstat.system.cs
    192700 ±  8%      +5.9%     204006        vmstat.system.in
     76742            +3.7%      79550        proc-vmstat.nr_active_anon
     25578 ±  2%     +10.1%      28154 ±  2%  proc-vmstat.nr_shmem
     76742            +3.7%      79550        proc-vmstat.nr_zone_active_anon
  34211075 ± 14%     +28.7%   44023085 ±  3%  cpuidle.C1.time
   2628057 ± 37%     +59.1%    4179955 ±  3%  cpuidle.C1.usage
    200488 ± 20%     +75.5%     351836 ± 13%  cpuidle.POLL.time
     57706 ± 49%     +93.1%     111419 ± 18%  cpuidle.POLL.usage
      2.08 ± 14%     +20.7%       2.51 ±  4%  sched_debug.cfs_rq:/.nr_spread_over.avg
     50.05 ± 36%     -38.0%      31.04 ± 24%  sched_debug.cpu.cpu_load[1].max
      0.25 ± 14%     +17.4%       0.29 ±  7%  sched_debug.cpu.nr_running.stddev
     42905 ± 12%     +26.2%      54143 ±  3%  sched_debug.cpu.nr_switches.avg
     19996 ± 17%     +43.0%      28586 ±  7%  sched_debug.cpu.nr_switches.min
     43774 ± 12%     +26.5%      55370 ±  3%  sched_debug.cpu.sched_count.avg
     20285 ± 17%     +43.1%      29024 ±  8%  sched_debug.cpu.sched_count.min
     19949 ± 12%     +28.0%      25530 ±  3%  sched_debug.cpu.sched_goidle.avg
      9260 ± 16%     +44.9%      13422 ±  8%  sched_debug.cpu.sched_goidle.min
     23108 ± 12%     +26.0%      29110 ±  3%  sched_debug.cpu.ttwu_count.avg
     25905 ± 11%     +25.8%      32593 ±  2%  sched_debug.cpu.ttwu_count.max
     21683 ± 12%     +26.0%      27323 ±  3%  sched_debug.cpu.ttwu_count.min
     73.74            -3.6       70.12        perf-stat.cache-miss-rate%
  2.83e+10 ±  2%      +5.5%  2.985e+10        perf-stat.cache-misses
 3.838e+10           +10.9%  4.257e+10        perf-stat.cache-references
   6787110 ±  9%     +35.5%    9197959 ±  3%  perf-stat.context-switches
      3.17           +11.1%       3.52        perf-stat.cpi
 7.609e+13           +14.5%  8.715e+13        perf-stat.cpu-cycles
     21380           +11.1%      23753        perf-stat.cpu-migrations
      0.08            -0.0        0.07        perf-stat.dTLB-load-miss-rate%
 6.634e+12            +3.8%  6.888e+12        perf-stat.dTLB-loads
      0.13            -0.0        0.12        perf-stat.dTLB-store-miss-rate%
 1.135e+09            -5.6%  1.071e+09        perf-stat.dTLB-store-misses
 2.184e+09            -2.3%  2.135e+09        perf-stat.iTLB-load-misses
 2.404e+13            +3.1%  2.479e+13        perf-stat.instructions
     11006            +5.5%      11614        perf-stat.instructions-per-iTLB-miss
      0.32           -10.0%       0.28        perf-stat.ipc
 2.898e+09           +15.2%  3.337e+09 ±  2%  perf-stat.node-load-misses
 7.235e+08           +19.1%  8.615e+08 ±  4%  perf-stat.node-store-misses
      4975            +5.3%       5236        perf-stat.path-length
      1.10            -0.1        0.97 ±  2%  perf-profile.calltrace.cycles-pp.rwsem_spin_on_owner.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.vma_link
      0.68 ±  2%      -0.1        0.59        perf-profile.calltrace.cycles-pp.vma_interval_tree_insert.vma_link.mmap_region.do_mmap.vm_mmap_pgoff
      0.59 ±  3%      -0.0        0.55 ±  2%  perf-profile.calltrace.cycles-pp.swapgs_restore_regs_and_return_to_usermode
      1.04 ±  5%      +0.1        1.19 ±  3%  perf-profile.calltrace.cycles-pp.task_numa_work.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.04 ±  5%      +0.1        1.19 ±  3%  perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.04 ±  5%      +0.1        1.19 ±  3%  perf-profile.calltrace.cycles-pp.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.64 ±  2%      +0.4        2.00        perf-profile.calltrace.cycles-pp.page_fault
      1.57 ±  2%      +0.4        1.93        perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault
      1.59 ±  2%      +0.4        1.96        perf-profile.calltrace.cycles-pp.do_page_fault.page_fault
      0.65            +0.5        1.19 ±  3%  perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
      0.00            +1.0        1.03 ±  4%  perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
      1.10            -0.1        0.97 ±  2%  perf-profile.children.cycles-pp.rwsem_spin_on_owner
      0.68 ±  2%      -0.1        0.59        perf-profile.children.cycles-pp.vma_interval_tree_insert
      0.62 ±  2%      -0.1        0.55 ±  2%  perf-profile.children.cycles-pp.native_irq_return_iret
      0.47            -0.1        0.41        perf-profile.children.cycles-pp.sync_regs
      0.32 ±  4%      -0.1        0.27 ±  3%  perf-profile.children.cycles-pp.get_unmapped_area
      0.26 ±  3%      -0.1        0.21 ±  3%  perf-profile.children.cycles-pp.__perf_sw_event
      0.21 ±  4%      -0.0        0.17 ±  5%  perf-profile.children.cycles-pp.find_vma
      0.27 ±  3%      -0.0        0.22 ±  3%  perf-profile.children.cycles-pp.unmapped_area_topdown
      0.11 ± 42%      -0.0        0.07 ±  7%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      0.29 ±  2%      -0.0        0.24 ±  3%  perf-profile.children.cycles-pp.arch_get_unmapped_area_topdown
      0.19 ±  4%      -0.0        0.15 ±  2%  perf-profile.children.cycles-pp.___perf_sw_event
      0.13 ±  3%      -0.0        0.10 ±  4%  perf-profile.children.cycles-pp.vma_policy_mof
      0.59 ±  3%      -0.0        0.56 ±  2%  perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
      0.16 ±  2%      -0.0        0.13        perf-profile.children.cycles-pp.__rb_insert_augmented
      0.19 ±  2%      -0.0        0.17 ±  4%  perf-profile.children.cycles-pp.perf_event_mmap
      0.08 ± 10%      -0.0        0.06 ±  6%  perf-profile.children.cycles-pp.vmacache_find
      0.08 ± 10%      -0.0        0.06 ±  6%  perf-profile.children.cycles-pp.__fget
      0.08            -0.0        0.07 ±  6%  perf-profile.children.cycles-pp.d_path
      0.09 ±  4%      +0.0        0.13 ±  3%  perf-profile.children.cycles-pp.__vma_link_rb
      0.01 ±173%      +0.1        0.12 ±  4%  perf-profile.children.cycles-pp.osq_unlock
      1.19 ±  3%      +0.2        1.36 ±  2%  perf-profile.children.cycles-pp.exit_to_usermode_loop
      1.19 ±  3%      +0.2        1.36 ±  2%  perf-profile.children.cycles-pp.task_work_run
      1.19 ±  3%      +0.2        1.36 ±  2%  perf-profile.children.cycles-pp.task_numa_work
      1.66 ±  2%      +0.4        2.01        perf-profile.children.cycles-pp.page_fault
      1.59 ±  2%      +0.4        1.96        perf-profile.children.cycles-pp.do_page_fault
      1.58 ±  2%      +0.4        1.95        perf-profile.children.cycles-pp.__do_page_fault
      0.66            +0.5        1.20 ±  4%  perf-profile.children.cycles-pp.handle_mm_fault
      0.47            +0.6        1.04 ±  4%  perf-profile.children.cycles-pp.__handle_mm_fault
      1.09            -0.1        0.96 ±  2%  perf-profile.self.cycles-pp.rwsem_spin_on_owner
      0.67 ±  2%      -0.1        0.59 ±  2%  perf-profile.self.cycles-pp.vma_interval_tree_insert
      0.62 ±  2%      -0.1        0.55 ±  3%  perf-profile.self.cycles-pp.native_irq_return_iret
      0.46            -0.1        0.40        perf-profile.self.cycles-pp.sync_regs
      0.43            -0.1        0.37 ±  2%  perf-profile.self.cycles-pp.swapgs_restore_regs_and_return_to_usermode
      0.30 ±  5%      -0.1        0.25 ±  3%  perf-profile.self.cycles-pp.__do_page_fault
      0.11 ± 42%      -0.0        0.07 ±  7%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      0.27 ±  3%      -0.0        0.22 ±  3%  perf-profile.self.cycles-pp.unmapped_area_topdown
      0.17 ±  3%      -0.0        0.13 ±  3%  perf-profile.self.cycles-pp.___perf_sw_event
      0.14 ±  8%      -0.0        0.11        perf-profile.self.cycles-pp.mmap_region
      0.18 ±  2%      -0.0        0.15 ±  2%  perf-profile.self.cycles-pp.handle_mm_fault
      0.15            -0.0        0.12 ±  4%  perf-profile.self.cycles-pp.__rb_insert_augmented
      0.12 ±  4%      -0.0        0.10 ±  7%  perf-profile.self.cycles-pp.find_vma
      0.08 ±  5%      -0.0        0.06        perf-profile.self.cycles-pp.vma_policy_mof
      0.08 ±  8%      -0.0        0.06 ±  6%  perf-profile.self.cycles-pp.vmacache_find
      0.08 ± 10%      -0.0        0.06 ±  6%  perf-profile.self.cycles-pp.__fget
      0.07 ±  6%      -0.0        0.05 ±  8%  perf-profile.self.cycles-pp.__perf_sw_event
      0.06            -0.0        0.05        perf-profile.self.cycles-pp.d_path
      0.08 ±  5%      +0.0        0.10 ±  7%  perf-profile.self.cycles-pp.down_write
      0.11 ±  4%      +0.0        0.16 ±  2%  perf-profile.self.cycles-pp.up_write
      0.00            +0.1        0.08 ±  5%  perf-profile.self.cycles-pp.__vma_link_rb
      0.18 ±  9%      +0.1        0.28 ±  6%  perf-profile.self.cycles-pp.rwsem_down_write_failed
      0.01 ±173%      +0.1        0.12 ±  4%  perf-profile.self.cycles-pp.osq_unlock
      1.05 ±  4%      +0.2        1.26 ±  2%  perf-profile.self.cycles-pp.task_numa_work
      0.35 ±  2%      +0.6        0.95 ±  5%  perf-profile.self.cycles-pp.__handle_mm_fault


                                                                                
                           vm-scalability.time.system_time                      
                                                                                
  30000 +-+-----------------------------------------------------------------+   
        |                       O                               O           |   
  25000 O-O  O O  O O  O O O  O    O O  O O O  O O  O O  O O O              |   
        |.+..+.+..+.+..+.+.+..+.+..+.+..+.+          .+..+.+.+..+.+..+.+..+.|   
        |                                 :    +    +                       |   
  20000 +-+                               :    :    :                       |   
        |                                  :  ::   :                        |   
  15000 +-+                                :  : :  :                        |   
        |                                  :  : :  :                        |   
  10000 +-+                                :  : :  :                        |   
        |                                  : :  : :                         |   
        |                                  : :  : :                         |   
   5000 +-+                                 ::   ::                         |   
        |                                   :    :                          |   
      0 +-+-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                         vm-scalability.time.elapsed_time                       
                                                                                
  400 +-+-------------------------------------------------------------------+   
      O  O O  O O  O O  O O  O O  O O  O O  O O  O O  O O  O O  O           |   
  350 +-+                                                                   |   
  300 +-++.+..+.+..+.+..+.+..+.+..+.+..+.+    +    +..+.+..+.+..+.+..+.+..+.|   
      |                                  :    :    :                        |   
  250 +-+                                 :   ::   :                        |   
      |                                   :  : :  :                         |   
  200 +-+                                 :  : :  :                         |   
      |                                   :  : :  :                         |   
  150 +-+                                  : :  : :                         |   
  100 +-+                                  : :  : :                         |   
      |                                    : :  : :                         |   
   50 +-+                                  ::   ::                          |   
      |                                     :    :                          |   
    0 +-+-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                       vm-scalability.time.elapsed_time.max                     
                                                                                
  400 +-+-------------------------------------------------------------------+   
      O  O O  O O  O O  O O  O O  O O  O O  O O  O O  O O  O O  O           |   
  350 +-+                                                                   |   
  300 +-++.+..+.+..+.+..+.+..+.+..+.+..+.+    +    +..+.+..+.+..+.+..+.+..+.|   
      |                                  :    :    :                        |   
  250 +-+                                 :   ::   :                        |   
      |                                   :  : :  :                         |   
  200 +-+                                 :  : :  :                         |   
      |                                   :  : :  :                         |   
  150 +-+                                  : :  : :                         |   
  100 +-+                                  : :  : :                         |   
      |                                    : :  : :                         |   
   50 +-+                                  ::   ::                          |   
      |                                     :    :                          |   
    0 +-+-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                               vm-scalability.throughput                        
                                                                                
    2e+07 +-+---------------------------------------------------------------+   
  1.8e+07 +-+..+.+.+..+.+..+.+.+..+.+.+..+.+    +   +..+.+..+.+.+..+.+.+..+.|   
          |                                :    :   :                       |   
  1.6e+07 O-O  O O O  O O  O O O  O O O  O O O  O O O  O O  O O O           |   
  1.4e+07 +-+                              :   ::   :                       |   
          |                                 :  : : :                        |   
  1.2e+07 +-+                               :  : : :                        |   
    1e+07 +-+                               :  : : :                        |   
    8e+06 +-+                               : :  : :                        |   
          |                                 : :  : :                        |   
    6e+06 +-+                               : :  : :                        |   
    4e+06 +-+                                ::   :                         |   
          |                                  :    :                         |   
    2e+06 +-+                                :    :                         |   
        0 +-+---------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                vm-scalability.median                           
                                                                                
  250000 +-+----------------------------------------------------------------+   
         |.+..+.+..+.+.+..+.+..+.+.+..+    +   +    +.+..+.+..+.+.+..+.+..+.|   
         |                                 :   :    :                       |   
  200000 O-O  O O  O O O  O O  O O O  O O  O O O  O O O  O O  O O           |   
         |                                 :   ::   :                       |   
         |                                  : : :  :                        |   
  150000 +-+                                : : :  :                        |   
         |                                  : : :  :                        |   
  100000 +-+                                : :  : :                        |   
         |                                  : :  : :                        |   
         |                                  : :  : :                        |   
   50000 +-+                                 :   ::                         |   
         |                                   :    :                         |   
         |                                   :    :                         |   
       0 +-+----------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-bdw-ep3b: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 64G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode:
  gcc-7/performance/x86_64-rhel-7.2/100%/debian-x86_64-2018-04-03.cgz/300s/lkp-bdw-ep3b/shell8/unixbench/0xb00002e

commit: 
  f32f004cdd ("ida: Convert to XArray")
  eb797a8ee0 ("page cache: Rearrange address_space")

f32f004cddf86d63 eb797a8ee0ab4cd03df556980c 
---------------- -------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
           :4           25%           1:4     dmesg.WARNING:at_ip___perf_sw_event/0x
           :4           25%           1:4     dmesg.WARNING:at_ip__slab_free/0x
           :4           25%           1:4     dmesg.WARNING:at_ip_filename_lookup/0x
          1:4          -25%            :4     dmesg.WARNING:at_ip_fsnotify/0x
          1:4          -25%            :4     dmesg.WARNING:at_ip_perf_event_mmap_output/0x
          1:4            9%           2:4     perf-profile.children.cycles-pp.error_entry
          1:4            7%           1:4     perf-profile.self.cycles-pp.error_entry
         %stddev     %change         %stddev
             \          |                \  
     39940           +20.9%      48288        unixbench.score
    641.38            +1.0%     647.51        unixbench.time.elapsed_time
    641.38            +1.0%     647.51        unixbench.time.elapsed_time.max
  24220392           +29.1%   31263367 ±  2%  unixbench.time.involuntary_context_switches
 1.836e+09           +20.8%  2.217e+09        unixbench.time.minor_page_faults
      7052            +2.1%       7200        unixbench.time.percent_of_cpu_this_job_got
     36337            -2.7%      35347        unixbench.time.system_time
      8893           +26.8%      11279 ±  2%  unixbench.time.user_time
  85632909            +4.3%   89281018        unixbench.time.voluntary_context_switches
  15301176           +21.7%   18622462        unixbench.workload
  42884872           +19.0%   51033794        softirqs.RCU
  43387938 ±  6%     -12.9%   37797790 ±  2%  cpuidle.C1.usage
   1155156 ±  4%     -24.0%     878414 ±  4%  cpuidle.POLL.usage
    152008 ±  9%     +15.2%     175048 ±  7%  meminfo.DirectMap4k
    255196           +13.6%     289806        meminfo.Shmem
    100.25           +26.7%     127.00 ±  2%  vmstat.procs.r
    337268            +3.6%     349315        vmstat.system.cs
     19.30            -1.7       17.59        mpstat.cpu.idle%
      0.01 ± 30%      +0.0        0.02 ± 35%  mpstat.cpu.iowait%
     15.54            +3.6       19.11        mpstat.cpu.usr%
 7.086e+08           +20.8%   8.56e+08 ±  2%  numa-numastat.node0.local_node
 7.086e+08           +20.8%  8.561e+08 ±  2%  numa-numastat.node0.numa_hit
 7.053e+08           +21.3%  8.558e+08        numa-numastat.node1.local_node
 7.053e+08           +21.3%  8.558e+08        numa-numastat.node1.numa_hit
      2274            +2.1%       2322        turbostat.Avg_MHz
  43383925 ±  6%     -12.9%   37793914 ±  2%  turbostat.C1
     14.41 ±  2%     -17.4%      11.90 ±  7%  turbostat.CPU%c1
    243.24            +2.2%     248.48        turbostat.PkgWatt
     14.16            +8.1%      15.30        turbostat.RAMWatt
      6807 ±  4%     +48.1%      10084 ±  5%  slabinfo.files_cache.active_objs
      6807 ±  4%     +48.2%      10090 ±  5%  slabinfo.files_cache.num_objs
      3616           +23.8%       4475 ±  5%  slabinfo.kmalloc-256.active_objs
      3616           +23.8%       4475 ±  5%  slabinfo.kmalloc-256.num_objs
     17134 ±  4%     +13.8%      19493 ±  2%  slabinfo.proc_inode_cache.active_objs
     17134 ±  4%     +14.0%      19524 ±  2%  slabinfo.proc_inode_cache.num_objs
      3393 ±  6%     +17.7%       3994 ±  3%  slabinfo.sock_inode_cache.active_objs
      3393 ±  6%     +17.7%       3994 ±  3%  slabinfo.sock_inode_cache.num_objs
      1241 ±  8%     +15.0%       1427 ±  6%  slabinfo.task_group.active_objs
      1241 ±  8%     +15.0%       1427 ±  6%  slabinfo.task_group.num_objs
    713963 ± 14%     -20.9%     564405 ±  4%  numa-meminfo.node0.FilePages
     20539 ±  9%     -19.7%      16496 ± 15%  numa-meminfo.node0.Mapped
   1281514 ±  9%     -10.6%    1145886 ±  4%  numa-meminfo.node0.MemUsed
    683.75 ± 11%     +54.1%       1053 ± 12%  numa-meminfo.node0.Mlocked
     48239 ±  4%     -19.8%      38686 ± 17%  numa-meminfo.node0.SReclaimable
    669804 ± 15%     +27.8%     856173 ±  3%  numa-meminfo.node1.FilePages
      4719 ± 56%    +227.5%      15456 ± 43%  numa-meminfo.node1.Inactive
      4644 ± 57%    +232.0%      15419 ± 43%  numa-meminfo.node1.Inactive(anon)
     14609 ± 12%     +47.2%      21499 ± 11%  numa-meminfo.node1.Mapped
     40126 ±  6%     +29.6%      52000 ± 12%  numa-meminfo.node1.SReclaimable
    119605 ± 96%    +131.3%     276593 ±  3%  numa-meminfo.node1.Shmem
    130207           +18.0%     153583 ±  5%  numa-meminfo.node1.Slab
    547652 ±  4%      +5.4%     577200 ±  4%  numa-meminfo.node1.Unevictable
    178549 ± 14%     -21.0%     141123 ±  4%  numa-vmstat.node0.nr_file_pages
      5179 ±  9%     -20.3%       4130 ± 16%  numa-vmstat.node0.nr_mapped
    170.50 ± 12%     +54.5%     263.50 ± 13%  numa-vmstat.node0.nr_mlock
     12061 ±  4%     -19.8%       9670 ± 17%  numa-vmstat.node0.nr_slab_reclaimable
 3.542e+08           +21.5%  4.304e+08 ±  2%  numa-vmstat.node0.numa_hit
 3.542e+08           +21.5%  4.304e+08 ±  2%  numa-vmstat.node0.numa_local
    167459 ± 15%     +27.9%     214111 ±  3%  numa-vmstat.node1.nr_file_pages
      1150 ± 57%    +234.3%       3845 ± 44%  numa-vmstat.node1.nr_inactive_anon
      3673 ± 11%     +49.4%       5488 ± 13%  numa-vmstat.node1.nr_mapped
     29911 ± 96%    +131.3%      69188 ±  3%  numa-vmstat.node1.nr_shmem
     10036 ±  6%     +29.5%      13000 ± 12%  numa-vmstat.node1.nr_slab_reclaimable
     22541           +12.7%      25401 ±  3%  numa-vmstat.node1.nr_slab_unreclaimable
    136898 ±  4%      +5.4%     144317 ±  4%  numa-vmstat.node1.nr_unevictable
      1150 ± 57%    +234.3%       3845 ± 44%  numa-vmstat.node1.nr_zone_inactive_anon
    136898 ±  4%      +5.4%     144317 ±  4%  numa-vmstat.node1.nr_zone_unevictable
 3.525e+08           +22.0%  4.301e+08 ±  2%  numa-vmstat.node1.numa_hit
 3.523e+08           +22.0%  4.299e+08 ±  2%  numa-vmstat.node1.numa_local
    149893            +6.3%     159383        proc-vmstat.nr_active_anon
    345938            +2.7%     355212        proc-vmstat.nr_file_pages
      6075            +9.3%       6643        proc-vmstat.nr_inactive_anon
     30490            +2.2%      31159        proc-vmstat.nr_kernel_stack
      8870            +8.1%       9585        proc-vmstat.nr_mapped
      9254            +2.3%       9471        proc-vmstat.nr_page_table_pages
     63803           +13.6%      72460        proc-vmstat.nr_shmem
     22097            +2.6%      22671        proc-vmstat.nr_slab_reclaimable
     47965            +6.9%      51274        proc-vmstat.nr_slab_unreclaimable
    149893            +6.3%     159383        proc-vmstat.nr_zone_active_anon
      6075            +9.3%       6643        proc-vmstat.nr_zone_inactive_anon
 1.414e+09           +21.1%  1.712e+09        proc-vmstat.numa_hit
 1.414e+09           +21.1%  1.712e+09        proc-vmstat.numa_local
     47447           +10.8%      52569        proc-vmstat.pgactivate
 1.487e+09           +21.1%  1.801e+09        proc-vmstat.pgalloc_normal
 1.846e+09           +20.7%  2.228e+09        proc-vmstat.pgfault
 1.487e+09           +21.1%  1.801e+09        proc-vmstat.pgfree
     63795           +20.6%      76962        proc-vmstat.thp_deferred_split_page
     63790           +20.6%      76940        proc-vmstat.thp_fault_alloc
  26894638           +20.9%   32515246        proc-vmstat.unevictable_pgs_culled
 1.556e+13            +4.9%  1.632e+13        perf-stat.branch-instructions
      1.54            +0.2        1.75        perf-stat.branch-miss-rate%
 2.399e+11           +19.4%  2.863e+11        perf-stat.branch-misses
 1.034e+11           +17.1%   1.21e+11 ±  2%  perf-stat.cache-misses
 9.784e+11           +18.6%  1.161e+12        perf-stat.cache-references
  2.18e+08            +4.6%  2.281e+08        perf-stat.context-switches
      1.60            -1.0%       1.59        perf-stat.cpi
 1.281e+14            +3.2%  1.322e+14        perf-stat.cpu-cycles
  46095973           +22.0%   56220581        perf-stat.cpu-migrations
      0.19            +0.0        0.23 ±  2%  perf-stat.dTLB-load-miss-rate%
 4.148e+10           +23.3%  5.113e+10 ±  2%  perf-stat.dTLB-load-misses
 2.213e+13            +2.1%  2.261e+13        perf-stat.dTLB-loads
 5.537e+09           +22.2%  6.767e+09 ±  2%  perf-stat.dTLB-store-misses
 8.001e+12           +20.3%  9.625e+12        perf-stat.dTLB-stores
 2.034e+10           +20.1%  2.443e+10        perf-stat.iTLB-load-misses
   1.3e+10           +17.1%  1.522e+10        perf-stat.iTLB-loads
 7.991e+13            +4.3%  8.332e+13        perf-stat.instructions
      3929           -13.2%       3411        perf-stat.instructions-per-iTLB-miss
  1.81e+09           +20.7%  2.184e+09        perf-stat.minor-faults
     91.68            -2.3       89.39        perf-stat.node-load-miss-rate%
 3.395e+10           +14.5%  3.889e+10        perf-stat.node-load-misses
 3.083e+09           +49.7%  4.616e+09 ±  3%  perf-stat.node-loads
 1.018e+10           +20.2%  1.224e+10 ±  2%  perf-stat.node-store-misses
 8.896e+09           +21.3%  1.079e+10 ±  2%  perf-stat.node-stores
  1.81e+09           +20.7%  2.184e+09        perf-stat.page-faults
   5222880           -14.3%    4475405        perf-stat.path-length
      1942 ±  7%     +37.3%       2665 ± 18%  sched_debug.cfs_rq:/.exec_clock.stddev
     47.72 ±  7%     +18.8%      56.67 ±  2%  sched_debug.cfs_rq:/.load_avg.avg
      4.95 ±  9%     +44.0%       7.14 ±  7%  sched_debug.cfs_rq:/.load_avg.min
    265960 ±  8%     +36.3%     362493 ± 15%  sched_debug.cfs_rq:/.min_vruntime.stddev
      0.33 ±  6%     -11.5%       0.29 ±  5%  sched_debug.cfs_rq:/.nr_running.stddev
      2.77 ±  7%     +88.4%       5.23 ±  3%  sched_debug.cfs_rq:/.nr_spread_over.avg
      7.18 ±  5%     +79.1%      12.86 ± 15%  sched_debug.cfs_rq:/.nr_spread_over.max
      0.27 ± 52%    +358.3%       1.25 ± 25%  sched_debug.cfs_rq:/.nr_spread_over.min
      1.57 ±  2%     +40.9%       2.21 ±  2%  sched_debug.cfs_rq:/.nr_spread_over.stddev
     27.36 ±  5%     +13.2%      30.98 ±  6%  sched_debug.cfs_rq:/.removed.load_avg.avg
      1263 ±  5%     +14.1%       1442 ±  6%  sched_debug.cfs_rq:/.removed.runnable_sum.avg
    265962 ±  8%     +36.3%     362499 ± 15%  sched_debug.cfs_rq:/.spread0.stddev
      8169 ± 28%     +64.2%      13411 ± 21%  sched_debug.cpu.avg_idle.min
    298.80 ± 11%     +57.4%     470.43 ± 28%  sched_debug.cpu.cpu_load[0].max
      7.46 ±  5%     +34.4%      10.03 ± 12%  sched_debug.cpu.cpu_load[1].avg
    212.55 ± 15%     +52.5%     324.14 ± 17%  sched_debug.cpu.cpu_load[1].max
     25.69 ±  9%     +52.4%      39.15 ± 19%  sched_debug.cpu.cpu_load[1].stddev
      7.18 ±  3%     +32.7%       9.53 ±  5%  sched_debug.cpu.cpu_load[2].avg
    151.27 ± 16%     +51.9%     229.80 ± 11%  sched_debug.cpu.cpu_load[2].max
     19.45 ± 11%     +50.2%      29.22 ± 10%  sched_debug.cpu.cpu_load[2].stddev
      6.92 ±  2%     +29.5%       8.96 ±  4%  sched_debug.cpu.cpu_load[3].avg
    106.45 ± 23%     +52.6%     162.50 ± 15%  sched_debug.cpu.cpu_load[3].max
     14.94 ± 15%     +48.0%      22.10 ± 10%  sched_debug.cpu.cpu_load[3].stddev
      6.45 ±  2%     +25.9%       8.12 ±  6%  sched_debug.cpu.cpu_load[4].avg
     79.82 ± 34%     +56.2%     124.64 ± 25%  sched_debug.cpu.cpu_load[4].max
     11.70 ± 23%     +47.3%      17.23 ± 17%  sched_debug.cpu.cpu_load[4].stddev
    422565 ± 24%     +22.7%     518276 ± 15%  sched_debug.cpu.load.max
      0.00 ±  3%     +23.4%       0.00 ± 14%  sched_debug.cpu.next_balance.stddev
      1.14 ±  5%     +31.4%       1.50 ±  7%  sched_debug.cpu.nr_running.avg
      3.16 ±  2%     +38.1%       4.36 ±  9%  sched_debug.cpu.nr_running.max
      0.71 ±  2%     +33.7%       0.95 ±  7%  sched_debug.cpu.nr_running.stddev
     39284 ±  7%     +47.6%      57979 ± 20%  sched_debug.cpu.nr_switches.stddev
      1658 ± 12%     +73.9%       2884 ± 11%  sched_debug.cpu.nr_uninterruptible.max
     -1902          +212.1%      -5938        sched_debug.cpu.nr_uninterruptible.min
    716.66 ±  9%    +107.5%       1487 ± 30%  sched_debug.cpu.nr_uninterruptible.stddev
     43221 ±  7%     +46.5%      63340 ± 18%  sched_debug.cpu.sched_count.stddev
    351785 ±  3%     -14.1%     302212        sched_debug.cpu.sched_goidle.min
     21765 ±  7%     +51.5%      32969 ± 24%  sched_debug.cpu.sched_goidle.stddev
    148360           +23.9%     183753 ±  2%  sched_debug.cpu.ttwu_local.avg
    152765           +24.4%     190065 ±  2%  sched_debug.cpu.ttwu_local.max
      2051 ± 10%     +87.1%       3838 ± 32%  sched_debug.cpu.ttwu_local.stddev
      0.00 ± 25%  +2.1e+05%       0.56 ± 46%  sched_debug.rt_rq:/.rt_time.avg
      0.02 ± 25%  +2.1e+05%      49.46 ± 46%  sched_debug.rt_rq:/.rt_time.max
      0.00 ± 25%  +2.1e+05%       5.24 ± 46%  sched_debug.rt_rq:/.rt_time.stddev
     20.84 ±  2%      -4.4       16.42 ±  3%  perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.unlink_file_vma
     21.69 ±  2%      -4.4       17.32 ±  2%  perf-profile.calltrace.cycles-pp.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.unlink_file_vma.free_pgtables
     17.27 ±  2%      -3.4       13.86 ±  3%  perf-profile.calltrace.cycles-pp.call_rwsem_down_write_failed.down_write.unlink_file_vma.free_pgtables.exit_mmap
     17.44            -3.4       14.05 ±  3%  perf-profile.calltrace.cycles-pp.down_write.unlink_file_vma.free_pgtables.exit_mmap.mmput
     36.96            -3.1       33.88        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
     36.94            -3.1       33.87        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
     12.69            -2.2       10.53 ±  2%  perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
     12.71            -2.2       10.55 ±  2%  perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
     13.36            -2.0       11.41        perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64
      8.96 ±  2%      -1.9        7.03 ±  3%  perf-profile.calltrace.cycles-pp.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.__vma_adjust.__split_vma
      8.63 ±  2%      -1.9        6.71 ±  3%  perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.__vma_adjust
     13.61            -1.9       11.71        perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
      8.11            -1.8        6.36 ±  3%  perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.copy_process
      8.44            -1.7        6.70 ±  2%  perf-profile.calltrace.cycles-pp.call_rwsem_down_write_failed.down_write.copy_process._do_fork.do_syscall_64
      8.44            -1.7        6.70 ±  2%  perf-profile.calltrace.cycles-pp.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.copy_process._do_fork
      8.50            -1.7        6.76 ±  2%  perf-profile.calltrace.cycles-pp.down_write.copy_process._do_fork.do_syscall_64.entry_SYSCALL_64_after_hwframe
      8.70 ±  2%      -1.7        7.01 ±  3%  perf-profile.calltrace.cycles-pp.unlink_file_vma.free_pgtables.exit_mmap.mmput.flush_old_exec
      9.06            -1.6        7.47 ±  2%  perf-profile.calltrace.cycles-pp.unlink_file_vma.free_pgtables.exit_mmap.mmput.do_exit
      9.36 ±  2%      -1.6        7.77 ±  2%  perf-profile.calltrace.cycles-pp.free_pgtables.exit_mmap.mmput.flush_old_exec.load_elf_binary
      9.71            -1.5        8.21 ±  2%  perf-profile.calltrace.cycles-pp.free_pgtables.exit_mmap.mmput.do_exit.do_group_exit
      6.71 ±  2%      -1.5        5.26 ±  3%  perf-profile.calltrace.cycles-pp.call_rwsem_down_write_failed.down_write.__vma_adjust.__split_vma.mprotect_fixup
      6.77 ±  2%      -1.4        5.32 ±  3%  perf-profile.calltrace.cycles-pp.down_write.__vma_adjust.__split_vma.mprotect_fixup.do_mprotect_pkey
      9.54            -1.3        8.23 ±  3%  perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
     10.17            -1.3        8.86 ±  4%  perf-profile.calltrace.cycles-pp.secondary_startup_64
      9.31            -1.3        8.01 ±  4%  perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
     10.04            -1.3        8.74 ±  3%  perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
      7.63 ±  2%      -1.3        6.33 ±  2%  perf-profile.calltrace.cycles-pp.__vma_adjust.__split_vma.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect
     10.04            -1.3        8.75 ±  3%  perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
     10.04            -1.3        8.75 ±  3%  perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
      7.34            -1.3        6.05 ±  2%  perf-profile.calltrace.cycles-pp.do_munmap.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff
      7.71 ±  2%      -1.3        6.43 ±  2%  perf-profile.calltrace.cycles-pp.__split_vma.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64
     10.84            -1.2        9.62        perf-profile.calltrace.cycles-pp.copy_process._do_fork.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
      8.01 ±  2%      -1.2        6.80 ±  2%  perf-profile.calltrace.cycles-pp.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe
      8.09 ±  2%      -1.2        6.90 ±  2%  perf-profile.calltrace.cycles-pp.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe
      8.10 ±  2%      -1.2        6.90 ±  2%  perf-profile.calltrace.cycles-pp.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe
     11.17            -1.2        9.99        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_fork
     11.16            -1.2        9.99        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
     11.15            -1.2        9.98        perf-profile.calltrace.cycles-pp._do_fork.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
     11.80            -1.1       10.73        perf-profile.calltrace.cycles-pp.__libc_fork
      4.28            -1.1        3.21 ±  3%  perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.vma_link
      4.44 ±  2%      -1.0        3.47 ±  2%  perf-profile.calltrace.cycles-pp.call_rwsem_down_write_failed.down_write.unlink_file_vma.free_pgtables.unmap_region
      4.45            -1.0        3.49 ±  2%  perf-profile.calltrace.cycles-pp.down_write.unlink_file_vma.free_pgtables.unmap_region.do_munmap
     11.18            -1.0       10.22        perf-profile.calltrace.cycles-pp.exit_mmap.mmput.flush_old_exec.load_elf_binary.search_binary_handler
      4.48            -1.0        3.52 ±  2%  perf-profile.calltrace.cycles-pp.unlink_file_vma.free_pgtables.unmap_region.do_munmap.mmap_region
     11.19            -1.0       10.23        perf-profile.calltrace.cycles-pp.mmput.flush_old_exec.load_elf_binary.search_binary_handler.__do_execve_file
      4.43            -0.9        3.48 ±  8%  perf-profile.calltrace.cycles-pp.call_rwsem_down_write_failed.down_write.vma_link.mmap_region.do_mmap
      4.43            -0.9        3.48 ±  8%  perf-profile.calltrace.cycles-pp.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.vma_link.mmap_region
      4.67            -0.9        3.73 ±  2%  perf-profile.calltrace.cycles-pp.free_pgtables.unmap_region.do_munmap.mmap_region.do_mmap
      4.45            -0.9        3.51 ±  9%  perf-profile.calltrace.cycles-pp.down_write.vma_link.mmap_region.do_mmap.vm_mmap_pgoff
     11.30            -0.9       10.38        perf-profile.calltrace.cycles-pp.flush_old_exec.load_elf_binary.search_binary_handler.__do_execve_file.__x64_sys_execve
      4.85            -0.9        3.98 ±  2%  perf-profile.calltrace.cycles-pp.unmap_region.do_munmap.mmap_region.do_mmap.vm_mmap_pgoff
      5.40            -0.8        4.57 ±  2%  perf-profile.calltrace.cycles-pp.vma_link.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff
      2.25            -0.5        1.78 ±  3%  perf-profile.calltrace.cycles-pp.down_write.__vma_adjust.__split_vma.do_munmap.mmap_region
      2.25            -0.5        1.78 ±  3%  perf-profile.calltrace.cycles-pp.call_rwsem_down_write_failed.down_write.__vma_adjust.__split_vma.do_munmap
      2.42            -0.4        1.98 ±  2%  perf-profile.calltrace.cycles-pp.__vma_adjust.__split_vma.do_munmap.mmap_region.do_mmap
      2.43            -0.4        1.99 ±  2%  perf-profile.calltrace.cycles-pp.__split_vma.do_munmap.mmap_region.do_mmap.vm_mmap_pgoff
     13.17            -0.3       12.82        perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group
     13.17            -0.3       12.83        perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      2.69 ±  2%      -0.1        2.56        perf-profile.calltrace.cycles-pp.filemap_map_pages.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
      0.56            +0.2        0.72 ±  2%  perf-profile.calltrace.cycles-pp._IO_default_xsputn
      0.63            +0.2        0.78 ±  3%  perf-profile.calltrace.cycles-pp.copy_strings.__do_execve_file.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.75            +0.2        0.92 ±  2%  perf-profile.calltrace.cycles-pp.wp_page_copy.do_wp_page.__handle_mm_fault.handle_mm_fault.__do_page_fault
      0.82            +0.2        1.01 ±  2%  perf-profile.calltrace.cycles-pp.do_wp_page.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
      0.94 ±  2%      +0.2        1.16        perf-profile.calltrace.cycles-pp._dl_addr
      0.64 ±  2%      +0.2        0.87 ±  4%  perf-profile.calltrace.cycles-pp.ret_from_fork
      0.63 ±  2%      +0.2        0.87 ±  4%  perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
      0.67 ±  2%      +0.2        0.91 ±  4%  perf-profile.calltrace.cycles-pp.page_remove_rmap.unmap_page_range.unmap_vmas.exit_mmap.mmput
      0.87            +0.2        1.12 ±  2%  perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.88            +0.2        1.13        perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.setlocale
      0.70 ±  2%      +0.3        0.96 ±  3%  perf-profile.calltrace.cycles-pp.arch_tlb_finish_mmu.tlb_finish_mmu.exit_mmap.mmput.flush_old_exec
      1.10            +0.3        1.37 ±  2%  perf-profile.calltrace.cycles-pp.__strcoll_l
      0.70 ±  2%      +0.3        0.97 ±  3%  perf-profile.calltrace.cycles-pp.tlb_finish_mmu.exit_mmap.mmput.flush_old_exec.load_elf_binary
      1.04            +0.3        1.32 ±  2%  perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.setlocale
      0.89            +0.3        1.17 ±  3%  perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.flush_old_exec
      0.95            +0.3        1.25 ±  3%  perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.flush_old_exec.load_elf_binary
      1.15            +0.3        1.46        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.setlocale
      4.73            +0.3        5.04        perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
      1.15            +0.3        1.46 ±  2%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.setlocale
      1.07            +0.3        1.39        perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
      1.09 ±  2%      +0.3        1.42        perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
      1.11 ±  2%      +0.3        1.45        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.mmap64
      1.11            +0.3        1.44        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
      0.26 ±100%      +0.3        0.60        perf-profile.calltrace.cycles-pp.copy_page_range.copy_process._do_fork.do_syscall_64.entry_SYSCALL_64_after_hwframe
      4.85            +0.3        5.19        perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
      0.39 ± 57%      +0.4        0.74 ±  3%  perf-profile.calltrace.cycles-pp.wait4
      1.20            +0.4        1.56        perf-profile.calltrace.cycles-pp.mmap64
     13.89            +0.4       14.25        perf-profile.calltrace.cycles-pp.load_elf_binary.search_binary_handler.__do_execve_file.__x64_sys_execve.do_syscall_64
     13.91            +0.4       14.28        perf-profile.calltrace.cycles-pp.search_binary_handler.__do_execve_file.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.78 ±  2%      +0.4        2.19        perf-profile.calltrace.cycles-pp.vfprintf.__vsnprintf_chk
      0.55 ±  3%      +0.4        0.99 ±  3%  perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.elf_map.load_elf_binary
      0.60 ±  4%      +0.5        1.06 ±  3%  perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.elf_map.load_elf_binary.search_binary_handler
      0.66 ±  3%      +0.5        1.12 ±  2%  perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.elf_map.load_elf_binary.search_binary_handler.__do_execve_file
      5.43            +0.5        5.91        perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault
      5.45            +0.5        5.92        perf-profile.calltrace.cycles-pp.do_page_fault.page_fault
      0.00            +0.5        0.51        perf-profile.calltrace.cycles-pp.page_fault.setlocale
      0.00            +0.5        0.51 ±  3%  perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
      5.58            +0.5        6.09        perf-profile.calltrace.cycles-pp.page_fault
      0.00            +0.5        0.52 ±  4%  perf-profile.calltrace.cycles-pp.smpboot_thread_fn.kthread.ret_from_fork
      0.00            +0.5        0.53 ±  2%  perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
      1.37 ±  2%      +0.5        1.90 ±  5%  perf-profile.calltrace.cycles-pp.arch_tlb_finish_mmu.tlb_finish_mmu.exit_mmap.mmput.do_exit
      1.38 ±  2%      +0.5        1.90 ±  5%  perf-profile.calltrace.cycles-pp.tlb_finish_mmu.exit_mmap.mmput.do_exit.do_group_exit
      0.00            +0.5        0.53 ±  3%  perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
      0.00            +0.5        0.54 ±  2%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
      1.73 ±  2%      +0.5        2.26 ±  3%  perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.do_exit
      0.00            +0.5        0.54 ±  3%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
      0.00            +0.5        0.54 ±  3%  perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
      2.69            +0.5        3.24        perf-profile.calltrace.cycles-pp.__vsnprintf_chk
      1.80 ±  2%      +0.6        2.35 ±  3%  perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.do_exit.do_group_exit
      0.00            +0.6        0.56 ±  2%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
      0.00            +0.6        0.56 ±  3%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read
      2.46            +0.6        3.03 ±  2%  perf-profile.calltrace.cycles-pp.setlocale
      0.00            +0.6        0.58 ±  3%  perf-profile.calltrace.cycles-pp.write
      0.00            +0.6        0.58 ±  2%  perf-profile.calltrace.cycles-pp.do_faccessat.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.12 ±173%      +0.6        0.71 ±  4%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.wait4
      0.12 ±173%      +0.6        0.71 ±  4%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.wait4
      0.00            +0.6        0.61 ±  3%  perf-profile.calltrace.cycles-pp.read
      0.00            +0.6        0.61 ±  5%  perf-profile.calltrace.cycles-pp.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +0.6        0.62 ±  5%  perf-profile.calltrace.cycles-pp.kernel_wait4.__do_sys_wait4.do_syscall_64.entry_SYSCALL_64_after_hwframe.wait4
      0.00            +0.6        0.63 ±  4%  perf-profile.calltrace.cycles-pp.__do_sys_wait4.do_syscall_64.entry_SYSCALL_64_after_hwframe.wait4
      1.63 ±  2%      +0.7        2.31 ±  5%  perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu.exit_mmap
      0.00            +0.7        0.74 ±  3%  perf-profile.calltrace.cycles-pp.vma_link.mmap_region.do_mmap.vm_mmap_pgoff.elf_map
      0.12 ±173%      +0.8        0.91 ±  3%  perf-profile.calltrace.cycles-pp.do_munmap.vm_munmap.elf_map.load_elf_binary.search_binary_handler
      2.05 ±  2%      +0.8        2.84 ±  4%  perf-profile.calltrace.cycles-pp.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu.exit_mmap.mmput
      0.13 ±173%      +0.8        0.92 ±  4%  perf-profile.calltrace.cycles-pp.vm_munmap.elf_map.load_elf_binary.search_binary_handler.__do_execve_file
     15.67            +0.8       16.48        perf-profile.calltrace.cycles-pp.__do_execve_file.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
     15.68            +0.8       16.50        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
     15.68            +0.8       16.50        perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
     15.68            +0.8       16.50        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.execve
     15.71            +0.8       16.53        perf-profile.calltrace.cycles-pp.execve
      1.16 ±  3%      +0.9        2.05 ±  3%  perf-profile.calltrace.cycles-pp.elf_map.load_elf_binary.search_binary_handler.__do_execve_file.__x64_sys_execve
     42.48            -8.6       33.85 ±  3%  perf-profile.children.cycles-pp.osq_lock
     44.24            -8.5       35.76 ±  2%  perf-profile.children.cycles-pp.rwsem_down_write_failed
     44.25            -8.5       35.78 ±  2%  perf-profile.children.cycles-pp.call_rwsem_down_write_failed
     44.77            -8.4       36.37 ±  2%  perf-profile.children.cycles-pp.down_write
     22.58            -4.1       18.50 ±  2%  perf-profile.children.cycles-pp.unlink_file_vma
     24.10            -3.9       20.25 ±  2%  perf-profile.children.cycles-pp.free_pgtables
     13.81            -1.8       11.97        perf-profile.children.cycles-pp.ksys_mmap_pgoff
     69.44            -1.8       67.68        perf-profile.children.cycles-pp.do_syscall_64
     69.50            -1.8       67.74        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
     10.31 ±  2%      -1.5        8.80 ±  2%  perf-profile.children.cycles-pp.__vma_adjust
     13.93            -1.5       12.43        perf-profile.children.cycles-pp.mmap_region
     10.40            -1.5        8.90 ±  2%  perf-profile.children.cycles-pp.__split_vma
     14.23            -1.4       12.80        perf-profile.children.cycles-pp.do_mmap
     14.44            -1.4       13.06        perf-profile.children.cycles-pp.vm_mmap_pgoff
      9.67            -1.3        8.35 ±  4%  perf-profile.children.cycles-pp.cpuidle_enter_state
     24.58            -1.3       23.26        perf-profile.children.cycles-pp.mmput
     24.55            -1.3       23.24        perf-profile.children.cycles-pp.exit_mmap
      9.44            -1.3        8.12 ±  4%  perf-profile.children.cycles-pp.intel_idle
     10.17            -1.3        8.86 ±  4%  perf-profile.children.cycles-pp.secondary_startup_64
     10.17            -1.3        8.86 ±  4%  perf-profile.children.cycles-pp.cpu_startup_entry
     10.17            -1.3        8.86 ±  4%  perf-profile.children.cycles-pp.do_idle
     10.04            -1.3        8.75 ±  3%  perf-profile.children.cycles-pp.start_secondary
     11.07            -1.2        9.82        perf-profile.children.cycles-pp.copy_process
      8.02 ±  2%      -1.2        6.80 ±  2%  perf-profile.children.cycles-pp.mprotect_fixup
     11.38            -1.2       10.18        perf-profile.children.cycles-pp._do_fork
      8.10 ±  2%      -1.2        6.90 ±  2%  perf-profile.children.cycles-pp.do_mprotect_pkey
      8.10 ±  2%      -1.2        6.90 ±  2%  perf-profile.children.cycles-pp.__x64_sys_mprotect
     11.84            -1.1       10.77        perf-profile.children.cycles-pp.__libc_fork
     11.51            -0.9       10.57        perf-profile.children.cycles-pp.flush_old_exec
      8.20            -0.8        7.44        perf-profile.children.cycles-pp.do_munmap
      5.36            -0.6        4.79        perf-profile.children.cycles-pp.unmap_region
      5.70 ±  2%      -0.5        5.21        perf-profile.children.cycles-pp.filemap_map_pages
      5.81            -0.4        5.39        perf-profile.children.cycles-pp.vma_link
      0.29 ±  3%      -0.1        0.15        perf-profile.children.cycles-pp.radix_tree_next_chunk
      0.39 ±  3%      -0.0        0.36        perf-profile.children.cycles-pp.time
      0.16 ±  4%      -0.0        0.13        perf-profile.children.cycles-pp.find_busiest_group
      0.30            -0.0        0.29        perf-profile.children.cycles-pp.load_balance
      0.23            -0.0        0.21 ±  2%  perf-profile.children.cycles-pp.__strcasecmp
      0.17            -0.0        0.15 ±  3%  perf-profile.children.cycles-pp.fopen
      0.05            +0.0        0.06        perf-profile.children.cycles-pp.__put_task_struct
      0.05            +0.0        0.06        perf-profile.children.cycles-pp.vm_brk_flags
      0.05            +0.0        0.06        perf-profile.children.cycles-pp.__perf_event__output_id_sample
      0.05            +0.0        0.06        perf-profile.children.cycles-pp.security_mmap_addr
      0.05            +0.0        0.06        perf-profile.children.cycles-pp.selinux_file_open
      0.06            +0.0        0.07        perf-profile.children.cycles-pp.__switch_to
      0.07 ±  5%      +0.0        0.09 ±  5%  perf-profile.children.cycles-pp.__x64_sys_pipe
      0.07 ±  5%      +0.0        0.09 ±  5%  perf-profile.children.cycles-pp.do_pipe2
      0.07 ±  5%      +0.0        0.09 ±  5%  perf-profile.children.cycles-pp.unmap_single_vma
      0.07            +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.__errno_location
      0.05            +0.0        0.06 ±  6%  perf-profile.children.cycles-pp.mem_cgroup_throttle_swaprate
      0.05 ±  8%      +0.0        0.07 ±  7%  perf-profile.children.cycles-pp.perf_event_task_output
      0.05 ±  8%      +0.0        0.07 ±  7%  perf-profile.children.cycles-pp.load_elf_phdrs
      0.17 ±  2%      +0.0        0.18 ±  2%  perf-profile.children.cycles-pp.sched_ttwu_pending
      0.06 ±  6%      +0.0        0.08 ±  6%  perf-profile.children.cycles-pp.cpumask_next
      0.06            +0.0        0.07 ±  5%  perf-profile.children.cycles-pp.kfree
      0.08 ±  6%      +0.0        0.09        perf-profile.children.cycles-pp.do_signal
      0.08 ±  6%      +0.0        0.09        perf-profile.children.cycles-pp.memcpy
      0.08 ±  5%      +0.0        0.09 ±  4%  perf-profile.children.cycles-pp.__pipe
      0.08            +0.0        0.10 ±  5%  perf-profile.children.cycles-pp.vfs_getattr
      0.08 ±  5%      +0.0        0.09 ±  4%  perf-profile.children.cycles-pp.security_inode_getattr
      0.08 ±  5%      +0.0        0.09 ±  4%  perf-profile.children.cycles-pp.avc_has_perm
      0.07 ±  6%      +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.fsnotify
      0.07            +0.0        0.09 ±  5%  perf-profile.children.cycles-pp.__tsearch
      0.07 ±  7%      +0.0        0.08        perf-profile.children.cycles-pp.memchr
      0.05 ±  8%      +0.0        0.07 ±  6%  perf-profile.children.cycles-pp.file_has_perm
      0.05 ±  8%      +0.0        0.07 ±  6%  perf-profile.children.cycles-pp.__alloc_fd
      0.05            +0.0        0.07 ±  7%  perf-profile.children.cycles-pp.free_unref_page_commit
      0.05            +0.0        0.07 ±  7%  perf-profile.children.cycles-pp.up_read
      0.29            +0.0        0.30 ±  2%  perf-profile.children.cycles-pp.save_stack_trace_tsk
      0.13 ±  3%      +0.0        0.14 ±  3%  perf-profile.children.cycles-pp.pipe_wait
      0.06            +0.0        0.08 ±  6%  perf-profile.children.cycles-pp.__put_anon_vma
      0.06 ± 11%      +0.0        0.08 ±  6%  perf-profile.children.cycles-pp.wake_up_page_bit
      0.05 ±  9%      +0.0        0.07        perf-profile.children.cycles-pp.dup_fd
      0.19 ±  2%      +0.0        0.21        perf-profile.children.cycles-pp.dequeue_entity
      0.10 ±  5%      +0.0        0.11 ±  3%  perf-profile.children.cycles-pp.move_queued_task
      0.10 ±  7%      +0.0        0.12 ±  3%  perf-profile.children.cycles-pp.__list_add_valid
      0.08 ±  6%      +0.0        0.09 ±  4%  perf-profile.children.cycles-pp.find_next_bit
      0.07 ±  5%      +0.0        0.09 ±  7%  perf-profile.children.cycles-pp.arch_dup_task_struct
      0.11            +0.0        0.13 ±  3%  perf-profile.children.cycles-pp.find_get_entry
      0.07 ± 10%      +0.0        0.09 ±  4%  perf-profile.children.cycles-pp.cp_new_stat
      0.07 ±  7%      +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.load_new_mm_cr3
      0.07 ±  7%      +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.set_next_entity
      0.07 ±  6%      +0.0        0.09 ±  5%  perf-profile.children.cycles-pp.__install_special_mapping
      0.06 ±  6%      +0.0        0.08        perf-profile.children.cycles-pp._IO_setb
      0.07 ±  6%      +0.0        0.09 ±  5%  perf-profile.children.cycles-pp.trailing_symlink
      0.06 ±  6%      +0.0        0.08        perf-profile.children.cycles-pp.setup_new_exec
      0.06 ±  6%      +0.0        0.08        perf-profile.children.cycles-pp.perf_event_task
      0.05            +0.0        0.07 ±  6%  perf-profile.children.cycles-pp.__legitimize_mnt
      0.05 ±  8%      +0.0        0.07        perf-profile.children.cycles-pp.vma_gap_callbacks_rotate
      0.05            +0.0        0.07 ±  6%  perf-profile.children.cycles-pp.security_file_free
      0.05 ±  8%      +0.0        0.07        perf-profile.children.cycles-pp.down_write_killable
      0.08 ±  5%      +0.0        0.10 ±  4%  perf-profile.children.cycles-pp.available_idle_cpu
      0.08 ±  5%      +0.0        0.10 ±  4%  perf-profile.children.cycles-pp._vm_normal_page
      0.08 ±  5%      +0.0        0.10        perf-profile.children.cycles-pp.expand_downwards
      0.06 ±  7%      +0.0        0.08 ±  6%  perf-profile.children.cycles-pp._copy_to_user
      0.06            +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.__unlock_page_memcg
      0.06 ±  7%      +0.0        0.08 ±  6%  perf-profile.children.cycles-pp.security_file_alloc
      0.17 ±  2%      +0.0        0.19 ±  4%  perf-profile.children.cycles-pp.osq_unlock
      0.10 ±  5%      +0.0        0.11 ±  4%  perf-profile.children.cycles-pp.finish_fault
      0.09 ±  4%      +0.0        0.11 ±  3%  perf-profile.children.cycles-pp.may_open
      0.09 ±  4%      +0.0        0.11 ±  3%  perf-profile.children.cycles-pp.perf_output_copy
      0.10 ±  4%      +0.0        0.12 ±  3%  perf-profile.children.cycles-pp.selinux_mmap_file
      0.07 ±  5%      +0.0        0.09 ±  4%  perf-profile.children.cycles-pp.try_charge
      0.07 ±  5%      +0.0        0.09 ±  4%  perf-profile.children.cycles-pp.___slab_alloc
      0.08 ±  6%      +0.0        0.10 ±  5%  perf-profile.children.cycles-pp.__pthread_once_slow
      0.07            +0.0        0.09        perf-profile.children.cycles-pp.move_page_tables
      0.07            +0.0        0.09        perf-profile.children.cycles-pp.touch_atime
      0.06 ±  6%      +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.queue_work_on
      0.09 ±  7%      +0.0        0.11        perf-profile.children.cycles-pp.filp_close
      0.08 ±  8%      +0.0        0.10        perf-profile.children.cycles-pp.selinux_vm_enough_memory
      0.09 ±  4%      +0.0        0.11 ±  4%  perf-profile.children.cycles-pp.sync_regs
      0.08            +0.0        0.10        perf-profile.children.cycles-pp.__inode_security_revalidate
      0.07 ±  6%      +0.0        0.09 ±  4%  perf-profile.children.cycles-pp.prepend_name
      0.06 ±  6%      +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.complete_walk
      0.06            +0.0        0.08 ±  8%  perf-profile.children.cycles-pp.unlock_page_memcg
      0.06 ± 11%      +0.0        0.08        perf-profile.children.cycles-pp.cred_has_capability
      0.06            +0.0        0.08        perf-profile.children.cycles-pp.security_file_open
      0.06            +0.0        0.08        perf-profile.children.cycles-pp.__queue_work
      0.05 ±  8%      +0.0        0.07 ± 11%  perf-profile.children.cycles-pp.copyin
      0.06 ±  9%      +0.0        0.08 ±  6%  perf-profile.children.cycles-pp.entry_SYSCALL_64_stage2
      0.06 ±  9%      +0.0        0.08 ±  6%  perf-profile.children.cycles-pp.simple_write_begin
      0.06 ±  7%      +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.security_file_permission
      0.14 ±  5%      +0.0        0.16 ±  2%  perf-profile.children.cycles-pp.switch_mm_irqs_off
      0.13            +0.0        0.15 ±  2%  perf-profile.children.cycles-pp.__anon_vma_prepare
      0.10 ±  7%      +0.0        0.12 ±  3%  perf-profile.children.cycles-pp.migration_cpu_stop
      0.09 ±  7%      +0.0        0.11 ±  3%  perf-profile.children.cycles-pp._IO_file_close
      0.09            +0.0        0.11 ±  3%  perf-profile.children.cycles-pp.__call_rcu
      0.07            +0.0        0.09 ±  4%  perf-profile.children.cycles-pp.kmem_cache_alloc_trace
      0.11 ±  4%      +0.0        0.13        perf-profile.children.cycles-pp.do_brk_flags
      0.10 ±  8%      +0.0        0.12 ±  4%  perf-profile.children.cycles-pp.prepare_creds
      0.09            +0.0        0.11 ±  3%  perf-profile.children.cycles-pp.put_task_stack
      0.08 ±  5%      +0.0        0.11 ±  4%  perf-profile.children.cycles-pp.__slab_alloc
      0.09 ±  5%      +0.0        0.11 ±  4%  perf-profile.children.cycles-pp.__x64_sys_brk
      0.08 ±  5%      +0.0        0.10        perf-profile.children.cycles-pp.map_vdso
      0.11 ±  7%      +0.0        0.14 ±  3%  perf-profile.children.cycles-pp.get_zeroed_page
      0.08 ±  5%      +0.0        0.11 ±  4%  perf-profile.children.cycles-pp.anon_vma_interval_tree_remove
      0.12 ±  3%      +0.0        0.15 ±  2%  perf-profile.children.cycles-pp.__libc_sigaction
      0.11 ±  4%      +0.0        0.13 ±  5%  perf-profile.children.cycles-pp.cpu_stopper_thread
      0.10 ±  5%      +0.0        0.12 ±  5%  perf-profile.children.cycles-pp.prepare_binprm
      0.09            +0.0        0.11 ±  4%  perf-profile.children.cycles-pp.__d_alloc
      0.09 ±  4%      +0.0        0.11 ±  3%  perf-profile.children.cycles-pp.simple_lookup
      0.08 ±  5%      +0.0        0.11 ±  4%  perf-profile.children.cycles-pp.vm_area_alloc
      0.04 ± 57%      +0.0        0.06 ±  6%  perf-profile.children.cycles-pp.alloc_pid
      0.11 ±  4%      +0.0        0.13 ±  3%  perf-profile.children.cycles-pp.brk
      0.08 ±  5%      +0.0        0.11 ±  4%  perf-profile.children.cycles-pp.memset_erms
      0.11 ±  3%      +0.0        0.14 ±  6%  perf-profile.children.cycles-pp.__get_user_8
      0.09 ±  5%      +0.0        0.11 ±  3%  perf-profile.children.cycles-pp.evict
      0.15 ±  3%      +0.0        0.17 ±  2%  perf-profile.children.cycles-pp.avc_has_perm_noaudit
      0.13 ±  3%      +0.0        0.15 ±  3%  perf-profile.children.cycles-pp.sched_move_task
      0.11 ±  7%      +0.0        0.14 ±  3%  perf-profile.children.cycles-pp.__dentry_kill
      0.10 ±  4%      +0.0        0.13 ±  5%  perf-profile.children.cycles-pp.unmapped_area_topdown
      0.10 ±  4%      +0.0        0.12 ±  4%  perf-profile.children.cycles-pp.__snprintf_chk
      0.09 ±  4%      +0.0        0.11 ±  7%  perf-profile.children.cycles-pp.select_idle_sibling
      0.08 ±  5%      +0.0        0.11 ±  8%  perf-profile.children.cycles-pp.lockref_get_not_dead
      0.07            +0.0        0.10 ±  4%  perf-profile.children.cycles-pp.d_add
      0.04 ± 57%      +0.0        0.07 ±  7%  perf-profile.children.cycles-pp.do_huge_pmd_anonymous_page
      0.11            +0.0        0.14 ±  3%  perf-profile.children.cycles-pp.fput
      0.11 ±  6%      +0.0        0.14 ±  3%  perf-profile.children.cycles-pp._exit
      0.12 ±  6%      +0.0        0.15 ±  5%  perf-profile.children.cycles-pp.security_vm_enough_memory_mm
      0.12 ±  3%      +0.0        0.15 ±  2%  perf-profile.children.cycles-pp.free_pgd_range
      0.11 ±  4%      +0.0        0.14 ±  3%  perf-profile.children.cycles-pp.vm_area_dup
      0.10 ±  7%      +0.0        0.13        perf-profile.children.cycles-pp.vma_interval_tree_augment_rotate
      0.10 ±  5%      +0.0        0.12 ±  4%  perf-profile.children.cycles-pp.rcu_all_qs
      0.09 ±  4%      +0.0        0.12 ±  3%  perf-profile.children.cycles-pp.munmap
      0.08 ±  6%      +0.0        0.11 ±  4%  perf-profile.children.cycles-pp.__mmdrop
      0.07 ±  6%      +0.0        0.10 ±  4%  perf-profile.children.cycles-pp.truncate_inode_pages_range
      0.04 ± 57%      +0.0        0.07 ±  6%  perf-profile.children.cycles-pp.grab_cache_page_write_begin
      0.12 ±  3%      +0.0        0.15 ±  2%  perf-profile.children.cycles-pp.__lru_cache_add
      0.11 ±  4%      +0.0        0.15 ±  3%  perf-profile.children.cycles-pp.prepend_path
      0.14 ±  3%      +0.0        0.17 ±  6%  perf-profile.children.cycles-pp.security_mmap_file
      0.14 ±  3%      +0.0        0.17 ±  7%  perf-profile.children.cycles-pp.mem_cgroup_try_charge
      0.12 ±  3%      +0.0        0.15 ±  9%  perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
      0.13            +0.0        0.16 ±  2%  perf-profile.children.cycles-pp.lockref_put_or_lock
      0.14 ±  6%      +0.0        0.17 ±  4%  perf-profile.children.cycles-pp.kernel_read
      0.12 ±  3%      +0.0        0.15 ±  3%  perf-profile.children.cycles-pp.__put_user_4
      0.09            +0.0        0.12 ±  8%  perf-profile.children.cycles-pp.file_free_rcu
      0.11 ±  4%      +0.0        0.15 ±  5%  perf-profile.children.cycles-pp.__x64_sys_close
      0.20 ±  3%      +0.0        0.23 ±  2%  perf-profile.children.cycles-pp.update_load_avg
      0.16            +0.0        0.20 ±  5%  perf-profile.children.cycles-pp.vfs_statx_fd
      0.17 ±  3%      +0.0        0.20 ±  3%  perf-profile.children.cycles-pp.d_path
      0.15 ±  3%      +0.0        0.19 ±  3%  perf-profile.children.cycles-pp.stop_one_cpu
      0.16            +0.0        0.20 ±  2%  perf-profile.children.cycles-pp.anon_vma_clone
      0.16 ±  2%      +0.0        0.20 ±  2%  perf-profile.children.cycles-pp.vma_compute_subtree_gap
      0.16 ±  2%      +0.0        0.20 ±  2%  perf-profile.children.cycles-pp.getenv
      0.15 ±  2%      +0.0        0.18 ±  2%  perf-profile.children.cycles-pp._init
      0.14 ±  5%      +0.0        0.18 ±  2%  perf-profile.children.cycles-pp.__pud_alloc
      0.12 ±  4%      +0.0        0.16 ±  4%  perf-profile.children.cycles-pp.__getrlimit
      0.09            +0.0        0.12 ±  4%  perf-profile.children.cycles-pp.wake_q_add
      0.30 ±  2%      +0.0        0.34 ±  3%  perf-profile.children.cycles-pp.strchrnul
      0.18 ±  2%      +0.0        0.21 ±  3%  perf-profile.children.cycles-pp.open_exec
      0.14            +0.0        0.18 ±  2%  perf-profile.children.cycles-pp.shift_arg_pages
      0.14 ±  5%      +0.0        0.18 ±  2%  perf-profile.children.cycles-pp.preempt_schedule_common
      0.19 ±  4%      +0.0        0.23        perf-profile.children.cycles-pp.getname_flags
      0.17 ±  4%      +0.0        0.21 ±  2%  perf-profile.children.cycles-pp.arch_get_unmapped_area_topdown
      0.14 ±  3%      +0.0        0.18 ±  5%  perf-profile.children.cycles-pp.__might_fault
      0.15 ±  2%      +0.0        0.18 ±  2%  perf-profile.children.cycles-pp.perf_event_mmap_output
      0.13 ±  6%      +0.0        0.17 ±  5%  perf-profile.children.cycles-pp.dentry_kill
      0.13 ±  5%      +0.0        0.17 ±  4%  perf-profile.children.cycles-pp.copy_user_generic_unrolled
      0.13 ±  3%      +0.0        0.17 ±  3%  perf-profile.children.cycles-pp.do_unlinkat
      0.11            +0.0        0.15 ±  2%  perf-profile.children.cycles-pp.inode_permission
      0.14 ±  5%      +0.0        0.18        perf-profile.children.cycles-pp.free_pcppages_bulk
      0.08            +0.0        0.12 ± 11%  perf-profile.children.cycles-pp.rcu_cblist_dequeue
      0.23 ±  2%      +0.0        0.27 ±  4%  perf-profile.children.cycles-pp.ptep_clear_flush
      0.18 ±  4%      +0.0        0.22        perf-profile.children.cycles-pp.change_protection_range
      0.16 ±  2%      +0.0        0.20 ±  4%  perf-profile.children.cycles-pp.memcpy_erms
      0.15 ±  3%      +0.0        0.20 ±  4%  perf-profile.children.cycles-pp.pagecache_get_page
      0.14 ±  3%      +0.0        0.18 ±  4%  perf-profile.children.cycles-pp.legitimize_path
      0.39 ±  2%      +0.0        0.43 ±  3%  perf-profile.children.cycles-pp.malloc
      0.54            +0.0        0.58        perf-profile.children.cycles-pp.ttwu_do_activate
      0.25 ±  3%      +0.0        0.29        perf-profile.children.cycles-pp.dequeue_task_fair
      0.19 ±  2%      +0.0        0.23 ±  3%  perf-profile.children.cycles-pp._copy_from_user
      0.18 ±  2%      +0.0        0.22 ±  3%  perf-profile.children.cycles-pp.vmacache_find
      0.18 ±  2%      +0.0        0.22        perf-profile.children.cycles-pp.open64
      0.14 ±  3%      +0.0        0.18        perf-profile.children.cycles-pp.lock_page_memcg
      0.01 ±173%      +0.0        0.05 ±  9%  perf-profile.children.cycles-pp.__follow_mount_rcu
      0.21 ±  6%      +0.0        0.26 ±  3%  perf-profile.children.cycles-pp._IO_file_xsputn
      0.17 ±  3%      +0.0        0.21 ±  3%  perf-profile.children.cycles-pp.signal_wake_up_state
      0.14 ±  3%      +0.0        0.18 ±  3%  perf-profile.children.cycles-pp.count
      0.13 ±  3%      +0.0        0.18 ±  2%  perf-profile.children.cycles-pp.unlinkat
      0.11 ±  7%      +0.0        0.15 ±  9%  perf-profile.children.cycles-pp.__get_vm_area_node
      0.09 ±  4%      +0.0        0.14 ± 13%  perf-profile.children.cycles-pp.alloc_vmap_area
      0.20 ±  2%      +0.0        0.24 ±  3%  perf-profile.children.cycles-pp.__vma_link_rb
      0.17 ±  2%      +0.0        0.21 ±  3%  perf-profile.children.cycles-pp.schedule_tail
      0.14 ±  3%      +0.0        0.19 ±  4%  perf-profile.children.cycles-pp.__rb_erase_color
      0.01 ±173%      +0.0        0.06 ±  7%  perf-profile.children.cycles-pp.insert_vm_struct
      0.01 ±173%      +0.0        0.06 ±  7%  perf-profile.children.cycles-pp.put_prev_entity
      0.19 ±  2%      +0.0        0.24 ±  2%  perf-profile.children.cycles-pp.copy_strings_kernel
      0.19 ±  2%      +0.0        0.24 ±  5%  perf-profile.children.cycles-pp.setup_arg_pages
      0.17 ±  4%      +0.0        0.22        perf-profile.children.cycles-pp.__pmd_alloc
      0.15 ±  3%      +0.0        0.19 ±  5%  perf-profile.children.cycles-pp.unlazy_walk
      0.14 ±  5%      +0.0        0.19 ±  4%  perf-profile.children.cycles-pp.__get_free_pages
      0.01 ±173%      +0.0        0.06        perf-profile.children.cycles-pp.__free_pages_ok
      0.15            +0.0        0.20 ±  4%  perf-profile.children.cycles-pp.mark_page_accessed
      0.15 ±  2%      +0.0        0.20 ±  3%  perf-profile.children.cycles-pp.d_alloc
      0.57            +0.0        0.61        perf-profile.children.cycles-pp.enqueue_entity
      0.23            +0.0        0.28 ±  4%  perf-profile.children.cycles-pp.__do_sys_newfstat
      0.16 ±  5%      +0.0        0.21 ±  6%  perf-profile.children.cycles-pp.__generic_file_write_iter
      0.15 ±  7%      +0.0        0.20 ±  5%  perf-profile.children.cycles-pp.generic_perform_write
      0.00            +0.1        0.05        perf-profile.children.cycles-pp.vma_merge
      0.00            +0.1        0.05        perf-profile.children.cycles-pp.free_one_page
      0.00            +0.1        0.05        perf-profile.children.cycles-pp.rcu_segcblist_enqueue
      0.00            +0.1        0.05        perf-profile.children.cycles-pp.__fget_light
      0.17 ±  7%      +0.1        0.22 ±  5%  perf-profile.children.cycles-pp.generic_file_write_iter
      0.24 ±  2%      +0.1        0.30 ±  3%  perf-profile.children.cycles-pp.anon_vma_fork
      0.19 ±  3%      +0.1        0.24 ±  4%  perf-profile.children.cycles-pp.mem_cgroup_try_charge_delay
      0.19            +0.1        0.24 ±  3%  perf-profile.children.cycles-pp.finish_task_switch
      0.18 ±  2%      +0.1        0.24 ±  3%  perf-profile.children.cycles-pp.__send_signal
      0.18 ±  2%      +0.1        0.23 ±  2%  perf-profile.children.cycles-pp.__pagevec_lru_add_fn
      0.00            +0.1        0.05 ±  8%  perf-profile.children.cycles-pp.__pagevec_release
      0.00            +0.1        0.05 ±  8%  perf-profile.children.cycles-pp.__perf_event_header__init_id
      0.00            +0.1        0.05 ±  8%  perf-profile.children.cycles-pp.check_preempt_curr
      0.21 ±  2%      +0.1        0.27 ±  5%  perf-profile.children.cycles-pp.autoremove_wake_function
      0.21 ±  3%      +0.1        0.27 ±  3%  perf-profile.children.cycles-pp.generic_file_read_iter
      0.30 ±  2%      +0.1        0.36 ±  3%  perf-profile.children.cycles-pp.wake_up_new_task
      0.25            +0.1        0.30        perf-profile.children.cycles-pp.get_unmapped_area
      0.19 ±  3%      +0.1        0.24 ±  4%  perf-profile.children.cycles-pp.get_user_arg_ptr
      0.17 ±  2%      +0.1        0.22 ±  3%  perf-profile.children.cycles-pp.pgd_alloc
      0.00            +0.1        0.05 ±  9%  perf-profile.children.cycles-pp.page_add_new_anon_rmap
      0.21 ±  2%      +0.1        0.26 ±  4%  perf-profile.children.cycles-pp.__might_sleep
      0.31            +0.1        0.36        perf-profile.children.cycles-pp.native_flush_tlb_one_user
      0.26 ±  3%      +0.1        0.31 ±  2%  perf-profile.children.cycles-pp.do_open_execat
      0.20 ±  4%      +0.1        0.25 ±  7%  perf-profile.children.cycles-pp.__vmalloc_node_range
      0.00            +0.1        0.06 ±  7%  perf-profile.children.cycles-pp.__mod_node_page_state
      0.00            +0.1        0.06 ±  7%  perf-profile.children.cycles-pp.anon_vma_interval_tree_insert
      0.22            +0.1        0.28 ±  4%  perf-profile.children.cycles-pp.__wake_up_common_lock
      0.22 ±  3%      +0.1        0.28 ±  4%  perf-profile.children.cycles-pp.do_notify_parent
      0.28 ±  2%      +0.1        0.34 ±  5%  perf-profile.children.cycles-pp.__rb_insert_augmented
      0.23 ±  2%      +0.1        0.29        perf-profile.children.cycles-pp.strlen
      0.19 ±  2%      +0.1        0.25 ±  5%  perf-profile.children.cycles-pp.copyout
      0.19 ±  4%      +0.1        0.25 ±  3%  perf-profile.children.cycles-pp.mm_init
      0.64            +0.1        0.70        perf-profile.children.cycles-pp.enqueue_task_fair
      0.29 ±  2%      +0.1        0.35 ±  3%  perf-profile.children.cycles-pp.___perf_sw_event
      0.24 ±  2%      +0.1        0.30 ±  3%  perf-profile.children.cycles-pp.terminate_walk
      0.26 ±  3%      +0.1        0.33        perf-profile.children.cycles-pp.free_unref_page_list
      0.23            +0.1        0.30 ±  5%  perf-profile.children.cycles-pp.__wake_up_common
      0.32            +0.1        0.38        perf-profile.children.cycles-pp.__perf_sw_event
      0.28 ±  2%      +0.1        0.35 ±  4%  perf-profile.children.cycles-pp.get_user_pages_remote
      0.25 ±  2%      +0.1        0.32 ±  4%  perf-profile.children.cycles-pp.pipe_write
      0.22 ±  3%      +0.1        0.29 ±  5%  perf-profile.children.cycles-pp.copy_page_to_iter
      0.23 ±  2%      +0.1        0.29 ±  2%  perf-profile.children.cycles-pp._IO_file_open
      0.32 ±  3%      +0.1        0.38 ±  2%  perf-profile.children.cycles-pp._fini
      0.28            +0.1        0.34 ±  4%  perf-profile.children.cycles-pp.__get_user_pages
      0.34 ±  2%      +0.1        0.41        perf-profile.children.cycles-pp.unlock_page
      0.30 ±  2%      +0.1        0.36 ±  4%  perf-profile.children.cycles-pp.__pte_alloc
      0.32            +0.1        0.39        perf-profile.children.cycles-pp.vma_interval_tree_remove
      0.29 ±  2%      +0.1        0.36 ±  3%  perf-profile.children.cycles-pp.pipe_read
      0.23            +0.1        0.30 ±  2%  perf-profile.children.cycles-pp.do_dentry_open
      0.10 ±  5%      +0.1        0.16 ± 10%  perf-profile.children.cycles-pp.run_ksoftirqd
      0.22 ±  3%      +0.1        0.29 ±  3%  perf-profile.children.cycles-pp.d_alloc_parallel
      0.39            +0.1        0.46 ±  3%  perf-profile.children.cycles-pp.__fxstat64
      0.34            +0.1        0.41        perf-profile.children.cycles-pp.flush_tlb_func_common
      0.27 ±  3%      +0.1        0.34 ±  3%  perf-profile.children.cycles-pp.__d_lookup_rcu
      0.34 ±  2%      +0.1        0.41 ±  2%  perf-profile.children.cycles-pp._IO_padn
      0.16 ±  2%      +0.1        0.23 ±  3%  perf-profile.children.cycles-pp.release_task
      0.24            +0.1        0.31 ±  3%  perf-profile.children.cycles-pp.__lookup_slow
      0.00            +0.1        0.07 ± 11%  perf-profile.children.cycles-pp.do_prlimit
      0.32            +0.1        0.40 ±  2%  perf-profile.children.cycles-pp.selinux_inode_permission
      0.39 ±  2%      +0.1        0.46 ±  2%  perf-profile.children.cycles-pp.find_vma
      0.00            +0.1        0.08 ±  6%  perf-profile.children.cycles-pp.__x64_sys_getrlimit
      0.59            +0.1        0.67 ±  2%  perf-profile.children.cycles-pp.schedule
      0.28            +0.1        0.36 ±  2%  perf-profile.children.cycles-pp.__list_del_entry_valid
      0.35 ±  2%      +0.1        0.42 ±  2%  perf-profile.children.cycles-pp.security_inode_permission
      0.32 ±  3%      +0.1        0.40 ±  3%  perf-profile.children.cycles-pp.__do_sys_newstat
      0.27 ±  3%      +0.1        0.35 ±  2%  perf-profile.children.cycles-pp.lookup_slow
      0.33 ±  2%      +0.1        0.41 ±  2%  perf-profile.children.cycles-pp._cond_resched
      0.32            +0.1        0.40 ±  3%  perf-profile.children.cycles-pp.remove_vma
      0.30 ±  2%      +0.1        0.39 ±  2%  perf-profile.children.cycles-pp.perf_iterate_sb
      0.13 ±  5%      +0.1        0.21 ± 14%  perf-profile.children.cycles-pp.find_vmap_area
      0.33 ±  3%      +0.1        0.41 ±  3%  perf-profile.children.cycles-pp.vfs_statx
      0.39 ±  2%      +0.1        0.47 ±  2%  perf-profile.children.cycles-pp._IO_fwrite
      0.29 ±  2%      +0.1        0.38 ±  3%  perf-profile.children.cycles-pp.exit_to_usermode_loop
      0.18 ±  8%      +0.1        0.27 ± 12%  perf-profile.children.cycles-pp.__vunmap
      0.43            +0.1        0.52 ±  2%  perf-profile.children.cycles-pp.flush_tlb_mm_range
      0.37 ±  3%      +0.1        0.46 ±  4%  perf-profile.children.cycles-pp.___might_sleep
      0.40 ±  3%      +0.1        0.49 ±  2%  perf-profile.children.cycles-pp.__slab_free
      0.29            +0.1        0.38 ±  2%  perf-profile.children.cycles-pp.create_elf_tables
      0.18 ±  7%      +0.1        0.27 ± 10%  perf-profile.children.cycles-pp.free_work
      0.37 ±  2%      +0.1        0.46        perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
      0.18 ±  4%      +0.1        0.28 ±  4%  perf-profile.children.cycles-pp.wait_consider_task
      0.39 ±  2%      +0.1        0.48 ±  3%  perf-profile.children.cycles-pp.__xstat64
      0.39            +0.1        0.48 ±  2%  perf-profile.children.cycles-pp.pte_alloc_one
      0.29 ±  2%      +0.1        0.39 ±  2%  perf-profile.children.cycles-pp.__alloc_file
      0.40 ±  2%      +0.1        0.49 ±  2%  perf-profile.children.cycles-pp.unlink_anon_vmas
      0.32 ±  2%      +0.1        0.41 ±  3%  perf-profile.children.cycles-pp.alloc_empty_file
      0.44            +0.1        0.54        perf-profile.children.cycles-pp.page_add_file_rmap
      0.41 ±  2%      +0.1        0.51 ±  2%  perf-profile.children.cycles-pp.sched_exec
      0.39            +0.1        0.48        perf-profile.children.cycles-pp.copy_page
      0.31 ±  3%      +0.1        0.41 ±  3%  perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      0.52            +0.1        0.62 ±  3%  perf-profile.children.cycles-pp.native_irq_return_iret
      0.20 ±  6%      +0.1        0.30 ±  9%  perf-profile.children.cycles-pp.process_one_work
      0.36 ±  2%      +0.1        0.47 ±  3%  perf-profile.children.cycles-pp.__fput
      0.51            +0.1        0.62 ±  2%  perf-profile.children.cycles-pp.copy_page_range
      0.40 ±  2%      +0.1        0.50 ±  3%  perf-profile.children.cycles-pp.__clear_user
      0.36 ±  2%      +0.1        0.47        perf-profile.children.cycles-pp.__x64_sys_munmap
      0.35            +0.1        0.46        perf-profile.children.cycles-pp.strnlen_user
      0.23 ±  5%      +0.1        0.34 ±  9%  perf-profile.children.cycles-pp.worker_thread
      0.48 ±  2%      +0.1        0.59 ±  2%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      0.40 ±  2%      +0.1        0.51 ±  3%  perf-profile.children.cycles-pp.dput
      0.44            +0.1        0.55 ±  4%  perf-profile.children.cycles-pp.lookup_fast
      1.14            +0.1        1.25 ±  4%  perf-profile.children.cycles-pp.__softirqentry_text_start
      0.42 ±  2%      +0.1        0.54 ±  2%  perf-profile.children.cycles-pp.free_pages_and_swap_cache
      1.08 ±  2%      +0.1        1.20 ±  2%  perf-profile.children.cycles-pp.__sched_text_start
      0.51            +0.1        0.63 ±  2%  perf-profile.children.cycles-pp.ksys_read
      0.40            +0.1        0.52 ±  4%  perf-profile.children.cycles-pp.smpboot_thread_fn
      0.46            +0.1        0.58 ±  3%  perf-profile.children.cycles-pp.write
      0.49            +0.1        0.62 ±  3%  perf-profile.children.cycles-pp.read
      0.42 ±  2%      +0.1        0.55 ±  3%  perf-profile.children.cycles-pp.__vfs_write
      0.44            +0.1        0.57 ±  2%  perf-profile.children.cycles-pp.task_work_run
      0.99 ±  2%      +0.1        1.12 ±  4%  perf-profile.children.cycles-pp.rcu_process_callbacks
      0.55 ±  2%      +0.1        0.67 ±  2%  perf-profile.children.cycles-pp.kmem_cache_free
      0.54            +0.1        0.66 ±  3%  perf-profile.children.cycles-pp.__vfs_read
      0.62            +0.1        0.75 ±  3%  perf-profile.children.cycles-pp.select_task_rq_fair
      0.46            +0.1        0.59 ±  4%  perf-profile.children.cycles-pp.vfs_write
      0.45 ±  2%      +0.1        0.58 ±  2%  perf-profile.children.cycles-pp.do_faccessat
      0.47            +0.1        0.60 ±  3%  perf-profile.children.cycles-pp.ksys_write
      0.15 ±  6%      +0.1        0.28 ±  7%  perf-profile.children.cycles-pp.queued_write_lock_slowpath
      0.14 ±  8%      +0.1        0.27 ±  9%  perf-profile.children.cycles-pp.queued_read_lock_slowpath
      0.51 ±  3%      +0.1        0.65        perf-profile.children.cycles-pp.kmem_cache_alloc
      0.58            +0.1        0.72 ±  2%  perf-profile.children.cycles-pp.__entry_SYSCALL_64_trampoline
      0.62            +0.1        0.77        perf-profile.children.cycles-pp.vfs_read
      0.63            +0.2        0.78 ±  2%  perf-profile.children.cycles-pp.perf_event_mmap
      0.62            +0.2        0.77 ±  2%  perf-profile.children.cycles-pp.alloc_pages_vma
      0.35 ±  4%      +0.2        0.51 ±  4%  perf-profile.children.cycles-pp.lru_add_drain
      0.35 ±  3%      +0.2        0.52 ±  5%  perf-profile.children.cycles-pp.lru_add_drain_cpu
      0.60 ±  2%      +0.2        0.78 ±  2%  perf-profile.children.cycles-pp.path_lookupat
      0.62 ±  2%      +0.2        0.80 ±  2%  perf-profile.children.cycles-pp.filename_lookup
      0.43 ±  2%      +0.2        0.61 ±  3%  perf-profile.children.cycles-pp.pagevec_lru_move_fn
      0.64            +0.2        0.83 ±  3%  perf-profile.children.cycles-pp.walk_component
      0.75 ±  2%      +0.2        0.95 ±  2%  perf-profile.children.cycles-pp._IO_default_xsputn
      0.86            +0.2        1.07 ±  3%  perf-profile.children.cycles-pp.clear_page_erms
      0.84            +0.2        1.04 ±  3%  perf-profile.children.cycles-pp.copy_strings
      0.41 ±  4%      +0.2        0.62 ±  5%  perf-profile.children.cycles-pp.do_wait
      0.43 ±  5%      +0.2        0.64 ±  5%  perf-profile.children.cycles-pp.__do_sys_wait4
      0.42 ±  4%      +0.2        0.64 ±  5%  perf-profile.children.cycles-pp.kernel_wait4
      0.95 ±  2%      +0.2        1.16        perf-profile.children.cycles-pp._dl_addr
      0.95            +0.2        1.17 ±  2%  perf-profile.children.cycles-pp.wp_page_copy
      0.80            +0.2        1.02 ±  2%  perf-profile.children.cycles-pp.link_path_walk
      0.93            +0.2        1.16        perf-profile.children.cycles-pp.vma_interval_tree_insert
      0.95            +0.2        1.18        perf-profile.children.cycles-pp.alloc_set_pte
      0.51 ±  3%      +0.2        0.74 ±  3%  perf-profile.children.cycles-pp.wait4
      0.63 ±  2%      +0.2        0.87 ±  4%  perf-profile.children.cycles-pp.kthread
      1.04            +0.2        1.27 ±  2%  perf-profile.children.cycles-pp.do_wp_page
      1.29            +0.2        1.54 ±  3%  perf-profile.children.cycles-pp.wake_up_q
      2.06            +0.3        2.33 ±  2%  perf-profile.children.cycles-pp.up_write
      9.02            +0.3        9.29        perf-profile.children.cycles-pp.handle_mm_fault
      1.53            +0.3        1.79 ±  2%  perf-profile.children.cycles-pp.rwsem_wake
      0.81            +0.3        1.08 ±  3%  perf-profile.children.cycles-pp.ret_from_fork
      1.53            +0.3        1.81 ±  2%  perf-profile.children.cycles-pp.call_rwsem_wake
      1.10            +0.3        1.38 ±  2%  perf-profile.children.cycles-pp.__strcoll_l
      0.81 ±  2%      +0.3        1.08 ±  4%  perf-profile.children.cycles-pp._raw_spin_lock
      0.93            +0.3        1.23        perf-profile.children.cycles-pp.rwsem_spin_on_owner
      1.24            +0.3        1.54 ±  3%  perf-profile.children.cycles-pp.get_page_from_freelist
      1.35            +0.3        1.69 ±  3%  perf-profile.children.cycles-pp.__alloc_pages_nodemask
      1.68            +0.4        2.04 ±  3%  perf-profile.children.cycles-pp.try_to_wake_up
      1.22            +0.4        1.58        perf-profile.children.cycles-pp.mmap64
     14.16            +0.4       14.53        perf-profile.children.cycles-pp.load_elf_binary
      1.05 ±  2%      +0.4        1.42 ±  4%  perf-profile.children.cycles-pp.page_remove_rmap
     14.19            +0.4       14.56        perf-profile.children.cycles-pp.search_binary_handler
      2.09 ±  2%      +0.4        2.52        perf-profile.children.cycles-pp.vfprintf
      9.80            +0.5       10.26        perf-profile.children.cycles-pp.__do_page_fault
      9.82            +0.5       10.29        perf-profile.children.cycles-pp.do_page_fault
      1.27 ±  2%      +0.5        1.79 ±  5%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      1.96            +0.5        2.49 ±  2%  perf-profile.children.cycles-pp.path_openat
      2.01            +0.5        2.54        perf-profile.children.cycles-pp.do_sys_open
      1.98            +0.5        2.52 ±  2%  perf-profile.children.cycles-pp.do_filp_open
      0.86 ±  2%      +0.5        1.40 ±  2%  perf-profile.children.cycles-pp.vm_munmap
     10.22            +0.6       10.78        perf-profile.children.cycles-pp.page_fault
      2.73            +0.6        3.29        perf-profile.children.cycles-pp.__vsnprintf_chk
      2.76            +0.6        3.39 ±  2%  perf-profile.children.cycles-pp.setlocale
      1.77 ±  2%      +0.7        2.50 ±  4%  perf-profile.children.cycles-pp.release_pages
      1.47            +0.8        2.27 ±  6%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      2.12 ±  2%      +0.8        2.93 ±  4%  perf-profile.children.cycles-pp.tlb_flush_mmu_free
     15.71            +0.8       16.53        perf-profile.children.cycles-pp.execve
     16.01            +0.8       16.84        perf-profile.children.cycles-pp.__do_execve_file
     16.02            +0.8       16.86        perf-profile.children.cycles-pp.__x64_sys_execve
      2.24 ±  2%      +0.8        3.09 ±  4%  perf-profile.children.cycles-pp.arch_tlb_finish_mmu
      2.25 ±  2%      +0.8        3.10 ±  4%  perf-profile.children.cycles-pp.tlb_finish_mmu
      2.77            +0.9        3.62 ±  3%  perf-profile.children.cycles-pp.unmap_page_range
      2.89            +0.9        3.77 ±  3%  perf-profile.children.cycles-pp.unmap_vmas
      1.18 ±  3%      +0.9        2.09 ±  2%  perf-profile.children.cycles-pp.elf_map
     41.69            -8.5       33.23 ±  3%  perf-profile.self.cycles-pp.osq_lock
      9.44            -1.3        8.12 ±  4%  perf-profile.self.cycles-pp.intel_idle
      4.14 ±  2%      -0.6        3.53        perf-profile.self.cycles-pp.filemap_map_pages
      0.54 ±  2%      -0.2        0.37        perf-profile.self.cycles-pp.rwsem_down_write_failed
      0.28 ±  3%      -0.1        0.15 ±  2%  perf-profile.self.cycles-pp.radix_tree_next_chunk
      0.13 ±  6%      -0.0        0.11 ±  4%  perf-profile.self.cycles-pp.find_busiest_group
      0.05            +0.0        0.06        perf-profile.self.cycles-pp.anon_vma_fork
      0.05            +0.0        0.06        perf-profile.self.cycles-pp.__call_rcu
      0.05            +0.0        0.06        perf-profile.self.cycles-pp.__unlock_page_memcg
      0.06            +0.0        0.07        perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.06            +0.0        0.07        perf-profile.self.cycles-pp.do_wp_page
      0.06            +0.0        0.07        perf-profile.self.cycles-pp.kfree
      0.06            +0.0        0.07        perf-profile.self.cycles-pp.__switch_to
      0.07 ±  5%      +0.0        0.08 ±  5%  perf-profile.self.cycles-pp.find_next_bit
      0.07            +0.0        0.08 ±  5%  perf-profile.self.cycles-pp.__tsearch
      0.05 ±  8%      +0.0        0.07 ±  7%  perf-profile.self.cycles-pp.path_openat
      0.05            +0.0        0.06 ±  6%  perf-profile.self.cycles-pp.d_alloc_parallel
      0.05            +0.0        0.06 ±  6%  perf-profile.self.cycles-pp.unlock_page_memcg
      0.05            +0.0        0.06 ±  6%  perf-profile.self.cycles-pp.up_read
      0.06            +0.0        0.07 ±  5%  perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
      0.06 ±  6%      +0.0        0.08 ±  6%  perf-profile.self.cycles-pp.__perf_sw_event
      0.06 ±  7%      +0.0        0.07        perf-profile.self.cycles-pp.strncpy_from_user
      0.09 ±  4%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.update_load_avg
      0.07            +0.0        0.09 ±  5%  perf-profile.self.cycles-pp.find_get_entry
      0.05            +0.0        0.07 ±  7%  perf-profile.self.cycles-pp.__d_alloc
      0.05 ±  9%      +0.0        0.07        perf-profile.self.cycles-pp.create_elf_tables
      0.17 ±  3%      +0.0        0.18 ±  2%  perf-profile.self.cycles-pp.osq_unlock
      0.07 ± 11%      +0.0        0.09        perf-profile.self.cycles-pp.__alloc_pages_nodemask
      0.07 ±  5%      +0.0        0.09        perf-profile.self.cycles-pp.getenv
      0.07 ± 11%      +0.0        0.09        perf-profile.self.cycles-pp._vm_normal_page
      0.11 ±  7%      +0.0        0.12 ±  4%  perf-profile.self.cycles-pp.__sched_text_start
      0.09 ±  4%      +0.0        0.11 ±  6%  perf-profile.self.cycles-pp._cond_resched
      0.09 ±  4%      +0.0        0.11        perf-profile.self.cycles-pp.__list_add_valid
      0.08 ±  5%      +0.0        0.10        perf-profile.self.cycles-pp.perf_iterate_sb
      0.07 ±  7%      +0.0        0.08 ±  5%  perf-profile.self.cycles-pp.__vma_adjust
      0.08 ±  8%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.sync_regs
      0.07 ±  7%      +0.0        0.08 ±  5%  perf-profile.self.cycles-pp.load_new_mm_cr3
      0.06 ±  6%      +0.0        0.08        perf-profile.self.cycles-pp._IO_setb
      0.05            +0.0        0.07 ±  6%  perf-profile.self.cycles-pp.__legitimize_mnt
      0.08 ±  5%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.__libc_fork
      0.08 ±  5%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.change_protection_range
      0.06            +0.0        0.08 ±  5%  perf-profile.self.cycles-pp.memchr
      0.08 ±  6%      +0.0        0.10 ±  5%  perf-profile.self.cycles-pp.anon_vma_interval_tree_remove
      0.07 ± 12%      +0.0        0.09 ±  9%  perf-profile.self.cycles-pp.try_charge
      0.08 ±  5%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.inode_permission
      0.07            +0.0        0.09        perf-profile.self.cycles-pp.avc_has_perm
      0.08            +0.0        0.10 ±  7%  perf-profile.self.cycles-pp.free_pgd_range
      0.07 ±  7%      +0.0        0.09 ±  5%  perf-profile.self.cycles-pp.prepend_name
      0.05 ±  8%      +0.0        0.07 ±  5%  perf-profile.self.cycles-pp.entry_SYSCALL_64_stage2
      0.08 ± 10%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.available_idle_cpu
      0.10 ±  8%      +0.0        0.12 ±  4%  perf-profile.self.cycles-pp.unlink_anon_vmas
      0.10 ±  7%      +0.0        0.12 ±  6%  perf-profile.self.cycles-pp.copy_user_generic_unrolled
      0.08 ±  8%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.memset_erms
      0.07 ±  5%      +0.0        0.10 ±  5%  perf-profile.self.cycles-pp.rcu_all_qs
      0.08 ±  6%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.lockref_get_not_dead
      0.06 ±  9%      +0.0        0.08 ±  5%  perf-profile.self.cycles-pp.memcpy
      0.08 ±  5%      +0.0        0.10        perf-profile.self.cycles-pp.fput
      0.12 ±  6%      +0.0        0.15 ±  4%  perf-profile.self.cycles-pp.do_syscall_64
      0.07 ±  5%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.__snprintf_chk
      0.04 ± 57%      +0.0        0.06 ±  6%  perf-profile.self.cycles-pp.vm_area_dup
      0.11 ±  4%      +0.0        0.13 ±  3%  perf-profile.self.cycles-pp.__get_user_8
      0.08 ±  5%      +0.0        0.11 ±  7%  perf-profile.self.cycles-pp.file_free_rcu
      0.08            +0.0        0.11 ±  4%  perf-profile.self.cycles-pp.__vma_link_rb
      0.13 ±  5%      +0.0        0.16 ±  5%  perf-profile.self.cycles-pp.copy_page_range
      0.11 ±  3%      +0.0        0.14 ±  3%  perf-profile.self.cycles-pp.perf_event_mmap
      0.18 ±  2%      +0.0        0.21 ±  2%  perf-profile.self.cycles-pp.find_vma
      0.14            +0.0        0.17 ±  2%  perf-profile.self.cycles-pp.copy_process
      0.10 ±  4%      +0.0        0.13        perf-profile.self.cycles-pp.link_path_walk
      0.08            +0.0        0.11 ±  4%  perf-profile.self.cycles-pp.__fput
      0.08 ±  8%      +0.0        0.11 ±  4%  perf-profile.self.cycles-pp.free_pcppages_bulk
      0.04 ± 57%      +0.0        0.07 ±  7%  perf-profile.self.cycles-pp.do_mmap
      0.04 ± 57%      +0.0        0.07 ±  7%  perf-profile.self.cycles-pp.kmem_cache_alloc_trace
      0.10 ±  4%      +0.0        0.13 ±  3%  perf-profile.self.cycles-pp.vma_interval_tree_augment_rotate
      0.15 ±  2%      +0.0        0.18 ±  2%  perf-profile.self.cycles-pp.vma_compute_subtree_gap
      0.14 ±  3%      +0.0        0.17 ±  4%  perf-profile.self.cycles-pp.avc_has_perm_noaudit
      0.12            +0.0        0.15 ±  2%  perf-profile.self.cycles-pp.lock_page_memcg
      0.11 ±  4%      +0.0        0.15 ±  2%  perf-profile.self.cycles-pp._init
      0.10 ±  5%      +0.0        0.13 ±  3%  perf-profile.self.cycles-pp.unmapped_area_topdown
      0.11            +0.0        0.14 ±  3%  perf-profile.self.cycles-pp.lockref_put_or_lock
      0.03 ±100%      +0.0        0.06        perf-profile.self.cycles-pp.unmap_vmas
      0.18            +0.0        0.21 ±  4%  perf-profile.self.cycles-pp.handle_mm_fault
      0.09            +0.0        0.12 ±  4%  perf-profile.self.cycles-pp.wake_q_add
      0.15 ±  5%      +0.0        0.18 ±  2%  perf-profile.self.cycles-pp.kmem_cache_free
      0.08 ±  5%      +0.0        0.12 ±  5%  perf-profile.self.cycles-pp.queued_write_lock_slowpath
      0.18 ±  3%      +0.0        0.22 ±  3%  perf-profile.self.cycles-pp._IO_file_xsputn
      0.17 ±  2%      +0.0        0.21 ±  3%  perf-profile.self.cycles-pp.strchrnul
      0.16 ±  5%      +0.0        0.20 ±  4%  perf-profile.self.cycles-pp.memcpy_erms
      0.12 ±  3%      +0.0        0.16 ±  4%  perf-profile.self.cycles-pp.__rb_erase_color
      0.12 ±  7%      +0.0        0.15 ±  3%  perf-profile.self.cycles-pp.__pagevec_lru_add_fn
      0.17 ±  2%      +0.0        0.21 ±  2%  perf-profile.self.cycles-pp.vmacache_find
      0.08            +0.0        0.12 ± 11%  perf-profile.self.cycles-pp.rcu_cblist_dequeue
      0.16            +0.0        0.20        perf-profile.self.cycles-pp.mmap_region
      0.14            +0.0        0.18 ±  2%  perf-profile.self.cycles-pp.mark_page_accessed
      0.07 ±  6%      +0.0        0.11 ± 11%  perf-profile.self.cycles-pp.queued_read_lock_slowpath
      0.21 ±  2%      +0.0        0.26 ±  4%  perf-profile.self.cycles-pp.get_page_from_freelist
      0.17 ±  2%      +0.0        0.22        perf-profile.self.cycles-pp.malloc
      0.16 ±  2%      +0.0        0.21 ±  2%  perf-profile.self.cycles-pp.strlen
      0.01 ±173%      +0.0        0.06 ±  7%  perf-profile.self.cycles-pp.down_read_trylock
      0.01 ±173%      +0.0        0.06 ±  7%  perf-profile.self.cycles-pp.__errno_location
      0.17 ±  2%      +0.0        0.22 ±  3%  perf-profile.self.cycles-pp.selinux_inode_permission
      0.26            +0.0        0.30 ±  5%  perf-profile.self.cycles-pp.__rb_insert_augmented
      0.18 ±  2%      +0.0        0.23 ±  5%  perf-profile.self.cycles-pp.__might_sleep
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.load_elf_binary
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.page_fault
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.pagevec_lru_move_fn
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.flush_tlb_mm_range
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.__inode_security_revalidate
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.dup_fd
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.acct_collect
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.vma_merge
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.rcu_segcblist_enqueue
      0.24 ±  2%      +0.1        0.30 ±  2%  perf-profile.self.cycles-pp.___perf_sw_event
      0.00            +0.1        0.05 ±  8%  perf-profile.self.cycles-pp.mem_cgroup_throttle_swaprate
      0.00            +0.1        0.05 ±  8%  perf-profile.self.cycles-pp.anon_vma_interval_tree_insert
      0.44 ±  2%      +0.1        0.49 ±  4%  perf-profile.self.cycles-pp.down_write
      0.00            +0.1        0.05 ±  9%  perf-profile.self.cycles-pp.__might_fault
      0.00            +0.1        0.05 ±  9%  perf-profile.self.cycles-pp.__mod_node_page_state
      0.22            +0.1        0.28 ±  3%  perf-profile.self.cycles-pp.__vsnprintf_chk
      0.15 ±  2%      +0.1        0.21 ±  5%  perf-profile.self.cycles-pp.__alloc_file
      0.30            +0.1        0.36        perf-profile.self.cycles-pp.native_flush_tlb_one_user
      0.18 ±  4%      +0.1        0.24 ±  3%  perf-profile.self.cycles-pp.__do_page_fault
      0.31 ±  2%      +0.1        0.37        perf-profile.self.cycles-pp._IO_padn
      0.33 ±  3%      +0.1        0.40        perf-profile.self.cycles-pp.unlock_page
      0.21 ±  3%      +0.1        0.28        perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.20 ±  2%      +0.1        0.27 ±  4%  perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      0.36            +0.1        0.43 ±  2%  perf-profile.self.cycles-pp.page_add_file_rmap
      0.31            +0.1        0.38        perf-profile.self.cycles-pp.vma_interval_tree_remove
      0.27 ±  3%      +0.1        0.34 ±  3%  perf-profile.self.cycles-pp.__d_lookup_rcu
      0.28            +0.1        0.35 ±  3%  perf-profile.self.cycles-pp.__list_del_entry_valid
      0.41            +0.1        0.49 ±  3%  perf-profile.self.cycles-pp.select_task_rq_fair
      0.34            +0.1        0.42        perf-profile.self.cycles-pp.swapgs_restore_regs_and_return_to_usermode
      0.35 ±  3%      +0.1        0.43 ±  3%  perf-profile.self.cycles-pp.___might_sleep
      0.29 ±  3%      +0.1        0.38        perf-profile.self.cycles-pp.kmem_cache_alloc
      0.36 ±  3%      +0.1        0.45        perf-profile.self.cycles-pp._IO_fwrite
      0.44            +0.1        0.53        perf-profile.self.cycles-pp.__handle_mm_fault
      0.39 ±  3%      +0.1        0.48 ±  2%  perf-profile.self.cycles-pp.__slab_free
      0.32 ±  2%      +0.1        0.41        perf-profile.self.cycles-pp.alloc_set_pte
      0.38            +0.1        0.47        perf-profile.self.cycles-pp.copy_page
      0.52            +0.1        0.62 ±  3%  perf-profile.self.cycles-pp.native_irq_return_iret
      0.34 ±  2%      +0.1        0.45 ±  2%  perf-profile.self.cycles-pp.strnlen_user
      0.41 ±  2%      +0.1        0.52 ±  2%  perf-profile.self.cycles-pp.free_pages_and_swap_cache
      0.47 ±  2%      +0.1        0.59 ±  2%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      0.51            +0.1        0.64 ±  4%  perf-profile.self.cycles-pp._raw_spin_lock
      0.53            +0.1        0.66        perf-profile.self.cycles-pp.__entry_SYSCALL_64_trampoline
      0.65 ±  2%      +0.2        0.82        perf-profile.self.cycles-pp._IO_default_xsputn
      0.84            +0.2        1.05 ±  3%  perf-profile.self.cycles-pp.clear_page_erms
      0.91 ±  2%      +0.2        1.13        perf-profile.self.cycles-pp._dl_addr
      0.91            +0.2        1.14        perf-profile.self.cycles-pp.vma_interval_tree_insert
      1.08            +0.3        1.35 ±  2%  perf-profile.self.cycles-pp.__strcoll_l
      0.91            +0.3        1.20        perf-profile.self.cycles-pp.rwsem_spin_on_owner
      0.92 ±  2%      +0.3        1.26 ±  4%  perf-profile.self.cycles-pp.page_remove_rmap
      1.33            +0.4        1.72 ±  2%  perf-profile.self.cycles-pp.unmap_page_range
      1.75 ±  2%      +0.4        2.18        perf-profile.self.cycles-pp.vfprintf
      1.23 ±  2%      +0.5        1.73 ±  5%  perf-profile.self.cycles-pp.release_pages
      1.47            +0.8        2.26 ±  6%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Rong Chen

View attachment "config-4.19.0-rc5-00261-geb797a8" of type "text/plain" (167777 bytes)

View attachment "job.yaml" of type "text/plain" (4949 bytes)

View attachment "reproduce" of type "text/plain" (794 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ