lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Date:   Mon, 5 Jul 2021 17:58:33 +0800
From:   kernel test robot <rong.a.chen@...el.com>
To:     Waiman Long <longman@...hat.com>
Cc:     Linus Torvalds <torvalds@...ux-foundation.org>,
        Shakeel Butt <shakeelb@...gle.com>,
        Alex Shi <alex.shi@...ux.alibaba.com>,
        Chris Down <chris@...isdown.name>,
        Christoph Lameter <cl@...ux.com>,
        David Rientjes <rientjes@...gle.com>,
        Johannes Weiner <hannes@...xchg.org>,
        Joonsoo Kim <iamjoonsoo.kim@....com>,
        Masayoshi Mizuma <msys.mizuma@...il.com>,
        Matthew Wilcox <willy@...radead.org>,
        Michal Hocko <mhocko@...nel.org>,
        Muchun Song <songmuchun@...edance.com>,
        Pekka Enberg <penberg@...nel.org>,
        Roman Gushchin <guro@...com>, Tejun Heo <tj@...nel.org>,
        Vladimir Davydov <vdavydov.dev@...il.com>,
        Vlastimil Babka <vbabka@...e.cz>,
        Wei Yang <richard.weiyang@...il.com>,
        Xing Zhengjun <zhengjun.xing@...ux.intel.com>,
        Yafang Shao <laoar.shao@...il.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org,
        lkp@...el.com, ying.huang@...el.com, feng.tang@...el.com
Subject: [mm/memcg] 68ac5b3c8d: hackbench.throughput 42.4% improvement

Greeting,

FYI, we noticed a 42.4% improvement of hackbench.throughput due to commit:


commit: 68ac5b3c8db2fda00af594eca4100aceaf927c0e ("mm/memcg: cache vmstat data in percpu memcg_stock_pcp")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master


in testcase: hackbench
on test machine: 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
with following parameters:

	nr_threads: 100%
	iterations: 4
	mode: threads
	ipc: socket
	cpufreq_governor: performance
	ucode: 0x5003006

test-description: Hackbench is both a benchmark and a stress test for the Linux kernel scheduler.
test-url: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/sched/cfs-scheduler/hackbench.c

In addition to that, the commit also has significant impact on the following tests:

+------------------+----------------------------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput 32.7% improvement                    |
| test machine     | 96 threads 2 sockets Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory |
| test parameters  | cpufreq_governor=performance                                         |
|                  | ipc=socket                                                           |
|                  | iterations=8                                                         |
|                  | mode=process                                                         |
|                  | nr_threads=1600%                                                     |
|                  | ucode=0x4003006                                                      |
+------------------+----------------------------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput 40.7% improvement                    |
| test machine     | 104 threads 2 sockets Skylake with 192G memory                       |
| test parameters  | cpufreq_governor=performance                                         |
|                  | ipc=socket                                                           |
|                  | iterations=4                                                         |
|                  | mode=threads                                                         |
|                  | nr_threads=100%                                                      |
|                  | ucode=0x2006a0a                                                      |
+------------------+----------------------------------------------------------------------+




Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install                job.yaml  # job file is attached in this email
        bin/lkp split-job --compatible job.yaml  # generate the yaml file for lkp run
        bin/lkp run                    generated-yaml-file

=========================================================================================
compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
  gcc-9/performance/socket/4/x86_64-rhel-8.3/threads/100%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2ap4/hackbench/0x5003006

commit: 
  fdbcb2a6d6 ("mm/memcg: move mod_objcg_state() to memcontrol.c")
  68ac5b3c8d ("mm/memcg: cache vmstat data in percpu memcg_stock_pcp")

fdbcb2a6d6778e0b 68ac5b3c8db2fda00af594eca41 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
     64004 ±  5%     +42.4%      91157 ±  2%  hackbench.throughput
      1533 ±  2%     -33.0%       1027 ±  3%  hackbench.time.elapsed_time
      1533 ±  2%     -33.0%       1027 ±  3%  hackbench.time.elapsed_time.max
 4.679e+09 ±  4%     +13.9%  5.327e+09 ±  3%  hackbench.time.involuntary_context_switches
    282365 ±  2%     -33.7%     187168 ±  3%  hackbench.time.system_time
      4044 ±  2%     +11.1%       4492        hackbench.time.user_time
 7.581e+09            +2.9%  7.802e+09        hackbench.time.voluntary_context_switches
    284299 ± 26%     -75.0%      71180 ± 22%  numa-meminfo.node3.AnonPages.max
      1580 ±  2%     -32.0%       1074 ±  3%  uptime.boot
   8025999 ±  3%     +59.8%   12822101 ±  5%  vmstat.system.cs
      2.69 ±  4%      +1.5        4.23 ±  7%  mpstat.cpu.all.idle%
      1.37 ±  3%      +0.9        2.24 ±  3%  mpstat.cpu.all.usr%
   1338201 ± 22%     -48.0%     696011 ± 22%  numa-numastat.node0.local_node
   1368199 ± 20%     -45.4%     746623 ± 18%  numa-numastat.node0.numa_hit
      0.55 ± 13%     +39.3%       0.76 ±  9%  turbostat.CPU%c1
  1.58e+09 ± 11%     -27.7%  1.142e+09 ± 15%  turbostat.IRQ
     46.55 ±223%    +523.7%     290.31        turbostat.PkgWatt
    138673            -3.0%     134515        proc-vmstat.nr_kernel_stack
   4798903 ±  6%     -21.4%    3773357 ±  3%  proc-vmstat.numa_hit
   4538312 ±  6%     -22.6%    3512942 ±  4%  proc-vmstat.numa_local
   9270219 ± 10%     -27.2%    6747732 ±  6%  proc-vmstat.pgalloc_normal
   5263974 ±  6%     -16.0%    4421768 ±  6%  proc-vmstat.pgfault
   8639280 ± 11%     -29.2%    6114666 ±  6%  proc-vmstat.pgfree
    201629 ±  3%     -26.0%     149254 ±  5%  proc-vmstat.pgreuse
     14.09 ±  2%     -15.2%      11.94        perf-stat.i.MPKI
 1.679e+10 ±  3%     +47.6%  2.478e+10 ±  5%  perf-stat.i.branch-instructions
 1.962e+08 ±  2%     +46.4%  2.873e+08 ±  4%  perf-stat.i.branch-misses
     23.61 ±  5%      -6.5       17.08 ± 13%  perf-stat.i.cache-miss-rate%
 1.103e+09 ±  2%     +24.5%  1.374e+09 ±  4%  perf-stat.i.cache-references
   8258347 ±  3%     +61.2%   13309378 ±  5%  perf-stat.i.context-switches
      7.73 ±  2%     -31.5%       5.30 ±  5%  perf-stat.i.cpi
    757865 ±  4%     +54.3%    1169348 ±  3%  perf-stat.i.cpu-migrations
 2.305e+10 ±  3%     +46.4%  3.374e+10 ±  5%  perf-stat.i.dTLB-loads
 1.351e+10 ±  3%     +48.0%  1.999e+10 ±  4%  perf-stat.i.dTLB-stores
 1.029e+08           +29.9%  1.336e+08 ±  3%  perf-stat.i.iTLB-load-misses
   1627438 ±  9%     +22.9%    2000194 ±  9%  perf-stat.i.iTLB-loads
 8.229e+10 ±  3%     +46.5%  1.206e+11 ±  5%  perf-stat.i.instructions
    784.62 ±  2%     +13.1%     887.30 ±  2%  perf-stat.i.instructions-per-iTLB-miss
      0.14 ±  3%     +46.5%       0.20 ±  5%  perf-stat.i.ipc
    282.63 ±  3%     +46.6%     414.21 ±  5%  perf-stat.i.metric.M/sec
      3392 ±  5%     +24.6%       4227 ±  3%  perf-stat.i.minor-faults
      3395 ±  5%     +24.7%       4232 ±  3%  perf-stat.i.page-faults
     13.40 ±  3%     -15.8%      11.28        perf-stat.overall.MPKI
     23.13 ±  6%      -6.7       16.39 ± 13%  perf-stat.overall.cache-miss-rate%
      7.30 ±  3%     -31.3%       5.02 ±  5%  perf-stat.overall.cpi
    799.61 ±  2%     +12.9%     903.10 ±  3%  perf-stat.overall.instructions-per-iTLB-miss
      0.14 ±  3%     +45.9%       0.20 ±  5%  perf-stat.overall.ipc
 1.647e+10 ±  3%     +46.6%  2.415e+10 ±  6%  perf-stat.ps.branch-instructions
 1.919e+08 ±  2%     +45.4%   2.79e+08 ±  4%  perf-stat.ps.branch-misses
 1.081e+09 ±  2%     +22.5%  1.324e+09 ±  5%  perf-stat.ps.cache-references
   8024383 ±  3%     +59.9%   12828939 ±  5%  perf-stat.ps.context-switches
    706549 ±  4%     +52.4%    1077133 ±  4%  perf-stat.ps.cpu-migrations
  2.26e+10 ±  2%     +45.6%  3.289e+10 ±  5%  perf-stat.ps.dTLB-loads
 1.324e+10 ±  2%     +47.3%   1.95e+10 ±  5%  perf-stat.ps.dTLB-stores
  1.01e+08           +28.8%    1.3e+08 ±  3%  perf-stat.ps.iTLB-load-misses
   1553977 ±  8%     +24.9%    1940929 ±  8%  perf-stat.ps.iTLB-loads
 8.074e+10 ±  3%     +45.5%  1.175e+11 ±  5%  perf-stat.ps.instructions
      3373 ±  5%     +25.2%       4223 ±  4%  perf-stat.ps.minor-faults
   1548676 ± 14%     +41.1%    2185441 ±  8%  perf-stat.ps.node-stores
      3376 ±  5%     +25.2%       4227 ±  4%  perf-stat.ps.page-faults
     70.29 ± 13%     -64.8%      24.74 ± 17%  perf-sched.sch_delay.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
    113.50 ±101%     -96.1%       4.42 ±201%  perf-sched.sch_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
     11944 ±134%     -99.8%      18.11 ±219%  perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
     51600 ± 20%     -48.4%      26647 ± 16%  perf-sched.sch_delay.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
     17653 ± 71%     -93.8%       1093 ±203%  perf-sched.sch_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
     46.30 ± 13%     -66.5%      15.49 ± 19%  perf-sched.total_sch_delay.average.ms
     51658 ± 20%     -48.1%      26788 ± 16%  perf-sched.total_sch_delay.max.ms
    175.44 ± 16%     -62.1%      66.54 ± 18%  perf-sched.total_wait_and_delay.average.ms
   1854505 ±  4%     +10.3%    2045473 ±  2%  perf-sched.total_wait_and_delay.count.ms
    104015 ± 20%     -48.5%      53554 ± 16%  perf-sched.total_wait_and_delay.max.ms
    129.14 ± 17%     -60.5%      51.04 ± 18%  perf-sched.total_wait_time.average.ms
     54569 ± 18%     -49.1%      27787 ± 16%  perf-sched.total_wait_time.max.ms
      2614 ± 39%     -66.8%     868.44 ± 81%  perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
    106.50 ± 19%     -71.5%      30.39 ± 27%  perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
    260.22 ± 26%     -65.9%      88.85 ± 35%  perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
    253.08 ± 15%     -85.5%      36.62 ±101%  perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
    296.64 ± 26%     -73.9%      77.38 ± 49%  perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
    344.94 ± 24%     -85.2%      51.19 ±104%  perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg
    206.16 ± 15%     -57.7%      87.25 ± 15%  perf-sched.wait_and_delay.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      3615 ± 15%     -57.0%       1556 ± 15%  perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
    611366 ± 10%     +25.0%     764401 ±  5%  perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
     35418 ±  5%     -30.1%      24750 ± 10%  perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
      1907 ±  5%     -66.1%     647.33 ±103%  perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
      1009 ±  6%     -29.4%     712.67 ± 45%  perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
    305.67 ± 18%     -22.0%     238.33 ± 10%  perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
     80491 ± 40%     -46.6%      42961 ± 38%  perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
     50832 ± 26%     -79.3%      10535 ±101%  perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
     43152 ± 35%     -62.7%      16096 ± 49%  perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
     35946 ± 21%     -70.9%      10460 ±100%  perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg
    103767 ± 19%     -48.7%      53272 ± 16%  perf-sched.wait_and_delay.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
     61843 ± 22%     -56.8%      26686 ± 13%  perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
      2613 ± 39%     -66.6%     871.99 ± 80%  perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
    290.13 ± 37%     -76.1%      69.34 ± 96%  perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
    104.89 ± 18%     -71.6%      29.74 ± 26%  perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
    258.62 ± 25%     -66.6%      86.50 ± 33%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
    249.20 ± 16%     -74.7%      63.10 ± 23%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
    296.60 ± 26%     -71.1%      85.74 ± 26%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
    344.89 ± 24%     -82.5%      60.26 ± 76%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg
    135.87 ± 18%     -54.0%      62.51 ± 16%  perf-sched.wait_time.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      3502 ± 16%     -55.7%       1551 ± 15%  perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
     14789 ±105%     -76.7%       3442 ±219%  perf-sched.wait_time.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
     50293 ± 22%     -74.6%      12773 ± 76%  perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
     54516 ± 18%     -49.2%      27716 ± 16%  perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
     53531 ± 19%     -49.2%      27167 ± 14%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
     47573 ± 14%     -58.8%      19614 ± 14%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
     43152 ± 35%     -56.8%      18656 ± 19%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
     35946 ± 21%     -64.0%      12942 ± 66%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg
     52900 ± 21%     -48.7%      27133 ± 17%  perf-sched.wait_time.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
     52252 ± 19%     -48.9%      26686 ± 13%  perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
      7.50 ±  7%      -5.6        1.89 ± 29%  perf-profile.calltrace.cycles-pp.mod_objcg_state.kfree.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
      5.58 ±  9%      -4.6        1.01 ± 48%  perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
     42.60 ±  2%      -2.6       40.00 ±  3%  perf-profile.calltrace.cycles-pp.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read
     42.67 ±  2%      -2.6       40.11 ±  3%  perf-profile.calltrace.cycles-pp.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read
     25.82 ±  2%      -2.2       23.62 ±  6%  perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
     25.96 ±  2%      -2.1       23.83 ±  6%  perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
     26.25 ±  2%      -2.0       24.24 ±  6%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
     26.28 ±  2%      -2.0       24.28 ±  6%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_read
     26.63 ±  2%      -1.9       24.74 ±  6%  perf-profile.calltrace.cycles-pp.__libc_read
      5.57 ±  6%      +2.4        7.95 ± 21%  perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
     20.20 ±  9%     -16.4        3.84 ± 26%  perf-profile.children.cycles-pp.mod_objcg_state
      9.12 ± 17%      -8.8        0.33 ± 33%  perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
      9.00 ± 17%      -8.7        0.29 ± 37%  perf-profile.children.cycles-pp.__mod_memcg_state
     42.62 ±  2%      -2.6       40.03 ±  3%  perf-profile.children.cycles-pp.unix_stream_read_generic
     42.67 ±  2%      -2.6       40.11 ±  3%  perf-profile.children.cycles-pp.unix_stream_recvmsg
      2.71 ± 14%      -1.4        1.27 ± 60%  perf-profile.children.cycles-pp.asm_sysvec_call_function_single
     98.60            -0.7       97.87        perf-profile.children.cycles-pp.do_syscall_64
     98.72            -0.7       98.05        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.27 ± 15%      -0.1        0.14 ± 45%  perf-profile.children.cycles-pp.flush_smp_call_function_queue
      0.27 ± 14%      -0.1        0.16 ± 35%  perf-profile.children.cycles-pp.ttwu_queue_wakelist
      0.14 ± 17%      -0.1        0.08 ±  6%  perf-profile.children.cycles-pp.aa_file_perm
      0.10 ± 18%      -0.1        0.03 ±103%  perf-profile.children.cycles-pp.__smp_call_single_queue
      0.10 ± 18%      -0.1        0.03 ±101%  perf-profile.children.cycles-pp.llist_add_batch
      0.09 ± 22%      -0.1        0.03 ±105%  perf-profile.children.cycles-pp.llist_reverse_order
      0.08 ±  6%      +0.0        0.10        perf-profile.children.cycles-pp.__ksize
      0.05 ±  8%      +0.0        0.08 ± 13%  perf-profile.children.cycles-pp.rb_insert_color
      0.06 ± 46%      +0.0        0.10 ±  4%  perf-profile.children.cycles-pp.resched_curr
      0.16 ±  5%      +0.0        0.20 ±  4%  perf-profile.children.cycles-pp.security_socket_sendmsg
      0.13 ±  4%      +0.0        0.17 ±  4%  perf-profile.children.cycles-pp.__check_heap_object
      0.05 ±  7%      +0.0        0.09 ±  5%  perf-profile.children.cycles-pp.__pthread_enable_asynccancel
      0.10 ±  6%      +0.0        0.15 ± 11%  perf-profile.children.cycles-pp.clear_buddies
      0.10 ±  4%      +0.0        0.15 ±  9%  perf-profile.children.cycles-pp.security_socket_recvmsg
      0.11 ± 16%      +0.1        0.16 ±  4%  perf-profile.children.cycles-pp.___perf_sw_event
      0.08 ±  5%      +0.1        0.14 ± 45%  perf-profile.children.cycles-pp.asm_sysvec_reschedule_ipi
      0.26 ±  3%      +0.1        0.31 ±  7%  perf-profile.children.cycles-pp.mutex_lock
      0.02 ±141%      +0.1        0.07 ± 10%  perf-profile.children.cycles-pp.rcu_all_qs
      0.14 ±  4%      +0.1        0.20 ±  2%  perf-profile.children.cycles-pp.update_min_vruntime
      0.09 ± 16%      +0.1        0.15 ± 16%  perf-profile.children.cycles-pp.__build_skb_around
      0.07 ± 17%      +0.1        0.14 ± 24%  perf-profile.children.cycles-pp.rb_erase
      0.10 ±  4%      +0.1        0.16 ±  4%  perf-profile.children.cycles-pp.copyin
      0.09 ± 13%      +0.1        0.16 ± 30%  perf-profile.children.cycles-pp.skb_unlink
      0.00            +0.1        0.06 ± 19%  perf-profile.children.cycles-pp.wakeup_preempt_entity
      0.12 ±  3%      +0.1        0.19 ±  7%  perf-profile.children.cycles-pp.sock_recvmsg
      0.60 ±  2%      +0.1        0.66 ±  3%  perf-profile.children.cycles-pp.__check_object_size
      0.01 ±223%      +0.1        0.08 ± 18%  perf-profile.children.cycles-pp.fput
      0.08 ±  8%      +0.1        0.15 ±  7%  perf-profile.children.cycles-pp.unix_write_space
      0.09 ±  9%      +0.1        0.16 ±  7%  perf-profile.children.cycles-pp.__list_del_entry_valid
      0.20 ±  5%      +0.1        0.28 ±  9%  perf-profile.children.cycles-pp.___might_sleep
      0.16 ±  5%      +0.1        0.25 ±  5%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      0.12 ±  6%      +0.1        0.21 ±  3%  perf-profile.children.cycles-pp.__pthread_disable_asynccancel
      0.00            +0.1        0.08 ±  8%  perf-profile.children.cycles-pp.__list_add_valid
      0.13 ±  3%      +0.1        0.21 ± 10%  perf-profile.children.cycles-pp.__entry_text_start
      0.19 ± 11%      +0.1        0.27 ± 10%  perf-profile.children.cycles-pp.put_prev_entity
      0.18 ±  6%      +0.1        0.26 ±  8%  perf-profile.children.cycles-pp._copy_from_iter
      0.12 ±  8%      +0.1        0.21 ± 10%  perf-profile.children.cycles-pp.fput_many
      0.47 ±  8%      +0.1        0.57 ± 11%  perf-profile.children.cycles-pp.prepare_to_wait
      0.20 ±  8%      +0.1        0.31 ±  6%  perf-profile.children.cycles-pp.__enqueue_entity
      0.40 ±  7%      +0.1        0.51 ±  8%  perf-profile.children.cycles-pp.__update_load_avg_se
      0.20 ±  9%      +0.1        0.32 ± 11%  perf-profile.children.cycles-pp.check_preempt_wakeup
      0.21 ±  8%      +0.1        0.34 ±  2%  perf-profile.children.cycles-pp.pick_next_entity
      0.30 ± 10%      +0.1        0.44 ± 34%  perf-profile.children.cycles-pp.update_cfs_group
      0.26 ±  7%      +0.1        0.40 ±  2%  perf-profile.children.cycles-pp.check_preempt_curr
      0.63 ±  8%      +0.2        0.79 ±  7%  perf-profile.children.cycles-pp.reweight_entity
      0.34 ±  6%      +0.2        0.51 ±  9%  perf-profile.children.cycles-pp.skb_copy_datagram_from_iter
      0.28 ±  7%      +0.2        0.46 ±  4%  perf-profile.children.cycles-pp.ttwu_do_wakeup
      0.34 ±  7%      +0.2        0.51 ± 14%  perf-profile.children.cycles-pp.__fget_files
      0.51 ±  5%      +0.2        0.68 ±  7%  perf-profile.children.cycles-pp.__fdget_pos
      0.31 ±  8%      +0.2        0.49 ± 12%  perf-profile.children.cycles-pp.__switch_to_asm
      0.48 ±  5%      +0.2        0.67 ±  7%  perf-profile.children.cycles-pp.__fget_light
      0.29 ± 11%      +0.2        0.49 ±  5%  perf-profile.children.cycles-pp.set_next_entity
      0.48 ±  5%      +0.3        0.74 ± 12%  perf-profile.children.cycles-pp.switch_fpu_return
      0.36 ±  8%      +0.3        0.68 ± 33%  perf-profile.children.cycles-pp.page_counter_cancel
      1.40 ±  6%      +0.4        1.75 ±  9%  perf-profile.children.cycles-pp.update_load_avg
      0.48 ±  7%      +0.4        0.83 ± 33%  perf-profile.children.cycles-pp.page_counter_uncharge
      0.58 ±  8%      +0.4        0.98 ± 36%  perf-profile.children.cycles-pp.page_counter_try_charge
      0.89 ±  7%      +0.5        1.42 ±  3%  perf-profile.children.cycles-pp.pick_next_task_fair
      0.81 ± 20%      +0.9        1.75 ± 20%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      1.94 ±  7%      +1.0        2.98 ±  7%  perf-profile.children.cycles-pp._raw_spin_lock
      8.30 ±  7%      +3.1       11.36 ± 23%  perf-profile.children.cycles-pp.get_obj_cgroup_from_current
      9.00 ± 17%      -8.7        0.29 ± 37%  perf-profile.self.cycles-pp.__mod_memcg_state
     10.31 ±  7%      -6.9        3.42 ± 28%  perf-profile.self.cycles-pp.mod_objcg_state
      1.49 ±  8%      -1.4        0.06 ±  8%  perf-profile.self.cycles-pp.refill_obj_stock
      0.38 ± 19%      -0.3        0.04 ±112%  perf-profile.self.cycles-pp.obj_cgroup_charge_pages
      0.35 ± 19%      -0.3        0.05 ± 83%  perf-profile.self.cycles-pp.obj_cgroup_uncharge_pages
      0.17 ± 15%      -0.1        0.07 ± 84%  perf-profile.self.cycles-pp.flush_smp_call_function_queue
      0.11 ± 18%      -0.1        0.04 ±107%  perf-profile.self.cycles-pp.sched_ttwu_pending
      0.14 ± 16%      -0.1        0.07        perf-profile.self.cycles-pp.aa_file_perm
      0.18 ±  5%      -0.1        0.11 ±  3%  perf-profile.self.cycles-pp._copy_to_iter
      0.10 ± 18%      -0.1        0.03 ±101%  perf-profile.self.cycles-pp.llist_add_batch
      0.09 ± 20%      -0.1        0.03 ±105%  perf-profile.self.cycles-pp.llist_reverse_order
      0.19 ±  9%      -0.1        0.13 ± 11%  perf-profile.self.cycles-pp.new_sync_write
      0.07 ±  6%      +0.0        0.09 ±  5%  perf-profile.self.cycles-pp.__ksize
      0.09            +0.0        0.11 ±  9%  perf-profile.self.cycles-pp.aa_sk_perm
      0.05 ±  8%      +0.0        0.08 ± 13%  perf-profile.self.cycles-pp.unix_stream_recvmsg
      0.05 ±  8%      +0.0        0.08 ± 14%  perf-profile.self.cycles-pp.rb_insert_color
      0.09 ±  7%      +0.0        0.12 ± 19%  perf-profile.self.cycles-pp.mutex_unlock
      0.06 ± 46%      +0.0        0.09 ±  7%  perf-profile.self.cycles-pp.resched_curr
      0.12 ±  3%      +0.0        0.16 ±  5%  perf-profile.self.cycles-pp.__check_heap_object
      0.05 ±  7%      +0.0        0.09 ±  4%  perf-profile.self.cycles-pp.__pthread_enable_asynccancel
      0.09 ±  9%      +0.0        0.13 ± 11%  perf-profile.self.cycles-pp.clear_buddies
      0.12 ±  9%      +0.0        0.17 ±  6%  perf-profile.self.cycles-pp.__alloc_skb
      0.07 ± 11%      +0.0        0.12 ±  4%  perf-profile.self.cycles-pp.set_next_entity
      0.00            +0.1        0.05 ±  7%  perf-profile.self.cycles-pp.obj_cgroup_charge
      0.10 ± 10%      +0.1        0.15 ±  5%  perf-profile.self.cycles-pp.pick_next_entity
      0.02 ±141%      +0.1        0.07 ± 14%  perf-profile.self.cycles-pp._copy_from_iter
      0.13 ±  7%      +0.1        0.18 ±  8%  perf-profile.self.cycles-pp.sock_write_iter
      0.14 ±  5%      +0.1        0.20 ±  3%  perf-profile.self.cycles-pp.update_min_vruntime
      0.02 ±142%      +0.1        0.08 ± 10%  perf-profile.self.cycles-pp.sock_alloc_send_pskb
      0.00            +0.1        0.06 ±  8%  perf-profile.self.cycles-pp.unix_destruct_scm
      0.00            +0.1        0.06 ±  8%  perf-profile.self.cycles-pp.ksys_read
      0.20 ±  8%      +0.1        0.26 ±  7%  perf-profile.self.cycles-pp.sock_def_readable
      0.08 ± 16%      +0.1        0.14 ±  2%  perf-profile.self.cycles-pp.___perf_sw_event
      0.05            +0.1        0.11 ± 14%  perf-profile.self.cycles-pp.__libc_write
      0.05 ± 71%      +0.1        0.10 ± 23%  perf-profile.self.cycles-pp.__build_skb_around
      0.07 ±  6%      +0.1        0.13 ±  5%  perf-profile.self.cycles-pp.unix_write_space
      0.08 ±  4%      +0.1        0.14 ± 12%  perf-profile.self.cycles-pp.__entry_text_start
      0.24 ±  6%      +0.1        0.30 ±  6%  perf-profile.self.cycles-pp.enqueue_entity
      0.07 ± 18%      +0.1        0.13 ± 26%  perf-profile.self.cycles-pp.rb_erase
      0.00            +0.1        0.06 ± 11%  perf-profile.self.cycles-pp.rcu_all_qs
      0.00            +0.1        0.06 ± 19%  perf-profile.self.cycles-pp.wakeup_preempt_entity
      0.12 ±  4%      +0.1        0.18 ±  7%  perf-profile.self.cycles-pp.new_sync_read
      0.08 ±  8%      +0.1        0.15 ±  9%  perf-profile.self.cycles-pp.__list_del_entry_valid
      0.12 ±  6%      +0.1        0.18 ± 13%  perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.09 ± 15%      +0.1        0.16 ±  6%  perf-profile.self.cycles-pp.consume_skb
      0.07 ± 10%      +0.1        0.15 ± 18%  perf-profile.self.cycles-pp.check_preempt_wakeup
      0.00            +0.1        0.08 ±  6%  perf-profile.self.cycles-pp.__list_add_valid
      0.09 ±  4%      +0.1        0.17 ±  7%  perf-profile.self.cycles-pp.sock_read_iter
      0.19 ±  3%      +0.1        0.27 ±  8%  perf-profile.self.cycles-pp.___might_sleep
      0.16 ±  3%      +0.1        0.24 ±  7%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      0.12 ±  4%      +0.1        0.20 ±  4%  perf-profile.self.cycles-pp.__pthread_disable_asynccancel
      0.15 ±  8%      +0.1        0.24 ± 11%  perf-profile.self.cycles-pp.vfs_read
      0.11 ±  8%      +0.1        0.20 ± 10%  perf-profile.self.cycles-pp.fput_many
      0.16 ±  4%      +0.1        0.26 ±  3%  perf-profile.self.cycles-pp.prepare_to_wait
      0.13 ±  5%      +0.1        0.22 ±  5%  perf-profile.self.cycles-pp.pick_next_task_fair
      0.20 ±  9%      +0.1        0.30 ±  6%  perf-profile.self.cycles-pp.__enqueue_entity
      0.40 ±  8%      +0.1        0.50 ±  8%  perf-profile.self.cycles-pp.__update_load_avg_se
      0.30 ±  9%      +0.1        0.43 ± 33%  perf-profile.self.cycles-pp.update_cfs_group
      0.72 ±  6%      +0.2        0.88 ±  3%  perf-profile.self.cycles-pp.unix_stream_read_generic
      0.20 ± 22%      +0.2        0.36 ± 17%  perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
      0.32 ±  7%      +0.2        0.49 ± 13%  perf-profile.self.cycles-pp.__fget_files
      0.31 ±  8%      +0.2        0.49 ± 12%  perf-profile.self.cycles-pp.__switch_to_asm
      0.58 ±  7%      +0.2        0.76 ± 10%  perf-profile.self.cycles-pp.update_load_avg
      0.90 ±  5%      +0.2        1.12 ±  7%  perf-profile.self.cycles-pp.__schedule
      0.46 ±  5%      +0.3        0.73 ± 12%  perf-profile.self.cycles-pp.switch_fpu_return
      0.48 ±  4%      +0.3        0.75 ±  2%  perf-profile.self.cycles-pp.update_curr
      0.36 ±  8%      +0.3        0.67 ± 32%  perf-profile.self.cycles-pp.page_counter_cancel
      0.46 ±  8%      +0.3        0.79 ± 34%  perf-profile.self.cycles-pp.page_counter_try_charge
      1.20 ±  9%      +0.8        2.04 ± 43%  perf-profile.self.cycles-pp.drain_obj_stock
      0.81 ± 20%      +0.9        1.75 ± 20%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
      4.48 ±  8%      +2.6        7.07 ± 21%  perf-profile.self.cycles-pp.kfree
      8.14 ±  7%      +3.0       11.17 ± 23%  perf-profile.self.cycles-pp.get_obj_cgroup_from_current
 1.531e+09 ± 15%     -36.3%  9.749e+08 ± 20%  interrupts.CAL:Function_call_interrupts
   3068881 ±  2%     -33.0%    2057122 ±  3%  interrupts.CPU0.LOC:Local_timer_interrupts
   3068930 ±  2%     -33.0%    2057189 ±  3%  interrupts.CPU1.LOC:Local_timer_interrupts
   1920898 ±  5%     +18.5%    2276692 ± 12%  interrupts.CPU1.RES:Rescheduling_interrupts
   3069047 ±  2%     -33.0%    2057271 ±  3%  interrupts.CPU10.LOC:Local_timer_interrupts
   3068860 ±  2%     -33.0%    2057039 ±  3%  interrupts.CPU100.LOC:Local_timer_interrupts
   3068788 ±  2%     -33.0%    2057016 ±  3%  interrupts.CPU101.LOC:Local_timer_interrupts
   3068902 ±  2%     -33.0%    2056953 ±  3%  interrupts.CPU102.LOC:Local_timer_interrupts
   3068874 ±  2%     -33.0%    2057082 ±  3%  interrupts.CPU103.LOC:Local_timer_interrupts
   3068837 ±  2%     -33.0%    2057099 ±  3%  interrupts.CPU104.LOC:Local_timer_interrupts
   3068782 ±  2%     -33.0%    2057017 ±  3%  interrupts.CPU105.LOC:Local_timer_interrupts
   3068825 ±  2%     -33.0%    2056945 ±  3%  interrupts.CPU106.LOC:Local_timer_interrupts
   3068774 ±  2%     -33.0%    2056953 ±  3%  interrupts.CPU107.LOC:Local_timer_interrupts
   3068907 ±  2%     -33.0%    2057107 ±  3%  interrupts.CPU108.LOC:Local_timer_interrupts
   3068930 ±  2%     -33.0%    2057005 ±  3%  interrupts.CPU109.LOC:Local_timer_interrupts
   3069038 ±  2%     -33.0%    2057015 ±  3%  interrupts.CPU11.LOC:Local_timer_interrupts
   3068767 ±  2%     -33.0%    2057060 ±  3%  interrupts.CPU110.LOC:Local_timer_interrupts
   3069063 ±  2%     -33.0%    2057173 ±  3%  interrupts.CPU111.LOC:Local_timer_interrupts
   2047029 ± 20%     +50.9%    3089234 ± 19%  interrupts.CPU111.RES:Rescheduling_interrupts
   3068794 ±  2%     -33.0%    2057038 ±  3%  interrupts.CPU112.LOC:Local_timer_interrupts
   3068806 ±  2%     -33.0%    2057275 ±  3%  interrupts.CPU113.LOC:Local_timer_interrupts
   3068865 ±  2%     -33.0%    2057053 ±  3%  interrupts.CPU114.LOC:Local_timer_interrupts
   3068846 ±  2%     -33.0%    2056994 ±  3%  interrupts.CPU115.LOC:Local_timer_interrupts
   3068875 ±  2%     -33.0%    2057000 ±  3%  interrupts.CPU116.LOC:Local_timer_interrupts
   3068933 ±  2%     -33.0%    2057012 ±  3%  interrupts.CPU117.LOC:Local_timer_interrupts
   2181060 ± 15%     +29.6%    2826706 ± 17%  interrupts.CPU117.RES:Rescheduling_interrupts
   3068848 ±  2%     -33.0%    2057041 ±  3%  interrupts.CPU118.LOC:Local_timer_interrupts
   1928528 ± 16%     +66.8%    3216476 ± 22%  interrupts.CPU118.RES:Rescheduling_interrupts
   3068981 ±  2%     -33.0%    2057056 ±  3%  interrupts.CPU119.LOC:Local_timer_interrupts
   3068882 ±  2%     -33.0%    2057126 ±  3%  interrupts.CPU12.LOC:Local_timer_interrupts
   3068504 ±  2%     -33.0%    2056694 ±  3%  interrupts.CPU120.LOC:Local_timer_interrupts
   3068536 ±  2%     -33.0%    2056659 ±  3%  interrupts.CPU121.LOC:Local_timer_interrupts
  10108354 ± 22%     -45.9%    5470800 ± 50%  interrupts.CPU122.CAL:Function_call_interrupts
   3068562 ±  2%     -33.0%    2056653 ±  3%  interrupts.CPU122.LOC:Local_timer_interrupts
   3068480 ±  2%     -33.0%    2056677 ±  3%  interrupts.CPU123.LOC:Local_timer_interrupts
   3068501 ±  2%     -33.0%    2056658 ±  3%  interrupts.CPU124.LOC:Local_timer_interrupts
   3068580 ±  2%     -33.0%    2056697 ±  3%  interrupts.CPU125.LOC:Local_timer_interrupts
   3068654 ±  2%     -33.0%    2056619 ±  3%  interrupts.CPU126.LOC:Local_timer_interrupts
   3068613 ±  2%     -33.0%    2056633 ±  3%  interrupts.CPU127.LOC:Local_timer_interrupts
   8613086 ± 30%     -38.0%    5335817 ± 40%  interrupts.CPU128.CAL:Function_call_interrupts
   3068601 ±  2%     -33.0%    2056671 ±  3%  interrupts.CPU128.LOC:Local_timer_interrupts
   3068506 ±  2%     -33.0%    2056669 ±  3%  interrupts.CPU129.LOC:Local_timer_interrupts
   3068927 ±  2%     -33.0%    2057122 ±  3%  interrupts.CPU13.LOC:Local_timer_interrupts
   3068604 ±  2%     -33.0%    2056721 ±  3%  interrupts.CPU130.LOC:Local_timer_interrupts
   3068608 ±  2%     -33.0%    2056674 ±  3%  interrupts.CPU131.LOC:Local_timer_interrupts
   3068614 ±  2%     -33.0%    2056682 ±  3%  interrupts.CPU132.LOC:Local_timer_interrupts
   3068628 ±  2%     -33.0%    2056688 ±  3%  interrupts.CPU133.LOC:Local_timer_interrupts
   3068579 ±  2%     -33.0%    2056754 ±  3%  interrupts.CPU134.LOC:Local_timer_interrupts
   9908620 ± 17%     -41.8%    5762355 ± 37%  interrupts.CPU135.CAL:Function_call_interrupts
   3068551 ±  2%     -33.0%    2056654 ±  3%  interrupts.CPU135.LOC:Local_timer_interrupts
   2579601 ± 13%     +26.8%    3271812 ±  8%  interrupts.CPU135.RES:Rescheduling_interrupts
   9893222 ± 13%     -41.9%    5750291 ± 41%  interrupts.CPU136.CAL:Function_call_interrupts
   3068594 ±  2%     -33.0%    2056682 ±  3%  interrupts.CPU136.LOC:Local_timer_interrupts
   8859106 ± 29%     -36.4%    5638024 ± 45%  interrupts.CPU137.CAL:Function_call_interrupts
   3068569 ±  2%     -33.0%    2056686 ±  3%  interrupts.CPU137.LOC:Local_timer_interrupts
  10450231 ± 29%     -50.0%    5220116 ± 43%  interrupts.CPU138.CAL:Function_call_interrupts
   3068487 ±  2%     -33.0%    2056626 ±  3%  interrupts.CPU138.LOC:Local_timer_interrupts
  10266532 ± 14%     -46.7%    5467357 ± 40%  interrupts.CPU139.CAL:Function_call_interrupts
   3068574 ±  2%     -33.0%    2056681 ±  3%  interrupts.CPU139.LOC:Local_timer_interrupts
   3068883 ±  2%     -33.0%    2057013 ±  3%  interrupts.CPU14.LOC:Local_timer_interrupts
   3068438 ±  2%     -33.0%    2056701 ±  3%  interrupts.CPU140.LOC:Local_timer_interrupts
  10997529 ± 34%     -52.2%    5258673 ± 43%  interrupts.CPU141.CAL:Function_call_interrupts
   3068575 ±  2%     -33.0%    2056672 ±  3%  interrupts.CPU141.LOC:Local_timer_interrupts
  10525182 ± 20%     -45.5%    5740160 ± 36%  interrupts.CPU142.CAL:Function_call_interrupts
   3068527 ±  2%     -33.0%    2056675 ±  3%  interrupts.CPU142.LOC:Local_timer_interrupts
   3068512 ±  2%     -33.0%    2056710 ±  3%  interrupts.CPU143.LOC:Local_timer_interrupts
  15399949 ± 14%     -41.8%    8962485 ± 27%  interrupts.CPU144.CAL:Function_call_interrupts
   3068419 ±  2%     -33.0%    2056607 ±  3%  interrupts.CPU144.LOC:Local_timer_interrupts
  10281339 ± 11%     -36.3%    6544192 ± 28%  interrupts.CPU145.CAL:Function_call_interrupts
   3068492 ±  2%     -33.0%    2056585 ±  3%  interrupts.CPU145.LOC:Local_timer_interrupts
  10754518 ± 10%     -35.4%    6942483 ± 32%  interrupts.CPU146.CAL:Function_call_interrupts
   3068391 ±  2%     -33.0%    2056699 ±  3%  interrupts.CPU146.LOC:Local_timer_interrupts
  11596622 ± 11%     -49.4%    5862855 ± 14%  interrupts.CPU147.CAL:Function_call_interrupts
   3068489 ±  2%     -33.0%    2056558 ±  3%  interrupts.CPU147.LOC:Local_timer_interrupts
   3068564 ±  2%     -33.0%    2056604 ±  3%  interrupts.CPU148.LOC:Local_timer_interrupts
  10264948 ± 18%     -41.0%    6052360 ± 24%  interrupts.CPU149.CAL:Function_call_interrupts
   3068493 ±  2%     -33.0%    2056571 ±  3%  interrupts.CPU149.LOC:Local_timer_interrupts
   3068985 ±  2%     -33.0%    2057059 ±  3%  interrupts.CPU15.LOC:Local_timer_interrupts
  10640620 ± 21%     -36.8%    6719798 ± 27%  interrupts.CPU150.CAL:Function_call_interrupts
   3068456 ±  2%     -33.0%    2056548 ±  3%  interrupts.CPU150.LOC:Local_timer_interrupts
  11941375 ± 27%     -45.5%    6507800 ± 38%  interrupts.CPU151.CAL:Function_call_interrupts
   3068543 ±  2%     -33.0%    2056582 ±  3%  interrupts.CPU151.LOC:Local_timer_interrupts
   3068545 ±  2%     -33.0%    2056610 ±  3%  interrupts.CPU152.LOC:Local_timer_interrupts
   3068506 ±  2%     -33.0%    2056602 ±  3%  interrupts.CPU153.LOC:Local_timer_interrupts
   3068491 ±  2%     -33.0%    2056578 ±  3%  interrupts.CPU154.LOC:Local_timer_interrupts
   3068416 ±  2%     -33.0%    2056585 ±  3%  interrupts.CPU155.LOC:Local_timer_interrupts
  11182149 ± 28%     -43.0%    6371185 ± 22%  interrupts.CPU156.CAL:Function_call_interrupts
   3068566 ±  2%     -33.0%    2056586 ±  3%  interrupts.CPU156.LOC:Local_timer_interrupts
  10314090 ± 21%     -37.9%    6403854 ± 41%  interrupts.CPU157.CAL:Function_call_interrupts
   3068567 ±  2%     -33.0%    2056610 ±  3%  interrupts.CPU157.LOC:Local_timer_interrupts
  11129901 ± 16%     -38.7%    6827535 ± 42%  interrupts.CPU158.CAL:Function_call_interrupts
   3068396 ±  2%     -33.0%    2056570 ±  3%  interrupts.CPU158.LOC:Local_timer_interrupts
   3068626 ±  2%     -33.0%    2056594 ±  3%  interrupts.CPU159.LOC:Local_timer_interrupts
   3068810 ±  2%     -33.0%    2057138 ±  3%  interrupts.CPU16.LOC:Local_timer_interrupts
   2006008 ±  8%     +21.9%    2445856 ±  8%  interrupts.CPU16.RES:Rescheduling_interrupts
   3068549 ±  2%     -33.0%    2056601 ±  3%  interrupts.CPU160.LOC:Local_timer_interrupts
   3068397 ±  2%     -33.0%    2056605 ±  3%  interrupts.CPU161.LOC:Local_timer_interrupts
  12689653 ± 26%     -45.5%    6921228 ± 42%  interrupts.CPU162.CAL:Function_call_interrupts
   3068611 ±  2%     -33.0%    2056601 ±  3%  interrupts.CPU162.LOC:Local_timer_interrupts
  11554610 ± 25%     -51.1%    5646100 ± 27%  interrupts.CPU163.CAL:Function_call_interrupts
   3068522 ±  2%     -33.0%    2056610 ±  3%  interrupts.CPU163.LOC:Local_timer_interrupts
  11058349 ± 22%     -52.9%    5206973 ± 30%  interrupts.CPU164.CAL:Function_call_interrupts
   3068378 ±  2%     -33.0%    2056602 ±  3%  interrupts.CPU164.LOC:Local_timer_interrupts
   3068367 ±  2%     -33.0%    2056597 ±  3%  interrupts.CPU165.LOC:Local_timer_interrupts
  10546615 ± 15%     -41.1%    6216711 ± 24%  interrupts.CPU166.CAL:Function_call_interrupts
   3068546 ±  2%     -33.0%    2056587 ±  3%  interrupts.CPU166.LOC:Local_timer_interrupts
   3068381 ±  2%     -33.0%    2056590 ±  3%  interrupts.CPU167.LOC:Local_timer_interrupts
  14348960 ± 26%     -44.5%    7968793 ± 28%  interrupts.CPU168.CAL:Function_call_interrupts
   3068429 ±  2%     -33.0%    2056515 ±  3%  interrupts.CPU168.LOC:Local_timer_interrupts
   9037033 ±  8%     -40.9%    5343981 ± 33%  interrupts.CPU169.CAL:Function_call_interrupts
   3068245 ±  2%     -33.0%    2056521 ±  3%  interrupts.CPU169.LOC:Local_timer_interrupts
   3068796 ±  2%     -33.0%    2057061 ±  3%  interrupts.CPU17.LOC:Local_timer_interrupts
   3068458 ±  2%     -33.0%    2056570 ±  3%  interrupts.CPU170.LOC:Local_timer_interrupts
   3068456 ±  2%     -33.0%    2056537 ±  3%  interrupts.CPU171.LOC:Local_timer_interrupts
   2955398 ±  8%     +18.1%    3491658 ±  9%  interrupts.CPU171.RES:Rescheduling_interrupts
   3068436 ±  2%     -33.0%    2056565 ±  3%  interrupts.CPU172.LOC:Local_timer_interrupts
   2682491 ± 11%     +25.5%    3366650 ±  7%  interrupts.CPU172.RES:Rescheduling_interrupts
  10459980 ± 17%     -35.6%    6741253 ± 40%  interrupts.CPU173.CAL:Function_call_interrupts
   3068448 ±  2%     -33.0%    2056554 ±  3%  interrupts.CPU173.LOC:Local_timer_interrupts
   3068386 ±  2%     -33.0%    2056544 ±  3%  interrupts.CPU174.LOC:Local_timer_interrupts
  10400439 ± 22%     -42.0%    6030941 ± 19%  interrupts.CPU175.CAL:Function_call_interrupts
   3068481 ±  2%     -33.0%    2056548 ±  3%  interrupts.CPU175.LOC:Local_timer_interrupts
   3068419 ±  2%     -33.0%    2056525 ±  3%  interrupts.CPU176.LOC:Local_timer_interrupts
   9716947 ± 21%     -36.1%    6213209 ± 38%  interrupts.CPU177.CAL:Function_call_interrupts
   3068339 ±  2%     -33.0%    2056576 ±  3%  interrupts.CPU177.LOC:Local_timer_interrupts
   2786012 ± 13%     +26.8%    3533937 ±  3%  interrupts.CPU177.RES:Rescheduling_interrupts
   9637199 ± 27%     -48.9%    4919892 ± 36%  interrupts.CPU178.CAL:Function_call_interrupts
   3068474 ±  2%     -33.0%    2056541 ±  3%  interrupts.CPU178.LOC:Local_timer_interrupts
   9888292 ± 37%     -46.3%    5308589 ± 23%  interrupts.CPU179.CAL:Function_call_interrupts
   3068405 ±  2%     -33.0%    2056544 ±  3%  interrupts.CPU179.LOC:Local_timer_interrupts
   3069027 ±  2%     -33.0%    2057052 ±  3%  interrupts.CPU18.LOC:Local_timer_interrupts
   3068383 ±  2%     -33.0%    2056569 ±  3%  interrupts.CPU180.LOC:Local_timer_interrupts
   3068302 ±  2%     -33.0%    2056495 ±  3%  interrupts.CPU181.LOC:Local_timer_interrupts
  10805869 ± 23%     -46.2%    5813818 ± 28%  interrupts.CPU182.CAL:Function_call_interrupts
   3068259 ±  2%     -33.0%    2056541 ±  3%  interrupts.CPU182.LOC:Local_timer_interrupts
   2762944 ±  6%     +31.1%    3623514 ± 14%  interrupts.CPU182.RES:Rescheduling_interrupts
  10793214 ± 25%     -45.8%    5852092 ± 23%  interrupts.CPU183.CAL:Function_call_interrupts
   3068289 ±  2%     -33.0%    2056543 ±  3%  interrupts.CPU183.LOC:Local_timer_interrupts
   2586038 ± 12%     +30.6%    3377221 ± 12%  interrupts.CPU183.RES:Rescheduling_interrupts
   3068277 ±  2%     -33.0%    2056545 ±  3%  interrupts.CPU184.LOC:Local_timer_interrupts
   3068374 ±  2%     -33.0%    2056550 ±  3%  interrupts.CPU185.LOC:Local_timer_interrupts
   3068253 ±  2%     -33.0%    2056481 ±  3%  interrupts.CPU186.LOC:Local_timer_interrupts
   2649495 ± 12%     +31.9%    3494494 ± 10%  interrupts.CPU186.RES:Rescheduling_interrupts
  10025765 ± 21%     -43.0%    5712695 ± 25%  interrupts.CPU187.CAL:Function_call_interrupts
   3068329 ±  2%     -33.0%    2056551 ±  3%  interrupts.CPU187.LOC:Local_timer_interrupts
   2892028 ± 13%     +26.9%    3670638 ± 12%  interrupts.CPU187.RES:Rescheduling_interrupts
  11151288 ± 17%     -37.4%    6986064 ± 32%  interrupts.CPU188.CAL:Function_call_interrupts
   3068414 ±  2%     -33.0%    2056521 ±  3%  interrupts.CPU188.LOC:Local_timer_interrupts
   3068465 ±  2%     -33.0%    2056552 ±  3%  interrupts.CPU189.LOC:Local_timer_interrupts
   3068875 ±  2%     -33.0%    2057045 ±  3%  interrupts.CPU19.LOC:Local_timer_interrupts
   3068371 ±  2%     -33.0%    2056571 ±  3%  interrupts.CPU190.LOC:Local_timer_interrupts
   3068404 ±  2%     -33.0%    2056531 ±  3%  interrupts.CPU191.LOC:Local_timer_interrupts
   2761094 ± 11%     +22.6%    3384449 ± 13%  interrupts.CPU191.RES:Rescheduling_interrupts
   3068844 ±  2%     -33.0%    2057030 ±  3%  interrupts.CPU2.LOC:Local_timer_interrupts
   3068960 ±  2%     -33.0%    2057035 ±  3%  interrupts.CPU20.LOC:Local_timer_interrupts
   3068786 ±  2%     -33.0%    2057052 ±  3%  interrupts.CPU21.LOC:Local_timer_interrupts
   3068957 ±  2%     -33.0%    2057028 ±  3%  interrupts.CPU22.LOC:Local_timer_interrupts
   3068805 ±  2%     -33.0%    2057075 ±  3%  interrupts.CPU23.LOC:Local_timer_interrupts
   3068509 ±  2%     -33.0%    2056726 ±  3%  interrupts.CPU24.LOC:Local_timer_interrupts
   3068428 ±  2%     -33.0%    2056699 ±  3%  interrupts.CPU25.LOC:Local_timer_interrupts
   3068521 ±  2%     -33.0%    2056656 ±  3%  interrupts.CPU26.LOC:Local_timer_interrupts
   3068485 ±  2%     -33.0%    2056654 ±  3%  interrupts.CPU27.LOC:Local_timer_interrupts
   3068591 ±  2%     -33.0%    2056697 ±  3%  interrupts.CPU28.LOC:Local_timer_interrupts
   3068586 ±  2%     -33.0%    2056722 ±  3%  interrupts.CPU29.LOC:Local_timer_interrupts
   3068781 ±  2%     -33.0%    2057281 ±  3%  interrupts.CPU3.LOC:Local_timer_interrupts
   2058882 ±  8%     +37.5%    2830697 ± 16%  interrupts.CPU3.RES:Rescheduling_interrupts
   9634161 ± 25%     -48.5%    4959876 ± 37%  interrupts.CPU30.CAL:Function_call_interrupts
   3068605 ±  2%     -33.0%    2056691 ±  3%  interrupts.CPU30.LOC:Local_timer_interrupts
   3068530 ±  2%     -33.0%    2056720 ±  3%  interrupts.CPU31.LOC:Local_timer_interrupts
   3068594 ±  2%     -33.0%    2056673 ±  3%  interrupts.CPU32.LOC:Local_timer_interrupts
   3068489 ±  2%     -33.0%    2056665 ±  3%  interrupts.CPU33.LOC:Local_timer_interrupts
   9626755 ± 23%     -43.7%    5423271 ± 39%  interrupts.CPU34.CAL:Function_call_interrupts
   3068542 ±  2%     -33.0%    2056673 ±  3%  interrupts.CPU34.LOC:Local_timer_interrupts
   3068547 ±  2%     -33.0%    2056637 ±  3%  interrupts.CPU35.LOC:Local_timer_interrupts
   3068584 ±  2%     -33.0%    2056645 ±  3%  interrupts.CPU36.LOC:Local_timer_interrupts
   2308345 ±  3%     +21.9%    2814386 ±  5%  interrupts.CPU36.RES:Rescheduling_interrupts
   3068617 ±  2%     -33.0%    2056701 ±  3%  interrupts.CPU37.LOC:Local_timer_interrupts
   2447075 ±  5%     +18.6%    2902611 ±  8%  interrupts.CPU37.RES:Rescheduling_interrupts
   9333901 ± 18%     -43.7%    5253242 ± 46%  interrupts.CPU38.CAL:Function_call_interrupts
   3068597 ±  2%     -33.0%    2056701 ±  3%  interrupts.CPU38.LOC:Local_timer_interrupts
   8973018 ± 11%     -29.1%    6363051 ± 41%  interrupts.CPU39.CAL:Function_call_interrupts
   3068594 ±  2%     -33.0%    2056719 ±  3%  interrupts.CPU39.LOC:Local_timer_interrupts
   3068897 ±  2%     -33.0%    2057142 ±  3%  interrupts.CPU4.LOC:Local_timer_interrupts
   8916276 ± 24%     -38.4%    5488654 ± 40%  interrupts.CPU40.CAL:Function_call_interrupts
   3068610 ±  2%     -33.0%    2056665 ±  3%  interrupts.CPU40.LOC:Local_timer_interrupts
   3068577 ±  2%     -33.0%    2056681 ±  3%  interrupts.CPU41.LOC:Local_timer_interrupts
  10118061 ± 29%     -48.1%    5247165 ± 42%  interrupts.CPU42.CAL:Function_call_interrupts
   3068526 ±  2%     -33.0%    2056558 ±  3%  interrupts.CPU42.LOC:Local_timer_interrupts
   9709031 ± 12%     -42.0%    5627612 ± 37%  interrupts.CPU43.CAL:Function_call_interrupts
   3068543 ±  2%     -33.0%    2056676 ±  3%  interrupts.CPU43.LOC:Local_timer_interrupts
   9925720 ± 28%     -50.1%    4948073 ± 39%  interrupts.CPU44.CAL:Function_call_interrupts
   3068519 ±  2%     -33.0%    2056749 ±  3%  interrupts.CPU44.LOC:Local_timer_interrupts
   9581695 ± 18%     -42.5%    5508025 ± 43%  interrupts.CPU45.CAL:Function_call_interrupts
   3068590 ±  2%     -33.0%    2056696 ±  3%  interrupts.CPU45.LOC:Local_timer_interrupts
   9195132 ± 24%     -35.5%    5931145 ± 43%  interrupts.CPU46.CAL:Function_call_interrupts
   3068544 ±  2%     -33.0%    2056696 ±  3%  interrupts.CPU46.LOC:Local_timer_interrupts
   3068531 ±  2%     -33.0%    2056571 ±  3%  interrupts.CPU47.LOC:Local_timer_interrupts
   3068482 ±  2%     -33.0%    2056580 ±  3%  interrupts.CPU48.LOC:Local_timer_interrupts
  10877934 ± 17%     -38.7%    6666999 ± 38%  interrupts.CPU49.CAL:Function_call_interrupts
   3068444 ±  2%     -33.0%    2056577 ±  3%  interrupts.CPU49.LOC:Local_timer_interrupts
   3068920 ±  2%     -33.0%    2057131 ±  3%  interrupts.CPU5.LOC:Local_timer_interrupts
  10434336 ± 20%     -42.8%    5966100 ± 30%  interrupts.CPU50.CAL:Function_call_interrupts
   3068377 ±  2%     -33.0%    2056590 ±  3%  interrupts.CPU50.LOC:Local_timer_interrupts
   3068408 ±  2%     -33.0%    2056586 ±  3%  interrupts.CPU51.LOC:Local_timer_interrupts
   2394615 ±  5%     +12.6%    2695289 ±  4%  interrupts.CPU51.RES:Rescheduling_interrupts
  11720117 ± 14%     -50.6%    5785902 ± 24%  interrupts.CPU52.CAL:Function_call_interrupts
   3068577 ±  2%     -33.0%    2056563 ±  3%  interrupts.CPU52.LOC:Local_timer_interrupts
   2253641 ±  6%     +27.7%    2878787 ±  8%  interrupts.CPU52.RES:Rescheduling_interrupts
  10368352 ± 19%     -49.2%    5265238 ± 31%  interrupts.CPU53.CAL:Function_call_interrupts
   3068506 ±  2%     -33.0%    2056579 ±  3%  interrupts.CPU53.LOC:Local_timer_interrupts
   3068443 ±  2%     -33.0%    2056580 ±  3%  interrupts.CPU54.LOC:Local_timer_interrupts
  11142575 ± 15%     -40.7%    6611607 ± 34%  interrupts.CPU55.CAL:Function_call_interrupts
   3068447 ±  2%     -33.0%    2056593 ±  3%  interrupts.CPU55.LOC:Local_timer_interrupts
  10507699 ± 21%     -40.8%    6221308 ± 25%  interrupts.CPU56.CAL:Function_call_interrupts
   3068546 ±  2%     -33.0%    2056569 ±  3%  interrupts.CPU56.LOC:Local_timer_interrupts
  11506471 ± 17%     -45.7%    6247067 ± 38%  interrupts.CPU57.CAL:Function_call_interrupts
   3068520 ±  2%     -33.0%    2056614 ±  3%  interrupts.CPU57.LOC:Local_timer_interrupts
  10149739 ± 20%     -39.9%    6095837 ± 23%  interrupts.CPU58.CAL:Function_call_interrupts
   3068431 ±  2%     -33.0%    2056573 ±  3%  interrupts.CPU58.LOC:Local_timer_interrupts
   3068377 ±  2%     -33.0%    2056566 ±  3%  interrupts.CPU59.LOC:Local_timer_interrupts
   3068862 ±  2%     -33.0%    2056994 ±  3%  interrupts.CPU6.LOC:Local_timer_interrupts
   3068645 ±  2%     -33.0%    2056612 ±  3%  interrupts.CPU60.LOC:Local_timer_interrupts
   9349585 ±  9%     -33.4%    6228620 ± 34%  interrupts.CPU61.CAL:Function_call_interrupts
   3068468 ±  2%     -33.0%    2056547 ±  3%  interrupts.CPU61.LOC:Local_timer_interrupts
  10179611 ± 22%     -45.1%    5589466 ± 24%  interrupts.CPU62.CAL:Function_call_interrupts
   3068405 ±  2%     -33.0%    2056589 ±  3%  interrupts.CPU62.LOC:Local_timer_interrupts
   9645667 ± 23%     -42.5%    5546114 ± 24%  interrupts.CPU63.CAL:Function_call_interrupts
   3068614 ±  2%     -33.0%    2056621 ±  3%  interrupts.CPU63.LOC:Local_timer_interrupts
   2437604 ± 14%     +20.8%    2943790 ±  9%  interrupts.CPU63.RES:Rescheduling_interrupts
   3068509 ±  2%     -33.0%    2056525 ±  3%  interrupts.CPU64.LOC:Local_timer_interrupts
   2251691 ±  6%     +29.9%    2925998 ± 13%  interrupts.CPU64.RES:Rescheduling_interrupts
   3068329 ±  2%     -33.0%    2056632 ±  3%  interrupts.CPU65.LOC:Local_timer_interrupts
   3068522 ±  2%     -33.0%    2056593 ±  3%  interrupts.CPU66.LOC:Local_timer_interrupts
  10646786 ± 21%     -41.8%    6197071 ± 33%  interrupts.CPU67.CAL:Function_call_interrupts
   3068457 ±  2%     -33.0%    2056518 ±  3%  interrupts.CPU67.LOC:Local_timer_interrupts
  10774555 ± 35%     -44.0%    6033539 ± 40%  interrupts.CPU68.CAL:Function_call_interrupts
   3068410 ±  2%     -33.0%    2056612 ±  3%  interrupts.CPU68.LOC:Local_timer_interrupts
   2318101 ± 11%     +27.2%    2947490 ±  9%  interrupts.CPU68.RES:Rescheduling_interrupts
  10792638 ± 23%     -41.9%    6272497 ± 25%  interrupts.CPU69.CAL:Function_call_interrupts
   3068572 ±  2%     -33.0%    2056539 ±  3%  interrupts.CPU69.LOC:Local_timer_interrupts
   2308398 ± 12%     +30.7%    3017171 ± 10%  interrupts.CPU69.RES:Rescheduling_interrupts
   3068931 ±  2%     -33.0%    2057070 ±  3%  interrupts.CPU7.LOC:Local_timer_interrupts
  10843214 ± 28%     -50.0%    5417943 ± 23%  interrupts.CPU70.CAL:Function_call_interrupts
   3068472 ±  2%     -33.0%    2056640 ±  3%  interrupts.CPU70.LOC:Local_timer_interrupts
   9594747 ± 20%     -38.5%    5903738 ± 32%  interrupts.CPU71.CAL:Function_call_interrupts
   3068422 ±  2%     -33.0%    2056599 ±  3%  interrupts.CPU71.LOC:Local_timer_interrupts
   3068473 ±  2%     -33.0%    2056592 ±  3%  interrupts.CPU72.LOC:Local_timer_interrupts
   1474907 ± 14%     +30.4%    1923058 ±  8%  interrupts.CPU72.RES:Rescheduling_interrupts
   3068248 ±  2%     -33.0%    2056570 ±  3%  interrupts.CPU73.LOC:Local_timer_interrupts
  10473721 ± 30%     -44.1%    5855397 ± 29%  interrupts.CPU74.CAL:Function_call_interrupts
   3068397 ±  2%     -33.0%    2056552 ±  3%  interrupts.CPU74.LOC:Local_timer_interrupts
   2372716 ± 11%     +25.9%    2986766 ±  8%  interrupts.CPU74.RES:Rescheduling_interrupts
   3068444 ±  2%     -33.0%    2056444 ±  3%  interrupts.CPU75.LOC:Local_timer_interrupts
  11003452 ± 27%     -53.5%    5111244 ± 29%  interrupts.CPU76.CAL:Function_call_interrupts
   3068425 ±  2%     -33.0%    2056621 ±  3%  interrupts.CPU76.LOC:Local_timer_interrupts
   2395091 ±  3%     +23.6%    2961177 ±  5%  interrupts.CPU76.RES:Rescheduling_interrupts
  10304316 ± 22%     -47.7%    5391627 ± 37%  interrupts.CPU77.CAL:Function_call_interrupts
   3068453 ±  2%     -33.0%    2056581 ±  3%  interrupts.CPU77.LOC:Local_timer_interrupts
   3068435 ±  2%     -33.0%    2056550 ±  3%  interrupts.CPU78.LOC:Local_timer_interrupts
   3068478 ±  2%     -33.0%    2056579 ±  3%  interrupts.CPU79.LOC:Local_timer_interrupts
   3068902 ±  2%     -33.0%    2057100 ±  3%  interrupts.CPU8.LOC:Local_timer_interrupts
   9206669 ± 19%     -31.1%    6341522 ± 30%  interrupts.CPU80.CAL:Function_call_interrupts
   3068424 ±  2%     -33.0%    2056560 ±  3%  interrupts.CPU80.LOC:Local_timer_interrupts
   3068415 ±  2%     -33.0%    2056524 ±  3%  interrupts.CPU81.LOC:Local_timer_interrupts
   2256903 ±  7%     +35.1%    3048124 ±  9%  interrupts.CPU81.RES:Rescheduling_interrupts
  10279497 ± 22%     -40.6%    6102509 ± 32%  interrupts.CPU82.CAL:Function_call_interrupts
   3068481 ±  2%     -33.0%    2056568 ±  3%  interrupts.CPU82.LOC:Local_timer_interrupts
   2299462 ± 13%     +33.1%    3059500 ± 16%  interrupts.CPU82.RES:Rescheduling_interrupts
   3068382 ±  2%     -33.0%    2056553 ±  3%  interrupts.CPU83.LOC:Local_timer_interrupts
   2334299 ± 15%     +38.1%    3223620 ±  8%  interrupts.CPU83.RES:Rescheduling_interrupts
   3068415 ±  2%     -33.0%    2056565 ±  3%  interrupts.CPU84.LOC:Local_timer_interrupts
   3068289 ±  2%     -33.0%    2056562 ±  3%  interrupts.CPU85.LOC:Local_timer_interrupts
   3068338 ±  2%     -33.0%    2056564 ±  3%  interrupts.CPU86.LOC:Local_timer_interrupts
   9737271 ± 21%     -37.1%    6127891 ± 32%  interrupts.CPU87.CAL:Function_call_interrupts
   3068239 ±  2%     -33.0%    2056566 ±  3%  interrupts.CPU87.LOC:Local_timer_interrupts
   2450244 ± 12%     +19.3%    2923936 ±  7%  interrupts.CPU87.RES:Rescheduling_interrupts
   3068293 ±  2%     -33.0%    2056580 ±  3%  interrupts.CPU88.LOC:Local_timer_interrupts
   3068428 ±  2%     -33.0%    2056514 ±  3%  interrupts.CPU89.LOC:Local_timer_interrupts
   3068866 ±  2%     -33.0%    2057208 ±  3%  interrupts.CPU9.LOC:Local_timer_interrupts
  10771708 ± 22%     -40.4%    6418307 ± 41%  interrupts.CPU90.CAL:Function_call_interrupts
   3068285 ±  2%     -33.0%    2056556 ±  3%  interrupts.CPU90.LOC:Local_timer_interrupts
   3068278 ±  2%     -33.0%    2056556 ±  3%  interrupts.CPU91.LOC:Local_timer_interrupts
  11162745 ± 21%     -48.6%    5740747 ± 33%  interrupts.CPU92.CAL:Function_call_interrupts
   3068462 ±  2%     -33.0%    2056557 ±  3%  interrupts.CPU92.LOC:Local_timer_interrupts
   3068440 ±  2%     -33.0%    2056573 ±  3%  interrupts.CPU93.LOC:Local_timer_interrupts
   2384662 ±  6%     +27.4%    3038680 ± 11%  interrupts.CPU93.RES:Rescheduling_interrupts
  11224988 ± 21%     -50.3%    5582800 ± 29%  interrupts.CPU94.CAL:Function_call_interrupts
   3068420 ±  2%     -33.0%    2056571 ±  3%  interrupts.CPU94.LOC:Local_timer_interrupts
   2166238 ±  8%     +34.8%    2920149 ±  9%  interrupts.CPU94.RES:Rescheduling_interrupts
   3068516 ±  2%     -33.0%    2056621 ±  3%  interrupts.CPU95.LOC:Local_timer_interrupts
   3068777 ±  2%     -33.0%    2057005 ±  3%  interrupts.CPU96.LOC:Local_timer_interrupts
   1558944 ± 11%     +42.2%    2217344 ± 14%  interrupts.CPU96.RES:Rescheduling_interrupts
   3068840 ±  2%     -33.0%    2057051 ±  3%  interrupts.CPU97.LOC:Local_timer_interrupts
   3068904 ±  2%     -33.0%    2057003 ±  3%  interrupts.CPU98.LOC:Local_timer_interrupts
   3068839 ±  2%     -33.0%    2057016 ±  3%  interrupts.CPU99.LOC:Local_timer_interrupts
 5.892e+08 ±  2%     -33.0%  3.949e+08 ±  3%  interrupts.LOC:Local_timer_interrupts
    768.00           -25.0%     576.00        interrupts.MCP:Machine_check_polls
 4.829e+08 ±  2%     +17.6%  5.679e+08 ±  2%  interrupts.RES:Rescheduling_interrupts
    193152 ±  9%     -27.0%     141046 ± 10%  softirqs.CPU0.RCU
     20466 ±  5%     -19.9%      16386 ±  9%  softirqs.CPU0.SCHED
    198795 ±  8%     -27.6%     143897 ± 11%  softirqs.CPU1.RCU
    199997 ±  6%     -27.7%     144536 ± 10%  softirqs.CPU10.RCU
     16821           -23.2%      12913 ± 11%  softirqs.CPU10.SCHED
    201971 ±  6%     -27.8%     145893 ±  9%  softirqs.CPU100.RCU
    203243 ±  6%     -30.3%     141724 ± 11%  softirqs.CPU101.RCU
     16916 ±  3%     -23.6%      12922 ± 15%  softirqs.CPU101.SCHED
    202568 ±  6%     -28.1%     145584 ±  9%  softirqs.CPU102.RCU
     16083 ±  3%     -20.3%      12823 ± 11%  softirqs.CPU102.SCHED
    202688 ±  7%     -28.3%     145381 ± 10%  softirqs.CPU103.RCU
    201501 ±  6%     -28.4%     144361 ± 10%  softirqs.CPU104.RCU
     16532 ±  3%     -23.4%      12665 ± 11%  softirqs.CPU104.SCHED
    199791 ±  6%     -28.2%     143353 ± 10%  softirqs.CPU105.RCU
    201042 ±  6%     -28.3%     144152 ± 10%  softirqs.CPU106.RCU
    202312 ±  6%     -28.1%     145480 ± 10%  softirqs.CPU107.RCU
     16444 ±  3%     -23.2%      12634 ± 13%  softirqs.CPU107.SCHED
    203312 ±  7%     -28.5%     145443 ± 10%  softirqs.CPU108.RCU
     16406 ±  2%     -22.8%      12662 ± 13%  softirqs.CPU108.SCHED
    202511 ±  7%     -28.7%     144477 ± 10%  softirqs.CPU109.RCU
    201374 ±  6%     -28.2%     144674 ± 10%  softirqs.CPU11.RCU
     16619 ±  3%     -23.1%      12773 ± 11%  softirqs.CPU11.SCHED
    195860 ±  3%     -26.3%     144303 ±  9%  softirqs.CPU110.RCU
     16435 ±  3%     -21.9%      12831 ± 12%  softirqs.CPU110.SCHED
    200642 ±  6%     -28.5%     143450 ± 10%  softirqs.CPU111.RCU
     16380 ±  3%     -23.6%      12510 ± 11%  softirqs.CPU111.SCHED
    211528 ±  6%     -28.9%     150296 ±  8%  softirqs.CPU112.RCU
     16474 ±  3%     -22.3%      12807 ± 11%  softirqs.CPU112.SCHED
    204714 ±  9%     -26.6%     150278 ±  8%  softirqs.CPU113.RCU
    211999 ±  6%     -28.9%     150629 ±  9%  softirqs.CPU114.RCU
    206050 ±  9%     -26.8%     150897 ±  8%  softirqs.CPU115.RCU
     16399 ±  3%     -23.5%      12552 ± 11%  softirqs.CPU115.SCHED
    205829 ± 10%     -26.7%     150890 ±  8%  softirqs.CPU116.RCU
    211008 ±  6%     -28.6%     150563 ±  9%  softirqs.CPU117.RCU
    211452 ±  6%     -29.1%     149985 ±  9%  softirqs.CPU118.RCU
     16363 ±  2%     -21.9%      12772 ± 12%  softirqs.CPU118.SCHED
    211001 ±  6%     -28.8%     150251 ±  9%  softirqs.CPU119.RCU
     16215 ±  3%     -21.6%      12715 ± 11%  softirqs.CPU119.SCHED
    201592 ±  6%     -27.9%     145412 ± 10%  softirqs.CPU12.RCU
    214923 ±  5%     -28.9%     152762 ±  9%  softirqs.CPU120.RCU
     21496 ±  5%     -22.8%      16594 ± 15%  softirqs.CPU120.SCHED
    216154 ±  6%     -29.0%     153419 ±  9%  softirqs.CPU121.RCU
     23229 ±  8%     -22.2%      18076 ± 20%  softirqs.CPU121.SCHED
    214137 ±  6%     -28.7%     152664 ±  9%  softirqs.CPU122.RCU
     23979 ±  6%     -29.8%      16833 ± 14%  softirqs.CPU122.SCHED
    214695 ±  6%     -28.9%     152625 ±  9%  softirqs.CPU123.RCU
     23136 ±  6%     -25.7%      17189 ± 17%  softirqs.CPU123.SCHED
    214970 ±  6%     -29.0%     152602 ±  9%  softirqs.CPU124.RCU
    214197 ±  6%     -29.0%     152008 ±  9%  softirqs.CPU125.RCU
     23064 ±  7%     -22.0%      17979 ± 19%  softirqs.CPU125.SCHED
    214136 ±  6%     -28.9%     152201 ±  9%  softirqs.CPU126.RCU
    214555 ±  5%     -29.0%     152287 ±  9%  softirqs.CPU127.RCU
     22979 ±  6%     -25.8%      17052 ± 15%  softirqs.CPU127.SCHED
    209028 ±  6%     -28.9%     148635 ± 10%  softirqs.CPU128.RCU
    208002 ±  6%     -28.8%     148083 ±  9%  softirqs.CPU129.RCU
     23928 ± 11%     -27.1%      17436 ± 16%  softirqs.CPU129.SCHED
    202575 ±  6%     -28.5%     144932 ± 10%  softirqs.CPU13.RCU
    209069 ±  6%     -29.2%     148034 ±  9%  softirqs.CPU130.RCU
     23435 ±  7%     -28.9%      16657 ± 16%  softirqs.CPU130.SCHED
    207986 ±  6%     -29.1%     147499 ±  9%  softirqs.CPU131.RCU
     23531 ±  8%     -24.7%      17709 ± 20%  softirqs.CPU131.SCHED
    208157 ±  6%     -29.1%     147628 ±  9%  softirqs.CPU132.RCU
     23491 ±  3%     -25.2%      17567 ± 20%  softirqs.CPU132.SCHED
    209272 ±  6%     -28.8%     148934 ±  9%  softirqs.CPU133.RCU
     23297 ±  7%     -25.1%      17460 ± 16%  softirqs.CPU133.SCHED
    209483 ±  6%     -28.9%     148915 ±  9%  softirqs.CPU134.RCU
     23150 ±  8%     -24.6%      17465 ± 17%  softirqs.CPU134.SCHED
    208558 ±  6%     -30.7%     144575 ± 12%  softirqs.CPU135.RCU
     23637 ± 11%     -25.3%      17667 ± 18%  softirqs.CPU135.SCHED
    208527 ±  6%     -28.7%     148690 ± 10%  softirqs.CPU136.RCU
     23160 ±  6%     -26.6%      17008 ± 15%  softirqs.CPU136.SCHED
    208023 ±  5%     -28.8%     148204 ± 10%  softirqs.CPU137.RCU
     23032 ±  5%     -25.5%      17159 ± 17%  softirqs.CPU137.SCHED
    207076 ±  6%     -28.6%     147817 ±  9%  softirqs.CPU138.RCU
     22784 ±  7%     -23.3%      17468 ± 17%  softirqs.CPU138.SCHED
    209164 ±  6%     -28.8%     148824 ±  9%  softirqs.CPU139.RCU
     24161 ± 10%     -21.8%      18898 ± 20%  softirqs.CPU139.SCHED
    196930 ±  4%     -27.0%     143744 ± 10%  softirqs.CPU14.RCU
     16312 ±  2%     -22.1%      12708 ± 12%  softirqs.CPU14.SCHED
    208960 ±  6%     -29.0%     148425 ±  9%  softirqs.CPU140.RCU
     23188 ±  9%     -24.5%      17507 ± 17%  softirqs.CPU140.SCHED
    207359 ±  6%     -28.7%     147876 ±  9%  softirqs.CPU141.RCU
     23048 ± 10%     -24.6%      17386 ± 16%  softirqs.CPU141.SCHED
    207289 ±  6%     -28.7%     147752 ±  9%  softirqs.CPU142.RCU
     24215 ±  9%     -30.2%      16912 ± 14%  softirqs.CPU142.SCHED
    207500 ±  6%     -29.2%     146807 ± 10%  softirqs.CPU143.RCU
     24472 ± 12%     -26.8%      17911 ± 17%  softirqs.CPU143.SCHED
    206965 ±  6%     -27.9%     149195 ±  9%  softirqs.CPU144.RCU
     21705 ± 10%     -18.8%      17632 ±  8%  softirqs.CPU144.SCHED
    207306 ±  6%     -28.4%     148376 ±  9%  softirqs.CPU145.RCU
     22890 ±  3%     -17.9%      18789 ± 11%  softirqs.CPU145.SCHED
    206457 ±  6%     -28.3%     147988 ±  9%  softirqs.CPU146.RCU
     23549 ±  6%     -21.3%      18533 ± 11%  softirqs.CPU146.SCHED
    206651 ±  6%     -28.2%     148428 ±  9%  softirqs.CPU147.RCU
     24170 ±  5%     -23.1%      18583 ±  9%  softirqs.CPU147.SCHED
    206659 ±  6%     -28.6%     147586 ±  9%  softirqs.CPU148.RCU
     23554 ±  3%     -22.2%      18317 ±  6%  softirqs.CPU148.SCHED
    205141 ±  6%     -28.2%     147197 ±  9%  softirqs.CPU149.RCU
     23640 ±  6%     -23.1%      18177 ±  9%  softirqs.CPU149.SCHED
    199223 ±  6%     -29.6%     140263 ± 12%  softirqs.CPU15.RCU
     16350 ±  2%     -24.3%      12381 ±  8%  softirqs.CPU15.SCHED
    205951 ±  6%     -28.4%     147445 ±  9%  softirqs.CPU150.RCU
     23317 ±  8%     -18.7%      18965 ± 13%  softirqs.CPU150.SCHED
    207287 ±  6%     -27.9%     149366 ±  9%  softirqs.CPU151.RCU
     24764 ±  5%     -28.4%      17742 ± 10%  softirqs.CPU151.SCHED
    206844 ±  6%     -28.5%     147853 ±  9%  softirqs.CPU152.RCU
     23983 ±  5%     -23.5%      18358 ±  9%  softirqs.CPU152.SCHED
    205850 ±  6%     -28.4%     147337 ±  9%  softirqs.CPU153.RCU
     24407 ±  4%     -24.1%      18528 ±  8%  softirqs.CPU153.SCHED
    205499 ±  6%     -28.3%     147405 ±  9%  softirqs.CPU154.RCU
     24013 ±  8%     -20.9%      18990 ±  9%  softirqs.CPU154.SCHED
    205700 ±  6%     -28.2%     147610 ±  9%  softirqs.CPU155.RCU
     23743 ±  3%     -18.4%      19368 ±  8%  softirqs.CPU155.SCHED
    207376 ±  6%     -28.1%     149119 ±  9%  softirqs.CPU156.RCU
     24700 ±  3%     -23.5%      18889 ±  9%  softirqs.CPU156.SCHED
    207621 ±  6%     -28.5%     148425 ±  9%  softirqs.CPU157.RCU
     24326 ±  6%     -21.3%      19154 ± 12%  softirqs.CPU157.SCHED
    206327 ±  6%     -28.2%     148212 ±  9%  softirqs.CPU158.RCU
     23142 ±  8%     -16.4%      19341 ± 11%  softirqs.CPU158.SCHED
    205910 ±  6%     -28.5%     147251 ±  9%  softirqs.CPU159.RCU
     22984 ±  4%     -16.4%      19217 ±  7%  softirqs.CPU159.SCHED
    210167 ±  6%     -28.1%     151069 ±  9%  softirqs.CPU16.RCU
     16472 ±  3%     -23.2%      12658 ± 10%  softirqs.CPU16.SCHED
    216218 ±  6%     -29.0%     153586 ±  9%  softirqs.CPU160.RCU
     24504 ±  7%     -21.6%      19220 ±  6%  softirqs.CPU160.SCHED
    216579 ±  6%     -28.9%     154091 ±  9%  softirqs.CPU161.RCU
     24635 ±  4%     -26.6%      18091 ±  7%  softirqs.CPU161.SCHED
    216831 ±  6%     -28.7%     154593 ±  9%  softirqs.CPU162.RCU
     24022 ±  5%     -23.3%      18418 ± 11%  softirqs.CPU162.SCHED
    217526 ±  6%     -28.7%     155182 ±  9%  softirqs.CPU163.RCU
     24587 ±  4%     -23.8%      18726 ± 11%  softirqs.CPU163.SCHED
    216490 ±  6%     -28.7%     154426 ±  9%  softirqs.CPU164.RCU
     22371 ±  5%     -19.7%      17971 ± 11%  softirqs.CPU164.SCHED
    216040 ±  6%     -28.8%     153740 ±  9%  softirqs.CPU165.RCU
     24296 ±  7%     -23.9%      18477 ±  9%  softirqs.CPU165.SCHED
    216407 ±  6%     -28.8%     154003 ±  9%  softirqs.CPU166.RCU
     23290 ±  4%     -23.5%      17808 ± 10%  softirqs.CPU166.SCHED
    216708 ±  6%     -29.0%     153826 ±  9%  softirqs.CPU167.RCU
     23006 ±  6%     -23.1%      17685 ± 11%  softirqs.CPU167.SCHED
    206126 ±  6%     -28.2%     148036 ± 10%  softirqs.CPU168.RCU
     23048 ± 12%     -27.3%      16762 ±  3%  softirqs.CPU168.SCHED
    206232 ±  6%     -28.8%     146881 ±  9%  softirqs.CPU169.RCU
     22923 ±  8%     -17.2%      18971 ±  7%  softirqs.CPU169.SCHED
    209777 ±  6%     -27.7%     151673 ±  9%  softirqs.CPU17.RCU
    205532 ±  6%     -28.7%     146562 ±  9%  softirqs.CPU170.RCU
     24167 ±  9%     -23.1%      18573 ± 12%  softirqs.CPU170.SCHED
    205693 ±  6%     -28.9%     146334 ±  9%  softirqs.CPU171.RCU
     23912 ± 15%     -24.1%      18158 ±  7%  softirqs.CPU171.SCHED
    205799 ±  6%     -28.8%     146575 ±  9%  softirqs.CPU172.RCU
     23413 ±  9%     -23.0%      18026 ± 10%  softirqs.CPU172.SCHED
    206142 ±  6%     -28.8%     146788 ±  9%  softirqs.CPU173.RCU
     23375 ±  9%     -21.0%      18472 ±  4%  softirqs.CPU173.SCHED
    206185 ±  6%     -28.8%     146701 ±  9%  softirqs.CPU174.RCU
     24200 ± 11%     -23.0%      18639 ±  6%  softirqs.CPU174.SCHED
    205957 ±  6%     -28.6%     147038 ±  9%  softirqs.CPU175.RCU
     24269 ±  8%     -24.5%      18330 ± 10%  softirqs.CPU175.SCHED
    202506 ±  6%     -28.5%     144871 ±  9%  softirqs.CPU176.RCU
     23579 ±  7%     -19.4%      19008 ±  8%  softirqs.CPU176.SCHED
    203330 ±  6%     -28.1%     146103 ±  9%  softirqs.CPU177.RCU
     23615 ±  9%     -20.8%      18712 ±  7%  softirqs.CPU177.SCHED
    204672 ±  6%     -28.0%     147343 ±  9%  softirqs.CPU178.RCU
     23916 ±  9%     -22.9%      18429 ±  7%  softirqs.CPU178.SCHED
    204454 ±  6%     -28.3%     146649 ±  9%  softirqs.CPU179.RCU
     23892 ± 11%     -19.0%      19352 ± 10%  softirqs.CPU179.SCHED
    208829 ±  6%     -28.1%     150229 ±  9%  softirqs.CPU18.RCU
     16298 ±  2%     -20.9%      12891 ± 14%  softirqs.CPU18.SCHED
    203207 ±  6%     -28.4%     145583 ±  9%  softirqs.CPU180.RCU
     24035 ± 10%     -20.6%      19074 ± 10%  softirqs.CPU180.SCHED
    202998 ±  6%     -28.6%     144903 ±  9%  softirqs.CPU181.RCU
    202865 ±  6%     -28.3%     145452 ±  9%  softirqs.CPU182.RCU
    204074 ±  6%     -28.3%     146359 ±  9%  softirqs.CPU183.RCU
     24052 ±  9%     -21.9%      18788 ±  8%  softirqs.CPU183.SCHED
    204385 ±  6%     -28.7%     145695 ±  9%  softirqs.CPU184.RCU
     24065 ±  6%     -21.3%      18951 ± 10%  softirqs.CPU184.SCHED
    204303 ±  6%     -28.3%     146551 ±  9%  softirqs.CPU185.RCU
     24618 ±  9%     -21.5%      19320 ±  7%  softirqs.CPU185.SCHED
    202553 ±  6%     -28.4%     144931 ±  9%  softirqs.CPU186.RCU
     22909 ±  8%     -21.0%      18106 ± 12%  softirqs.CPU186.SCHED
    203676 ±  6%     -28.4%     145832 ±  9%  softirqs.CPU187.RCU
     24933 ± 15%     -26.2%      18407 ± 14%  softirqs.CPU187.SCHED
    202625 ±  6%     -28.3%     145229 ±  9%  softirqs.CPU188.RCU
     24220 ± 11%     -24.2%      18369 ±  7%  softirqs.CPU188.SCHED
    203753 ±  6%     -28.5%     145671 ±  9%  softirqs.CPU189.RCU
     23515 ± 13%     -18.8%      19083 ±  8%  softirqs.CPU189.SCHED
    210120 ±  6%     -27.4%     152454 ±  9%  softirqs.CPU19.RCU
     16227 ±  2%     -21.1%      12804 ± 13%  softirqs.CPU19.SCHED
    203955 ±  6%     -28.1%     146583 ±  9%  softirqs.CPU190.RCU
     23566 ±  7%     -17.8%      19369 ±  7%  softirqs.CPU190.SCHED
    203773 ±  6%     -28.8%     145160 ±  9%  softirqs.CPU191.RCU
     24056 ±  8%     -21.6%      18849 ±  6%  softirqs.CPU191.SCHED
    198967 ±  6%     -28.1%     143092 ± 10%  softirqs.CPU2.RCU
     17293 ±  4%     -19.6%      13907 ± 11%  softirqs.CPU2.SCHED
    209889 ±  6%     -27.6%     151960 ±  9%  softirqs.CPU20.RCU
    209219 ±  6%     -27.8%     151011 ±  9%  softirqs.CPU21.RCU
    209780 ±  6%     -28.1%     150848 ±  9%  softirqs.CPU22.RCU
    209554 ±  6%     -30.1%     146396 ±  3%  softirqs.CPU23.RCU
    217072 ±  5%     -29.1%     153886 ±  9%  softirqs.CPU24.RCU
     21938 ±  6%     -24.2%      16630 ± 16%  softirqs.CPU24.SCHED
    218118 ±  6%     -29.5%     153831 ±  9%  softirqs.CPU25.RCU
     23877 ± 11%     -29.0%      16943 ± 20%  softirqs.CPU25.SCHED
    216947 ±  6%     -29.1%     153895 ±  9%  softirqs.CPU26.RCU
     23484 ±  7%     -26.3%      17304 ± 15%  softirqs.CPU26.SCHED
    216514 ±  6%     -29.5%     152609 ±  9%  softirqs.CPU27.RCU
     23146 ±  7%     -26.9%      16915 ± 16%  softirqs.CPU27.SCHED
    215799 ±  5%     -29.3%     152597 ±  9%  softirqs.CPU28.RCU
     23394 ± 10%     -25.4%      17451 ± 17%  softirqs.CPU28.SCHED
    215717 ±  6%     -29.0%     153205 ±  9%  softirqs.CPU29.RCU
     23278 ±  8%     -24.8%      17496 ± 17%  softirqs.CPU29.SCHED
    202354 ±  6%     -29.4%     142794 ± 11%  softirqs.CPU3.RCU
     16384 ±  4%     -20.7%      12989 ± 11%  softirqs.CPU3.SCHED
    215070 ±  6%     -29.2%     152337 ±  9%  softirqs.CPU30.RCU
    217099 ±  6%     -29.1%     153870 ±  9%  softirqs.CPU31.RCU
     23577 ± 12%     -27.8%      17019 ± 16%  softirqs.CPU31.SCHED
    208449 ±  6%     -28.9%     148126 ± 10%  softirqs.CPU32.RCU
     22628 ± 13%     -22.5%      17545 ± 16%  softirqs.CPU32.SCHED
    208025 ±  6%     -29.1%     147574 ±  9%  softirqs.CPU33.RCU
     23533 ± 13%     -27.6%      17042 ± 16%  softirqs.CPU33.SCHED
    208432 ±  6%     -29.0%     148018 ± 10%  softirqs.CPU34.RCU
     23026 ±  5%     -27.3%      16737 ± 14%  softirqs.CPU34.SCHED
    207774 ±  6%     -28.9%     147669 ±  9%  softirqs.CPU35.RCU
     24112 ± 13%     -25.3%      18016 ± 20%  softirqs.CPU35.SCHED
    208130 ±  6%     -29.2%     147338 ±  9%  softirqs.CPU36.RCU
     22962 ±  7%     -25.1%      17190 ± 17%  softirqs.CPU36.SCHED
    209296 ±  6%     -29.1%     148353 ±  9%  softirqs.CPU37.RCU
     23993 ±  8%     -26.0%      17746 ± 16%  softirqs.CPU37.SCHED
    208823 ±  6%     -28.5%     149327 ± 10%  softirqs.CPU38.RCU
     23357 ± 10%     -22.0%      18228 ± 17%  softirqs.CPU38.SCHED
    209460 ±  6%     -30.2%     146293 ± 11%  softirqs.CPU39.RCU
    202232 ±  6%     -28.6%     144389 ± 10%  softirqs.CPU4.RCU
    208879 ±  6%     -29.1%     148189 ± 10%  softirqs.CPU40.RCU
     23097 ±  7%     -26.4%      16990 ± 14%  softirqs.CPU40.SCHED
    209006 ±  6%     -29.3%     147795 ±  9%  softirqs.CPU41.RCU
     23867 ±  9%     -28.3%      17118 ± 18%  softirqs.CPU41.SCHED
    207817 ±  6%     -29.2%     147220 ± 10%  softirqs.CPU42.RCU
     23410 ±  8%     -27.4%      16998 ± 16%  softirqs.CPU42.SCHED
    209588 ±  6%     -29.0%     148790 ± 10%  softirqs.CPU43.RCU
     24198 ± 13%     -24.5%      18280 ± 19%  softirqs.CPU43.SCHED
    207943 ±  6%     -28.8%     148055 ± 10%  softirqs.CPU44.RCU
     23126 ±  8%     -26.4%      17012 ± 14%  softirqs.CPU44.SCHED
    207889 ±  6%     -28.8%     147933 ± 10%  softirqs.CPU45.RCU
     23334 ±  9%     -25.5%      17380 ± 16%  softirqs.CPU45.SCHED
    207374 ±  6%     -29.0%     147309 ± 10%  softirqs.CPU46.RCU
     23398 ±  8%     -27.2%      17039 ± 16%  softirqs.CPU46.SCHED
    206686 ±  6%     -29.1%     146562 ± 10%  softirqs.CPU47.RCU
     23642 ±  8%     -25.4%      17628 ± 18%  softirqs.CPU47.SCHED
    206436 ±  6%     -28.6%     147323 ±  9%  softirqs.CPU48.RCU
     22261 ±  7%     -23.3%      17082 ±  8%  softirqs.CPU48.SCHED
    206917 ±  6%     -28.7%     147434 ±  9%  softirqs.CPU49.RCU
     22614 ±  2%     -18.9%      18331 ± 10%  softirqs.CPU49.SCHED
    202825 ±  7%     -29.3%     143367 ± 10%  softirqs.CPU5.RCU
     16444 ±  2%     -22.7%      12711 ± 11%  softirqs.CPU5.SCHED
    205598 ±  6%     -28.4%     147197 ±  9%  softirqs.CPU50.RCU
     23126 ±  3%     -20.3%      18430 ± 12%  softirqs.CPU50.SCHED
    205405 ±  6%     -28.5%     146785 ±  9%  softirqs.CPU51.RCU
     23817 ±  5%     -22.0%      18577 ±  7%  softirqs.CPU51.SCHED
    204054 ±  6%     -28.2%     146499 ±  9%  softirqs.CPU52.RCU
     23330 ±  3%     -21.4%      18343 ±  8%  softirqs.CPU52.SCHED
    206074 ±  6%     -28.2%     147912 ±  9%  softirqs.CPU53.RCU
     24062 ±  4%     -25.7%      17875 ±  6%  softirqs.CPU53.SCHED
    205523 ±  6%     -28.5%     146965 ±  9%  softirqs.CPU54.RCU
     23084 ±  8%     -20.1%      18443 ± 10%  softirqs.CPU54.SCHED
    206208 ±  6%     -28.7%     147108 ±  9%  softirqs.CPU55.RCU
     24389 ±  7%     -28.9%      17342 ± 10%  softirqs.CPU55.SCHED
    205438 ±  6%     -28.3%     147389 ±  9%  softirqs.CPU56.RCU
     23920 ±  4%     -23.5%      18287 ± 10%  softirqs.CPU56.SCHED
    205279 ±  6%     -28.5%     146806 ±  9%  softirqs.CPU57.RCU
     23767 ±  4%     -22.0%      18536 ± 10%  softirqs.CPU57.SCHED
    206739 ±  6%     -28.4%     148086 ±  9%  softirqs.CPU58.RCU
     23688 ± 10%     -20.3%      18881 ±  8%  softirqs.CPU58.SCHED
    205733 ±  6%     -28.3%     147452 ±  9%  softirqs.CPU59.RCU
     23463 ±  7%     -18.0%      19250 ±  9%  softirqs.CPU59.SCHED
    201813 ±  6%     -28.1%     145159 ± 10%  softirqs.CPU6.RCU
    206398 ±  6%     -28.5%     147496 ±  9%  softirqs.CPU60.RCU
     23978 ±  4%     -20.7%      19014 ±  8%  softirqs.CPU60.SCHED
    206583 ±  6%     -28.3%     148071 ±  9%  softirqs.CPU61.RCU
     23640 ±  7%     -16.0%      19853 ±  9%  softirqs.CPU61.SCHED
    205225 ±  6%     -28.2%     147391 ±  9%  softirqs.CPU62.RCU
     22642 ±  8%     -16.4%      18938 ±  8%  softirqs.CPU62.SCHED
    205633 ±  6%     -28.3%     147520 ±  9%  softirqs.CPU63.RCU
     23103 ±  4%     -17.6%      19045 ±  5%  softirqs.CPU63.SCHED
    214687 ±  6%     -28.6%     153332 ±  9%  softirqs.CPU64.RCU
     23454 ±  8%     -18.5%      19116 ±  5%  softirqs.CPU64.SCHED
    215835 ±  6%     -28.9%     153451 ±  8%  softirqs.CPU65.RCU
     24143 ±  7%     -25.8%      17908 ±  9%  softirqs.CPU65.SCHED
    215972 ±  6%     -28.9%     153606 ±  9%  softirqs.CPU66.RCU
     23860 ±  6%     -23.2%      18336 ±  9%  softirqs.CPU66.SCHED
    215406 ±  6%     -28.5%     154006 ±  9%  softirqs.CPU67.RCU
     22980 ±  3%     -19.1%      18589 ±  9%  softirqs.CPU67.SCHED
    214782 ±  6%     -28.8%     152916 ±  9%  softirqs.CPU68.RCU
     23292 ±  4%     -21.2%      18361 ± 12%  softirqs.CPU68.SCHED
    214593 ±  6%     -28.7%     152948 ±  9%  softirqs.CPU69.RCU
     23892 ±  5%     -22.9%      18425 ± 11%  softirqs.CPU69.SCHED
    201838 ±  6%     -28.5%     144292 ± 10%  softirqs.CPU7.RCU
    215956 ±  6%     -28.4%     154696 ±  9%  softirqs.CPU70.RCU
     23312 ±  8%     -22.7%      18026 ±  9%  softirqs.CPU70.SCHED
    215406 ±  6%     -28.7%     153489 ±  9%  softirqs.CPU71.RCU
     22148 ±  2%     -20.3%      17645 ± 10%  softirqs.CPU71.SCHED
    203716 ±  6%     -28.6%     145467 ±  9%  softirqs.CPU72.RCU
     21959 ±  9%     -23.2%      16861 ±  5%  softirqs.CPU72.SCHED
    203812 ±  6%     -28.3%     146175 ±  9%  softirqs.CPU73.RCU
    203810 ±  6%     -28.6%     145609 ±  9%  softirqs.CPU74.RCU
     23846 ± 10%     -22.9%      18395 ±  9%  softirqs.CPU74.SCHED
    203782 ±  6%     -28.6%     145431 ±  9%  softirqs.CPU75.RCU
     24210 ± 13%     -25.4%      18054 ±  5%  softirqs.CPU75.SCHED
    204045 ±  6%     -28.6%     145695 ±  9%  softirqs.CPU76.RCU
     24107 ±  4%     -24.1%      18309 ±  8%  softirqs.CPU76.SCHED
    204587 ±  6%     -28.6%     145982 ±  9%  softirqs.CPU77.RCU
     23196 ± 11%     -20.8%      18363 ±  6%  softirqs.CPU77.SCHED
    203973 ±  6%     -28.3%     146159 ±  9%  softirqs.CPU78.RCU
     23708 ±  8%     -22.7%      18333 ±  8%  softirqs.CPU78.SCHED
    204523 ±  6%     -28.9%     145363 ±  9%  softirqs.CPU79.RCU
     23358 ±  7%     -19.6%      18773 ±  8%  softirqs.CPU79.SCHED
    201301 ±  6%     -28.4%     144045 ± 10%  softirqs.CPU8.RCU
     16536 ±  2%     -25.0%      12401 ±  7%  softirqs.CPU8.SCHED
    202735 ±  6%     -28.4%     145162 ±  9%  softirqs.CPU80.RCU
     24125 ±  9%     -22.4%      18713 ±  7%  softirqs.CPU80.SCHED
    202638 ±  6%     -28.0%     145932 ±  9%  softirqs.CPU81.RCU
     23794 ± 11%     -23.9%      18098 ±  6%  softirqs.CPU81.SCHED
    204200 ±  6%     -28.2%     146676 ±  9%  softirqs.CPU82.RCU
     23274 ±  7%     -22.0%      18151 ±  9%  softirqs.CPU82.SCHED
    204940 ±  6%     -28.2%     147106 ±  9%  softirqs.CPU83.RCU
    202867 ±  6%     -27.9%     146170 ±  9%  softirqs.CPU84.RCU
     23219 ±  9%     -21.2%      18286 ±  6%  softirqs.CPU84.SCHED
    202233 ±  6%     -28.1%     145390 ±  9%  softirqs.CPU85.RCU
    202773 ±  6%     -28.2%     145568 ±  9%  softirqs.CPU86.RCU
     24198 ±  8%     -20.0%      19347 ±  7%  softirqs.CPU86.SCHED
    203138 ±  6%     -28.1%     145995 ±  9%  softirqs.CPU87.RCU
     24121 ± 10%     -22.9%      18590 ± 10%  softirqs.CPU87.SCHED
    204056 ±  6%     -28.3%     146306 ±  9%  softirqs.CPU88.RCU
     23484 ±  8%     -21.7%      18399 ±  9%  softirqs.CPU88.SCHED
    204461 ±  6%     -28.0%     147236 ±  9%  softirqs.CPU89.RCU
     23871 ± 10%     -20.9%      18892 ±  4%  softirqs.CPU89.SCHED
    199606 ±  6%     -28.1%     143488 ± 10%  softirqs.CPU9.RCU
    201165 ±  6%     -28.1%     144674 ±  9%  softirqs.CPU90.RCU
     23158 ±  9%     -20.5%      18403 ±  5%  softirqs.CPU90.SCHED
    203664 ±  6%     -28.3%     146065 ±  9%  softirqs.CPU91.RCU
     23715 ± 12%     -25.6%      17637 ±  8%  softirqs.CPU91.SCHED
    203117 ±  6%     -28.0%     146167 ±  9%  softirqs.CPU92.RCU
     24175 ± 10%     -22.6%      18704 ±  8%  softirqs.CPU92.SCHED
    202988 ±  6%     -27.9%     146298 ±  9%  softirqs.CPU93.RCU
     23486 ±  8%     -19.8%      18840 ±  7%  softirqs.CPU93.SCHED
    204067 ±  6%     -27.9%     147224 ±  9%  softirqs.CPU94.RCU
    203850 ±  6%     -28.1%     146595 ±  9%  softirqs.CPU95.RCU
     24015 ±  8%     -23.7%      18317 ± 11%  softirqs.CPU95.SCHED
    199699 ±  8%     -28.2%     143289 ±  9%  softirqs.CPU96.RCU
    197219 ± 11%     -28.1%     141731 ± 13%  softirqs.CPU97.RCU
     16779 ±  3%     -23.8%      12785 ± 13%  softirqs.CPU97.SCHED
    201869 ±  6%     -28.8%     143794 ±  9%  softirqs.CPU98.RCU
     16393 ±  3%     -21.2%      12917 ± 12%  softirqs.CPU98.SCHED
    201718 ±  7%     -30.3%     140575 ± 12%  softirqs.CPU99.RCU
  39732848 ±  6%     -28.5%   28390119 ±  9%  softirqs.RCU
   4184661 ±  4%     -22.6%    3239867 ±  6%  softirqs.SCHED
    253050 ±  6%     -31.0%     174587 ±  9%  softirqs.TIMER


                                                                                
                                hackbench.throughput                            
                                                                                
  140000 +------------------------------------------------------------------+   
         |                                                   O   O          |   
  120000 |OOO    OO   OO O    O  OO  O          O OOOOOO OOOO OOO           |   
         |   O O    OO     OO  O   O  O     O  O                            |   
  100000 |-+  O    O    O       O                                           |   
         |                                 O                      OOO OOO   |   
   80000 |-+                                                                |   
         |  +     +      +      +                     +      +    +    +  + |   
   60000 |++ +++.+ ++++++ +++.++ ++                +++ +.++++ ++++ ++.+ ++ +|   
         |                         +  +  +.+   ++ +                         |   
   40000 |-+                       :  :  : :   :: :                         |   
         |                         :  :  : :   :: :                         |   
   20000 |-+                        :: ::   : :  :                          |   
         |                          :: ::   : :  :                          |   
       0 +------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                             hackbench.time.system_time                         
                                                                                
  350000 +------------------------------------------------------------------+   
         |                         +  +  +.+    +                           |   
  300000 |++  + .+ + ++++  ++.++  +:  :  : :   +: ++++  .++++  +++   .+ +  +|   
         |  ++ +  + +    ++     ++ :  :  : :   :: :   ++     ++   +++  + ++ |   
  250000 |-+                       :  :  : :   :: :                         |   
         |                         :  :  : :   :: :                         |   
  200000 |-+                       :  :  : :   :: :                OO O     |   
         |              O       O  :  :  : :   :: :               O    OO   |   
  150000 |OOOOOO OOOOOOO O OO OO OOO::O::  O:  :O::  O   OOOO O             |   
         |                          :: ::   : :  : OO OO     O OOO          |   
  100000 |-+                        :: ::   : :  :                          |   
         |                          :: ::   : :  :                          |   
   50000 |-+                        :: ::   : :  :                          |   
         |                          :: ::   : :  :                          |   
       0 +------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                            hackbench.time.elapsed_time                         
                                                                                
  2000 +--------------------------------------------------------------------+   
  1800 |-+                                    +  +                          |   
       |+        +  ++.    ++     +  +  +.+   :+ :+.+    ++                 |   
  1600 |-++ .+++ ::+   ++ +  ++ .+:  :  : :   :: :   ++ +  ++. +++   ++.++ +|   
  1400 |-+ +    + +      +     +  :  :  : :   :: :     +      +   +++     + |   
       |                          :  :  : :   :: :                          |   
  1200 |-+                        :  :  : :   :: :                  O       |   
  1000 |-+   O                O   :  :  : :   :: :                OO OO O   |   
   800 |OOOO  OOOOOOO  O  OOOO O  O: :: :  : :OO:        O                  |   
       |             O  O        O :: ::   : :  :OO OOOOO OOO OOOO          |   
   600 |-+                         :: ::   : :  :                           |   
   400 |-+                         :: ::   : :  :                           |   
       |                           :: ::   : :  :                           |   
   200 |-+                         :: ::   : :  :                           |   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                          hackbench.time.elapsed_time.max                       
                                                                                
  2000 +--------------------------------------------------------------------+   
  1800 |-+                                    +  +                          |   
       |+        +  ++.    ++     +  +  +.+   :+ :+.+    ++                 |   
  1600 |-++ .+++ ::+   ++ +  ++ .+:  :  : :   :: :   ++ +  ++. +++   ++.++ +|   
  1400 |-+ +    + +      +     +  :  :  : :   :: :     +      +   +++     + |   
       |                          :  :  : :   :: :                          |   
  1200 |-+                        :  :  : :   :: :                  O       |   
  1000 |-+   O                O   :  :  : :   :: :                OO OO O   |   
   800 |OOOO  OOOOOOO  O  OOOO O  O: :: :  : :OO:        O                  |   
       |             O  O        O :: ::   : :  :OO OOOOO OOO OOOO          |   
   600 |-+                         :: ::   : :  :                           |   
   400 |-+                         :: ::   : :  :                           |   
       |                           :: ::   : :  :                           |   
   200 |-+                         :: ::   : :  :                           |   
     0 +--------------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-csl-2sp4: 96 threads 2 sockets Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory
=========================================================================================
compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
  gcc-9/performance/socket/8/x86_64-rhel-8.3/process/1600%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp4/hackbench/0x4003006

commit: 
  fdbcb2a6d6 ("mm/memcg: move mod_objcg_state() to memcontrol.c")
  68ac5b3c8d ("mm/memcg: cache vmstat data in percpu memcg_stock_pcp")

fdbcb2a6d6778e0b 68ac5b3c8db2fda00af594eca41 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
          0:6           -8%           0:6     perf-profile.children.cycles-pp.error_entry
         %stddev     %change         %stddev
             \          |                \  
     66846 ±  2%     +32.7%      88731 ±  2%  hackbench.throughput
      1616 ±  2%     -25.9%       1198 ±  2%  hackbench.time.elapsed_time
      1616 ±  2%     -25.9%       1198 ±  2%  hackbench.time.elapsed_time.max
 1.933e+09 ± 21%     -84.4%  3.018e+08 ± 19%  hackbench.time.involuntary_context_switches
      8906            -2.1%       8720        hackbench.time.percent_of_cpu_this_job_got
    141381 ±  2%     -27.1%     103040 ±  2%  hackbench.time.system_time
      2576 ± 17%     -43.9%       1444 ±  3%  hackbench.time.user_time
 2.635e+09 ± 18%     -73.6%   6.95e+08 ±  9%  hackbench.time.voluntary_context_switches
   2989130 ± 64%     -67.4%     975670 ±111%  cpuidle.POLL.usage
   5035530 ± 62%     -61.2%    1951377 ± 45%  numa-meminfo.node1.MemUsed
      1653 ±  2%     -25.3%       1235 ±  2%  uptime.boot
   5919793 ± 65%     -55.4%    2642061 ±  7%  numa-vmstat.node1.numa_hit
   5855661 ± 66%     -55.6%    2597226 ±  7%  numa-vmstat.node1.numa_local
     12579 ± 13%     -46.9%       6676 ±  3%  vmstat.procs.r
   2882862 ± 21%     -69.6%     876931 ± 10%  vmstat.system.cs
      7.30 ±  3%      +2.3        9.59        mpstat.cpu.all.idle%
      0.89 ±  3%      -0.1        0.81        mpstat.cpu.all.irq%
      0.02 ±  2%      +0.0        0.03 ±  3%  mpstat.cpu.all.soft%
      1.62 ± 18%      -0.4        1.25        mpstat.cpu.all.usr%
    209656            -4.6%     200066 ±  3%  proc-vmstat.nr_active_anon
    816698            +2.6%     838072        proc-vmstat.nr_kernel_stack
    215126            -4.3%     205815 ±  3%  proc-vmstat.nr_shmem
    679175 ±  2%      +4.9%     712630        proc-vmstat.nr_slab_unreclaimable
    209656            -4.6%     200066 ±  3%  proc-vmstat.nr_zone_active_anon
    270489 ±  5%     +12.9%     305469 ±  3%  slabinfo.kmalloc-512.active_objs
    183677 ±  7%     +16.4%     213837 ±  4%  slabinfo.skbuff_head_cache.active_objs
     17316 ± 11%     +17.0%      20252 ±  4%  slabinfo.skbuff_head_cache.active_slabs
    554151 ± 11%     +17.0%     648092 ±  4%  slabinfo.skbuff_head_cache.num_objs
     17316 ± 11%     +17.0%      20252 ±  4%  slabinfo.skbuff_head_cache.num_slabs
      0.23 ±  9%      +0.1        0.31 ±  4%  turbostat.C1%
      4.63 ± 18%      +2.3        6.96 ± 15%  turbostat.C1E%
      5.99 ±  5%     +35.3%       8.10 ±  3%  turbostat.CPU%c1
 4.141e+08 ± 17%     -42.8%  2.368e+08 ± 15%  turbostat.IRQ
     33.06 ±100%    +115.6%      71.28        turbostat.RAMWatt
     11.17 ±  3%     -18.6%       9.09 ±  2%  perf-stat.i.MPKI
 9.476e+09 ± 11%     -25.5%  7.058e+09 ±  2%  perf-stat.i.branch-instructions
      0.81 ±  3%      -0.2        0.60 ±  3%  perf-stat.i.branch-miss-rate%
  79936790 ± 14%     -46.6%   42700294 ±  2%  perf-stat.i.branch-misses
     30.13 ±  5%      +9.5       39.61        perf-stat.i.cache-miss-rate%
 4.864e+08 ± 10%     -36.9%  3.069e+08 ±  3%  perf-stat.i.cache-references
   2395540 ± 22%     -67.0%     790961 ±  7%  perf-stat.i.context-switches
 2.355e+11           -10.7%  2.103e+11        perf-stat.i.cpu-cycles
      2082 ±  2%      +5.2%       2191 ±  2%  perf-stat.i.cycles-between-cache-misses
      0.23 ± 30%      -0.1        0.12 ± 29%  perf-stat.i.dTLB-load-miss-rate%
  35585799 ± 40%     -58.5%   14773839 ± 35%  perf-stat.i.dTLB-load-misses
 1.258e+10 ± 10%     -23.8%  9.593e+09 ±  2%  perf-stat.i.dTLB-loads
 6.796e+09 ± 11%     -26.1%  5.023e+09 ±  2%  perf-stat.i.dTLB-stores
  47077684 ± 13%     -51.5%   22838344 ±  3%  perf-stat.i.iTLB-load-misses
  19003694 ± 22%     -67.3%    6214874 ±  5%  perf-stat.i.iTLB-loads
 4.606e+10 ± 10%     -25.3%  3.441e+10 ±  2%  perf-stat.i.instructions
      2554 ±  8%     +35.3%       3456 ±  3%  perf-stat.i.instructions-per-iTLB-miss
    503.38 ±  7%     +64.2%     826.53 ± 11%  perf-stat.i.major-faults
      2.44           -11.8%       2.15        perf-stat.i.metric.GHz
      1067 ±  8%     -12.7%     932.12 ±  2%  perf-stat.i.metric.K/sec
    307.12 ± 10%     -26.3%     226.32 ±  2%  perf-stat.i.metric.M/sec
     33337 ±  4%     +35.2%      45074 ±  2%  perf-stat.i.minor-faults
     55.51 ±  5%      -9.1       46.40 ±  3%  perf-stat.i.node-load-miss-rate%
  26572946 ± 12%     -35.7%   17095614 ±  2%  perf-stat.i.node-load-misses
  20452087 ±  3%     +23.4%   25241388 ±  4%  perf-stat.i.node-loads
  17722287 ±  3%     +45.5%   25780743 ±  3%  perf-stat.i.node-stores
     33840 ±  4%     +35.6%      45900 ±  2%  perf-stat.i.page-faults
     10.95 ±  3%     -18.1%       8.96 ±  2%  perf-stat.overall.MPKI
      0.88 ±  2%      -0.2        0.63        perf-stat.overall.branch-miss-rate%
     31.07 ±  9%     +20.0       51.11 ±  2%  perf-stat.overall.cache-miss-rate%
      1528 ±  2%     -20.7%       1212 ±  3%  perf-stat.overall.cycles-between-cache-misses
      0.30 ± 33%      -0.1        0.17 ± 36%  perf-stat.overall.dTLB-load-miss-rate%
     72.51 ±  3%     +12.1       84.57        perf-stat.overall.iTLB-load-miss-rate%
    907.89           +44.1%       1308        perf-stat.overall.instructions-per-iTLB-miss
     52.18 ±  4%     -19.2       32.99        perf-stat.overall.node-load-miss-rate%
     28.66 ±  7%     -10.3       18.38        perf-stat.overall.node-store-miss-rate%
  95699062 ± 11%     -36.1%   61121486 ±  2%  perf-stat.ps.branch-misses
 1.812e+08 ±  2%     +23.4%  2.236e+08 ±  3%  perf-stat.ps.cache-misses
 5.873e+08 ±  7%     -25.5%  4.376e+08 ±  3%  perf-stat.ps.cache-references
   2870044 ± 21%     -69.7%     868230 ± 10%  perf-stat.ps.context-switches
 2.769e+11            -2.2%  2.708e+11        perf-stat.ps.cpu-cycles
     41347 ± 25%     -46.6%      22081 ±  3%  perf-stat.ps.cpu-migrations
  45624790 ± 39%     -48.5%   23501280 ± 37%  perf-stat.ps.dTLB-load-misses
  59221419 ±  9%     -37.0%   37312739        perf-stat.ps.iTLB-load-misses
  22801344 ± 21%     -70.1%    6820471 ± 10%  perf-stat.ps.iTLB-loads
    164.81 ±  7%     +45.0%     238.96 ±  9%  perf-stat.ps.major-faults
     12346 ±  2%     +33.5%      16479 ±  2%  perf-stat.ps.minor-faults
  28298925 ±  7%     -30.5%   19674114 ±  2%  perf-stat.ps.node-load-misses
  25850519 ±  2%     +54.6%   39977675 ±  3%  perf-stat.ps.node-loads
  24700103 ±  4%     +84.5%   45563654 ±  2%  perf-stat.ps.node-stores
     12511 ±  2%     +33.6%      16718 ±  2%  perf-stat.ps.page-faults
  8.67e+13 ±  7%     -32.5%  5.849e+13        perf-stat.total.instructions
      0.00 ±152%  +1.9e+07%     642.60 ±104%  perf-sched.sch_delay.avg.ms.rcu_gp_kthread.kthread.ret_from_fork
    136.59 ± 25%     -77.9%      30.24 ± 14%  perf-sched.sch_delay.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      0.01           +21.4%       0.01 ±  5%  perf-sched.sch_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
      0.46 ±217%   +8382.5%      38.95 ± 78%  perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
    116306 ±  4%     -55.9%      51249 ± 10%  perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.01 ±104%    +1e+08%      10700 ±146%  perf-sched.sch_delay.max.ms.rcu_gp_kthread.kthread.ret_from_fork
      1.80 ±222%  +4.9e+05%       8729 ±221%  perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
     91893 ± 26%     -45.5%      50088 ± 11%  perf-sched.sch_delay.max.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
    117071 ±  4%     -55.7%      51831 ± 10%  perf-sched.sch_delay.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
     88.71 ±221%  +33947.1%      30203 ± 63%  perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
     85.67 ± 23%     -66.4%      28.77 ±  9%  perf-sched.total_sch_delay.average.ms
    117076 ±  4%     -55.7%      51831 ± 10%  perf-sched.total_sch_delay.max.ms
   1024233 ±  2%     -20.5%     814352 ±  4%  perf-sched.total_wait_and_delay.count.ms
    234198 ±  4%     -55.7%     103802 ± 10%  perf-sched.total_wait_and_delay.max.ms
    117346 ±  4%     -55.2%      52524 ±  9%  perf-sched.total_wait_time.max.ms
     25223 ± 44%     -71.3%       7243 ± 40%  perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      1144 ± 10%     -62.6%     427.53 ± 25%  perf-sched.wait_and_delay.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
    182.67 ± 21%    +424.5%     958.17 ± 53%  perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
     32.00 ±108%    +242.7%     109.67 ± 21%  perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
    406629 ±  4%     -49.1%     207148 ± 11%  perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2931 ± 19%    +107.7%       6087 ± 14%  perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
    217.50 ± 26%     +80.8%     393.33 ± 15%  perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
     64.83 ± 76%    +197.4%     192.83 ± 32%  perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
     21.83 ±101%    +280.2%      83.00 ± 20%  perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg
     11787 ± 20%    +376.9%      56211 ± 18%  perf-sched.wait_and_delay.count.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
    116.67 ±  5%     +54.6%     180.33 ±  9%  perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
    223.67 ± 17%    +381.1%       1076 ± 47%  perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
    162248 ± 29%     -52.9%      76448 ± 32%  perf-sched.wait_and_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
    220396 ±  6%     -54.1%     101251 ± 11%  perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
    123552 ± 15%     -23.4%      94618 ± 10%  perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
    130467 ± 34%     -60.5%      51519 ± 10%  perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
    165624 ± 29%     -39.9%      99599 ± 10%  perf-sched.wait_and_delay.max.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
    234198 ±  4%     -55.7%     103736 ± 10%  perf-sched.wait_and_delay.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
    116481 ±  4%     -55.8%      51474 ± 11%  perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
    116612 ±  4%     -40.1%      69804 ± 26%  perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork
     22554 ± 42%     -68.9%       7024 ± 40%  perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.06 ± 97%  +1.9e+06%       1048 ±221%  perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
      0.63 ±144%  +3.3e+05%       2076 ±142%  perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      1.06 ±206%    +562.6%       7.01 ± 91%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__alloc_pages.alloc_pages_vma.wp_page_copy
      0.84 ±204%    +1e+06%       8433 ±177%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables
      0.34 ±190%  +1.2e+06%       3939 ± 81%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.remove_vma.exit_mmap.mmput
      0.05 ± 57%  +2.7e+06%       1299 ±114%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.zap_pte_range.unmap_page_range.unmap_vmas
      0.26 ±223%  +7.5e+05%       1948 ±199%  perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      1007 ±  9%     -60.6%     397.28 ± 27%  perf-sched.wait_time.avg.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
    115581 ±  4%     -55.1%      51872 ± 10%  perf-sched.wait_time.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      0.27 ±165%    +3e+06%       8348 ±222%  perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
    117266 ±  4%     -55.3%      52368 ±  9%  perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
      4.07 ±141%  +4.3e+05%      17691 ±140%  perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      2.05 ±207%    +747.8%      17.36 ± 62%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__alloc_pages.alloc_pages_vma.wp_page_copy
    115804 ±  4%     -54.9%      52181 ±  9%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
      3.56 ±213%  +6.7e+05%      23768 ± 81%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables
    107150 ± 13%     -51.9%      51516 ± 10%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
      7.64 ±149%  +3.1e+05%      23760 ±101%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.down_write.unlink_anon_vmas.free_pgtables
      1.54 ±209%  +1.9e+06%      29014 ± 72%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.remove_vma.exit_mmap.mmput
      0.26 ± 58%    +1e+07%      26321 ±100%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.zap_pte_range.unmap_page_range.unmap_vmas
      0.52 ±223%  +1.8e+06%       9162 ±212%  perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
    113782 ±  4%     -54.3%      52040 ± 10%  perf-sched.wait_time.max.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
    117327 ±  4%     -55.5%      52228 ±  9%  perf-sched.wait_time.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
    116481 ±  4%     -55.8%      51474 ± 11%  perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
    116541 ±  4%     -55.8%      51558 ± 10%  perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork
    114184 ±  8%     -20.6%      90666 ± 10%  softirqs.CPU0.RCU
    114289 ±  7%     -21.0%      90265 ±  7%  softirqs.CPU1.RCU
    119909 ±  5%     -25.2%      89635 ± 14%  softirqs.CPU10.RCU
    120164 ±  5%     -23.7%      91634 ± 11%  softirqs.CPU11.RCU
    116012 ± 10%     -21.7%      90853 ± 11%  softirqs.CPU12.RCU
    119396 ±  4%     -24.3%      90341 ± 11%  softirqs.CPU13.RCU
    116589 ±  7%     -22.0%      90927 ± 11%  softirqs.CPU14.RCU
    119794 ±  5%     -24.3%      90667 ± 11%  softirqs.CPU15.RCU
    133127 ±  4%     -25.2%      99573 ± 12%  softirqs.CPU16.RCU
    133994 ±  5%     -26.9%      97966 ± 13%  softirqs.CPU17.RCU
    135938 ±  5%     -25.7%     101014 ± 12%  softirqs.CPU18.RCU
    134987 ±  4%     -25.6%     100440 ± 11%  softirqs.CPU19.RCU
    119761 ±  5%     -24.1%      90949 ± 11%  softirqs.CPU2.RCU
    135228 ±  4%     -25.6%     100584 ± 11%  softirqs.CPU20.RCU
    134220 ±  4%     -25.2%     100354 ± 11%  softirqs.CPU21.RCU
    133607 ±  5%     -25.5%      99481 ± 11%  softirqs.CPU22.RCU
    132953 ±  5%     -25.3%      99254 ± 11%  softirqs.CPU23.RCU
    125453 ±  5%     -22.5%      97263 ± 11%  softirqs.CPU24.RCU
     52187 ± 10%     -15.3%      44184 ±  3%  softirqs.CPU24.SCHED
    125561 ±  5%     -23.2%      96476 ± 12%  softirqs.CPU25.RCU
     52772 ± 10%     -16.0%      44354 ±  3%  softirqs.CPU25.SCHED
    123367 ±  5%     -21.9%      96395 ± 11%  softirqs.CPU26.RCU
     52972 ± 10%     -15.6%      44713 ±  3%  softirqs.CPU26.SCHED
    122905 ±  5%     -23.2%      94372 ± 13%  softirqs.CPU27.RCU
     53253 ± 11%     -16.5%      44456 ±  2%  softirqs.CPU27.SCHED
    123695 ±  6%     -22.4%      95926 ± 12%  softirqs.CPU28.RCU
     52960 ± 10%     -15.7%      44628 ±  3%  softirqs.CPU28.SCHED
    121430 ±  5%     -21.5%      95279 ± 12%  softirqs.CPU29.RCU
     53015 ± 10%     -15.7%      44673 ±  4%  softirqs.CPU29.SCHED
    119944 ±  4%     -24.4%      90706 ± 11%  softirqs.CPU3.RCU
    124998 ±  5%     -24.5%      94335 ± 10%  softirqs.CPU30.RCU
     53154 ± 11%     -16.3%      44481 ±  3%  softirqs.CPU30.SCHED
    125493 ±  6%     -23.3%      96239 ± 12%  softirqs.CPU31.RCU
     53027 ± 11%     -16.7%      44164 ±  3%  softirqs.CPU31.SCHED
    119025 ±  4%     -23.7%      90870 ± 12%  softirqs.CPU32.RCU
     53067 ± 11%     -16.2%      44476 ±  3%  softirqs.CPU32.SCHED
    117982 ±  5%     -23.0%      90844 ± 12%  softirqs.CPU33.RCU
     52814 ± 11%     -16.1%      44305 ±  3%  softirqs.CPU33.SCHED
    117877 ±  5%     -23.0%      90742 ± 12%  softirqs.CPU34.RCU
     53003 ± 10%     -15.8%      44647 ±  3%  softirqs.CPU34.SCHED
    118295 ±  4%     -23.2%      90796 ± 12%  softirqs.CPU35.RCU
     52789 ± 11%     -16.3%      44182 ±  3%  softirqs.CPU35.SCHED
    120083 ±  4%     -23.8%      91525 ± 12%  softirqs.CPU36.RCU
     52671 ± 10%     -16.7%      43875 ±  3%  softirqs.CPU36.SCHED
    120626 ±  4%     -24.1%      91537 ± 12%  softirqs.CPU37.RCU
     52745 ± 11%     -16.0%      44288 ±  3%  softirqs.CPU37.SCHED
    120928 ±  5%     -24.8%      90960 ± 11%  softirqs.CPU38.RCU
     53088 ± 10%     -16.5%      44341 ±  3%  softirqs.CPU38.SCHED
    118700 ±  5%     -23.7%      90620 ± 12%  softirqs.CPU39.RCU
     52680 ± 11%     -16.0%      44227 ±  4%  softirqs.CPU39.SCHED
    120774 ±  5%     -24.2%      91567 ± 10%  softirqs.CPU4.RCU
    119135 ±  5%     -24.0%      90523 ± 11%  softirqs.CPU40.RCU
     53445 ± 11%     -17.1%      44317 ±  4%  softirqs.CPU40.SCHED
    118090 ±  4%     -23.2%      90743 ± 12%  softirqs.CPU41.RCU
     53166 ± 11%     -17.2%      44045 ±  3%  softirqs.CPU41.SCHED
    120017 ±  5%     -23.5%      91862 ± 12%  softirqs.CPU42.RCU
     52816 ± 10%     -16.1%      44303 ±  3%  softirqs.CPU42.SCHED
    119579 ±  4%     -23.9%      91010 ± 12%  softirqs.CPU43.RCU
     53167 ± 10%     -16.9%      44207 ±  3%  softirqs.CPU43.SCHED
    119165 ±  5%     -23.4%      91228 ± 11%  softirqs.CPU44.RCU
     52960 ± 11%     -17.0%      43966 ±  3%  softirqs.CPU44.SCHED
    117999 ±  4%     -23.4%      90426 ± 12%  softirqs.CPU45.RCU
     53045 ± 11%     -16.4%      44338 ±  4%  softirqs.CPU45.SCHED
    118010 ±  4%     -23.2%      90685 ± 11%  softirqs.CPU46.RCU
     52727 ± 12%     -16.2%      44176 ±  3%  softirqs.CPU46.SCHED
    118424 ±  4%     -23.2%      90912 ± 12%  softirqs.CPU47.RCU
     52314 ± 11%     -16.0%      43941 ±  3%  softirqs.CPU47.SCHED
    114734 ± 12%     -20.3%      91490 ± 12%  softirqs.CPU48.RCU
    112041 ±  8%     -20.3%      89350 ±  8%  softirqs.CPU49.RCU
    120529 ±  4%     -23.5%      92261 ± 11%  softirqs.CPU5.RCU
    117409 ±  6%     -22.5%      90939 ± 11%  softirqs.CPU50.RCU
    118919 ±  6%     -23.2%      91370 ± 11%  softirqs.CPU51.RCU
    118703 ±  6%     -22.3%      92263 ± 12%  softirqs.CPU52.RCU
    118382 ±  6%     -22.6%      91577 ± 11%  softirqs.CPU53.RCU
    119135 ±  6%     -22.7%      92103 ± 11%  softirqs.CPU54.RCU
    119265 ±  6%     -23.0%      91884 ± 11%  softirqs.CPU55.RCU
    117401 ±  6%     -23.0%      90404 ± 11%  softirqs.CPU56.RCU
    119001 ±  6%     -23.0%      91686 ± 13%  softirqs.CPU57.RCU
    118121 ±  5%     -23.8%      90029 ± 14%  softirqs.CPU58.RCU
    118586 ±  6%     -22.2%      92244 ± 11%  softirqs.CPU59.RCU
    120277 ±  4%     -24.4%      90959 ± 12%  softirqs.CPU6.RCU
    115244 ± 11%     -20.1%      92044 ± 11%  softirqs.CPU60.RCU
    118905 ±  5%     -22.9%      91630 ± 11%  softirqs.CPU61.RCU
    114662 ±  8%     -20.8%      90861 ± 11%  softirqs.CPU62.RCU
    118251 ±  5%     -22.4%      91707 ± 11%  softirqs.CPU63.RCU
    134012 ±  5%     -24.9%     100669 ± 11%  softirqs.CPU64.RCU
    134233 ±  5%     -27.5%      97372 ± 14%  softirqs.CPU65.RCU
    137160 ±  4%     -25.5%     102174 ± 10%  softirqs.CPU66.RCU
    135877 ±  5%     -25.6%     101076 ± 11%  softirqs.CPU67.RCU
    134922 ±  5%     -24.9%     101268 ± 10%  softirqs.CPU68.RCU
    134720 ±  4%     -23.6%     102974 ± 12%  softirqs.CPU69.RCU
    119952 ±  4%     -23.1%      92197 ± 10%  softirqs.CPU7.RCU
    132918 ±  5%     -24.8%      99972 ± 11%  softirqs.CPU70.RCU
    132785 ±  4%     -24.6%     100098 ± 11%  softirqs.CPU71.RCU
    127056 ±  5%     -23.5%      97160 ± 11%  softirqs.CPU72.RCU
     53218 ± 10%     -15.7%      44852 ±  4%  softirqs.CPU72.SCHED
    126480 ±  6%     -23.9%      96268 ± 12%  softirqs.CPU73.RCU
     53562 ± 11%     -16.9%      44500 ±  4%  softirqs.CPU73.SCHED
    124660 ±  5%     -22.9%      96164 ± 11%  softirqs.CPU74.RCU
     53777 ± 11%     -16.5%      44879 ±  2%  softirqs.CPU74.SCHED
    124245 ±  5%     -24.1%      94295 ± 13%  softirqs.CPU75.RCU
     53874 ± 12%     -16.3%      45068 ±  3%  softirqs.CPU75.SCHED
    125293 ±  6%     -23.6%      95673 ± 11%  softirqs.CPU76.RCU
     54049 ± 11%     -17.3%      44711 ±  3%  softirqs.CPU76.SCHED
    121514 ±  4%     -21.8%      94985 ± 11%  softirqs.CPU77.RCU
     53659 ± 11%     -16.9%      44576 ±  3%  softirqs.CPU77.SCHED
    125807 ±  5%     -24.7%      94691 ±  9%  softirqs.CPU78.RCU
     53936 ± 12%     -17.1%      44693 ±  4%  softirqs.CPU78.SCHED
    126021 ±  5%     -23.6%      96298 ± 11%  softirqs.CPU79.RCU
     53883 ± 12%     -17.3%      44557 ±  3%  softirqs.CPU79.SCHED
    119710 ±  5%     -23.6%      91480 ± 11%  softirqs.CPU8.RCU
    118989 ±  5%     -23.5%      90984 ± 11%  softirqs.CPU80.RCU
     54159 ± 12%     -17.4%      44760 ±  3%  softirqs.CPU80.SCHED
    119117 ±  5%     -23.3%      91361 ± 11%  softirqs.CPU81.RCU
     53858 ± 11%     -16.8%      44802 ±  3%  softirqs.CPU81.SCHED
    119650 ±  5%     -23.3%      91808 ± 12%  softirqs.CPU82.RCU
     54282 ± 12%     -17.9%      44592 ±  3%  softirqs.CPU82.SCHED
    120091 ±  5%     -23.3%      92117 ± 12%  softirqs.CPU83.RCU
     53733 ± 11%     -16.9%      44641 ±  3%  softirqs.CPU83.SCHED
    119260 ±  4%     -23.8%      90851 ± 12%  softirqs.CPU84.RCU
     53562 ± 12%     -16.1%      44931 ±  2%  softirqs.CPU84.SCHED
    119637 ±  4%     -24.0%      90966 ± 12%  softirqs.CPU85.RCU
     54234 ± 12%     -17.7%      44625 ±  3%  softirqs.CPU85.SCHED
    119399 ±  4%     -23.5%      91375 ± 11%  softirqs.CPU86.RCU
     53875 ± 12%     -15.9%      45325 ±  5%  softirqs.CPU86.SCHED
    119617 ±  5%     -23.7%      91211 ± 12%  softirqs.CPU87.RCU
     54188 ± 11%     -17.5%      44681 ±  3%  softirqs.CPU87.SCHED
    120626 ±  5%     -23.6%      92204 ± 12%  softirqs.CPU88.RCU
     54135 ± 13%     -17.4%      44711 ±  3%  softirqs.CPU88.SCHED
    120031 ±  5%     -23.2%      92197 ± 12%  softirqs.CPU89.RCU
     54187 ± 12%     -17.6%      44624 ±  3%  softirqs.CPU89.SCHED
    120401 ±  4%     -24.1%      91370 ± 11%  softirqs.CPU9.RCU
    119423 ±  4%     -23.6%      91228 ± 12%  softirqs.CPU90.RCU
     53697 ± 11%     -17.3%      44396 ±  3%  softirqs.CPU90.SCHED
    119554 ±  4%     -23.3%      91678 ± 12%  softirqs.CPU91.RCU
     54189 ± 12%     -17.4%      44748 ±  3%  softirqs.CPU91.SCHED
    119817 ±  4%     -23.7%      91366 ± 12%  softirqs.CPU92.RCU
     54229 ± 12%     -17.4%      44793 ±  2%  softirqs.CPU92.SCHED
    119474 ±  5%     -23.3%      91670 ± 12%  softirqs.CPU93.RCU
     53702 ± 11%     -16.9%      44639 ±  3%  softirqs.CPU93.SCHED
    119441 ±  4%     -23.1%      91846 ± 11%  softirqs.CPU94.RCU
     53648 ± 12%     -16.9%      44594 ±  3%  softirqs.CPU94.SCHED
    120184 ±  3%     -24.1%      91251 ± 11%  softirqs.CPU95.RCU
     53668 ± 12%     -17.6%      44196 ±  3%  softirqs.CPU95.SCHED
  11743411 ±  5%     -23.6%    8973612 ± 11%  softirqs.RCU
   5039118 ±  3%     -11.6%    4452992        softirqs.SCHED
    180426 ±  3%     -21.1%     142298 ±  6%  softirqs.TIMER
      1579 ±  2%     -25.8%       1171 ±  2%  interrupts.293:PCI-MSI.327680-edge.xhci_hcd
  67312831 ± 28%     -85.6%    9672064 ± 11%  interrupts.CAL:Function_call_interrupts
   1000046 ± 73%     -89.1%     108653 ± 35%  interrupts.CPU0.CAL:Function_call_interrupts
    435107 ± 21%     -64.8%     153129 ± 24%  interrupts.CPU0.RES:Rescheduling_interrupts
    441799 ± 53%     -78.0%      97386 ± 12%  interrupts.CPU1.CAL:Function_call_interrupts
    476047 ± 19%     -71.2%     137010 ± 24%  interrupts.CPU1.RES:Rescheduling_interrupts
      1579 ±  2%     -25.8%       1171 ±  2%  interrupts.CPU10.293:PCI-MSI.327680-edge.xhci_hcd
    446904 ± 52%     -78.0%      98261 ± 28%  interrupts.CPU10.CAL:Function_call_interrupts
    602379 ± 27%     -75.6%     146834 ± 31%  interrupts.CPU10.RES:Rescheduling_interrupts
    564799 ± 44%     -85.2%      83384 ± 22%  interrupts.CPU11.CAL:Function_call_interrupts
    598246 ± 24%     -78.8%     126665 ± 17%  interrupts.CPU11.RES:Rescheduling_interrupts
    514319 ± 51%     -83.9%      82697 ± 10%  interrupts.CPU12.CAL:Function_call_interrupts
    459552 ± 23%     -70.0%     137653 ± 22%  interrupts.CPU12.RES:Rescheduling_interrupts
    410806 ± 43%     -77.1%      94196 ± 21%  interrupts.CPU13.CAL:Function_call_interrupts
    520277 ± 34%     -71.0%     150899 ± 13%  interrupts.CPU13.RES:Rescheduling_interrupts
    487114 ± 32%     -81.8%      88472 ± 17%  interrupts.CPU14.CAL:Function_call_interrupts
    642852 ± 20%     -79.7%     130512 ± 20%  interrupts.CPU14.RES:Rescheduling_interrupts
    583107 ± 36%     -85.6%      83840 ± 25%  interrupts.CPU15.CAL:Function_call_interrupts
    598732 ± 18%     -75.7%     145702 ± 17%  interrupts.CPU15.RES:Rescheduling_interrupts
    528273 ± 50%     -79.7%     107349 ± 18%  interrupts.CPU16.CAL:Function_call_interrupts
    787747 ± 38%     -79.4%     162249 ± 27%  interrupts.CPU16.RES:Rescheduling_interrupts
    665311 ± 34%     -85.8%      94151 ± 21%  interrupts.CPU17.CAL:Function_call_interrupts
    636120 ± 19%     -75.8%     154164 ± 14%  interrupts.CPU17.RES:Rescheduling_interrupts
    586596 ± 47%     -84.0%      93957 ± 18%  interrupts.CPU18.CAL:Function_call_interrupts
    656525 ± 39%     -76.1%     156643 ± 17%  interrupts.CPU18.RES:Rescheduling_interrupts
    513755 ± 42%     -83.9%      82705 ± 26%  interrupts.CPU19.CAL:Function_call_interrupts
    527848 ± 11%     -71.5%     150534 ± 32%  interrupts.CPU19.RES:Rescheduling_interrupts
    526368 ± 51%     -81.4%      97845 ± 10%  interrupts.CPU2.CAL:Function_call_interrupts
    592304 ± 26%     -73.9%     154321 ± 22%  interrupts.CPU2.RES:Rescheduling_interrupts
    523825 ± 49%     -81.4%      97332 ± 33%  interrupts.CPU20.CAL:Function_call_interrupts
    535373 ± 28%     -69.8%     161508 ± 26%  interrupts.CPU20.RES:Rescheduling_interrupts
    523377 ± 28%     -80.6%     101433 ± 24%  interrupts.CPU21.CAL:Function_call_interrupts
    559624 ± 30%     -73.9%     146323 ± 23%  interrupts.CPU21.RES:Rescheduling_interrupts
    635043 ± 40%     -85.9%      89347 ± 16%  interrupts.CPU22.CAL:Function_call_interrupts
    595030 ± 12%     -76.6%     139054 ± 29%  interrupts.CPU22.RES:Rescheduling_interrupts
    536335 ± 44%     -82.7%      92971 ±  8%  interrupts.CPU23.CAL:Function_call_interrupts
    620371 ± 25%     -77.4%     139904 ± 20%  interrupts.CPU23.RES:Rescheduling_interrupts
   1260708 ± 34%     -88.6%     144250 ± 38%  interrupts.CPU24.CAL:Function_call_interrupts
    361897 ± 44%     -72.3%     100327 ± 13%  interrupts.CPU24.RES:Rescheduling_interrupts
    781039 ± 59%     -86.2%     107963 ± 26%  interrupts.CPU25.CAL:Function_call_interrupts
    587739 ± 38%     -81.6%     108021 ± 24%  interrupts.CPU25.RES:Rescheduling_interrupts
    821224 ± 50%     -88.3%      95907 ± 22%  interrupts.CPU26.CAL:Function_call_interrupts
    651514 ± 39%     -81.9%     118003 ± 20%  interrupts.CPU26.RES:Rescheduling_interrupts
   1130601 ± 46%     -89.6%     117172 ± 46%  interrupts.CPU27.CAL:Function_call_interrupts
    729288 ± 39%     -82.3%     128807 ± 26%  interrupts.CPU27.RES:Rescheduling_interrupts
    855810 ± 50%     -85.8%     121530 ± 28%  interrupts.CPU28.CAL:Function_call_interrupts
    632698 ± 51%     -81.5%     116816 ± 21%  interrupts.CPU28.RES:Rescheduling_interrupts
    793580 ± 49%     -86.3%     108455 ± 30%  interrupts.CPU29.CAL:Function_call_interrupts
    643494 ± 39%     -83.1%     108795 ± 29%  interrupts.CPU29.RES:Rescheduling_interrupts
    445293 ± 37%     -79.7%      90248 ± 27%  interrupts.CPU3.CAL:Function_call_interrupts
    534975 ± 29%     -72.6%     146737 ± 19%  interrupts.CPU3.RES:Rescheduling_interrupts
    839914 ± 49%     -90.1%      82742 ± 23%  interrupts.CPU30.CAL:Function_call_interrupts
    500713 ± 40%     -79.6%     102122 ± 23%  interrupts.CPU30.RES:Rescheduling_interrupts
    757309 ± 58%     -86.5%     102592 ± 20%  interrupts.CPU31.CAL:Function_call_interrupts
    575342 ± 48%     -80.1%     114516 ± 19%  interrupts.CPU31.RES:Rescheduling_interrupts
    827692 ± 51%     -85.7%     118517 ± 32%  interrupts.CPU32.CAL:Function_call_interrupts
    624069 ± 45%     -82.0%     112286 ± 24%  interrupts.CPU32.RES:Rescheduling_interrupts
    741985 ± 55%     -86.4%     101076 ± 19%  interrupts.CPU33.CAL:Function_call_interrupts
    649128 ± 41%     -82.9%     110727 ± 19%  interrupts.CPU33.RES:Rescheduling_interrupts
   1012379 ± 38%     -90.2%      99150 ± 22%  interrupts.CPU34.CAL:Function_call_interrupts
    677784 ± 24%     -84.3%     106581 ± 21%  interrupts.CPU34.RES:Rescheduling_interrupts
    751227 ± 46%     -85.7%     107452 ± 32%  interrupts.CPU35.CAL:Function_call_interrupts
    534488 ± 42%     -77.7%     119311 ± 16%  interrupts.CPU35.RES:Rescheduling_interrupts
    749457 ± 45%     -86.6%     100264 ± 38%  interrupts.CPU36.CAL:Function_call_interrupts
    502370 ± 31%     -80.7%      96905 ± 23%  interrupts.CPU36.RES:Rescheduling_interrupts
    741542 ± 55%     -88.1%      88529 ± 15%  interrupts.CPU37.CAL:Function_call_interrupts
    554068 ± 38%     -81.2%     104428 ± 19%  interrupts.CPU37.RES:Rescheduling_interrupts
    825045 ± 51%     -87.2%     105916 ± 32%  interrupts.CPU38.CAL:Function_call_interrupts
    629181 ± 45%     -79.9%     126330 ± 36%  interrupts.CPU38.RES:Rescheduling_interrupts
    984619 ± 46%     -90.7%      91637 ± 22%  interrupts.CPU39.CAL:Function_call_interrupts
    650113 ± 33%     -79.3%     134709 ± 35%  interrupts.CPU39.RES:Rescheduling_interrupts
    488311 ± 48%     -79.8%      98424 ± 23%  interrupts.CPU4.CAL:Function_call_interrupts
    544078 ± 24%     -75.0%     135853 ±  8%  interrupts.CPU4.RES:Rescheduling_interrupts
    965807 ± 53%     -88.9%     107209 ± 23%  interrupts.CPU40.CAL:Function_call_interrupts
    660162 ± 45%     -81.7%     120858 ± 19%  interrupts.CPU40.RES:Rescheduling_interrupts
   1031514 ± 50%     -91.3%      89924 ± 22%  interrupts.CPU41.CAL:Function_call_interrupts
    644803 ± 38%     -82.0%     115873 ± 19%  interrupts.CPU41.RES:Rescheduling_interrupts
    768801 ± 48%     -86.6%     102982 ± 25%  interrupts.CPU42.CAL:Function_call_interrupts
    579411 ± 38%     -81.5%     107132 ± 15%  interrupts.CPU42.RES:Rescheduling_interrupts
    901315 ± 62%     -90.2%      88679 ± 25%  interrupts.CPU43.CAL:Function_call_interrupts
    601838 ± 45%     -81.2%     113099 ± 17%  interrupts.CPU43.RES:Rescheduling_interrupts
   1000987 ± 39%     -91.4%      85664 ± 16%  interrupts.CPU44.CAL:Function_call_interrupts
    555650 ± 49%     -78.4%     120169 ± 10%  interrupts.CPU44.RES:Rescheduling_interrupts
    974069 ± 58%     -88.5%     112212 ± 39%  interrupts.CPU45.CAL:Function_call_interrupts
    663758 ± 41%     -81.5%     122815 ± 20%  interrupts.CPU45.RES:Rescheduling_interrupts
   1012208 ± 51%     -90.5%      96126 ± 17%  interrupts.CPU46.CAL:Function_call_interrupts
    577721 ± 41%     -81.1%     109166 ± 22%  interrupts.CPU46.RES:Rescheduling_interrupts
   1030696 ± 49%     -89.6%     107235 ± 18%  interrupts.CPU47.CAL:Function_call_interrupts
    611193 ± 63%     -81.7%     112079 ± 28%  interrupts.CPU47.RES:Rescheduling_interrupts
    714260 ± 59%     -81.9%     129280 ± 33%  interrupts.CPU48.CAL:Function_call_interrupts
    405969 ± 28%     -59.6%     164096 ± 27%  interrupts.CPU48.RES:Rescheduling_interrupts
    447297 ± 76%     -80.4%      87848 ± 13%  interrupts.CPU49.CAL:Function_call_interrupts
    527389 ± 26%     -70.2%     157134 ± 32%  interrupts.CPU49.RES:Rescheduling_interrupts
    581797 ± 38%     -83.8%      94157 ±  9%  interrupts.CPU5.CAL:Function_call_interrupts
    572139 ± 21%     -76.0%     137144 ± 21%  interrupts.CPU5.RES:Rescheduling_interrupts
    491168 ± 49%     -80.3%      96730 ± 30%  interrupts.CPU50.CAL:Function_call_interrupts
    578016 ± 21%     -70.6%     169729 ± 33%  interrupts.CPU50.RES:Rescheduling_interrupts
    414679 ± 56%     -79.3%      85659 ± 17%  interrupts.CPU51.CAL:Function_call_interrupts
    561418 ± 35%     -73.1%     150936 ± 21%  interrupts.CPU51.RES:Rescheduling_interrupts
    528740 ± 40%     -83.1%      89499 ± 11%  interrupts.CPU52.CAL:Function_call_interrupts
    643967 ± 34%     -78.8%     136534 ± 13%  interrupts.CPU52.RES:Rescheduling_interrupts
    568698 ± 35%     -82.9%      97199 ± 35%  interrupts.CPU53.CAL:Function_call_interrupts
    692481 ± 22%     -80.8%     132633 ± 22%  interrupts.CPU53.RES:Rescheduling_interrupts
    430150 ± 55%     -77.9%      95165 ± 19%  interrupts.CPU54.CAL:Function_call_interrupts
    404094 ± 32%     -64.3%     144257 ± 24%  interrupts.CPU54.RES:Rescheduling_interrupts
    360051 ± 52%     -76.1%      86110 ± 23%  interrupts.CPU55.CAL:Function_call_interrupts
    366324 ± 37%     -54.8%     165553 ± 39%  interrupts.CPU55.RES:Rescheduling_interrupts
    446431 ± 52%     -80.1%      88875 ± 11%  interrupts.CPU56.CAL:Function_call_interrupts
    476211 ± 32%     -68.0%     152559 ± 29%  interrupts.CPU56.RES:Rescheduling_interrupts
    421204 ± 46%     -75.1%     104697 ± 24%  interrupts.CPU57.CAL:Function_call_interrupts
    489921 ± 24%     -63.9%     177068 ± 55%  interrupts.CPU57.RES:Rescheduling_interrupts
    482474 ± 69%     -80.7%      93088 ± 27%  interrupts.CPU58.CAL:Function_call_interrupts
    623436 ± 26%     -78.0%     136961 ± 30%  interrupts.CPU58.RES:Rescheduling_interrupts
    579962 ± 54%     -84.6%      89163 ± 25%  interrupts.CPU59.CAL:Function_call_interrupts
    681328 ± 31%     -79.8%     137811 ± 16%  interrupts.CPU59.RES:Rescheduling_interrupts
    483707 ± 51%     -80.2%      95706 ± 21%  interrupts.CPU6.CAL:Function_call_interrupts
    459200 ± 28%     -69.3%     140942 ± 23%  interrupts.CPU6.RES:Rescheduling_interrupts
    470781 ± 47%     -80.2%      93374 ± 20%  interrupts.CPU60.CAL:Function_call_interrupts
    469003 ± 36%     -69.2%     144391 ± 29%  interrupts.CPU60.RES:Rescheduling_interrupts
    461804 ± 56%     -76.1%     110191 ± 24%  interrupts.CPU61.CAL:Function_call_interrupts
    520967 ± 29%     -68.9%     161787 ± 23%  interrupts.CPU61.RES:Rescheduling_interrupts
    422032 ± 56%     -78.0%      92640 ± 21%  interrupts.CPU62.CAL:Function_call_interrupts
    543922 ± 31%     -75.6%     132790 ± 19%  interrupts.CPU62.RES:Rescheduling_interrupts
    587153 ± 55%     -84.7%      89631 ± 30%  interrupts.CPU63.CAL:Function_call_interrupts
    708789 ± 16%     -78.0%     156124 ± 26%  interrupts.CPU63.RES:Rescheduling_interrupts
    585654 ± 47%     -82.3%     103552 ± 21%  interrupts.CPU64.CAL:Function_call_interrupts
    688309 ± 21%     -75.8%     166368 ± 28%  interrupts.CPU64.RES:Rescheduling_interrupts
    581141 ± 35%     -84.3%      91047 ± 16%  interrupts.CPU65.CAL:Function_call_interrupts
    616229 ± 26%     -75.1%     153443 ± 22%  interrupts.CPU65.RES:Rescheduling_interrupts
    473840 ± 58%     -79.7%      96228 ± 15%  interrupts.CPU66.CAL:Function_call_interrupts
    529502 ± 34%     -65.9%     180396 ± 28%  interrupts.CPU66.RES:Rescheduling_interrupts
    514035 ± 41%     -81.4%      95624 ± 24%  interrupts.CPU67.CAL:Function_call_interrupts
    567972 ± 24%     -72.3%     157117 ± 24%  interrupts.CPU67.RES:Rescheduling_interrupts
    488636 ± 54%     -78.8%     103497 ± 27%  interrupts.CPU68.CAL:Function_call_interrupts
    588887 ± 26%     -71.7%     166658 ± 29%  interrupts.CPU68.RES:Rescheduling_interrupts
    473929 ± 50%     -77.8%     105198 ± 12%  interrupts.CPU69.CAL:Function_call_interrupts
    536660 ± 27%     -74.0%     139760 ± 23%  interrupts.CPU69.RES:Rescheduling_interrupts
    457894 ± 56%     -79.8%      92717 ± 30%  interrupts.CPU7.CAL:Function_call_interrupts
    470890 ± 51%     -66.4%     158208 ± 32%  interrupts.CPU7.RES:Rescheduling_interrupts
    639309 ± 51%     -84.6%      98184 ± 28%  interrupts.CPU70.CAL:Function_call_interrupts
    734558 ± 19%     -80.8%     141367 ± 43%  interrupts.CPU70.RES:Rescheduling_interrupts
    662121 ± 52%     -85.8%      93861 ± 24%  interrupts.CPU71.CAL:Function_call_interrupts
    640700 ± 34%     -78.7%     136432 ± 18%  interrupts.CPU71.RES:Rescheduling_interrupts
   1123094 ± 40%     -89.1%     122248 ± 12%  interrupts.CPU72.CAL:Function_call_interrupts
    367557 ± 41%     -72.8%     100138 ± 14%  interrupts.CPU72.RES:Rescheduling_interrupts
    636351 ± 64%     -83.6%     104327 ± 37%  interrupts.CPU73.CAL:Function_call_interrupts
    501361 ± 42%     -77.4%     113354 ± 28%  interrupts.CPU73.RES:Rescheduling_interrupts
    747849 ± 57%     -85.1%     111620 ± 38%  interrupts.CPU74.CAL:Function_call_interrupts
    617694 ± 33%     -76.8%     143288 ± 30%  interrupts.CPU74.RES:Rescheduling_interrupts
    860857 ± 56%     -84.5%     133631 ± 39%  interrupts.CPU75.CAL:Function_call_interrupts
    650519 ± 39%     -79.5%     133493 ± 29%  interrupts.CPU75.RES:Rescheduling_interrupts
    820666 ± 55%     -83.8%     132983 ± 26%  interrupts.CPU76.CAL:Function_call_interrupts
    553758 ± 27%     -78.2%     120779 ± 14%  interrupts.CPU76.RES:Rescheduling_interrupts
    785724 ± 62%     -85.7%     112086 ± 37%  interrupts.CPU77.CAL:Function_call_interrupts
    656443 ± 39%     -84.2%     103477 ± 36%  interrupts.CPU77.RES:Rescheduling_interrupts
    793606 ± 45%     -87.5%      98917 ± 29%  interrupts.CPU78.CAL:Function_call_interrupts
    579013 ± 43%     -82.0%     104251 ± 29%  interrupts.CPU78.RES:Rescheduling_interrupts
    727899 ± 64%     -85.7%     103882 ± 34%  interrupts.CPU79.CAL:Function_call_interrupts
    544650 ± 39%     -78.5%     117180 ± 31%  interrupts.CPU79.RES:Rescheduling_interrupts
    563180 ± 36%     -84.6%      86491 ± 27%  interrupts.CPU8.CAL:Function_call_interrupts
    524707 ± 26%     -72.8%     142526 ± 25%  interrupts.CPU8.RES:Rescheduling_interrupts
    929710 ± 53%     -88.7%     104876 ± 32%  interrupts.CPU80.CAL:Function_call_interrupts
    669860 ± 40%     -79.7%     135924 ± 39%  interrupts.CPU80.RES:Rescheduling_interrupts
    771837 ± 62%     -86.8%     102018 ± 44%  interrupts.CPU81.CAL:Function_call_interrupts
    599243 ± 43%     -77.5%     134873 ± 36%  interrupts.CPU81.RES:Rescheduling_interrupts
    996596 ± 63%     -89.8%     101544 ± 23%  interrupts.CPU82.CAL:Function_call_interrupts
    758943 ± 36%     -86.2%     105049 ± 17%  interrupts.CPU82.RES:Rescheduling_interrupts
    720908 ± 64%     -85.4%     105078 ± 42%  interrupts.CPU83.CAL:Function_call_interrupts
    596410 ± 50%     -78.1%     130869 ± 27%  interrupts.CPU83.RES:Rescheduling_interrupts
    595394 ± 52%     -84.9%      89866 ± 29%  interrupts.CPU84.CAL:Function_call_interrupts
    438388 ± 15%     -78.8%      92874 ± 27%  interrupts.CPU84.RES:Rescheduling_interrupts
    500965 ± 41%     -76.8%     116426 ± 14%  interrupts.CPU85.RES:Rescheduling_interrupts
    565738 ± 40%     -74.7%     143308 ± 48%  interrupts.CPU86.RES:Rescheduling_interrupts
    855623 ± 41%     -87.5%     106874 ± 25%  interrupts.CPU87.CAL:Function_call_interrupts
    657859 ± 42%     -77.8%     146024 ± 37%  interrupts.CPU87.RES:Rescheduling_interrupts
    984596 ± 68%     -88.9%     109768 ± 21%  interrupts.CPU88.CAL:Function_call_interrupts
    616449 ± 34%     -81.4%     114878 ± 23%  interrupts.CPU88.RES:Rescheduling_interrupts
    957564 ± 70%     -89.1%     104327 ± 25%  interrupts.CPU89.CAL:Function_call_interrupts
    685537 ± 46%     -83.7%     112008 ± 20%  interrupts.CPU89.RES:Rescheduling_interrupts
    505120 ± 35%     -82.9%      86292 ± 14%  interrupts.CPU9.CAL:Function_call_interrupts
    557760 ± 23%     -71.8%     157380 ± 35%  interrupts.CPU9.RES:Rescheduling_interrupts
    594960 ± 42%     -79.0%     124824 ± 16%  interrupts.CPU90.RES:Rescheduling_interrupts
    972190 ± 55%     -89.1%     105610 ± 56%  interrupts.CPU91.CAL:Function_call_interrupts
    737043 ± 39%     -84.3%     115867 ± 20%  interrupts.CPU91.RES:Rescheduling_interrupts
    870362 ± 47%     -89.2%      93606 ± 31%  interrupts.CPU92.CAL:Function_call_interrupts
    595357 ± 43%     -78.5%     128171 ± 24%  interrupts.CPU92.RES:Rescheduling_interrupts
    966701 ± 61%     -87.4%     122268 ± 34%  interrupts.CPU93.CAL:Function_call_interrupts
    612011 ± 28%     -77.7%     136594 ± 23%  interrupts.CPU93.RES:Rescheduling_interrupts
   1067318 ± 53%     -90.2%     104449 ± 19%  interrupts.CPU94.CAL:Function_call_interrupts
    699603 ± 32%     -83.8%     113008 ± 27%  interrupts.CPU94.RES:Rescheduling_interrupts
   1086161 ± 42%     -91.1%      96762 ± 20%  interrupts.CPU95.CAL:Function_call_interrupts
    685521 ± 49%     -82.9%     116896 ± 34%  interrupts.CPU95.RES:Rescheduling_interrupts
    432.00 ± 11%     -33.3%     288.00        interrupts.MCP:Machine_check_polls
  55987281 ± 21%     -77.1%   12807267 ± 11%  interrupts.RES:Rescheduling_interrupts
     11.78 ± 26%      -8.9        2.88 ±  7%  perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event
     11.12 ± 27%      -8.4        2.68 ±  8%  perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow
     11.08 ± 27%      -8.4        2.66 ±  8%  perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow
      9.99 ±  6%      -8.0        2.04 ±  2%  perf-profile.calltrace.cycles-pp.mod_objcg_state.kfree.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
      9.93 ± 20%      -6.4        3.49 ± 11%  perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
      9.78 ± 20%      -6.4        3.35 ± 15%  perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg.sock_sendmsg
      9.72 ± 20%      -6.4        3.31 ± 15%  perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable.unix_stream_sendmsg
      9.67 ± 20%      -6.4        3.29 ± 15%  perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable
      6.34 ± 22%      -6.2        0.17 ±223%  perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
      6.16 ±  7%      -6.2        0.00        perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
      7.75 ±  7%      -6.1        1.62        perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
     10.12 ± 20%      -6.1        4.02 ± 10%  perf-profile.calltrace.cycles-pp.sock_def_readable.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
      6.10 ±  7%      -6.1        0.00        perf-profile.calltrace.cycles-pp.mod_objcg_state.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
      8.02 ± 20%      -5.2        2.83 ±  6%  perf-profile.calltrace.cycles-pp.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
      7.96 ± 20%      -5.2        2.80 ±  6%  perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      7.90 ± 20%      -5.1        2.78 ±  6%  perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg
      5.74 ± 26%      -4.1        1.60 ±  7%  perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr
      5.71 ± 26%      -4.1        1.57 ±  7%  perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime
      3.90 ± 28%      -3.2        0.68 ±  8%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule
      3.90 ± 28%      -3.2        0.68 ±  8%  perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule
      3.78 ± 26%      -3.1        0.66 ±  8%  perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch
      3.88 ± 25%      -2.7        1.14 ±  6%  perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.schedule_timeout.unix_stream_read_generic
      3.62 ± 26%      -2.6        1.03 ±  7%  perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
      3.58 ± 26%      -2.6        1.01 ±  7%  perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
      3.47 ± 25%      -2.4        1.04 ±  6%  perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.schedule_timeout
      3.34 ± 25%      -2.3        1.00 ±  6%  perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
      3.23 ± 26%      -2.3        0.90 ±  8%  perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
      3.20 ± 25%      -2.2        0.96 ±  6%  perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule
      3.15 ± 25%      -2.2        0.92 ±  6%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair
      2.93 ± 20%      -2.2        0.72 ± 15%  perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
      3.02 ± 20%      -2.2        0.81 ± 14%  perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
      3.08 ± 25%      -2.2        0.89 ±  7%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity
      2.91 ± 27%      -2.1        0.79 ±  8%  perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up
      2.84 ± 26%      -2.0        0.80 ±  7%  perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
      2.79 ± 27%      -2.0        0.75 ±  8%  perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate
      2.78 ± 26%      -2.0        0.77 ±  7%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common
      2.74 ± 27%      -2.0        0.74 ±  8%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.enqueue_entity.enqueue_task_fair
      2.69 ± 27%      -2.0        0.73 ±  8%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.enqueue_entity
      2.67 ± 26%      -1.9        0.74 ±  8%  perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function
      2.66 ± 26%      -1.9        0.74 ±  8%  perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up
      2.66 ± 26%      -1.9        0.73 ±  8%  perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template
      2.51 ± 27%      -1.8        0.75 ±  7%  perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.schedule_timeout.unix_stream_read_generic
      2.31 ± 29%      -1.6        0.69 ±  8%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.schedule_timeout
      1.91 ± 37%      -1.4        0.47 ±100%  perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
      1.73 ± 38%      -1.3        0.46 ±100%  perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
      0.55 ±  4%      +0.2        0.77 ±  2%  perf-profile.calltrace.cycles-pp.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
      0.74 ± 11%      +0.3        1.07 ±  3%  perf-profile.calltrace.cycles-pp.get_mem_cgroup_from_objcg.obj_cgroup_uncharge_pages.drain_obj_stock.refill_obj_stock.kfree
     47.29            +0.4       47.67        perf-profile.calltrace.cycles-pp.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write
     47.56            +0.4       47.98        perf-profile.calltrace.cycles-pp.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write.ksys_write
     47.95            +0.4       48.38        perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
     47.74            +0.5       48.20        perf-profile.calltrace.cycles-pp.sock_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
      0.83 ±  9%      +0.5        1.37 ±  5%  perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
      0.00            +0.5        0.54 ±  4%  perf-profile.calltrace.cycles-pp.unix_write_space.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all
      0.76 ± 10%      +0.6        1.32 ±  5%  perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags
      0.00            +0.6        0.61 ±  2%  perf-profile.calltrace.cycles-pp.__check_object_size.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
      0.00            +0.6        0.62 ±  2%  perf-profile.calltrace.cycles-pp.skb_release_data.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      0.00            +0.6        0.62        perf-profile.calltrace.cycles-pp.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic
      0.58 ±  4%      +0.6        1.23 ±  3%  perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
      0.60 ±  4%      +0.7        1.26 ±  3%  perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
      0.75 ±  3%      +0.7        1.41 ±  3%  perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic
      1.11 ± 11%      +0.8        1.87 ±  4%  perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.drain_obj_stock.refill_obj_stock.kmem_cache_free.unix_stream_read_generic
      0.00            +0.8        0.80        perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
      1.18 ±  2%      +0.9        2.08 ±  2%  perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg
      1.19 ±  2%      +0.9        2.09 ±  2%  perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      1.20 ±  2%      +0.9        2.10 ±  2%  perf-profile.calltrace.cycles-pp.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
      0.83 ±  6%      +1.1        1.94 ±  2%  perf-profile.calltrace.cycles-pp.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      0.82 ±  6%      +1.1        1.93 ±  2%  perf-profile.calltrace.cycles-pp.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
      0.75 ±  6%      +1.1        1.86 ±  2%  perf-profile.calltrace.cycles-pp.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb
      0.78 ±  6%      +1.1        1.91 ±  2%  perf-profile.calltrace.cycles-pp.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic
      0.00            +1.1        1.14 ±  3%  perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
      0.00            +1.2        1.18 ±  6%  perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc_node.__alloc_skb
      0.00            +1.2        1.22 ±  8%  perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.drain_obj_stock.refill_obj_stock.kmem_cache_free
      0.00            +1.5        1.46 ±  7%  perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.drain_obj_stock.refill_obj_stock.kfree
      1.44 ± 11%      +1.7        3.12 ±  9%  perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.drain_obj_stock.refill_obj_stock.kfree.consume_skb
      2.53 ± 21%      +1.7        4.22        perf-profile.calltrace.cycles-pp.refill_obj_stock.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      1.46 ± 12%      +1.8        3.22 ± 11%  perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
     37.73            +1.8       39.52        perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
     37.92            +1.8       39.74        perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.51 ± 11%      +1.9        3.38 ±  5%  perf-profile.calltrace.cycles-pp.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
     44.66            +1.9       46.61        perf-profile.calltrace.cycles-pp.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read
     44.74            +2.0       46.71        perf-profile.calltrace.cycles-pp.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read
     45.13            +2.1       47.21        perf-profile.calltrace.cycles-pp.sock_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64
     45.27            +2.1       47.37        perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +2.2        2.18 ±  9%  perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.drain_obj_stock.refill_obj_stock
     17.32 ±  7%      +2.2       19.57        perf-profile.calltrace.cycles-pp.kfree.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      0.18 ±141%      +2.5        2.63 ± 13%  perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve
      1.59 ± 11%      +2.6        4.14        perf-profile.calltrace.cycles-pp.drain_obj_stock.refill_obj_stock.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg
      3.15 ± 17%      +2.6        5.72 ±  2%  perf-profile.calltrace.cycles-pp.refill_obj_stock.kfree.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
     17.76 ±  6%      +3.0       20.73        perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
      2.38 ± 16%      +3.2        5.57        perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
      2.21 ± 20%      +3.4        5.62 ±  2%  perf-profile.calltrace.cycles-pp.drain_obj_stock.refill_obj_stock.kfree.consume_skb.unix_stream_read_generic
     18.88 ±  6%      +4.4       23.30        perf-profile.calltrace.cycles-pp.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
     34.61 ±  6%      +4.5       39.16        perf-profile.calltrace.cycles-pp.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
     34.65 ±  6%      +4.6       39.22        perf-profile.calltrace.cycles-pp.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
     35.25 ±  6%      +5.2       40.41        perf-profile.calltrace.cycles-pp.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
      5.31 ±  7%      +7.2       12.52 ±  2%  perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
     30.03 ±  6%     -24.9        5.10        perf-profile.children.cycles-pp.mod_objcg_state
     17.84 ±  8%     -17.6        0.26 ± 15%  perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
     17.60 ±  8%     -17.4        0.18 ± 22%  perf-profile.children.cycles-pp.__mod_memcg_state
     14.38 ± 20%      -9.2        5.19 ±  3%  perf-profile.children.cycles-pp.perf_tp_event
     14.00 ± 20%      -9.0        5.03 ±  4%  perf-profile.children.cycles-pp.perf_swevent_overflow
     13.95 ± 20%      -9.0        4.99 ±  4%  perf-profile.children.cycles-pp.__perf_event_overflow
     13.86 ± 20%      -8.9        4.93 ±  4%  perf-profile.children.cycles-pp.perf_event_output_forward
     13.57 ± 20%      -8.8        4.79 ±  3%  perf-profile.children.cycles-pp.perf_prepare_sample
     12.77 ± 21%      -8.3        4.45 ±  4%  perf-profile.children.cycles-pp.perf_callchain
     12.72 ± 21%      -8.3        4.42 ±  4%  perf-profile.children.cycles-pp.get_perf_callchain
     11.23 ± 20%      -7.2        4.04 ±  4%  perf-profile.children.cycles-pp.schedule
     11.22 ± 20%      -7.2        4.06 ±  4%  perf-profile.children.cycles-pp.__schedule
     10.15 ± 20%      -6.2        3.97 ± 10%  perf-profile.children.cycles-pp.__wake_up_common_lock
     10.12 ± 20%      -6.1        4.02 ± 10%  perf-profile.children.cycles-pp.sock_def_readable
     10.00 ± 20%      -6.1        3.91 ± 10%  perf-profile.children.cycles-pp.__wake_up_common
      9.93 ± 20%      -6.1        3.87 ± 10%  perf-profile.children.cycles-pp.autoremove_wake_function
      9.88 ± 20%      -6.0        3.84 ± 10%  perf-profile.children.cycles-pp.try_to_wake_up
      8.18 ± 21%      -5.4        2.82 ±  4%  perf-profile.children.cycles-pp.perf_callchain_kernel
      8.23 ± 19%      -4.7        3.48 ±  2%  perf-profile.children.cycles-pp.schedule_timeout
      7.34 ± 19%      -4.4        2.97 ±  2%  perf-profile.children.cycles-pp.update_curr
      6.89 ± 19%      -4.1        2.80 ±  2%  perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
      6.22 ± 22%      -4.1        2.14 ±  4%  perf-profile.children.cycles-pp.unwind_next_frame
      4.78 ± 22%      -3.3        1.46 ±  6%  perf-profile.children.cycles-pp.perf_trace_sched_switch
      4.28 ± 20%      -2.8        1.49 ±  4%  perf-profile.children.cycles-pp.perf_callchain_user
      3.56 ± 21%      -2.7        0.91 ± 11%  perf-profile.children.cycles-pp.exit_to_user_mode_prepare
      3.67 ± 21%      -2.6        1.02 ± 10%  perf-profile.children.cycles-pp.syscall_exit_to_user_mode
      3.99 ± 20%      -2.6        1.40 ±  4%  perf-profile.children.cycles-pp.__get_user_nocheck_8
      3.98 ± 20%      -2.5        1.46 ±  3%  perf-profile.children.cycles-pp.ttwu_do_activate
      3.94 ± 20%      -2.5        1.43 ±  3%  perf-profile.children.cycles-pp.enqueue_task_fair
      4.13 ± 19%      -2.5        1.66 ±  2%  perf-profile.children.cycles-pp.dequeue_task_fair
      3.55 ± 21%      -2.3        1.28 ±  3%  perf-profile.children.cycles-pp.enqueue_entity
      3.79 ± 19%      -2.3        1.52 ±  2%  perf-profile.children.cycles-pp.dequeue_entity
      3.35 ± 22%      -2.2        1.15 ±  3%  perf-profile.children.cycles-pp.__unwind_start
      3.09 ± 20%      -1.9        1.15 ±  3%  perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
      2.48 ± 20%      -1.6        0.84 ±  4%  perf-profile.children.cycles-pp.asm_exc_page_fault
      1.76 ± 20%      -1.1        0.61 ±  3%  perf-profile.children.cycles-pp.__orc_find
      0.96 ± 20%      -0.6        0.32 ±  3%  perf-profile.children.cycles-pp.unwind_get_return_address
      0.97 ± 23%      -0.6        0.32 ±  4%  perf-profile.children.cycles-pp.orc_find
      0.77 ± 20%      -0.5        0.25 ±  4%  perf-profile.children.cycles-pp.__kernel_text_address
      0.78 ± 18%      -0.5        0.28 ±  3%  perf-profile.children.cycles-pp.exc_page_fault
      0.70 ± 20%      -0.4        0.27 ±  4%  perf-profile.children.cycles-pp.native_irq_return_iret
      0.65 ± 22%      -0.4        0.23 ±  3%  perf-profile.children.cycles-pp.pick_next_task_fair
      0.63 ± 19%      -0.4        0.21 ±  4%  perf-profile.children.cycles-pp.kernelmode_fixup_or_oops
      0.60 ± 20%      -0.4        0.19 ±  4%  perf-profile.children.cycles-pp.kernel_text_address
      0.61 ± 22%      -0.4        0.20 ±  6%  perf-profile.children.cycles-pp.stack_access_ok
      0.55 ± 20%      -0.4        0.18 ±  3%  perf-profile.children.cycles-pp.fixup_exception
      0.48 ± 20%      -0.3        0.16 ±  4%  perf-profile.children.cycles-pp.search_exception_tables
      0.47 ± 19%      -0.3        0.15 ±  3%  perf-profile.children.cycles-pp.search_extable
      0.44 ± 20%      -0.3        0.15 ±  5%  perf-profile.children.cycles-pp.bsearch
      0.52 ± 16%      -0.3        0.25 ±  6%  perf-profile.children.cycles-pp.switch_mm_irqs_off
      0.46 ± 20%      -0.3        0.19 ±  3%  perf-profile.children.cycles-pp.update_load_avg
      0.44 ± 16%      -0.2        0.20 ±  2%  perf-profile.children.cycles-pp.__perf_event_header__init_id
      0.30 ± 19%      -0.2        0.10 ±  4%  perf-profile.children.cycles-pp.cmp_ex_search
      0.36 ± 17%      -0.2        0.17 ±  6%  perf-profile.children.cycles-pp.switch_fpu_return
      0.25 ± 25%      -0.2        0.06 ±  9%  perf-profile.children.cycles-pp.__enqueue_entity
      0.28 ± 20%      -0.2        0.09 ±  5%  perf-profile.children.cycles-pp.load_new_mm_cr3
      0.26 ± 31%      -0.2        0.08 ± 36%  perf-profile.children.cycles-pp.cpumask_next_wrap
      0.26 ± 15%      -0.2        0.09 ±  4%  perf-profile.children.cycles-pp.reweight_entity
      0.23 ± 20%      -0.1        0.09 ±  5%  perf-profile.children.cycles-pp.__switch_to_asm
      0.18 ± 23%      -0.1        0.04 ± 44%  perf-profile.children.cycles-pp.finish_task_switch
      0.21 ± 20%      -0.1        0.08 ±  6%  perf-profile.children.cycles-pp.perf_output_begin_forward
      0.21 ± 22%      -0.1        0.08 ±  8%  perf-profile.children.cycles-pp.ftrace_graph_ret_addr
      0.23 ± 16%      -0.1        0.11        perf-profile.children.cycles-pp.perf_event_pid_type
      0.20 ± 32%      -0.1        0.08 ± 20%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      0.18 ± 21%      -0.1        0.07 ± 10%  perf-profile.children.cycles-pp.update_rq_clock
      0.20 ± 12%      -0.1        0.08 ±  5%  perf-profile.children.cycles-pp.__switch_to
      0.20 ± 20%      -0.1        0.09 ±  4%  perf-profile.children.cycles-pp.sched_clock_cpu
      0.19 ± 20%      -0.1        0.08        perf-profile.children.cycles-pp.__update_load_avg_se
      0.15 ± 13%      -0.1        0.05 ±  8%  perf-profile.children.cycles-pp.ttwu_do_wakeup
      0.17 ± 18%      -0.1        0.08 ±  6%  perf-profile.children.cycles-pp.native_sched_clock
      0.19 ± 14%      -0.1        0.10 ±  4%  perf-profile.children.cycles-pp.__task_pid_nr_ns
      0.13 ± 17%      -0.1        0.05 ± 45%  perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
      0.18 ± 15%      -0.1        0.10 ±  4%  perf-profile.children.cycles-pp.set_next_entity
      0.13 ± 18%      -0.1        0.05 ±  7%  perf-profile.children.cycles-pp.cpuacct_charge
      0.16 ± 17%      -0.1        0.10 ±  3%  perf-profile.children.cycles-pp.prepare_to_wait
      0.13 ± 16%      -0.1        0.08 ±  7%  perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
      0.10 ±  9%      -0.0        0.06 ±  7%  perf-profile.children.cycles-pp.update_cfs_group
      0.09 ± 21%      -0.0        0.06 ±  8%  perf-profile.children.cycles-pp.get_stack_info
      0.09 ±  6%      -0.0        0.06 ±  6%  perf-profile.children.cycles-pp.security_socket_getpeersec_dgram
      0.15 ±  6%      +0.0        0.17 ±  3%  perf-profile.children.cycles-pp.__cond_resched
      0.09 ±  5%      +0.0        0.12 ±  4%  perf-profile.children.cycles-pp.wait_for_unix_gc
      0.22 ±  5%      +0.0        0.26 ±  2%  perf-profile.children.cycles-pp.__virt_addr_valid
      0.21 ±  5%      +0.0        0.25 ±  2%  perf-profile.children.cycles-pp.security_socket_recvmsg
      0.19 ±  4%      +0.0        0.23        perf-profile.children.cycles-pp.__might_sleep
      0.20 ±  3%      +0.0        0.24 ±  2%  perf-profile.children.cycles-pp.security_socket_sendmsg
      0.02 ±141%      +0.0        0.06 ±  7%  perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
      0.32 ±  4%      +0.1        0.38        perf-profile.children.cycles-pp.aa_sk_perm
      0.24 ±  5%      +0.1        0.30 ±  2%  perf-profile.children.cycles-pp.sock_recvmsg
      0.55 ±  4%      +0.1        0.60 ±  2%  perf-profile.children.cycles-pp.security_file_permission
      0.00            +0.1        0.06 ±  8%  perf-profile.children.cycles-pp.syscall_enter_from_user_mode
      0.28 ±  4%      +0.1        0.34 ±  3%  perf-profile.children.cycles-pp.___might_sleep
      0.10 ± 11%      +0.1        0.17 ±  4%  perf-profile.children.cycles-pp.refill_stock
      0.34 ±  6%      +0.1        0.41 ±  2%  perf-profile.children.cycles-pp.__fget_light
      0.22 ±  6%      +0.1        0.30 ±  4%  perf-profile.children.cycles-pp.fsnotify
      0.07 ± 15%      +0.1        0.15 ±  2%  perf-profile.children.cycles-pp.drain_stock
      0.38 ±  5%      +0.1        0.46 ±  2%  perf-profile.children.cycles-pp.__fdget_pos
      0.09 ±  9%      +0.1        0.17 ±  2%  perf-profile.children.cycles-pp.__ksize
      0.18 ±  4%      +0.1        0.27 ±  3%  perf-profile.children.cycles-pp.__entry_text_start
      0.18 ±  6%      +0.1        0.28 ±  2%  perf-profile.children.cycles-pp.__check_heap_object
      0.04 ± 71%      +0.1        0.14 ±  4%  perf-profile.children.cycles-pp.try_charge
      0.22 ±  5%      +0.1        0.33 ±  2%  perf-profile.children.cycles-pp.copyin
      0.37 ±  4%      +0.2        0.52 ±  3%  perf-profile.children.cycles-pp._copy_from_iter
      0.31 ±  3%      +0.2        0.47 ±  3%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      0.22 ±  3%      +0.2        0.39 ±  3%  perf-profile.children.cycles-pp.mutex_lock
      0.09 ± 10%      +0.2        0.28 ±  5%  perf-profile.children.cycles-pp.get_partial_node
      0.13 ±  6%      +0.2        0.32 ±  4%  perf-profile.children.cycles-pp.mutex_unlock
      0.16 ±  4%      +0.2        0.35 ±  4%  perf-profile.children.cycles-pp.skb_queue_tail
      0.67 ±  5%      +0.3        0.95 ±  2%  perf-profile.children.cycles-pp.skb_copy_datagram_from_iter
      0.16 ±  6%      +0.3        0.44 ±  5%  perf-profile.children.cycles-pp.___slab_alloc
      0.16 ±  7%      +0.3        0.45 ±  4%  perf-profile.children.cycles-pp.__slab_alloc
      0.47 ±  3%      +0.3        0.76 ±  2%  perf-profile.children.cycles-pp.simple_copy_to_iter
      0.30 ±  2%      +0.3        0.60 ±  2%  perf-profile.children.cycles-pp.skb_set_owner_w
      0.13 ±  6%      +0.3        0.44 ±  4%  perf-profile.children.cycles-pp.skb_unlink
      0.32 ±  5%      +0.3        0.66 ±  3%  perf-profile.children.cycles-pp.unix_write_space
      0.21 ± 10%      +0.4        0.56 ±  3%  perf-profile.children.cycles-pp.__build_skb_around
      0.73 ±  3%      +0.4        1.11        perf-profile.children.cycles-pp.__check_object_size
     47.31            +0.4       47.69        perf-profile.children.cycles-pp.unix_stream_sendmsg
      0.44 ±  5%      +0.4        0.84 ±  4%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
     47.57            +0.4       48.00        perf-profile.children.cycles-pp.sock_sendmsg
     47.96            +0.4       48.39        perf-profile.children.cycles-pp.new_sync_write
     48.51            +0.4       48.96        perf-profile.children.cycles-pp.vfs_write
     47.74            +0.5       48.20        perf-profile.children.cycles-pp.sock_write_iter
      0.28 ± 13%      +0.5        0.76        perf-profile.children.cycles-pp.skb_release_data
     48.76            +0.5       49.27        perf-profile.children.cycles-pp.ksys_write
      0.28 ±  6%      +0.5        0.80 ±  2%  perf-profile.children.cycles-pp.__slab_free
      0.74 ±  4%      +0.8        1.53 ±  3%  perf-profile.children.cycles-pp.copyout
      0.91 ±  3%      +0.8        1.71 ±  2%  perf-profile.children.cycles-pp._copy_to_iter
      0.92 ±  3%      +0.9        1.83 ±  2%  perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      1.43 ±  2%      +1.1        2.52 ±  2%  perf-profile.children.cycles-pp.__skb_datagram_iter
      1.44 ±  2%      +1.1        2.54 ±  2%  perf-profile.children.cycles-pp.skb_copy_datagram_iter
      1.45 ±  2%      +1.1        2.56 ±  2%  perf-profile.children.cycles-pp.unix_stream_read_actor
      0.34 ± 13%      +1.2        1.53 ±  7%  perf-profile.children.cycles-pp.propagate_protected_usage
      1.36 ±  7%      +1.3        2.63 ±  3%  perf-profile.children.cycles-pp._raw_spin_lock
      1.01 ±  6%      +1.4        2.36 ±  2%  perf-profile.children.cycles-pp.skb_release_all
      1.00 ±  6%      +1.4        2.35 ±  2%  perf-profile.children.cycles-pp.skb_release_head_state
      0.91 ±  7%      +1.4        2.26 ±  2%  perf-profile.children.cycles-pp.sock_wfree
      0.95 ±  6%      +1.4        2.33 ±  2%  perf-profile.children.cycles-pp.unix_destruct_scm
     44.68            +2.0       46.63        perf-profile.children.cycles-pp.unix_stream_read_generic
     44.74            +2.0       46.71        perf-profile.children.cycles-pp.unix_stream_recvmsg
     45.13            +2.1       47.21        perf-profile.children.cycles-pp.sock_read_iter
     45.27            +2.1       47.38        perf-profile.children.cycles-pp.new_sync_read
      0.58 ± 12%      +2.2        2.74 ±  9%  perf-profile.children.cycles-pp.page_counter_cancel
      2.86 ± 11%      +2.2        5.05 ±  5%  perf-profile.children.cycles-pp.obj_cgroup_charge
     45.86            +2.2       48.06        perf-profile.children.cycles-pp.vfs_read
      2.71 ± 12%      +2.2        4.91 ±  5%  perf-profile.children.cycles-pp.obj_cgroup_charge_pages
     46.09            +2.2       48.31        perf-profile.children.cycles-pp.ksys_read
     17.33 ±  7%      +2.2       19.57        perf-profile.children.cycles-pp.kfree
      3.13 ± 11%      +2.3        5.47 ±  3%  perf-profile.children.cycles-pp.obj_cgroup_uncharge_pages
      0.77 ± 12%      +2.6        3.39 ±  7%  perf-profile.children.cycles-pp.page_counter_uncharge
     17.80 ±  6%      +3.0       20.76        perf-profile.children.cycles-pp.kmem_cache_alloc_node
      0.89 ± 13%      +3.4        4.32 ±  6%  perf-profile.children.cycles-pp.page_counter_try_charge
      6.15 ± 11%      +3.8        9.95        perf-profile.children.cycles-pp.refill_obj_stock
     18.88 ±  6%      +4.4       23.30        perf-profile.children.cycles-pp.consume_skb
     34.62 ±  6%      +4.6       39.18        perf-profile.children.cycles-pp.__alloc_skb
     34.66 ±  6%      +4.6       39.22        perf-profile.children.cycles-pp.alloc_skb_with_frags
     35.25 ±  6%      +5.2       40.41        perf-profile.children.cycles-pp.sock_alloc_send_pskb
      4.45 ± 11%      +5.3        9.77        perf-profile.children.cycles-pp.drain_obj_stock
      8.01 ±  7%     +10.1       18.12 ±  2%  perf-profile.children.cycles-pp.get_obj_cgroup_from_current
     17.60 ±  8%     -17.4        0.18 ± 23%  perf-profile.self.cycles-pp.__mod_memcg_state
     11.95 ±  7%      -7.1        4.83        perf-profile.self.cycles-pp.mod_objcg_state
      2.76 ± 22%      -1.8        0.95 ±  4%  perf-profile.self.cycles-pp.unwind_next_frame
      1.64 ± 11%      -1.5        0.11 ±  6%  perf-profile.self.cycles-pp.refill_obj_stock
      2.30 ± 20%      -1.5        0.80 ±  4%  perf-profile.self.cycles-pp.__get_user_nocheck_8
      1.75 ± 20%      -1.1        0.61 ±  3%  perf-profile.self.cycles-pp.__orc_find
      0.80 ± 12%      -0.7        0.08 ± 19%  perf-profile.self.cycles-pp.obj_cgroup_charge_pages
      0.78 ± 12%      -0.7        0.08 ± 19%  perf-profile.self.cycles-pp.obj_cgroup_uncharge_pages
      0.92 ± 22%      -0.6        0.31 ±  3%  perf-profile.self.cycles-pp.orc_find
      0.70 ± 22%      -0.5        0.24 ±  5%  perf-profile.self.cycles-pp.perf_callchain_kernel
      0.70 ± 20%      -0.4        0.27 ±  4%  perf-profile.self.cycles-pp.native_irq_return_iret
      0.54 ± 23%      -0.4        0.18 ±  6%  perf-profile.self.cycles-pp.stack_access_ok
      0.48 ± 13%      -0.2        0.24 ±  6%  perf-profile.self.cycles-pp.__schedule
      0.30 ± 22%      -0.2        0.09 ±  5%  perf-profile.self.cycles-pp.__unwind_start
      0.29 ± 19%      -0.2        0.09 ±  4%  perf-profile.self.cycles-pp.cmp_ex_search
      0.36 ± 16%      -0.2        0.16 ±  5%  perf-profile.self.cycles-pp.switch_fpu_return
      0.25 ± 25%      -0.2        0.06 ±  9%  perf-profile.self.cycles-pp.__enqueue_entity
      0.28 ± 20%      -0.2        0.09 ±  4%  perf-profile.self.cycles-pp.load_new_mm_cr3
      0.27 ± 22%      -0.2        0.08 ±  5%  perf-profile.self.cycles-pp.kernel_text_address
      0.26 ± 18%      -0.2        0.08 ± 11%  perf-profile.self.cycles-pp.perf_callchain_user
      0.23 ± 20%      -0.1        0.09 ±  5%  perf-profile.self.cycles-pp.__switch_to_asm
      0.22 ± 18%      -0.1        0.09 ±  4%  perf-profile.self.cycles-pp.perf_tp_event
      0.20 ± 22%      -0.1        0.06 ±  7%  perf-profile.self.cycles-pp.unwind_get_return_address
      0.23 ± 14%      -0.1        0.10 ±  4%  perf-profile.self.cycles-pp.perf_prepare_sample
      0.16 ± 28%      -0.1        0.04 ±100%  perf-profile.self.cycles-pp.cpumask_next_wrap
      0.20 ± 32%      -0.1        0.08 ± 20%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
      0.19 ± 20%      -0.1        0.07 ±  8%  perf-profile.self.cycles-pp.get_perf_callchain
      0.18 ± 20%      -0.1        0.06 ±  7%  perf-profile.self.cycles-pp.perf_output_begin_forward
      0.18 ± 19%      -0.1        0.07 ±  7%  perf-profile.self.cycles-pp.update_curr
      0.18 ± 20%      -0.1        0.08 ±  6%  perf-profile.self.cycles-pp.__update_load_avg_se
      0.16 ± 13%      -0.1        0.06 ±  9%  perf-profile.self.cycles-pp.__switch_to
      0.15 ± 20%      -0.1        0.05 ±  7%  perf-profile.self.cycles-pp.bsearch
      0.15 ± 12%      -0.1        0.05 ±  7%  perf-profile.self.cycles-pp.reweight_entity
      0.18 ± 24%      -0.1        0.08 ±  7%  perf-profile.self.cycles-pp.ftrace_graph_ret_addr
      0.12 ± 31%      -0.1        0.02 ± 99%  perf-profile.self.cycles-pp._find_next_bit
      0.16 ± 20%      -0.1        0.08 ±  6%  perf-profile.self.cycles-pp.native_sched_clock
      0.14 ± 22%      -0.1        0.05 ±  8%  perf-profile.self.cycles-pp.__kernel_text_address
      0.12 ± 20%      -0.1        0.04 ± 71%  perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
      0.14 ± 18%      -0.1        0.06        perf-profile.self.cycles-pp.update_load_avg
      0.18 ± 15%      -0.1        0.09 ±  5%  perf-profile.self.cycles-pp.__task_pid_nr_ns
      0.18 ± 12%      -0.1        0.10 ±  8%  perf-profile.self.cycles-pp.perf_trace_sched_switch
      0.23 ± 11%      -0.1        0.16 ±  7%  perf-profile.self.cycles-pp.switch_mm_irqs_off
      0.12 ± 19%      -0.1        0.05        perf-profile.self.cycles-pp.cpuacct_charge
      0.13 ± 19%      -0.1        0.06        perf-profile.self.cycles-pp.try_to_wake_up
      0.11 ± 15%      -0.1        0.05 ±  7%  perf-profile.self.cycles-pp.enqueue_entity
      0.12 ± 17%      -0.1        0.06 ±  7%  perf-profile.self.cycles-pp.enqueue_task_fair
      0.10 ± 17%      -0.1        0.04 ± 45%  perf-profile.self.cycles-pp.dequeue_task_fair
      0.11            -0.1        0.06 ±  9%  perf-profile.self.cycles-pp.__mod_memcg_lruvec_state
      0.10 ± 10%      -0.0        0.06 ±  7%  perf-profile.self.cycles-pp.update_cfs_group
      0.09 ±  9%      -0.0        0.05 ± 13%  perf-profile.self.cycles-pp.perf_event_output_forward
      0.08 ± 14%      -0.0        0.06 ±  8%  perf-profile.self.cycles-pp.prepare_to_wait
      0.08 ±  6%      -0.0        0.06        perf-profile.self.cycles-pp.rcu_read_unlock_strict
      0.06 ± 11%      +0.0        0.09 ±  4%  perf-profile.self.cycles-pp.security_file_permission
      0.18 ±  7%      +0.0        0.21 ±  4%  perf-profile.self.cycles-pp.sock_write_iter
      0.21 ±  4%      +0.0        0.25 ±  3%  perf-profile.self.cycles-pp.__virt_addr_valid
      0.08 ±  8%      +0.0        0.12 ±  4%  perf-profile.self.cycles-pp._copy_from_iter
      0.17 ±  5%      +0.0        0.20 ±  2%  perf-profile.self.cycles-pp.__might_sleep
      0.02 ±141%      +0.0        0.06 ±  7%  perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
      0.01 ±223%      +0.0        0.06 ±  8%  perf-profile.self.cycles-pp.syscall_exit_to_user_mode
      0.18 ±  5%      +0.0        0.23 ±  2%  perf-profile.self.cycles-pp.aa_sk_perm
      0.07            +0.0        0.12 ±  4%  perf-profile.self.cycles-pp.obj_cgroup_charge
      0.01 ±223%      +0.0        0.06 ±  6%  perf-profile.self.cycles-pp.unix_destruct_scm
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.__fdget_pos
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.sock_recvmsg
      0.00            +0.1        0.05 ±  7%  perf-profile.self.cycles-pp.skb_copy_datagram_from_iter
      0.15 ±  5%      +0.1        0.20 ±  3%  perf-profile.self.cycles-pp.sock_read_iter
      0.27 ±  3%      +0.1        0.33 ±  4%  perf-profile.self.cycles-pp.___might_sleep
      0.34 ±  6%      +0.1        0.40 ±  3%  perf-profile.self.cycles-pp.__fget_light
      0.22 ±  6%      +0.1        0.28 ±  3%  perf-profile.self.cycles-pp.fsnotify
      0.10 ±  4%      +0.1        0.18 ±  4%  perf-profile.self.cycles-pp.unix_write_space
      0.15 ±  4%      +0.1        0.22 ±  3%  perf-profile.self.cycles-pp.__entry_text_start
      0.08 ±  8%      +0.1        0.17        perf-profile.self.cycles-pp.__ksize
      0.18 ±  5%      +0.1        0.27 ±  2%  perf-profile.self.cycles-pp.__check_heap_object
      0.07 ± 11%      +0.1        0.16 ±  2%  perf-profile.self.cycles-pp.___slab_alloc
      0.11 ±  3%      +0.1        0.20 ±  3%  perf-profile.self.cycles-pp.consume_skb
      0.00            +0.1        0.10 ±  7%  perf-profile.self.cycles-pp.get_partial_node
      0.19 ±  5%      +0.1        0.32 ±  2%  perf-profile.self.cycles-pp.__alloc_skb
      0.14 ±  5%      +0.1        0.29 ±  2%  perf-profile.self.cycles-pp.mutex_lock
      0.31 ±  3%      +0.2        0.47 ±  3%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      0.12 ±  6%      +0.2        0.31 ±  5%  perf-profile.self.cycles-pp.mutex_unlock
      0.26 ±  4%      +0.3        0.51 ±  3%  perf-profile.self.cycles-pp.__check_object_size
      0.30 ±  2%      +0.3        0.59 ±  2%  perf-profile.self.cycles-pp.skb_set_owner_w
      0.15 ± 11%      +0.3        0.46 ±  4%  perf-profile.self.cycles-pp.__build_skb_around
      0.18 ±  3%      +0.3        0.53 ±  5%  perf-profile.self.cycles-pp.sock_def_readable
      0.57 ±  4%      +0.4        0.96 ±  2%  perf-profile.self.cycles-pp.unix_stream_read_generic
      0.43 ±  6%      +0.4        0.84 ±  4%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.28 ± 13%      +0.5        0.76        perf-profile.self.cycles-pp.skb_release_data
      0.28 ±  6%      +0.5        0.79 ±  2%  perf-profile.self.cycles-pp.__slab_free
      5.10 ±  6%      +0.8        5.86 ±  3%  perf-profile.self.cycles-pp.kmem_cache_alloc_node
      0.91 ±  3%      +0.9        1.82 ±  3%  perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      0.58 ±  9%      +1.0        1.58 ±  2%  perf-profile.self.cycles-pp.sock_wfree
      0.34 ± 13%      +1.2        1.52 ±  7%  perf-profile.self.cycles-pp.propagate_protected_usage
      1.15 ±  5%      +1.4        2.53 ±  2%  perf-profile.self.cycles-pp._raw_spin_lock
      0.58 ± 12%      +2.2        2.74 ±  9%  perf-profile.self.cycles-pp.page_counter_cancel
      5.49 ±  6%      +2.2        7.66 ±  2%  perf-profile.self.cycles-pp.__kmalloc_node_track_caller
      0.73 ± 13%      +2.7        3.39 ±  8%  perf-profile.self.cycles-pp.page_counter_try_charge
      1.32 ± 11%      +2.9        4.26 ±  3%  perf-profile.self.cycles-pp.drain_obj_stock
      3.94 ±  7%      +3.9        7.86        perf-profile.self.cycles-pp.kmem_cache_free
      3.97 ±  7%      +7.3       11.30 ±  2%  perf-profile.self.cycles-pp.kfree
      7.90 ±  7%     +10.0       17.91 ±  2%  perf-profile.self.cycles-pp.get_obj_cgroup_from_current



***************************************************************************************************
lkp-skl-fpga01: 104 threads 2 sockets Skylake with 192G memory
=========================================================================================
compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
  gcc-9/performance/socket/4/x86_64-rhel-8.3/threads/100%/debian-10.4-x86_64-20200603.cgz/lkp-skl-fpga01/hackbench/0x2006a0a

commit: 
  fdbcb2a6d6 ("mm/memcg: move mod_objcg_state() to memcontrol.c")
  68ac5b3c8d ("mm/memcg: cache vmstat data in percpu memcg_stock_pcp")

fdbcb2a6d6778e0b 68ac5b3c8db2fda00af594eca41 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
     58594 ±  2%     +40.7%      82459        hackbench.throughput
    876.58           -31.4%     601.44        hackbench.time.elapsed_time
    876.58           -31.4%     601.44        hackbench.time.elapsed_time.max
 2.049e+09 ±  2%     +14.4%  2.344e+09 ±  3%  hackbench.time.involuntary_context_switches
     80437           -33.3%      53613        hackbench.time.system_time
      9246           -15.3%       7833        hackbench.time.user_time
 3.259e+09            +7.2%  3.493e+09 ±  2%  hackbench.time.voluntary_context_switches
   7604601 ± 19%     -25.2%    5686636 ± 13%  cpuidle.C1.usage
    926.06           -29.7%     651.41        uptime.boot
      2.47 ±  6%      +1.0        3.45 ±  4%  mpstat.cpu.all.idle%
      9.97            +2.2       12.20        mpstat.cpu.all.usr%
   6057764 ±  3%     +59.8%    9678625 ±  2%  vmstat.system.cs
    730949           +30.7%     955036 ±  3%  vmstat.system.in
   7598688 ± 19%     -25.3%    5677672 ± 13%  turbostat.C1
      0.44 ±  2%     +54.6%       0.68 ±  2%  turbostat.CPU%c1
 6.632e+08 ±  2%     -10.2%  5.956e+08 ±  3%  turbostat.IRQ
      4.86 ± 20%     +52.8       57.63 ± 25%  turbostat.PKG_%
    355.39            +4.7%     372.20        turbostat.PkgWatt
    123.74            -4.4%     118.32        turbostat.RAMWatt
     45898           -14.9%      39056 ±  3%  slabinfo.kmalloc-512.active_objs
      1681 ±  2%     -16.9%       1397 ±  3%  slabinfo.kmalloc-512.active_slabs
     53826 ±  2%     -16.9%      44743 ±  3%  slabinfo.kmalloc-512.num_objs
      1681 ±  2%     -16.9%       1397 ±  3%  slabinfo.kmalloc-512.num_slabs
     36866 ±  2%     -16.9%      30624 ±  3%  slabinfo.skbuff_head_cache.active_objs
      1381 ±  2%     -19.1%       1117 ±  4%  slabinfo.skbuff_head_cache.active_slabs
     44208 ±  2%     -19.1%      35782 ±  4%  slabinfo.skbuff_head_cache.num_objs
      1381 ±  2%     -19.1%       1117 ±  4%  slabinfo.skbuff_head_cache.num_slabs
    191781 ±  3%     -13.1%     166659 ± 11%  proc-vmstat.nr_active_anon
     71587            +2.5%      73405 ±  2%  proc-vmstat.nr_anon_pages
    812663            -2.2%     794697        proc-vmstat.nr_file_pages
    220808            -8.5%     201937        proc-vmstat.nr_shmem
    191781 ±  3%     -13.1%     166659 ± 11%  proc-vmstat.nr_zone_active_anon
   2969387            +8.5%    3223037 ±  2%  proc-vmstat.numa_hit
   2875316            +8.8%    3128922 ±  2%  proc-vmstat.numa_local
    245303            -1.6%     241489        proc-vmstat.pgactivate
   5358774 ±  2%     +24.1%    6652818 ±  3%  proc-vmstat.pgalloc_normal
   2267435 ±  3%     -20.8%    1795974 ±  3%  proc-vmstat.pgfault
   5077096 ±  2%     +25.5%    6372725 ±  3%  proc-vmstat.pgfree
    109324 ±  2%     -28.6%      78108        proc-vmstat.pgreuse
      0.00 ±141%  +1.5e+05%       6.71 ± 71%  perf-sched.sch_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read
      1.22 ± 11%     -58.1%       0.51 ± 37%  perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.09 ±127%    +119.6%       0.19 ± 51%  perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
      0.00 ±141%  +1.5e+05%       6.72 ± 71%  perf-sched.sch_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read
     15588 ±  6%     -26.1%      11520 ± 16%  perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
      7662 ± 28%     -89.5%     802.08 ±187%  perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
      2234 ±139%     -99.3%      14.55 ± 52%  perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg
     16317 ± 11%     -21.7%      12780 ± 13%  perf-sched.sch_delay.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      0.97 ±138%    +205.9%       2.98        perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
     16552 ±  9%     -21.9%      12925 ± 13%  perf-sched.total_sch_delay.max.ms
     33139 ±  9%     -21.9%      25866 ± 13%  perf-sched.total_wait_and_delay.max.ms
     16912 ±  9%     -19.3%      13649 ± 12%  perf-sched.total_wait_time.max.ms
    134.61 ± 15%     -29.2%      95.29 ± 22%  perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
     11728 ±  5%     -67.7%       3789 ± 46%  perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
     15675 ± 13%     -39.7%       9455 ± 32%  perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
     31257 ±  6%     -26.0%      23132 ± 16%  perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
     32739 ± 10%     -21.9%      25565 ± 13%  perf-sched.wait_and_delay.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
     15848 ±  7%     -23.2%      12170 ± 12%  perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
    384.57 ± 38%     -53.8%     177.56 ± 57%  perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
    134.61 ± 15%     -29.2%      95.29 ± 22%  perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr
     14420 ± 11%     -45.9%       7800 ± 32%  perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
      3754 ± 68%     -75.8%     909.20 ±144%  perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
     16912 ±  9%     -19.3%      13649 ± 12%  perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
     16327 ±  8%     -22.0%      12732 ±  9%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
     16535 ± 10%     -21.3%      13018 ± 14%  perf-sched.wait_time.max.ms.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
     15848 ±  7%     -23.2%      12170 ± 12%  perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
     14.02 ±  2%     -16.7%      11.68        perf-stat.i.MPKI
 1.293e+10           +47.8%   1.91e+10 ±  2%  perf-stat.i.branch-instructions
 2.072e+08           +50.3%  3.114e+08        perf-stat.i.branch-misses
     10.77 ±  5%      -4.6        6.20        perf-stat.i.cache-miss-rate%
  90573100 ±  6%     -29.7%   63695030 ±  4%  perf-stat.i.cache-misses
  8.59e+08           +23.6%  1.062e+09 ±  3%  perf-stat.i.cache-references
   6175726 ±  3%     +58.8%    9804802 ±  2%  perf-stat.i.context-switches
      4.79           -33.1%       3.20 ±  2%  perf-stat.i.cpi
    411441 ±  3%     +65.9%     682632 ±  6%  perf-stat.i.cpu-migrations
      3333 ±  6%     +52.4%       5079 ±  4%  perf-stat.i.cycles-between-cache-misses
      0.08            -0.0        0.08 ±  2%  perf-stat.i.dTLB-load-miss-rate%
  14103769           +39.0%   19602583        perf-stat.i.dTLB-load-misses
 1.813e+10           +45.7%  2.642e+10 ±  2%  perf-stat.i.dTLB-loads
      0.01 ±  4%      -0.0        0.01 ±  8%  perf-stat.i.dTLB-store-miss-rate%
 1.097e+10           +46.9%  1.612e+10        perf-stat.i.dTLB-stores
  17167651           +51.2%   25957899        perf-stat.i.iTLB-load-misses
   6491659           +70.1%   11045291 ±  6%  perf-stat.i.iTLB-loads
 6.344e+10           +46.4%   9.29e+10 ±  2%  perf-stat.i.instructions
      0.22           +46.6%       0.32 ±  2%  perf-stat.i.ipc
    411.82           +46.4%     602.85 ±  2%  perf-stat.i.metric.M/sec
      2569 ±  2%     +14.4%       2939 ±  3%  perf-stat.i.minor-faults
  25184851 ±  7%     -15.5%   21292414 ± 15%  perf-stat.i.node-load-misses
   2862951 ±  2%     -13.3%    2480788 ±  7%  perf-stat.i.node-loads
     90.43            +1.2       91.60        perf-stat.i.node-store-miss-rate%
   8837079 ±  4%     +17.3%   10368046 ±  6%  perf-stat.i.node-store-misses
      2570 ±  2%     +14.4%       2941 ±  3%  perf-stat.i.page-faults
     13.60           -16.1%      11.41        perf-stat.overall.MPKI
     10.66 ±  5%      -4.6        6.05        perf-stat.overall.cache-miss-rate%
      4.61           -32.5%       3.11 ±  2%  perf-stat.overall.cpi
      3189 ±  6%     +41.5%       4511 ±  3%  perf-stat.overall.cycles-between-cache-misses
      0.08            -0.0        0.07 ±  2%  perf-stat.overall.dTLB-load-miss-rate%
      0.01 ±  3%      -0.0        0.01 ±  9%  perf-stat.overall.dTLB-store-miss-rate%
     72.72            -2.4       70.35        perf-stat.overall.iTLB-load-miss-rate%
      3674            -2.8%       3571 ±  2%  perf-stat.overall.instructions-per-iTLB-miss
      0.22           +48.2%       0.32 ±  2%  perf-stat.overall.ipc
 1.272e+10           +49.1%  1.896e+10 ±  2%  perf-stat.ps.branch-instructions
 2.043e+08           +51.4%  3.094e+08        perf-stat.ps.branch-misses
  90522814 ±  5%     -29.7%   63654107 ±  3%  perf-stat.ps.cache-misses
 8.489e+08           +24.0%  1.053e+09 ±  3%  perf-stat.ps.cache-references
   6061055 ±  3%     +60.0%    9699719 ±  2%  perf-stat.ps.context-switches
    396845 ±  3%     +69.0%     670636 ±  6%  perf-stat.ps.cpu-migrations
  14020909           +39.7%   19593701        perf-stat.ps.dTLB-load-misses
 1.786e+10           +46.9%  2.624e+10 ±  2%  perf-stat.ps.dTLB-loads
 1.081e+10           +48.1%  1.601e+10        perf-stat.ps.dTLB-stores
  16997675           +52.0%   25833087        perf-stat.ps.iTLB-load-misses
   6375875           +71.0%   10905261 ±  6%  perf-stat.ps.iTLB-loads
 6.244e+10           +47.7%  9.225e+10 ±  2%  perf-stat.ps.instructions
      2525 ±  2%     +15.2%       2910 ±  3%  perf-stat.ps.minor-faults
  24698062 ±  6%     -14.9%   21009262 ± 15%  perf-stat.ps.node-load-misses
   2896962 ±  2%     -12.4%    2537682 ±  7%  perf-stat.ps.node-loads
   8640562 ±  4%     +19.0%   10280989 ±  6%  perf-stat.ps.node-store-misses
      2526 ±  2%     +15.2%       2912 ±  3%  perf-stat.ps.page-faults
      1755           -31.3%       1205        interrupts.9:IO-APIC.9-fasteoi.acpi
 1.303e+08 ± 13%     -31.0%   89891349 ± 18%  interrupts.CAL:Function_call_interrupts
   1756087           -31.3%    1205651        interrupts.CPU0.LOC:Local_timer_interrupts
      1755           -31.3%       1205        interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi
   1756083           -31.3%    1205641        interrupts.CPU1.LOC:Local_timer_interrupts
   1756283           -31.4%    1205637        interrupts.CPU10.LOC:Local_timer_interrupts
   3130211           +14.6%    3586704 ±  2%  interrupts.CPU10.RES:Rescheduling_interrupts
   1756032           -31.3%    1205570        interrupts.CPU100.LOC:Local_timer_interrupts
   2193714 ± 20%     -44.4%    1220194 ± 53%  interrupts.CPU101.CAL:Function_call_interrupts
   1756012           -31.3%    1205569        interrupts.CPU101.LOC:Local_timer_interrupts
   1713690 ±  5%     -36.5%    1088363 ± 44%  interrupts.CPU102.CAL:Function_call_interrupts
   1756044           -31.3%    1205539        interrupts.CPU102.LOC:Local_timer_interrupts
   1660404 ± 34%     -54.6%     753033 ± 38%  interrupts.CPU103.CAL:Function_call_interrupts
   1756050           -31.3%    1205534        interrupts.CPU103.LOC:Local_timer_interrupts
   1756187           -31.4%    1205604        interrupts.CPU11.LOC:Local_timer_interrupts
   2966878 ±  4%     +14.6%    3400360 ±  6%  interrupts.CPU11.RES:Rescheduling_interrupts
   1756030           -31.3%    1205609        interrupts.CPU12.LOC:Local_timer_interrupts
   2885830 ± 10%     +23.4%    3561488 ±  5%  interrupts.CPU12.RES:Rescheduling_interrupts
   1756038           -31.3%    1205642        interrupts.CPU13.LOC:Local_timer_interrupts
   1755967           -31.3%    1205608        interrupts.CPU14.LOC:Local_timer_interrupts
   2869980 ±  3%     +26.0%    3615978 ±  7%  interrupts.CPU14.RES:Rescheduling_interrupts
   1756056           -31.3%    1205614        interrupts.CPU15.LOC:Local_timer_interrupts
   2966138 ±  6%     +12.7%    3343478 ±  4%  interrupts.CPU15.RES:Rescheduling_interrupts
   1756030           -31.3%    1205612        interrupts.CPU16.LOC:Local_timer_interrupts
   3168638 ± 10%     +16.1%    3677516 ±  9%  interrupts.CPU16.RES:Rescheduling_interrupts
   1756080           -31.3%    1205608        interrupts.CPU17.LOC:Local_timer_interrupts
   2936901 ± 10%     +16.2%    3411368 ±  3%  interrupts.CPU17.RES:Rescheduling_interrupts
   1756057           -31.3%    1205588        interrupts.CPU18.LOC:Local_timer_interrupts
   2969000 ±  3%     +17.5%    3489665 ±  6%  interrupts.CPU18.RES:Rescheduling_interrupts
   1756069           -31.3%    1205594        interrupts.CPU19.LOC:Local_timer_interrupts
   2782993 ±  2%     +14.4%    3184389 ±  4%  interrupts.CPU19.RES:Rescheduling_interrupts
   1756028           -31.3%    1205645        interrupts.CPU2.LOC:Local_timer_interrupts
   1755968           -31.3%    1205599        interrupts.CPU20.LOC:Local_timer_interrupts
   2947698 ±  6%     +15.5%    3403935 ±  6%  interrupts.CPU20.RES:Rescheduling_interrupts
   1756098           -31.3%    1205604        interrupts.CPU21.LOC:Local_timer_interrupts
   1756007           -31.3%    1205655        interrupts.CPU22.LOC:Local_timer_interrupts
   3030421 ±  3%     +14.0%    3455529 ±  6%  interrupts.CPU22.RES:Rescheduling_interrupts
   1756298           -31.4%    1205628        interrupts.CPU23.LOC:Local_timer_interrupts
   3121273 ±  3%     +17.8%    3677020 ±  9%  interrupts.CPU23.RES:Rescheduling_interrupts
   1756010           -31.3%    1205621        interrupts.CPU24.LOC:Local_timer_interrupts
   1756021           -31.3%    1205631        interrupts.CPU25.LOC:Local_timer_interrupts
   4335710 ± 14%     -36.8%    2739263 ± 43%  interrupts.CPU26.CAL:Function_call_interrupts
   1756035           -31.3%    1205541        interrupts.CPU26.LOC:Local_timer_interrupts
   1756047           -31.3%    1205558        interrupts.CPU27.LOC:Local_timer_interrupts
   3134334 ±  3%     +14.7%    3593896 ±  7%  interrupts.CPU27.RES:Rescheduling_interrupts
   1775870 ± 12%     -44.5%     984752 ± 53%  interrupts.CPU28.CAL:Function_call_interrupts
   1756012           -31.3%    1205555        interrupts.CPU28.LOC:Local_timer_interrupts
   1756067           -31.3%    1205576        interrupts.CPU29.LOC:Local_timer_interrupts
   1755936           -31.3%    1205627        interrupts.CPU3.LOC:Local_timer_interrupts
   3100930 ±  3%     +15.9%    3595156 ±  5%  interrupts.CPU3.RES:Rescheduling_interrupts
   1756116           -31.3%    1205681        interrupts.CPU30.LOC:Local_timer_interrupts
   1756050           -31.3%    1205626        interrupts.CPU31.LOC:Local_timer_interrupts
   1756078           -31.4%    1205545        interrupts.CPU32.LOC:Local_timer_interrupts
   2028831 ± 12%     -52.4%     966184 ± 54%  interrupts.CPU33.CAL:Function_call_interrupts
   1756051           -31.3%    1205644        interrupts.CPU33.LOC:Local_timer_interrupts
   1756065           -31.3%    1205560        interrupts.CPU34.LOC:Local_timer_interrupts
   1700094 ± 22%     -45.2%     931953 ± 40%  interrupts.CPU35.CAL:Function_call_interrupts
   1756018           -31.3%    1205608        interrupts.CPU35.LOC:Local_timer_interrupts
   1756097           -31.3%    1205561        interrupts.CPU36.LOC:Local_timer_interrupts
   1900638 ± 12%     -58.7%     784528 ± 44%  interrupts.CPU37.CAL:Function_call_interrupts
   1756059           -31.3%    1205566        interrupts.CPU37.LOC:Local_timer_interrupts
   1661249 ± 18%     -48.4%     856586 ± 55%  interrupts.CPU38.CAL:Function_call_interrupts
   1756054           -31.3%    1205562        interrupts.CPU38.LOC:Local_timer_interrupts
   1756036           -31.3%    1205555        interrupts.CPU39.LOC:Local_timer_interrupts
   1756081           -31.3%    1205646        interrupts.CPU4.LOC:Local_timer_interrupts
   1756068           -31.3%    1205633        interrupts.CPU40.LOC:Local_timer_interrupts
   2224396 ±  4%     -56.3%     970965 ± 38%  interrupts.CPU41.CAL:Function_call_interrupts
   1756154           -31.3%    1205609        interrupts.CPU41.LOC:Local_timer_interrupts
   1756099           -31.3%    1205645        interrupts.CPU42.LOC:Local_timer_interrupts
   1570767 ± 25%     -43.5%     887763 ± 41%  interrupts.CPU43.CAL:Function_call_interrupts
   1756063           -31.3%    1205540        interrupts.CPU43.LOC:Local_timer_interrupts
   1899270 ± 32%     -47.8%     992296 ± 59%  interrupts.CPU44.CAL:Function_call_interrupts
   1756085           -31.3%    1205622        interrupts.CPU44.LOC:Local_timer_interrupts
   1849890 ±  9%     -41.6%    1079682 ± 54%  interrupts.CPU45.CAL:Function_call_interrupts
   1756067           -31.3%    1205568        interrupts.CPU45.LOC:Local_timer_interrupts
   1756083           -31.3%    1205556        interrupts.CPU46.LOC:Local_timer_interrupts
   2083619 ±  6%     -53.6%     966803 ± 52%  interrupts.CPU47.CAL:Function_call_interrupts
   1756159           -31.4%    1205560        interrupts.CPU47.LOC:Local_timer_interrupts
   3321506 ±  2%     +12.0%    3719213 ±  6%  interrupts.CPU47.RES:Rescheduling_interrupts
   1735617 ± 12%     -35.2%    1124501 ± 33%  interrupts.CPU48.CAL:Function_call_interrupts
   1756085           -31.3%    1205608        interrupts.CPU48.LOC:Local_timer_interrupts
   1756070           -31.3%    1205544        interrupts.CPU49.LOC:Local_timer_interrupts
   1756167           -31.3%    1205621        interrupts.CPU5.LOC:Local_timer_interrupts
   3027953           +15.0%    3481679 ±  5%  interrupts.CPU5.RES:Rescheduling_interrupts
   1756016           -31.3%    1205609        interrupts.CPU50.LOC:Local_timer_interrupts
   1588161 ± 21%     -45.2%     871046 ± 34%  interrupts.CPU51.CAL:Function_call_interrupts
   1756163           -31.3%    1205628        interrupts.CPU51.LOC:Local_timer_interrupts
   1755990           -31.3%    1205570        interrupts.CPU52.LOC:Local_timer_interrupts
   1756046           -31.3%    1205565        interrupts.CPU53.LOC:Local_timer_interrupts
   2716149 ±  7%     +21.0%    3287450 ±  9%  interrupts.CPU53.RES:Rescheduling_interrupts
   1756063           -31.3%    1205586        interrupts.CPU54.LOC:Local_timer_interrupts
   1756003           -31.3%    1205592        interrupts.CPU55.LOC:Local_timer_interrupts
   3041413 ±  5%     +21.0%    3681265 ±  4%  interrupts.CPU55.RES:Rescheduling_interrupts
   1756173           -31.3%    1205653        interrupts.CPU56.LOC:Local_timer_interrupts
   3120053 ±  8%     +11.9%    3492355 ±  5%  interrupts.CPU56.RES:Rescheduling_interrupts
   1756039           -31.3%    1205600        interrupts.CPU57.LOC:Local_timer_interrupts
   1756021           -31.3%    1205585        interrupts.CPU58.LOC:Local_timer_interrupts
   2920530 ±  7%     +12.7%    3292319 ±  6%  interrupts.CPU58.RES:Rescheduling_interrupts
   1756023           -31.3%    1205592        interrupts.CPU59.LOC:Local_timer_interrupts
   1756033           -31.3%    1205626        interrupts.CPU6.LOC:Local_timer_interrupts
   1756029           -31.3%    1205605        interrupts.CPU60.LOC:Local_timer_interrupts
   2573666 ±  7%     +34.0%    3447925 ± 11%  interrupts.CPU60.RES:Rescheduling_interrupts
   1756033           -31.3%    1205600        interrupts.CPU61.LOC:Local_timer_interrupts
   3064524 ±  2%     +20.4%    3688455 ± 10%  interrupts.CPU61.RES:Rescheduling_interrupts
   1756011           -31.3%    1205607        interrupts.CPU62.LOC:Local_timer_interrupts
   1756034           -31.3%    1205597        interrupts.CPU63.LOC:Local_timer_interrupts
   2857476 ±  6%     +16.5%    3329589 ±  6%  interrupts.CPU63.RES:Rescheduling_interrupts
   1756039           -31.3%    1205619        interrupts.CPU64.LOC:Local_timer_interrupts
   1756011           -31.3%    1205621        interrupts.CPU65.LOC:Local_timer_interrupts
   1756032           -31.3%    1205711        interrupts.CPU66.LOC:Local_timer_interrupts
   3025406 ±  6%     +18.9%    3597977 ± 10%  interrupts.CPU66.RES:Rescheduling_interrupts
   1756037           -31.3%    1205625        interrupts.CPU67.LOC:Local_timer_interrupts
   2925726 ±  4%     +16.7%    3415111 ±  5%  interrupts.CPU67.RES:Rescheduling_interrupts
   1756040           -31.3%    1205569        interrupts.CPU68.LOC:Local_timer_interrupts
   2892746 ±  5%     +18.1%    3416078 ±  8%  interrupts.CPU68.RES:Rescheduling_interrupts
   1755976           -31.3%    1205639        interrupts.CPU69.LOC:Local_timer_interrupts
   1756062           -31.3%    1205657        interrupts.CPU7.LOC:Local_timer_interrupts
   2757608 ±  2%     +19.8%    3303249 ±  5%  interrupts.CPU7.RES:Rescheduling_interrupts
   1756014           -31.3%    1205689        interrupts.CPU70.LOC:Local_timer_interrupts
   1756028           -31.3%    1205544        interrupts.CPU71.LOC:Local_timer_interrupts
   2874234 ±  3%     +16.5%    3349906 ±  4%  interrupts.CPU71.RES:Rescheduling_interrupts
   1756053           -31.3%    1205643        interrupts.CPU72.LOC:Local_timer_interrupts
   1756051           -31.3%    1205653        interrupts.CPU73.LOC:Local_timer_interrupts
   1756002           -31.3%    1205609        interrupts.CPU74.LOC:Local_timer_interrupts
   2994525 ±  9%     +17.2%    3510382 ±  9%  interrupts.CPU74.RES:Rescheduling_interrupts
   1756006           -31.3%    1205610        interrupts.CPU75.LOC:Local_timer_interrupts
   3070769 ±  3%     +20.0%    3684382 ±  5%  interrupts.CPU75.RES:Rescheduling_interrupts
   1756026           -31.3%    1205626        interrupts.CPU76.LOC:Local_timer_interrupts
   1756039           -31.3%    1205618        interrupts.CPU77.LOC:Local_timer_interrupts
   4859129 ± 10%     -52.9%    2288910 ± 46%  interrupts.CPU78.CAL:Function_call_interrupts
   1756010           -31.3%    1205573        interrupts.CPU78.LOC:Local_timer_interrupts
   2448504 ± 14%     -57.7%    1036604 ± 45%  interrupts.CPU79.CAL:Function_call_interrupts
   1756068           -31.3%    1205615        interrupts.CPU79.LOC:Local_timer_interrupts
   1756007           -31.3%    1205598        interrupts.CPU8.LOC:Local_timer_interrupts
   2944757 ±  3%     +21.9%    3589350 ±  7%  interrupts.CPU8.RES:Rescheduling_interrupts
   1704443 ± 12%     -51.3%     830489 ± 40%  interrupts.CPU80.CAL:Function_call_interrupts
   1756090           -31.3%    1205586        interrupts.CPU80.LOC:Local_timer_interrupts
   1756119           -31.4%    1205553        interrupts.CPU81.LOC:Local_timer_interrupts
   1951835 ± 20%     -47.0%    1034865 ± 32%  interrupts.CPU82.CAL:Function_call_interrupts
   1756134           -31.4%    1205574        interrupts.CPU82.LOC:Local_timer_interrupts
   1756078           -31.3%    1205620        interrupts.CPU83.LOC:Local_timer_interrupts
   1837507 ± 31%     -51.7%     886753 ± 50%  interrupts.CPU84.CAL:Function_call_interrupts
   1756088           -31.3%    1205595        interrupts.CPU84.LOC:Local_timer_interrupts
   2088115 ± 16%     -61.4%     805593 ± 48%  interrupts.CPU85.CAL:Function_call_interrupts
   1756056           -31.3%    1205568        interrupts.CPU85.LOC:Local_timer_interrupts
   2016702 ± 33%     -63.2%     741626 ± 43%  interrupts.CPU86.CAL:Function_call_interrupts
   1756042           -31.3%    1205556        interrupts.CPU86.LOC:Local_timer_interrupts
   1756147           -31.4%    1205588        interrupts.CPU87.LOC:Local_timer_interrupts
   3362776 ±  2%     +13.6%    3819022 ±  3%  interrupts.CPU87.RES:Rescheduling_interrupts
   1756143           -31.3%    1205634        interrupts.CPU88.LOC:Local_timer_interrupts
   3217725 ±  7%     +10.5%    3554878 ±  4%  interrupts.CPU88.RES:Rescheduling_interrupts
   1972919 ± 32%     -66.3%     663970 ± 44%  interrupts.CPU89.CAL:Function_call_interrupts
   1756070           -31.3%    1205618        interrupts.CPU89.LOC:Local_timer_interrupts
   3189844 ±  4%      +9.9%    3506043 ±  4%  interrupts.CPU89.RES:Rescheduling_interrupts
   1756034           -31.3%    1205624        interrupts.CPU9.LOC:Local_timer_interrupts
   1734960 ± 19%     -53.4%     808272 ± 42%  interrupts.CPU90.CAL:Function_call_interrupts
   1756122           -31.3%    1205711        interrupts.CPU90.LOC:Local_timer_interrupts
   1818256           -42.8%    1039860 ± 51%  interrupts.CPU91.CAL:Function_call_interrupts
   1756039           -31.3%    1205569        interrupts.CPU91.LOC:Local_timer_interrupts
   2060769 ± 12%     -52.1%     987566 ± 45%  interrupts.CPU92.CAL:Function_call_interrupts
   1756061           -31.3%    1205564        interrupts.CPU92.LOC:Local_timer_interrupts
   1978905 ±  9%     -51.0%     969386 ± 39%  interrupts.CPU93.CAL:Function_call_interrupts
   1756138           -31.4%    1205583        interrupts.CPU93.LOC:Local_timer_interrupts
   3177803 ±  8%     +19.6%    3802054 ±  9%  interrupts.CPU93.RES:Rescheduling_interrupts
   1756068           -31.3%    1205561        interrupts.CPU94.LOC:Local_timer_interrupts
   3054954 ±  8%     +19.8%    3658583 ± 16%  interrupts.CPU94.RES:Rescheduling_interrupts
   1765762 ± 24%     -56.9%     760270 ± 37%  interrupts.CPU95.CAL:Function_call_interrupts
   1756126           -31.4%    1205530        interrupts.CPU95.LOC:Local_timer_interrupts
   1756081           -31.3%    1205578        interrupts.CPU96.LOC:Local_timer_interrupts
   1756089           -31.3%    1205573        interrupts.CPU97.LOC:Local_timer_interrupts
   3015595 ±  2%     +13.8%    3430508 ±  7%  interrupts.CPU97.RES:Rescheduling_interrupts
   1912617 ± 22%     -51.6%     925102 ± 52%  interrupts.CPU98.CAL:Function_call_interrupts
   1756082           -31.3%    1205586        interrupts.CPU98.LOC:Local_timer_interrupts
   1907270 ± 22%     -43.2%    1082601 ± 43%  interrupts.CPU99.CAL:Function_call_interrupts
   1756075           -31.3%    1205570        interrupts.CPU99.LOC:Local_timer_interrupts
 1.826e+08           -31.3%  1.254e+08        interrupts.LOC:Local_timer_interrupts
    208.00           -50.0%     104.00        interrupts.MCP:Machine_check_polls
 3.278e+08            +9.8%  3.601e+08        interrupts.RES:Rescheduling_interrupts
      5.90 ± 20%      -4.3        1.57 ± 10%  perf-profile.calltrace.cycles-pp.mod_objcg_state.kfree.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
      3.67 ± 13%      -2.4        1.24 ±  9%  perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
     35.11 ±  4%      -1.9       33.25        perf-profile.calltrace.cycles-pp.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read
     35.32 ±  4%      -1.8       33.49        perf-profile.calltrace.cycles-pp.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read
     36.03 ±  3%      -1.7       34.36        perf-profile.calltrace.cycles-pp.sock_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64
     36.32 ±  3%      -1.6       34.69        perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
     37.24 ±  3%      -1.5       35.77        perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
     37.82 ±  3%      -1.3       36.47        perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
     44.56            -1.2       43.41        perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
     52.26            -0.9       51.33        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
     52.40            -0.9       51.50        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write
      0.82 ± 20%      -0.6        0.26 ±100%  perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
     54.68            -0.4       54.25        perf-profile.calltrace.cycles-pp.__libc_write
      0.60 ± 12%      +0.1        0.74 ±  9%  perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.drain_obj_stock.refill_obj_stock.kmem_cache_free.unix_stream_read_generic
      0.62 ±  9%      +0.1        0.76 ±  4%  perf-profile.calltrace.cycles-pp.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb
      0.65 ± 15%      +0.2        0.82 ±  2%  perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_write
      0.66 ± 13%      +0.2        0.84 ±  3%  perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_read
      0.71 ± 15%      +0.2        0.90 ±  2%  perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_write
      0.82 ± 20%      +0.2        1.01 ±  3%  perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
      0.74 ± 15%      +0.2        0.92 ±  3%  perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_read
      0.71 ±  7%      +0.2        0.91 ± 10%  perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb
      0.76 ± 11%      +0.2        0.97 ±  2%  perf-profile.calltrace.cycles-pp.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic
      0.35 ± 70%      +0.2        0.56 ±  3%  perf-profile.calltrace.cycles-pp._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
      0.80 ± 11%      +0.2        1.01 ±  2%  perf-profile.calltrace.cycles-pp.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
      0.81 ± 11%      +0.2        1.03 ±  2%  perf-profile.calltrace.cycles-pp.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
      0.35 ± 70%      +0.2        0.57 ±  5%  perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up
      0.35 ± 70%      +0.3        0.62 ±  4%  perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common
      0.36 ± 70%      +0.3        0.63 ±  4%  perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.37 ± 70%      +0.3        0.64 ±  4%  perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
      0.36 ± 70%      +0.3        0.64 ±  4%  perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
      0.84 ± 11%      +0.3        1.12 ±  8%  perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.drain_obj_stock.refill_obj_stock.kfree.consume_skb
      0.36 ± 70%      +0.3        0.66 ±  5%  perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
      0.90 ± 19%      +0.3        1.24 ±  3%  perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.schedule_timeout.unix_stream_read_generic
      0.48 ± 70%      +0.4        0.85 ±  3%  perf-profile.calltrace.cycles-pp.prepare_to_wait.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
      1.05 ± 16%      +0.4        1.50 ±  7%  perf-profile.calltrace.cycles-pp.drain_obj_stock.refill_obj_stock.kmem_cache_free.unix_stream_read_generic.unix_stream_recvmsg
      0.00            +0.5        0.55 ±  5%  perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
      0.00            +0.7        0.67 ± 10%  perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.__kmalloc_node_track_caller.kmalloc_reserve
      1.42 ± 16%      +0.7        2.16 ±  7%  perf-profile.calltrace.cycles-pp.drain_obj_stock.refill_obj_stock.kfree.consume_skb.unix_stream_read_generic
      3.65 ± 23%      +0.8        4.50 ±  2%  perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
      2.29 ± 26%      +1.0        3.25 ±  6%  perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
      5.22 ± 26%      +1.9        7.15 ±  5%  perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
     13.43 ± 13%     -10.1        3.32 ± 10%  perf-profile.children.cycles-pp.mod_objcg_state
      4.19 ±  7%      -4.0        0.16 ± 16%  perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
      3.96 ±  8%      -3.8        0.11 ± 24%  perf-profile.children.cycles-pp.__mod_memcg_state
     35.15 ±  4%      -1.9       33.29        perf-profile.children.cycles-pp.unix_stream_read_generic
     35.33 ±  4%      -1.8       33.50        perf-profile.children.cycles-pp.unix_stream_recvmsg
     36.05 ±  3%      -1.7       34.38        perf-profile.children.cycles-pp.sock_read_iter
     36.33 ±  3%      -1.6       34.72        perf-profile.children.cycles-pp.new_sync_read
     37.25 ±  3%      -1.5       35.78        perf-profile.children.cycles-pp.vfs_read
     93.98            -1.4       92.63        perf-profile.children.cycles-pp.do_syscall_64
     37.84 ±  3%      -1.3       36.50        perf-profile.children.cycles-pp.ksys_read
     94.32            -1.3       93.03        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
     44.58            -1.2       43.42        perf-profile.children.cycles-pp.vfs_write
     54.81            -0.4       54.41        perf-profile.children.cycles-pp.__libc_write
      0.13 ±  9%      -0.1        0.04 ± 71%  perf-profile.children.cycles-pp.cgroup_rstat_updated
      0.06 ± 13%      +0.0        0.08 ±  6%  perf-profile.children.cycles-pp.syscall_enter_from_user_mode
      0.08 ± 10%      +0.0        0.10 ± 10%  perf-profile.children.cycles-pp.native_irq_return_iret
      0.09 ±  9%      +0.0        0.11 ±  5%  perf-profile.children.cycles-pp.__x64_sys_write
      0.09 ±  9%      +0.0        0.11 ±  4%  perf-profile.children.cycles-pp.__list_add_valid
      0.08 ± 20%      +0.0        0.11 ±  4%  perf-profile.children.cycles-pp.apparmor_socket_sendmsg
      0.10 ± 19%      +0.0        0.12 ±  3%  perf-profile.children.cycles-pp.apparmor_socket_recvmsg
      0.08 ± 12%      +0.0        0.10 ± 11%  perf-profile.children.cycles-pp.__cgroup_account_cputime
      0.20            +0.0        0.23 ±  4%  perf-profile.children.cycles-pp.preempt_schedule_common
      0.08 ± 17%      +0.0        0.12 ±  4%  perf-profile.children.cycles-pp.__x64_sys_read
      0.03 ± 70%      +0.0        0.07 ± 10%  perf-profile.children.cycles-pp.check_cfs_rq_runtime
      0.34 ±  3%      +0.0        0.38 ±  5%  perf-profile.children.cycles-pp.cpuacct_charge
      0.03 ± 70%      +0.0        0.08 ± 16%  perf-profile.children.cycles-pp.irqentry_exit_to_user_mode
      0.07 ± 71%      +0.0        0.11 ±  9%  perf-profile.children.cycles-pp.ktime_get
      0.09 ± 25%      +0.0        0.14 ±  5%  perf-profile.children.cycles-pp.rb_insert_color
      0.20 ±  6%      +0.0        0.24 ±  2%  perf-profile.children.cycles-pp.unix_write_space
      0.22 ± 10%      +0.0        0.27 ±  7%  perf-profile.children.cycles-pp.update_cfs_rq_h_load
      0.13 ±  7%      +0.0        0.18 ±  3%  perf-profile.children.cycles-pp.wait_for_unix_gc
      0.00            +0.1        0.05        perf-profile.children.cycles-pp.kmalloc_slab
      0.36 ±  3%      +0.1        0.41        perf-profile.children.cycles-pp.__cond_resched
      0.25 ± 16%      +0.1        0.30 ±  3%  perf-profile.children.cycles-pp.update_min_vruntime
      0.36 ± 11%      +0.1        0.41 ±  3%  perf-profile.children.cycles-pp.security_socket_recvmsg
      0.27 ± 13%      +0.1        0.33 ±  4%  perf-profile.children.cycles-pp.__check_heap_object
      0.06 ± 14%      +0.1        0.12 ± 11%  perf-profile.children.cycles-pp.wakeup_preempt_entity
      0.11 ±  4%      +0.1        0.18 ± 10%  perf-profile.children.cycles-pp.skb_release_data
      0.16 ± 21%      +0.1        0.22 ±  3%  perf-profile.children.cycles-pp.mutex_unlock
      0.00            +0.1        0.06 ± 11%  perf-profile.children.cycles-pp.put_pid
      0.28 ± 18%      +0.1        0.34 ±  5%  perf-profile.children.cycles-pp.copyin
      0.14 ± 15%      +0.1        0.20 ±  3%  perf-profile.children.cycles-pp.__ksize
      0.34 ±  9%      +0.1        0.40 ±  5%  perf-profile.children.cycles-pp.security_socket_sendmsg
      0.22 ± 23%      +0.1        0.29        perf-profile.children.cycles-pp.fput_many
      0.18 ± 16%      +0.1        0.25 ±  5%  perf-profile.children.cycles-pp.skb_unlink
      0.18 ± 12%      +0.1        0.25 ±  4%  perf-profile.children.cycles-pp.__list_del_entry_valid
      0.13 ± 20%      +0.1        0.20 ±  3%  perf-profile.children.cycles-pp.rb_erase
      0.40 ± 11%      +0.1        0.48 ±  3%  perf-profile.children.cycles-pp.sock_recvmsg
      0.34 ± 22%      +0.1        0.41 ±  3%  perf-profile.children.cycles-pp.__pthread_disable_asynccancel
      0.17 ±  7%      +0.1        0.25 ±  9%  perf-profile.children.cycles-pp.__build_skb_around
      0.49 ±  8%      +0.1        0.57 ±  4%  perf-profile.children.cycles-pp.common_file_perm
      0.49 ± 12%      +0.1        0.57 ±  3%  perf-profile.children.cycles-pp._copy_from_iter
      0.18 ± 24%      +0.1        0.27 ±  4%  perf-profile.children.cycles-pp.asm_sysvec_reschedule_ipi
      0.42 ± 21%      +0.1        0.51        perf-profile.children.cycles-pp.mutex_lock
      0.33 ±  9%      +0.1        0.43 ±  7%  perf-profile.children.cycles-pp.skb_set_owner_w
      0.36 ±  6%      +0.1        0.46 ± 11%  perf-profile.children.cycles-pp.__slab_free
      0.30 ± 15%      +0.1        0.40 ±  2%  perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
      0.37 ± 18%      +0.1        0.47 ±  6%  perf-profile.children.cycles-pp.check_preempt_wakeup
      0.23 ± 10%      +0.1        0.34 ±  9%  perf-profile.children.cycles-pp.propagate_protected_usage
      0.36 ± 15%      +0.1        0.47 ±  4%  perf-profile.children.cycles-pp.skb_queue_tail
      0.42 ±  5%      +0.1        0.55 ±  5%  perf-profile.children.cycles-pp.copyout
      0.43 ± 22%      +0.1        0.55 ±  3%  perf-profile.children.cycles-pp.__enqueue_entity
      0.62 ±  9%      +0.1        0.76 ±  4%  perf-profile.children.cycles-pp.sock_wfree
      0.58 ± 15%      +0.1        0.72 ±  4%  perf-profile.children.cycles-pp.__update_load_avg_se
      0.47 ± 21%      +0.2        0.63 ±  3%  perf-profile.children.cycles-pp.check_preempt_curr
      0.50 ± 20%      +0.2        0.66 ±  4%  perf-profile.children.cycles-pp.ttwu_do_wakeup
      0.61 ± 14%      +0.2        0.78 ±  4%  perf-profile.children.cycles-pp.__fget_files
      0.63 ± 22%      +0.2        0.82 ±  4%  perf-profile.children.cycles-pp.set_next_entity
      0.67 ± 10%      +0.2        0.86 ±  3%  perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      0.50 ± 19%      +0.2        0.69 ±  6%  perf-profile.children.cycles-pp.update_cfs_group
      0.84 ± 13%      +0.2        1.04 ±  2%  perf-profile.children.cycles-pp.__fget_light
      0.86 ± 13%      +0.2        1.06 ±  2%  perf-profile.children.cycles-pp.__fdget_pos
      0.65 ± 17%      +0.2        0.86 ±  4%  perf-profile.children.cycles-pp.prepare_to_wait
      0.78 ± 12%      +0.2        1.00 ±  2%  perf-profile.children.cycles-pp.unix_destruct_scm
      0.80 ± 11%      +0.2        1.02 ±  2%  perf-profile.children.cycles-pp.skb_release_head_state
      0.81 ± 11%      +0.2        1.03        perf-profile.children.cycles-pp.skb_release_all
      0.48 ± 15%      +0.3        0.76 ±  7%  perf-profile.children.cycles-pp.page_counter_cancel
      0.61 ± 14%      +0.3        0.94 ±  7%  perf-profile.children.cycles-pp.page_counter_uncharge
      1.50 ± 12%      +0.4        1.86 ±  8%  perf-profile.children.cycles-pp.obj_cgroup_uncharge_pages
      0.68 ± 13%      +0.4        1.06 ±  9%  perf-profile.children.cycles-pp.page_counter_try_charge
      1.65 ± 14%      +0.4        2.05 ±  3%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      1.50 ±  7%      +0.4        1.91 ±  6%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      1.51 ± 14%      +0.4        1.92 ±  2%  perf-profile.children.cycles-pp.__entry_text_start
      2.58 ± 17%      +1.1        3.66 ±  7%  perf-profile.children.cycles-pp.drain_obj_stock
      7.55 ± 26%      +2.9       10.46 ±  5%  perf-profile.children.cycles-pp.get_obj_cgroup_from_current
      9.13 ± 22%      -6.0        3.14 ± 10%  perf-profile.self.cycles-pp.mod_objcg_state
      3.95 ±  8%      -3.8        0.11 ± 24%  perf-profile.self.cycles-pp.__mod_memcg_state
      1.49 ± 21%      -1.4        0.10 ±  3%  perf-profile.self.cycles-pp.refill_obj_stock
      0.12 ±  7%      -0.1        0.04 ± 71%  perf-profile.self.cycles-pp.cgroup_rstat_updated
      0.06            +0.0        0.08 ±  6%  perf-profile.self.cycles-pp.sock_sendmsg
      0.09 ±  5%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.__list_add_valid
      0.08 ± 17%      +0.0        0.10 ±  3%  perf-profile.self.cycles-pp.apparmor_socket_recvmsg
      0.11 ±  7%      +0.0        0.13 ±  3%  perf-profile.self.cycles-pp._copy_from_iter
      0.09 ± 14%      +0.0        0.11 ±  3%  perf-profile.self.cycles-pp.obj_cgroup_charge
      0.07 ± 12%      +0.0        0.10 ±  3%  perf-profile.self.cycles-pp.__x64_sys_read
      0.03 ± 70%      +0.0        0.06 ±  7%  perf-profile.self.cycles-pp.sock_recvmsg
      0.11 ± 12%      +0.0        0.14 ±  4%  perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
      0.07 ± 11%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.ksys_write
      0.11 ± 19%      +0.0        0.15 ±  3%  perf-profile.self.cycles-pp.ksys_read
      0.14 ± 10%      +0.0        0.18 ± 10%  perf-profile.self.cycles-pp.__wake_up_common
      0.06 ± 72%      +0.0        0.10 ± 10%  perf-profile.self.cycles-pp.ktime_get
      0.26 ± 10%      +0.0        0.30 ±  3%  perf-profile.self.cycles-pp.sock_write_iter
      0.33 ±  5%      +0.0        0.38 ±  5%  perf-profile.self.cycles-pp.cpuacct_charge
      0.18 ± 19%      +0.0        0.22 ±  2%  perf-profile.self.cycles-pp.__libc_write
      0.09 ± 25%      +0.0        0.14 ±  5%  perf-profile.self.cycles-pp.rb_insert_color
      0.22 ± 10%      +0.0        0.27 ±  8%  perf-profile.self.cycles-pp.update_cfs_rq_h_load
      0.18 ±  7%      +0.0        0.23 ±  4%  perf-profile.self.cycles-pp.__libc_read
      0.10 ± 29%      +0.0        0.15 ±  6%  perf-profile.self.cycles-pp.check_preempt_curr
      0.16 ± 12%      +0.0        0.21 ±  7%  perf-profile.self.cycles-pp.check_preempt_wakeup
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.kmalloc_slab
      0.21 ± 17%      +0.1        0.26 ±  6%  perf-profile.self.cycles-pp.perf_trace_sched_wakeup_template
      0.00            +0.1        0.05 ±  7%  perf-profile.self.cycles-pp.check_cfs_rq_runtime
      0.15 ± 20%      +0.1        0.21 ±  6%  perf-profile.self.cycles-pp.unix_destruct_scm
      0.09 ±  5%      +0.1        0.15 ± 17%  perf-profile.self.cycles-pp.__build_skb_around
      0.17 ± 17%      +0.1        0.22 ±  3%  perf-profile.self.cycles-pp.unix_write_space
      0.24 ± 15%      +0.1        0.30 ±  2%  perf-profile.self.cycles-pp.update_min_vruntime
      0.27 ± 13%      +0.1        0.33 ±  4%  perf-profile.self.cycles-pp.__check_heap_object
      0.22 ± 12%      +0.1        0.27 ±  7%  perf-profile.self.cycles-pp.aa_sk_perm
      0.15 ± 20%      +0.1        0.21 ±  3%  perf-profile.self.cycles-pp.mutex_unlock
      0.14 ± 13%      +0.1        0.20 ±  4%  perf-profile.self.cycles-pp.__ksize
      0.00            +0.1        0.06 ±  6%  perf-profile.self.cycles-pp.wait_for_unix_gc
      0.16 ± 10%      +0.1        0.22 ±  5%  perf-profile.self.cycles-pp.__list_del_entry_valid
      0.11 ±  7%      +0.1        0.17 ±  9%  perf-profile.self.cycles-pp.skb_release_data
      0.22 ± 22%      +0.1        0.28        perf-profile.self.cycles-pp.fput_many
      0.25 ± 11%      +0.1        0.32 ±  2%  perf-profile.self.cycles-pp.__alloc_skb
      0.28 ± 24%      +0.1        0.35        perf-profile.self.cycles-pp.mutex_lock
      0.12 ± 23%      +0.1        0.19 ±  4%  perf-profile.self.cycles-pp.rb_erase
      0.32 ± 23%      +0.1        0.39 ±  3%  perf-profile.self.cycles-pp.__pthread_disable_asynccancel
      0.17 ± 19%      +0.1        0.24 ±  5%  perf-profile.self.cycles-pp.dequeue_entity
      0.05 ± 70%      +0.1        0.12 ±  9%  perf-profile.self.cycles-pp.wakeup_preempt_entity
      0.25 ±  9%      +0.1        0.33 ±  3%  perf-profile.self.cycles-pp.sock_def_readable
      0.38 ±  9%      +0.1        0.46 ±  6%  perf-profile.self.cycles-pp.common_file_perm
      0.30 ± 14%      +0.1        0.38 ±  3%  perf-profile.self.cycles-pp.sock_read_iter
      0.60 ±  8%      +0.1        0.69 ±  2%  perf-profile.self.cycles-pp.unix_stream_sendmsg
      0.38 ± 12%      +0.1        0.46 ±  3%  perf-profile.self.cycles-pp.try_to_wake_up
      0.46 ± 15%      +0.1        0.56 ±  3%  perf-profile.self.cycles-pp.select_task_rq_fair
      0.41 ± 11%      +0.1        0.50 ±  7%  perf-profile.self.cycles-pp.sock_wfree
      0.33 ± 10%      +0.1        0.42 ±  7%  perf-profile.self.cycles-pp.skb_set_owner_w
      0.32 ± 21%      +0.1        0.42 ±  2%  perf-profile.self.cycles-pp.prepare_to_wait
      0.32 ± 24%      +0.1        0.42 ±  3%  perf-profile.self.cycles-pp.pick_next_task_fair
      0.35 ±  6%      +0.1        0.46 ± 11%  perf-profile.self.cycles-pp.__slab_free
      0.23 ± 12%      +0.1        0.34 ±  9%  perf-profile.self.cycles-pp.propagate_protected_usage
      0.50 ± 24%      +0.1        0.62 ±  4%  perf-profile.self.cycles-pp.update_rq_clock
      0.41 ± 22%      +0.1        0.55 ±  3%  perf-profile.self.cycles-pp.__enqueue_entity
      0.57 ± 15%      +0.1        0.71 ±  4%  perf-profile.self.cycles-pp.__update_load_avg_se
      0.60 ± 13%      +0.2        0.76 ±  4%  perf-profile.self.cycles-pp.__fget_files
      0.49 ± 19%      +0.2        0.68 ±  6%  perf-profile.self.cycles-pp.update_cfs_group
      0.66 ± 11%      +0.2        0.84 ±  3%  perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      1.14 ± 16%      +0.3        1.40 ±  4%  perf-profile.self.cycles-pp.__schedule
      0.94 ± 17%      +0.3        1.19 ±  2%  perf-profile.self.cycles-pp.update_curr
      0.48 ± 15%      +0.3        0.76 ±  8%  perf-profile.self.cycles-pp.page_counter_cancel
      0.57 ± 13%      +0.3        0.89 ±  8%  perf-profile.self.cycles-pp.page_counter_try_charge
      1.31 ± 14%      +0.4        1.68 ±  2%  perf-profile.self.cycles-pp.__entry_text_start
      1.65 ± 14%      +0.4        2.05 ±  3%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      1.34 ± 17%      +0.4        1.78 ±  3%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      1.71 ± 20%      +0.5        2.17 ±  2%  perf-profile.self.cycles-pp._raw_spin_lock
      1.08 ± 23%      +0.7        1.76 ±  7%  perf-profile.self.cycles-pp.drain_obj_stock
      2.35 ± 10%      +2.1        4.44 ±  7%  perf-profile.self.cycles-pp.kmem_cache_free
      7.40 ± 26%      +2.9       10.27 ±  5%  perf-profile.self.cycles-pp.get_obj_cgroup_from_current
      2.60 ± 12%      +3.9        6.50 ±  5%  perf-profile.self.cycles-pp.kfree
    109386           -24.7%      82383        softirqs.CPU0.RCU
     15341 ±  5%     -22.1%      11946 ±  8%  softirqs.CPU0.SCHED
    113388           -26.0%      83901        softirqs.CPU1.RCU
     12942 ±  7%     -18.6%      10530 ±  9%  softirqs.CPU1.SCHED
    111188           -25.6%      82690        softirqs.CPU10.RCU
     11168 ±  2%     -23.0%       8595 ±  5%  softirqs.CPU10.SCHED
    109889           -26.3%      81004        softirqs.CPU100.RCU
     14260 ±  5%     -29.7%      10023 ± 14%  softirqs.CPU100.SCHED
    109411           -26.2%      80721        softirqs.CPU101.RCU
     14045 ±  8%     -31.3%       9652 ± 12%  softirqs.CPU101.SCHED
    109913           -25.6%      81743        softirqs.CPU102.RCU
     13603 ± 12%     -30.4%       9461 ± 12%  softirqs.CPU102.SCHED
    110978           -26.5%      81567        softirqs.CPU103.RCU
     14320 ±  7%     -27.5%      10377 ± 13%  softirqs.CPU103.SCHED
    113121 ±  2%     -26.1%      83547        softirqs.CPU11.RCU
     11268 ±  2%     -22.6%       8718 ±  7%  softirqs.CPU11.SCHED
    113553           -26.2%      83780        softirqs.CPU12.RCU
     11282 ±  3%     -24.2%       8551 ±  7%  softirqs.CPU12.SCHED
    112337 ±  2%     -26.2%      82874 ±  2%  softirqs.CPU13.RCU
     11479 ±  2%     -24.1%       8707 ±  7%  softirqs.CPU13.SCHED
    112127           -29.0%      79623 ±  9%  softirqs.CPU14.RCU
     11495 ±  2%     -23.8%       8757 ±  8%  softirqs.CPU14.SCHED
    126475 ±  2%     -27.8%      91367        softirqs.CPU15.RCU
     11571 ±  2%     -27.1%       8433 ±  6%  softirqs.CPU15.SCHED
    125147           -26.8%      91569        softirqs.CPU16.RCU
     11050 ±  5%     -23.8%       8422 ±  5%  softirqs.CPU16.SCHED
    126468           -28.5%      90429 ±  5%  softirqs.CPU17.RCU
     10891 ±  3%     -21.8%       8511 ±  7%  softirqs.CPU17.SCHED
    126847           -27.0%      92542        softirqs.CPU18.RCU
     10796 ±  3%     -20.3%       8609 ±  9%  softirqs.CPU18.SCHED
    127636           -27.0%      93207        softirqs.CPU19.RCU
     11135 ±  2%     -20.4%       8866 ±  6%  softirqs.CPU19.SCHED
    112062 ±  2%     -25.5%      83467        softirqs.CPU2.RCU
     11754           -22.2%       9144 ±  7%  softirqs.CPU2.SCHED
    113957 ± 11%     -20.2%      90907        softirqs.CPU20.RCU
     11197 ±  6%     -20.4%       8915 ± 10%  softirqs.CPU20.SCHED
    125383 ±  2%     -27.4%      91022        softirqs.CPU21.RCU
     11275 ±  3%     -26.4%       8302 ±  4%  softirqs.CPU21.SCHED
    124972 ±  2%     -27.9%      90088 ±  4%  softirqs.CPU22.RCU
     11370 ±  3%     -23.6%       8682 ±  5%  softirqs.CPU22.SCHED
    126353 ±  2%     -27.6%      91532        softirqs.CPU23.RCU
     11324 ±  3%     -24.5%       8545 ±  6%  softirqs.CPU23.SCHED
    126661 ±  2%     -26.9%      92532        softirqs.CPU24.RCU
     11120 ±  4%     -21.9%       8682 ±  7%  softirqs.CPU24.SCHED
    126860           -28.5%      90745 ±  6%  softirqs.CPU25.RCU
     10945           -20.9%       8656 ±  6%  softirqs.CPU25.SCHED
    110300           -26.6%      80912        softirqs.CPU26.RCU
     12543 ±  8%     -23.1%       9647 ± 12%  softirqs.CPU26.SCHED
    110556           -26.7%      81056        softirqs.CPU27.RCU
     14373 ± 13%     -29.9%      10080 ± 12%  softirqs.CPU27.SCHED
    111033           -27.2%      80849        softirqs.CPU28.RCU
     14005 ±  7%     -33.5%       9318 ± 16%  softirqs.CPU28.SCHED
    104861 ±  8%     -22.4%      81348        softirqs.CPU29.RCU
     14384 ±  5%     -31.6%       9837 ±  9%  softirqs.CPU29.SCHED
    112140 ±  2%     -26.3%      82656 ±  2%  softirqs.CPU3.RCU
     11745 ±  3%     -25.8%       8719 ±  5%  softirqs.CPU3.SCHED
    114089           -26.4%      83915        softirqs.CPU30.RCU
     14207 ±  5%     -33.6%       9430 ±  9%  softirqs.CPU30.SCHED
    113403 ±  2%     -25.2%      84800 ±  2%  softirqs.CPU31.RCU
     13544 ±  4%     -29.0%       9622 ± 12%  softirqs.CPU31.SCHED
    114790           -28.7%      81832 ±  8%  softirqs.CPU32.RCU
     14011 ±  6%     -30.6%       9722 ± 10%  softirqs.CPU32.SCHED
    114298           -26.8%      83675        softirqs.CPU33.RCU
     14702 ±  7%     -36.1%       9401 ± 11%  softirqs.CPU33.SCHED
    114243 ±  2%     -25.9%      84672        softirqs.CPU34.RCU
     13441 ±  5%     -28.9%       9561 ± 17%  softirqs.CPU34.SCHED
    115097 ±  2%     -26.5%      84570        softirqs.CPU35.RCU
     14226 ±  6%     -33.2%       9508 ± 12%  softirqs.CPU35.SCHED
    115565 ±  2%     -27.2%      84118        softirqs.CPU36.RCU
     13705 ± 11%     -28.6%       9782 ± 16%  softirqs.CPU36.SCHED
    116062 ±  3%     -26.7%      85110        softirqs.CPU37.RCU
     14157 ±  9%     -32.3%       9587 ± 15%  softirqs.CPU37.SCHED
    116212 ±  2%     -26.6%      85245 ±  2%  softirqs.CPU38.RCU
     13646 ±  9%     -30.9%       9434 ± 13%  softirqs.CPU38.SCHED
    114536           -26.5%      84202        softirqs.CPU39.RCU
     13891 ±  5%     -30.4%       9671 ± 11%  softirqs.CPU39.SCHED
    107336 ±  8%     -22.0%      83756 ±  3%  softirqs.CPU4.RCU
     11885 ±  3%     -26.3%       8758 ±  6%  softirqs.CPU4.SCHED
    114987           -26.4%      84595        softirqs.CPU40.RCU
     14076 ± 12%     -31.6%       9627 ± 15%  softirqs.CPU40.SCHED
    114484 ±  2%     -27.0%      83591        softirqs.CPU41.RCU
     13850 ±  6%     -35.9%       8874 ±  8%  softirqs.CPU41.SCHED
    116063           -27.1%      84667 ±  2%  softirqs.CPU42.RCU
     14224 ±  8%     -33.0%       9534 ± 17%  softirqs.CPU42.SCHED
    115606           -26.1%      85428 ±  2%  softirqs.CPU43.RCU
     13433 ±  3%     -32.0%       9135 ±  9%  softirqs.CPU43.SCHED
    115318 ±  2%     -26.7%      84516 ±  2%  softirqs.CPU44.RCU
     14774 ±  5%     -36.0%       9459 ± 13%  softirqs.CPU44.SCHED
    116588 ±  2%     -26.5%      85668        softirqs.CPU45.RCU
     13793 ± 10%     -29.3%       9758 ± 19%  softirqs.CPU45.SCHED
    115365           -26.7%      84519        softirqs.CPU46.RCU
     13898 ±  8%     -32.1%       9430 ± 11%  softirqs.CPU46.SCHED
    115826           -27.3%      84197        softirqs.CPU47.RCU
     13557 ±  5%     -27.1%       9879 ± 18%  softirqs.CPU47.SCHED
    115566           -27.2%      84157        softirqs.CPU48.RCU
     13896 ±  7%     -31.7%       9486 ± 15%  softirqs.CPU48.SCHED
    115762           -26.9%      84677        softirqs.CPU49.RCU
     14081 ±  5%     -31.2%       9683 ± 12%  softirqs.CPU49.SCHED
    112583           -26.2%      83127        softirqs.CPU5.RCU
     11497 ±  3%     -24.7%       8662 ±  6%  softirqs.CPU5.SCHED
    116177           -27.2%      84627        softirqs.CPU50.RCU
     13451 ± 13%     -31.8%       9172 ± 11%  softirqs.CPU50.SCHED
    116569           -26.6%      85517        softirqs.CPU51.RCU
     14579 ±  5%     -29.8%      10235 ± 18%  softirqs.CPU51.SCHED
    120241 ±  2%     -25.5%      89582 ±  2%  softirqs.CPU52.RCU
     10709 ±  3%     -22.8%       8268 ±  5%  softirqs.CPU52.SCHED
    123731 ±  2%     -26.0%      91542 ±  2%  softirqs.CPU53.RCU
     11339 ±  4%     -24.5%       8564 ±  4%  softirqs.CPU53.SCHED
    122537 ±  2%     -25.0%      91896 ±  2%  softirqs.CPU54.RCU
     11006 ±  9%     -31.3%       7566 ±  4%  softirqs.CPU54.SCHED
    124865 ±  2%     -26.6%      91657 ±  3%  softirqs.CPU55.RCU
     11073 ±  5%     -24.7%       8339 ±  5%  softirqs.CPU55.SCHED
     11530 ±  3%     -25.5%       8593 ±  5%  softirqs.CPU56.SCHED
    124438 ±  2%     -26.2%      91852        softirqs.CPU57.RCU
     10909 ±  3%     -22.1%       8502 ±  5%  softirqs.CPU57.SCHED
    125889 ±  2%     -25.7%      93510 ±  2%  softirqs.CPU58.RCU
     11441 ±  4%     -23.4%       8769 ±  7%  softirqs.CPU58.SCHED
    122714           -29.1%      87046 ± 10%  softirqs.CPU59.RCU
     11610 ±  2%     -24.3%       8785 ±  9%  softirqs.CPU59.SCHED
    113688           -25.4%      84834 ±  2%  softirqs.CPU6.RCU
     11723 ±  2%     -24.4%       8858 ±  6%  softirqs.CPU6.SCHED
    112505           -26.2%      83069        softirqs.CPU60.RCU
     11303 ±  3%     -23.7%       8628 ±  7%  softirqs.CPU60.SCHED
    111675           -25.5%      83152        softirqs.CPU61.RCU
     11008 ±  2%     -19.6%       8854 ± 10%  softirqs.CPU61.SCHED
    111654           -25.7%      82984        softirqs.CPU62.RCU
     10974 ±  2%     -21.4%       8622 ±  5%  softirqs.CPU62.SCHED
    113013           -25.2%      84589        softirqs.CPU63.RCU
     11383 ±  4%     -23.4%       8722 ±  7%  softirqs.CPU63.SCHED
    113944           -26.0%      84360        softirqs.CPU64.RCU
     11270 ±  2%     -21.9%       8807 ±  6%  softirqs.CPU64.SCHED
    112766           -26.4%      82988        softirqs.CPU65.RCU
     11198 ±  4%     -23.4%       8580 ±  5%  softirqs.CPU65.SCHED
    111974           -27.9%      80761 ±  7%  softirqs.CPU66.RCU
     11420 ±  3%     -24.7%       8602 ±  5%  softirqs.CPU66.SCHED
    111724           -25.8%      82949        softirqs.CPU67.RCU
     11407 ±  5%     -24.4%       8626 ±  6%  softirqs.CPU67.SCHED
    112065           -25.6%      83358        softirqs.CPU68.RCU
     11401 ±  9%     -24.9%       8559 ±  6%  softirqs.CPU68.SCHED
    114047           -29.1%      80823 ±  9%  softirqs.CPU69.RCU
     11465 ±  2%     -21.7%       8981 ± 12%  softirqs.CPU69.SCHED
    111811 ±  2%     -26.7%      81901 ±  7%  softirqs.CPU7.RCU
     11527 ±  2%     -21.7%       9028 ±  6%  softirqs.CPU7.SCHED
    113806           -25.5%      84749 ±  2%  softirqs.CPU70.RCU
     11231 ±  5%     -22.0%       8761 ±  7%  softirqs.CPU70.SCHED
    113642           -25.6%      84493 ±  2%  softirqs.CPU71.RCU
     11748 ±  8%     -25.6%       8746 ±  6%  softirqs.CPU71.SCHED
    107927 ±  5%     -22.8%      83345        softirqs.CPU72.RCU
     11561 ±  2%     -22.7%       8936 ± 12%  softirqs.CPU72.SCHED
    111669           -25.5%      83229        softirqs.CPU73.RCU
     11172 ±  4%     -24.1%       8482 ±  5%  softirqs.CPU73.SCHED
    113287           -29.4%      79942 ±  9%  softirqs.CPU74.RCU
     11715 ±  6%     -23.3%       8981 ±  7%  softirqs.CPU74.SCHED
    112331 ±  3%     -25.5%      83686 ±  2%  softirqs.CPU75.RCU
     11128 ±  3%     -22.1%       8671 ±  5%  softirqs.CPU75.SCHED
    111893 ±  3%     -24.9%      84048 ±  2%  softirqs.CPU76.RCU
     11210 ±  3%     -16.6%       9352 ±  8%  softirqs.CPU76.SCHED
    112187 ±  4%     -27.9%      80865 ±  7%  softirqs.CPU77.RCU
     11295 ±  4%     -22.2%       8789 ±  4%  softirqs.CPU77.SCHED
    118790           -26.0%      87858        softirqs.CPU78.RCU
     13396 ±  4%     -31.1%       9232 ±  9%  softirqs.CPU78.SCHED
    118851 ±  2%     -25.8%      88237        softirqs.CPU79.RCU
     13523 ±  7%     -24.3%      10236 ± 12%  softirqs.CPU79.SCHED
    111449           -26.0%      82490 ±  2%  softirqs.CPU8.RCU
     11214 ±  2%     -24.3%       8488 ±  5%  softirqs.CPU8.SCHED
    118692           -25.4%      88590        softirqs.CPU80.RCU
     14082 ±  4%     -26.7%      10327 ± 10%  softirqs.CPU80.SCHED
    116649 ±  5%     -24.6%      87957        softirqs.CPU81.RCU
     14559 ±  7%     -34.5%       9534 ± 13%  softirqs.CPU81.SCHED
    119797 ±  2%     -26.6%      87951        softirqs.CPU82.RCU
     14468 ±  7%     -33.9%       9557 ± 10%  softirqs.CPU82.SCHED
    118499           -25.7%      87989 ±  2%  softirqs.CPU83.RCU
    119911 ±  2%     -27.4%      87013 ±  4%  softirqs.CPU84.RCU
     14860 ±  3%     -31.7%      10147 ± 13%  softirqs.CPU84.SCHED
    119690           -26.6%      87829        softirqs.CPU85.RCU
     14141 ±  8%     -33.2%       9448 ± 11%  softirqs.CPU85.SCHED
    119016 ±  2%     -25.9%      88170        softirqs.CPU86.RCU
     13379 ±  8%     -30.4%       9309 ± 11%  softirqs.CPU86.SCHED
    119242 ±  2%     -25.9%      88373        softirqs.CPU87.RCU
     13944 ±  7%     -29.5%       9834 ± 12%  softirqs.CPU87.SCHED
    119490 ±  2%     -26.6%      87687        softirqs.CPU88.RCU
     14066 ± 11%     -29.7%       9885 ± 14%  softirqs.CPU88.SCHED
    118674 ±  2%     -25.8%      88035        softirqs.CPU89.RCU
     14090 ±  5%     -31.5%       9647 ± 13%  softirqs.CPU89.SCHED
    111284 ±  2%     -25.4%      82978 ±  2%  softirqs.CPU9.RCU
     10915           -19.4%       8802 ±  7%  softirqs.CPU9.SCHED
    111624 ±  2%     -26.2%      82405        softirqs.CPU90.RCU
     14343 ±  9%     -33.4%       9559 ± 14%  softirqs.CPU90.SCHED
    110317           -26.2%      81438        softirqs.CPU91.RCU
     13801 ±  4%     -29.0%       9803 ± 11%  softirqs.CPU91.SCHED
    110304 ±  2%     -25.9%      81721        softirqs.CPU92.RCU
     13623 ± 11%     -29.1%       9665 ± 13%  softirqs.CPU92.SCHED
    111772           -27.2%      81370        softirqs.CPU93.RCU
     14274 ±  6%     -33.8%       9448 ± 10%  softirqs.CPU93.SCHED
    110389           -26.1%      81624        softirqs.CPU94.RCU
     14238 ±  9%     -31.6%       9737 ± 16%  softirqs.CPU94.SCHED
    111415 ±  2%     -26.6%      81770        softirqs.CPU95.RCU
     13761 ±  8%     -34.1%       9066 ± 10%  softirqs.CPU95.SCHED
    110596           -26.1%      81782        softirqs.CPU96.RCU
     14044 ±  6%     -30.4%       9772 ± 14%  softirqs.CPU96.SCHED
    111174 ±  2%     -26.1%      82155        softirqs.CPU97.RCU
     14260 ± 12%     -30.7%       9878 ± 18%  softirqs.CPU97.SCHED
    111371 ±  2%     -27.0%      81298        softirqs.CPU98.RCU
     14319 ±  9%     -32.2%       9705 ± 14%  softirqs.CPU98.SCHED
    109567           -25.9%      81242        softirqs.CPU99.RCU
     13355 ±  9%     -26.5%       9817 ± 14%  softirqs.CPU99.SCHED
  12003152           -26.2%    8852871        softirqs.RCU
   1320123 ±  4%     -27.4%     958083 ±  4%  softirqs.SCHED
    124887           -25.9%      92537        softirqs.TIMER





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


---
0DAY/LKP+ Test Infrastructure                   Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org       Intel Corporation

Thanks,
Rong Chen


View attachment "config-5.13.0-00078-g68ac5b3c8db2" of type "text/plain" (174163 bytes)

View attachment "job-script" of type "text/plain" (8005 bytes)

View attachment "job.yaml" of type "text/plain" (5395 bytes)

View attachment "reproduce" of type "text/plain" (530 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ