lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <557E786D.4080906@intel.com>
Date:	Mon, 15 Jun 2015 15:02:05 +0800
From:	Huang Ying <ying.huang@...el.com>
To:	Jiang Liu <jiang.liu@...ux.intel.com>
CC:	Thomas Gleixner <tglx@...utronix.de>,
	LKML <linux-kernel@...r.kernel.org>, LKP ML <lkp@...org>
Subject: [lkp] [x86/MSI] 52f518a3a7c: -30.5% netperf.Throughput_tps

FYI, we noticed the below changes on

git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master
commit 52f518a3a7c2f80551a38d38be28bc9f335e713c ("x86/MSI: Use hierarchical irqdomains to manage MSI interrupts")


testcase/path_params/tbox_group: netperf/performance-300s-200%-TCP_RR/brickland3

3cb96f0c97330834  52f518a3a7c2f80551a38d38be  
----------------  --------------------------  
         %stddev     %change         %stddev
             \          |                \  
      1444 ±  1%     -30.5%       1004 ±  1%  netperf.Throughput_tps
    899.43 ±  3%     -91.6%      75.48 ±  3%  netperf.time.user_time
     10202 ±  0%     -92.4%        778 ±  1%  netperf.time.system_time
      3565 ±  4%     -92.5%        268 ±  3%  netperf.time.percent_of_cpu_this_job_got
     98265 ±  3%     -31.1%      67662 ±  4%  netperf.time.minor_page_faults
     25540 ± 11%     -88.3%       2991 ± 16%  netperf.time.involuntary_context_switches
  69135723 ±  1%     -30.0%   48364754 ±  1%  netperf.time.voluntary_context_switches
     37347 ±  4%     +13.1%      42256 ±  3%  uptime.idle
        44 ±  5%     -93.2%          3 ±  0%  vmstat.procs.r
    407528 ±  3%     -26.3%     300511 ±  4%  vmstat.system.cs
    446620 ±  4%     -92.6%      33083 ±  6%  vmstat.system.in
    178635 ±  1%     -14.6%     152519 ±  0%  meminfo.Active
     82997 ±  2%     -29.5%      58518 ±  1%  meminfo.Active(anon)
     36056 ±  3%     -63.2%      13281 ±  0%  meminfo.Shmem
    496435 ±  3%     +19.4%     592640 ±  2%  numa-meminfo.node0.MemUsed
    480933 ±  3%      -7.3%     445927 ±  2%  numa-meminfo.node2.MemUsed
      3997 ±  1%     +12.1%       4482 ±  3%  slabinfo.kmalloc-2048.active_objs
      4183 ±  1%     +12.2%       4694 ±  3%  slabinfo.kmalloc-2048.num_objs
       313 ± 15%     -78.0%         69 ± 30%  slabinfo.tw_sock_TCP.active_objs
       313 ± 15%     -78.0%         69 ± 30%  slabinfo.tw_sock_TCP.num_objs
   1016370 ±  1%     -90.2%      99283 ±  1%  softirqs.HRTIMER
   1556902 ±  0%     -84.0%     248959 ±  2%  softirqs.RCU
   3471142 ±  1%     -92.3%     267496 ±  3%  softirqs.SCHED
   4270170 ±  1%     -82.2%     758706 ±  2%  softirqs.TIMER
 2.347e+09 ±  6%     -63.5%  8.569e+08 ± 39%  cpuidle.C1-IVT-4S.time
  22319046 ±  3%     +62.6%   36299214 ±  3%  cpuidle.C1-IVT-4S.usage
 1.591e+10 ±  9%    +128.3%  3.632e+10 ±  3%  cpuidle.C6-IVT-4S.time
         7 ± 21%    +514.0%         44 ± 31%  cpuidle.POLL.usage
     25540 ± 11%     -88.3%       2991 ± 16%  time.involuntary_context_switches
     98265 ±  3%     -31.1%      67662 ±  4%  time.minor_page_faults
      3565 ±  4%     -92.5%        268 ±  3%  time.percent_of_cpu_this_job_got
     10202 ±  0%     -92.4%        778 ±  1%  time.system_time
    899.43 ±  3%     -91.6%      75.48 ±  3%  time.user_time
  69135723 ±  1%     -30.0%   48364754 ±  1%  time.voluntary_context_switches
    268764 ± 14%    +408.8%    1367464 ±  3%  numa-numastat.node0.numa_hit
    268758 ± 14%    +408.8%    1367424 ±  3%  numa-numastat.node0.local_node
    438574 ± 26%     -45.0%     241074 ±  2%  numa-numastat.node1.local_node
    438574 ± 26%     -45.0%     241076 ±  2%  numa-numastat.node1.numa_hit
    333989 ± 23%     -27.9%     240841 ±  3%  numa-numastat.node2.local_node
    333989 ± 23%     -27.9%     240844 ±  3%  numa-numastat.node2.numa_hit
    381233 ± 26%     -35.6%     245556 ±  3%  numa-numastat.node3.local_node
    381238 ± 26%     -35.6%     245561 ±  3%  numa-numastat.node3.numa_hit
       482 ±  4%     +11.9%        540 ±  1%  numa-vmstat.node0.nr_alloc_batch
    251510 ±  9%    +212.0%     784815 ±  8%  numa-vmstat.node0.numa_local
    289771 ±  8%    +184.0%     823025 ±  7%  numa-vmstat.node0.numa_hit
    454081 ± 12%     -25.4%     338719 ±  4%  numa-vmstat.node1.numa_hit
       521 ±  1%     +13.5%        592 ±  3%  numa-vmstat.node1.nr_alloc_batch
    395201 ± 14%     -28.9%     281018 ±  4%  numa-vmstat.node1.numa_local
       529 ±  3%     +10.1%        583 ±  1%  numa-vmstat.node2.nr_alloc_batch
       525 ±  6%     +12.8%        593 ±  1%  numa-vmstat.node3.nr_alloc_batch
     51.25 ±  3%     -92.9%       3.65 ±  3%  turbostat.%Busy
      1635 ±  3%     -93.0%        114 ±  3%  turbostat.Avg_MHz
     45.20 ±  4%     -70.1%      13.52 ± 21%  turbostat.CPU%c1
    357.80 ±  3%     -71.5%     101.83 ±  6%  turbostat.CorWatt
        53 ±  2%     -21.2%         42 ±  6%  turbostat.CoreTmp
        53 ±  1%     -20.7%         42 ±  6%  turbostat.PkgTmp
       424 ±  2%     -61.0%        165 ±  3%  turbostat.PkgWatt
     62.78 ±  1%      -8.9%      57.17 ±  3%  turbostat.RAMWatt
     20746 ±  2%     -29.5%      14628 ±  1%  proc-vmstat.nr_active_anon
      2024 ±  1%     +13.4%       2295 ±  0%  proc-vmstat.nr_alloc_batch
      9013 ±  3%     -63.2%       3319 ±  0%  proc-vmstat.nr_shmem
     49576 ±  5%     -60.3%      19667 ± 13%  proc-vmstat.numa_hint_faults
     44013 ±  5%     -64.6%      15598 ± 16%  proc-vmstat.numa_hint_faults_local
   1420966 ±  2%     +47.3%    2092933 ±  1%  proc-vmstat.numa_hit
   1420955 ±  2%     +47.3%    2092883 ±  1%  proc-vmstat.numa_local
     53703 ±  5%     -57.9%      22601 ± 13%  proc-vmstat.numa_pte_updates
      8470 ±  1%     -81.8%       1543 ±  3%  proc-vmstat.pgactivate
    162499 ±  5%     +90.3%     309245 ±  5%  proc-vmstat.pgalloc_dma32
   5918865 ±  2%     +85.7%   10992818 ±  3%  proc-vmstat.pgalloc_normal
   6085763 ±  2%     +86.0%   11317294 ±  3%  proc-vmstat.pgfree
      2579 ±  5%     -79.4%        532 ±  2%  latency_stats.avg.do_wait.SyS_wait4.system_call_fastpath
       858 ±  5%     +60.1%       1375 ±  4%  latency_stats.avg.cifs_user_readv.[cifs].cifs_strict_readv.[cifs].__vfs_read.vfs_read.SyS_read.system_call_fastpath
      2337 ±  4%     -79.5%        479 ±  2%  latency_stats.avg.pipe_wait.pipe_read.__vfs_read.vfs_read.SyS_read.system_call_fastpath
       875 ±  7%     +74.6%       1528 ± 10%  latency_stats.avg.wait_for_response.[cifs].SendReceive.[cifs].CIFSPOSIXCreate.[cifs].cifs_posix_open.[cifs].cifs_open.[cifs].do_dentry_open.vfs_open.do_last.path_openat.do_filp_open.do_sys_open.SyS_openat
       455 ±  1%    +108.8%        951 ±  1%  latency_stats.avg.sk_wait_data.tcp_recvmsg.inet_recvmsg.sock_recvmsg.SYSC_recvfrom.SyS_recvfrom.system_call_fastpath
       541 ± 14%    +137.6%       1287 ±  4%  latency_stats.avg.wait_for_response.[cifs].SendReceive.[cifs].CIFSSMBUnixQPathInfo.[cifs].cifs_get_inode_info_unix.[cifs].cifs_revalidate_dentry_attr.[cifs].cifs_revalidate_dentry.[cifs].cifs_d_revalidate.[cifs].lookup_fast.do_last.path_openat.do_filp_open.do_sys_open
       655 ± 20%     +96.9%       1291 ±  9%  latency_stats.avg.poll_schedule_timeout.do_select.core_sys_select.SyS_select.system_call_fastpath
  69119790 ±  1%     -30.0%   48351473 ±  1%  latency_stats.hits.sk_wait_data.tcp_recvmsg.inet_recvmsg.sock_recvmsg.SYSC_recvfrom.SyS_recvfrom.system_call_fastpath
       443 ±  3%     -10.0%        399 ±  2%  latency_stats.hits.poll_schedule_timeout.do_select.core_sys_select.SyS_select.system_call_fastpath
       190 ±  6%     -95.8%          8 ± 38%  latency_stats.hits.stop_one_cpu.sched_exec.do_execveat_common.SyS_execve.return_from_execve
      2157 ± 23%     +66.0%       3581 ± 22%  latency_stats.max.cifs_user_readv.[cifs].cifs_strict_readv.[cifs].__vfs_read.vfs_read.SyS_read.system_call_fastpath
      4912 ±  0%      +1.6%       4992 ±  0%  latency_stats.max.sk_wait_data.tcp_recvmsg.inet_recvmsg.sock_recvmsg.SYSC_recvfrom.SyS_recvfrom.system_call_fastpath
 3.151e+10 ±  0%     +46.1%  4.603e+10 ±  0%  latency_stats.sum.sk_wait_data.tcp_recvmsg.inet_recvmsg.sock_recvmsg.SYSC_recvfrom.SyS_recvfrom.system_call_fastpath
  16780629 ±  2%     -78.9%    3539898 ±  4%  latency_stats.sum.do_wait.SyS_wait4.system_call_fastpath
  25402986 ±  2%     -78.5%    5473439 ±  4%  latency_stats.sum.pipe_wait.pipe_read.__vfs_read.vfs_read.SyS_read.system_call_fastpath
     12072 ± 31%    +112.0%      25588 ± 10%  latency_stats.sum.wait_for_response.[cifs].SendReceive.[cifs].CIFSSMBUnixQPathInfo.[cifs].cifs_get_inode_info_unix.[cifs].cifs_revalidate_dentry_attr.[cifs].cifs_revalidate_dentry.[cifs].cifs_d_revalidate.[cifs].lookup_fast.do_last.path_openat.do_filp_open.do_sys_open
    290547 ± 19%     +77.5%     515581 ±  9%  latency_stats.sum.poll_schedule_timeout.do_select.core_sys_select.SyS_select.system_call_fastpath
     24407 ±  9%     +74.1%      42503 ±  9%  latency_stats.sum.wait_for_response.[cifs].SendReceive.[cifs].CIFSPOSIXCreate.[cifs].cifs_posix_open.[cifs].cifs_open.[cifs].do_dentry_open.vfs_open.do_last.path_openat.do_filp_open.do_sys_open.SyS_openat
    245753 ± 23%     +44.5%     355113 ±  9%  latency_stats.sum.cifs_user_readv.[cifs].cifs_strict_readv.[cifs].__vfs_read.vfs_read.SyS_read.system_call_fastpath
     17.85 ±  4%     -16.4%      14.92 ±  2%  perf-profile.cpu-cycles.ip_rcv.__netif_receive_skb_core.__netif_receive_skb.netif_receive_skb_internal.napi_gro_receive
      0.97 ±  4%     -85.4%       0.14 ± 32%  perf-profile.cpu-cycles.ktime_get.sched_clock_tick.scheduler_tick.update_process_times.tick_sched_handle
      2.19 ±  8%     -93.6%       0.14 ± 10%  perf-profile.cpu-cycles._raw_spin_lock.try_to_wake_up.default_wake_function.autoremove_wake_function.__wake_up_common
      3.88 ±  9%     -94.1%       0.23 ± 21%  perf-profile.cpu-cycles.load_balance.pick_next_task_fair.__schedule.schedule.schedule_timeout
      6.29 ±  7%     -96.7%       0.21 ± 40%  perf-profile.cpu-cycles._raw_spin_lock_irqsave.__account_scheduler_latency.enqueue_entity.enqueue_task_fair.enqueue_task
      0.94 ±  5%     -90.7%       0.09 ± 13%  perf-profile.cpu-cycles.local_apic_timer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.__schedule.schedule
      0.94 ±  5%     -90.6%       0.09 ± 13%  perf-profile.cpu-cycles.hrtimer_interrupt.local_apic_timer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.__schedule
      4.19 ±  2%     +61.7%       6.77 ±  3%  perf-profile.cpu-cycles.ktime_get.cpuidle_enter_state.cpuidle_enter.cpu_startup_entry.start_secondary
     15.10 ±  3%     -27.5%      10.94 ±  1%  perf-profile.cpu-cycles.inet_recvmsg.sock_recvmsg.SYSC_recvfrom.sys_recvfrom.system_call_fastpath
     15.19 ±  3%     -27.2%      11.07 ±  1%  perf-profile.cpu-cycles.sock_recvmsg.SYSC_recvfrom.sys_recvfrom.system_call_fastpath
     15.35 ±  3%     -27.0%      11.20 ±  1%  perf-profile.cpu-cycles.SYSC_recvfrom.sys_recvfrom.system_call_fastpath
     15.44 ±  3%     -27.0%      11.27 ±  1%  perf-profile.cpu-cycles.sys_recvfrom.system_call_fastpath
     14.72 ±  3%     -26.5%      10.82 ±  1%  perf-profile.cpu-cycles.tcp_recvmsg.inet_recvmsg.sock_recvmsg.SYSC_recvfrom.sys_recvfrom
      1.57 ±  3%     +33.4%       2.10 ±  4%  perf-profile.cpu-cycles.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.tcp_prequeue_process.tcp_recvmsg
     15.03 ±  4%     -25.0%      11.27 ±  2%  perf-profile.cpu-cycles.try_to_wake_up.default_wake_function.autoremove_wake_function.__wake_up_common.__wake_up_sync_key
     23.41 ±  3%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.ret_from_intr.cpu_startup_entry.start_secondary
      1.19 ± 12%     -80.5%       0.23 ± 19%  perf-profile.cpu-cycles.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__run_hrtimer
      4.24 ±  2%     +58.3%       6.72 ±  3%  perf-profile.cpu-cycles.read_hpet.ktime_get.cpuidle_enter_state.cpuidle_enter.cpu_startup_entry
     15.05 ±  4%     -24.9%      11.30 ±  2%  perf-profile.cpu-cycles.default_wake_function.autoremove_wake_function.__wake_up_common.__wake_up_sync_key.tcp_prequeue
      2.81 ±  9%     -90.6%       0.26 ± 20%  perf-profile.cpu-cycles.find_busiest_group.load_balance.pick_next_task_fair.__schedule.schedule
      9.12 ±  4%     -52.1%       4.36 ±  4%  perf-profile.cpu-cycles.sk_wait_data.tcp_recvmsg.inet_recvmsg.sock_recvmsg.SYSC_recvfrom
      0.46 ±  2%    +127.4%       1.04 ±  6%  perf-profile.cpu-cycles.pick_next_task_fair.__schedule.schedule.schedule_preempt_disabled.cpu_startup_entry
      2.52 ±  2%     -62.9%       0.94 ± 31%  perf-profile.cpu-cycles.__run_hrtimer.hrtimer_interrupt.local_apic_timer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
      1.58 ±  8%     -81.7%       0.29 ± 21%  perf-profile.cpu-cycles.tick_sched_handle.isra.18.tick_sched_timer.__run_hrtimer.hrtimer_interrupt.local_apic_timer_interrupt
      2.49 ±  1%     -64.0%       0.90 ± 33%  perf-profile.cpu-cycles.tick_sched_timer.__run_hrtimer.hrtimer_interrupt.local_apic_timer_interrupt.smp_apic_timer_interrupt
     15.06 ±  4%     -24.9%      11.32 ±  2%  perf-profile.cpu-cycles.autoremove_wake_function.__wake_up_common.__wake_up_sync_key.tcp_prequeue.tcp_v4_rcv
      1.35 ±  3%     +32.3%       1.79 ±  8%  perf-profile.cpu-cycles.deactivate_task.__schedule.schedule.schedule_timeout.sk_wait_data
     23.23 ±  3%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.do_IRQ.ret_from_intr.cpu_startup_entry.start_secondary
      5.90 ±  2%     -23.8%       4.50 ±  6%  perf-profile.cpu-cycles.read_hpet.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.tick_nohz_stop_idle
      1.72 ±  8%     -83.2%       0.29 ± 21%  perf-profile.cpu-cycles.update_process_times.tick_sched_handle.tick_sched_timer.__run_hrtimer.hrtimer_interrupt
      1.55 ±  6%     +48.4%       2.31 ±  5%  perf-profile.cpu-cycles.dev_queue_xmit_sk.ip_finish_output.ip_output.ip_local_out_sk.ip_queue_xmit
      1.35 ±  3%     +30.7%       1.76 ±  7%  perf-profile.cpu-cycles.dequeue_task.deactivate_task.__schedule.schedule.schedule_timeout
     15.75 ±  3%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.irq_exit.do_IRQ.ret_from_intr.cpu_startup_entry.start_secondary
      1.54 ±  5%     +48.3%       2.28 ±  5%  perf-profile.cpu-cycles.__dev_queue_xmit.dev_queue_xmit_sk.ip_finish_output.ip_output.ip_local_out_sk
      3.70 ±  1%     +49.0%       5.51 ±  1%  perf-profile.cpu-cycles.tcp_v4_do_rcv.tcp_prequeue_process.tcp_recvmsg.inet_recvmsg.sock_recvmsg
     72.13 ±  0%     -24.2%      54.66 ±  0%  perf-profile.cpu-cycles.cpu_startup_entry.start_secondary
      8.28 ±  5%     -61.0%       3.23 ±  5%  perf-profile.cpu-cycles.schedule.schedule_timeout.sk_wait_data.tcp_recvmsg.inet_recvmsg
      3.44 ±  3%     +51.2%       5.19 ±  1%  perf-profile.cpu-cycles.tcp_rcv_established.tcp_v4_do_rcv.tcp_prequeue_process.tcp_recvmsg.inet_recvmsg
      2.75 ±  9%     -91.1%       0.24 ± 20%  perf-profile.cpu-cycles.update_sd_lb_stats.find_busiest_group.load_balance.pick_next_task_fair.__schedule
      8.36 ±  5%     -61.0%       3.26 ±  5%  perf-profile.cpu-cycles.schedule_timeout.sk_wait_data.tcp_recvmsg.inet_recvmsg.sock_recvmsg
      0.33 ± 29%    +218.5%       1.06 ± 10%  perf-profile.cpu-cycles.sk_stream_alloc_skb.tcp_sendmsg.inet_sendmsg.sock_sendmsg.SYSC_sendto
     15.13 ±  4%     -24.2%      11.46 ±  2%  perf-profile.cpu-cycles.__wake_up_common.__wake_up_sync_key.tcp_prequeue.tcp_v4_rcv.ip_local_deliver_finish
     72.31 ±  0%     -24.2%      54.83 ±  0%  perf-profile.cpu-cycles.start_secondary
     14.20 ±  3%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.__do_softirq.irq_exit.do_IRQ.ret_from_intr.cpu_startup_entry
      1.30 ±  3%     +24.4%       1.62 ±  8%  perf-profile.cpu-cycles.dequeue_task_fair.dequeue_task.deactivate_task.__schedule.schedule
      1.34 ±  5%     +30.0%       1.74 ±  4%  perf-profile.cpu-cycles.sch_direct_xmit.__dev_queue_xmit.dev_queue_xmit_sk.ip_finish_output.ip_output
      8.93 ±  3%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.ret_from_intr.cpuidle_enter.cpu_startup_entry.start_secondary
     15.18 ±  1%     +26.1%      19.14 ±  3%  perf-profile.cpu-cycles.tick_nohz_idle_exit.cpu_startup_entry.start_secondary
      4.09 ±  0%      +9.8%       4.49 ±  4%  perf-profile.cpu-cycles.ktime_get.clockevents_program_event.tick_program_event.__remove_hrtimer.__hrtimer_start_range_ns
      8.08 ±  5%     -62.1%       3.06 ±  6%  perf-profile.cpu-cycles.__schedule.schedule.schedule_timeout.sk_wait_data.tcp_recvmsg
      3.87 ±  2%     +12.2%       4.35 ±  7%  perf-profile.cpu-cycles.read_hpet.ktime_get.__tick_nohz_idle_enter.tick_nohz_idle_enter.cpu_startup_entry
      5.80 ±  1%     -12.5%       5.07 ±  5%  perf-profile.cpu-cycles.ktime_get_with_offset.netif_receive_skb_internal.napi_gro_receive.ixgbe_clean_rx_irq.ixgbe_poll
      0.36 ±  4%    +188.3%       1.02 ±  5%  perf-profile.cpu-cycles.dev_hard_start_xmit.sch_direct_xmit.__dev_queue_xmit.dev_queue_xmit_sk.ip_finish_output
      4.72 ±  3%    +186.8%      13.52 ±  6%  perf-profile.cpu-cycles.cpuidle_enter_state.cpuidle_enter.cpu_startup_entry.start_secondary
     10.92 ±  4%     -11.1%       9.71 ±  4%  perf-profile.cpu-cycles.enqueue_entity.enqueue_task_fair.enqueue_task.activate_task.ttwu_do_activate
      2.95 ±  3%     -18.6%       2.40 ±  3%  perf-profile.cpu-cycles.__schedule.schedule.schedule_preempt_disabled.cpu_startup_entry.start_secondary
      3.83 ±  2%     +14.0%       4.36 ±  7%  perf-profile.cpu-cycles.ktime_get.__tick_nohz_idle_enter.tick_nohz_idle_enter.cpu_startup_entry.start_secondary
      8.92 ±  3%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.do_IRQ.ret_from_intr.cpuidle_enter.cpu_startup_entry.start_secondary
      7.38 ±  1%     +13.6%       8.38 ±  2%  perf-profile.cpu-cycles.sock_sendmsg.SYSC_sendto.sys_sendto.system_call_fastpath
     11.19 ±  4%     -11.1%       9.95 ±  2%  perf-profile.cpu-cycles.ttwu_do_activate.constprop.88.try_to_wake_up.default_wake_function.autoremove_wake_function.__wake_up_common
     15.76 ±  4%     -20.9%      12.46 ±  3%  perf-profile.cpu-cycles.__wake_up_sync_key.tcp_prequeue.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver
      7.73 ±  2%     +14.4%       8.85 ±  2%  perf-profile.cpu-cycles.SYSC_sendto.sys_sendto.system_call_fastpath
     13.61 ±  1%     +12.6%      15.31 ±  3%  perf-profile.cpu-cycles.tick_nohz_idle_enter.cpu_startup_entry.start_secondary
      2.97 ±  2%     +14.2%       3.40 ±  4%  perf-profile.cpu-cycles.ip_local_out_sk.ip_queue_xmit.tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames
      7.79 ±  2%     +14.7%       8.93 ±  2%  perf-profile.cpu-cycles.sys_sendto.system_call_fastpath
      4.63 ±  1%     +24.6%       5.77 ±  1%  perf-profile.cpu-cycles.tcp_prequeue_process.tcp_recvmsg.inet_recvmsg.sock_recvmsg.SYSC_recvfrom
      2.87 ±  2%     +13.8%       3.26 ±  3%  perf-profile.cpu-cycles.ip_output.ip_local_out_sk.ip_queue_xmit.tcp_transmit_skb.tcp_write_xmit
      6.59 ±  4%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.irq_exit.do_IRQ.ret_from_intr.cpuidle_enter.cpu_startup_entry
      3.04 ±  3%     -17.9%       2.49 ±  3%  perf-profile.cpu-cycles.schedule.schedule_preempt_disabled.cpu_startup_entry.start_secondary
     11.17 ±  4%     -18.6%       9.09 ±  2%  perf-profile.cpu-cycles.enqueue_task_fair.enqueue_task.activate_task.ttwu_do_activate.try_to_wake_up
     23.39 ±  2%     -12.3%      20.51 ±  1%  perf-profile.cpu-cycles.system_call_fastpath
      7.03 ±  1%     +16.0%       8.15 ±  2%  perf-profile.cpu-cycles.inet_sendmsg.sock_sendmsg.SYSC_sendto.sys_sendto.system_call_fastpath
      4.16 ±  0%     +14.6%       4.76 ±  4%  perf-profile.cpu-cycles.tick_program_event.__remove_hrtimer.__hrtimer_start_range_ns.hrtimer_start.tick_nohz_stop_sched_tick
      4.15 ±  0%     +14.7%       4.76 ±  4%  perf-profile.cpu-cycles.clockevents_program_event.tick_program_event.__remove_hrtimer.__hrtimer_start_range_ns.hrtimer_start
      3.05 ±  3%     -16.3%       2.55 ±  3%  perf-profile.cpu-cycles.schedule_preempt_disabled.cpu_startup_entry.start_secondary
      6.33 ±  4%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.__do_softirq.irq_exit.do_IRQ.ret_from_intr.cpuidle_enter
      6.78 ±  4%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.tick_irq_enter.irq_enter.do_IRQ.ret_from_intr.cpu_startup_entry
      6.75 ±  4%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.irq_enter.do_IRQ.ret_from_intr.cpu_startup_entry.start_secondary
      4.73 ±  3%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.sched_clock_tick.sched_clock_idle_wakeup_event.tick_nohz_stop_idle.tick_irq_enter.irq_enter
     10.95 ±  5%     -13.5%       9.47 ±  2%  perf-profile.cpu-cycles.activate_task.ttwu_do_activate.try_to_wake_up.default_wake_function.autoremove_wake_function
      4.62 ±  3%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.tick_nohz_stop_idle.tick_irq_enter
      4.21 ±  1%     +21.2%       5.10 ±  4%  perf-profile.cpu-cycles.ktime_get.tick_nohz_idle_exit.cpu_startup_entry.start_secondary
      1.29 ±  4%     +21.1%       1.57 ±  3%  perf-profile.cpu-cycles.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.tcp_prequeue_process
      9.15 ±  1%     +16.1%      10.63 ±  2%  perf-profile.cpu-cycles.tick_nohz_stop_sched_tick.__tick_nohz_idle_enter.tick_nohz_idle_enter.cpu_startup_entry.start_secondary
     10.94 ±  5%     -13.5%       9.46 ±  2%  perf-profile.cpu-cycles.enqueue_task.activate_task.ttwu_do_activate.try_to_wake_up.default_wake_function
      0.17 ± 35%    +479.8%       0.96 ± 10%  perf-profile.cpu-cycles.__alloc_skb.sk_stream_alloc_skb.tcp_sendmsg.inet_sendmsg.sock_sendmsg
      4.32 ±  9%     -88.6%       0.49 ± 13%  perf-profile.cpu-cycles.pick_next_task_fair.__schedule.schedule.schedule_timeout.sk_wait_data
      4.54 ±  3%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.tick_nohz_stop_idle.tick_irq_enter.irq_enter.do_IRQ.ret_from_intr
     16.82 ±  4%     -15.0%      14.30 ±  3%  perf-profile.cpu-cycles.ip_local_deliver.ip_rcv_finish.ip_rcv.__netif_receive_skb_core.__netif_receive_skb
     16.81 ±  4%     -15.0%      14.28 ±  3%  perf-profile.cpu-cycles.ip_local_deliver_finish.ip_local_deliver.ip_rcv_finish.ip_rcv.__netif_receive_skb_core
      4.28 ±  2%     -14.4%       3.66 ±  6%  perf-profile.cpu-cycles.ktime_get.clockevents_program_event.tick_program_event.__remove_hrtimer.hrtimer_try_to_cancel
     24.19 ±  3%     -15.8%      20.36 ±  3%  perf-profile.cpu-cycles.napi_gro_receive.ixgbe_clean_rx_irq.ixgbe_poll.net_rx_action.__do_softirq
     23.93 ±  3%     -15.8%      20.15 ±  3%  perf-profile.cpu-cycles.netif_receive_skb_internal.napi_gro_receive.ixgbe_clean_rx_irq.ixgbe_poll.net_rx_action
      4.03 ±  1%     +15.2%       4.64 ±  3%  perf-profile.cpu-cycles.tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_push.tcp_sendmsg
      9.82 ±  4%     -14.5%       8.40 ±  3%  perf-profile.cpu-cycles.__account_scheduler_latency.enqueue_entity.enqueue_task_fair.enqueue_task.activate_task
     13.12 ±  1%     +15.9%      15.21 ±  3%  perf-profile.cpu-cycles.__tick_nohz_idle_enter.tick_nohz_idle_enter.cpu_startup_entry.start_secondary
      6.71 ±  1%     +18.5%       7.95 ±  2%  perf-profile.cpu-cycles.tcp_sendmsg.inet_sendmsg.sock_sendmsg.SYSC_sendto.sys_sendto
      8.57 ±  1%     +17.2%      10.05 ±  3%  perf-profile.cpu-cycles.hrtimer_start.tick_nohz_stop_sched_tick.__tick_nohz_idle_enter.tick_nohz_idle_enter.cpu_startup_entry
     16.71 ±  4%     -16.1%      14.02 ±  3%  perf-profile.cpu-cycles.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver.ip_rcv_finish.ip_rcv
      0.14 ± 15%    +573.8%       0.94 ±  9%  perf-profile.cpu-cycles.tcp_send_delayed_ack.__tcp_ack_snd_check.tcp_rcv_established.tcp_v4_do_rcv.tcp_prequeue_process
      0.10 ± 23%    +830.0%       0.93 ±  5%  perf-profile.cpu-cycles.ixgbe_xmit_frame.dev_hard_start_xmit.sch_direct_xmit.__dev_queue_xmit.dev_queue_xmit_sk
      0.08 ±  6%   +1213.0%       1.01 ±  4%  perf-profile.cpu-cycles.is_module_text_address.print_context_stack.dump_trace.save_stack_trace_tsk.__account_scheduler_latency
      4.48 ±  3%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.sched_clock_idle_wakeup_event.tick_nohz_stop_idle.tick_irq_enter.irq_enter.do_IRQ
      4.57 ±  2%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.ktime_get.tick_irq_enter.irq_enter.do_IRQ.ret_from_intr
      8.57 ±  1%     +16.9%      10.02 ±  3%  perf-profile.cpu-cycles.__hrtimer_start_range_ns.hrtimer_start.tick_nohz_stop_sched_tick.__tick_nohz_idle_enter.tick_nohz_idle_enter
     17.72 ±  4%     -16.3%      14.84 ±  2%  perf-profile.cpu-cycles.ip_rcv_finish.ip_rcv.__netif_receive_skb_core.__netif_receive_skb.netif_receive_skb_internal
      4.57 ±  2%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.read_hpet.ktime_get.tick_irq_enter.irq_enter.do_IRQ
      0.96 ±  4%     -85.4%       0.14 ± 35%  perf-profile.cpu-cycles.read_hpet.ktime_get.sched_clock_tick.scheduler_tick.update_process_times
     18.03 ±  4%     -16.6%      15.03 ±  2%  perf-profile.cpu-cycles.__netif_receive_skb.netif_receive_skb_internal.napi_gro_receive.ixgbe_clean_rx_irq.ixgbe_poll
      4.99 ±  1%     +17.0%       5.84 ±  2%  perf-profile.cpu-cycles.tcp_push.tcp_sendmsg.inet_sendmsg.sock_sendmsg.SYSC_sendto
     18.00 ±  4%     -16.7%      15.00 ±  2%  perf-profile.cpu-cycles.__netif_receive_skb_core.__netif_receive_skb.netif_receive_skb_internal.napi_gro_receive.ixgbe_clean_rx_irq
      3.16 ±  1%     +20.8%       3.82 ±  3%  perf-profile.cpu-cycles.ip_queue_xmit.tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_push
      4.87 ±  2%     +18.7%       5.78 ±  2%  perf-profile.cpu-cycles.__tcp_push_pending_frames.tcp_push.tcp_sendmsg.inet_sendmsg.sock_sendmsg
      2.36 ±  1%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.irq_enter.do_IRQ.ret_from_intr.cpuidle_enter.cpu_startup_entry
      0.23 ±  6%    +443.4%       1.23 ±  3%  perf-profile.cpu-cycles.is_module_text_address.__kernel_text_address.print_context_stack.dump_trace.save_stack_trace_tsk
      0.47 ±  4%    +684.2%       3.65 ±  3%  perf-profile.cpu-cycles.__kernel_text_address.print_context_stack.dump_trace.save_stack_trace_tsk.__account_scheduler_latency
      1.37 ±  4%    +239.8%       4.65 ±  7%  perf-profile.cpu-cycles.tick_nohz_stop_idle.tick_nohz_idle_exit.cpu_startup_entry.start_secondary
      1.34 ±  4%    +243.2%       4.61 ±  7%  perf-profile.cpu-cycles.sched_clock_tick.sched_clock_idle_wakeup_event.tick_nohz_stop_idle.tick_nohz_idle_exit.cpu_startup_entry
      0.43 ±  7%   +1444.2%       6.64 ± 15%  perf-profile.cpu-cycles.intel_idle.cpuidle_enter_state.cpuidle_enter.cpu_startup_entry.start_secondary
      0.54 ±  7%   +3863.4%      21.27 ±  3%  perf-profile.cpu-cycles.ixgbe_clean_rx_irq.ixgbe_poll.net_rx_action.__do_softirq.run_ksoftirqd
      0.54 ±  7%   +4049.4%      22.41 ±  2%  perf-profile.cpu-cycles.ixgbe_poll.net_rx_action.__do_softirq.run_ksoftirqd.smpboot_thread_fn
      0.55 ±  6%   +3975.2%      22.48 ±  2%  perf-profile.cpu-cycles.net_rx_action.__do_softirq.run_ksoftirqd.smpboot_thread_fn.kthread
      1.28 ±  3%    +246.7%       4.43 ±  7%  perf-profile.cpu-cycles.ktime_get.sched_clock_tick.sched_clock_idle_wakeup_event.tick_nohz_stop_idle.tick_nohz_idle_exit
      4.21 ±  0%     +18.0%       4.97 ±  3%  perf-profile.cpu-cycles.__remove_hrtimer.__hrtimer_start_range_ns.hrtimer_start.tick_nohz_stop_sched_tick.__tick_nohz_idle_enter
      5.58 ±  1%     -15.9%       4.69 ±  3%  perf-profile.cpu-cycles.read_hpet.ktime_get_with_offset.netif_receive_skb_internal.napi_gro_receive.ixgbe_clean_rx_irq
      4.77 ±  1%     +19.6%       5.70 ±  2%  perf-profile.cpu-cycles.tcp_write_xmit.__tcp_push_pending_frames.tcp_push.tcp_sendmsg.inet_sendmsg
      1.33 ±  4%    +248.1%       4.62 ±  7%  perf-profile.cpu-cycles.sched_clock_idle_wakeup_event.tick_nohz_stop_idle.tick_nohz_idle_exit.cpu_startup_entry.start_secondary
      1.92 ±  2%    +283.1%       7.36 ±  2%  perf-profile.cpu-cycles.dump_trace.save_stack_trace_tsk.__account_scheduler_latency.enqueue_entity.enqueue_task_fair
      1.93 ±  2%    +285.6%       7.46 ±  2%  perf-profile.cpu-cycles.save_stack_trace_tsk.__account_scheduler_latency.enqueue_entity.enqueue_task_fair.enqueue_task
      2.20 ±  3%     +19.0%       2.61 ±  4%  perf-profile.cpu-cycles.ip_finish_output.ip_output.ip_local_out_sk.ip_queue_xmit.tcp_transmit_skb
     16.33 ±  4%     -17.9%      13.40 ±  3%  perf-profile.cpu-cycles.tcp_prequeue.tcp_v4_rcv.ip_local_deliver_finish.ip_local_deliver.ip_rcv_finish
      0.99 ±  4%   +2179.8%      22.61 ±  2%  perf-profile.cpu-cycles.run_ksoftirqd.smpboot_thread_fn.kthread.ret_from_fork
      0.97 ±  4%   +2217.9%      22.60 ±  2%  perf-profile.cpu-cycles.__do_softirq.run_ksoftirqd.smpboot_thread_fn.kthread.ret_from_fork
      1.13 ±  3%   +1906.2%      22.67 ±  2%  perf-profile.cpu-cycles.smpboot_thread_fn.kthread.ret_from_fork
      1.15 ±  2%   +1878.6%      22.82 ±  2%  perf-profile.cpu-cycles.kthread.ret_from_fork
      1.02 ±  2%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.x86_64_start_reservations.x86_64_start_kernel
      1.15 ±  2%   +1879.0%      22.82 ±  2%  perf-profile.cpu-cycles.ret_from_fork
      2.34 ±  1%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.tick_irq_enter.irq_enter.do_IRQ.ret_from_intr.cpuidle_enter
      1.62 ±  4%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.__tick_nohz_idle_enter.tick_nohz_irq_exit.irq_exit.do_IRQ.ret_from_intr
      1.61 ±  3%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.read_hpet.ktime_get.__tick_nohz_idle_enter.tick_nohz_irq_exit.irq_exit
      1.56 ±  4%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.tick_nohz_irq_exit.irq_exit.do_IRQ.ret_from_intr.cpu_startup_entry
      1.48 ±  3%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.ktime_get.__tick_nohz_idle_enter.tick_nohz_irq_exit.irq_exit.do_IRQ
      0.21 ± 12%    +351.6%       0.96 ±  8%  perf-profile.cpu-cycles.__tcp_ack_snd_check.tcp_rcv_established.tcp_v4_do_rcv.tcp_prequeue_process.tcp_recvmsg
      1.72 ±  2%    +301.4%       6.90 ±  2%  perf-profile.cpu-cycles.print_context_stack.dump_trace.save_stack_trace_tsk.__account_scheduler_latency.enqueue_entity
      1.02 ±  2%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.x86_64_start_kernel
      1.02 ±  2%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.rest_init.start_kernel.x86_64_start_reservations.x86_64_start_kernel
      1.02 ±  2%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.start_kernel.x86_64_start_reservations.x86_64_start_kernel
      4.21 ±  1%     +20.6%       5.08 ±  4%  perf-profile.cpu-cycles.read_hpet.ktime_get.tick_nohz_idle_exit.cpu_startup_entry.start_secondary
       528 ±  6%     +61.7%        854 ±  1%  sched_debug.cfs_rq[0]:/.tg_runnable_contrib
     24162 ±  6%     +62.3%      39214 ±  1%  sched_debug.cfs_rq[0]:/.avg->runnable_avg_sum
   4882658 ±  9%     -96.6%     167048 ±  8%  sched_debug.cfs_rq[0]:/.min_vruntime
     37303 ±  1%     -89.7%       3826 ±  4%  sched_debug.cfs_rq[0]:/.tg->runnable_avg
     77353 ±  9%     +81.9%     140694 ±  7%  sched_debug.cfs_rq[0]:/.exec_clock
     38349 ±  2%     -89.4%       4070 ±  3%  sched_debug.cfs_rq[100]:/.tg->runnable_avg
     38365 ±  2%     -89.4%       4071 ±  3%  sched_debug.cfs_rq[101]:/.tg->runnable_avg
     38369 ±  2%     -89.4%       4074 ±  3%  sched_debug.cfs_rq[102]:/.tg->runnable_avg
     38382 ±  2%     -89.4%       4075 ±  3%  sched_debug.cfs_rq[103]:/.tg->runnable_avg
     38383 ±  2%     -89.4%       4077 ±  3%  sched_debug.cfs_rq[104]:/.tg->runnable_avg
     38391 ±  2%     -89.4%       4079 ±  3%  sched_debug.cfs_rq[105]:/.tg->runnable_avg
     38388 ±  2%     -89.4%       4083 ±  3%  sched_debug.cfs_rq[106]:/.tg->runnable_avg
     38402 ±  2%     -89.4%       4084 ±  3%  sched_debug.cfs_rq[107]:/.tg->runnable_avg
     38401 ±  2%     -89.4%       4087 ±  3%  sched_debug.cfs_rq[108]:/.tg->runnable_avg
     38413 ±  2%     -89.4%       4088 ±  3%  sched_debug.cfs_rq[109]:/.tg->runnable_avg
       548 ±  3%     -93.4%         36 ± 22%  sched_debug.cfs_rq[10]:/.tg_runnable_contrib
     37450 ±  1%     -89.7%       3851 ±  4%  sched_debug.cfs_rq[10]:/.tg->runnable_avg
   5772463 ±  8%     -98.4%      93465 ± 22%  sched_debug.cfs_rq[10]:/.min_vruntime
     76594 ±  8%     -91.7%       6366 ± 15%  sched_debug.cfs_rq[10]:/.exec_clock
     25036 ±  3%     -93.3%       1688 ± 20%  sched_debug.cfs_rq[10]:/.avg->runnable_avg_sum
     38410 ±  2%     -89.4%       4089 ±  3%  sched_debug.cfs_rq[110]:/.tg->runnable_avg
     38417 ±  2%     -89.4%       4088 ±  3%  sched_debug.cfs_rq[111]:/.tg->runnable_avg
     38435 ±  2%     -89.4%       4091 ±  3%  sched_debug.cfs_rq[112]:/.tg->runnable_avg
     38435 ±  2%     -89.4%       4092 ±  3%  sched_debug.cfs_rq[113]:/.tg->runnable_avg
     38428 ±  2%     -89.3%       4095 ±  3%  sched_debug.cfs_rq[114]:/.tg->runnable_avg
     38439 ±  1%     -89.3%       4096 ±  3%  sched_debug.cfs_rq[115]:/.tg->runnable_avg
     38445 ±  2%     -89.3%       4095 ±  3%  sched_debug.cfs_rq[116]:/.tg->runnable_avg
     38449 ±  2%     -89.3%       4097 ±  3%  sched_debug.cfs_rq[117]:/.tg->runnable_avg
     38461 ±  2%     -89.3%       4100 ±  3%  sched_debug.cfs_rq[118]:/.tg->runnable_avg
     38460 ±  1%     -89.3%       4102 ±  3%  sched_debug.cfs_rq[119]:/.tg->runnable_avg
     76608 ±  7%     -91.7%       6356 ± 16%  sched_debug.cfs_rq[11]:/.exec_clock
       546 ±  2%     -93.5%         35 ± 25%  sched_debug.cfs_rq[11]:/.tg_runnable_contrib
     37465 ±  1%     -89.7%       3852 ±  4%  sched_debug.cfs_rq[11]:/.tg->runnable_avg
     24985 ±  2%     -93.3%       1664 ± 24%  sched_debug.cfs_rq[11]:/.avg->runnable_avg_sum
   5775629 ±  7%     -98.4%      91917 ± 12%  sched_debug.cfs_rq[11]:/.min_vruntime
     76166 ±  8%     -92.2%       5923 ± 22%  sched_debug.cfs_rq[12]:/.exec_clock
   5756491 ±  8%     -98.5%      86026 ± 20%  sched_debug.cfs_rq[12]:/.min_vruntime
         5 ± 31%    -100.0%          0 ±  0%  sched_debug.cfs_rq[12]:/.runnable_load_avg
     37492 ±  1%     -89.7%       3853 ±  4%  sched_debug.cfs_rq[12]:/.tg->runnable_avg
     75892 ±  8%     -91.8%       6211 ± 19%  sched_debug.cfs_rq[13]:/.exec_clock
     37506 ±  1%     -89.7%       3856 ±  4%  sched_debug.cfs_rq[13]:/.tg->runnable_avg
   5727275 ±  7%     -98.4%      93734 ± 18%  sched_debug.cfs_rq[13]:/.min_vruntime
     37528 ±  1%     -89.7%       3857 ±  4%  sched_debug.cfs_rq[14]:/.tg->runnable_avg
     25080 ±  5%     -94.4%       1394 ± 43%  sched_debug.cfs_rq[14]:/.avg->runnable_avg_sum
     75791 ±  8%     -92.2%       5907 ± 24%  sched_debug.cfs_rq[14]:/.exec_clock
       550 ±  4%     -94.6%         29 ± 43%  sched_debug.cfs_rq[14]:/.tg_runnable_contrib
         6 ± 34%    -100.0%          0 ±  0%  sched_debug.cfs_rq[14]:/.runnable_load_avg
   5717919 ±  7%     -98.4%      90360 ± 22%  sched_debug.cfs_rq[14]:/.min_vruntime
     37554 ±  1%     -89.7%       3860 ±  4%  sched_debug.cfs_rq[15]:/.tg->runnable_avg
     71299 ± 11%     -96.4%       2537 ± 18%  sched_debug.cfs_rq[15]:/.exec_clock
   5471648 ± 11%     -99.1%      50259 ± 22%  sched_debug.cfs_rq[15]:/.min_vruntime
     71273 ± 12%     -96.7%       2353 ± 10%  sched_debug.cfs_rq[16]:/.exec_clock
   5469742 ± 11%     -99.0%      53306 ± 24%  sched_debug.cfs_rq[16]:/.min_vruntime
     37563 ±  1%     -89.7%       3864 ±  4%  sched_debug.cfs_rq[16]:/.tg->runnable_avg
2807093.70 ± 36%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[17]:/.MIN_vruntime
         6 ± 31%    -100.0%          0 ±  0%  sched_debug.cfs_rq[17]:/.runnable_load_avg
     37568 ±  1%     -89.7%       3865 ±  4%  sched_debug.cfs_rq[17]:/.tg->runnable_avg
     71654 ± 12%     -96.9%       2229 ± 14%  sched_debug.cfs_rq[17]:/.exec_clock
   5474236 ± 11%     -99.1%      50364 ± 25%  sched_debug.cfs_rq[17]:/.min_vruntime
2807093.70 ± 36%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[17]:/.max_vruntime
     37574 ±  1%     -89.7%       3866 ±  4%  sched_debug.cfs_rq[18]:/.tg->runnable_avg
2611368.18 ± 47%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[18]:/.max_vruntime
2611367.85 ± 47%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[18]:/.MIN_vruntime
   5469964 ± 11%     -99.2%      42263 ± 36%  sched_debug.cfs_rq[18]:/.min_vruntime
         4 ± 28%    -100.0%          0 ±  0%  sched_debug.cfs_rq[19]:/.runnable_load_avg
   5460827 ± 11%     -99.2%      46250 ± 33%  sched_debug.cfs_rq[19]:/.min_vruntime
     37603 ±  1%     -89.7%       3871 ±  4%  sched_debug.cfs_rq[19]:/.tg->runnable_avg
   5931126 ±  8%     -86.0%     829332 ± 28%  sched_debug.cfs_rq[1]:/.min_vruntime
     37333 ±  1%     -89.7%       3834 ±  4%  sched_debug.cfs_rq[1]:/.tg->runnable_avg
   1048433 ± 16%     -36.8%     662283 ± 34%  sched_debug.cfs_rq[1]:/.spread0
   5501605 ± 11%     -99.1%      46820 ± 29%  sched_debug.cfs_rq[20]:/.min_vruntime
     71374 ± 11%     -97.4%       1825 ± 22%  sched_debug.cfs_rq[20]:/.exec_clock
     37630 ±  1%     -89.7%       3876 ±  4%  sched_debug.cfs_rq[20]:/.tg->runnable_avg
         4 ± 49%    -100.0%          0 ±  0%  sched_debug.cfs_rq[20]:/.runnable_load_avg
   5435674 ± 11%     -99.1%      48697 ± 31%  sched_debug.cfs_rq[21]:/.min_vruntime
     37645 ±  1%     -89.7%       3875 ±  4%  sched_debug.cfs_rq[21]:/.tg->runnable_avg
         7 ± 40%    -100.0%          0 ±  0%  sched_debug.cfs_rq[21]:/.load
       122 ± 48%    -100.0%          0 ±  0%  sched_debug.cfs_rq[21]:/.utilization_load_avg
     70205 ± 12%     -97.3%       1867 ± 36%  sched_debug.cfs_rq[21]:/.exec_clock
         7 ± 44%    -100.0%          0 ±  0%  sched_debug.cfs_rq[22]:/.load
     37661 ±  1%     -89.7%       3876 ±  4%  sched_debug.cfs_rq[22]:/.tg->runnable_avg
   5491828 ± 11%     -99.2%      45143 ± 26%  sched_debug.cfs_rq[22]:/.min_vruntime
         6 ± 32%    -100.0%          0 ±  0%  sched_debug.cfs_rq[23]:/.load
         4 ± 35%    -100.0%          0 ±  0%  sched_debug.cfs_rq[23]:/.runnable_load_avg
        90 ± 40%    -100.0%          0 ±  0%  sched_debug.cfs_rq[23]:/.utilization_load_avg
     37652 ±  1%     -89.7%       3879 ±  4%  sched_debug.cfs_rq[23]:/.tg->runnable_avg
   5499468 ± 11%     -99.2%      44829 ± 30%  sched_debug.cfs_rq[23]:/.min_vruntime
     37651 ±  1%     -89.7%       3882 ±  4%  sched_debug.cfs_rq[24]:/.tg->runnable_avg
         7 ± 47%    -100.0%          0 ±  0%  sched_debug.cfs_rq[24]:/.load
2002338.86 ± 45%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[24]:/.MIN_vruntime
   5458480 ± 11%     -99.2%      45851 ± 22%  sched_debug.cfs_rq[24]:/.min_vruntime
2002338.86 ± 45%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[24]:/.max_vruntime
       499 ± 11%     -98.4%          7 ± 49%  sched_debug.cfs_rq[24]:/.tg_runnable_contrib
     22791 ± 11%     -98.3%        393 ± 47%  sched_debug.cfs_rq[24]:/.avg->runnable_avg_sum
   5501548 ± 11%     -99.2%      42017 ± 39%  sched_debug.cfs_rq[25]:/.min_vruntime
     37675 ±  1%     -89.7%       3883 ±  4%  sched_debug.cfs_rq[25]:/.tg->runnable_avg
     37701 ±  1%     -89.7%       3887 ±  4%  sched_debug.cfs_rq[26]:/.tg->runnable_avg
   5460570 ± 11%     -99.2%      42120 ± 20%  sched_debug.cfs_rq[26]:/.min_vruntime
     72021 ±  8%     -97.9%       1500 ± 39%  sched_debug.cfs_rq[26]:/.exec_clock
   5443461 ± 11%     -99.3%      39281 ± 30%  sched_debug.cfs_rq[27]:/.min_vruntime
     37720 ±  1%     -89.7%       3890 ±  4%  sched_debug.cfs_rq[27]:/.tg->runnable_avg
     70407 ± 11%     -97.6%       1657 ± 41%  sched_debug.cfs_rq[27]:/.exec_clock
     37737 ±  1%     -89.7%       3890 ±  4%  sched_debug.cfs_rq[28]:/.tg->runnable_avg
     71813 ± 12%     -98.0%       1424 ± 44%  sched_debug.cfs_rq[28]:/.exec_clock
   5482457 ± 11%     -99.2%      42096 ± 24%  sched_debug.cfs_rq[28]:/.min_vruntime
   5478035 ± 11%     -99.2%      41368 ± 31%  sched_debug.cfs_rq[29]:/.min_vruntime
     37748 ±  1%     -89.7%       3893 ±  4%  sched_debug.cfs_rq[29]:/.tg->runnable_avg
   5924938 ±  8%     -86.5%     799517 ± 29%  sched_debug.cfs_rq[2]:/.min_vruntime
     37344 ±  1%     -89.7%       3839 ±  4%  sched_debug.cfs_rq[2]:/.tg->runnable_avg
   1042225 ± 20%     -39.3%     632468 ± 35%  sched_debug.cfs_rq[2]:/.spread0
     37765 ±  1%     -89.7%       3895 ±  4%  sched_debug.cfs_rq[30]:/.tg->runnable_avg
2514014.52 ± 44%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[30]:/.max_vruntime
   5791091 ± 11%     -99.1%      49961 ± 26%  sched_debug.cfs_rq[30]:/.min_vruntime
2514014.52 ± 44%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[30]:/.MIN_vruntime
     76335 ± 12%     -97.0%       2285 ± 16%  sched_debug.cfs_rq[30]:/.exec_clock
     76769 ± 11%     -97.4%       2026 ± 16%  sched_debug.cfs_rq[31]:/.exec_clock
2370182.85 ± 47%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[31]:/.MIN_vruntime
   5806067 ± 11%     -99.2%      46410 ± 22%  sched_debug.cfs_rq[31]:/.min_vruntime
2370183.19 ± 47%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[31]:/.max_vruntime
     37783 ±  1%     -89.7%       3903 ±  4%  sched_debug.cfs_rq[31]:/.tg->runnable_avg
     77107 ± 12%     -97.4%       1980 ± 30%  sched_debug.cfs_rq[32]:/.exec_clock
   5865705 ± 12%     -99.2%      45792 ± 32%  sched_debug.cfs_rq[32]:/.min_vruntime
     37808 ±  1%     -89.7%       3909 ±  4%  sched_debug.cfs_rq[32]:/.tg->runnable_avg
     24950 ±  9%     -96.2%        943 ± 23%  sched_debug.cfs_rq[33]:/.avg->runnable_avg_sum
     37805 ±  1%     -89.6%       3914 ±  4%  sched_debug.cfs_rq[33]:/.tg->runnable_avg
   5827248 ± 12%     -99.2%      44413 ± 27%  sched_debug.cfs_rq[33]:/.min_vruntime
     76627 ± 13%     -97.6%       1812 ± 27%  sched_debug.cfs_rq[33]:/.exec_clock
       546 ±  8%     -96.4%         19 ± 22%  sched_debug.cfs_rq[33]:/.tg_runnable_contrib
     37804 ±  1%     -89.6%       3915 ±  4%  sched_debug.cfs_rq[34]:/.tg->runnable_avg
     76396 ± 12%     -97.7%       1738 ± 29%  sched_debug.cfs_rq[34]:/.exec_clock
2052042.52 ± 30%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[34]:/.MIN_vruntime
   5789564 ± 11%     -99.2%      43627 ± 23%  sched_debug.cfs_rq[34]:/.min_vruntime
2052042.52 ± 30%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[34]:/.max_vruntime
     37810 ±  1%     -89.6%       3915 ±  4%  sched_debug.cfs_rq[35]:/.tg->runnable_avg
   5793549 ± 11%     -99.3%      40722 ± 15%  sched_debug.cfs_rq[35]:/.min_vruntime
     76584 ± 12%     -97.8%       1682 ± 27%  sched_debug.cfs_rq[35]:/.exec_clock
       553 ±  8%     -98.9%          6 ± 36%  sched_debug.cfs_rq[36]:/.tg_runnable_contrib
     37800 ±  1%     -89.6%       3917 ±  4%  sched_debug.cfs_rq[36]:/.tg->runnable_avg
     25249 ±  8%     -98.7%        330 ± 28%  sched_debug.cfs_rq[36]:/.avg->runnable_avg_sum
     76722 ± 13%     -97.9%       1576 ± 41%  sched_debug.cfs_rq[36]:/.exec_clock
   5843915 ± 12%     -99.3%      40252 ± 43%  sched_debug.cfs_rq[36]:/.min_vruntime
     75923 ± 12%     -97.7%       1718 ± 38%  sched_debug.cfs_rq[37]:/.exec_clock
   5768738 ± 11%     -99.3%      42192 ± 39%  sched_debug.cfs_rq[37]:/.min_vruntime
       535 ±  9%     -98.8%          6 ± 28%  sched_debug.cfs_rq[37]:/.tg_runnable_contrib
     37811 ±  1%     -89.6%       3924 ±  4%  sched_debug.cfs_rq[37]:/.tg->runnable_avg
     24479 ±  9%     -98.7%        316 ± 26%  sched_debug.cfs_rq[37]:/.avg->runnable_avg_sum
         7 ± 32%    -100.0%          0 ±  0%  sched_debug.cfs_rq[38]:/.load
     37833 ±  1%     -89.6%       3925 ±  4%  sched_debug.cfs_rq[38]:/.tg->runnable_avg
         5 ± 30%    -100.0%          0 ±  0%  sched_debug.cfs_rq[38]:/.runnable_load_avg
   5826004 ± 12%     -99.3%      43456 ± 44%  sched_debug.cfs_rq[38]:/.min_vruntime
       544 ± 10%     -98.4%          8 ± 27%  sched_debug.cfs_rq[39]:/.tg_runnable_contrib
         6 ± 21%    -100.0%          0 ±  0%  sched_debug.cfs_rq[39]:/.runnable_load_avg
     76688 ± 13%     -98.2%       1415 ± 35%  sched_debug.cfs_rq[39]:/.exec_clock
     37837 ±  1%     -89.6%       3925 ±  4%  sched_debug.cfs_rq[39]:/.tg->runnable_avg
     24864 ±  9%     -98.2%        454 ± 23%  sched_debug.cfs_rq[39]:/.avg->runnable_avg_sum
   5835422 ± 12%     -99.4%      36160 ± 31%  sched_debug.cfs_rq[39]:/.min_vruntime
     26232 ±  5%     -22.4%      20344 ±  8%  sched_debug.cfs_rq[3]:/.avg->runnable_avg_sum
       574 ±  6%     -22.5%        445 ±  8%  sched_debug.cfs_rq[3]:/.tg_runnable_contrib
     37364 ±  1%     -89.7%       3840 ±  4%  sched_debug.cfs_rq[3]:/.tg->runnable_avg
         9 ± 49%    +335.2%         39 ± 44%  sched_debug.cfs_rq[3]:/.load
   5891140 ±  8%     -87.4%     740621 ± 29%  sched_debug.cfs_rq[3]:/.min_vruntime
   1008386 ± 18%     -43.1%     573573 ± 35%  sched_debug.cfs_rq[3]:/.spread0
     76125 ± 12%     -98.0%       1494 ± 47%  sched_debug.cfs_rq[40]:/.exec_clock
     37838 ±  1%     -89.6%       3926 ±  4%  sched_debug.cfs_rq[40]:/.tg->runnable_avg
   5784431 ± 11%     -99.3%      38831 ± 36%  sched_debug.cfs_rq[40]:/.min_vruntime
     76346 ± 13%     -98.1%       1455 ± 41%  sched_debug.cfs_rq[41]:/.exec_clock
2279884.43 ± 47%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[41]:/.MIN_vruntime
   5802087 ± 12%     -99.3%      40195 ± 34%  sched_debug.cfs_rq[41]:/.min_vruntime
         5 ± 29%    -100.0%          0 ±  0%  sched_debug.cfs_rq[41]:/.runnable_load_avg
2279884.76 ± 47%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[41]:/.max_vruntime
     37855 ±  1%     -89.6%       3929 ±  4%  sched_debug.cfs_rq[41]:/.tg->runnable_avg
       134 ± 42%    -100.0%          0 ±  0%  sched_debug.cfs_rq[41]:/.utilization_load_avg
     37858 ±  1%     -89.6%       3932 ±  4%  sched_debug.cfs_rq[42]:/.tg->runnable_avg
   5839210 ± 12%     -99.3%      39152 ± 47%  sched_debug.cfs_rq[42]:/.min_vruntime
     76766 ± 13%     -98.2%       1387 ± 46%  sched_debug.cfs_rq[42]:/.exec_clock
         5 ± 20%    -100.0%          0 ±  0%  sched_debug.cfs_rq[42]:/.runnable_load_avg
     76865 ± 13%     -98.1%       1491 ± 35%  sched_debug.cfs_rq[43]:/.exec_clock
         5 ± 34%    -100.0%          0 ±  0%  sched_debug.cfs_rq[43]:/.runnable_load_avg
         7 ± 44%    -100.0%          0 ±  0%  sched_debug.cfs_rq[43]:/.load
3274498.62 ± 42%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[43]:/.max_vruntime
   5845980 ± 12%     -99.3%      42925 ± 28%  sched_debug.cfs_rq[43]:/.min_vruntime
3274498.62 ± 42%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[43]:/.MIN_vruntime
     37884 ±  1%     -89.6%       3934 ±  4%  sched_debug.cfs_rq[43]:/.tg->runnable_avg
     76461 ± 13%     -97.9%       1582 ± 36%  sched_debug.cfs_rq[44]:/.exec_clock
   5801262 ± 12%     -99.3%      39028 ± 29%  sched_debug.cfs_rq[44]:/.min_vruntime
     37904 ±  1%     -89.6%       3939 ±  4%  sched_debug.cfs_rq[44]:/.tg->runnable_avg
     37919 ±  1%     -89.6%       3945 ±  4%  sched_debug.cfs_rq[45]:/.tg->runnable_avg
     75128 ±  8%     -96.8%       2372 ± 25%  sched_debug.cfs_rq[45]:/.exec_clock
   5733040 ±  8%     -99.2%      47682 ± 38%  sched_debug.cfs_rq[45]:/.min_vruntime
     74823 ±  8%     -97.4%       1957 ± 15%  sched_debug.cfs_rq[46]:/.exec_clock
2181820.38 ± 44%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[46]:/.MIN_vruntime
   5706201 ±  7%     -99.2%      45069 ± 29%  sched_debug.cfs_rq[46]:/.min_vruntime
     37931 ±  1%     -89.6%       3949 ±  4%  sched_debug.cfs_rq[46]:/.tg->runnable_avg
2181820.38 ± 44%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[46]:/.max_vruntime
     37937 ±  1%     -89.6%       3952 ±  4%  sched_debug.cfs_rq[47]:/.tg->runnable_avg
     75941 ±  7%     -97.7%       1740 ± 26%  sched_debug.cfs_rq[47]:/.exec_clock
   5795769 ±  7%     -99.3%      43188 ± 29%  sched_debug.cfs_rq[47]:/.min_vruntime
     74784 ±  7%     -97.9%       1582 ± 27%  sched_debug.cfs_rq[48]:/.exec_clock
   5703468 ±  7%     -99.3%      39942 ± 31%  sched_debug.cfs_rq[48]:/.min_vruntime
     37947 ±  1%     -89.6%       3954 ±  4%  sched_debug.cfs_rq[48]:/.tg->runnable_avg
         5 ± 33%    -100.0%          0 ±  0%  sched_debug.cfs_rq[49]:/.runnable_load_avg
     75045 ±  7%     -97.9%       1592 ± 23%  sched_debug.cfs_rq[49]:/.exec_clock
2349144.49 ± 34%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[49]:/.MIN_vruntime
   5717654 ±  7%     -99.3%      41062 ± 37%  sched_debug.cfs_rq[49]:/.min_vruntime
2349144.49 ± 34%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[49]:/.max_vruntime
     37976 ±  1%     -89.6%       3956 ±  4%  sched_debug.cfs_rq[49]:/.tg->runnable_avg
     26117 ±  3%     -50.7%      12870 ±  7%  sched_debug.cfs_rq[4]:/.avg->runnable_avg_sum
     37364 ±  1%     -89.7%       3843 ±  4%  sched_debug.cfs_rq[4]:/.tg->runnable_avg
     78451 ±  8%     -47.9%      40905 ± 20%  sched_debug.cfs_rq[4]:/.exec_clock
       571 ±  4%     -50.5%        282 ±  7%  sched_debug.cfs_rq[4]:/.tg_runnable_contrib
   5828333 ±  8%     -91.7%     486365 ± 28%  sched_debug.cfs_rq[4]:/.min_vruntime
    945560 ± 28%     -66.2%     319316 ± 41%  sched_debug.cfs_rq[4]:/.spread0
        92 ± 24%    -100.0%          0 ±  0%  sched_debug.cfs_rq[50]:/.utilization_load_avg
     75109 ±  7%     -97.7%       1699 ± 37%  sched_debug.cfs_rq[50]:/.exec_clock
         6 ± 49%    -100.0%          0 ±  0%  sched_debug.cfs_rq[50]:/.load
   5726100 ±  6%     -99.2%      45365 ± 30%  sched_debug.cfs_rq[50]:/.min_vruntime
       543 ± 11%     -98.9%          6 ± 49%  sched_debug.cfs_rq[50]:/.tg_runnable_contrib
     24764 ± 11%     -98.7%        328 ± 39%  sched_debug.cfs_rq[50]:/.avg->runnable_avg_sum
     37997 ±  1%     -89.6%       3959 ±  4%  sched_debug.cfs_rq[50]:/.tg->runnable_avg
         4 ± 34%    -100.0%          0 ±  0%  sched_debug.cfs_rq[50]:/.runnable_load_avg
   5701479 ±  6%     -99.3%      37520 ± 36%  sched_debug.cfs_rq[51]:/.min_vruntime
     38008 ±  1%     -89.6%       3959 ±  4%  sched_debug.cfs_rq[51]:/.tg->runnable_avg
     74668 ±  7%     -98.1%       1411 ± 41%  sched_debug.cfs_rq[51]:/.exec_clock
     25473 ± 12%     -98.4%        408 ± 48%  sched_debug.cfs_rq[52]:/.avg->runnable_avg_sum
     75465 ±  8%     -98.1%       1450 ± 23%  sched_debug.cfs_rq[52]:/.exec_clock
3567331.44 ± 29%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[52]:/.MIN_vruntime
   5767898 ±  7%     -99.3%      38845 ± 30%  sched_debug.cfs_rq[52]:/.min_vruntime
         8 ± 41%    -100.0%          0 ±  0%  sched_debug.cfs_rq[52]:/.runnable_load_avg
3567331.77 ± 29%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[52]:/.max_vruntime
     38019 ±  1%     -89.6%       3962 ±  4%  sched_debug.cfs_rq[52]:/.tg->runnable_avg
     38036 ±  2%     -89.6%       3966 ±  4%  sched_debug.cfs_rq[53]:/.tg->runnable_avg
     75493 ±  8%     -98.0%       1520 ± 30%  sched_debug.cfs_rq[53]:/.exec_clock
2564844.76 ± 34%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[53]:/.max_vruntime
   5768160 ±  7%     -99.2%      44245 ± 35%  sched_debug.cfs_rq[53]:/.min_vruntime
2564844.76 ± 34%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[53]:/.MIN_vruntime
     75523 ±  9%     -98.1%       1443 ± 40%  sched_debug.cfs_rq[54]:/.exec_clock
     38055 ±  2%     -89.6%       3969 ±  4%  sched_debug.cfs_rq[54]:/.tg->runnable_avg
   5757595 ±  8%     -99.3%      41312 ± 28%  sched_debug.cfs_rq[54]:/.min_vruntime
   5731615 ±  7%     -99.3%      38668 ± 27%  sched_debug.cfs_rq[55]:/.min_vruntime
     75160 ±  8%     -98.0%       1473 ± 39%  sched_debug.cfs_rq[55]:/.exec_clock
     38085 ±  2%     -89.6%       3971 ±  4%  sched_debug.cfs_rq[55]:/.tg->runnable_avg
       552 ± 13%     -99.0%          5 ± 49%  sched_debug.cfs_rq[56]:/.tg_runnable_contrib
   5759032 ±  7%     -99.3%      37467 ± 32%  sched_debug.cfs_rq[56]:/.min_vruntime
     38101 ±  2%     -89.6%       3980 ±  4%  sched_debug.cfs_rq[56]:/.tg->runnable_avg
     25129 ± 13%     -98.8%        291 ± 44%  sched_debug.cfs_rq[56]:/.avg->runnable_avg_sum
     75483 ±  8%     -98.1%       1421 ± 48%  sched_debug.cfs_rq[56]:/.exec_clock
         5 ± 29%    -100.0%          0 ±  0%  sched_debug.cfs_rq[57]:/.runnable_load_avg
     74873 ±  8%     -97.9%       1582 ± 49%  sched_debug.cfs_rq[57]:/.exec_clock
   5707036 ±  7%     -99.2%      43462 ± 45%  sched_debug.cfs_rq[57]:/.min_vruntime
     38113 ±  2%     -89.6%       3981 ±  4%  sched_debug.cfs_rq[57]:/.tg->runnable_avg
2056049.76 ± 47%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[58]:/.MIN_vruntime
2056049.76 ± 47%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[58]:/.max_vruntime
   5638172 ±  8%     -99.3%      40579 ± 41%  sched_debug.cfs_rq[58]:/.min_vruntime
     38110 ±  1%     -89.6%       3979 ±  4%  sched_debug.cfs_rq[58]:/.tg->runnable_avg
     74655 ±  8%     -98.3%       1291 ± 34%  sched_debug.cfs_rq[59]:/.exec_clock
         9 ± 48%    -100.0%          0 ±  0%  sched_debug.cfs_rq[59]:/.load
     38125 ±  1%     -89.6%       3980 ±  4%  sched_debug.cfs_rq[59]:/.tg->runnable_avg
   5687789 ±  7%     -99.3%      37622 ± 41%  sched_debug.cfs_rq[59]:/.min_vruntime
       569 ±  5%     -89.4%         60 ± 18%  sched_debug.cfs_rq[5]:/.tg_runnable_contrib
     78491 ±  9%     -87.5%       9836 ±  6%  sched_debug.cfs_rq[5]:/.exec_clock
     37386 ±  1%     -89.7%       3845 ±  4%  sched_debug.cfs_rq[5]:/.tg->runnable_avg
     26026 ±  6%     -89.3%       2782 ± 17%  sched_debug.cfs_rq[5]:/.avg->runnable_avg_sum
   5868941 ±  8%     -97.8%     132022 ± 13%  sched_debug.cfs_rq[5]:/.min_vruntime
     50960 ±  9%     -99.1%        438 ± 44%  sched_debug.cfs_rq[60]:/.exec_clock
     38133 ±  1%     -89.6%       3981 ±  4%  sched_debug.cfs_rq[60]:/.tg->runnable_avg
     51720 ± 14%     -97.9%       1072 ± 40%  sched_debug.cfs_rq[61]:/.exec_clock
   4337125 ± 13%     -99.2%      35916 ± 38%  sched_debug.cfs_rq[61]:/.min_vruntime
     38160 ±  1%     -89.6%       3984 ±  4%  sched_debug.cfs_rq[61]:/.tg->runnable_avg
         3 ± 39%    -100.0%          0 ±  0%  sched_debug.cfs_rq[62]:/.load
     52202 ± 13%     -98.8%        626 ± 34%  sched_debug.cfs_rq[62]:/.exec_clock
     38148 ±  1%     -89.5%       3986 ±  4%  sched_debug.cfs_rq[62]:/.tg->runnable_avg
     14412 ± 18%     -98.6%        194 ± 48%  sched_debug.cfs_rq[62]:/.avg->runnable_avg_sum
         2 ± 40%    -100.0%          0 ±  0%  sched_debug.cfs_rq[62]:/.runnable_load_avg
     38154 ±  2%     -89.6%       3986 ±  4%  sched_debug.cfs_rq[63]:/.tg->runnable_avg
   1268924 ± 47%     -97.9%      26358 ± 34%  sched_debug.cfs_rq[64]:/.min_vruntime
        64 ± 42%     -95.8%          2 ± 41%  sched_debug.cfs_rq[64]:/.tg_runnable_contrib
     38173 ±  2%     -89.6%       3988 ±  4%  sched_debug.cfs_rq[64]:/.tg->runnable_avg
      2993 ± 41%     -95.1%        148 ± 36%  sched_debug.cfs_rq[64]:/.avg->runnable_avg_sum
     38152 ±  2%     -89.6%       3985 ±  4%  sched_debug.cfs_rq[65]:/.tg->runnable_avg
         7 ± 47%    -100.0%          0 ±  0%  sched_debug.cfs_rq[65]:/.nr_spread_over
     38162 ±  2%     -89.5%       3989 ±  3%  sched_debug.cfs_rq[66]:/.tg->runnable_avg
        10 ± 30%    -100.0%          0 ±  0%  sched_debug.cfs_rq[66]:/.nr_spread_over
     15875 ± 49%     -95.7%        680 ± 31%  sched_debug.cfs_rq[66]:/.exec_clock
     38175 ±  2%     -89.5%       3990 ±  3%  sched_debug.cfs_rq[67]:/.tg->runnable_avg
     38206 ±  2%     -89.6%       3991 ±  3%  sched_debug.cfs_rq[68]:/.tg->runnable_avg
     38221 ±  2%     -89.6%       3992 ±  3%  sched_debug.cfs_rq[69]:/.tg->runnable_avg
     15369 ± 47%     -94.7%        812 ± 43%  sched_debug.cfs_rq[69]:/.exec_clock
     25441 ±  4%     -91.8%       2076 ± 13%  sched_debug.cfs_rq[6]:/.avg->runnable_avg_sum
       557 ±  4%     -92.0%         44 ± 13%  sched_debug.cfs_rq[6]:/.tg_runnable_contrib
         6 ± 45%    -100.0%          0 ±  0%  sched_debug.cfs_rq[6]:/.load
   5815571 ±  7%     -98.1%     110120 ± 10%  sched_debug.cfs_rq[6]:/.min_vruntime
     77709 ±  8%     -89.8%       7891 ±  9%  sched_debug.cfs_rq[6]:/.exec_clock
         5 ± 33%    -100.0%          0 ±  0%  sched_debug.cfs_rq[6]:/.runnable_load_avg
     37407 ±  1%     -89.7%       3849 ±  4%  sched_debug.cfs_rq[6]:/.tg->runnable_avg
     38225 ±  2%     -89.5%       4001 ±  4%  sched_debug.cfs_rq[70]:/.tg->runnable_avg
     38235 ±  2%     -89.5%       4005 ±  4%  sched_debug.cfs_rq[71]:/.tg->runnable_avg
     38238 ±  2%     -89.5%       4007 ±  4%  sched_debug.cfs_rq[72]:/.tg->runnable_avg
     38253 ±  2%     -89.5%       4007 ±  4%  sched_debug.cfs_rq[73]:/.tg->runnable_avg
         9 ± 27%    -100.0%          0 ±  0%  sched_debug.cfs_rq[73]:/.nr_spread_over
         7 ± 45%    -100.0%          0 ±  0%  sched_debug.cfs_rq[74]:/.nr_spread_over
     38240 ±  2%     -89.5%       4012 ±  4%  sched_debug.cfs_rq[74]:/.tg->runnable_avg
     38235 ±  2%     -89.5%       4010 ±  4%  sched_debug.cfs_rq[75]:/.tg->runnable_avg
     38233 ±  2%     -89.5%       4010 ±  4%  sched_debug.cfs_rq[76]:/.tg->runnable_avg
     38239 ±  2%     -89.5%       4012 ±  4%  sched_debug.cfs_rq[77]:/.tg->runnable_avg
     38260 ±  2%     -89.5%       4015 ±  4%  sched_debug.cfs_rq[78]:/.tg->runnable_avg
     38284 ±  2%     -89.5%       4019 ±  3%  sched_debug.cfs_rq[79]:/.tg->runnable_avg
     25894 ±  3%     -92.4%       1956 ± 12%  sched_debug.cfs_rq[7]:/.avg->runnable_avg_sum
     78030 ±  8%     -90.8%       7193 ±  8%  sched_debug.cfs_rq[7]:/.exec_clock
   5863768 ±  7%     -98.2%     105821 ± 12%  sched_debug.cfs_rq[7]:/.min_vruntime
       566 ±  3%     -92.6%         42 ± 12%  sched_debug.cfs_rq[7]:/.tg_runnable_contrib
     37430 ±  1%     -89.7%       3850 ±  4%  sched_debug.cfs_rq[7]:/.tg->runnable_avg
     38292 ±  2%     -89.5%       4021 ±  3%  sched_debug.cfs_rq[80]:/.tg->runnable_avg
     38300 ±  2%     -89.5%       4022 ±  3%  sched_debug.cfs_rq[81]:/.tg->runnable_avg
     38303 ±  2%     -89.5%       4025 ±  3%  sched_debug.cfs_rq[82]:/.tg->runnable_avg
     38286 ±  2%     -89.5%       4028 ±  3%  sched_debug.cfs_rq[83]:/.tg->runnable_avg
     38294 ±  2%     -89.5%       4032 ±  3%  sched_debug.cfs_rq[84]:/.tg->runnable_avg
     38304 ±  2%     -89.5%       4034 ±  3%  sched_debug.cfs_rq[85]:/.tg->runnable_avg
     38321 ±  2%     -89.5%       4035 ±  3%  sched_debug.cfs_rq[86]:/.tg->runnable_avg
     38326 ±  2%     -89.5%       4039 ±  3%  sched_debug.cfs_rq[87]:/.tg->runnable_avg
     38322 ±  2%     -89.5%       4040 ±  3%  sched_debug.cfs_rq[88]:/.tg->runnable_avg
     38313 ±  2%     -89.4%       4044 ±  3%  sched_debug.cfs_rq[89]:/.tg->runnable_avg
     37445 ±  1%     -89.7%       3853 ±  4%  sched_debug.cfs_rq[8]:/.tg->runnable_avg
       553 ±  4%     -92.0%         44 ±  5%  sched_debug.cfs_rq[8]:/.tg_runnable_contrib
     25270 ±  4%     -91.8%       2076 ±  5%  sched_debug.cfs_rq[8]:/.avg->runnable_avg_sum
     77680 ±  8%     -91.2%       6799 ±  6%  sched_debug.cfs_rq[8]:/.exec_clock
   5833584 ±  8%     -98.3%      99293 ± 11%  sched_debug.cfs_rq[8]:/.min_vruntime
     38307 ±  2%     -89.4%       4044 ±  3%  sched_debug.cfs_rq[90]:/.tg->runnable_avg
     38316 ±  2%     -89.4%       4047 ±  3%  sched_debug.cfs_rq[91]:/.tg->runnable_avg
      4788 ± 48%     -95.1%        235 ± 35%  sched_debug.cfs_rq[91]:/.avg->runnable_avg_sum
       103 ± 49%     -95.8%          4 ± 39%  sched_debug.cfs_rq[91]:/.tg_runnable_contrib
     38320 ±  2%     -89.4%       4051 ±  3%  sched_debug.cfs_rq[92]:/.tg->runnable_avg
     38332 ±  2%     -89.4%       4052 ±  3%  sched_debug.cfs_rq[93]:/.tg->runnable_avg
     38341 ±  2%     -89.4%       4053 ±  3%  sched_debug.cfs_rq[94]:/.tg->runnable_avg
     38329 ±  2%     -89.4%       4056 ±  3%  sched_debug.cfs_rq[95]:/.tg->runnable_avg
     38315 ±  2%     -89.4%       4060 ±  3%  sched_debug.cfs_rq[96]:/.tg->runnable_avg
     38318 ±  2%     -89.4%       4061 ±  3%  sched_debug.cfs_rq[97]:/.tg->runnable_avg
     38321 ±  2%     -89.4%       4064 ±  3%  sched_debug.cfs_rq[98]:/.tg->runnable_avg
     38326 ±  2%     -89.4%       4066 ±  3%  sched_debug.cfs_rq[99]:/.tg->runnable_avg
2599395.82 ± 28%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[9]:/.MIN_vruntime
     37435 ±  1%     -89.7%       3849 ±  4%  sched_debug.cfs_rq[9]:/.tg->runnable_avg
         7 ± 45%    -100.0%          0 ±  0%  sched_debug.cfs_rq[9]:/.runnable_load_avg
     25086 ±  4%     -93.1%       1741 ± 14%  sched_debug.cfs_rq[9]:/.avg->runnable_avg_sum
     76760 ±  8%     -91.3%       6708 ± 10%  sched_debug.cfs_rq[9]:/.exec_clock
2599396.15 ± 28%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[9]:/.max_vruntime
       550 ±  4%     -93.2%         37 ± 15%  sched_debug.cfs_rq[9]:/.tg_runnable_contrib
   5775897 ±  7%     -98.3%      99243 ± 18%  sched_debug.cfs_rq[9]:/.min_vruntime
         7 ± 39%  +11559.1%        855 ±  0%  sched_debug.cpu#0.cpu_load[3]
         8 ± 35%   +9767.3%        855 ±  0%  sched_debug.cpu#0.cpu_load[2]
        10 ± 25%   +8443.3%        854 ±  0%  sched_debug.cpu#0.cpu_load[1]
         6 ± 39%  +14133.3%        854 ±  0%  sched_debug.cpu#0.cpu_load[4]
    866410 ± 20%     -42.2%     500354 ±  0%  sched_debug.cpu#0.max_idle_balance_cost
    401816 ±  6%   +5302.7%   21709084 ±  9%  sched_debug.cpu#0.ttwu_count
    312394 ± 14%   +1631.2%    5408220 ± 18%  sched_debug.cpu#1.sched_goidle
    177603 ± 24%     -87.5%      22288 ± 17%  sched_debug.cpu#1.ttwu_local
    303227 ±  7%     -46.2%     163110 ± 12%  sched_debug.cpu#1.avg_idle
    695685 ± 12%   +1457.9%   10837841 ± 18%  sched_debug.cpu#1.nr_switches
    699186 ± 12%   +1450.1%   10837966 ± 18%  sched_debug.cpu#1.sched_count
    461875 ±  8%     -94.9%      23602 ± 15%  sched_debug.cpu#1.ttwu_count
    262569 ±  5%    +240.9%     895031 ± 14%  sched_debug.cpu#10.avg_idle
    145636 ±  7%     -30.0%     101980 ± 10%  sched_debug.cpu#10.nr_load_updates
         4 ± 36%    -100.0%          0 ±  0%  sched_debug.cpu#10.cpu_load[1]
         3 ± 42%    -100.0%          0 ±  0%  sched_debug.cpu#10.cpu_load[3]
         3 ± 34%    -100.0%          0 ±  0%  sched_debug.cpu#10.cpu_load[2]
    534226 ± 10%     -99.7%       1612 ± 29%  sched_debug.cpu#10.ttwu_count
    182313 ± 10%     -99.3%       1351 ± 29%  sched_debug.cpu#10.ttwu_local
    912692 ± 29%     -45.2%     500000 ±  0%  sched_debug.cpu#100.max_idle_balance_cost
    890003 ± 25%     -43.5%     502419 ±  0%  sched_debug.cpu#101.max_idle_balance_cost
    895162 ± 26%     -42.0%     518876 ±  8%  sched_debug.cpu#102.max_idle_balance_cost
   1279055 ± 16%     -18.8%    1038141 ±  8%  sched_debug.cpu#102.avg_idle
   1210646 ± 23%     -19.2%     978478 ±  1%  sched_debug.cpu#103.avg_idle
    873261 ± 25%     -42.7%     500000 ±  0%  sched_debug.cpu#103.max_idle_balance_cost
    864952 ± 24%     -42.2%     500000 ±  0%  sched_debug.cpu#104.max_idle_balance_cost
    887413 ± 31%     -43.7%     500000 ±  0%  sched_debug.cpu#105.max_idle_balance_cost
    854103 ± 27%     -40.9%     504735 ±  2%  sched_debug.cpu#106.max_idle_balance_cost
    864081 ± 28%     -42.0%     501124 ±  0%  sched_debug.cpu#107.max_idle_balance_cost
    898657 ± 28%     -42.5%     516711 ±  6%  sched_debug.cpu#108.max_idle_balance_cost
         3 ± 42%    -100.0%          0 ±  0%  sched_debug.cpu#11.cpu_load[2]
    543726 ±  4%     -99.7%       1760 ± 32%  sched_debug.cpu#11.ttwu_count
    145709 ±  7%     -30.3%     101564 ±  9%  sched_debug.cpu#11.nr_load_updates
    270996 ±  1%    +229.9%     894099 ± 14%  sched_debug.cpu#11.avg_idle
         2 ± 37%    -100.0%          0 ±  0%  sched_debug.cpu#11.cpu_load[3]
    185681 ± 10%     -99.2%       1455 ± 30%  sched_debug.cpu#11.ttwu_local
    873654 ± 27%     -42.5%     502080 ±  0%  sched_debug.cpu#110.max_idle_balance_cost
    858843 ± 30%     -41.1%     505597 ±  1%  sched_debug.cpu#111.max_idle_balance_cost
    872547 ± 28%     -42.7%     500133 ±  0%  sched_debug.cpu#112.max_idle_balance_cost
    850260 ± 28%     -40.6%     505082 ±  2%  sched_debug.cpu#113.max_idle_balance_cost
    960366 ± 23%     -47.8%     500984 ±  0%  sched_debug.cpu#114.max_idle_balance_cost
    869532 ± 27%     -42.2%     502156 ±  0%  sched_debug.cpu#115.max_idle_balance_cost
    884039 ± 27%     -42.9%     505044 ±  1%  sched_debug.cpu#116.max_idle_balance_cost
    855531 ± 30%     -41.2%     502901 ±  1%  sched_debug.cpu#117.max_idle_balance_cost
    844778 ± 29%     -40.2%     505544 ±  1%  sched_debug.cpu#118.max_idle_balance_cost
    867345 ± 27%     -41.1%     510969 ±  4%  sched_debug.cpu#119.max_idle_balance_cost
    145505 ±  8%     -34.3%      95650 ± 15%  sched_debug.cpu#12.nr_load_updates
         5 ± 44%    -100.0%          0 ±  0%  sched_debug.cpu#12.cpu_load[1]
    186685 ± 11%     -99.3%       1223 ± 33%  sched_debug.cpu#12.ttwu_local
         4 ± 49%    -100.0%          0 ±  0%  sched_debug.cpu#12.cpu_load[2]
    280891 ±  3%    +222.5%     905991 ± 14%  sched_debug.cpu#12.avg_idle
         3 ± 39%    -100.0%          0 ±  0%  sched_debug.cpu#12.cpu_load[3]
    187388 ± 11%     -99.3%       1223 ± 30%  sched_debug.cpu#13.ttwu_local
    145475 ±  8%     -31.9%      99013 ± 11%  sched_debug.cpu#13.nr_load_updates
    546320 ±  8%     -99.7%       1508 ± 21%  sched_debug.cpu#13.ttwu_count
         3 ± 27%    -100.0%          0 ±  0%  sched_debug.cpu#13.cpu_load[1]
    282818 ±  5%    +212.5%     883684 ± 13%  sched_debug.cpu#13.avg_idle
    145427 ±  7%     -34.5%      95288 ± 17%  sched_debug.cpu#14.nr_load_updates
         4 ± 41%    -100.0%          0 ±  0%  sched_debug.cpu#14.cpu_load[3]
    283614 ±  6%    +204.3%     862952 ± 17%  sched_debug.cpu#14.avg_idle
    548763 ± 10%     -99.7%       1508 ± 26%  sched_debug.cpu#14.ttwu_count
    187787 ± 10%     -99.3%       1258 ± 24%  sched_debug.cpu#14.ttwu_local
         4 ± 46%    -100.0%          0 ±  0%  sched_debug.cpu#14.cpu_load[2]
    910931 ± 10%     -90.9%      83254 ± 28%  sched_debug.cpu#15.sched_count
    401751 ± 10%     -89.7%      41393 ± 28%  sched_debug.cpu#15.sched_goidle
    532305 ±  9%     -99.3%       3824 ± 15%  sched_debug.cpu#15.ttwu_count
    227026 ± 15%     -98.7%       2840 ± 12%  sched_debug.cpu#15.ttwu_local
         2 ± 46%    -100.0%          0 ±  0%  sched_debug.cpu#15.cpu_load[2]
    907583 ± 10%     -90.8%      83244 ± 28%  sched_debug.cpu#15.nr_switches
    147307 ±  8%     -64.1%      52925 ± 18%  sched_debug.cpu#15.nr_load_updates
    271896 ± 28%    +268.2%    1001194 ±  0%  sched_debug.cpu#15.avg_idle
    228547 ± 14%     -99.6%       1006 ± 46%  sched_debug.cpu#16.ttwu_local
    146688 ±  7%     -64.8%      51642 ± 18%  sched_debug.cpu#16.nr_load_updates
    538118 ± 10%     -99.7%       1836 ± 36%  sched_debug.cpu#16.ttwu_count
    401589 ±  9%     -89.9%      40532 ± 27%  sched_debug.cpu#16.sched_goidle
    912282 ± 10%     -91.1%      81551 ± 27%  sched_debug.cpu#16.sched_count
    274129 ± 10%    +258.6%     983056 ±  2%  sched_debug.cpu#16.avg_idle
    909937 ± 10%     -91.0%      81542 ± 27%  sched_debug.cpu#16.nr_switches
    399818 ± 10%     -90.8%      36949 ± 41%  sched_debug.cpu#17.sched_goidle
    251250 ± 33%    +290.1%     980211 ±  2%  sched_debug.cpu#17.avg_idle
    146193 ±  8%     -67.5%      47493 ± 30%  sched_debug.cpu#17.nr_load_updates
    907652 ± 10%     -91.8%      74306 ± 41%  sched_debug.cpu#17.sched_count
    905857 ± 10%     -91.8%      74297 ± 41%  sched_debug.cpu#17.nr_switches
    535902 ±  9%     -99.7%       1645 ± 28%  sched_debug.cpu#17.ttwu_count
    535263 ±  9%     -99.6%       2125 ± 33%  sched_debug.cpu#18.ttwu_count
    904084 ± 11%     -93.0%      62907 ± 44%  sched_debug.cpu#18.nr_switches
    274939 ±  9%    +266.2%    1006769 ±  2%  sched_debug.cpu#18.avg_idle
    226681 ± 14%     -99.5%       1123 ± 40%  sched_debug.cpu#18.ttwu_local
    905555 ± 11%     -93.1%      62916 ± 43%  sched_debug.cpu#18.sched_count
    146165 ±  7%     -71.0%      42321 ± 30%  sched_debug.cpu#18.nr_load_updates
    399028 ± 10%     -92.2%      31022 ± 44%  sched_debug.cpu#18.sched_goidle
    146308 ±  8%     -71.5%      41686 ± 35%  sched_debug.cpu#19.nr_load_updates
    529715 ±  9%     -99.6%       2212 ± 43%  sched_debug.cpu#19.ttwu_count
    296225 ± 10%    +232.9%     986018 ±  1%  sched_debug.cpu#19.avg_idle
         3 ± 38%    +605.3%         22 ± 29%  sched_debug.cpu#2.cpu_load[2]
    683586 ±  9%   +1403.7%   10278920 ± 19%  sched_debug.cpu#2.sched_count
         3 ± 41%    +460.9%         21 ± 46%  sched_debug.cpu#2.cpu_load[1]
    420236 ±  7%     -95.2%      20376 ± 17%  sched_debug.cpu#2.ttwu_count
    154717 ± 17%     -87.2%      19810 ± 18%  sched_debug.cpu#2.ttwu_local
    680385 ±  9%   +1410.7%   10278585 ± 19%  sched_debug.cpu#2.nr_switches
    300978 ± 12%     -50.5%     149090 ± 40%  sched_debug.cpu#2.avg_idle
    300267 ± 10%   +1608.3%    5129477 ± 19%  sched_debug.cpu#2.sched_goidle
    226744 ± 15%     -99.7%        791 ± 12%  sched_debug.cpu#20.ttwu_local
         2 ± 41%    -100.0%          0 ±  0%  sched_debug.cpu#20.cpu_load[3]
    539626 ± 11%     -99.7%       1714 ± 13%  sched_debug.cpu#20.ttwu_count
         3 ± 48%    -100.0%          0 ±  0%  sched_debug.cpu#20.cpu_load[1]
    264020 ± 16%    +277.1%     995578 ±  0%  sched_debug.cpu#20.avg_idle
    145629 ±  7%     -72.8%      39540 ± 38%  sched_debug.cpu#20.nr_load_updates
    525878 ± 11%     -99.6%       1905 ± 16%  sched_debug.cpu#21.ttwu_count
    145597 ±  8%     -72.4%      40198 ± 40%  sched_debug.cpu#21.nr_load_updates
         6 ± 40%    -100.0%          0 ±  0%  sched_debug.cpu#21.load
    226311 ± 17%     -99.6%        954 ± 17%  sched_debug.cpu#21.ttwu_local
         2 ± 47%    -100.0%          0 ±  0%  sched_debug.cpu#21.cpu_load[3]
    282162 ± 10%    +268.8%    1040716 ±  4%  sched_debug.cpu#21.avg_idle
         2 ± 40%    -100.0%          0 ±  0%  sched_debug.cpu#21.cpu_load[2]
         4 ± 40%    -100.0%          0 ±  0%  sched_debug.cpu#22.cpu_load[1]
         3 ± 34%    -100.0%          0 ±  0%  sched_debug.cpu#22.cpu_load[2]
    145443 ±  7%     -73.8%      38078 ± 39%  sched_debug.cpu#22.nr_load_updates
         2 ± 47%    -100.0%          0 ±  0%  sched_debug.cpu#22.cpu_load[3]
    547461 ± 10%     -99.6%       2080 ± 41%  sched_debug.cpu#22.ttwu_count
    293095 ±  9%    +242.6%    1004209 ±  2%  sched_debug.cpu#22.avg_idle
    538395 ± 10%     -99.7%       1761 ± 30%  sched_debug.cpu#23.ttwu_count
    145290 ±  8%     -77.3%      33007 ± 45%  sched_debug.cpu#23.nr_load_updates
    312631 ± 11%    +217.5%     992648 ±  1%  sched_debug.cpu#23.avg_idle
         5 ± 42%    -100.0%          0 ±  0%  sched_debug.cpu#23.load
    274102 ±  9%    +261.8%     991658 ±  1%  sched_debug.cpu#24.avg_idle
    145062 ±  8%     -75.6%      35376 ± 39%  sched_debug.cpu#24.nr_load_updates
    305640 ±  8%    +215.8%     965318 ±  7%  sched_debug.cpu#25.avg_idle
    554385 ±  4%     -99.6%       2368 ± 47%  sched_debug.cpu#25.ttwu_count
    146404 ±  6%     -77.2%      33321 ± 47%  sched_debug.cpu#25.nr_load_updates
    562613 ± 11%     -99.6%       2233 ± 32%  sched_debug.cpu#26.ttwu_count
         2 ± 47%    -100.0%          0 ±  0%  sched_debug.cpu#26.cpu_load[2]
    291842 ±  8%    +239.5%     990850 ±  4%  sched_debug.cpu#26.avg_idle
         3 ± 37%    -100.0%          0 ±  0%  sched_debug.cpu#27.cpu_load[2]
         4 ± 44%    -100.0%          0 ±  0%  sched_debug.cpu#27.cpu_load[1]
    296953 ±  8%    +234.6%     993660 ±  3%  sched_debug.cpu#27.avg_idle
    227387 ± 14%     -99.6%        875 ± 46%  sched_debug.cpu#28.ttwu_local
    541565 ±  9%     -99.7%       1729 ± 35%  sched_debug.cpu#28.ttwu_count
    287000 ±  9%    +247.4%     996956 ±  1%  sched_debug.cpu#28.avg_idle
    292577 ±  5%    +240.4%     995812 ±  9%  sched_debug.cpu#29.avg_idle
         2 ± 49%    -100.0%          0 ±  0%  sched_debug.cpu#29.cpu_load[0]
         2 ± 31%    -100.0%          0 ±  0%  sched_debug.cpu#29.cpu_load[1]
         3 ± 27%    -100.0%          0 ±  0%  sched_debug.cpu#29.cpu_load[2]
    538059 ± 11%     -99.6%       1942 ± 46%  sched_debug.cpu#29.ttwu_count
         2 ± 40%    -100.0%          0 ±  0%  sched_debug.cpu#29.cpu_load[3]
    858348 ± 11%    +988.5%    9343383 ± 19%  sched_debug.cpu#3.nr_switches
    583644 ± 14%     -96.9%      18374 ± 19%  sched_debug.cpu#3.ttwu_count
    188558 ± 11%     -90.6%      17658 ± 20%  sched_debug.cpu#3.ttwu_local
    861080 ± 11%    +985.1%    9343434 ± 19%  sched_debug.cpu#3.sched_count
         3 ± 28%    +325.0%         14 ± 46%  sched_debug.cpu#3.cpu_load[2]
    283363 ±  7%     -36.9%     178917 ±  5%  sched_debug.cpu#3.avg_idle
         3 ± 27%    +366.7%         14 ± 35%  sched_debug.cpu#3.cpu_load[3]
    365993 ± 11%   +1174.1%    4662957 ± 19%  sched_debug.cpu#3.sched_goidle
    272296 ±  5%    +263.0%     988390 ±  3%  sched_debug.cpu#30.avg_idle
    886491 ±  7%     -90.7%      82339 ± 23%  sched_debug.cpu#30.nr_switches
    193322 ±  6%     -99.6%        752 ± 14%  sched_debug.cpu#30.ttwu_local
    890539 ±  7%     -90.7%      82392 ± 23%  sched_debug.cpu#30.sched_count
    145061 ±  7%     -66.4%      48735 ± 20%  sched_debug.cpu#30.nr_load_updates
    385340 ±  7%     -89.4%      40937 ± 23%  sched_debug.cpu#30.sched_goidle
    558855 ± 11%     -99.7%       1486 ± 14%  sched_debug.cpu#30.ttwu_count
    607204 ± 10%     -99.7%       1669 ± 28%  sched_debug.cpu#31.ttwu_count
    948582 ± 13%     -92.4%      72412 ± 23%  sched_debug.cpu#31.sched_count
    147499 ±  6%     -70.2%      44027 ± 20%  sched_debug.cpu#31.nr_load_updates
    231115 ± 34%     -99.7%        707 ± 17%  sched_debug.cpu#31.ttwu_local
    414152 ± 16%     -91.3%      35947 ± 23%  sched_debug.cpu#31.sched_goidle
    945517 ± 13%     -92.3%      72360 ± 23%  sched_debug.cpu#31.nr_switches
    269881 ± 11%    +260.8%     973666 ±  4%  sched_debug.cpu#31.avg_idle
    146258 ±  6%     -72.3%      40536 ± 31%  sched_debug.cpu#32.nr_load_updates
    212916 ± 19%     -99.7%        682 ± 15%  sched_debug.cpu#32.ttwu_local
    616801 ± 13%     -99.8%       1439 ± 11%  sched_debug.cpu#32.ttwu_count
    886655 ± 10%     -92.6%      65901 ± 36%  sched_debug.cpu#32.sched_count
    884019 ± 10%     -92.6%      65851 ± 36%  sched_debug.cpu#32.nr_switches
    287282 ±  4%    +221.9%     924747 ±  3%  sched_debug.cpu#32.avg_idle
    386416 ± 11%     -91.5%      32707 ± 36%  sched_debug.cpu#32.sched_goidle
    381805 ±  6%     -92.2%      29619 ± 35%  sched_debug.cpu#33.sched_goidle
    882037 ±  6%     -93.2%      59641 ± 35%  sched_debug.cpu#33.sched_count
    264381 ±  4%    +251.9%     930329 ±  7%  sched_debug.cpu#33.avg_idle
    580711 ± 13%     -99.8%       1425 ±  8%  sched_debug.cpu#33.ttwu_count
         2 ± 37%    -100.0%          0 ±  0%  sched_debug.cpu#33.cpu_load[2]
    198746 ±  7%     -99.7%        677 ± 13%  sched_debug.cpu#33.ttwu_local
    879687 ±  6%     -93.2%      59612 ± 35%  sched_debug.cpu#33.nr_switches
    145237 ±  7%     -73.9%      37912 ± 30%  sched_debug.cpu#33.nr_load_updates
    558432 ± 12%     -99.7%       1937 ± 40%  sched_debug.cpu#34.ttwu_count
    271737 ±  6%    +267.3%     998140 ±  8%  sched_debug.cpu#34.avg_idle
    144986 ±  7%     -74.4%      37118 ± 42%  sched_debug.cpu#34.nr_load_updates
    192853 ±  6%     -99.6%        781 ± 15%  sched_debug.cpu#34.ttwu_local
    255816 ± 24%    +287.7%     991833 ±  1%  sched_debug.cpu#35.avg_idle
    144906 ±  7%     -74.4%      37168 ± 35%  sched_debug.cpu#35.nr_load_updates
    554138 ± 12%     -99.7%       1813 ±  9%  sched_debug.cpu#35.ttwu_count
    379077 ±  6%     -92.2%      29594 ± 42%  sched_debug.cpu#35.sched_goidle
    879048 ±  7%     -93.2%      59608 ± 42%  sched_debug.cpu#35.sched_count
    876729 ±  7%     -93.2%      59559 ± 42%  sched_debug.cpu#35.nr_switches
    192396 ±  6%     -99.6%        796 ± 19%  sched_debug.cpu#35.ttwu_local
    144811 ±  7%     -76.8%      33645 ± 44%  sched_debug.cpu#36.nr_load_updates
    198260 ±  3%     -99.7%        681 ± 15%  sched_debug.cpu#36.ttwu_local
         3 ± 38%    -100.0%          0 ±  0%  sched_debug.cpu#36.cpu_load[2]
    608120 ± 19%     -99.7%       1607 ± 10%  sched_debug.cpu#36.ttwu_count
    272324 ± 10%    +271.7%    1012309 ±  4%  sched_debug.cpu#36.avg_idle
         4 ± 47%    -100.0%          0 ±  0%  sched_debug.cpu#36.cpu_load[0]
         3 ± 48%    -100.0%          0 ±  0%  sched_debug.cpu#36.cpu_load[1]
    144880 ±  8%     -75.5%      35491 ± 42%  sched_debug.cpu#37.nr_load_updates
    551377 ± 10%     -99.7%       1487 ± 10%  sched_debug.cpu#37.ttwu_count
    193906 ±  7%     -99.6%        680 ± 18%  sched_debug.cpu#37.ttwu_local
    277677 ±  3%    +256.2%     989065 ±  1%  sched_debug.cpu#37.avg_idle
         4 ± 49%    -100.0%          0 ±  0%  sched_debug.cpu#38.cpu_load[1]
    271911 ±  6%    +259.7%     978143 ±  1%  sched_debug.cpu#38.avg_idle
         4 ± 37%    -100.0%          0 ±  0%  sched_debug.cpu#38.cpu_load[2]
    588979 ± 20%     -99.7%       1876 ± 49%  sched_debug.cpu#38.ttwu_count
    195832 ±  6%     -99.7%        662 ± 13%  sched_debug.cpu#38.ttwu_local
         3 ± 42%    -100.0%          0 ±  0%  sched_debug.cpu#38.cpu_load[3]
    859499 ±  7%     -94.2%      49501 ± 48%  sched_debug.cpu#39.nr_switches
    197080 ±  5%     -99.7%        616 ± 14%  sched_debug.cpu#39.ttwu_local
    144875 ±  8%     -78.5%      31180 ± 38%  sched_debug.cpu#39.nr_load_updates
    861583 ±  7%     -94.2%      50175 ± 46%  sched_debug.cpu#39.sched_count
    372809 ±  7%     -93.4%      24588 ± 48%  sched_debug.cpu#39.sched_goidle
    585682 ± 15%     -99.8%       1422 ± 11%  sched_debug.cpu#39.ttwu_count
    254598 ± 28%    +291.0%     995522 ±  0%  sched_debug.cpu#39.avg_idle
    384496 ±  7%    +624.1%    2784133 ± 20%  sched_debug.cpu#4.sched_goidle
    903009 ±  6%    +517.8%    5579241 ± 20%  sched_debug.cpu#4.sched_count
    900237 ±  6%    +519.7%    5579190 ± 20%  sched_debug.cpu#4.nr_switches
    556589 ±  9%     -97.9%      11720 ± 21%  sched_debug.cpu#4.ttwu_count
    193650 ± 19%     -94.2%      11155 ± 20%  sched_debug.cpu#4.ttwu_local
    279229 ±  6%    +261.3%    1008863 ±  1%  sched_debug.cpu#40.avg_idle
    564514 ± 13%     -99.8%       1408 ± 18%  sched_debug.cpu#40.ttwu_count
         2 ± 37%    -100.0%          0 ±  0%  sched_debug.cpu#40.cpu_load[0]
         2 ± 47%    -100.0%          0 ±  0%  sched_debug.cpu#40.cpu_load[4]
    196449 ±  8%     -99.7%        599 ± 25%  sched_debug.cpu#40.ttwu_local
    195020 ±  4%     -99.7%        665 ± 32%  sched_debug.cpu#41.ttwu_local
    569391 ± 15%     -99.7%       1443 ± 22%  sched_debug.cpu#41.ttwu_count
    250174 ± 24%    +299.0%     998144 ±  0%  sched_debug.cpu#41.avg_idle
         7 ± 46%    -100.0%          0 ±  0%  sched_debug.cpu#41.load
    271444 ±  4%    +267.2%     996765 ±  3%  sched_debug.cpu#42.avg_idle
         3 ± 35%    -100.0%          0 ±  0%  sched_debug.cpu#42.cpu_load[2]
         4 ± 43%    -100.0%          0 ±  0%  sched_debug.cpu#42.cpu_load[1]
         3 ± 27%    -100.0%          0 ±  0%  sched_debug.cpu#42.cpu_load[3]
    594265 ± 15%     -99.8%       1417 ± 26%  sched_debug.cpu#42.ttwu_count
    198649 ±  7%     -99.7%        578 ± 29%  sched_debug.cpu#42.ttwu_local
    270822 ±  9%    +262.7%     982391 ±  2%  sched_debug.cpu#43.avg_idle
         3 ± 14%    -100.0%          0 ±  0%  sched_debug.cpu#43.cpu_load[2]
    595801 ± 15%     -99.8%       1474 ± 35%  sched_debug.cpu#43.ttwu_count
         2 ± 24%    -100.0%          0 ±  0%  sched_debug.cpu#43.cpu_load[3]
    199411 ±  4%     -99.7%        613 ± 22%  sched_debug.cpu#43.ttwu_local
         4 ± 25%    -100.0%          0 ±  0%  sched_debug.cpu#43.cpu_load[1]
         5 ± 23%    -100.0%          0 ±  0%  sched_debug.cpu#43.cpu_load[0]
    144742 ±  8%     -79.2%      30152 ± 47%  sched_debug.cpu#43.nr_load_updates
    276311 ±  3%    +258.0%     989070 ±  2%  sched_debug.cpu#44.avg_idle
    582806 ± 15%     -99.8%       1362 ± 30%  sched_debug.cpu#44.ttwu_count
    144761 ±  8%     -80.1%      28756 ± 45%  sched_debug.cpu#44.nr_load_updates
    144710 ±  7%     -67.6%      46849 ± 27%  sched_debug.cpu#45.nr_load_updates
    851364 ± 13%     -90.0%      85464 ± 31%  sched_debug.cpu#45.sched_count
    847235 ± 13%     -89.9%      85423 ± 31%  sched_debug.cpu#45.nr_switches
    372059 ± 14%     -88.6%      42469 ± 31%  sched_debug.cpu#45.sched_goidle
    203310 ± 19%     -99.6%        843 ± 17%  sched_debug.cpu#45.ttwu_local
    281126 ±  7%    +253.4%     993587 ±  5%  sched_debug.cpu#45.avg_idle
    573414 ± 12%     -99.7%       1515 ± 21%  sched_debug.cpu#45.ttwu_count
    861853 ± 12%     -92.4%      65635 ± 43%  sched_debug.cpu#46.nr_switches
    273192 ±  7%    +268.5%    1006815 ±  2%  sched_debug.cpu#46.avg_idle
    864903 ± 12%     -92.4%      65678 ± 43%  sched_debug.cpu#46.sched_count
    376696 ± 13%     -91.3%      32592 ± 43%  sched_debug.cpu#46.sched_goidle
    552783 ±  9%     -99.7%       1450 ± 14%  sched_debug.cpu#46.ttwu_count
    201314 ± 20%     -99.6%        706 ± 19%  sched_debug.cpu#46.ttwu_local
    144703 ±  7%     -74.6%      36762 ± 34%  sched_debug.cpu#46.nr_load_updates
       589 ± 34%    -100.0%          0 ±  0%  sched_debug.cpu#46.curr->pid
    584555 ±  7%     -99.7%       1599 ± 16%  sched_debug.cpu#47.ttwu_count
    203884 ± 19%     -99.6%        723 ± 23%  sched_debug.cpu#47.ttwu_local
    144799 ±  7%     -76.4%      34218 ± 46%  sched_debug.cpu#47.nr_load_updates
    274538 ±  6%    +260.1%     988726 ±  1%  sched_debug.cpu#47.avg_idle
         3 ± 35%    -100.0%          0 ±  0%  sched_debug.cpu#48.cpu_load[1]
    144673 ±  8%     -78.2%      31538 ± 48%  sched_debug.cpu#48.nr_load_updates
    543724 ±  4%     -99.7%       1672 ± 18%  sched_debug.cpu#48.ttwu_count
    199514 ± 22%     -99.7%        684 ± 34%  sched_debug.cpu#48.ttwu_local
         2 ± 37%    -100.0%          0 ±  0%  sched_debug.cpu#48.cpu_load[2]
         3 ± 44%    -100.0%          0 ±  0%  sched_debug.cpu#48.cpu_load[0]
    280359 ±  8%    +263.5%    1019216 ±  4%  sched_debug.cpu#48.avg_idle
    564757 ±  6%     -99.7%       1635 ± 18%  sched_debug.cpu#49.ttwu_count
    203456 ± 21%     -99.7%        710 ± 29%  sched_debug.cpu#49.ttwu_local
    144637 ±  7%     -78.9%      30527 ± 47%  sched_debug.cpu#49.nr_load_updates
    302435 ± 19%    +233.8%    1009483 ±  3%  sched_debug.cpu#49.avg_idle
         7 ± 35%    -100.0%          0 ±  0%  sched_debug.cpu#5.load
    576004 ± 11%     -99.5%       2984 ± 18%  sched_debug.cpu#5.ttwu_count
    184101 ± 10%     -98.6%       2497 ± 17%  sched_debug.cpu#5.ttwu_local
    256732 ±  5%    +131.1%     593404 ± 27%  sched_debug.cpu#5.avg_idle
         2 ± 49%    -100.0%          0 ±  0%  sched_debug.cpu#50.cpu_load[3]
    200344 ± 20%     -99.7%        688 ± 28%  sched_debug.cpu#50.ttwu_local
         2 ± 31%    -100.0%          0 ±  0%  sched_debug.cpu#50.cpu_load[2]
    563146 ±  5%     -99.7%       1908 ± 44%  sched_debug.cpu#50.ttwu_count
         2 ± 41%    -100.0%          0 ±  0%  sched_debug.cpu#50.cpu_load[1]
    244904 ± 22%    +302.4%     985599 ±  1%  sched_debug.cpu#50.avg_idle
    144490 ±  7%     -79.9%      29035 ± 47%  sched_debug.cpu#51.nr_load_updates
    273633 ±  7%    +265.2%     999259 ±  2%  sched_debug.cpu#51.avg_idle
    555030 ±  5%     -99.7%       1398 ± 23%  sched_debug.cpu#51.ttwu_count
    202800 ± 21%     -99.7%        617 ± 28%  sched_debug.cpu#51.ttwu_local
    204402 ± 20%     -99.7%        672 ± 35%  sched_debug.cpu#52.ttwu_local
         6 ± 37%    -100.0%          0 ±  0%  sched_debug.cpu#52.cpu_load[2]
    266105 ±  8%    +268.6%     980753 ±  3%  sched_debug.cpu#52.avg_idle
         8 ± 40%    -100.0%          0 ±  0%  sched_debug.cpu#52.cpu_load[1]
         4 ± 33%    -100.0%          0 ±  0%  sched_debug.cpu#52.cpu_load[3]
    590313 ± 10%     -99.8%       1394 ± 15%  sched_debug.cpu#52.ttwu_count
    246340 ± 20%    +293.7%     969729 ±  5%  sched_debug.cpu#53.avg_idle
    580316 ± 10%     -99.7%       1479 ± 17%  sched_debug.cpu#53.ttwu_count
    202868 ± 20%     -99.7%        559 ± 24%  sched_debug.cpu#53.ttwu_local
    275898 ±  8%    +251.8%     970687 ±  6%  sched_debug.cpu#54.avg_idle
    580835 ± 12%     -99.8%       1420 ± 27%  sched_debug.cpu#54.ttwu_count
    203860 ± 21%     -99.7%        669 ± 25%  sched_debug.cpu#54.ttwu_local
    561511 ± 12%     -99.8%       1320 ± 32%  sched_debug.cpu#55.ttwu_count
    199859 ± 21%     -99.7%        615 ± 44%  sched_debug.cpu#55.ttwu_local
    269166 ±  3%    +267.9%     990163 ±  1%  sched_debug.cpu#55.avg_idle
    592219 ± 10%     -99.8%       1394 ± 20%  sched_debug.cpu#56.ttwu_count
    203190 ± 20%     -99.7%        622 ± 21%  sched_debug.cpu#56.ttwu_local
    285148 ±  8%    +244.9%     983467 ±  2%  sched_debug.cpu#56.avg_idle
    289395 ±  8%    +243.8%     994802 ±  1%  sched_debug.cpu#57.avg_idle
    201174 ± 22%     -99.7%        588 ± 32%  sched_debug.cpu#57.ttwu_local
    561875 ±  9%     -99.8%       1333 ± 32%  sched_debug.cpu#57.ttwu_count
    532410 ± 13%     -99.8%       1321 ± 14%  sched_debug.cpu#58.ttwu_count
    199150 ± 23%     -99.7%        623 ± 30%  sched_debug.cpu#58.ttwu_local
    281144 ±  9%    +248.6%     979942 ±  4%  sched_debug.cpu#58.avg_idle
    206191 ± 20%     -99.7%        576 ± 34%  sched_debug.cpu#59.ttwu_local
    290618 ±  4%    +237.8%     981621 ±  2%  sched_debug.cpu#59.avg_idle
    568974 ±  8%     -99.8%       1378 ± 21%  sched_debug.cpu#59.ttwu_count
    145823 ±  7%     -17.5%     120332 ±  3%  sched_debug.cpu#6.nr_load_updates
    179817 ± 10%     -99.1%       1693 ± 19%  sched_debug.cpu#6.ttwu_local
    546345 ±  8%     -99.6%       2115 ± 21%  sched_debug.cpu#6.ttwu_count
    278607 ±  3%    +199.1%     833241 ± 18%  sched_debug.cpu#6.avg_idle
         2 ± 47%    -100.0%          0 ±  0%  sched_debug.cpu#6.cpu_load[3]
    851921 ±  8%     -40.4%     508151 ± 46%  sched_debug.cpu#6.nr_switches
    854405 ±  8%     -40.5%     508206 ± 46%  sched_debug.cpu#6.sched_count
         1 ± 35%    -100.0%          0 ±  0%  sched_debug.cpu#6.cpu_load[4]
    137215 ±  8%     -92.7%       9982 ± 37%  sched_debug.cpu#60.nr_load_updates
    916934 ±  7%     -39.4%     555767 ± 14%  sched_debug.cpu#60.max_idle_balance_cost
    496648 ± 21%    +124.0%    1112535 ± 14%  sched_debug.cpu#60.avg_idle
    523174 ± 17%     +90.8%     997981 ±  1%  sched_debug.cpu#61.avg_idle
    207882 ± 14%     -99.9%        283 ± 41%  sched_debug.cpu#61.ttwu_local
    925690 ±  4%     -45.3%     506045 ±  2%  sched_debug.cpu#61.max_idle_balance_cost
    303435 ± 20%     -99.8%        571 ± 21%  sched_debug.cpu#61.ttwu_count
    203648 ± 13%     -99.9%        158 ± 29%  sched_debug.cpu#62.ttwu_local
    904843 ±  5%     -44.4%     502645 ±  1%  sched_debug.cpu#62.max_idle_balance_cost
    305661 ± 24%     -99.9%        261 ± 30%  sched_debug.cpu#62.ttwu_count
       529 ± 32%    -100.0%          0 ±  0%  sched_debug.cpu#62.curr->pid
   1080713 ± 10%     -53.7%     500459 ±  0%  sched_debug.cpu#63.max_idle_balance_cost
   1699969 ± 10%     -41.0%    1003536 ±  2%  sched_debug.cpu#64.avg_idle
   1096675 ±  9%     -54.0%     504808 ±  2%  sched_debug.cpu#64.max_idle_balance_cost
      2374 ± 48%     -91.1%        211 ± 27%  sched_debug.cpu#64.ttwu_count
   1073352 ±  9%     -51.1%     524554 ±  7%  sched_debug.cpu#65.max_idle_balance_cost
   1599502 ±  9%     -34.4%    1049932 ±  7%  sched_debug.cpu#65.avg_idle
     39322 ± 42%     -72.4%      10848 ± 40%  sched_debug.cpu#66.nr_load_updates
   1409179 ± 19%     -27.9%    1015680 ±  3%  sched_debug.cpu#66.avg_idle
   1076703 ±  7%     -52.6%     510769 ±  4%  sched_debug.cpu#66.max_idle_balance_cost
   1086785 ±  9%     -53.0%     510290 ±  4%  sched_debug.cpu#67.max_idle_balance_cost
   1411451 ± 16%     -29.9%     989837 ±  2%  sched_debug.cpu#68.avg_idle
     38684 ± 43%     -66.9%      12790 ± 34%  sched_debug.cpu#68.nr_load_updates
   1108217 ±  7%     -54.9%     500229 ±  0%  sched_debug.cpu#68.max_idle_balance_cost
   1077891 ±  7%     -47.7%     563733 ± 22%  sched_debug.cpu#69.max_idle_balance_cost
    566207 ±  7%     -99.7%       1748 ± 21%  sched_debug.cpu#7.ttwu_count
    262075 ±  9%    +236.5%     881956 ± 13%  sched_debug.cpu#7.avg_idle
    838568 ±  9%     -53.9%     386340 ± 44%  sched_debug.cpu#7.sched_count
    357789 ±  9%     -46.2%     192547 ± 44%  sched_debug.cpu#7.sched_goidle
    836222 ±  9%     -53.8%     386289 ± 44%  sched_debug.cpu#7.nr_switches
    145736 ±  7%     -22.7%     112707 ±  3%  sched_debug.cpu#7.nr_load_updates
    182859 ± 10%     -99.2%       1443 ± 23%  sched_debug.cpu#7.ttwu_local
   1073432 ± 10%     -51.3%     522486 ±  8%  sched_debug.cpu#70.max_idle_balance_cost
   1530831 ± 15%     -35.2%     992013 ±  1%  sched_debug.cpu#71.avg_idle
   1054705 ±  6%     -52.6%     500000 ±  0%  sched_debug.cpu#71.max_idle_balance_cost
   1069098 ±  9%     -50.7%     527311 ±  9%  sched_debug.cpu#72.max_idle_balance_cost
     38445 ± 45%     -61.5%      14811 ± 40%  sched_debug.cpu#72.nr_load_updates
   1356913 ± 13%     -22.4%    1053465 ±  9%  sched_debug.cpu#72.avg_idle
   1332336 ± 10%     -26.8%     975029 ±  6%  sched_debug.cpu#73.avg_idle
   1058495 ±  8%     -51.8%     510063 ±  3%  sched_debug.cpu#73.max_idle_balance_cost
         2 ± 47%    -100.0%          0 ±  0%  sched_debug.cpu#73.cpu_load[1]
   1067890 ±  8%     -52.7%     504615 ±  1%  sched_debug.cpu#74.max_idle_balance_cost
    682497 ± 27%     -26.7%     500478 ±  0%  sched_debug.cpu#76.max_idle_balance_cost
    693411 ± 28%     -27.4%     503241 ±  1%  sched_debug.cpu#77.max_idle_balance_cost
    734302 ± 27%     -30.9%     507513 ±  2%  sched_debug.cpu#78.max_idle_balance_cost
    681179 ± 28%     -21.9%     531938 ± 10%  sched_debug.cpu#79.max_idle_balance_cost
    540200 ±  8%     -99.7%       1700 ± 13%  sched_debug.cpu#8.ttwu_count
    178504 ± 11%     -99.2%       1373 ± 19%  sched_debug.cpu#8.ttwu_local
    145742 ±  7%     -25.5%     108570 ±  3%  sched_debug.cpu#8.nr_load_updates
    361282 ±  8%     -51.3%     176052 ± 43%  sched_debug.cpu#8.sched_goidle
    276970 ±  8%    +230.5%     915429 ± 10%  sched_debug.cpu#8.avg_idle
    846491 ±  7%     -58.3%     353331 ± 43%  sched_debug.cpu#8.sched_count
    843911 ±  7%     -58.1%     353246 ± 43%  sched_debug.cpu#8.nr_switches
      1359 ± 48%     -59.4%        552 ±  9%  sched_debug.cpu#81.ttwu_count
    682840 ± 27%     -26.2%     503819 ±  1%  sched_debug.cpu#81.max_idle_balance_cost
    680252 ± 27%     -24.4%     513984 ±  4%  sched_debug.cpu#82.max_idle_balance_cost
    688877 ± 33%     -26.0%     509483 ±  3%  sched_debug.cpu#83.max_idle_balance_cost
    670276 ± 22%     -24.9%     503665 ±  1%  sched_debug.cpu#85.max_idle_balance_cost
    680305 ± 22%     -25.8%     504652 ±  1%  sched_debug.cpu#86.max_idle_balance_cost
      1390 ± 44%     -65.2%        483 ±  6%  sched_debug.cpu#87.ttwu_count
    711200 ± 26%     -29.2%     503212 ±  1%  sched_debug.cpu#88.max_idle_balance_cost
    649693 ± 26%     -22.7%     502075 ±  0%  sched_debug.cpu#89.max_idle_balance_cost
    531933 ±  7%     -99.7%       1700 ± 30%  sched_debug.cpu#9.ttwu_count
    180813 ± 10%     -99.2%       1424 ± 27%  sched_debug.cpu#9.ttwu_local
    855358 ±  8%     -58.0%     359635 ± 48%  sched_debug.cpu#9.nr_switches
    857752 ±  8%     -58.1%     359679 ± 48%  sched_debug.cpu#9.sched_count
    366428 ±  8%     -51.1%     179269 ± 48%  sched_debug.cpu#9.sched_goidle
    255284 ± 21%    +256.0%     908885 ± 12%  sched_debug.cpu#9.avg_idle
    145661 ±  7%     -28.0%     104899 ±  5%  sched_debug.cpu#9.nr_load_updates
         3 ± 33%    -100.0%          0 ±  0%  sched_debug.cpu#9.cpu_load[2]
         2 ± 38%    -100.0%          0 ±  0%  sched_debug.cpu#9.cpu_load[3]
         3 ± 40%    -100.0%          0 ±  0%  sched_debug.cpu#9.cpu_load[1]
    880077 ± 25%     -43.2%     500000 ±  0%  sched_debug.cpu#90.max_idle_balance_cost
   1270327 ± 11%     -21.3%    1000000 ±  0%  sched_debug.cpu#91.avg_idle
    886286 ± 22%     -43.6%     500000 ±  0%  sched_debug.cpu#91.max_idle_balance_cost
    861842 ± 25%     -42.0%     500000 ±  0%  sched_debug.cpu#92.max_idle_balance_cost
    877600 ± 26%     -37.3%     550542 ± 20%  sched_debug.cpu#93.max_idle_balance_cost
    872812 ± 25%     -42.7%     500149 ±  0%  sched_debug.cpu#94.max_idle_balance_cost
    880573 ± 26%     -43.0%     501897 ±  0%  sched_debug.cpu#95.max_idle_balance_cost
    898390 ± 24%     -43.6%     506730 ±  2%  sched_debug.cpu#96.max_idle_balance_cost
   1233750 ± 15%     -17.9%    1013461 ±  2%  sched_debug.cpu#96.avg_idle
   1279764 ± 17%     -22.4%     993620 ±  1%  sched_debug.cpu#97.avg_idle
    887849 ± 23%     -43.7%     500000 ±  0%  sched_debug.cpu#97.max_idle_balance_cost
    881541 ± 22%     -43.0%     502208 ±  0%  sched_debug.cpu#98.max_idle_balance_cost
    883063 ± 25%     -43.1%     502857 ±  1%  sched_debug.cpu#99.max_idle_balance_cost

brickland3: Brickland Ivy Bridge-EX
Memory: 512G




            stderr.netperf:send_omni:recv_data_failed:No_route_to_host

    1 ++----------------*-----*--------*------*-----------------------------*
      |                 :     :        :      :                             |
      |                 :     :        ::     :                             |
  0.8 ++               : :   : :      : :    : :                           :|
      |                : :   : :      : :    : :                           :|
      |                : :   : :      : :    : :                           :|
  0.6 ++               : :   : :      :  :   : :                           :|
      |                :  :  :  :     :  :   :  :                          :|
  0.4 ++              :   : :   :    :   :  :   :                         : |
      |               :   : :   :    :    : :   :                         : |
      |               :   : :   :    :    : :   :                         : |
  0.2 ++              :   : :   :    :    : :   :                         : |
      |              :     :     :  :     ::     :                       :  |
      |              :     :     :  :      :     :                       :  |
    0 *+-*--*--*--*--*-----*-----*--*------*-----*--*--*--*--*--*--*--*--*--+


                                last_state.booting

    1 ++------------------------------------------------------*-------------*
      |                                                       :             |
      |                                                       ::            |
  0.8 ++                                                     : :           :|
      |                                                      : :           :|
      |                                                      : :           :|
  0.6 ++                                                     :  :          :|
      |                                                      :  :          :|
  0.4 ++                                                    :   :         : |
      |                                                     :    :        : |
      |                                                     :    :        : |
  0.2 ++                                                    :    :        : |
      |                                                    :     :       :  |
      |                                                    :      :      :  |
    0 *+--*--*---*--*---*--*---*--*---*--*---*--*---*--*---*------*--*---*--+


                                   softirqs.TIMER

  6e+06 ++------------------------------------------------------------------+
        |                                                                   |
  5e+06 ++                                                 *                |
        |  .*.                                            +:                |
        |.*   *.*.*. .*    *   *   *.*     *.*         *.* :   .*.*.*.*.*.*.*
  4e+06 *+          * :    :   :   : :     : :     *   :   :  *             |
        |              :   :   :: :  :     : :     :  :    :  :             |
  3e+06 ++             :  : : : : :   :   :   :    :: :    :  :             |
        |              :  : : : : :   :   :   :   : : :     ::              |
  2e+06 ++             :  : : : : :   :   :   :   : : :     ::              |
        |               : : : :  :    :   :   :   :  :      ::              |
        |               *:   :   *    :   :   :   :  *      ::              |
  1e+06 O+  O   O O   O  O   O O   O   O : O   O : O   O O OO O O O O       |
        | O   O         O* O *         : :   O : :   O      *               |
      0 ++----------O------------O---O-*-O-----*-O--------------------------+


                                   softirqs.NET_RX

  7e+07 ++------------------------------------------------------------------+
        |                                                                   |
  6e+07 *+*.*.*.*.*   *    *   *   *.*     *.*     *   *.*.*  *.*.*.*.*.*.*.*
        |          : ::    :   :   : :     : :     :   :   :  :             |
  5e+07 ++         : ::    :   :   : :     : :     :   :   :  :             |
        |           *  :   ::  :: :   :    :  :    :: :    :  :             |
  4e+07 ++             :  : : : : :   :   :   :   : : :    : :              |
        |              :  : : : : :   :   :   :   : : :    : :              |
  3e+07 ++             :  : : : : :   :   :   :   : : :     ::              |
        |              :  : : :  :    :   :   :   : : :     ::              |
  2e+07 ++              : : : :  :    :   :   :   :  :      ::              |
        |               : :  ::  *     :  :    :  :  :      ::              |
  1e+07 ++              *:   :         : :     : :   *      :               |
        |                :   :         : :     : :          :               |
      0 O+O-O-O-O-O-O-O-OO-O-O-O-O-O-O-O-O-O-O-O-O-O-O-O-O-OO-O-O-O-O-------+


                                    softirqs.SCHED

  4.5e+06 ++-----------------------------------------------*----------------+
          |                                               ::                |
    4e+06 ++ .*.   *         *                            ::                |
  3.5e+06 ++*   *.* :  *     :   *  *.*     *.*    *   *.* :     *.**.*.*.*.*
          *         : +:     :   :  : :     : :    :   :    :   +           |
    3e+06 ++         * :     ::  :: : :     : :    :   :    :  *            |
  2.5e+06 ++            :   : : : : :  :   :   :   :: :     :  :            |
          |             :   : : : : :  :   :   :   :: :     :  :            |
    2e+06 ++            :   : : : ::   :   :   :   :: :     : :             |
  1.5e+06 ++            :   : : :  :   :   :   :  : : :     : :             |
          |              :  : : :  :   :   :   :  :  :      : :             |
    1e+06 ++             :  :  ::  *   :   :   :  :  :       ::             |
   500000 ++             * :   :        : :     : :  *       :              |
          O   O   OO   O  +O   O O  O   O : O   O :O   O O O O O O OO       |
        0 ++O---O----O---O-*-O-*---O--O-*-O---O-*-O--O-------*--------------+


                                     softirqs.RCU

  1.8e+06 ++-----------------------------------------------*----------------+
          |      .**         *       .*      .*           +:    .*.         |
  1.6e+06 *+ .*.*   + .*     :   *  * :     * :    *   *.* :   *   **.*.*.*.*
  1.4e+06 ++*        * :     :   :  : :     : :    :   :   :   :            |
          |            :     ::  :: : :     : :    :   :    :  :            |
  1.2e+06 ++            :   : : : : :  :    :  :   :: :     : :             |
    1e+06 ++            :   : : : : :  :   :   :   :: :     : :             |
          |             :   : : : ::   :   :   :   :: :     : :             |
   800000 ++            :   : : :  :   :   :   :   :: :     : :             |
   600000 ++             :  : : :  :   :   :   :  :  :      : :             |
          |              :  :  ::  *   :   :   :  :  :       ::             |
   400000 ++             * :   :        :  :    : :  *       :              |
   200000 O+  O   OO   O  +O   O O  O   O : O   O :O   O   O O O O OO       |
          | O            O *   *        : :   O : :  O   O   *              |
        0 ++----O----O-------O-----O--O-*-O-----*-O-------------------------+


                                cpuidle.C1-IVT-4S.usage

    4e+07 ++----------------------------------------------------------------+
          O   O   OO   O   O   O O  O   O       O  O   O O O O   O  O       |
  3.5e+07 ++                                O                  O   O        |
    3e+07 ++                                                                |
          |                                                                 |
  2.5e+07 ++                                                                |
          *.*.*.*.*    *     *   *  *.*     *.*    *   *.*.    *. .**.*. .*.*
    2e+07 ++       *. +:     :   :  : :     : :    :   :   *   : *      *   |
          |          *  :    ::  :: :  :    :  :   :: :    :   :            |
  1.5e+07 ++            :   : : : : :  :   :   :   :: :     : :             |
    1e+07 ++            :   : : : ::   :   :   :   :: :     : :             |
          |             :   : : :  :   :   :   :  : : :     : :             |
    5e+06 ++             :  :  ::  *    :  :    : :  :      : :             |
          |              *.:   :        : :     : :  *       :              |
        0 ++O---O----O---O-*-O-*---O--O-*-O---O-*-O--O-------*--------------+


                                cpuidle.C6-IVT-4S.time

  1.2e+11 ++----------------------------------------------------------------+
          |                                                                 |
    1e+11 ++         *             *                                        |
          |          :             :                                        |
          |          :             :                 *                      |
    8e+10 ++        : :           ::                 :                      |
          |         : :           ::                 ::                     |
    6e+10 ++        : :  *        : :               : :                     |
          |         : :  :        : :               : :                     |
    4e+10 ++        : : : :       : :               : :                     |
          O   O   OO   O: :O   O O  O   O   O   O  O:  O O O O O O OO       |
          |        :   :  :      :  :              :   :                    |
    2e+10 *+*.*.*.**   *   :.*.*.*  *.*     *.*    *   *.*.*   *.*.**.*.*.*.*
          |     O          * O         +   +   +   : O      + +             |
        0 ++O--------O---O---------O--O-*-O---O-*-O----------*--------------+


                                 turbostat.Avg_MHz

  1800 ++-------------------------------------------------*-----*-----------+
       |.*. .*.*.*   *     *   *    .*     .*     *   *. +:    +  *.*. .*.*.|
  1600 *+  *     :   :     :   :   * :    * :     :   : * :   *       *     *
  1400 ++        :   :     :   :   : :    : :     :   :   :   :             |
       |          : : :    :   ::  : :    : :     ::  :    :  :             |
  1200 ++         : : :   : : : : :   :   :  :   : : :     : :              |
  1000 ++         : : :   : : : : :   :   :  :   : : :     : :              |
       |          : : :   : : : : :   :   :  :   : : :     : :              |
   800 ++         : : :   : : : : :   :   :  :   : : :     : :              |
   600 ++          :  :   : : : : :   :  :   :   : : :     : :              |
       |           :   :  : : :  ::   :  :   :   :  ::      ::              |
   400 ++          *   : :   :   :     : :    : :   :       :               |
   200 ++              : :   :   :     : :    : :   :       :               |
       O   O   O O   O *.O   O O * O   O :O   O : O * O O O O O O O O       |
     0 ++O---O-----O---O-*-O-*---O---O-*-O--O-*-O---O-----------------------+


                                 turbostat.%Busy

  60 ++---------------------------------------------------------------------+
     |    .*.*.*         *                                *    .*.          |
  50 *+*.*     :   *     :   *    .*       .*     *   *. +:   *   *.*. .*.*.|
     |         :   :     :   :   * :      * :     :   : * :   :       *     *
     |          :  :     :   :   : :      : :     :   :    :  :             |
  40 ++         : : :   : :  ::  :  :    :   :    :: :     : :              |
     |          : : :   : : : : :   :    :   :   : : :     : :              |
  30 ++         : : :   : : : : :   :    :   :   : : :     : :              |
     |          : : :   : : : : :   :    :   :   : : :     : :              |
  20 ++          :  :   : : : : :   :   :    :   : : :     : :              |
     |           :  :   : : : : :   :   :    :   : : :      ::              |
     |           *   : :   ::  :     :  :     :  :  :       :               |
  10 ++              : :   :   :     :  :     : :   :       :               |
     O   O   O O   O *.O   O O * O   O :  O   O : O * O O O O O O O O       |
   0 ++O---O-----O---O-*-O-*---O---O-*-O----O-*-O---O-----------------------+


                                turbostat.CPU%c1

  50 *+-------------------------------------------------------*-------------+
  45 ++*   *.*.*   *     *   *    .*       .*     *   *.      : *.*.*. .*.*.|
     |  + +    :   :     :   :   * :      * :     :   : *.*   :       *     *
  40 ++  *     :   :     :   :   : :      : :     :   :   :  :              |
  35 ++         : : :    :   ::  : :      : :     ::  :   :  :              |
     |          : : :   : : : : :   :    :   :   : : :     : :              |
  30 ++         : : :   : : : : :   :    :   :   : : :     : :              |
  25 ++         : : :   : : : : :   :    :   :   : : :     : :              |
  20 ++         : : :   : : : : :   :    :   :   : : :     : :              |
     |           :  :   : : : : :   :   : O  :   : : :      : O             |
  15 O+  O   O O : O : O: :O:O ::   :   :    :   :  ::      O     O         |
  10 ++          *   : :   :   : O   O  :     O : O : O O O *   O   O       |
     |               : :   :   :     :  :     : :   :                       |
   5 ++              *.*   :   *     : :    O : :   O                       |
   0 ++O---O-----O---O---O-*---O---O-*-O------*-O---------------------------+


                                 turbostat.CPU%c6

  100 ++O---O---------O---O-*-----------------------O-----------------------+
   90 ++               .*   :   *           O       *                       |
      O   O   O O   O * O   O O : O   O       O   O : O O O *   O   O       |
   80 ++          *   : :   :   :         O         :       O O   O         |
   70 ++          :   : :  : :  ::                  :       :               |
      |           :   :  : : : : :                 : :      ::              |
   60 ++         : : :   : : : : :                 : :     : :              |
   50 ++         : : :   : : : : :                 : :     : :              |
   40 ++         : : :   : : : : :                 : :     : :              |
      |          : : :   : : : : :                 : :     : :              |
   30 ++         : : :   : : : :  :                : :     : :              |
   20 ++         : : :    :   :   :               :   :    :  :             |
      |         :   :     :   :   :               :   :   :   :             |
   10 ++ .*.    :   :     :   :   *.     .*.      :   :.*.:   :      .*.   .*
    0 *+*---*-*-*-O-*-----*---*-O---O-*-O---*-*-O-*---*---*---*-*-*-*---*-*-+


                                turbostat.CoreTmp

  60 ++---------------------------------------------------------------------+
     *.*.*.*.*.*   *     *   *    .*       .*         *.*.*      .*.*.      |
  50 ++        :   :     :   :   * :      * :     *   :    :  *.*     *.*.*.*
     O   O   O  : :O:  O: :O: :  : :      O :     :  :O    : :              |
     |         O: : :   : : :O: :O : O    : : O   O: :  O   O:O O O O       |
  40 ++          :  :  :   :  : :   :    :   :    :: :      *               |
     | O   O     *   O.* O *   :    :    :  O:   :  O     O                 |
  30 ++              *         *    :    :   :   :  *                       |
     |                              :    :   :   :                          |
  20 ++                             :   :    :   :                          |
     |                              :   :    :   :                          |
     |                               :  :     :  :                          |
  10 ++                              :  :     : :                           |
     |                               : :      : :                           |
   0 ++----------O-------------O---O-*-O------*-O---------------------------+


                                turbostat.PkgTmp

  60 ++---------------------------------------------------------------------+
     *.*.*.*.*.*   *     *   *    .*       .*         *.*.*    .*.*.       .*
  50 ++        :   :     :   :   * :      * :     *   :    :  *     *.*.*.* |
     |   O   O  : : :  O: :O: :  : :      O :     :  :O    : :O             |
     O         O: :O:   : : :O: :O : O    : : O   O: :  O   O:  O O O       |
  40 ++          :  :  :   :  : :   :    :   :    :: :      *               |
     | O   O     *   O.* O *   :    :    :  O:   :  O     O                 |
  30 ++              *         *    :    :   :   :  *                       |
     |                              :    :   :   :                          |
  20 ++                             :   :    :   :                          |
     |                              :   :    :   :                          |
     |                               :  :     :  :                          |
  10 ++                              :  :     : :                           |
     |                               : :      : :                           |
   0 ++----------O-------------O---O-*-O------*-O---------------------------+


                                turbostat.Pkg%pc2

  70 ++---------------------------------------------------------------------+
     |                           O                                          |
  60 ++                              O        O       O   O                 |
     O   O         O   O                                O       O   O       |
  50 ++      O O           O O                    O         O     O         |
     |                                    O                                 |
  40 ++                                                       O             |
     | O   O         O   O *                O                               |
  30 ++               .*   :                        O       *               |
     |           *   * :  : :  *                    *       :               |
  20 ++          :   :  : : :  ::                   :      : :              |
     |          : : :   : : : : :                  : :     : :              |
  10 ++         : : :   : : : :  :                 : :     : :              |
     |  .*.    :   :     :   :   *.      .*.      :   :.*.:   :      .*.    |
   0 *+*---*-*-*-O-*-----*---*-O---O-*-O----*-*-O-*---*---*---*-*-*-*---*-*-*


                                 turbostat.PkgWatt

  450 *+*---*-*-*---------*---*-----*-------*-------------------*-------*---+
      |   *     :   *     :   :   * :     * :     *   *.*.*   *   *.*.*   *.*
  400 ++        :   :     :   :   : :     : :     :   :   :   :             |
  350 ++         :  :     ::  ::  : :     : :     :   :   :   :             |
      |          : : :   : : : : :   :    :  :    :: :     : :              |
  300 ++         : : :   : : : : :   :   :   :   : : :     : :              |
  250 ++         : : :   : : : : :   :   :   :   : : :     : :              |
      |          : : :   : : : : :   :   :   :   : : :     : :              |
  200 ++          :  :   : : : : :   :   :   :   : : :      :               |
  150 O+  O   O O : O : O:  O:O ::O  :O  :O  :O  :O: :O O O O O O O O       |
      |           *   : :   :   :    :   :   :   :  :       *               |
  100 ++              *.:   :   :     : :   O : :   :                       |
   50 ++O   O         O * O *   *     : :     : :   O                       |
      |                               : :     : :                           |
    0 ++----------O-------------O---O-*-O-----*-O---------------------------+


                                 turbostat.CorWatt

  400 ++--------------------------------------------------------------------+
      *.*. .*.*.*   *     *   *    .*      .*     *   *. .*   *.*.*.*. .*.*.|
  350 ++  *     :   :     :   :   * :     * :     :   : * :   :       *     *
  300 ++        :   :     :   :   : :     : :     :   :   :   :             |
      |          : : :    ::  ::  : :     : :     :: :     : :              |
  250 ++         : : :   : : : : :   :    :  :   : : :     : :              |
      |          : : :   : : : : :   :   :   :   : : :     : :              |
  200 ++         : : :   : : : : :   :   :   :   : : :     : :              |
      |          : : :   : : : : :   :   :   :   : : :     : :              |
  150 ++          :  :   : : : : :   :   :   :   : : :      :               |
  100 O+  O   O   : O : O:  O:  :    :   :O  :   :O :       O O   O         |
      |         O *   : :   : O : O   O  :    O :   : O O O *   O   O       |
   50 ++              *.:   :   *     : :   O : :   *                       |
      | O   O         O * O *         : :     : :   O                       |
    0 ++----------O-------------O---O-*-O-----*-O---------------------------+


                              netperf.Throughput_tps

  1600 ++-------------------------------------------------------------------+
       *.*.*.*.*.*   *     *   *   *.*    *.*         *.*.*   *.*.*. .*.*. .*
  1400 ++        :   :     :   :   : :    : :     *   :   :   :     *     * |
  1200 ++        :   :     :   :   : :    : :     :   :   :   :             |
       |         :   :     :   :   : :    : :     :   :   :   :             |
  1000 O+  O   O O: :O:  O: :O:O: :O  :O  O  :O   O: :O O O:O:O O O O       |
       |          : : :   : : : : :   :   :  :   : : :     : :              |
   800 ++         : : :   : : : : :   :   :  :   : : :     : :              |
       |          : : :   : : : : :   :  :   :   : : :     : :              |
   600 ++         : : :   : : : : :   :  :   :   : : :     : :              |
   400 ++         : : :   : : : : :   :  :   :   : : :     : :              |
       |           :   : :   :   :     : :    :  :  :       :               |
   200 ++          :   : :   :   :     : :    : :   :       :               |
       |           :   : :   :   :     : :    : :   :       :               |
     0 ++O---O-----O---O-*-O-*---O---O-*-O--O-*-O---O-------*---------------+


                              netperf.time.user_time

  1000 ++*-*----------------------------------------------------------------+
   900 ++   :                                     *           *.*.*.   .*.*.|
       *    :   .*.*.*             *.     *.      :           :     *.*     *
   800 ++    :.*     :     *   *   : *    : *     :           :             |
   700 ++    *       :     :   :   : :    : :     ::          :             |
       |              :    :   :  :  :    : :    : :  *.*.*  :              |
   600 ++             :   : : : : :   :   :  :   : :  :   :  :              |
   500 ++             :   : : : : :   :   :  :   : : :    :  :              |
   400 ++             :   : : : : :   :   :  :   : : :     : :              |
       |              :   : : : : :   :  :   :   : : :     : :              |
   300 ++              :  : : :  :    :  :   :   :  ::     : :              |
   200 ++              :  : : :  :    :  :   :  :   :      ::               |
       |               * :   :   *     : :    : :   *       :               |
   100 O+  O   O O   O  +O   O O   O   O :O   O : O   O O O O O O O O       |
     0 ++O---O-----O---O-*-O-*---O---O-*-O--O-*-O---O-------*---------------+


                              netperf.time.system_time

  12000 ++------------------------------------------------------------------+
        |    .*.           *                              .*                |
  10000 ++*.*   *.*   *    :   *   *.*     *.*         *.* :   .*.*.*.*.*.*.*
        *          + +:    :   :   : :     : :     *   :   :  *             |
        |           * :    :   :   : :     : :     :   :   :  :             |
   8000 ++             :  : :  :: :   :    :  :    :  :    :  :             |
        |              :  : : : : :   :   :   :   : : :    : :              |
   6000 ++             :  : : : : :   :   :   :   : : :    : :              |
        |              :  : : : : :   :   :   :   : : :     ::              |
   4000 ++             :  : : :  :    :   :   :   : : :     ::              |
        |               : : : :  :    :   :   :   :  :      ::              |
        |               ::   ::  *     :  :    :  :  :      ::              |
   2000 ++              *:   :         : :     : :   *      :               |
        O   O   O O   O  O   O O   O   O : O   O : O   O O OO O O O O       |
      0 ++O---O-----O---O*-O-*---O---O-*-O---O-*-O---O------*---------------+


                     netperf.time.percent_of_cpu_this_job_got

  4000 ++-------------------------------------------------*-----------------+
       |.*. .*.*.*   *     *   *    .*     .*         *. +:    .*.*.*. .*.*.|
  3500 *+  *     :   :     :   :   * :    * :     *   : * :   *       *     *
  3000 ++        :   :     :   :   : :    : :     :   :   :   :             |
       |          : : :    :   ::  : :    : :     :   :    :  :             |
  2500 ++         : : :   : : : : :   :   :  :    :: :     :  :             |
       |          : : :   : : : : :   :   :  :   : : :     : :              |
  2000 ++         : : :   : : : : :   :   :  :   : : :     : :              |
       |          : : :   : : : : :   :   :  :   : : :     : :              |
  1500 ++          :  :   : : : : :   :  :   :   : : :     : :              |
  1000 ++          :   :  : : :  ::   :  :   :   : : :     : :              |
       |           *   : :   :   :     : :    :  :  :       ::              |
   500 ++              : :   :   :     : :    : :   :       :               |
       O   O   O O   O *.O   O O * O   O :O   O : O * O O O O O O O O       |
     0 ++O---O-----O---O-*-O-*---O---O-*-O--O-*-O---O-------*---------------+


                       netperf.time.voluntary_context_switches

  7e+07 *+*-*-*-*-*---*----*---*---*-*-----*-*---------*-*-*--*-*-*-*-*-*-*-*
        |          : ::    :   :   : :     : :     *   :   :  :             |
  6e+07 ++         : ::    :   :   : :     : :     :   :   :  :             |
        |           *  :   :   :: :  :     : :     :  :    :  :             |
  5e+07 O+  O   O O   O: O: :O:O: :O  :O  :O  :O   O: :O O OO:  O O O       |
        |              :  : : : : :   :   :   :   : : :    : :O             |
  4e+07 ++             :  : : : : :   :   :   :   : : :    : :              |
        |              :  : : : : :   :   :   :   : : :     ::              |
  3e+07 ++             :  : : : : :   :   :   :   : : :     ::              |
        |               : : : :  :    :   :   :   :  :      ::              |
  2e+07 ++              : : : :  :    :   :   :   :  :      ::              |
        |               ::   :   *     : :     : :   *      :               |
  1e+07 ++              *:   :         : :     : :          :               |
        |                :   :         : :     : :          :               |
      0 ++O---O-----O---O*-O-*---O---O-*-O---O-*-O---O------*---------------+


                      netperf.time.involuntary_context_switches

  35000 ++------------------------------------------------------------------+
        |                                                *.                 |
  30000 ++    *                                         +  *     .*         |
        |    + :           *   *                       *   :    *  :  *     *
  25000 ++  *  : .*   *    :   :   *.*     *.*         :   :    :  : + +   +|
        |  :    *  + +:    :   ::  : :     : :     *   :   :   :    *   *.* |
  20000 ++ :        *  :  : : : : :  :     : :     :  :    :   :            |
        *.*            :  : : : : :   :   :   :    :: :    :  *             |
  15000 ++             :  : : : : :   :   :   :   : : :     : :             |
        |              :  : : :  ::   :   :   :   : : :     ::              |
  10000 ++              : : : :  :    :   :   :   : ::      ::              |
        |               : : : :  *    :   :   :   :  :      ::              |
   5000 ++              *:   :         : :     : :   *      ::O             |
        O   O   O O   O  O   O O   O   O : O   O : O   O O OO   O O O       |
      0 ++O---O-----O---O*-O-*---O---O-*-O---O-*-O---O------*---------------+


                                  time.user_time

  1000 ++*-*----------------------------------------------------------------+
   900 ++   :                                     *           *.*.*.   .*.*.|
       *    :   .*.*.*             *.     *.      :           :     *.*     *
   800 ++    :.*     :     *   *   : *    : *     :           :             |
   700 ++    *       :     :   :   : :    : :     ::          :             |
       |              :    :   :  :  :    : :    : :  *.*.*  :              |
   600 ++             :   : : : : :   :   :  :   : :  :   :  :              |
   500 ++             :   : : : : :   :   :  :   : : :    :  :              |
   400 ++             :   : : : : :   :   :  :   : : :     : :              |
       |              :   : : : : :   :  :   :   : : :     : :              |
   300 ++              :  : : :  :    :  :   :   :  ::     : :              |
   200 ++              :  : : :  :    :  :   :  :   :      ::               |
       |               * :   :   *     : :    : :   *       :               |
   100 O+  O   O O   O  +O   O O   O   O :O   O : O   O O O O O O O O       |
     0 ++O---O-----O---O-*-O-*---O---O-*-O--O-*-O---O-------*---------------+


                                  time.system_time

  12000 ++------------------------------------------------------------------+
        |    .*.           *                              .*                |
  10000 ++*.*   *.*   *    :   *   *.*     *.*         *.* :   .*.*.*.*.*.*.*
        *          + +:    :   :   : :     : :     *   :   :  *             |
        |           * :    :   :   : :     : :     :   :   :  :             |
   8000 ++             :  : :  :: :   :    :  :    :  :    :  :             |
        |              :  : : : : :   :   :   :   : : :    : :              |
   6000 ++             :  : : : : :   :   :   :   : : :    : :              |
        |              :  : : : : :   :   :   :   : : :     ::              |
   4000 ++             :  : : :  :    :   :   :   : : :     ::              |
        |               : : : :  :    :   :   :   :  :      ::              |
        |               ::   ::  *     :  :    :  :  :      ::              |
   2000 ++              *:   :         : :     : :   *      :               |
        O   O   O O   O  O   O O   O   O : O   O : O   O O OO O O O O       |
      0 ++O---O-----O---O*-O-*---O---O-*-O---O-*-O---O------*---------------+


                         time.percent_of_cpu_this_job_got

  4000 ++-------------------------------------------------*-----------------+
       |.*. .*.*.*   *     *   *    .*     .*         *. +:    .*.*.*. .*.*.|
  3500 *+  *     :   :     :   :   * :    * :     *   : * :   *       *     *
  3000 ++        :   :     :   :   : :    : :     :   :   :   :             |
       |          : : :    :   ::  : :    : :     :   :    :  :             |
  2500 ++         : : :   : : : : :   :   :  :    :: :     :  :             |
       |          : : :   : : : : :   :   :  :   : : :     : :              |
  2000 ++         : : :   : : : : :   :   :  :   : : :     : :              |
       |          : : :   : : : : :   :   :  :   : : :     : :              |
  1500 ++          :  :   : : : : :   :  :   :   : : :     : :              |
  1000 ++          :   :  : : :  ::   :  :   :   : : :     : :              |
       |           *   : :   :   :     : :    :  :  :       ::              |
   500 ++              : :   :   :     : :    : :   :       :               |
       O   O   O O   O *.O   O O * O   O :O   O : O * O O O O O O O O       |
     0 ++O---O-----O---O-*-O-*---O---O-*-O--O-*-O---O-------*---------------+


                           time.voluntary_context_switches

  7e+07 *+*-*-*-*-*---*----*---*---*-*-----*-*---------*-*-*--*-*-*-*-*-*-*-*
        |          : ::    :   :   : :     : :     *   :   :  :             |
  6e+07 ++         : ::    :   :   : :     : :     :   :   :  :             |
        |           *  :   :   :: :  :     : :     :  :    :  :             |
  5e+07 O+  O   O O   O: O: :O:O: :O  :O  :O  :O   O: :O O OO:  O O O       |
        |              :  : : : : :   :   :   :   : : :    : :O             |
  4e+07 ++             :  : : : : :   :   :   :   : : :    : :              |
        |              :  : : : : :   :   :   :   : : :     ::              |
  3e+07 ++             :  : : : : :   :   :   :   : : :     ::              |
        |               : : : :  :    :   :   :   :  :      ::              |
  2e+07 ++              : : : :  :    :   :   :   :  :      ::              |
        |               ::   :   *     : :     : :   *      :               |
  1e+07 ++              *:   :         : :     : :          :               |
        |                :   :         : :     : :          :               |
      0 ++O---O-----O---O*-O-*---O---O-*-O---O-*-O---O------*---------------+


                          time.involuntary_context_switches

  35000 ++------------------------------------------------------------------+
        |                                                *.                 |
  30000 ++    *                                         +  *     .*         |
        |    + :           *   *                       *   :    *  :  *     *
  25000 ++  *  : .*   *    :   :   *.*     *.*         :   :    :  : + +   +|
        |  :    *  + +:    :   ::  : :     : :     *   :   :   :    *   *.* |
  20000 ++ :        *  :  : : : : :  :     : :     :  :    :   :            |
        *.*            :  : : : : :   :   :   :    :: :    :  *             |
  15000 ++             :  : : : : :   :   :   :   : : :     : :             |
        |              :  : : :  ::   :   :   :   : : :     ::              |
  10000 ++              : : : :  :    :   :   :   : ::      ::              |
        |               : : : :  *    :   :   :   :  :      ::              |
   5000 ++              *:   :         : :     : :   *      ::O             |
        O   O   O O   O  O   O O   O   O : O   O : O   O O OO   O O O       |
      0 ++O---O-----O---O*-O-*---O---O-*-O---O-*-O---O------*---------------+


                                 vmstat.procs.r

  50 ++----*---*------------------------------------------------------------+
     |    +  * :   *     *   *     *        *         *. .*    .*.*.*   *.  |
     *.*.*     :   :     :   :    +:       +:     *   : * :   *      + +  *.*
  40 ++         :  :     :   :   * :      * :     :   :   :   :       *     |
     |          : : :    :   ::  : :      : :     :   :    :  :             |
     |          : : :   : : : :  :  :    :   :    :: :     : :              |
  30 ++         : : :   : : : : :   :    :   :   : : :     : :              |
     |          : : :   : : : : :   :    :   :   : : :     : :              |
  20 ++         : : :   : : : : :   :    :   :   : : :     : :              |
     |           :  :   : : : : :   :   :    :   : : :     : :              |
     |           :   :  : : :  ::   :   :    :   : : :      ::              |
  10 ++          *   : :   :   :     :  :     :  :  :       :               |
     |               : :   :   :     :  :     : :   :       :               |
     O   O   O O   O *.O   O O * O   O :  O   O : O * O O O O O O O O       |
   0 ++O---O-----O---O-*-O-*---O---O-*-O----O-*-O---O-----------------------+


                                  vmstat.system.in

  500000 ++-----------------------------------------------------------------+
  450000 *+*. .*.*.*  *     *   *    .*     .*     *   *. .*   *.**.*. .*.*.|
         |   *     :  :     :   :   * :    * :     :   : * :   :      *     *
  400000 ++        :  :     :   :   : :    : :     :   :   :   :            |
  350000 ++        : : :    :   ::  : :    : :     ::  :   :   :            |
         |         : : :   : : : : :   :   :  :   : : :     : :             |
  300000 ++         :: :   : : : : :   :   :  :   : : :     : :             |
  250000 ++         :: :   : : : : :   :   :  :   : : :     : :             |
  200000 ++         :: :   : : : : :   :   :  :   : : :     : :             |
         |          :  :   : : : : :   :  :   :   : : :     : :             |
  150000 ++         :   :  : : :  ::   :  :   :   :  ::     : :             |
  100000 ++         *   : :   :   :     : :    : :   :       :              |
         |              : :   :   :     : :    : :   :       :              |
   50000 O+  O   O O  O *.O   O O * O   O :O   O : O * O O O O O OO O       |
       0 ++O---O----O---O-*-O-*---O---O-*-O--O-*-O---O-------*--------------+


                             numa-numastat.node0.numa_hit

  1.6e+06 ++----------------------------------------------------------------+
          |       O            O                O        O O                |
  1.4e+06 O+  O    O   O   O     O  O   O   O      O   O     O   O OO       |
          |                                                    O            |
  1.2e+06 ++                                                                |
    1e+06 ++                                                                |
          |                                                                 |
   800000 ++                                                                |
          |          *             *                                        |
   600000 ++         ::            :                                        |
   400000 ++        : :           : :                                       |
          |    .*.  :  :.*        : *.      *.      .*.              .*.*.  |
   200000 *+*.*   **   *  :  *. .*    *    :  *    *   *.*.*   *.*.**     *.*
          |               : +  *       +   :   +   :        + +             |
        0 ++O---O----O---O-*-O-----O--O-*-O---O-*-O--O-------*--------------+


                            numa-numastat.node0.local_node

  1.6e+06 ++----------------------------------------------------------------+
          |       O            O                O        O O                |
  1.4e+06 O+  O    O   O   O     O  O   O   O      O   O     O   O OO       |
          |                                                    O            |
  1.2e+06 ++                                                                |
    1e+06 ++                                                                |
          |                                                                 |
   800000 ++                                                                |
          |          *             *                                        |
   600000 ++         ::            :                                        |
   400000 ++        : :           : :                                       |
          |    .*.  :  :.*        : *.      *.      .*.              .*.*.  |
   200000 *+*.*   **   *  :  *. .*    *    :  *    *   *.*.*   *.*.**     *.*
          |               : +  *       +   :   +   :        + +             |
        0 ++O---O----O---O-*-O-----O--O-*-O---O-*-O--O-------*--------------+


                             numa-vmstat.node0.numa_hit

   1e+06 ++-----------------------------------------------------------------+
  900000 ++                                                O                |
         |       O            O                O   O             OO         |
  800000 O+  O     O  O   O     O   O   O  O           O O   O      O       |
  700000 ++                                                    O            |
         |                                                                  |
  600000 ++                                                                 |
  500000 ++         *             *                                         |
  400000 ++         ::           : :                                        |
         |    .*.  : :           : :                .*.                .*.  |
  300000 *+*.*   *.*  *.*. .*. .*   *.*    *.*     *   *.*.*   *.**.*.*   *.*
  200000 ++    O          * O *        :   :  :    :        + +             |
         | O            O              :   : O:   :  O       *              |
  100000 ++                             : :    :  :                         |
       0 ++---------O-------------O---O-*-O----*-O--------------------------+


                            numa-vmstat.node0.numa_local

  900000 ++------------------------------------------------O----------------+
         |       O            O                O   O             OO         |
  800000 O+  O        O   O         O   O  O           O O   O              |
  700000 ++        O            O                              O    O       |
         |                                                                  |
  600000 ++                                                                 |
  500000 ++                                                                 |
         |          *             *                                         |
  400000 ++         ::           : :                                        |
  300000 ++        : :           : :                .*                 .*.  |
         |.*.*.*.*.*  *.*   *. .*   *.*    *.*     *  + .*.*   *.**.*.*   *.*
  200000 *+              + +  *        :   :  :    :   *    + +             |
  100000 ++O   O        O * O          :   : O:   :  O       *              |
         |                              : :    :  :                         |
       0 ++---------O-------------O---O-*-O----*-O--------------------------+


                                 proc-vmstat.numa_hit

    3e+06 ++----------------------------------------------------------------+
          |          *                                                      |
  2.5e+06 ++         :                                                      |
          |         : :                                                     |
          O   O   OO: :O   O   O O *O   O   O   O  O     O O O   O OO       |
    2e+06 ++        : :           ::                   O       O            |
          |        :   :          : :                *.                     |
  1.5e+06 ++ .*.*.**   *.    *   *  *.*     *.*     +  *.*.*    .*.* .*.*. .*
          *.*            *   :   :    :     : :    *       :   *    *     * |
    1e+06 ++             :  : : :      :   :   :   :        :  :            |
          |               : : : :      :   :   :   :        : :             |
          |               : :  :       :   :   :   :        : :             |
   500000 ++               :   *       :   :   :  :         : :             |
          |     O          * O          : :     : :  O       :              |
        0 ++O--------O---O---------O--O-*-O---O-*-O----------*--------------+


                                proc-vmstat.numa_local

    3e+06 ++----------------------------------------------------------------+
          |          *                                                      |
  2.5e+06 ++         :                                                      |
          |         : :                                                     |
          O   O   OO: :O   O   O O *O   O   O   O  O     O O O   O OO       |
    2e+06 ++        : :           ::                   O       O            |
          |        :   :          : :                *.                     |
  1.5e+06 ++ .*.*.**   *.    *   *  *.*     *.*     +  *.*.*    .*.* .*.*. .*
          *.*            *   :   :    :     : :    *       :   *    *     * |
    1e+06 ++             :  : : :      :   :   :   :        :  :            |
          |               : : : :      :   :   :   :        : :             |
          |               : :  :       :   :   :   :        : :             |
   500000 ++               :   *       :   :   :  :         : :             |
          |     O          * O          : :     : :  O       :              |
        0 ++O--------O---O---------O--O-*-O---O-*-O----------*--------------+


                              proc-vmstat.pgalloc_dma32

  350000 ++-----------------------------------------------------------------+
         O   O   O    O   O   O                O   O     O O O    O         |
  300000 ++        O            O   O   O  O           O       O O          |
         |                                                          O       |
  250000 ++                                                                 |
         |                                                                  |
  200000 ++    *   *                                                        |
         *.   + + + *.*     *   *    .*     .*     *   *.*.*    .**.     .*.*
  150000 ++*.*   *    :     :   ::  * :    * :     :   :   :   *    *.*.*   |
         |             :   : : : : +   :   :  :   : : :     :  :            |
  100000 ++            :   : : :  *    :   :  :   : : :     : :             |
         |              :  : : :       :   :  :   :  :      : :             |
   50000 ++             *  : : :       :  :   :   :  *      : :             |
         |               +:   :         : :    : :           :              |
       0 ++O---O----O---O-*-O-*---O---O-*-O--O-*-O---O-------*--------------+


                              proc-vmstat.pgalloc_normal

  1.2e+07 ++----------------------------------------------------------------+
          O   O   O    O   O   O    O   O   O   O  O     O O O   O O        |
    1e+07 ++       O             O                     O       O    O       |
          |                                                                 |
          |                                                                 |
    8e+06 ++                                                                |
          |         .*.                                                     |
    6e+06 *+*.*.*.**   *     *   *  *.*     *.*        *.*.*   *.*.**.*.*.*.*
          |            :     :   :: : :     : :    *   :   :   :            |
    4e+06 ++            :   : : : ::   :   :   :   :: :     : :             |
          |             :   : : :  *   :   :   :   :: :     : :             |
          |              :  : : :      :   :   :   : *      : :             |
    2e+06 ++             *  : : :      :   :   :  :         : :             |
          |               +:   :        : :     : :          :              |
        0 ++O---O----O---O-*-O-*---O--O-*-O---O-*-O--O-------*--------------+


                                  proc-vmstat.pgfree

  1.2e+07 ++------O------------O----------------O--O-----O-O-----O-O--------+
          O   O    O   O   O     O  O   O   O          O     O      O       |
    1e+07 ++                                                   O            |
          |                                                                 |
          |                                                                 |
    8e+06 ++                                                                |
          |    .*.  .*.                                                     |
    6e+06 *+*.*   **   *     *   *  *.*     *.*    *   *.*.*   *.*.**.*.*.*.*
          |             :    :   :: : :     : :    ::  :   :   :            |
    4e+06 ++            :   : : : ::   :   :   :   :: :     : :             |
          |              :  : : :  *   :   :   :   : ::     : :             |
          |              *  : : :      :   :   :  :  *      : :             |
    2e+06 ++              : :  :       :   :   :  :         : :             |
          |               ::   *        : :     : :          :              |
        0 ++O---O----O---O-*-O-----O--O-*-O---O-*-O--O-------*--------------+


                            proc-vmstat.numa_hint_faults

  70000 ++------------------------------------------------------------------+
        |                                                                   |
  60000 ++                           *       *                  *           |
        |                           +:      +:          .*.*    :+   .*.    |
  50000 ++ .*   *.                 * :     * :         *   :   :  *.*   *.*.*
        |.*  + +  *.*.     *   *   :  :    :  :        :   :   :            |
  40000 *+    *       *    :   :: :   :    :  :    *  :    :  *             |
        |              :   :   :: :   :   :   :    :: :    :  :             |
  30000 ++             :  : : :  :    :   :   :    :: :    :  :             |
        |               : : : :  *    :   :   :   :  :      ::O             |
  20000 ++  O   O O     *O: :O:O      :   :O  :   :  * O O OO:  O O         |
        O             O : : : :        O  :    O  :O        ::      O       |
  10000 ++               :   :     O   : :     :  :         ::              |
        |                :   :         : :     : :          :               |
      0 ++O---O-----O---O*-O-*---O---O-*-O---O-*-O---O------*---------------+


                         proc-vmstat.numa_hint_faults_local

  60000 ++------------------------------------------------------------------+
        |                            *       *                  *           |
  50000 ++                          +:      +:                  ::          |
        |  .*                      * :     * :          .*.*   : :   .*.    |
        | *  + .*.         *   *   : :     : :         *   :   :  *.*   *.*.*
  40000 ++    *   *.*.*    :   :  :   :    :  :        :   :  :             |
        *             :    :   :: :   :   :   :    *  :    :  *             |
  30000 ++             :  : : : : :   :   :   :    :  :    :  :             |
        |              :  : : : : :   :   :   :    :: :    :  :             |
  20000 ++              : : : :  :    :   :   :   : : :     ::              |
        |   O   O O     *O: :O:O *    :   :O  :   :  :   O  O:O O O         |
        O             O : : : :    O   :  :    O  :O * O   O::      O       |
  10000 ++               :   :         O :     :  :         ::              |
        |                :   :         : :     : :          :               |
      0 ++O---O-----O---O*-O-*---O---O-*-O---O-*-O---O------*---------------+


	[*] bisect-good sample
	[O] bisect-bad  sample

To reproduce:

	apt-get install ruby
	git clone git://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git
	cd lkp-tests
	bin/setup-local job.yaml # the job file attached in this email
	bin/run-local   job.yaml


Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Ying Huang


View attachment "job.yaml" of type "text/plain" (3379 bytes)

View attachment "reproduce" of type "text/plain" (9477 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ